2023-11-26T06:58:37.910 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-26T06:58:37.910 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} --name rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353 --verbose --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353/orig.config.yaml 2023-11-26T06:58:37.954 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-26T06:58:38.042 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} email: null first_in_suite: false job_id: '7467353' kernel: client: sha1: distro flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 44c2237fe77713214148c1fd622c872f346f1b99 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 44c2237fe77713214148c1fd622c872f346f1b99 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-mgr-vol-misc sha1: 44c2237fe77713214148c1fd622c872f346f1b99 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1863 sha1: 44c2237fe77713214148c1fd622c872f346f1b99 sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-mgr-vol-misc suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 44c2237fe77713214148c1fd622c872f346f1b99 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-26_06:51:12 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-26T06:58:38.042 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa; will attempt to use it 2023-11-26T06:58:38.042 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks 2023-11-26T06:58:38.042 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-26T06:58:38.044 INFO:teuthology.task.internal:Checking packages... 2023-11-26T06:58:38.065 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '44c2237fe77713214148c1fd622c872f346f1b99' 2023-11-26T06:58:38.066 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-26T06:58:38.066 INFO:teuthology.packaging:ref: None 2023-11-26T06:58:38.066 INFO:teuthology.packaging:tag: None 2023-11-26T06:58:38.066 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-26T06:58:38.066 INFO:teuthology.packaging:sha1: 44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T06:58:38.066 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-26T06:58:38.215 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484.g44c2237f 2023-11-26T06:58:38.215 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-26T06:58:38.222 INFO:teuthology.task.internal:no buildpackages task found 2023-11-26T06:58:38.222 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-26T06:58:38.260 INFO:teuthology.task.internal:Saving configuration 2023-11-26T06:58:38.277 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-26T06:58:38.294 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-26T06:58:38.319 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:38.187444', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-26T06:58:38.342 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:38.186196', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-26T06:58:38.342 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-26T06:58:38.349 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-26T06:58:38.349 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-26T06:58:38.349 INFO:teuthology.run_tasks:Running task console_log... 2023-11-26T06:58:38.446 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcc3c371b80>, signals=[15]) 2023-11-26T06:58:38.446 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-26T06:58:38.453 INFO:teuthology.task.internal:Opening connections... 2023-11-26T06:58:38.453 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T06:58:38.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T06:58:38.521 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T06:58:38.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T06:58:38.588 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-26T06:58:38.595 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2023-11-26T06:58:38.615 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2023-11-26T06:58:38.616 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2023-11-26T06:58:38.675 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="Red Hat Enterprise Linux" 2023-11-26T06:58:38.675 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="8.6 (Ootpa)" 2023-11-26T06:58:38.675 INFO:teuthology.orchestra.run.smithi083.stdout:ID="rhel" 2023-11-26T06:58:38.675 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="fedora" 2023-11-26T06:58:38.675 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="8.6" 2023-11-26T06:58:38.675 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el8" 2023-11-26T06:58:38.675 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-26T06:58:38.675 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2023-11-26T06:58:38.675 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-26T06:58:38.675 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://www.redhat.com/" 2023-11-26T06:58:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-26T06:58:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-26T06:58:38.676 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T06:58:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-26T06:58:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-26T06:58:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-26T06:58:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-26T06:58:38.677 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2023-11-26T06:58:38.699 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2023-11-26T06:58:38.719 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2023-11-26T06:58:38.719 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2023-11-26T06:58:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="Red Hat Enterprise Linux" 2023-11-26T06:58:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="8.6 (Ootpa)" 2023-11-26T06:58:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:ID="rhel" 2023-11-26T06:58:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="fedora" 2023-11-26T06:58:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="8.6" 2023-11-26T06:58:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el8" 2023-11-26T06:58:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-26T06:58:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2023-11-26T06:58:38.779 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-26T06:58:38.780 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://www.redhat.com/" 2023-11-26T06:58:38.780 INFO:teuthology.orchestra.run.smithi203.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-26T06:58:38.780 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-26T06:58:38.780 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T06:58:38.780 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-26T06:58:38.780 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-26T06:58:38.780 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-26T06:58:38.780 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-26T06:58:38.780 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2023-11-26T06:58:38.799 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-26T06:58:38.808 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-26T06:58:38.836 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-26T06:58:38.836 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2023-11-26T06:58:38.838 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2023-11-26T06:58:38.856 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-26T06:58:38.869 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-26T06:58:38.869 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2023-11-26T06:58:38.897 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2023-11-26T06:58:38.923 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-26T06:58:38.976 INFO:teuthology.run_tasks:Running task kernel... 2023-11-26T06:58:38.991 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-26T06:58:38.992 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-26T06:58:38.992 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-26T06:58:38.992 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-26T06:58:38.993 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-26T06:58:39.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T06:58:39.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T06:58:39.011 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2023-11-26T06:58:39.012 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2023-11-26T06:58:39.069 INFO:teuthology.orchestra.run.smithi083.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-26T06:58:39.069 INFO:teuthology.task.kernel:Running kernel on smithi083: 4.18.0-372.9.1.el8.x86_64 2023-11-26T06:58:39.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum install -y kernel 2023-11-26T06:58:39.071 INFO:teuthology.orchestra.run.smithi203.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-26T06:58:39.071 INFO:teuthology.task.kernel:Running kernel on smithi203: 4.18.0-372.9.1.el8.x86_64 2023-11-26T06:58:39.071 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum install -y kernel 2023-11-26T06:58:40.239 INFO:teuthology.orchestra.run.smithi083.stdout:Updating Subscription Management repositories. 2023-11-26T06:58:40.239 INFO:teuthology.orchestra.run.smithi083.stdout:Unable to read consumer identity 2023-11-26T06:58:40.242 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-11-26T06:58:40.243 INFO:teuthology.orchestra.run.smithi203.stdout:Unable to read consumer identity 2023-11-26T06:58:41.708 INFO:teuthology.orchestra.run.smithi083.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-11-26T06:58:41.811 INFO:teuthology.orchestra.run.smithi203.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-11-26T06:58:42.500 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2023-11-26T06:58:42.937 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2023-11-26T06:58:46.092 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 427 kB/s | 24 kB 00:00 2023-11-26T06:58:46.395 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 457 kB/s | 24 kB 00:00 2023-11-26T06:58:46.795 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-26T06:58:46.855 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-11-26T06:58:46.856 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2023-11-26T06:58:46.856 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-11-26T06:58:46.962 DEBUG:teuthology.orchestra.run.smithi083:> echo no | sudo yum reinstall kernel || true 2023-11-26T06:58:47.114 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-26T06:58:47.166 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2023-11-26T06:58:47.167 INFO:teuthology.orchestra.run.smithi203.stdout:Nothing to do. 2023-11-26T06:58:47.167 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2023-11-26T06:58:47.286 DEBUG:teuthology.orchestra.run.smithi203:> echo no | sudo yum reinstall kernel || true 2023-11-26T06:58:47.293 INFO:teuthology.orchestra.run.smithi083.stdout:Updating Subscription Management repositories. 2023-11-26T06:58:47.293 INFO:teuthology.orchestra.run.smithi083.stdout:Unable to read consumer identity 2023-11-26T06:58:47.549 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:01 ago on Sun 26 Nov 2023 06:58:46 AM UTC. 2023-11-26T06:58:47.589 INFO:teuthology.orchestra.run.smithi083.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-26T06:58:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:Error: No packages marked for reinstall. 2023-11-26T06:58:47.658 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-11-26T06:58:47.658 INFO:teuthology.orchestra.run.smithi203.stdout:Unable to read consumer identity 2023-11-26T06:58:47.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum reinstall -y kernel || true 2023-11-26T06:58:47.922 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:01 ago on Sun 26 Nov 2023 06:58:46 AM UTC. 2023-11-26T06:58:47.962 INFO:teuthology.orchestra.run.smithi203.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-26T06:58:47.992 INFO:teuthology.orchestra.run.smithi203.stderr:Error: No packages marked for reinstall. 2023-11-26T06:58:48.004 INFO:teuthology.orchestra.run.smithi083.stdout:Updating Subscription Management repositories. 2023-11-26T06:58:48.004 INFO:teuthology.orchestra.run.smithi083.stdout:Unable to read consumer identity 2023-11-26T06:58:48.040 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum reinstall -y kernel || true 2023-11-26T06:58:48.257 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:02 ago on Sun 26 Nov 2023 06:58:46 AM UTC. 2023-11-26T06:58:48.297 INFO:teuthology.orchestra.run.smithi083.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-26T06:58:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:Error: No packages marked for reinstall. 2023-11-26T06:58:48.366 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2023-11-26T06:58:48.371 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-11-26T06:58:48.371 INFO:teuthology.orchestra.run.smithi203.stdout:Unable to read consumer identity 2023-11-26T06:58:48.446 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-26T06:58:48.446 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-26T06:58:48.446 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-26T06:58:48.446 DEBUG:teuthology.parallel:result is None 2023-11-26T06:58:48.625 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:00:02 ago on Sun 26 Nov 2023 06:58:46 AM UTC. 2023-11-26T06:58:48.667 INFO:teuthology.orchestra.run.smithi203.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-26T06:58:48.690 INFO:teuthology.orchestra.run.smithi203.stderr:Error: No packages marked for reinstall. 2023-11-26T06:58:48.737 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q kernel | sort -rV | head -n 1 2023-11-26T06:58:48.816 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-26T06:58:48.817 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-26T06:58:48.817 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-26T06:58:48.817 DEBUG:teuthology.parallel:result is None 2023-11-26T06:58:48.817 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-26T06:58:48.824 INFO:teuthology.task.internal:Creating test directory... 2023-11-26T06:58:48.825 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-26T06:58:48.827 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-26T06:58:48.843 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-26T06:58:48.863 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-26T06:58:48.889 INFO:teuthology.task.internal:Creating archive directory... 2023-11-26T06:58:48.889 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-26T06:58:48.892 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-26T06:58:48.913 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-26T06:58:48.920 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-26T06:58:48.920 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-26T06:58:48.949 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-26T06:58:48.990 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T06:58:48.997 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T06:58:49.009 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T06:58:49.010 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T06:58:49.011 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-26T06:58:49.018 INFO:teuthology.task.internal:Configuring sudo... 2023-11-26T06:58:49.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-26T06:58:49.054 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-26T06:58:49.081 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-26T06:58:49.089 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-26T06:58:49.089 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-26T06:58:49.121 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-26T06:58:49.137 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-26T06:58:49.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-26T06:58:49.269 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-26T06:58:49.351 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-26T06:58:49.393 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T06:58:49.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-26T06:58:49.460 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-26T06:58:49.505 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-26T06:58:49.550 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-26T06:58:49.630 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-26T06:58:49.675 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T06:58:49.675 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-26T06:58:49.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2023-11-26T06:58:49.746 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2023-11-26T06:58:49.802 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-26T06:58:49.840 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-26T06:58:50.153 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-26T06:58:50.161 INFO:teuthology.task.internal:Starting timer... 2023-11-26T06:58:50.162 INFO:teuthology.run_tasks:Running task pcp... 2023-11-26T06:58:50.190 INFO:teuthology.run_tasks:Running task selinux... 2023-11-26T06:58:50.215 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-26T06:58:50.215 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2023-11-26T06:58:50.279 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2023-11-26T06:58:50.281 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2023-11-26T06:58:50.345 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2023-11-26T06:58:50.347 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-26T06:58:50.347 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2023-11-26T06:58:50.367 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2023-11-26T06:58:50.367 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2023-11-26T06:58:50.386 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2023-11-26T06:58:50.386 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2023-11-26T06:58:50.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-26T06:58:50.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T06:58:50.434 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-26T06:58:50.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T06:58:50.462 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-26T06:58:50.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2023-11-26T06:58:50.500 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2023-11-26T06:58:50.531 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-26T06:58:50.541 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-26T06:58:50.541 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-26T06:58:50.556 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-26T06:58:50.608 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-26T07:07:59.082 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2023-11-26T07:07:59.083 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-11-26T07:07:59.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:07:59.155 DEBUG:teuthology.orchestra.run.smithi083:> true 2023-11-26T07:07:59.229 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-11-26T07:07:59.229 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-11-26T07:07:59.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:07:59.298 DEBUG:teuthology.orchestra.run.smithi203:> true 2023-11-26T07:07:59.324 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-11-26T07:07:59.324 INFO:teuthology.run_tasks:Running task clock... 2023-11-26T07:07:59.338 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-26T07:07:59.338 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-26T07:07:59.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:07:59.341 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-26T07:07:59.341 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:07:59.371 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-26T07:07:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-26T07:07:59.396 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-26T07:07:59.411 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-26T07:07:59.467 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2023-11-26T07:07:59.476 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2023-11-26T07:07:59.479 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2023-11-26T07:07:59.487 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2023-11-26T07:07:59.494 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-26T07:07:59.502 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-26T07:07:59.508 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-26T07:07:59.515 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-26T07:07:59.580 INFO:teuthology.orchestra.run.smithi083.stderr:bash: ntpq: command not found 2023-11-26T07:07:59.582 INFO:teuthology.orchestra.run.smithi203.stderr:bash: ntpq: command not found 2023-11-26T07:07:59.583 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-26T07:07:59.583 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2023-11-26T07:07:59.583 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-26T07:07:59.583 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-26T07:07:59.583 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-26T07:07:59.583 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-26T07:07:59.586 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-26T07:07:59.586 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2023-11-26T07:07:59.587 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-26T07:07:59.587 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-26T07:07:59.587 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-26T07:07:59.587 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-26T07:07:59.587 INFO:teuthology.run_tasks:Running task install... 2023-11-26T07:07:59.596 DEBUG:teuthology.task.install:project ceph 2023-11-26T07:07:59.596 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}} 2023-11-26T07:07:59.596 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'} 2023-11-26T07:07:59.596 INFO:teuthology.task.install:Using flavor: default 2023-11-26T07:07:59.602 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-26T07:07:59.602 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-26T07:07:59.603 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99', 'tag': None, 'wait_for_package': False} 2023-11-26T07:07:59.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:07:59.605 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99', 'tag': None, 'wait_for_package': False} 2023-11-26T07:07:59.605 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:07:59.784 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/centos/8/flavors/default/ 2023-11-26T07:07:59.784 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.g44c2237f 2023-11-26T07:07:59.838 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/centos/8/flavors/default/ 2023-11-26T07:07:59.838 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.g44c2237f 2023-11-26T07:07:59.953 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-26T07:07:59.954 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:07:59.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-26T07:07:59.981 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-26T07:07:59.981 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/44c2237fe77713214148c1fd622c872f346f1b99/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-26T07:07:59.985 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-26T07:07:59.985 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:07:59.985 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-26T07:08:00.012 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-26T07:08:00.012 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/44c2237fe77713214148c1fd622c872f346f1b99/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-26T07:08:00.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-26T07:08:00.079 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-26T07:08:00.126 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-26T07:08:00.155 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-26T07:08:00.191 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2023-11-26T07:08:00.193 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2023-11-26T07:08:00.221 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2023-11-26T07:08:00.223 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2023-11-26T07:08:00.563 INFO:teuthology.orchestra.run.smithi083.stdout:Updating Subscription Management repositories. 2023-11-26T07:08:00.595 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-11-26T07:08:01.557 INFO:teuthology.orchestra.run.smithi203.stdout:44 files removed 2023-11-26T07:08:01.580 INFO:teuthology.orchestra.run.smithi083.stdout:44 files removed 2023-11-26T07:08:01.609 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-26T07:08:01.633 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-26T07:08:01.941 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-11-26T07:08:01.964 INFO:teuthology.orchestra.run.smithi083.stdout:Updating Subscription Management repositories. 2023-11-26T07:08:03.220 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 137 kB/s | 81 kB 00:00 2023-11-26T07:08:03.221 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 140 kB/s | 81 kB 00:00 2023-11-26T07:08:03.513 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 36 kB/s | 9.5 kB 00:00 2023-11-26T07:08:03.546 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 32 kB/s | 9.5 kB 00:00 2023-11-26T07:08:03.772 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 6.3 kB/s | 1.5 kB 00:00 2023-11-26T07:08:03.822 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 5.9 kB/s | 1.5 kB 00:00 2023-11-26T07:08:03.940 INFO:teuthology.orchestra.run.smithi083.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-11-26T07:08:03.989 INFO:teuthology.orchestra.run.smithi203.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-11-26T07:08:09.290 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 3.0 MB/s | 16 MB 00:05 2023-11-26T07:08:09.348 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 3.0 MB/s | 16 MB 00:05 2023-11-26T07:08:12.815 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 624 kB/s | 24 kB 00:00 2023-11-26T07:08:12.890 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 534 kB/s | 24 kB 00:00 2023-11-26T07:08:13.675 INFO:teuthology.orchestra.run.smithi083.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-11-26T07:08:13.817 INFO:teuthology.orchestra.run.smithi203.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-11-26T07:08:21.551 INFO:teuthology.orchestra.run.smithi083.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 MB/s | 53 MB 00:00 2023-11-26T07:08:21.868 INFO:teuthology.orchestra.run.smithi203.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-11-26T07:08:32.724 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-26T07:08:32.725 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-26T07:08:32.739 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-26T07:08:32.740 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-26T07:08:32.741 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-26T07:08:32.742 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-26T07:08:32.744 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-26T07:08:32.748 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-26T07:08:32.748 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-26T07:08:32.877 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2023-11-26T07:08:32.890 INFO:teuthology.orchestra.run.smithi083.stdout:=========================================================================================================================== 2023-11-26T07:08:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2023-11-26T07:08:32.890 INFO:teuthology.orchestra.run.smithi083.stdout:=========================================================================================================================== 2023-11-26T07:08:32.890 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2023-11-26T07:08:32.890 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-26T07:08:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-26T07:08:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 6.4 k 2023-11-26T07:08:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 5.4 M 2023-11-26T07:08:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 919 k 2023-11-26T07:08:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 171 k 2023-11-26T07:08:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 1.6 M 2023-11-26T07:08:32.891 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 133 k 2023-11-26T07:08:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 1.8 M 2023-11-26T07:08:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 7.4 M 2023-11-26T07:08:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 50 k 2023-11-26T07:08:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 14 M 2023-11-26T07:08:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 49 M 2023-11-26T07:08:32.892 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 262 k 2023-11-26T07:08:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 409 k 2023-11-26T07:08:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 263 k 2023-11-26T07:08:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 18 k 2023-11-26T07:08:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-26T07:08:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-26T07:08:32.893 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-26T07:08:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 31 k 2023-11-26T07:08:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 838 k 2023-11-26T07:08:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 139 k 2023-11-26T07:08:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-26T07:08:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-26T07:08:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-26T07:08:32.894 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-26T07:08:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-26T07:08:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-26T07:08:32.895 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 196 k 2023-11-26T07:08:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 371 k 2023-11-26T07:08:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 365 k 2023-11-26T07:08:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 107 k 2023-11-26T07:08:32.896 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 93 k 2023-11-26T07:08:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 3.6 M 2023-11-26T07:08:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 181 k 2023-11-26T07:08:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-26T07:08:32.897 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2023-11-26T07:08:32.897 INFO:teuthology.orchestra.run.smithi083.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-26T07:08:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 3.8 M 2023-11-26T07:08:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 3.7 M 2023-11-26T07:08:32.898 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2023-11-26T07:08:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-26T07:08:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 23 M 2023-11-26T07:08:32.898 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 28 k 2023-11-26T07:08:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 2.3 M 2023-11-26T07:08:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 243 k 2023-11-26T07:08:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 4.6 M 2023-11-26T07:08:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 18 M 2023-11-26T07:08:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 15 k 2023-11-26T07:08:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 24 k 2023-11-26T07:08:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-26T07:08:32.899 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-26T07:08:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-26T07:08:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-26T07:08:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-26T07:08:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 183 k 2023-11-26T07:08:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-26T07:08:32.900 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-26T07:08:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-26T07:08:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-26T07:08:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-26T07:08:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-26T07:08:32.901 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 517 k 2023-11-26T07:08:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-26T07:08:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 6.9 M 2023-11-26T07:08:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-26T07:08:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-26T07:08:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-26T07:08:32.902 INFO:teuthology.orchestra.run.smithi083.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-26T07:08:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-26T07:08:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-26T07:08:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-26T07:08:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-26T07:08:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-26T07:08:32.903 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-26T07:08:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-26T07:08:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-26T07:08:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-26T07:08:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-26T07:08:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-26T07:08:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-26T07:08:32.904 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 45 k 2023-11-26T07:08:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 128 k 2023-11-26T07:08:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-26T07:08:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-26T07:08:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-26T07:08:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-26T07:08:32.905 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-26T07:08:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-26T07:08:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-26T07:08:32.906 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-26T07:08:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-26T07:08:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-26T07:08:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-26T07:08:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-26T07:08:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-26T07:08:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-26T07:08:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-26T07:08:32.907 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-26T07:08:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-26T07:08:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-26T07:08:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-26T07:08:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-26T07:08:32.908 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-26T07:08:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-26T07:08:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-26T07:08:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-26T07:08:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-26T07:08:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-26T07:08:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-26T07:08:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-26T07:08:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-26T07:08:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-26T07:08:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-26T07:08:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-26T07:08:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-26T07:08:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-26T07:08:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-26T07:08:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-26T07:08:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-26T07:08:32.910 INFO:teuthology.orchestra.run.smithi083.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-26T07:08:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-26T07:08:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-26T07:08:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-26T07:08:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-26T07:08:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-26T07:08:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-26T07:08:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-26T07:08:32.911 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-26T07:08:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-26T07:08:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-26T07:08:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-26T07:08:32.912 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling module streams: 2023-11-26T07:08:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql 10 2023-11-26T07:08:32.912 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:08:32.912 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2023-11-26T07:08:32.912 INFO:teuthology.orchestra.run.smithi083.stdout:=========================================================================================================================== 2023-11-26T07:08:32.913 INFO:teuthology.orchestra.run.smithi083.stdout:Install 128 Packages 2023-11-26T07:08:32.913 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 3 Packages 2023-11-26T07:08:32.913 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:08:32.913 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 209 M 2023-11-26T07:08:32.914 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2023-11-26T07:08:32.999 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-26T07:08:33.001 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-26T07:08:33.015 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-26T07:08:33.016 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-26T07:08:33.017 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-26T07:08:33.019 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-26T07:08:33.021 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-26T07:08:33.024 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-26T07:08:33.026 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-26T07:08:33.157 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2023-11-26T07:08:33.170 INFO:teuthology.orchestra.run.smithi203.stdout:=========================================================================================================================== 2023-11-26T07:08:33.170 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2023-11-26T07:08:33.170 INFO:teuthology.orchestra.run.smithi203.stdout:=========================================================================================================================== 2023-11-26T07:08:33.170 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2023-11-26T07:08:33.170 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-26T07:08:33.170 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-26T07:08:33.170 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 6.4 k 2023-11-26T07:08:33.170 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 5.4 M 2023-11-26T07:08:33.170 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 919 k 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 171 k 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 1.6 M 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 133 k 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 1.8 M 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 7.4 M 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 50 k 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 14 M 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 49 M 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 262 k 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 409 k 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 263 k 2023-11-26T07:08:33.171 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 18 k 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 31 k 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 838 k 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 139 k 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-26T07:08:33.172 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 196 k 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 371 k 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 365 k 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 107 k 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 93 k 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 3.6 M 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 181 k 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 3.8 M 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 3.7 M 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2023-11-26T07:08:33.173 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 23 M 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 28 k 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 2.3 M 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 243 k 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 4.6 M 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 18 M 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.g44c2237f.el8 ceph-noarch 15 k 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 24 k 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-26T07:08:33.174 INFO:teuthology.orchestra.run.smithi203.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 183 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 517 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 6.9 M 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-26T07:08:33.175 INFO:teuthology.orchestra.run.smithi203.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-26T07:08:33.176 INFO:teuthology.orchestra.run.smithi203.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 45 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:18.0.0-7484.g44c2237f.el8 ceph 128 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-26T07:08:33.177 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-26T07:08:33.178 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-26T07:08:33.179 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-26T07:08:33.180 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-26T07:08:33.181 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-26T07:08:33.181 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-26T07:08:33.181 INFO:teuthology.orchestra.run.smithi203.stdout:Enabling module streams: 2023-11-26T07:08:33.181 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql 10 2023-11-26T07:08:33.181 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:08:33.181 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2023-11-26T07:08:33.181 INFO:teuthology.orchestra.run.smithi203.stdout:=========================================================================================================================== 2023-11-26T07:08:33.181 INFO:teuthology.orchestra.run.smithi203.stdout:Install 128 Packages 2023-11-26T07:08:33.181 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 3 Packages 2023-11-26T07:08:33.181 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:08:33.182 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 209 M 2023-11-26T07:08:33.182 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2023-11-26T07:08:33.260 INFO:teuthology.orchestra.run.smithi083.stdout:(1/131): ceph-18.0.0-7484.g44c2237f.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2023-11-26T07:08:33.468 INFO:teuthology.orchestra.run.smithi203.stdout:(1/131): ceph-18.0.0-7484.g44c2237f.el8.x86_64. 54 kB/s | 6.4 kB 00:00 2023-11-26T07:08:33.668 INFO:teuthology.orchestra.run.smithi083.stdout:(2/131): ceph-fuse-18.0.0-7484.g44c2237f.el8.x8 2.2 MB/s | 919 kB 00:00 2023-11-26T07:08:33.802 INFO:teuthology.orchestra.run.smithi083.stdout:(3/131): ceph-immutable-object-cache-18.0.0-748 1.3 MB/s | 171 kB 00:00 2023-11-26T07:08:33.844 INFO:teuthology.orchestra.run.smithi203.stdout:(2/131): ceph-fuse-18.0.0-7484.g44c2237f.el8.x8 2.4 MB/s | 919 kB 00:00 2023-11-26T07:08:33.927 INFO:teuthology.orchestra.run.smithi083.stdout:(4/131): ceph-base-18.0.0-7484.g44c2237f.el8.x8 6.3 MB/s | 5.4 MB 00:00 2023-11-26T07:08:33.944 INFO:teuthology.orchestra.run.smithi203.stdout:(3/131): ceph-immutable-object-cache-18.0.0-748 1.7 MB/s | 171 kB 00:00 2023-11-26T07:08:34.211 INFO:teuthology.orchestra.run.smithi083.stdout:(5/131): ceph-mgr-18.0.0-7484.g44c2237f.el8.x86 5.7 MB/s | 1.6 MB 00:00 2023-11-26T07:08:34.569 INFO:teuthology.orchestra.run.smithi203.stdout:(4/131): ceph-base-18.0.0-7484.g44c2237f.el8.x8 4.4 MB/s | 5.4 MB 00:01 2023-11-26T07:08:34.669 INFO:teuthology.orchestra.run.smithi083.stdout:(6/131): ceph-mds-18.0.0-7484.g44c2237f.el8.x86 2.7 MB/s | 2.3 MB 00:00 2023-11-26T07:08:34.837 INFO:teuthology.orchestra.run.smithi203.stdout:(5/131): ceph-mds-18.0.0-7484.g44c2237f.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-11-26T07:08:34.928 INFO:teuthology.orchestra.run.smithi083.stdout:(7/131): ceph-mon-18.0.0-7484.g44c2237f.el8.x86 6.5 MB/s | 4.6 MB 00:00 2023-11-26T07:08:34.937 INFO:teuthology.orchestra.run.smithi203.stdout:(6/131): ceph-mgr-18.0.0-7484.g44c2237f.el8.x86 4.4 MB/s | 1.6 MB 00:00 2023-11-26T07:08:35.779 INFO:teuthology.orchestra.run.smithi203.stdout:(7/131): ceph-mon-18.0.0-7484.g44c2237f.el8.x86 4.9 MB/s | 4.6 MB 00:00 2023-11-26T07:08:35.862 INFO:teuthology.orchestra.run.smithi083.stdout:(8/131): ceph-common-18.0.0-7484.g44c2237f.el8. 8.1 MB/s | 23 MB 00:02 2023-11-26T07:08:35.954 INFO:teuthology.orchestra.run.smithi083.stdout:(9/131): ceph-selinux-18.0.0-7484.g44c2237f.el8 266 kB/s | 24 kB 00:00 2023-11-26T07:08:38.862 INFO:teuthology.orchestra.run.smithi203.stdout:(8/131): ceph-radosgw-18.0.0-7484.g44c2237f.el8 4.5 MB/s | 14 MB 00:03 2023-11-26T07:08:38.937 INFO:teuthology.orchestra.run.smithi203.stdout:(9/131): ceph-selinux-18.0.0-7484.g44c2237f.el8 325 kB/s | 24 kB 00:00 2023-11-26T07:08:39.162 INFO:teuthology.orchestra.run.smithi083.stdout:(10/131): ceph-radosgw-18.0.0-7484.g44c2237f.el 3.3 MB/s | 14 MB 00:04 2023-11-26T07:08:39.562 INFO:teuthology.orchestra.run.smithi083.stdout:(11/131): cephfs-mirror-18.0.0-7484.g44c2237f.e 658 kB/s | 263 kB 00:00 2023-11-26T07:08:39.663 INFO:teuthology.orchestra.run.smithi083.stdout:(12/131): libcephfs-devel-18.0.0-7484.g44c2237f 310 kB/s | 31 kB 00:00 2023-11-26T07:08:39.773 INFO:teuthology.orchestra.run.smithi203.stdout:(10/131): ceph-osd-18.0.0-7484.g44c2237f.el8.x8 3.7 MB/s | 18 MB 00:04 2023-11-26T07:08:40.071 INFO:teuthology.orchestra.run.smithi203.stdout:(11/131): cephfs-mirror-18.0.0-7484.g44c2237f.e 877 kB/s | 263 kB 00:00 2023-11-26T07:08:40.213 INFO:teuthology.orchestra.run.smithi203.stdout:(12/131): libcephfs-devel-18.0.0-7484.g44c2237f 219 kB/s | 31 kB 00:00 2023-11-26T07:08:40.238 INFO:teuthology.orchestra.run.smithi083.stdout:(13/131): libcephfs2-18.0.0-7484.g44c2237f.el8. 1.4 MB/s | 838 kB 00:00 2023-11-26T07:08:40.388 INFO:teuthology.orchestra.run.smithi083.stdout:(14/131): libcephsqlite-18.0.0-7484.g44c2237f.e 1.2 MB/s | 183 kB 00:00 2023-11-26T07:08:40.547 INFO:teuthology.orchestra.run.smithi083.stdout:(15/131): librados-devel-18.0.0-7484.g44c2237f. 878 kB/s | 139 kB 00:00 2023-11-26T07:08:40.922 INFO:teuthology.orchestra.run.smithi083.stdout:(16/131): libradosstriper1-18.0.0-7484.g44c2237 1.3 MB/s | 517 kB 00:00 2023-11-26T07:08:41.139 INFO:teuthology.orchestra.run.smithi203.stdout:(13/131): libcephfs2-18.0.0-7484.g44c2237f.el8. 907 kB/s | 838 kB 00:00 2023-11-26T07:08:41.396 INFO:teuthology.orchestra.run.smithi203.stdout:(14/131): libcephsqlite-18.0.0-7484.g44c2237f.e 712 kB/s | 183 kB 00:00 2023-11-26T07:08:41.564 INFO:teuthology.orchestra.run.smithi203.stdout:(15/131): librados-devel-18.0.0-7484.g44c2237f. 828 kB/s | 139 kB 00:00 2023-11-26T07:08:41.956 INFO:teuthology.orchestra.run.smithi203.stdout:(16/131): libradosstriper1-18.0.0-7484.g44c2237 1.3 MB/s | 517 kB 00:00 2023-11-26T07:08:44.565 INFO:teuthology.orchestra.run.smithi203.stdout:(17/131): ceph-common-18.0.0-7484.g44c2237f.el8 2.0 MB/s | 23 MB 00:11 2023-11-26T07:08:44.681 INFO:teuthology.orchestra.run.smithi203.stdout:(18/131): python3-ceph-argparse-18.0.0-7484.g44 385 kB/s | 45 kB 00:00 2023-11-26T07:08:44.890 INFO:teuthology.orchestra.run.smithi203.stdout:(19/131): python3-ceph-common-18.0.0-7484.g44c2 617 kB/s | 128 kB 00:00 2023-11-26T07:08:45.156 INFO:teuthology.orchestra.run.smithi083.stdout:(17/131): librgw2-18.0.0-7484.g44c2237f.el8.x86 1.6 MB/s | 6.9 MB 00:04 2023-11-26T07:08:45.216 INFO:teuthology.orchestra.run.smithi203.stdout:(20/131): python3-cephfs-18.0.0-7484.g44c2237f. 604 kB/s | 196 kB 00:00 2023-11-26T07:08:45.256 INFO:teuthology.orchestra.run.smithi083.stdout:(18/131): python3-ceph-argparse-18.0.0-7484.g44 449 kB/s | 45 kB 00:00 2023-11-26T07:08:45.333 INFO:teuthology.orchestra.run.smithi203.stdout:(21/131): librgw2-18.0.0-7484.g44c2237f.el8.x86 2.0 MB/s | 6.9 MB 00:03 2023-11-26T07:08:45.499 INFO:teuthology.orchestra.run.smithi203.stdout:(22/131): python3-rbd-18.0.0-7484.g44c2237f.el8 2.1 MB/s | 365 kB 00:00 2023-11-26T07:08:45.525 INFO:teuthology.orchestra.run.smithi203.stdout:(23/131): python3-rados-18.0.0-7484.g44c2237f.e 1.2 MB/s | 371 kB 00:00 2023-11-26T07:08:45.531 INFO:teuthology.orchestra.run.smithi083.stdout:(19/131): ceph-osd-18.0.0-7484.g44c2237f.el8.x8 1.7 MB/s | 18 MB 00:10 2023-11-26T07:08:45.556 INFO:teuthology.orchestra.run.smithi083.stdout:(20/131): python3-ceph-common-18.0.0-7484.g44c2 428 kB/s | 128 kB 00:00 2023-11-26T07:08:45.608 INFO:teuthology.orchestra.run.smithi203.stdout:(24/131): python3-rgw-18.0.0-7484.g44c2237f.el8 983 kB/s | 107 kB 00:00 2023-11-26T07:08:45.634 INFO:teuthology.orchestra.run.smithi203.stdout:(25/131): rbd-fuse-18.0.0-7484.g44c2237f.el8.x8 860 kB/s | 93 kB 00:00 2023-11-26T07:08:45.723 INFO:teuthology.orchestra.run.smithi083.stdout:(21/131): python3-cephfs-18.0.0-7484.g44c2237f. 1.0 MB/s | 196 kB 00:00 2023-11-26T07:08:45.824 INFO:teuthology.orchestra.run.smithi083.stdout:(22/131): python3-rados-18.0.0-7484.g44c2237f.e 1.4 MB/s | 371 kB 00:00 2023-11-26T07:08:45.843 INFO:teuthology.orchestra.run.smithi203.stdout:(26/131): rbd-nbd-18.0.0-7484.g44c2237f.el8.x86 866 kB/s | 181 kB 00:00 2023-11-26T07:08:45.924 INFO:teuthology.orchestra.run.smithi083.stdout:(23/131): python3-rgw-18.0.0-7484.g44c2237f.el8 1.0 MB/s | 107 kB 00:00 2023-11-26T07:08:45.968 INFO:teuthology.orchestra.run.smithi203.stdout:(27/131): ceph-grafana-dashboards-18.0.0-7484.g 223 kB/s | 28 kB 00:00 2023-11-26T07:08:46.024 INFO:teuthology.orchestra.run.smithi083.stdout:(24/131): rbd-fuse-18.0.0-7484.g44c2237f.el8.x8 936 kB/s | 93 kB 00:00 2023-11-26T07:08:46.049 INFO:teuthology.orchestra.run.smithi083.stdout:(25/131): python3-rbd-18.0.0-7484.g44c2237f.el8 1.1 MB/s | 365 kB 00:00 2023-11-26T07:08:46.110 INFO:teuthology.orchestra.run.smithi203.stdout:(28/131): ceph-mgr-cephadm-18.0.0-7484.g44c2237 935 kB/s | 133 kB 00:00 2023-11-26T07:08:46.241 INFO:teuthology.orchestra.run.smithi083.stdout:(26/131): rbd-nbd-18.0.0-7484.g44c2237f.el8.x86 943 kB/s | 181 kB 00:00 2023-11-26T07:08:46.367 INFO:teuthology.orchestra.run.smithi083.stdout:(27/131): ceph-grafana-dashboards-18.0.0-7484.g 223 kB/s | 28 kB 00:00 2023-11-26T07:08:46.484 INFO:teuthology.orchestra.run.smithi083.stdout:(28/131): ceph-mgr-cephadm-18.0.0-7484.g44c2237 1.1 MB/s | 133 kB 00:00 2023-11-26T07:08:46.985 INFO:teuthology.orchestra.run.smithi203.stdout:(29/131): rbd-mirror-18.0.0-7484.g44c2237f.el8. 2.6 MB/s | 3.6 MB 00:01 2023-11-26T07:08:47.360 INFO:teuthology.orchestra.run.smithi203.stdout:(30/131): ceph-mgr-dashboard-18.0.0-7484.g44c22 1.4 MB/s | 1.8 MB 00:01 2023-11-26T07:08:47.653 INFO:teuthology.orchestra.run.smithi203.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7484.g44 832 kB/s | 243 kB 00:00 2023-11-26T07:08:47.744 INFO:teuthology.orchestra.run.smithi203.stdout:(32/131): ceph-mgr-rook-18.0.0-7484.g44c2237f.e 547 kB/s | 50 kB 00:00 2023-11-26T07:08:47.828 INFO:teuthology.orchestra.run.smithi203.stdout:(33/131): ceph-prometheus-alerts-18.0.0-7484.g4 180 kB/s | 15 kB 00:00 2023-11-26T07:08:48.051 INFO:teuthology.orchestra.run.smithi083.stdout:(29/131): ceph-mgr-dashboard-18.0.0-7484.g44c22 1.1 MB/s | 1.8 MB 00:01 2023-11-26T07:08:48.078 INFO:teuthology.orchestra.run.smithi203.stdout:(34/131): ceph-volume-18.0.0-7484.g44c2237f.el8 1.0 MB/s | 262 kB 00:00 2023-11-26T07:08:48.404 INFO:teuthology.orchestra.run.smithi203.stdout:(35/131): cephadm-18.0.0-7484.g44c2237f.el8.noa 1.2 MB/s | 409 kB 00:00 2023-11-26T07:08:48.479 INFO:teuthology.orchestra.run.smithi203.stdout:(36/131): cephfs-top-18.0.0-7484.g44c2237f.el8. 242 kB/s | 18 kB 00:00 2023-11-26T07:08:48.509 INFO:teuthology.orchestra.run.smithi083.stdout:(30/131): ceph-test-18.0.0-7484.g44c2237f.el8.x 3.9 MB/s | 49 MB 00:12 2023-11-26T07:08:48.576 INFO:teuthology.orchestra.run.smithi083.stdout:(31/131): rbd-mirror-18.0.0-7484.g44c2237f.el8. 1.4 MB/s | 3.6 MB 00:02 2023-11-26T07:08:48.668 INFO:teuthology.orchestra.run.smithi083.stdout:(32/131): ceph-mgr-rook-18.0.0-7484.g44c2237f.e 548 kB/s | 50 kB 00:00 2023-11-26T07:08:48.702 INFO:teuthology.orchestra.run.smithi083.stdout:(33/131): ceph-mgr-modules-core-18.0.0-7484.g44 1.2 MB/s | 243 kB 00:00 2023-11-26T07:08:48.760 INFO:teuthology.orchestra.run.smithi083.stdout:(34/131): ceph-prometheus-alerts-18.0.0-7484.g4 163 kB/s | 15 kB 00:00 2023-11-26T07:08:48.788 INFO:teuthology.orchestra.run.smithi203.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 994 kB/s | 306 kB 00:00 2023-11-26T07:08:48.863 INFO:teuthology.orchestra.run.smithi203.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 786 kB/s | 59 kB 00:00 2023-11-26T07:08:48.938 INFO:teuthology.orchestra.run.smithi203.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2023-11-26T07:08:49.047 INFO:teuthology.orchestra.run.smithi203.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2023-11-26T07:08:49.069 INFO:teuthology.orchestra.run.smithi083.stdout:(35/131): cephadm-18.0.0-7484.g44c2237f.el8.noa 1.3 MB/s | 409 kB 00:00 2023-11-26T07:08:49.114 INFO:teuthology.orchestra.run.smithi203.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 655 kB/s | 44 kB 00:00 2023-11-26T07:08:49.153 INFO:teuthology.orchestra.run.smithi083.stdout:(36/131): ceph-volume-18.0.0-7484.g44c2237f.el8 581 kB/s | 262 kB 00:00 2023-11-26T07:08:49.170 INFO:teuthology.orchestra.run.smithi083.stdout:(37/131): cephfs-top-18.0.0-7484.g44c2237f.el8. 180 kB/s | 18 kB 00:00 2023-11-26T07:08:49.189 INFO:teuthology.orchestra.run.smithi203.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2023-11-26T07:08:49.248 INFO:teuthology.orchestra.run.smithi203.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 567 kB/s | 33 kB 00:00 2023-11-26T07:08:49.315 INFO:teuthology.orchestra.run.smithi203.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 245 kB/s | 16 kB 00:00 2023-11-26T07:08:49.382 INFO:teuthology.orchestra.run.smithi203.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-11-26T07:08:49.482 INFO:teuthology.orchestra.run.smithi203.stdout:(46/131): ceph-mgr-diskprediction-local-18.0.0- 3.0 MB/s | 7.4 MB 00:02 2023-11-26T07:08:49.508 INFO:teuthology.orchestra.run.smithi203.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2023-11-26T07:08:49.575 INFO:teuthology.orchestra.run.smithi203.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-11-26T07:08:49.642 INFO:teuthology.orchestra.run.smithi203.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-11-26T07:08:49.695 INFO:teuthology.orchestra.run.smithi083.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 112 kB/s | 59 kB 00:00 2023-11-26T07:08:49.709 INFO:teuthology.orchestra.run.smithi203.stdout:(50/131): python3-google-auth-1.1.1-10.el8.noar 359 kB/s | 81 kB 00:00 2023-11-26T07:08:49.751 INFO:teuthology.orchestra.run.smithi203.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 11 MB/s | 1.2 MB 00:00 2023-11-26T07:08:49.785 INFO:teuthology.orchestra.run.smithi203.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 647 kB/s | 49 kB 00:00 2023-11-26T07:08:49.818 INFO:teuthology.orchestra.run.smithi203.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 878 kB/s | 59 kB 00:00 2023-11-26T07:08:49.860 INFO:teuthology.orchestra.run.smithi203.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 800 kB/s | 60 kB 00:00 2023-11-26T07:08:49.886 INFO:teuthology.orchestra.run.smithi203.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-11-26T07:08:49.887 INFO:teuthology.orchestra.run.smithi083.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 417 kB/s | 306 kB 00:00 2023-11-26T07:08:49.912 INFO:teuthology.orchestra.run.smithi083.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 346 kB/s | 75 kB 00:00 2023-11-26T07:08:49.919 INFO:teuthology.orchestra.run.smithi203.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 265 kB/s | 16 kB 00:00 2023-11-26T07:08:49.953 INFO:teuthology.orchestra.run.smithi203.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 503 kB/s | 34 kB 00:00 2023-11-26T07:08:50.020 INFO:teuthology.orchestra.run.smithi203.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 964 kB/s | 64 kB 00:00 2023-11-26T07:08:50.045 INFO:teuthology.orchestra.run.smithi203.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-11-26T07:08:50.079 INFO:teuthology.orchestra.run.smithi203.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 319 kB/s | 19 kB 00:00 2023-11-26T07:08:50.088 INFO:teuthology.orchestra.run.smithi083.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 249 kB/s | 44 kB 00:00 2023-11-26T07:08:50.104 INFO:teuthology.orchestra.run.smithi203.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 408 kB/s | 24 kB 00:00 2023-11-26T07:08:50.130 INFO:teuthology.orchestra.run.smithi083.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.9 MB/s | 476 kB 00:00 2023-11-26T07:08:50.146 INFO:teuthology.orchestra.run.smithi203.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 437 kB/s | 29 kB 00:00 2023-11-26T07:08:50.195 INFO:teuthology.orchestra.run.smithi203.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 304 kB/s | 27 kB 00:00 2023-11-26T07:08:50.230 INFO:teuthology.orchestra.run.smithi203.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-11-26T07:08:50.271 INFO:teuthology.orchestra.run.smithi083.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 233 kB/s | 33 kB 00:00 2023-11-26T07:08:50.280 INFO:teuthology.orchestra.run.smithi203.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2023-11-26T07:08:50.306 INFO:teuthology.orchestra.run.smithi203.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 813 kB/s | 61 kB 00:00 2023-11-26T07:08:50.339 INFO:teuthology.orchestra.run.smithi203.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-11-26T07:08:50.355 INFO:teuthology.orchestra.run.smithi083.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 693 kB/s | 185 kB 00:00 2023-11-26T07:08:50.357 INFO:teuthology.orchestra.run.smithi203.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 455 kB/s | 23 kB 00:00 2023-11-26T07:08:50.390 INFO:teuthology.orchestra.run.smithi203.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.1 MB/s | 70 kB 00:00 2023-11-26T07:08:50.405 INFO:teuthology.orchestra.run.smithi083.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 122 kB/s | 16 kB 00:00 2023-11-26T07:08:50.516 INFO:teuthology.orchestra.run.smithi203.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 9.9 MB/s | 1.7 MB 00:00 2023-11-26T07:08:50.547 INFO:teuthology.orchestra.run.smithi083.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 899 kB/s | 173 kB 00:00 2023-11-26T07:08:50.581 INFO:teuthology.orchestra.run.smithi083.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-11-26T07:08:50.649 INFO:teuthology.orchestra.run.smithi203.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 828 kB/s | 110 kB 00:00 2023-11-26T07:08:50.706 INFO:teuthology.orchestra.run.smithi083.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 512 kB/s | 81 kB 00:00 2023-11-26T07:08:50.732 INFO:teuthology.orchestra.run.smithi083.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 74 kB/s | 11 kB 00:00 2023-11-26T07:08:50.849 INFO:teuthology.orchestra.run.smithi083.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 124 kB/s | 18 kB 00:00 2023-11-26T07:08:50.883 INFO:teuthology.orchestra.run.smithi203.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 232 kB/s | 54 kB 00:00 2023-11-26T07:08:50.933 INFO:teuthology.orchestra.run.smithi203.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-11-26T07:08:50.983 INFO:teuthology.orchestra.run.smithi203.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-11-26T07:08:50.999 INFO:teuthology.orchestra.run.smithi083.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 325 kB/s | 49 kB 00:00 2023-11-26T07:08:51.026 INFO:teuthology.orchestra.run.smithi203.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 928 kB/s | 39 kB 00:00 2023-11-26T07:08:51.124 INFO:teuthology.orchestra.run.smithi083.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 3.0 MB/s | 1.2 MB 00:00 2023-11-26T07:08:51.150 INFO:teuthology.orchestra.run.smithi083.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 391 kB/s | 59 kB 00:00 2023-11-26T07:08:51.168 INFO:teuthology.orchestra.run.smithi203.stdout:(76/131): python3-scipy-1.0.0-21.module+el8.5.0 18 MB/s | 14 MB 00:00 2023-11-26T07:08:51.201 INFO:teuthology.orchestra.run.smithi203.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 3.8 MB/s | 688 kB 00:00 2023-11-26T07:08:51.235 INFO:teuthology.orchestra.run.smithi203.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-11-26T07:08:51.275 INFO:teuthology.orchestra.run.smithi083.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 401 kB/s | 60 kB 00:00 2023-11-26T07:08:51.277 INFO:teuthology.orchestra.run.smithi203.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.0 MB/s | 157 kB 00:00 2023-11-26T07:08:51.302 INFO:teuthology.orchestra.run.smithi203.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2023-11-26T07:08:51.317 INFO:teuthology.orchestra.run.smithi083.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-11-26T07:08:51.344 INFO:teuthology.orchestra.run.smithi203.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-11-26T07:08:51.386 INFO:teuthology.orchestra.run.smithi203.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 642 kB/s | 27 kB 00:00 2023-11-26T07:08:51.426 INFO:teuthology.orchestra.run.smithi083.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 104 kB/s | 16 kB 00:00 2023-11-26T07:08:51.428 INFO:teuthology.orchestra.run.smithi203.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-26T07:08:51.459 INFO:teuthology.orchestra.run.smithi083.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 237 kB/s | 34 kB 00:00 2023-11-26T07:08:51.470 INFO:teuthology.orchestra.run.smithi203.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-11-26T07:08:51.554 INFO:teuthology.orchestra.run.smithi203.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-11-26T07:08:51.568 INFO:teuthology.orchestra.run.smithi083.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-11-26T07:08:51.601 INFO:teuthology.orchestra.run.smithi083.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 453 kB/s | 64 kB 00:00 2023-11-26T07:08:51.604 INFO:teuthology.orchestra.run.smithi203.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.9 MB/s | 457 kB 00:00 2023-11-26T07:08:51.646 INFO:teuthology.orchestra.run.smithi203.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-11-26T07:08:51.710 INFO:teuthology.orchestra.run.smithi083.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 132 kB/s | 19 kB 00:00 2023-11-26T07:08:51.713 INFO:teuthology.orchestra.run.smithi203.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 15 MB/s | 3.7 MB 00:00 2023-11-26T07:08:51.739 INFO:teuthology.orchestra.run.smithi203.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 695 kB/s | 64 kB 00:00 2023-11-26T07:08:51.744 INFO:teuthology.orchestra.run.smithi083.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 168 kB/s | 24 kB 00:00 2023-11-26T07:08:51.797 INFO:teuthology.orchestra.run.smithi203.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.5 MB/s | 126 kB 00:00 2023-11-26T07:08:51.831 INFO:teuthology.orchestra.run.smithi203.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 218 kB/s | 20 kB 00:00 2023-11-26T07:08:51.844 INFO:teuthology.orchestra.run.smithi083.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 218 kB/s | 29 kB 00:00 2023-11-26T07:08:51.857 INFO:teuthology.orchestra.run.smithi203.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.7 MB/s | 103 kB 00:00 2023-11-26T07:08:51.869 INFO:teuthology.orchestra.run.smithi083.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 220 kB/s | 27 kB 00:00 2023-11-26T07:08:51.890 INFO:teuthology.orchestra.run.smithi203.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-11-26T07:08:51.923 INFO:teuthology.orchestra.run.smithi203.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 5.8 MB/s | 198 kB 00:00 2023-11-26T07:08:52.003 INFO:teuthology.orchestra.run.smithi083.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.6 MB/s | 255 kB 00:00 2023-11-26T07:08:52.028 INFO:teuthology.orchestra.run.smithi083.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-11-26T07:08:52.032 INFO:teuthology.orchestra.run.smithi203.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 28 MB/s | 4.9 MB 00:00 2023-11-26T07:08:52.074 INFO:teuthology.orchestra.run.smithi203.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-11-26T07:08:52.145 INFO:teuthology.orchestra.run.smithi083.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 432 kB/s | 61 kB 00:00 2023-11-26T07:08:52.149 INFO:teuthology.orchestra.run.smithi203.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-11-26T07:08:52.170 INFO:teuthology.orchestra.run.smithi083.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 602 kB/s | 85 kB 00:00 2023-11-26T07:08:52.174 INFO:teuthology.orchestra.run.smithi203.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2023-11-26T07:08:52.225 INFO:teuthology.orchestra.run.smithi203.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 16 MB/s | 808 kB 00:00 2023-11-26T07:08:52.279 INFO:teuthology.orchestra.run.smithi083.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 171 kB/s | 23 kB 00:00 2023-11-26T07:08:52.305 INFO:teuthology.orchestra.run.smithi083.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-26T07:08:52.325 INFO:teuthology.orchestra.run.smithi203.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-11-26T07:08:52.401 INFO:teuthology.orchestra.run.smithi203.stdout:(101/131): postgresql-server-10.21-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-11-26T07:08:52.418 INFO:teuthology.orchestra.run.smithi203.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 506 kB/s | 47 kB 00:00 2023-11-26T07:08:52.443 INFO:teuthology.orchestra.run.smithi203.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 4.7 MB/s | 200 kB 00:00 2023-11-26T07:08:52.469 INFO:teuthology.orchestra.run.smithi203.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-11-26T07:08:52.494 INFO:teuthology.orchestra.run.smithi203.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-26T07:08:52.519 INFO:teuthology.orchestra.run.smithi203.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 957 kB/s | 48 kB 00:00 2023-11-26T07:08:52.528 INFO:teuthology.orchestra.run.smithi083.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 4.9 MB/s | 1.7 MB 00:00 2023-11-26T07:08:52.544 INFO:teuthology.orchestra.run.smithi203.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-26T07:08:52.570 INFO:teuthology.orchestra.run.smithi203.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 848 kB/s | 43 kB 00:00 2023-11-26T07:08:52.595 INFO:teuthology.orchestra.run.smithi203.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-11-26T07:08:52.621 INFO:teuthology.orchestra.run.smithi203.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-11-26T07:08:52.654 INFO:teuthology.orchestra.run.smithi203.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-11-26T07:08:52.680 INFO:teuthology.orchestra.run.smithi203.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 319 kB/s | 19 kB 00:00 2023-11-26T07:08:52.705 INFO:teuthology.orchestra.run.smithi203.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-11-26T07:08:52.705 INFO:teuthology.orchestra.run.smithi083.stdout:(71/131): python3-scipy-1.0.0-21.module+el8.5.0 34 MB/s | 14 MB 00:00 2023-11-26T07:08:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 570 kB/s | 110 kB 00:00 2023-11-26T07:08:52.739 INFO:teuthology.orchestra.run.smithi083.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-11-26T07:08:52.747 INFO:teuthology.orchestra.run.smithi203.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 8.2 MB/s | 559 kB 00:00 2023-11-26T07:08:52.772 INFO:teuthology.orchestra.run.smithi203.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 267 kB/s | 18 kB 00:00 2023-11-26T07:08:52.773 INFO:teuthology.orchestra.run.smithi083.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-11-26T07:08:52.798 INFO:teuthology.orchestra.run.smithi083.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-11-26T07:08:52.823 INFO:teuthology.orchestra.run.smithi083.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 774 kB/s | 39 kB 00:00 2023-11-26T07:08:52.836 INFO:teuthology.orchestra.run.smithi203.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.7 MB/s | 605 kB 00:00 2023-11-26T07:08:52.857 INFO:teuthology.orchestra.run.smithi083.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-11-26T07:08:52.872 INFO:teuthology.orchestra.run.smithi203.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.3 MB/s | 238 kB 00:00 2023-11-26T07:08:52.899 INFO:teuthology.orchestra.run.smithi083.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2023-11-26T07:08:52.906 INFO:teuthology.orchestra.run.smithi203.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 2.1 MB/s | 72 kB 00:00 2023-11-26T07:08:52.924 INFO:teuthology.orchestra.run.smithi083.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-11-26T07:08:52.965 INFO:teuthology.orchestra.run.smithi203.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 18 MB/s | 2.3 MB 00:00 2023-11-26T07:08:52.982 INFO:teuthology.orchestra.run.smithi203.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-11-26T07:08:53.007 INFO:teuthology.orchestra.run.smithi203.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-11-26T07:08:53.008 INFO:teuthology.orchestra.run.smithi083.stdout:(80/131): ceph-mgr-diskprediction-local-18.0.0- 1.5 MB/s | 7.4 MB 00:04 2023-11-26T07:08:53.033 INFO:teuthology.orchestra.run.smithi203.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-11-26T07:08:53.033 INFO:teuthology.orchestra.run.smithi083.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2023-11-26T07:08:53.049 INFO:teuthology.orchestra.run.smithi203.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-11-26T07:08:53.066 INFO:teuthology.orchestra.run.smithi203.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-11-26T07:08:53.067 INFO:teuthology.orchestra.run.smithi083.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.9 MB/s | 713 kB 00:00 2023-11-26T07:08:53.094 INFO:teuthology.orchestra.run.smithi083.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 501 kB/s | 27 kB 00:00 2023-11-26T07:08:53.100 INFO:teuthology.orchestra.run.smithi203.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 13 MB/s | 644 kB 00:00 2023-11-26T07:08:53.117 INFO:teuthology.orchestra.run.smithi083.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 6.2 MB/s | 320 kB 00:00 2023-11-26T07:08:53.126 INFO:teuthology.orchestra.run.smithi203.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.8 MB/s | 171 kB 00:00 2023-11-26T07:08:53.142 INFO:teuthology.orchestra.run.smithi083.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 873 kB/s | 44 kB 00:00 2023-11-26T07:08:53.151 INFO:teuthology.orchestra.run.smithi203.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-11-26T07:08:53.185 INFO:teuthology.orchestra.run.smithi083.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-11-26T07:08:53.226 INFO:teuthology.orchestra.run.smithi083.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-11-26T07:08:53.260 INFO:teuthology.orchestra.run.smithi083.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2023-11-26T07:08:53.319 INFO:teuthology.orchestra.run.smithi083.stdout:(89/131): python3-numpy-1.14.3-10.el8.x86_64.rp 19 MB/s | 3.7 MB 00:00 2023-11-26T07:08:53.344 INFO:teuthology.orchestra.run.smithi083.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.5 MB/s | 126 kB 00:00 2023-11-26T07:08:53.369 INFO:teuthology.orchestra.run.smithi083.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-11-26T07:08:53.486 INFO:teuthology.orchestra.run.smithi083.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 729 kB/s | 103 kB 00:00 2023-11-26T07:08:53.745 INFO:teuthology.orchestra.run.smithi083.stdout:(93/131): python3-babel-2.5.1-7.el8.noarch.rpm 6.5 MB/s | 4.8 MB 00:00 2023-11-26T07:08:53.779 INFO:teuthology.orchestra.run.smithi083.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 1.7 MB/s | 709 kB 00:00 2023-11-26T07:08:53.804 INFO:teuthology.orchestra.run.smithi083.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2023-11-26T07:08:53.854 INFO:teuthology.orchestra.run.smithi083.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.9 MB/s | 202 kB 00:00 2023-11-26T07:08:53.896 INFO:teuthology.orchestra.run.smithi083.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-11-26T07:08:54.013 INFO:teuthology.orchestra.run.smithi083.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 9.2 MB/s | 4.9 MB 00:00 2023-11-26T07:08:54.093 INFO:teuthology.orchestra.run.smithi203.stdout:(128/131): ceph-test-18.0.0-7484.g44c2237f.el8. 3.3 MB/s | 49 MB 00:15 2023-11-26T07:08:54.097 INFO:teuthology.orchestra.run.smithi083.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-11-26T07:08:54.143 INFO:teuthology.orchestra.run.smithi203.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-26T07:08:54.147 INFO:teuthology.orchestra.run.smithi083.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-11-26T07:08:54.189 INFO:teuthology.orchestra.run.smithi083.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-11-26T07:08:54.240 INFO:teuthology.orchestra.run.smithi083.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-11-26T07:08:54.265 INFO:teuthology.orchestra.run.smithi083.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.6 MB/s | 200 kB 00:00 2023-11-26T07:08:54.290 INFO:teuthology.orchestra.run.smithi083.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-11-26T07:08:54.317 INFO:teuthology.orchestra.run.smithi083.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-26T07:08:54.333 INFO:teuthology.orchestra.run.smithi083.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.1 MB/s | 48 kB 00:00 2023-11-26T07:08:54.358 INFO:teuthology.orchestra.run.smithi083.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-11-26T07:08:54.375 INFO:teuthology.orchestra.run.smithi083.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 1.0 MB/s | 43 kB 00:00 2023-11-26T07:08:54.409 INFO:teuthology.orchestra.run.smithi083.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-11-26T07:08:54.434 INFO:teuthology.orchestra.run.smithi083.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-11-26T07:08:54.501 INFO:teuthology.orchestra.run.smithi083.stdout:(111/131): postgresql-server-10.21-2.module+el8 8.4 MB/s | 5.1 MB 00:00 2023-11-26T07:08:54.526 INFO:teuthology.orchestra.run.smithi083.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 204 kB/s | 19 kB 00:00 2023-11-26T07:08:54.551 INFO:teuthology.orchestra.run.smithi083.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-11-26T07:08:54.594 INFO:teuthology.orchestra.run.smithi083.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.0 MB/s | 560 kB 00:00 2023-11-26T07:08:54.619 INFO:teuthology.orchestra.run.smithi083.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 267 kB/s | 18 kB 00:00 2023-11-26T07:08:54.652 INFO:teuthology.orchestra.run.smithi083.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.3 MB/s | 559 kB 00:00 2023-11-26T07:08:54.677 INFO:teuthology.orchestra.run.smithi083.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-11-26T07:08:54.703 INFO:teuthology.orchestra.run.smithi083.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-11-26T07:08:54.728 INFO:teuthology.orchestra.run.smithi083.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-11-26T07:08:54.753 INFO:teuthology.orchestra.run.smithi083.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-26T07:08:54.778 INFO:teuthology.orchestra.run.smithi083.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-11-26T07:08:54.804 INFO:teuthology.orchestra.run.smithi083.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-11-26T07:08:54.829 INFO:teuthology.orchestra.run.smithi083.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-11-26T07:08:54.855 INFO:teuthology.orchestra.run.smithi083.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 763 kB/s | 39 kB 00:00 2023-11-26T07:08:54.897 INFO:teuthology.orchestra.run.smithi083.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-11-26T07:08:54.922 INFO:teuthology.orchestra.run.smithi083.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-11-26T07:08:54.947 INFO:teuthology.orchestra.run.smithi083.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-11-26T07:08:55.006 INFO:teuthology.orchestra.run.smithi083.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.6 MB/s | 2.3 MB 00:00 2023-11-26T07:08:55.065 INFO:teuthology.orchestra.run.smithi083.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-11-26T07:08:55.310 INFO:teuthology.orchestra.run.smithi203.stdout:(130/131): librados2-18.0.0-7484.g44c2237f.el8. 1.7 MB/s | 3.8 MB 00:02 2023-11-26T07:08:56.035 INFO:teuthology.orchestra.run.smithi203.stdout:(131/131): librbd1-18.0.0-7484.g44c2237f.el8.x8 1.3 MB/s | 3.7 MB 00:02 2023-11-26T07:08:56.038 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2023-11-26T07:08:56.038 INFO:teuthology.orchestra.run.smithi203.stdout:Total 9.2 MB/s | 209 MB 00:22 2023-11-26T07:08:56.740 INFO:teuthology.orchestra.run.smithi083.stdout:(130/131): librbd1-18.0.0-7484.g44c2237f.el8.x8 2.1 MB/s | 3.7 MB 00:01 2023-11-26T07:08:56.864 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2023-11-26T07:08:56.985 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2023-11-26T07:08:56.986 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2023-11-26T07:08:57.173 INFO:teuthology.orchestra.run.smithi083.stdout:(131/131): librados2-18.0.0-7484.g44c2237f.el8. 1.7 MB/s | 3.8 MB 00:02 2023-11-26T07:08:57.177 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2023-11-26T07:08:57.177 INFO:teuthology.orchestra.run.smithi083.stdout:Total 8.6 MB/s | 209 MB 00:24 2023-11-26T07:08:58.033 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2023-11-26T07:08:58.096 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2023-11-26T07:08:58.100 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2023-11-26T07:08:58.154 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2023-11-26T07:08:58.155 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2023-11-26T07:08:59.312 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2023-11-26T07:08:59.316 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2023-11-26T07:09:03.386 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2023-11-26T07:09:03.539 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-26T07:09:03.695 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-26T07:09:03.914 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-26T07:09:04.075 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-26T07:09:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2023-11-26T07:09:04.201 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-26T07:09:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-26T07:09:04.417 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-26T07:09:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-26T07:09:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-26T07:09:04.824 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-26T07:09:04.874 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-26T07:09:04.998 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-26T07:09:05.041 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:18.0.0-7484.g44c2237f.el8.x86_64 6/134 2023-11-26T07:09:05.207 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-26T07:09:05.451 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:18.0.0-7484.g44c2237f.el8.x86_64 6/134 2023-11-26T07:09:05.619 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-26T07:09:05.633 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:18.0.0-7484.g44c2237f.el8.x86_64 7/134 2023-11-26T07:09:05.766 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:18.0.0-7484.g44c2237f.el8.x86_64 7/134 2023-11-26T07:09:05.882 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:18.0.0-7484.g44c2237f.el8.x86_64 6/134 2023-11-26T07:09:05.949 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:18.0.0-7484.g44c2237f.el8.x86_64 8/134 2023-11-26T07:09:06.050 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.g44c2237f.el8.x86_64 8/134 2023-11-26T07:09:06.250 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:18.0.0-7484.g44c2237f.el8.x86_64 9/134 2023-11-26T07:09:06.287 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:18.0.0-7484.g44c2237f.el8.x86_64 6/134 2023-11-26T07:09:06.482 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:18.0.0-7484.g44c2237f.el8.x86_64 7/134 2023-11-26T07:09:06.496 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-26T07:09:06.604 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:18.0.0-7484.g44c2237f.el8.x86_64 7/134 2023-11-26T07:09:06.719 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-26T07:09:06.773 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:18.0.0-7484.g44c2237f.el8.x86_64 8/134 2023-11-26T07:09:06.855 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.g44c2237f.el8.x86_64 8/134 2023-11-26T07:09:07.048 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:18.0.0-7484.g44c2237f.el8.x86_64 9/134 2023-11-26T07:09:07.140 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-26T07:09:07.301 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-26T07:09:07.381 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-26T07:09:07.510 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-26T07:09:07.583 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:18.0.0-7484.g44c2237f.el8.x86_64 14/134 2023-11-26T07:09:07.694 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.g44c2237f.el8.x86_64 14/134 2023-11-26T07:09:07.849 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:18.0.0-7484.g44c2237f.el8.x86 15/134 2023-11-26T07:09:07.920 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-26T07:09:07.925 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.g44c2237f.el8.x86 15/134 2023-11-26T07:09:08.066 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-26T07:09:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-26T07:09:08.196 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-26T07:09:08.324 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-26T07:09:08.340 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:18.0.0-7484.g44c2237f.el8.x86_64 14/134 2023-11-26T07:09:08.439 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.g44c2237f.el8.x86_64 14/134 2023-11-26T07:09:08.582 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:18.0.0-7484.g44c2237f.el8.x86 15/134 2023-11-26T07:09:08.651 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.g44c2237f.el8.x86 15/134 2023-11-26T07:09:08.806 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-26T07:09:08.925 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-26T07:09:08.931 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-26T07:09:09.090 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-26T07:09:09.099 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-26T07:09:09.683 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-26T07:09:09.701 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-26T07:09:09.866 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-26T07:09:10.090 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-26T07:09:10.570 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-26T07:09:10.683 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-26T07:09:10.848 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-26T07:09:10.889 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-26T07:09:11.059 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-26T07:09:11.261 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-26T07:09:11.430 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-26T07:09:11.594 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-26T07:09:11.648 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-26T07:09:11.875 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-26T07:09:11.876 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-26T07:09:12.039 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:18.0.0-7484.g44c2237f.el8. 26/134 2023-11-26T07:09:12.116 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-26T07:09:12.193 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.g44c2237f.el 27/134 2023-11-26T07:09:12.335 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-26T07:09:12.520 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-26T07:09:12.737 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-26T07:09:12.950 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-26T07:09:13.083 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:18.0.0-7484.g44c2237f.el8.x86_6 28/134 2023-11-26T07:09:13.138 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:18.0.0-7484.g44c2237f.el8. 26/134 2023-11-26T07:09:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.g44c2237f.el 27/134 2023-11-26T07:09:13.390 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-26T07:09:13.685 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-26T07:09:13.940 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-26T07:09:14.146 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:18.0.0-7484.g44c2237f.el8.x86_6 28/134 2023-11-26T07:09:14.165 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-26T07:09:14.446 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-26T07:09:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-26T07:09:14.965 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-26T07:09:15.173 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-26T07:09:15.948 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-26T07:09:16.283 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-26T07:09:16.460 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-26T07:09:16.650 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-26T07:09:16.862 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:18.0.0-7484.g44c2237f.el8.x86_64 36/134 2023-11-26T07:09:16.938 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-26T07:09:17.011 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:18.0.0-7484.g44c2237f.el8.x86_6 37/134 2023-11-26T07:09:17.167 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-26T07:09:17.231 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-26T07:09:17.382 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-26T07:09:17.400 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-26T07:09:17.550 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-26T07:09:17.625 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-26T07:09:17.701 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-26T07:09:17.811 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:18.0.0-7484.g44c2237f.el8.x86_64 36/134 2023-11-26T07:09:17.903 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-26T07:09:17.968 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:18.0.0-7484.g44c2237f.el8.x86_6 37/134 2023-11-26T07:09:18.110 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-26T07:09:18.145 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-26T07:09:18.324 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-26T07:09:18.499 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-26T07:09:18.517 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-26T07:09:18.642 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-26T07:09:18.810 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-26T07:09:18.839 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-26T07:09:18.999 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-26T07:09:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-26T07:09:19.211 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-26T07:09:19.423 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-26T07:09:19.469 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-26T07:09:19.709 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-26T07:09:19.790 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-26T07:09:19.898 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-26T07:09:20.039 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-26T07:09:20.118 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-26T07:09:20.207 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-26T07:09:20.376 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-26T07:09:20.419 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-26T07:09:20.578 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-26T07:09:20.704 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-26T07:09:20.725 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-26T07:09:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-26T07:09:21.071 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-26T07:09:21.165 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-26T07:09:21.335 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-26T07:09:21.389 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-26T07:09:21.554 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-26T07:09:21.561 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-26T07:09:21.725 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-26T07:09:21.770 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-26T07:09:21.954 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-26T07:09:22.042 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-26T07:09:22.098 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-26T07:09:22.247 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-26T07:09:22.316 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-26T07:09:22.319 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-26T07:09:22.551 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-26T07:09:22.559 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-26T07:09:22.742 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-26T07:09:22.795 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-26T07:09:22.922 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-26T07:09:22.962 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-26T07:09:23.003 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-26T07:09:23.123 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-26T07:09:23.226 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-26T07:09:23.296 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-26T07:09:23.352 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-26T07:09:23.414 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-26T07:09:23.579 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-26T07:09:23.602 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-26T07:09:23.758 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-26T07:09:23.791 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-26T07:09:23.963 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-26T07:09:24.040 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-26T07:09:24.048 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-26T07:09:24.250 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-26T07:09:24.270 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-26T07:09:24.414 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-26T07:09:24.442 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-26T07:09:24.579 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-26T07:09:24.607 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-26T07:09:24.757 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-26T07:09:24.796 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-26T07:09:25.016 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-26T07:09:25.046 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-26T07:09:25.198 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-26T07:09:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-26T07:09:25.349 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-26T07:09:25.416 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-26T07:09:25.527 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-26T07:09:25.565 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-26T07:09:25.722 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-26T07:09:25.745 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-26T07:09:25.932 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-26T07:09:25.932 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-26T07:09:26.114 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-26T07:09:26.151 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-26T07:09:26.265 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-26T07:09:26.398 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-26T07:09:26.418 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-26T07:09:26.617 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-26T07:09:26.645 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-26T07:09:26.802 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-26T07:09:26.817 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-26T07:09:26.995 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-26T07:09:27.002 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-26T07:09:27.252 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-26T07:09:27.487 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-26T07:09:27.653 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-26T07:09:27.866 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-26T07:09:28.114 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-26T07:09:28.359 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-26T07:09:28.587 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-26T07:09:28.873 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-26T07:09:28.945 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-26T07:09:29.198 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-26T07:09:29.207 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-26T07:09:29.366 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-26T07:09:29.461 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-26T07:09:29.736 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-26T07:09:30.026 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-26T07:09:30.181 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-26T07:09:30.201 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-26T07:09:30.346 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:18.0.0-7484.g44c2237f.el8.x86_64 92/134 2023-11-26T07:09:30.421 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:18.0.0-7484.g44c2237f.el8.x86_64 92/134 2023-11-26T07:09:30.596 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:18.0.0-7484.g44c2237f.el8.x86_64 93/134 2023-11-26T07:09:31.014 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-26T07:09:31.179 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:18.0.0-7484.g44c2237f.el8.x86_64 92/134 2023-11-26T07:09:31.235 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:18.0.0-7484.g44c2237f.el8.x86_64 92/134 2023-11-26T07:09:31.412 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:18.0.0-7484.g44c2237f.el8.x86_64 93/134 2023-11-26T07:09:34.401 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 94/134 2023-11-26T07:09:34.612 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 94/134 2023-11-26T07:09:35.117 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 94/134 2023-11-26T07:09:35.199 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 94/134 2023-11-26T07:09:35.362 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 94/134 2023-11-26T07:09:35.474 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:18.0.0-7484.g44c2237f.el8.x86_64 95/134 2023-11-26T07:09:35.539 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.g44c2237f.el8.x86_64 95/134 2023-11-26T07:09:35.686 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:18.0.0-7484.g44c2237f.el8.x86_64 96/134 2023-11-26T07:09:35.961 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 94/134 2023-11-26T07:09:36.484 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:18.0.0-7484.g44c2237f.el8.x86_64 95/134 2023-11-26T07:09:36.564 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.g44c2237f.el8.x86_64 95/134 2023-11-26T07:09:36.711 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:18.0.0-7484.g44c2237f.el8.x86_64 96/134 2023-11-26T07:09:54.427 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.g44c2237f.el8.x86_64 96/134 2023-11-26T07:09:54.427 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2023-11-26T07:09:54.427 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2023-11-26T07:09:54.427 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /dev 2023-11-26T07:09:54.427 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /run 2023-11-26T07:09:54.427 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2023-11-26T07:09:54.427 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2023-11-26T07:09:54.427 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2023-11-26T07:09:54.428 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2023-11-26T07:09:54.428 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /dev 2023-11-26T07:09:54.428 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:09:54.776 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:18.0.0-7484.g44c2237f.el8.noarch 97/134 2023-11-26T07:09:55.392 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.g44c2237f.el8.x86_64 96/134 2023-11-26T07:09:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2023-11-26T07:09:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2023-11-26T07:09:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /dev 2023-11-26T07:09:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /run 2023-11-26T07:09:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2023-11-26T07:09:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2023-11-26T07:09:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2023-11-26T07:09:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2023-11-26T07:09:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /dev 2023-11-26T07:09:55.393 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:09:55.734 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:18.0.0-7484.g44c2237f.el8.noarch 97/134 2023-11-26T07:09:56.374 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.g44c2237f.el8.noarch 97/134 2023-11-26T07:09:56.554 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:18.0.0-7484.g44c2237f.el8.x86_64 98/134 2023-11-26T07:09:56.775 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.g44c2237f.el8.x86_64 98/134 2023-11-26T07:09:56.925 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:18.0.0-7484.g44c2237f.el8.x86_64 99/134 2023-11-26T07:09:57.331 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.g44c2237f.el8.noarch 97/134 2023-11-26T07:09:57.384 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.g44c2237f.el8.x86_64 99/134 2023-11-26T07:09:57.522 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:18.0.0-7484.g44c2237f.el8.x86_64 98/134 2023-11-26T07:09:57.625 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:18.0.0-7484.g44c2237f.el8.x86_64 100/134 2023-11-26T07:09:57.642 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.g44c2237f.el8.x86_64 100/134 2023-11-26T07:09:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.g44c2237f.el8.x86_64 98/134 2023-11-26T07:09:57.883 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:18.0.0-7484.g44c2237f.el8.x86_64 99/134 2023-11-26T07:09:58.340 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.g44c2237f.el8.x86_64 99/134 2023-11-26T07:09:58.550 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:18.0.0-7484.g44c2237f.el8.x86_64 100/134 2023-11-26T07:09:58.567 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.g44c2237f.el8.x86_64 100/134 2023-11-26T07:09:59.318 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:18.0.0-7484.g44c2237f.el8.noarch 101/134 2023-11-26T07:09:59.473 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:18.0.0-7484.g44c2237f.el8.noarch 101/134 2023-11-26T07:09:59.615 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.g44c2237f.e 102/134 2023-11-26T07:09:59.782 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.g44c2237f. 103/134 2023-11-26T07:09:59.958 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.g44c2237f.el8.noa 104/134 2023-11-26T07:10:00.184 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.g44c2237f.el8.noa 104/134 2023-11-26T07:10:00.297 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:18.0.0-7484.g44c2237f.el8.noarch 101/134 2023-11-26T07:10:00.486 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.g44c2237f.el8.n 105/134 2023-11-26T07:10:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:18.0.0-7484.g44c2237f.el8.noarch 101/134 2023-11-26T07:10:00.615 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.g44c2237f.e 102/134 2023-11-26T07:10:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.g44c2237f. 103/134 2023-11-26T07:10:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.g44c2237f.el8.noa 104/134 2023-11-26T07:10:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.g44c2237f.el8.noa 104/134 2023-11-26T07:10:01.455 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.g44c2237f.el8.n 105/134 2023-11-26T07:10:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.g44c2237f.el8.n 105/134 2023-11-26T07:10:01.599 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.g44c 106/134 2023-11-26T07:10:01.693 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.g44c 106/134 2023-11-26T07:10:02.054 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.g44c2237f.el 107/134 2023-11-26T07:10:02.236 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:18.0.0-7484.g44c2237f.el8.x86_64 108/134 2023-11-26T07:10:02.274 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.g44c2237f.el8.x86_64 108/134 2023-11-26T07:10:02.432 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.g44c2237f.el8.noarch 109/134 2023-11-26T07:10:02.443 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.g44c2237f.el8.noarch 109/134 2023-11-26T07:10:02.473 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.g44c2237f.el8.n 105/134 2023-11-26T07:10:02.641 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.g44c 106/134 2023-11-26T07:10:02.732 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.g44c 106/134 2023-11-26T07:10:03.098 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.g44c2237f.el 107/134 2023-11-26T07:10:03.255 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:18.0.0-7484.g44c2237f.el8.x86_64 108/134 2023-11-26T07:10:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.g44c2237f.el8.x86_64 108/134 2023-11-26T07:10:03.483 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.g44c2237f.el8.noarch 109/134 2023-11-26T07:10:03.494 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.g44c2237f.el8.noarch 109/134 2023-11-26T07:10:03.768 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:18.0.0-7484.g44c2237f.el8.x86_64 110/134 2023-11-26T07:10:03.983 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:18.0.0-7484.g44c2237f.el8.x86_64 111/134 2023-11-26T07:10:04.017 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.g44c2237f.el8.x86_64 111/134 2023-11-26T07:10:04.174 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.g44c22 112/134 2023-11-26T07:10:04.226 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.g44c22 112/134 2023-11-26T07:10:04.374 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 113/134 2023-11-26T07:10:04.734 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 113/134 2023-11-26T07:10:04.805 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:18.0.0-7484.g44c2237f.el8.x86_64 110/134 2023-11-26T07:10:04.882 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 114/134 2023-11-26T07:10:05.045 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:18.0.0-7484.g44c2237f.el8.x86_64 111/134 2023-11-26T07:10:05.079 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.g44c2237f.el8.x86_64 111/134 2023-11-26T07:10:05.232 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.g44c22 112/134 2023-11-26T07:10:05.279 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.g44c22 112/134 2023-11-26T07:10:05.432 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 113/134 2023-11-26T07:10:05.792 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 113/134 2023-11-26T07:10:05.966 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 114/134 2023-11-26T07:10:09.585 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 114/134 2023-11-26T07:10:09.872 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:18.0.0-7484.g44c2237f.el8.x86_64 115/134 2023-11-26T07:10:10.014 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-26T07:10:10.209 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-26T07:10:10.756 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 114/134 2023-11-26T07:10:11.057 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:18.0.0-7484.g44c2237f.el8.x86_64 115/134 2023-11-26T07:10:11.144 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-26T07:10:11.223 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-26T07:10:11.414 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-26T07:10:11.468 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-26T07:10:11.641 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 119/134 2023-11-26T07:10:11.835 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-26T07:10:12.019 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:18.0.0-7484.g44c2237f.el8.x86_ 121/134 2023-11-26T07:10:12.165 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-26T07:10:12.283 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-26T07:10:12.577 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-26T07:10:12.767 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 119/134 2023-11-26T07:10:12.952 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-26T07:10:13.135 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:18.0.0-7484.g44c2237f.el8.x86_ 121/134 2023-11-26T07:10:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-26T07:10:14.382 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-26T07:10:14.630 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-26T07:10:14.648 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-26T07:10:14.891 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:18.0.0-7484.g44c2237f.el8.noarch 124/134 2023-11-26T07:10:15.138 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 125/134 2023-11-26T07:10:15.406 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:18.0.0-7484.g44c2237f.el8.x86_64 126/134 2023-11-26T07:10:15.541 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-26T07:10:15.732 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-26T07:10:15.781 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-26T07:10:15.799 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-26T07:10:15.888 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-26T07:10:15.897 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-26T07:10:16.008 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:18.0.0-7484.g44c2237f.el8.noarch 124/134 2023-11-26T07:10:16.117 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-26T07:10:16.255 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 125/134 2023-11-26T07:10:16.305 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-26T07:10:16.385 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-26T07:10:16.517 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:18.0.0-7484.g44c2237f.el8.x86_64 126/134 2023-11-26T07:10:16.613 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-26T07:10:16.626 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-26T07:10:16.626 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-26T07:10:16.808 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-26T07:10:16.808 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-26T07:10:16.832 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-26T07:10:16.981 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-26T07:10:16.989 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-26T07:10:17.037 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-26T07:10:17.141 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-26T07:10:17.245 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-26T07:10:17.322 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-26T07:10:17.406 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-26T07:10:17.597 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-26T07:10:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-26T07:10:17.610 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-26T07:10:17.749 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-26T07:10:17.749 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:18.0.0-7484.g44c2237f.el8.x86_64 1/134 2023-11-26T07:10:17.749 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:18.0.0-7484.g44c2237f.el8.x86_64 2/134 2023-11-26T07:10:17.750 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 3/134 2023-11-26T07:10:17.750 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 4/134 2023-11-26T07:10:17.750 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.g44c22 5/134 2023-11-26T07:10:17.750 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:18.0.0-7484.g44c2237f.el8.x86_64 6/134 2023-11-26T07:10:17.750 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:18.0.0-7484.g44c2237f.el8.x86_64 7/134 2023-11-26T07:10:17.750 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:18.0.0-7484.g44c2237f.el8.x86_64 8/134 2023-11-26T07:10:17.751 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:18.0.0-7484.g44c2237f.el8.x86_64 9/134 2023-11-26T07:10:17.751 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.g44c2237f.el8.x86_64 10/134 2023-11-26T07:10:17.751 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:18.0.0-7484.g44c2237f.el8.x86_64 11/134 2023-11-26T07:10:17.751 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:18.0.0-7484.g44c2237f.el8.x86_64 12/134 2023-11-26T07:10:17.751 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 13/134 2023-11-26T07:10:17.751 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.g44c2237f.el8.x86_ 14/134 2023-11-26T07:10:17.751 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:18.0.0-7484.g44c2237f.el8.x86_64 15/134 2023-11-26T07:10:17.752 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:18.0.0-7484.g44c2237f.el8.x86_64 16/134 2023-11-26T07:10:17.752 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:18.0.0-7484.g44c2237f.el8.x86_6 17/134 2023-11-26T07:10:17.752 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:18.0.0-7484.g44c2237f.el8.x86 18/134 2023-11-26T07:10:17.752 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:18.0.0-7484.g44c2237f.el8.x86_64 19/134 2023-11-26T07:10:17.752 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.g44c2237f.el 20/134 2023-11-26T07:10:17.752 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.g44c2237f.el8. 21/134 2023-11-26T07:10:17.753 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:18.0.0-7484.g44c2237f.el8.x86_6 22/134 2023-11-26T07:10:17.753 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:18.0.0-7484.g44c2237f.el8.x86_64 23/134 2023-11-26T07:10:17.753 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:18.0.0-7484.g44c2237f.el8.x86_64 24/134 2023-11-26T07:10:17.753 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:18.0.0-7484.g44c2237f.el8.x86_64 25/134 2023-11-26T07:10:17.753 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 26/134 2023-11-26T07:10:17.753 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 27/134 2023-11-26T07:10:17.753 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:18.0.0-7484.g44c2237f.el8.x86_64 28/134 2023-11-26T07:10:17.753 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.g44c2237f. 29/134 2023-11-26T07:10:17.754 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.g44c2237f.el8.noa 30/134 2023-11-26T07:10:17.762 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.g44c2237f.el8.n 31/134 2023-11-26T07:10:17.762 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.g44c 32/134 2023-11-26T07:10:17.762 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.g44c2237f.el 33/134 2023-11-26T07:10:17.763 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.g44c2237f.el8.noarch 34/134 2023-11-26T07:10:17.763 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.g44c2237f.e 35/134 2023-11-26T07:10:17.763 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:18.0.0-7484.g44c2237f.el8.noarch 36/134 2023-11-26T07:10:17.763 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:18.0.0-7484.g44c2237f.el8.noarch 37/134 2023-11-26T07:10:17.763 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:18.0.0-7484.g44c2237f.el8.noarch 38/134 2023-11-26T07:10:17.763 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-26T07:10:17.763 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-26T07:10:17.764 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-26T07:10:17.764 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-26T07:10:17.764 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-26T07:10:17.764 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-26T07:10:17.764 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-26T07:10:17.764 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-26T07:10:17.764 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-26T07:10:17.764 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-26T07:10:17.764 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-26T07:10:17.764 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-26T07:10:17.765 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-26T07:10:17.765 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-26T07:10:17.765 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-26T07:10:17.765 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-26T07:10:17.765 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-26T07:10:17.765 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-26T07:10:17.765 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-26T07:10:17.766 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-26T07:10:17.766 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-26T07:10:17.766 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-26T07:10:17.766 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-26T07:10:17.766 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-26T07:10:17.766 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-26T07:10:17.766 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-26T07:10:17.766 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-26T07:10:17.766 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-26T07:10:17.766 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-26T07:10:17.766 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-26T07:10:17.767 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-26T07:10:17.767 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-26T07:10:17.767 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-26T07:10:17.767 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-26T07:10:17.767 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-26T07:10:17.767 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-26T07:10:17.767 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-26T07:10:17.767 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-26T07:10:17.767 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-26T07:10:17.767 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-26T07:10:17.767 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-26T07:10:17.768 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-26T07:10:17.769 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-26T07:10:17.770 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:18.0.0-7484.g44c2237f.el8.x86_64 129/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-26T07:10:17.771 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:18.0.0-7484.g44c2237f.el8.x86_64 131/134 2023-11-26T07:10:17.772 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-26T07:10:17.772 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-26T07:10:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-26T07:10:17.817 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-26T07:10:18.105 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-26T07:10:18.296 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-26T07:10:18.787 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-26T07:10:18.787 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:18.0.0-7484.g44c2237f.el8.x86_64 1/134 2023-11-26T07:10:18.787 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:18.0.0-7484.g44c2237f.el8.x86_64 2/134 2023-11-26T07:10:18.787 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 3/134 2023-11-26T07:10:18.787 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 4/134 2023-11-26T07:10:18.787 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.g44c22 5/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:18.0.0-7484.g44c2237f.el8.x86_64 6/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:18.0.0-7484.g44c2237f.el8.x86_64 7/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:18.0.0-7484.g44c2237f.el8.x86_64 8/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:18.0.0-7484.g44c2237f.el8.x86_64 9/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.g44c2237f.el8.x86_64 10/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:18.0.0-7484.g44c2237f.el8.x86_64 11/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:18.0.0-7484.g44c2237f.el8.x86_64 12/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 13/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.g44c2237f.el8.x86_ 14/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:18.0.0-7484.g44c2237f.el8.x86_64 15/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:18.0.0-7484.g44c2237f.el8.x86_64 16/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:18.0.0-7484.g44c2237f.el8.x86_6 17/134 2023-11-26T07:10:18.788 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:18.0.0-7484.g44c2237f.el8.x86 18/134 2023-11-26T07:10:18.789 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:18.0.0-7484.g44c2237f.el8.x86_64 19/134 2023-11-26T07:10:18.789 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.g44c2237f.el 20/134 2023-11-26T07:10:18.789 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.g44c2237f.el8. 21/134 2023-11-26T07:10:18.789 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:18.0.0-7484.g44c2237f.el8.x86_6 22/134 2023-11-26T07:10:18.789 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:18.0.0-7484.g44c2237f.el8.x86_64 23/134 2023-11-26T07:10:18.789 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:18.0.0-7484.g44c2237f.el8.x86_64 24/134 2023-11-26T07:10:18.789 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:18.0.0-7484.g44c2237f.el8.x86_64 25/134 2023-11-26T07:10:18.789 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 26/134 2023-11-26T07:10:18.789 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 27/134 2023-11-26T07:10:18.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:18.0.0-7484.g44c2237f.el8.x86_64 28/134 2023-11-26T07:10:18.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.g44c2237f. 29/134 2023-11-26T07:10:18.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.g44c2237f.el8.noa 30/134 2023-11-26T07:10:18.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.g44c2237f.el8.n 31/134 2023-11-26T07:10:18.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.g44c 32/134 2023-11-26T07:10:18.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.g44c2237f.el 33/134 2023-11-26T07:10:18.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.g44c2237f.el8.noarch 34/134 2023-11-26T07:10:18.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.g44c2237f.e 35/134 2023-11-26T07:10:18.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:18.0.0-7484.g44c2237f.el8.noarch 36/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:18.0.0-7484.g44c2237f.el8.noarch 37/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:18.0.0-7484.g44c2237f.el8.noarch 38/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-26T07:10:18.800 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-26T07:10:18.801 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-26T07:10:18.802 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-26T07:10:18.803 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-26T07:10:18.804 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-26T07:10:18.805 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:18.0.0-7484.g44c2237f.el8.x86_64 129/134 2023-11-26T07:10:18.806 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-26T07:10:18.807 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:18.0.0-7484.g44c2237f.el8.x86_64 131/134 2023-11-26T07:10:18.807 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-26T07:10:18.807 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-26T07:10:20.573 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-26T07:10:20.573 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:59 ago on Sun 26 Nov 2023 07:08:21 AM UTC. 2023-11-26T07:10:21.515 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-26T07:10:21.515 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:02:00 ago on Sun 26 Nov 2023 07:08:21 AM UTC. 2023-11-26T07:10:23.085 INFO:teuthology.orchestra.run.smithi203.stdout:Installed products updated. 2023-11-26T07:10:23.306 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:23.306 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2023-11-26T07:10:23.307 INFO:teuthology.orchestra.run.smithi203.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-26T07:10:23.307 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.307 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.307 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2023-11-26T07:10:23.307 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-26T07:10:23.307 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.1-7.el8.noarch 2023-11-26T07:10:23.307 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.0.4-10.el8.x86_64 2023-11-26T07:10:23.307 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.307 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.307 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.307 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.308 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.1-9.el8.x86_64 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: jq-1.6-3.el8.x86_64 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-26T07:10:23.309 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: libpq-13.5-1.el8.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.310 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-26T07:10:23.311 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-26T07:10:23.312 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-26T07:10:23.313 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-26T07:10:23.314 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-26T07:10:23.314 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-26T07:10:23.314 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-26T07:10:23.314 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-26T07:10:23.314 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-26T07:10:23.314 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-26T07:10:23.314 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-2.6-1.el8.noarch 2023-11-26T07:10:23.314 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-26T07:10:23.314 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-26T07:10:23.314 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-26T07:10:23.314 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-26T07:10:23.315 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-26T07:10:23.316 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-26T07:10:23.317 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-26T07:10:23.317 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-26T07:10:23.317 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-26T07:10:23.317 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-26T07:10:23.317 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:23.317 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2023-11-26T07:10:23.493 DEBUG:teuthology.parallel:result is None 2023-11-26T07:10:24.063 INFO:teuthology.orchestra.run.smithi083.stdout:Installed products updated. 2023-11-26T07:10:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:24.287 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2023-11-26T07:10:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-26T07:10:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.287 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.287 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2023-11-26T07:10:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-26T07:10:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.1-7.el8.noarch 2023-11-26T07:10:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.0.4-10.el8.x86_64 2023-11-26T07:10:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.288 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:24.289 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.290 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.291 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:18.0.0-7484.g44c2237f.el8.noarch 2023-11-26T07:10:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-26T07:10:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.1-9.el8.x86_64 2023-11-26T07:10:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-26T07:10:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-26T07:10:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: jq-1.6-3.el8.x86_64 2023-11-26T07:10:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-26T07:10:24.292 INFO:teuthology.orchestra.run.smithi083.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-26T07:10:24.293 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-26T07:10:24.293 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.293 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.293 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.293 INFO:teuthology.orchestra.run.smithi083.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-26T07:10:24.293 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-26T07:10:24.293 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-26T07:10:24.294 INFO:teuthology.orchestra.run.smithi083.stdout: libpq-13.5-1.el8.x86_64 2023-11-26T07:10:24.294 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-26T07:10:24.294 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-26T07:10:24.294 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.294 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.294 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-26T07:10:24.295 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.295 INFO:teuthology.orchestra.run.smithi083.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-26T07:10:24.295 INFO:teuthology.orchestra.run.smithi083.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-26T07:10:24.295 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-26T07:10:24.295 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-26T07:10:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-26T07:10:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-26T07:10:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-26T07:10:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-26T07:10:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-26T07:10:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-26T07:10:24.296 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-26T07:10:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-26T07:10:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-26T07:10:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-26T07:10:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-26T07:10:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-26T07:10:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-26T07:10:24.297 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-26T07:10:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-26T07:10:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-26T07:10:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-26T07:10:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-26T07:10:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.298 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-26T07:10:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-26T07:10:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-26T07:10:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-26T07:10:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-26T07:10:24.299 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-26T07:10:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-26T07:10:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-26T07:10:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-26T07:10:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-26T07:10:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-26T07:10:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-26T07:10:24.300 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-26T07:10:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-26T07:10:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-26T07:10:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-26T07:10:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-26T07:10:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-26T07:10:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-26T07:10:24.301 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-26T07:10:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-26T07:10:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-26T07:10:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-2.6-1.el8.noarch 2023-11-26T07:10:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-26T07:10:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-26T07:10:24.302 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-26T07:10:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-26T07:10:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-26T07:10:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-26T07:10:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-26T07:10:24.303 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-26T07:10:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-26T07:10:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-26T07:10:24.304 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-26T07:10:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-26T07:10:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-26T07:10:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-26T07:10:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-26T07:10:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-26T07:10:24.305 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-26T07:10:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-26T07:10:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-26T07:10:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-26T07:10:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-26T07:10:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:18.0.0-7484.g44c2237f.el8.x86_64 2023-11-26T07:10:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-26T07:10:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-26T07:10:24.306 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-26T07:10:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-26T07:10:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-26T07:10:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-26T07:10:24.307 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:24.307 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2023-11-26T07:10:24.528 DEBUG:teuthology.parallel:result is None 2023-11-26T07:10:24.529 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:10:24.749 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-26T07:10:24.773 INFO:teuthology.orchestra.run.smithi083.stdout:18.0.0-7484.g44c2237f.el8 2023-11-26T07:10:24.773 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.g44c2237f.el8 2023-11-26T07:10:24.773 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.g44c2237f is installed. 2023-11-26T07:10:24.774 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:10:24.989 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-26T07:10:25.017 INFO:teuthology.orchestra.run.smithi203.stdout:18.0.0-7484.g44c2237f.el8 2023-11-26T07:10:25.017 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.g44c2237f.el8 2023-11-26T07:10:25.017 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.g44c2237f is installed. 2023-11-26T07:10:25.018 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-26T07:10:25.018 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:25.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-26T07:10:25.061 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:10:25.061 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-26T07:10:25.108 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-26T07:10:25.109 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:25.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2023-11-26T07:10:25.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-26T07:10:25.208 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:10:25.208 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2023-11-26T07:10:25.239 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-26T07:10:25.308 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-26T07:10:25.308 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:25.308 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-26T07:10:25.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-26T07:10:25.405 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:10:25.405 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-26T07:10:25.434 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-26T07:10:25.502 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-26T07:10:25.502 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:25.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2023-11-26T07:10:25.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-26T07:10:25.597 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:10:25.597 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2023-11-26T07:10:25.624 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-26T07:10:25.691 INFO:teuthology.run_tasks:Running task ceph... 2023-11-26T07:10:25.757 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-26T07:10:25.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /var/log/ceph 2023-11-26T07:10:25.759 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /var/log/ceph 2023-11-26T07:10:25.786 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-26T07:10:25.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-26T07:10:25.826 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-26T07:10:25.854 INFO:tasks.ceph:Creating extra log directories... 2023-11-26T07:10:25.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-26T07:10:25.892 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-26T07:10:25.927 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-26T07:10:25.927 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-26T07:10:25.927 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-26T07:10:25.927 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:25.927 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-26T07:10:25.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-26T07:10:26.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-26T07:10:26.128 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:10:26.128 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-26T07:10:26.144 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-26T07:10:26.282 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-26T07:10:26.313 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-26T07:10:26.313 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-26T07:10:26.314 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}', 'email': None, 'first_in_suite': False, 'job_id': '7467353', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-mgr-vol-misc', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1863, 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-mgr-vol-misc', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '44c2237fe77713214148c1fd622c872f346f1b99', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi083.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi203.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-26_06:51:12', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-26T07:10:26.314 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-26T07:10:26.316 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-26T07:10:26.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/run/ceph 2023-11-26T07:10:26.373 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/run/ceph 2023-11-26T07:10:26.431 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:26.431 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2023-11-26T07:10:26.455 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-26T07:10:26.455 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2023-11-26T07:10:26.512 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-26T07:10:26.512 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:10:26.512 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 387684 Links: 1 2023-11-26T07:10:26.512 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:10:26.512 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-11-26T07:10:26.513 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-11-26 07:10:18.495984626 +0000 2023-11-26T07:10:26.513 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-11-26 07:07:27.011238411 +0000 2023-11-26T07:10:26.513 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-11-26 07:07:27.011238411 +0000 2023-11-26T07:10:26.513 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2023-11-26T07:10:26.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-26T07:10:26.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:10:26.578 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:10:26.578 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000257845 s, 2.0 MB/s 2023-11-26T07:10:26.579 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-26T07:10:26.637 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2023-11-26T07:10:26.694 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-26T07:10:26.694 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:10:26.695 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 385872 Links: 1 2023-11-26T07:10:26.695 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:10:26.695 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-11-26T07:10:26.695 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-11-26 07:10:18.495984626 +0000 2023-11-26T07:10:26.696 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-11-26 07:07:27.519226362 +0000 2023-11-26T07:10:26.696 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-11-26 07:07:27.519226362 +0000 2023-11-26T07:10:26.696 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2023-11-26T07:10:26.696 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-26T07:10:26.761 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:10:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:10:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000247765 s, 2.1 MB/s 2023-11-26T07:10:26.762 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-26T07:10:26.819 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2023-11-26T07:10:26.877 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-26T07:10:26.877 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:10:26.877 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 385975 Links: 1 2023-11-26T07:10:26.877 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:10:26.877 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-11-26T07:10:26.877 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-11-26 07:10:18.495984626 +0000 2023-11-26T07:10:26.877 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-11-26 07:07:28.008214764 +0000 2023-11-26T07:10:26.877 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-11-26 07:07:28.008214764 +0000 2023-11-26T07:10:26.877 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2023-11-26T07:10:26.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-26T07:10:26.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:10:26.942 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:10:26.943 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000239095 s, 2.1 MB/s 2023-11-26T07:10:26.944 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-26T07:10:27.001 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2023-11-26T07:10:27.057 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-26T07:10:27.058 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:10:27.058 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 394418 Links: 1 2023-11-26T07:10:27.058 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:10:27.058 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2023-11-26T07:10:27.058 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2023-11-26 07:10:18.495984626 +0000 2023-11-26T07:10:27.058 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2023-11-26 07:07:28.558201720 +0000 2023-11-26T07:10:27.059 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2023-11-26 07:07:28.558201720 +0000 2023-11-26T07:10:27.059 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2023-11-26T07:10:27.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-26T07:10:27.123 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:10:27.123 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:10:27.123 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000226731 s, 2.3 MB/s 2023-11-26T07:10:27.124 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-26T07:10:27.182 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:10:27.182 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:10:27.182 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2023-11-26T07:10:27.202 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-26T07:10:27.202 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2023-11-26T07:10:27.262 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-26T07:10:27.262 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:10:27.262 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 390562 Links: 1 2023-11-26T07:10:27.262 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:10:27.262 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-11-26T07:10:27.262 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-11-26 07:10:17.449801669 +0000 2023-11-26T07:10:27.262 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-11-26 07:07:28.698153693 +0000 2023-11-26T07:10:27.262 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-11-26 07:07:28.698153693 +0000 2023-11-26T07:10:27.262 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-11-26T07:10:27.263 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-26T07:10:27.332 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-11-26T07:10:27.333 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-11-26T07:10:27.333 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000272488 s, 1.9 MB/s 2023-11-26T07:10:27.334 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-26T07:10:27.395 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2023-11-26T07:10:27.454 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-26T07:10:27.454 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:10:27.455 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 391720 Links: 1 2023-11-26T07:10:27.455 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:10:27.455 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-11-26T07:10:27.455 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-11-26 07:10:17.450801644 +0000 2023-11-26T07:10:27.455 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-11-26 07:07:29.200141172 +0000 2023-11-26T07:10:27.455 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-11-26 07:07:29.200141172 +0000 2023-11-26T07:10:27.455 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-11-26T07:10:27.455 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-26T07:10:27.522 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-11-26T07:10:27.522 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-11-26T07:10:27.522 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000256321 s, 2.0 MB/s 2023-11-26T07:10:27.523 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-26T07:10:27.585 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2023-11-26T07:10:27.641 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-26T07:10:27.642 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:10:27.642 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 388839 Links: 1 2023-11-26T07:10:27.642 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:10:27.642 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-11-26T07:10:27.642 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-11-26 07:10:17.450801644 +0000 2023-11-26T07:10:27.642 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-11-26 07:07:29.721128179 +0000 2023-11-26T07:10:27.642 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-11-26 07:07:29.721128179 +0000 2023-11-26T07:10:27.642 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-11-26T07:10:27.642 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-26T07:10:27.709 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-11-26T07:10:27.709 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-11-26T07:10:27.709 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000213205 s, 2.4 MB/s 2023-11-26T07:10:27.710 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-26T07:10:27.768 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2023-11-26T07:10:27.825 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-26T07:10:27.825 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:10:27.825 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 393221 Links: 1 2023-11-26T07:10:27.825 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:10:27.826 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-11-26T07:10:27.826 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-11-26 07:10:17.450801644 +0000 2023-11-26T07:10:27.826 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-11-26 07:07:30.230115484 +0000 2023-11-26T07:10:27.826 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-11-26 07:07:30.230115484 +0000 2023-11-26T07:10:27.826 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-11-26T07:10:27.826 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-26T07:10:27.890 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-11-26T07:10:27.891 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-11-26T07:10:27.891 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000215326 s, 2.4 MB/s 2023-11-26T07:10:27.892 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-26T07:10:27.949 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:10:27.950 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-26T07:10:27.950 INFO:tasks.ceph:Generating config... 2023-11-26T07:10:27.951 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-26T07:10:27.951 INFO:tasks.ceph:[client] debug client = 20 2023-11-26T07:10:27.951 INFO:tasks.ceph:[client] debug ms = 1 2023-11-26T07:10:27.951 INFO:tasks.ceph:[client] pid file = 2023-11-26T07:10:27.951 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-26T07:10:27.951 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-26T07:10:27.952 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-26T07:10:27.952 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-26T07:10:27.953 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-26T07:10:27.953 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-26T07:10:27.953 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-26T07:10:27.953 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-26T07:10:27.953 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-26T07:10:27.953 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-26T07:10:27.953 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-26T07:10:27.953 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-26T07:10:27.953 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-26T07:10:27.953 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-26T07:10:27.953 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-26T07:10:27.953 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-26T07:10:27.953 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-26T07:10:27.954 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-26T07:10:27.954 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-26T07:10:27.954 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-26T07:10:27.954 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-26T07:10:27.954 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-26T07:10:27.954 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-26T07:10:27.954 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-26T07:10:27.954 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-26T07:10:27.954 INFO:tasks.ceph:Setting up mon.a... 2023-11-26T07:10:27.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-26T07:10:28.007 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.keyring 2023-11-26T07:10:28.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-26T07:10:28.056 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-26T07:10:28.082 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.83'), ('mon.b', '172.21.15.203'), ('mon.c', '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]')] 2023-11-26T07:10:28.083 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.83,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-26T07:10:28.084 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:28.084 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-26T07:10:28.140 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.83 --add b 172.21.15.203 --addv c '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:28.228 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:28.228 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: generated fsid 0aae1049-cc8c-4011-a01f-378673e57a96 2023-11-26T07:10:28.228 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = pacific 2023-11-26T07:10:28.228 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 0 2023-11-26T07:10:28.228 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 0aae1049-cc8c-4011-a01f-378673e57a96 2023-11-26T07:10:28.228 INFO:teuthology.orchestra.run.smithi083.stdout:last_changed 2023-11-26T07:10:28.226971+0000 2023-11-26T07:10:28.228 INFO:teuthology.orchestra.run.smithi083.stdout:created 2023-11-26T07:10:28.226971+0000 2023-11-26T07:10:28.229 INFO:teuthology.orchestra.run.smithi083.stdout:min_mon_release 16 (pacific) 2023-11-26T07:10:28.229 INFO:teuthology.orchestra.run.smithi083.stdout:election_strategy: 1 2023-11-26T07:10:28.229 INFO:teuthology.orchestra.run.smithi083.stdout:0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2023-11-26T07:10:28.229 INFO:teuthology.orchestra.run.smithi083.stdout:1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.b 2023-11-26T07:10:28.229 INFO:teuthology.orchestra.run.smithi083.stdout:2: [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] mon.c 2023-11-26T07:10:28.229 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-26T07:10:28.230 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-26T07:10:28.246 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0aae1049-cc8c-4011-a01f-378673e57a96... 2023-11-26T07:10:28.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-26T07:10:28.288 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-26T07:10:28.331 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2023-11-26T07:10:28.331 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2023-11-26T07:10:28.331 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-26T07:10:28.331 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2023-11-26T07:10:28.331 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.331 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2023-11-26T07:10:28.331 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.331 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.332 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.333 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.83,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on skipped message = False 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 0aae1049-cc8c-4011-a01f-378673e57a96 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.334 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2023-11-26T07:10:28.335 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2023-11-26T07:10:28.336 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: bdev async discard = True 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: bdev enable discard = True 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore allocator = bitmap 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 1/20 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 1/20 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 4/10 2023-11-26T07:10:28.337 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:10:28.338 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: debug finisher = 20 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:10:28.339 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2023-11-26T07:10:28.340 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2023-11-26T07:10:28.341 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2023-11-26T07:10:28.342 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2023-11-26T07:10:28.343 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2023-11-26T07:10:28.343 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2023-11-26T07:10:28.343 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = True 2023-11-26T07:10:28.343 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2023-11-26T07:10:28.343 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = True 2023-11-26T07:10:28.343 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2023-11-26T07:10:28.343 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2023-11-26T07:10:28.343 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2023-11-26T07:10:28.343 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2023-11-26T07:10:28.344 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2023-11-26T07:10:28.344 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-26T07:10:28.344 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2023-11-26T07:10:28.344 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.344 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2023-11-26T07:10:28.344 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.344 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2023-11-26T07:10:28.344 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.344 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2023-11-26T07:10:28.344 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2023-11-26T07:10:28.344 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2023-11-26T07:10:28.345 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2023-11-26T07:10:28.346 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.83,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on skipped message = False 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = 0aae1049-cc8c-4011-a01f-378673e57a96 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.347 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2023-11-26T07:10:28.348 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: bdev async discard = True 2023-11-26T07:10:28.349 INFO:teuthology.orchestra.run.smithi203.stdout: bdev enable discard = True 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore allocator = bitmap 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 1/20 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 1/20 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 4/10 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2023-11-26T07:10:28.350 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: debug finisher = 20 2023-11-26T07:10:28.351 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-26T07:10:28.352 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2023-11-26T07:10:28.353 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2023-11-26T07:10:28.354 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = True 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = True 2023-11-26T07:10:28.355 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2023-11-26T07:10:28.356 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2023-11-26T07:10:28.356 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2023-11-26T07:10:28.357 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-26T07:10:28.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-26T07:10:28.454 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-26T07:10:28.454 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:28.454 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-26T07:10:28.470 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:28.471 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-26T07:10:28.528 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:10:28.528 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:28.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-26T07:10:28.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-26T07:10:28.604 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:28.604 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:28.661 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:10:28.661 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:10:28.661 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-26T07:10:28.661 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-26T07:10:28.698 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:10:28.698 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:28.754 INFO:tasks.ceph:Setting up mon nodes... 2023-11-26T07:10:28.755 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-26T07:10:28.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-26T07:10:28.808 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-26T07:10:28.810 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-26T07:10:28.874 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-26T07:10:28.877 INFO:tasks.ceph:Setting up mds nodes... 2023-11-26T07:10:28.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-26T07:10:28.931 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-26T07:10:28.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-26T07:10:28.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-26T07:10:29.052 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-26T07:10:29.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-26T07:10:29.080 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-26T07:10:29.135 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-26T07:10:29.137 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-26T07:10:29.163 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-26T07:10:29.257 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-26T07:10:29.259 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-26T07:10:29.285 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-26T07:10:29.286 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-26T07:10:29.329 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-26T07:10:29.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-26T07:10:29.384 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-26T07:10:29.398 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-26T07:10:29.441 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-26T07:10:29.455 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-26T07:10:29.539 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-26T07:10:29.551 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-26T07:10:29.552 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-26T07:10:29.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-26T07:10:29.578 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:10:29.578 INFO:tasks.ceph:role: osd.0 2023-11-26T07:10:29.578 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:10:29.578 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-26T07:10:29.651 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:10:29.652 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:10:29.652 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:10:29.652 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:10:29.652 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:10:29.652 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:10:29.652 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:10:29.652 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:10:29.652 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:10:29.652 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:10:30.080 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-11-26T07:10:30.082 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-11-26T07:10:30.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-26T07:10:30.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-26T07:10:30.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-26T07:10:30.342 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:10:30.342 INFO:tasks.ceph:role: osd.1 2023-11-26T07:10:30.342 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:10:30.342 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-26T07:10:30.415 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:10:30.415 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:10:30.416 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:10:30.416 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:10:30.416 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:10:30.416 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:10:30.416 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:10:30.416 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:10:30.416 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:10:30.416 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:10:30.844 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-11-26T07:10:30.845 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-11-26T07:10:30.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-26T07:10:30.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-26T07:10:30.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-26T07:10:31.059 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:10:31.059 INFO:tasks.ceph:role: osd.2 2023-11-26T07:10:31.060 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:10:31.060 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-26T07:10:31.135 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:10:31.135 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:10:31.135 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:10:31.136 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:10:31.136 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:10:31.136 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:10:31.136 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:10:31.136 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:10:31.136 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:10:31.136 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:10:31.577 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-11-26T07:10:31.579 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-11-26T07:10:31.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-26T07:10:31.757 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-26T07:10:31.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-26T07:10:31.870 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:10:31.870 INFO:tasks.ceph:role: osd.3 2023-11-26T07:10:31.870 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:10:31.870 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-26T07:10:31.948 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:10:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:10:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:10:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:10:31.948 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:10:31.948 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:10:31.949 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:10:31.949 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:10:31.949 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:10:31.949 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:10:32.386 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2023-11-26T07:10:32.388 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2023-11-26T07:10:32.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-26T07:10:32.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-26T07:10:32.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:32.696+0000 7f979ad92640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-26T07:10:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:32.696+0000 7f979ad92640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-26T07:10:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:32.696+0000 7f979ad92640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-26T07:10:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:32.696+0000 7f979ad92640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-26T07:10:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:32.696+0000 7f979ad92640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-26T07:10:32.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:32.697+0000 7f979ad92640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-26T07:10:35.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-26T07:10:35.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:35.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:35.690+0000 7fdfd596b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-26T07:10:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:35.690+0000 7fdfd596b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-26T07:10:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:35.690+0000 7fdfd596b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-26T07:10:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:35.690+0000 7fdfd596b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-26T07:10:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:35.690+0000 7fdfd596b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-26T07:10:35.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:35.691+0000 7fdfd596b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-26T07:10:38.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-26T07:10:38.590 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:38.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:38.680+0000 7f22a5913640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-26T07:10:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:38.680+0000 7f22a5913640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-26T07:10:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:38.680+0000 7f22a5913640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-26T07:10:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:38.680+0000 7f22a5913640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-26T07:10:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:38.680+0000 7f22a5913640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-26T07:10:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:38.680+0000 7f22a5913640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-26T07:10:41.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-26T07:10:41.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:41.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:41.677+0000 7f378d408640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-26T07:10:41.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:41.677+0000 7f378d408640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-26T07:10:41.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:41.677+0000 7f378d408640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-26T07:10:41.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:41.677+0000 7f378d408640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-26T07:10:41.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:41.677+0000 7f378d408640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-26T07:10:41.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:10:41.678+0000 7f378d408640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-26T07:10:44.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-26T07:10:44.588 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-26T07:10:44.621 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:10:44.621 INFO:tasks.ceph:role: osd.4 2023-11-26T07:10:44.621 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:10:44.621 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-26T07:10:44.698 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:10:44.698 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:10:44.698 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:10:44.698 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:10:44.698 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:10:44.698 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:10:44.698 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:10:44.698 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:10:44.698 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:10:44.698 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:10:45.126 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2023-11-26T07:10:45.128 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2023-11-26T07:10:45.128 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-26T07:10:45.324 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-26T07:10:45.362 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-26T07:10:45.432 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:10:45.432 INFO:tasks.ceph:role: osd.5 2023-11-26T07:10:45.432 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:10:45.432 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-26T07:10:45.506 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:10:45.506 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:10:45.506 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:10:45.507 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:10:45.507 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:10:45.507 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:10:45.507 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:10:45.507 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:10:45.507 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:10:45.507 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:10:45.916 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2023-11-26T07:10:45.918 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2023-11-26T07:10:45.918 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-26T07:10:46.024 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-26T07:10:46.057 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-26T07:10:46.126 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:10:46.126 INFO:tasks.ceph:role: osd.6 2023-11-26T07:10:46.126 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:10:46.126 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-26T07:10:46.201 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:10:46.202 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:10:46.202 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:10:46.202 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:10:46.202 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:10:46.202 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:10:46.202 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:10:46.202 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:10:46.202 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:10:46.202 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:10:46.658 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2023-11-26T07:10:46.659 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2023-11-26T07:10:46.660 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-26T07:10:46.849 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-26T07:10:46.883 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-26T07:10:46.951 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:10:46.952 INFO:tasks.ceph:role: osd.7 2023-11-26T07:10:46.952 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:10:46.952 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-26T07:10:47.027 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:10:47.028 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:10:47.028 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:10:47.028 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:10:47.028 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:10:47.028 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:10:47.029 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:10:47.029 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:10:47.029 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:10:47.029 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:10:47.449 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2023-11-26T07:10:47.451 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2023-11-26T07:10:47.451 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-26T07:10:47.644 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-26T07:10:47.678 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:47.784 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:47.782+0000 7fce712d0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-26T07:10:47.784 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:47.782+0000 7fce712d0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-26T07:10:47.784 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:47.782+0000 7fce712d0640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-26T07:10:47.784 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:47.782+0000 7fce712d0640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-26T07:10:47.784 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:47.782+0000 7fce712d0640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-26T07:10:47.785 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:47.783+0000 7fce712d0640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-26T07:10:50.648 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-26T07:10:50.680 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:50.772 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:50.769+0000 7f331fa2e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-26T07:10:50.772 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:50.769+0000 7f331fa2e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-26T07:10:50.772 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:50.769+0000 7f331fa2e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-26T07:10:50.772 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:50.769+0000 7f331fa2e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-26T07:10:50.772 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:50.769+0000 7f331fa2e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-26T07:10:50.772 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:50.770+0000 7f331fa2e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-26T07:10:53.638 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-26T07:10:53.669 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:53.761 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:53.758+0000 7fc00955d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-26T07:10:53.761 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:53.758+0000 7fc00955d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-26T07:10:53.761 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:53.758+0000 7fc00955d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-26T07:10:53.761 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:53.758+0000 7fc00955d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-26T07:10:53.761 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:53.758+0000 7fc00955d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-26T07:10:53.761 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:53.759+0000 7fc00955d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-26T07:10:56.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:10:56.318 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:10:56.631 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-26T07:10:56.665 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:10:56.754 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:56.752+0000 7fd53e3bc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-26T07:10:56.755 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:56.752+0000 7fd53e3bc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-26T07:10:56.755 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:56.752+0000 7fd53e3bc640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-26T07:10:56.755 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:56.752+0000 7fd53e3bc640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-26T07:10:56.755 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:56.752+0000 7fd53e3bc640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-26T07:10:56.755 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:10:56.753+0000 7fd53e3bc640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-26T07:10:59.629 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-26T07:10:59.662 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-26T07:10:59.662 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:59.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-26T07:10:59.694 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:59.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-26T07:10:59.765 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:59.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-26T07:10:59.835 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:59.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-26T07:10:59.903 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:59.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-26T07:10:59.972 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:10:59.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-26T07:11:00.039 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:11:00.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-26T07:11:00.107 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:00.107 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-26T07:11:00.139 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:00.139 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-26T07:11:00.210 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:00.210 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-26T07:11:00.280 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:00.280 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-26T07:11:00.350 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:00.350 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-26T07:11:00.419 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:00.419 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-26T07:11:00.488 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:00.488 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-26T07:11:00.556 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:11:00.556 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-26T07:11:00.576 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:11:00.576 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-26T07:11:00.635 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:00.635 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-26T07:11:00.653 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:00.653 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-26T07:11:00.711 INFO:tasks.ceph:Adding keys to all mons... 2023-11-26T07:11:00.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-26T07:11:00.713 DEBUG:teuthology.orchestra.run.smithi203:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-26T07:11:00.754 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.y] 2023-11-26T07:11:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBk72Jl3qkjMBAA+jcxYuDG/UHhB7W7HDwmaw== 2023-11-26T07:11:00.754 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2023-11-26T07:11:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBk72JlA8pxNxAAt5NuewAjm7yPThCguJ6sEA== 2023-11-26T07:11:00.754 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2023-11-26T07:11:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JlbKEQAxAAM2tBBQwM+tt2d8JU37YURg== 2023-11-26T07:11:00.755 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.0] 2023-11-26T07:11:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBo72JlhuSTKRAA6LRDOHBQVl5TL5tIHzyfzA== 2023-11-26T07:11:00.755 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.1] 2023-11-26T07:11:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBr72JlteU6KRAAuuHTYosYpl+KptywVmaeHA== 2023-11-26T07:11:00.755 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.2] 2023-11-26T07:11:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBu72JlEb6dKBAAebp4pnu9XZsSRJpojTZcvQ== 2023-11-26T07:11:00.755 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.3] 2023-11-26T07:11:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBx72JlbTJ3KBAA1JUxxAoyCzTtfgubUxSJEA== 2023-11-26T07:11:00.755 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.x] 2023-11-26T07:11:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBk72JlsA0VNBAAfSlnVquhI+vpLFwmgLGGVA== 2023-11-26T07:11:00.756 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2023-11-26T07:11:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JlHBgDCBAAk6egig5krY7kiMiHkMKrWg== 2023-11-26T07:11:00.756 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2023-11-26T07:11:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JlI3ZHDxAArtLmBVGr/vAB/YGwEFfhGw== 2023-11-26T07:11:00.757 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.4] 2023-11-26T07:11:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQB372JlufqqLhAAm6ZUt9juWj5yAIDlnCXqWg== 2023-11-26T07:11:00.757 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.5] 2023-11-26T07:11:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQB672Jl1L3vLRAAX06GcsntcmB+7L1CsfnJzw== 2023-11-26T07:11:00.757 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.6] 2023-11-26T07:11:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQB972Jlv41HLRAALtuyuREvGmTuDZt/RT0UcA== 2023-11-26T07:11:00.757 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.7] 2023-11-26T07:11:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCA72JlKCHsLBAAAXDzA7AN36wcLjhK97nlkg== 2023-11-26T07:11:00.757 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2023-11-26T07:11:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ== 2023-11-26T07:11:00.758 INFO:teuthology.orchestra.run.smithi083.stdout:[client.2] 2023-11-26T07:11:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JldAHlFhAAb3u9KljU9vssDl54zgno2Q== 2023-11-26T07:11:00.758 INFO:teuthology.orchestra.run.smithi083.stdout:[client.1] 2023-11-26T07:11:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JleNpIGhAAKwoQj4zB9YOxpFMD28juCA== 2023-11-26T07:11:00.758 INFO:teuthology.orchestra.run.smithi083.stdout:[client.3] 2023-11-26T07:11:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JluiQUIBAA06mqpeCzgpbSLo2/fulfmw== 2023-11-26T07:11:00.777 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.y] 2023-11-26T07:11:00.777 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBk72Jl3qkjMBAA+jcxYuDG/UHhB7W7HDwmaw== 2023-11-26T07:11:00.777 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2023-11-26T07:11:00.777 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBk72JlA8pxNxAAt5NuewAjm7yPThCguJ6sEA== 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBl72JlbKEQAxAAM2tBBQwM+tt2d8JU37YURg== 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.0] 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBo72JlhuSTKRAA6LRDOHBQVl5TL5tIHzyfzA== 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.1] 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBr72JlteU6KRAAuuHTYosYpl+KptywVmaeHA== 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.2] 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBu72JlEb6dKBAAebp4pnu9XZsSRJpojTZcvQ== 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.3] 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBx72JlbTJ3KBAA1JUxxAoyCzTtfgubUxSJEA== 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.x] 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBk72JlsA0VNBAAfSlnVquhI+vpLFwmgLGGVA== 2023-11-26T07:11:00.778 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBl72JlHBgDCBAAk6egig5krY7kiMiHkMKrWg== 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBl72JlI3ZHDxAArtLmBVGr/vAB/YGwEFfhGw== 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.4] 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQB372JlufqqLhAAm6ZUt9juWj5yAIDlnCXqWg== 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.5] 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQB672Jl1L3vLRAAX06GcsntcmB+7L1CsfnJzw== 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.6] 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQB972Jlv41HLRAALtuyuREvGmTuDZt/RT0UcA== 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.7] 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCA72JlKCHsLBAAAXDzA7AN36wcLjhK97nlkg== 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout:[client.0] 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ== 2023-11-26T07:11:00.779 INFO:teuthology.orchestra.run.smithi203.stdout:[client.2] 2023-11-26T07:11:00.780 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBl72JldAHlFhAAb3u9KljU9vssDl54zgno2Q== 2023-11-26T07:11:00.780 INFO:teuthology.orchestra.run.smithi203.stdout:[client.1] 2023-11-26T07:11:00.780 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBl72JleNpIGhAAKwoQj4zB9YOxpFMD28juCA== 2023-11-26T07:11:00.780 INFO:teuthology.orchestra.run.smithi203.stdout:[client.3] 2023-11-26T07:11:00.780 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQBl72JluiQUIBAA06mqpeCzgpbSLo2/fulfmw== 2023-11-26T07:11:00.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:11:00.801 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:11:00.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:11:00.879 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:11:00.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:11:00.940 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:11:00.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:00.999 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.055 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.111 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.166 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:11:01.221 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:11:01.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:11:01.277 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:11:01.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:11:01.335 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:11:01.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.394 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.449 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.505 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.561 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:11:01.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:11:01.617 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:11:01.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:11:01.676 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:11:01.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:11:01.734 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:11:01.791 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:11:01.793 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:11:01.849 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-26T07:11:01.849 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-26T07:11:01.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-26T07:11:01.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-26T07:11:01.998 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-26T07:11:02.025 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-26T07:11:02.123 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-26T07:11:02.148 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-26T07:11:02.214 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-26T07:11:02.313 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-26T07:11:02.339 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:11:02.341 DEBUG:teuthology.orchestra.run.smithi203:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:11:02.396 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-26T07:11:02.396 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-26T07:11:02.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-26T07:11:02.398 INFO:tasks.ceph.mon.a:Started 2023-11-26T07:11:02.399 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-26T07:11:02.399 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-26T07:11:02.438 INFO:tasks.ceph.mon.b:Started 2023-11-26T07:11:02.438 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-26T07:11:02.439 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-26T07:11:02.440 INFO:tasks.ceph.mon.c:Started 2023-11-26T07:11:02.440 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-26T07:11:02.441 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-26T07:11:02.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-26T07:11:02.443 INFO:tasks.ceph.mgr.y:Started 2023-11-26T07:11:02.443 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-26T07:11:02.443 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-26T07:11:02.445 INFO:tasks.ceph.mgr.x:Started 2023-11-26T07:11:02.445 INFO:tasks.ceph:Setting crush tunables to default 2023-11-26T07:11:02.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd crush tunables default 2023-11-26T07:11:02.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.565+0000 7f6b63b89700 1 Processor -- start 2023-11-26T07:11:02.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.566+0000 7f6b63b89700 1 -- start start 2023-11-26T07:11:02.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.566+0000 7f6b63b89700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6b5c0c6c30 0x7f6b5c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:02.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.566+0000 7f6b63b89700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5c0d0140 con 0x7f6b5c0cef40 2023-11-26T07:11:02.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.566+0000 7f6b63b89700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5c0cf3d0 con 0x7f6b5c15bf50 2023-11-26T07:11:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.566+0000 7f6b63b89700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5c15c330 con 0x7f6b5c0c6c30 2023-11-26T07:11:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.567+0000 7f6b61124700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6b5c0c6c30 0x7f6b5c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.567+0000 7f6b62126700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6b5c0cef40 0x7f6b5c0cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53460/0 (socket says 172.21.15.83:53460) 2023-11-26T07:11:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.567+0000 7f6b61124700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6b5c0c6c30 0x7f6b5c0cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33418/0 (socket says 172.21.15.83:33418) 2023-11-26T07:11:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.567+0000 7f6b62126700 1 -- 172.21.15.83:0/479409953 learned_addr learned my addr 172.21.15.83:0/479409953 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.567+0000 7f6b61124700 1 -- 172.21.15.83:0/479409953 >> v2:172.21.15.83:3300/0 conn(0x7f6b5c0c6c30 msgr2=0x7f6b5c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-26T07:11:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.567+0000 7f6b61124700 1 -- 172.21.15.83:0/479409953 >> v2:172.21.15.83:3300/0 conn(0x7f6b5c0c6c30 msgr2=0x7f6b5c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-26T07:11:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.567+0000 7f6b61124700 1 --2- 172.21.15.83:0/479409953 >> v2:172.21.15.83:3300/0 conn(0x7f6b5c0c6c30 0x7f6b5c0cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-26T07:11:02.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.567+0000 7f6b61124700 1 --2- 172.21.15.83:0/479409953 >> v2:172.21.15.83:3300/0 conn(0x7f6b5c0c6c30 0x7f6b5c0cea00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-26T07:11:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426208513 0 0) 0x7f6b5c0d0140 con 0x7f6b5c0cef40 2023-11-26T07:11:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b480035b0 con 0x7f6b5c0cef40 2023-11-26T07:11:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2165435756 0 0) 0x7f6b480035b0 con 0x7f6b5c0cef40 2023-11-26T07:11:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 >> v2:172.21.15.83:3300/0 conn(0x7f6b5c0c6c30 msgr2=0x7f6b5c0cea00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-26T07:11:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 --2- 172.21.15.83:0/479409953 >> v2:172.21.15.83:3300/0 conn(0x7f6b5c0c6c30 0x7f6b5c0cea00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 >> v1:172.21.15.203:6789/0 conn(0x7f6b5c15bf50 legacy=0x7f6b5c0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0d03e0 con 0x7f6b5c0cef40 2023-11-26T07:11:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f6b540019f0 con 0x7f6b5c0cef40 2023-11-26T07:11:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 >> v1:172.21.15.83:6789/0 conn(0x7f6b5c0cef40 legacy=0x7f6b5c0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b48003a00 0x7f6b48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 0x7f6b48004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.757+0000 7f6b60923700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b48005210 0x7f6b4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c15c330 con 0x7f6b48004850 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4800c160 con 0x7f6b48005210 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4800c320 con 0x7f6b48003a00 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b61124700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 0x7f6b48004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b61925700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b48003a00 0x7f6b48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b62126700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b48005210 0x7f6b4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b61124700 1 -- 172.21.15.83:0/479409953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b48003a00 msgr2=0x7f6b48003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b61124700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b48003a00 0x7f6b48003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:02.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b61124700 1 -- 172.21.15.83:0/479409953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b48005210 msgr2=0x7f6b4800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:02.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b61124700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b48005210 0x7f6b4800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:02.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b62126700 1 -- 172.21.15.83:0/479409953 reap_dead start 2023-11-26T07:11:02.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b61124700 1 -- 172.21.15.83:0/479409953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b480035b0 con 0x7f6b48004850 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b61925700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b48003a00 0x7f6b48003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b61124700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 0x7f6b48004cd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009890 tx=0x7f6b48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb05bef000afcb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800c070 con 0x7f6b48004850 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b58004020 con 0x7f6b48004850 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.758+0000 7f6b60923700 1 -- 172.21.15.83:0/479409953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b580043d0 con 0x7f6b48004850 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.759+0000 7f6b63b89700 1 -- 172.21.15.83:0/479409953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 msgr2=0x7f6b48004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.759+0000 7f6b63b89700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 0x7f6b48004cd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009890 tx=0x7f6b48005880 comp rx=0 tx=0).stop 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.759+0000 7f6b63b89700 1 -- 172.21.15.83:0/479409953 shutdown_connections 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.759+0000 7f6b63b89700 1 --2- 172.21.15.83:0/479409953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 0x7f6b48004cd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.759+0000 7f6b63b89700 1 -- 172.21.15.83:0/479409953 >> 172.21.15.83:0/479409953 conn(0x7f6b5c0b3780 msgr2=0x7f6b5c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.759+0000 7f6b63b89700 1 -- 172.21.15.83:0/479409953 shutdown_connections 2023-11-26T07:11:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.759+0000 7f6b63b89700 1 -- 172.21.15.83:0/479409953 wait complete. 2023-11-26T07:11:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.760+0000 7f6b63b89700 1 Processor -- start 2023-11-26T07:11:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.760+0000 7f6b63b89700 1 -- start start 2023-11-26T07:11:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b63b89700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 0x7f6b5c0b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b63b89700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b5c15d0f0 0x7f6b5c0af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b63b89700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c15d570 0x7f6b5c0afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b63b89700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0b6210 con 0x7f6b48004850 2023-11-26T07:11:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b63b89700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0b5840 con 0x7f6b5c15d570 2023-11-26T07:11:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b63b89700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c15a240 con 0x7f6b5c15d0f0 2023-11-26T07:11:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b61925700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 0x7f6b5c0b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b61925700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 0x7f6b5c0b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33438/0 (socket says 172.21.15.83:33438) 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b61925700 1 -- 172.21.15.83:0/3577688225 learned_addr learned my addr 172.21.15.83:0/3577688225 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b61124700 1 --2- 172.21.15.83:0/3577688225 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b5c15d0f0 0x7f6b5c0af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b62126700 1 --2- 172.21.15.83:0/3577688225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c15d570 0x7f6b5c0afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b61925700 1 -- 172.21.15.83:0/3577688225 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b5c15d0f0 msgr2=0x7f6b5c0af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b61925700 1 --2- 172.21.15.83:0/3577688225 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b5c15d0f0 0x7f6b5c0af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b61925700 1 -- 172.21.15.83:0/3577688225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c15d570 msgr2=0x7f6b5c0afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b61925700 1 --2- 172.21.15.83:0/3577688225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c15d570 0x7f6b5c0afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b61925700 1 -- 172.21.15.83:0/3577688225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0b4c30 con 0x7f6b48004850 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.761+0000 7f6b61925700 1 --2- 172.21.15.83:0/3577688225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 0x7f6b5c0b5cd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0078a0 tx=0x7f6b4c00e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.762+0000 7f6b527fc700 1 -- 172.21.15.83:0/3577688225 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c0050d0 con 0x7f6b48004850 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.762+0000 7f6b527fc700 1 -- 172.21.15.83:0/3577688225 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c005230 con 0x7f6b48004850 2023-11-26T07:11:02.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.762+0000 7f6b527fc700 1 -- 172.21.15.83:0/3577688225 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c003680 con 0x7f6b48004850 2023-11-26T07:11:02.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.762+0000 7f6b63b89700 1 -- 172.21.15.83:0/3577688225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5c0b4f10 con 0x7f6b48004850 2023-11-26T07:11:02.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.762+0000 7f6b63b89700 1 -- 172.21.15.83:0/3577688225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5c0b54d0 con 0x7f6b48004850 2023-11-26T07:11:02.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.762+0000 7f6b527fc700 1 -- 172.21.15.83:0/3577688225 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6b4c003ae0 con 0x7f6b48004850 2023-11-26T07:11:02.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.762+0000 7f6b527fc700 1 -- 172.21.15.83:0/3577688225 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6b4c01e440 con 0x7f6b48004850 2023-11-26T07:11:02.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.763+0000 7f6b63b89700 1 -- 172.21.15.83:0/3577688225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5c0b5690 con 0x7f6b48004850 2023-11-26T07:11:02.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.766+0000 7f6b527fc700 1 -- 172.21.15.83:0/3577688225 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f6b4c023060 con 0x7f6b48004850 2023-11-26T07:11:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.843+0000 7f6b63b89700 1 -- 172.21.15.83:0/3577688225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6b5c068fb0 con 0x7f6b48004850 2023-11-26T07:11:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.845+0000 7f6b527fc700 1 -- 172.21.15.83:0/3577688225 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6b4c019070 con 0x7f6b48004850 2023-11-26T07:11:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.846+0000 7f6b63b89700 1 -- 172.21.15.83:0/3577688225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 msgr2=0x7f6b5c0b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.846+0000 7f6b63b89700 1 --2- 172.21.15.83:0/3577688225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 0x7f6b5c0b5cd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0078a0 tx=0x7f6b4c00e740 comp rx=0 tx=0).stop 2023-11-26T07:11:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.846+0000 7f6b63b89700 1 -- 172.21.15.83:0/3577688225 shutdown_connections 2023-11-26T07:11:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.846+0000 7f6b63b89700 1 --2- 172.21.15.83:0/3577688225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b5c15d570 0x7f6b5c0afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.846+0000 7f6b63b89700 1 --2- 172.21.15.83:0/3577688225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b48004850 0x7f6b5c0b5cd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.846+0000 7f6b63b89700 1 --2- 172.21.15.83:0/3577688225 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b5c15d0f0 0x7f6b5c0af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.846+0000 7f6b63b89700 1 -- 172.21.15.83:0/3577688225 >> 172.21.15.83:0/3577688225 conn(0x7f6b5c0b3780 msgr2=0x7f6b5c0c7340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.846+0000 7f6b63b89700 1 -- 172.21.15.83:0/3577688225 shutdown_connections 2023-11-26T07:11:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:02.846+0000 7f6b63b89700 1 -- 172.21.15.83:0/3577688225 wait complete. 2023-11-26T07:11:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2023-11-26T07:11:02.860 INFO:tasks.ceph:check_enable_crimson: False 2023-11-26T07:11:02.860 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-26T07:11:02.860 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:11:02.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-26T07:11:02.879 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:02.877+0000 7f317dfe1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-26T07:11:02.884 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:02.882+0000 7f3809420200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-26T07:11:02.886 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:11:02.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-26T07:11:02.945 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:02.943+0000 7f317dfe1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-26T07:11:02.952 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:11:02.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-26T07:11:02.952 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:02.950+0000 7f3809420200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-26T07:11:03.018 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:11:03.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-26T07:11:03.021 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:03.018+0000 7f317dfe1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-26T07:11:03.031 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:03.029+0000 7f3809420200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-26T07:11:03.057 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:03.057 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-26T07:11:03.083 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:03.084 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-26T07:11:03.101 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:03.099+0000 7f317dfe1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-26T07:11:03.114 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:03.112+0000 7f3809420200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-26T07:11:03.127 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:03.127 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-26T07:11:03.193 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:03.193 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-26T07:11:03.254 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:03.252+0000 7f317dfe1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-26T07:11:03.256 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:03.253+0000 7f3809420200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-26T07:11:03.259 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 2941c9d0-0912-44e6-bce1-c2c8749bd713 0 2023-11-26T07:11:03.421 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.419+0000 7f553c590700 1 Processor -- start 2023-11-26T07:11:03.421 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.419+0000 7f553c590700 1 -- start start 2023-11-26T07:11:03.423 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553c590700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f553415a620 0x7f55340ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.423 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553c590700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55340b2f20 con 0x7f55340cc540 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553c590700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55340ce690 con 0x7f553406ecf0 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553c590700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f553415aa00 con 0x7f553415a620 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553a32c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f553415a620 0x7f55340ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553a32c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f553415a620 0x7f55340ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57646/0 (socket says 172.21.15.203:57646) 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553ab2d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f553406ecf0 0x7f55340b3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56852/0 (socket says 172.21.15.203:56852) 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553a32c700 1 -- 172.21.15.203:0/4071420213 learned_addr learned my addr 172.21.15.203:0/4071420213 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553a32c700 1 -- 172.21.15.203:0/4071420213 >> v1:172.21.15.203:6789/0 conn(0x7f553406ecf0 legacy=0x7f55340b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553a32c700 1 -- 172.21.15.203:0/4071420213 >> v1:172.21.15.83:6789/0 conn(0x7f55340cc540 legacy=0x7f553406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553a32c700 1 -- 172.21.15.203:0/4071420213 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55340ce960 con 0x7f553415a620 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553a32c700 1 --2- 172.21.15.203:0/4071420213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f553415a620 0x7f55340ce150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f55300098f0 tx=0x7f5530009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e858b7f20fc9c26b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553932a700 1 -- 172.21.15.203:0/4071420213 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553000b8a0 con 0x7f553415a620 2023-11-26T07:11:03.424 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553932a700 1 -- 172.21.15.203:0/4071420213 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553000ba00 con 0x7f553415a620 2023-11-26T07:11:03.425 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.420+0000 7f553932a700 1 -- 172.21.15.203:0/4071420213 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553000bd90 con 0x7f553415a620 2023-11-26T07:11:03.425 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.421+0000 7f553c590700 1 -- 172.21.15.203:0/4071420213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f553415a620 msgr2=0x7f55340ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.425 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.421+0000 7f553c590700 1 --2- 172.21.15.203:0/4071420213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f553415a620 0x7f55340ce150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f55300098f0 tx=0x7f5530009c00 comp rx=0 tx=0).stop 2023-11-26T07:11:03.425 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.421+0000 7f553c590700 1 -- 172.21.15.203:0/4071420213 shutdown_connections 2023-11-26T07:11:03.425 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.421+0000 7f553c590700 1 --2- 172.21.15.203:0/4071420213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f553415a620 0x7f55340ce150 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.425 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.421+0000 7f553c590700 1 -- 172.21.15.203:0/4071420213 >> 172.21.15.203:0/4071420213 conn(0x7f55340b2050 msgr2=0x7f55340b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:03.425 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.421+0000 7f553c590700 1 -- 172.21.15.203:0/4071420213 shutdown_connections 2023-11-26T07:11:03.425 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.421+0000 7f553c590700 1 -- 172.21.15.203:0/4071420213 wait complete. 2023-11-26T07:11:03.427 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.422+0000 7f553c590700 1 Processor -- start 2023-11-26T07:11:03.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.422+0000 7f553c590700 1 -- start start 2023-11-26T07:11:03.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f553c590700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f553406ecf0 0x7f553414ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f553c590700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55340cc540 0x7f553415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f553c590700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f553415a620 0x7f553415d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f553c590700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553415c150 con 0x7f553406ecf0 2023-11-26T07:11:03.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f553c590700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553415b540 con 0x7f553415a620 2023-11-26T07:11:03.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f553c590700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553415b820 con 0x7f55340cc540 2023-11-26T07:11:03.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f553ab2d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f553415a620 0x7f553415d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f5539b2b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55340cc540 0x7f553415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f553ab2d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f553415a620 0x7f553415d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57656/0 (socket says 172.21.15.203:57656) 2023-11-26T07:11:03.428 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f553ab2d700 1 -- 172.21.15.203:0/3223707927 learned_addr learned my addr 172.21.15.203:0/3223707927 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f553a32c700 1 --2- 172.21.15.203:0/3223707927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f553406ecf0 0x7f553414ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f5539b2b700 1 -- 172.21.15.203:0/3223707927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f553415a620 msgr2=0x7f553415d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f5539b2b700 1 --2- 172.21.15.203:0/3223707927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f553415a620 0x7f553415d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f5539b2b700 1 -- 172.21.15.203:0/3223707927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f553406ecf0 msgr2=0x7f553414ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f5539b2b700 1 --2- 172.21.15.203:0/3223707927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f553406ecf0 0x7f553414ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f5539b2b700 1 -- 172.21.15.203:0/3223707927 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5530009550 con 0x7f55340cc540 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.423+0000 7f553ab2d700 1 --2- 172.21.15.203:0/3223707927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f553415a620 0x7f553415d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.424+0000 7f5539b2b700 1 --2- 172.21.15.203:0/3223707927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55340cc540 0x7f553415ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5524000f60 tx=0x7f552400c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.424+0000 7f552affd700 1 -- 172.21.15.203:0/3223707927 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55240052d0 con 0x7f55340cc540 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.424+0000 7f552affd700 1 -- 172.21.15.203:0/3223707927 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5524005430 con 0x7f55340cc540 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.424+0000 7f553c590700 1 -- 172.21.15.203:0/3223707927 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553415bb00 con 0x7f55340cc540 2023-11-26T07:11:03.429 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.424+0000 7f552affd700 1 -- 172.21.15.203:0/3223707927 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55240038c0 con 0x7f55340cc540 2023-11-26T07:11:03.430 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.424+0000 7f553c590700 1 -- 172.21.15.203:0/3223707927 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553414b250 con 0x7f55340cc540 2023-11-26T07:11:03.430 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.424+0000 7f552affd700 1 -- 172.21.15.203:0/3223707927 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5524003d20 con 0x7f55340cc540 2023-11-26T07:11:03.430 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.424+0000 7f552affd700 1 -- 172.21.15.203:0/3223707927 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f552401c780 con 0x7f55340cc540 2023-11-26T07:11:03.430 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.425+0000 7f553c590700 1 -- 172.21.15.203:0/3223707927 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5534068d90 con 0x7f55340cc540 2023-11-26T07:11:03.430 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.427+0000 7f552affd700 1 -- 172.21.15.203:0/3223707927 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5524017070 con 0x7f55340cc540 2023-11-26T07:11:03.512 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.510+0000 7f553c590700 1 -- 172.21.15.203:0/3223707927 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2941c9d0-0912-44e6-bce1-c2c8749bd713", "id": 0} v 0) v1 -- 0x7f553415c2c0 con 0x7f55340cc540 2023-11-26T07:11:03.515 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.513+0000 7f552affd700 1 -- 172.21.15.203:0/3223707927 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2941c9d0-0912-44e6-bce1-c2c8749bd713", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f552400ce30 con 0x7f55340cc540 2023-11-26T07:11:03.515 INFO:teuthology.orchestra.run.smithi203.stdout:0 2023-11-26T07:11:03.516 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.513+0000 7f553c590700 1 -- 172.21.15.203:0/3223707927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55340cc540 msgr2=0x7f553415ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.516 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.513+0000 7f553c590700 1 --2- 172.21.15.203:0/3223707927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55340cc540 0x7f553415ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5524000f60 tx=0x7f552400c730 comp rx=0 tx=0).stop 2023-11-26T07:11:03.516 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.514+0000 7f553c590700 1 -- 172.21.15.203:0/3223707927 shutdown_connections 2023-11-26T07:11:03.516 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.514+0000 7f553c590700 1 --2- 172.21.15.203:0/3223707927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f553415a620 0x7f553415d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.516 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.514+0000 7f553c590700 1 --2- 172.21.15.203:0/3223707927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f553406ecf0 0x7f553414ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.516 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.514+0000 7f553c590700 1 --2- 172.21.15.203:0/3223707927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55340cc540 0x7f553415ac60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.516 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.514+0000 7f553c590700 1 -- 172.21.15.203:0/3223707927 >> 172.21.15.203:0/3223707927 conn(0x7f55340b2050 msgr2=0x7f55340b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:03.517 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.514+0000 7f553c590700 1 -- 172.21.15.203:0/3223707927 shutdown_connections 2023-11-26T07:11:03.517 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.514+0000 7f553c590700 1 -- 172.21.15.203:0/3223707927 wait complete. 2023-11-26T07:11:03.528 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 7c65463e-79cb-4487-aec7-5ef4fd9b7520 1 2023-11-26T07:11:03.641 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:03.639+0000 7f3809420200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-26T07:11:03.648 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.646+0000 7fa094fe6700 1 Processor -- start 2023-11-26T07:11:03.648 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.646+0000 7fa094fe6700 1 -- start start 2023-11-26T07:11:03.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa094fe6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa09015e160 0x7fa0900b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa094fe6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa0900746a0 0x7fa09006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa094fe6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0900b3c60 con 0x7fa0900afc90 2023-11-26T07:11:03.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa094fe6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0900b4500 con 0x7fa0900746a0 2023-11-26T07:11:03.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa094fe6700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0900b58c0 con 0x7fa09015e160 2023-11-26T07:11:03.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa08e59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa09015e160 0x7fa0900b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa08e59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa09015e160 0x7fa0900b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57658/0 (socket says 172.21.15.203:57658) 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa08e59c700 1 -- 172.21.15.203:0/2408066529 learned_addr learned my addr 172.21.15.203:0/2408066529 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa08ed9d700 1 --2- 172.21.15.203:0/2408066529 >> v2:172.21.15.83:3300/0 conn(0x7fa0900746a0 0x7fa09006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa08e59c700 1 -- 172.21.15.203:0/2408066529 >> v2:172.21.15.83:3300/0 conn(0x7fa0900746a0 msgr2=0x7fa09006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa08e59c700 1 --2- 172.21.15.203:0/2408066529 >> v2:172.21.15.83:3300/0 conn(0x7fa0900746a0 0x7fa09006d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa08e59c700 1 -- 172.21.15.203:0/2408066529 >> v1:172.21.15.83:6789/0 conn(0x7fa0900afc90 legacy=0x7fa090073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa08e59c700 1 -- 172.21.15.203:0/2408066529 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0900b0140 con 0x7fa09015e160 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa08e59c700 1 --2- 172.21.15.203:0/2408066529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa09015e160 0x7fa0900b5da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa09009efa0 tx=0x7fa080009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b63b66e092d36682 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.647+0000 7fa08d59a700 1 -- 172.21.15.203:0/2408066529 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080011830 con 0x7fa09015e160 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.648+0000 7fa08d59a700 1 -- 172.21.15.203:0/2408066529 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa080011990 con 0x7fa09015e160 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.648+0000 7fa08d59a700 1 -- 172.21.15.203:0/2408066529 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080011d20 con 0x7fa09015e160 2023-11-26T07:11:03.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.648+0000 7fa094fe6700 1 -- 172.21.15.203:0/2408066529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa09015e160 msgr2=0x7fa0900b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.653 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.648+0000 7fa094fe6700 1 --2- 172.21.15.203:0/2408066529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa09015e160 0x7fa0900b5da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa09009efa0 tx=0x7fa080009bd0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.653 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.648+0000 7fa094fe6700 1 -- 172.21.15.203:0/2408066529 shutdown_connections 2023-11-26T07:11:03.653 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.648+0000 7fa094fe6700 1 --2- 172.21.15.203:0/2408066529 >> v2:172.21.15.83:3300/0 conn(0x7fa0900746a0 0x7fa09006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.653 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.648+0000 7fa094fe6700 1 --2- 172.21.15.203:0/2408066529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa09015e160 0x7fa0900b5da0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.653 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.648+0000 7fa094fe6700 1 -- 172.21.15.203:0/2408066529 >> 172.21.15.203:0/2408066529 conn(0x7fa0900d0bf0 msgr2=0x7fa0900d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:03.653 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.648+0000 7fa094fe6700 1 -- 172.21.15.203:0/2408066529 shutdown_connections 2023-11-26T07:11:03.653 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.648+0000 7fa094fe6700 1 -- 172.21.15.203:0/2408066529 wait complete. 2023-11-26T07:11:03.655 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.650+0000 7fa094fe6700 1 Processor -- start 2023-11-26T07:11:03.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.650+0000 7fa094fe6700 1 -- start start 2023-11-26T07:11:03.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.650+0000 7fa094fe6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0900746a0 0x7fa0901f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.650+0000 7fa094fe6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0900afc90 0x7fa0901480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.650+0000 7fa094fe6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09015e160 0x7fa090148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.650+0000 7fa094fe6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa090158b80 con 0x7fa09015e160 2023-11-26T07:11:03.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.650+0000 7fa094fe6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa090157f00 con 0x7fa0900746a0 2023-11-26T07:11:03.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.650+0000 7fa094fe6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0901581e0 con 0x7fa0900afc90 2023-11-26T07:11:03.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.650+0000 7fa08dd9b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0900afc90 0x7fa0901480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08dd9b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0900afc90 0x7fa0901480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:48806/0 (socket says 172.21.15.203:48806) 2023-11-26T07:11:03.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08dd9b700 1 -- 172.21.15.203:0/3353810179 learned_addr learned my addr 172.21.15.203:0/3353810179 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:03.656 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08e59c700 1 --2- 172.21.15.203:0/3353810179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0900746a0 0x7fa0901f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08ed9d700 1 --2- 172.21.15.203:0/3353810179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09015e160 0x7fa090148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08dd9b700 1 -- 172.21.15.203:0/3353810179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0900746a0 msgr2=0x7fa0901f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08dd9b700 1 --2- 172.21.15.203:0/3353810179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0900746a0 0x7fa0901f3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08dd9b700 1 -- 172.21.15.203:0/3353810179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09015e160 msgr2=0x7fa090148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08dd9b700 1 --2- 172.21.15.203:0/3353810179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09015e160 0x7fa090148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08dd9b700 1 -- 172.21.15.203:0/3353810179 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa080009550 con 0x7fa0900afc90 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08e59c700 1 --2- 172.21.15.203:0/3353810179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0900746a0 0x7fa0901f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08ed9d700 1 --2- 172.21.15.203:0/3353810179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09015e160 0x7fa090148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa08dd9b700 1 --2- 172.21.15.203:0/3353810179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0900afc90 0x7fa0901480f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa08400c4e0 tx=0x7fa08400c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa07effd700 1 -- 172.21.15.203:0/3353810179 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0840052d0 con 0x7fa0900afc90 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa07effd700 1 -- 172.21.15.203:0/3353810179 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa084005430 con 0x7fa0900afc90 2023-11-26T07:11:03.657 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa094fe6700 1 -- 172.21.15.203:0/3353810179 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0901584c0 con 0x7fa0900afc90 2023-11-26T07:11:03.658 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa07effd700 1 -- 172.21.15.203:0/3353810179 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0840038c0 con 0x7fa0900afc90 2023-11-26T07:11:03.658 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa094fe6700 1 -- 172.21.15.203:0/3353810179 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0901f3770 con 0x7fa0900afc90 2023-11-26T07:11:03.658 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa07effd700 1 -- 172.21.15.203:0/3353810179 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa084003d20 con 0x7fa0900afc90 2023-11-26T07:11:03.658 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.651+0000 7fa07effd700 1 -- 172.21.15.203:0/3353810179 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fa084011e30 con 0x7fa0900afc90 2023-11-26T07:11:03.658 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.652+0000 7fa094fe6700 1 -- 172.21.15.203:0/3353810179 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa090068fb0 con 0x7fa0900afc90 2023-11-26T07:11:03.658 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.655+0000 7fa07effd700 1 -- 172.21.15.203:0/3353810179 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa084017070 con 0x7fa0900afc90 2023-11-26T07:11:03.691 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:03.688+0000 7f317dfe1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-26T07:11:03.755 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.753+0000 7fa094fe6700 1 -- 172.21.15.203:0/3353810179 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7c65463e-79cb-4487-aec7-5ef4fd9b7520", "id": 1} v 0) v1 -- 0x7fa0900b20a0 con 0x7fa0900afc90 2023-11-26T07:11:03.756 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:03.754+0000 7f3809420200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-26T07:11:03.758 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.756+0000 7fa07effd700 1 -- 172.21.15.203:0/3353810179 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7c65463e-79cb-4487-aec7-5ef4fd9b7520", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa084011590 con 0x7fa0900afc90 2023-11-26T07:11:03.759 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-11-26T07:11:03.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.757+0000 7fa094fe6700 1 -- 172.21.15.203:0/3353810179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0900afc90 msgr2=0x7fa0901480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.757+0000 7fa094fe6700 1 --2- 172.21.15.203:0/3353810179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0900afc90 0x7fa0901480f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa08400c4e0 tx=0x7fa08400c7f0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.757+0000 7fa094fe6700 1 -- 172.21.15.203:0/3353810179 shutdown_connections 2023-11-26T07:11:03.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.757+0000 7fa094fe6700 1 --2- 172.21.15.203:0/3353810179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09015e160 0x7fa090148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.757+0000 7fa094fe6700 1 --2- 172.21.15.203:0/3353810179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0900746a0 0x7fa0901f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.757+0000 7fa094fe6700 1 --2- 172.21.15.203:0/3353810179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0900afc90 0x7fa0901480f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.757+0000 7fa094fe6700 1 -- 172.21.15.203:0/3353810179 >> 172.21.15.203:0/3353810179 conn(0x7fa0900d0bf0 msgr2=0x7fa0901619f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:03.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.757+0000 7fa094fe6700 1 -- 172.21.15.203:0/3353810179 shutdown_connections 2023-11-26T07:11:03.760 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.757+0000 7fa094fe6700 1 -- 172.21.15.203:0/3353810179 wait complete. 2023-11-26T07:11:03.774 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3 2 2023-11-26T07:11:03.813 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:03.811+0000 7f317dfe1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-26T07:11:03.834 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:03.831+0000 7f3809420200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-26T07:11:03.895 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.893+0000 7f4340499700 1 Processor -- start 2023-11-26T07:11:03.895 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.893+0000 7f4340499700 1 -- start start 2023-11-26T07:11:03.897 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f4340499700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f43380746a0 0x7f43380b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.897 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f4340499700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43380b3c60 con 0x7f43380afc90 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f4340499700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43380b58c0 con 0x7f433815e160 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f4340499700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43380b4500 con 0x7f43380746a0 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f433da34700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f43380746a0 0x7f43380b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f433e235700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f433815e160 0x7f4338073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56872/0 (socket says 172.21.15.203:56872) 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f433e235700 1 -- 172.21.15.203:0/2791196253 learned_addr learned my addr 172.21.15.203:0/2791196253 (peer_addr_for_me v1:172.21.15.203:0/0) 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f433da34700 1 -- 172.21.15.203:0/2791196253 >> v1:172.21.15.203:6789/0 conn(0x7f433815e160 legacy=0x7f4338073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f433da34700 1 -- 172.21.15.203:0/2791196253 >> v1:172.21.15.83:6789/0 conn(0x7f43380afc90 legacy=0x7f433806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f433da34700 1 -- 172.21.15.203:0/2791196253 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433806bdc0 con 0x7f43380746a0 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f433da34700 1 --2- 172.21.15.203:0/2791196253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43380746a0 0x7f43380b5da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f432c009840 tx=0x7f432c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f093080ec7a3ca3e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f433d233700 1 -- 172.21.15.203:0/2791196253 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c011830 con 0x7f43380746a0 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f433d233700 1 -- 172.21.15.203:0/2791196253 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c011990 con 0x7f43380746a0 2023-11-26T07:11:03.898 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.894+0000 7f433d233700 1 -- 172.21.15.203:0/2791196253 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c011d20 con 0x7f43380746a0 2023-11-26T07:11:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.895+0000 7f4340499700 1 -- 172.21.15.203:0/2791196253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43380746a0 msgr2=0x7f43380b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.895+0000 7f4340499700 1 --2- 172.21.15.203:0/2791196253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43380746a0 0x7f43380b5da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f432c009840 tx=0x7f432c009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.895+0000 7f4340499700 1 -- 172.21.15.203:0/2791196253 shutdown_connections 2023-11-26T07:11:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.895+0000 7f4340499700 1 --2- 172.21.15.203:0/2791196253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43380746a0 0x7f43380b5da0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.895+0000 7f4340499700 1 -- 172.21.15.203:0/2791196253 >> 172.21.15.203:0/2791196253 conn(0x7f43380d0bf0 msgr2=0x7f43380d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.895+0000 7f4340499700 1 -- 172.21.15.203:0/2791196253 shutdown_connections 2023-11-26T07:11:03.899 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.895+0000 7f4340499700 1 -- 172.21.15.203:0/2791196253 wait complete. 2023-11-26T07:11:03.901 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.896+0000 7f4340499700 1 Processor -- start 2023-11-26T07:11:03.901 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.896+0000 7f4340499700 1 -- start start 2023-11-26T07:11:03.901 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f4340499700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43380746a0 0x7f4338153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f4340499700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43380afc90 0x7f4338141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f4340499700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f433815e160 0x7f4338154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f4340499700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4338155200 con 0x7f43380afc90 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f4340499700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433815e750 con 0x7f43380746a0 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f4340499700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4338153a00 con 0x7f433815e160 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f433e235700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43380746a0 0x7f4338153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f433e235700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43380746a0 0x7f4338153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57676/0 (socket says 172.21.15.203:57676) 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f433e235700 1 -- 172.21.15.203:0/3749454770 learned_addr learned my addr 172.21.15.203:0/3749454770 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f433ea36700 1 --2- 172.21.15.203:0/3749454770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f433815e160 0x7f4338154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f433da34700 1 --2- 172.21.15.203:0/3749454770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43380afc90 0x7f4338141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f433e235700 1 -- 172.21.15.203:0/3749454770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f433815e160 msgr2=0x7f4338154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.902 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f433e235700 1 --2- 172.21.15.203:0/3749454770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f433815e160 0x7f4338154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f433e235700 1 -- 172.21.15.203:0/3749454770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43380afc90 msgr2=0x7f4338141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f433e235700 1 --2- 172.21.15.203:0/3749454770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43380afc90 0x7f4338141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f433e235700 1 -- 172.21.15.203:0/3749454770 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c0094a0 con 0x7f43380746a0 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f433e235700 1 --2- 172.21.15.203:0/3749454770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43380746a0 0x7f4338153420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f432800cbc0 tx=0x7f432800ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f4336ffd700 1 -- 172.21.15.203:0/3749454770 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432800a9e0 con 0x7f43380746a0 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f4336ffd700 1 -- 172.21.15.203:0/3749454770 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432800ab40 con 0x7f43380746a0 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f4340499700 1 -- 172.21.15.203:0/3749454770 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4338153ce0 con 0x7f43380746a0 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f4336ffd700 1 -- 172.21.15.203:0/3749454770 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328012400 con 0x7f43380746a0 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.897+0000 7f4340499700 1 -- 172.21.15.203:0/3749454770 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43381542a0 con 0x7f43380746a0 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.898+0000 7f4336ffd700 1 -- 172.21.15.203:0/3749454770 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4328012560 con 0x7f43380746a0 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.898+0000 7f4336ffd700 1 -- 172.21.15.203:0/3749454770 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f4328012c30 con 0x7f43380746a0 2023-11-26T07:11:03.903 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.898+0000 7f4340499700 1 -- 172.21.15.203:0/3749454770 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4338068fb0 con 0x7f43380746a0 2023-11-26T07:11:03.904 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.901+0000 7f4336ffd700 1 -- 172.21.15.203:0/3749454770 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4328021060 con 0x7f43380746a0 2023-11-26T07:11:03.910 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:03.908+0000 7f317dfe1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-26T07:11:03.913 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:03.911+0000 7f3809420200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-26T07:11:03.989 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.987+0000 7f4340499700 1 -- 172.21.15.203:0/3749454770 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3", "id": 2} v 0) v1 -- 0x7f433815e890 con 0x7f43380746a0 2023-11-26T07:11:03.994 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.992+0000 7f4336ffd700 1 -- 172.21.15.203:0/3749454770 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4328017070 con 0x7f43380746a0 2023-11-26T07:11:03.994 INFO:teuthology.orchestra.run.smithi203.stdout:2 2023-11-26T07:11:03.995 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.993+0000 7f4340499700 1 -- 172.21.15.203:0/3749454770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43380746a0 msgr2=0x7f4338153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:03.995 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.993+0000 7f4340499700 1 --2- 172.21.15.203:0/3749454770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43380746a0 0x7f4338153420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f432800cbc0 tx=0x7f432800ced0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.996 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.993+0000 7f4340499700 1 -- 172.21.15.203:0/3749454770 shutdown_connections 2023-11-26T07:11:03.996 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.993+0000 7f4340499700 1 --2- 172.21.15.203:0/3749454770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f433815e160 0x7f4338154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.996 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.993+0000 7f4340499700 1 --2- 172.21.15.203:0/3749454770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f43380746a0 0x7f4338153420 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.996 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.993+0000 7f4340499700 1 --2- 172.21.15.203:0/3749454770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43380afc90 0x7f4338141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:03.996 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.993+0000 7f4340499700 1 -- 172.21.15.203:0/3749454770 >> 172.21.15.203:0/3749454770 conn(0x7f43380d0bf0 msgr2=0x7f43380d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:03.996 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.993+0000 7f4340499700 1 -- 172.21.15.203:0/3749454770 shutdown_connections 2023-11-26T07:11:03.996 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:03.993+0000 7f4340499700 1 -- 172.21.15.203:0/3749454770 wait complete. 2023-11-26T07:11:04.006 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:04.003+0000 7f317dfe1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-26T07:11:04.007 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 10dbc082-3b8c-46a8-81c2-e1e471c78783 3 2023-11-26T07:11:04.171 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.169+0000 7f2a65450700 1 Processor -- start 2023-11-26T07:11:04.172 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.169+0000 7f2a65450700 1 -- start start 2023-11-26T07:11:04.173 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a65450700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2a60144760 0x7f2a600b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.173 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a65450700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2a600b4cd0 0x7f2a6006db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.173 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a65450700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a6006e0d0 0x7f2a6006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.173 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a65450700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600b4300 con 0x7f2a600b4cd0 2023-11-26T07:11:04.173 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a65450700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a60144b40 con 0x7f2a60144760 2023-11-26T07:11:04.173 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a65450700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6015ec80 con 0x7f2a6006e0d0 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5effd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2a60144760 0x7f2a600b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5f7fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a6006e0d0 0x7f2a6006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5effd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2a60144760 0x7f2a600b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57682/0 (socket says 172.21.15.203:57682) 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5effd700 1 -- 172.21.15.203:0/3907064485 learned_addr learned my addr 172.21.15.203:0/3907064485 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5e7fc700 1 --2- 172.21.15.203:0/3907064485 >> v2:172.21.15.83:3300/0 conn(0x7f2a600b4cd0 0x7f2a6006db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5effd700 1 -- 172.21.15.203:0/3907064485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a6006e0d0 msgr2=0x7f2a6006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5effd700 1 --2- 172.21.15.203:0/3907064485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a6006e0d0 0x7f2a6006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5effd700 1 -- 172.21.15.203:0/3907064485 >> v2:172.21.15.83:3300/0 conn(0x7f2a600b4cd0 msgr2=0x7f2a6006db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5effd700 1 --2- 172.21.15.203:0/3907064485 >> v2:172.21.15.83:3300/0 conn(0x7f2a600b4cd0 0x7f2a6006db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5effd700 1 -- 172.21.15.203:0/3907064485 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6006e6e0 con 0x7f2a60144760 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5f7fe700 1 --2- 172.21.15.203:0/3907064485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a6006e0d0 0x7f2a6006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5e7fc700 1 --2- 172.21.15.203:0/3907064485 >> v2:172.21.15.83:3300/0 conn(0x7f2a600b4cd0 0x7f2a6006db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:04.174 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.170+0000 7f2a5effd700 1 --2- 172.21.15.203:0/3907064485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a60144760 0x7f2a600b4790 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2a60068c60 tx=0x7f2a50009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565f586f41d1fded server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:04.175 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a5dffb700 1 -- 172.21.15.203:0/3907064485 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011830 con 0x7f2a60144760 2023-11-26T07:11:04.175 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a5dffb700 1 -- 172.21.15.203:0/3907064485 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50011990 con 0x7f2a60144760 2023-11-26T07:11:04.175 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a5dffb700 1 -- 172.21.15.203:0/3907064485 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011d20 con 0x7f2a60144760 2023-11-26T07:11:04.178 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a65450700 1 -- 172.21.15.203:0/3907064485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a60144760 msgr2=0x7f2a600b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.178 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a65450700 1 --2- 172.21.15.203:0/3907064485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a60144760 0x7f2a600b4790 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2a60068c60 tx=0x7f2a50009bd0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.178 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a65450700 1 -- 172.21.15.203:0/3907064485 shutdown_connections 2023-11-26T07:11:04.178 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a65450700 1 --2- 172.21.15.203:0/3907064485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a6006e0d0 0x7f2a6006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a65450700 1 --2- 172.21.15.203:0/3907064485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a60144760 0x7f2a600b4790 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a65450700 1 --2- 172.21.15.203:0/3907064485 >> v2:172.21.15.83:3300/0 conn(0x7f2a600b4cd0 0x7f2a6006db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a65450700 1 -- 172.21.15.203:0/3907064485 >> 172.21.15.203:0/3907064485 conn(0x7f2a60071a70 msgr2=0x7f2a60071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a65450700 1 -- 172.21.15.203:0/3907064485 shutdown_connections 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.171+0000 7f2a65450700 1 -- 172.21.15.203:0/3907064485 wait complete. 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.173+0000 7f2a65450700 1 Processor -- start 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.173+0000 7f2a65450700 1 -- start start 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.173+0000 7f2a65450700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a6006e0d0 0x7f2a6006ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.173+0000 7f2a65450700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a600b4cd0 0x7f2a600cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.173+0000 7f2a65450700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a60144760 0x7f2a601f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.173+0000 7f2a65450700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6015b600 con 0x7f2a60144760 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.173+0000 7f2a65450700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60143fd0 con 0x7f2a600b4cd0 2023-11-26T07:11:04.179 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.173+0000 7f2a65450700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60143a10 con 0x7f2a6006e0d0 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5e7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a600b4cd0 0x7f2a600cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5e7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a600b4cd0 0x7f2a600cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57688/0 (socket says 172.21.15.203:57688) 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5effd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a6006e0d0 0x7f2a6006ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5e7fc700 1 -- 172.21.15.203:0/4252777812 learned_addr learned my addr 172.21.15.203:0/4252777812 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5f7fe700 1 --2- 172.21.15.203:0/4252777812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a60144760 0x7f2a601f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5e7fc700 1 -- 172.21.15.203:0/4252777812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a6006e0d0 msgr2=0x7f2a6006ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5e7fc700 1 --2- 172.21.15.203:0/4252777812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a6006e0d0 0x7f2a6006ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5e7fc700 1 -- 172.21.15.203:0/4252777812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a60144760 msgr2=0x7f2a601f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5e7fc700 1 --2- 172.21.15.203:0/4252777812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a60144760 0x7f2a601f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5e7fc700 1 -- 172.21.15.203:0/4252777812 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a58009a70 con 0x7f2a600b4cd0 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5f7fe700 1 --2- 172.21.15.203:0/4252777812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a60144760 0x7f2a601f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a5e7fc700 1 --2- 172.21.15.203:0/4252777812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a600b4cd0 0x7f2a600cf500 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e810 tx=0x7f2a5400eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:04.180 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a477fe700 1 -- 172.21.15.203:0/4252777812 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a54004230 con 0x7f2a600b4cd0 2023-11-26T07:11:04.181 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a477fe700 1 -- 172.21.15.203:0/4252777812 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a540043b0 con 0x7f2a600b4cd0 2023-11-26T07:11:04.181 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a65450700 1 -- 172.21.15.203:0/4252777812 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a50009550 con 0x7f2a600b4cd0 2023-11-26T07:11:04.181 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a477fe700 1 -- 172.21.15.203:0/4252777812 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540038e0 con 0x7f2a600b4cd0 2023-11-26T07:11:04.181 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a65450700 1 -- 172.21.15.203:0/4252777812 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6015b220 con 0x7f2a600b4cd0 2023-11-26T07:11:04.181 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a477fe700 1 -- 172.21.15.203:0/4252777812 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2a54003d40 con 0x7f2a600b4cd0 2023-11-26T07:11:04.181 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.174+0000 7f2a477fe700 1 -- 172.21.15.203:0/4252777812 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f2a54013b90 con 0x7f2a600b4cd0 2023-11-26T07:11:04.181 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.175+0000 7f2a65450700 1 -- 172.21.15.203:0/4252777812 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a60068fb0 con 0x7f2a600b4cd0 2023-11-26T07:11:04.181 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.177+0000 7f2a477fe700 1 -- 172.21.15.203:0/4252777812 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f2a54020060 con 0x7f2a600b4cd0 2023-11-26T07:11:04.264 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.262+0000 7f2a65450700 1 -- 172.21.15.203:0/4252777812 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "10dbc082-3b8c-46a8-81c2-e1e471c78783", "id": 3} v 0) v1 -- 0x7f2a6015b740 con 0x7f2a600b4cd0 2023-11-26T07:11:04.269 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.267+0000 7f2a477fe700 1 -- 172.21.15.203:0/4252777812 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "10dbc082-3b8c-46a8-81c2-e1e471c78783", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2a6015b740 con 0x7f2a600b4cd0 2023-11-26T07:11:04.269 INFO:teuthology.orchestra.run.smithi203.stdout:3 2023-11-26T07:11:04.270 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.268+0000 7f2a65450700 1 -- 172.21.15.203:0/4252777812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a600b4cd0 msgr2=0x7f2a600cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.270 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.268+0000 7f2a65450700 1 --2- 172.21.15.203:0/4252777812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a600b4cd0 0x7f2a600cf500 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400e810 tx=0x7f2a5400eb20 comp rx=0 tx=0).stop 2023-11-26T07:11:04.270 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.268+0000 7f2a65450700 1 -- 172.21.15.203:0/4252777812 shutdown_connections 2023-11-26T07:11:04.270 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.268+0000 7f2a65450700 1 --2- 172.21.15.203:0/4252777812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a60144760 0x7f2a601f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.270 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.268+0000 7f2a65450700 1 --2- 172.21.15.203:0/4252777812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a6006e0d0 0x7f2a6006ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.270 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.268+0000 7f2a65450700 1 --2- 172.21.15.203:0/4252777812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a600b4cd0 0x7f2a600cf500 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.271 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.268+0000 7f2a65450700 1 -- 172.21.15.203:0/4252777812 >> 172.21.15.203:0/4252777812 conn(0x7f2a60071a70 msgr2=0x7f2a600b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:04.271 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.268+0000 7f2a65450700 1 -- 172.21.15.203:0/4252777812 shutdown_connections 2023-11-26T07:11:04.271 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.268+0000 7f2a65450700 1 -- 172.21.15.203:0/4252777812 wait complete. 2023-11-26T07:11:04.282 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 777669fb-d21e-4bff-b20a-514e5803d4ce 4 2023-11-26T07:11:04.355 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:04.352+0000 7f3809420200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-26T07:11:04.400 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.397+0000 7f2e8b942700 1 Processor -- start 2023-11-26T07:11:04.400 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.398+0000 7f2e8b942700 1 -- start start 2023-11-26T07:11:04.402 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.398+0000 7f2e8b942700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2e8415be70 0x7f2e840cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.402 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e8b942700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e840b5ee0 con 0x7f2e840add40 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e8b942700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8406ba00 con 0x7f2e840746a0 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e8b942700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8406bfc0 con 0x7f2e8415be70 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e896de700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2e8415be70 0x7f2e840cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e896de700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2e8415be70 0x7f2e840cfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57692/0 (socket says 172.21.15.203:57692) 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e896de700 1 -- 172.21.15.203:0/250635528 learned_addr learned my addr 172.21.15.203:0/250635528 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e89edf700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2e840746a0 0x7f2e840b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56898/0 (socket says 172.21.15.203:56898) 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e896de700 1 -- 172.21.15.203:0/250635528 >> v1:172.21.15.203:6789/0 conn(0x7f2e840746a0 legacy=0x7f2e840b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e896de700 1 -- 172.21.15.203:0/250635528 >> v1:172.21.15.83:6789/0 conn(0x7f2e840add40 legacy=0x7f2e84073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e7bfff700 1 -- 172.21.15.203:0/250635528 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383388293 0 0) 0x7f2e8406ba00 con 0x7f2e840746a0 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e896de700 1 -- 172.21.15.203:0/250635528 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8406bce0 con 0x7f2e8415be70 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e896de700 1 --2- 172.21.15.203:0/250635528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e8415be70 0x7f2e840cfdf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e800098f0 tx=0x7f2e80009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29c5f5735fd3bc09 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e7bfff700 1 -- 172.21.15.203:0/250635528 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000b8a0 con 0x7f2e8415be70 2023-11-26T07:11:04.403 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e7bfff700 1 -- 172.21.15.203:0/250635528 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8000ba00 con 0x7f2e8415be70 2023-11-26T07:11:04.404 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.399+0000 7f2e7bfff700 1 -- 172.21.15.203:0/250635528 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000bd90 con 0x7f2e8415be70 2023-11-26T07:11:04.404 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.400+0000 7f2e8b942700 1 -- 172.21.15.203:0/250635528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e8415be70 msgr2=0x7f2e840cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.404 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.400+0000 7f2e8b942700 1 --2- 172.21.15.203:0/250635528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e8415be70 0x7f2e840cfdf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e800098f0 tx=0x7f2e80009c00 comp rx=0 tx=0).stop 2023-11-26T07:11:04.404 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.400+0000 7f2e8b942700 1 -- 172.21.15.203:0/250635528 shutdown_connections 2023-11-26T07:11:04.404 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.400+0000 7f2e8b942700 1 --2- 172.21.15.203:0/250635528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e8415be70 0x7f2e840cfdf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.404 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.400+0000 7f2e8b942700 1 -- 172.21.15.203:0/250635528 >> 172.21.15.203:0/250635528 conn(0x7f2e840d0bf0 msgr2=0x7f2e840d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:04.404 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.400+0000 7f2e8b942700 1 -- 172.21.15.203:0/250635528 shutdown_connections 2023-11-26T07:11:04.404 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.400+0000 7f2e8b942700 1 -- 172.21.15.203:0/250635528 wait complete. 2023-11-26T07:11:04.406 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.402+0000 7f2e8b942700 1 Processor -- start 2023-11-26T07:11:04.406 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.402+0000 7f2e8b942700 1 -- start start 2023-11-26T07:11:04.406 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.402+0000 7f2e8b942700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e840746a0 0x7f2e8409c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.406 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e8b942700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e840add40 0x7f2e8409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.406 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e8b942700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e8415be70 0x7f2e8409af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e8b942700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8409c290 con 0x7f2e840add40 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e8b942700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e841540d0 con 0x7f2e840746a0 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e8b942700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8409b610 con 0x7f2e8415be70 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e896de700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e840746a0 0x7f2e8409c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e896de700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e840746a0 0x7f2e8409c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57698/0 (socket says 172.21.15.203:57698) 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e896de700 1 -- 172.21.15.203:0/2956551617 learned_addr learned my addr 172.21.15.203:0/2956551617 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e89edf700 1 --2- 172.21.15.203:0/2956551617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e8415be70 0x7f2e8409af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e88edd700 1 --2- 172.21.15.203:0/2956551617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e840add40 0x7f2e8409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e896de700 1 -- 172.21.15.203:0/2956551617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e8415be70 msgr2=0x7f2e8409af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e896de700 1 --2- 172.21.15.203:0/2956551617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e8415be70 0x7f2e8409af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e896de700 1 -- 172.21.15.203:0/2956551617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e840add40 msgr2=0x7f2e8409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.407 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e896de700 1 --2- 172.21.15.203:0/2956551617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e840add40 0x7f2e8409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.408 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e896de700 1 -- 172.21.15.203:0/2956551617 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e80009550 con 0x7f2e840746a0 2023-11-26T07:11:04.408 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e896de700 1 --2- 172.21.15.203:0/2956551617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e840746a0 0x7f2e8409c890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e80000c00 tx=0x7f2e80004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:04.408 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e79ffb700 1 -- 172.21.15.203:0/2956551617 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8001a8f0 con 0x7f2e840746a0 2023-11-26T07:11:04.408 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e79ffb700 1 -- 172.21.15.203:0/2956551617 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8001e070 con 0x7f2e840746a0 2023-11-26T07:11:04.408 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e8b942700 1 -- 172.21.15.203:0/2956551617 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8409b8f0 con 0x7f2e840746a0 2023-11-26T07:11:04.408 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.403+0000 7f2e79ffb700 1 -- 172.21.15.203:0/2956551617 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8001ac50 con 0x7f2e840746a0 2023-11-26T07:11:04.408 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.404+0000 7f2e8b942700 1 -- 172.21.15.203:0/2956551617 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8409beb0 con 0x7f2e840746a0 2023-11-26T07:11:04.408 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.404+0000 7f2e79ffb700 1 -- 172.21.15.203:0/2956551617 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2e8002d750 con 0x7f2e840746a0 2023-11-26T07:11:04.408 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.404+0000 7f2e79ffb700 1 -- 172.21.15.203:0/2956551617 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f2e80011a20 con 0x7f2e840746a0 2023-11-26T07:11:04.409 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.405+0000 7f2e8b942700 1 -- 172.21.15.203:0/2956551617 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e840b3a10 con 0x7f2e840746a0 2023-11-26T07:11:04.411 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.408+0000 7f2e79ffb700 1 -- 172.21.15.203:0/2956551617 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f2e80020070 con 0x7f2e840746a0 2023-11-26T07:11:04.466 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:04.464+0000 7f317dfe1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-26T07:11:04.499 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.497+0000 7f2e8b942700 1 -- 172.21.15.203:0/2956551617 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "777669fb-d21e-4bff-b20a-514e5803d4ce", "id": 4} v 0) v1 -- 0x7f2e84068fb0 con 0x7f2e840746a0 2023-11-26T07:11:04.504 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.502+0000 7f2e79ffb700 1 -- 172.21.15.203:0/2956551617 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "777669fb-d21e-4bff-b20a-514e5803d4ce", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2e80015020 con 0x7f2e840746a0 2023-11-26T07:11:04.504 INFO:teuthology.orchestra.run.smithi203.stdout:4 2023-11-26T07:11:04.505 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.503+0000 7f2e8b942700 1 -- 172.21.15.203:0/2956551617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e840746a0 msgr2=0x7f2e8409c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.503+0000 7f2e8b942700 1 --2- 172.21.15.203:0/2956551617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e840746a0 0x7f2e8409c890 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e80000c00 tx=0x7f2e80004750 comp rx=0 tx=0).stop 2023-11-26T07:11:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.503+0000 7f2e8b942700 1 -- 172.21.15.203:0/2956551617 shutdown_connections 2023-11-26T07:11:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.503+0000 7f2e8b942700 1 --2- 172.21.15.203:0/2956551617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e8415be70 0x7f2e8409af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.503+0000 7f2e8b942700 1 --2- 172.21.15.203:0/2956551617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e840746a0 0x7f2e8409c890 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.503+0000 7f2e8b942700 1 --2- 172.21.15.203:0/2956551617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e840add40 0x7f2e8409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.503+0000 7f2e8b942700 1 -- 172.21.15.203:0/2956551617 >> 172.21.15.203:0/2956551617 conn(0x7f2e840d0bf0 msgr2=0x7f2e840af710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.503+0000 7f2e8b942700 1 -- 172.21.15.203:0/2956551617 shutdown_connections 2023-11-26T07:11:04.506 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.503+0000 7f2e8b942700 1 -- 172.21.15.203:0/2956551617 wait complete. 2023-11-26T07:11:04.518 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 75d30c7f-9ed3-45e8-be57-a79c37c2f1f6 5 2023-11-26T07:11:04.640 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.638+0000 7f2fd4c56700 1 Processor -- start 2023-11-26T07:11:04.640 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.638+0000 7f2fd4c56700 1 -- start start 2023-11-26T07:11:04.643 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.639+0000 7f2fd4c56700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd00746a0 0x7f2fd00add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.643 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.639+0000 7f2fd4c56700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd006b080 con 0x7f2fd015f9e0 2023-11-26T07:11:04.643 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.639+0000 7f2fd4c56700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd015c290 con 0x7f2fd00cf610 2023-11-26T07:11:04.643 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.639+0000 7f2fd4c56700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd00cf9f0 con 0x7f2fd00746a0 2023-11-26T07:11:04.643 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fce59c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2fd00cf610 0x7f2fd00b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56906/0 (socket says 172.21.15.203:56906) 2023-11-26T07:11:04.643 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fced9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd00746a0 0x7f2fd00add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.643 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fce59c700 1 -- 172.21.15.203:0/3865791488 learned_addr learned my addr 172.21.15.203:0/3865791488 (peer_addr_for_me v1:172.21.15.203:0/0) 2023-11-26T07:11:04.643 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fcd59a700 1 -- 172.21.15.203:0/3865791488 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252660979 0 0) 0x7f2fd015c290 con 0x7f2fd00cf610 2023-11-26T07:11:04.643 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fcd59a700 1 -- 172.21.15.203:0/3865791488 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fb80035b0 con 0x7f2fd00cf610 2023-11-26T07:11:04.644 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fced9d700 1 -- 172.21.15.203:0/3865791488 >> v1:172.21.15.203:6789/0 conn(0x7f2fd00cf610 legacy=0x7f2fd00b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.644 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fced9d700 1 -- 172.21.15.203:0/3865791488 >> v1:172.21.15.83:6789/0 conn(0x7f2fd015f9e0 legacy=0x7f2fd0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.644 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fcd59a700 1 -- 172.21.15.203:0/3865791488 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604297100 0 0) 0x7f2fd006b080 con 0x7f2fd015f9e0 2023-11-26T07:11:04.644 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fced9d700 1 -- 172.21.15.203:0/3865791488 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd015ff80 con 0x7f2fd00746a0 2023-11-26T07:11:04.644 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fced9d700 1 --2- 172.21.15.203:0/3865791488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd00746a0 0x7f2fd00add40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009dc0 tx=0x7f2fc800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed4a09937a55a26 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:04.644 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fcd59a700 1 -- 172.21.15.203:0/3865791488 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800e070 con 0x7f2fd00746a0 2023-11-26T07:11:04.644 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fcd59a700 1 -- 172.21.15.203:0/3865791488 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc800bb20 con 0x7f2fd00746a0 2023-11-26T07:11:04.644 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.640+0000 7f2fcd59a700 1 -- 172.21.15.203:0/3865791488 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800bed0 con 0x7f2fd00746a0 2023-11-26T07:11:04.645 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.641+0000 7f2fd4c56700 1 -- 172.21.15.203:0/3865791488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd00746a0 msgr2=0x7f2fd00add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.645 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.641+0000 7f2fd4c56700 1 --2- 172.21.15.203:0/3865791488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd00746a0 0x7f2fd00add40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009dc0 tx=0x7f2fc800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:04.645 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.641+0000 7f2fd4c56700 1 -- 172.21.15.203:0/3865791488 shutdown_connections 2023-11-26T07:11:04.645 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.641+0000 7f2fd4c56700 1 --2- 172.21.15.203:0/3865791488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd00746a0 0x7f2fd00add40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.645 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.641+0000 7f2fd4c56700 1 -- 172.21.15.203:0/3865791488 >> 172.21.15.203:0/3865791488 conn(0x7f2fd00d0bf0 msgr2=0x7f2fd00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:04.645 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.641+0000 7f2fd4c56700 1 -- 172.21.15.203:0/3865791488 shutdown_connections 2023-11-26T07:11:04.645 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.641+0000 7f2fd4c56700 1 -- 172.21.15.203:0/3865791488 wait complete. 2023-11-26T07:11:04.646 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.643+0000 7f2fd4c56700 1 Processor -- start 2023-11-26T07:11:04.646 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.643+0000 7f2fd4c56700 1 -- start start 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fd4c56700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd00746a0 0x7f2fd01f1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fd4c56700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd00cf610 0x7f2fd0155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fd4c56700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd015f9e0 0x7f2fd015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fd4c56700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd0157770 con 0x7f2fd00746a0 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fd4c56700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd01f15d0 con 0x7f2fd00cf610 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fd4c56700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd01572e0 con 0x7f2fd015f9e0 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fcdd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd00cf610 0x7f2fd0155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fcdd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd00cf610 0x7f2fd0155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57712/0 (socket says 172.21.15.203:57712) 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fcdd9b700 1 -- 172.21.15.203:0/1062955226 learned_addr learned my addr 172.21.15.203:0/1062955226 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fce59c700 1 --2- 172.21.15.203:0/1062955226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd00746a0 0x7f2fd01f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fced9d700 1 --2- 172.21.15.203:0/1062955226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd015f9e0 0x7f2fd015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fcdd9b700 1 -- 172.21.15.203:0/1062955226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd015f9e0 msgr2=0x7f2fd015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fcdd9b700 1 --2- 172.21.15.203:0/1062955226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd015f9e0 0x7f2fd015a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fcdd9b700 1 -- 172.21.15.203:0/1062955226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd00746a0 msgr2=0x7f2fd01f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fcdd9b700 1 --2- 172.21.15.203:0/1062955226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd00746a0 0x7f2fd01f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fcdd9b700 1 -- 172.21.15.203:0/1062955226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8009a70 con 0x7f2fd00cf610 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fced9d700 1 --2- 172.21.15.203:0/1062955226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd015f9e0 0x7f2fd015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fcdd9b700 1 --2- 172.21.15.203:0/1062955226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd00cf610 0x7f2fd0155c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c550 tx=0x7f2fc400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fbeffd700 1 -- 172.21.15.203:0/1062955226 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40054e0 con 0x7f2fd00cf610 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fbeffd700 1 -- 172.21.15.203:0/1062955226 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc4003680 con 0x7f2fd00cf610 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fbeffd700 1 -- 172.21.15.203:0/1062955226 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4003a30 con 0x7f2fd00cf610 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.645+0000 7f2fd4c56700 1 -- 172.21.15.203:0/1062955226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd01566d0 con 0x7f2fd00cf610 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.646+0000 7f2fd4c56700 1 -- 172.21.15.203:0/1062955226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd0156c90 con 0x7f2fd00cf610 2023-11-26T07:11:04.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.646+0000 7f2fbeffd700 1 -- 172.21.15.203:0/1062955226 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2fc401c440 con 0x7f2fd00cf610 2023-11-26T07:11:04.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.646+0000 7f2fbeffd700 1 -- 172.21.15.203:0/1062955226 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f2fc4012980 con 0x7f2fd00cf610 2023-11-26T07:11:04.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.646+0000 7f2fd4c56700 1 -- 172.21.15.203:0/1062955226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd0157450 con 0x7f2fd00cf610 2023-11-26T07:11:04.652 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.650+0000 7f2fbeffd700 1 -- 172.21.15.203:0/1062955226 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f2fc4017070 con 0x7f2fd00cf610 2023-11-26T07:11:04.736 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.734+0000 7f2fd4c56700 1 -- 172.21.15.203:0/1062955226 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "75d30c7f-9ed3-45e8-be57-a79c37c2f1f6", "id": 5} v 0) v1 -- 0x7f2fd0068fb0 con 0x7f2fd00cf610 2023-11-26T07:11:04.742 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.740+0000 7f2fbeffd700 1 -- 172.21.15.203:0/1062955226 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "75d30c7f-9ed3-45e8-be57-a79c37c2f1f6", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2fc4012400 con 0x7f2fd00cf610 2023-11-26T07:11:04.742 INFO:teuthology.orchestra.run.smithi203.stdout:5 2023-11-26T07:11:04.743 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.741+0000 7f2fd4c56700 1 -- 172.21.15.203:0/1062955226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd00cf610 msgr2=0x7f2fd0155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.743 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.741+0000 7f2fd4c56700 1 --2- 172.21.15.203:0/1062955226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd00cf610 0x7f2fd0155c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c550 tx=0x7f2fc400c860 comp rx=0 tx=0).stop 2023-11-26T07:11:04.743 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.741+0000 7f2fd4c56700 1 -- 172.21.15.203:0/1062955226 shutdown_connections 2023-11-26T07:11:04.743 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.741+0000 7f2fd4c56700 1 --2- 172.21.15.203:0/1062955226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2fd015f9e0 0x7f2fd015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.743 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.741+0000 7f2fd4c56700 1 --2- 172.21.15.203:0/1062955226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fd00746a0 0x7f2fd01f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.743 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.741+0000 7f2fd4c56700 1 --2- 172.21.15.203:0/1062955226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2fd00cf610 0x7f2fd0155c80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.744 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.741+0000 7f2fd4c56700 1 -- 172.21.15.203:0/1062955226 >> 172.21.15.203:0/1062955226 conn(0x7f2fd00d0bf0 msgr2=0x7f2fd00b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:04.744 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.741+0000 7f2fd4c56700 1 -- 172.21.15.203:0/1062955226 shutdown_connections 2023-11-26T07:11:04.744 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.741+0000 7f2fd4c56700 1 -- 172.21.15.203:0/1062955226 wait complete. 2023-11-26T07:11:04.755 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new b639b30a-d697-4550-8610-09f074b23c40 6 2023-11-26T07:11:04.885 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.883+0000 7f5c2f3e7700 1 Processor -- start 2023-11-26T07:11:04.888 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.883+0000 7f5c2f3e7700 1 -- start start 2023-11-26T07:11:04.888 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2f3e7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c281405e0 0x7f5c2806cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.888 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2f3e7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5c2815e380 0x7f5c28070e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.888 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2f3e7700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2806e100 con 0x7f5c2815e960 2023-11-26T07:11:04.889 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2f3e7700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c281409c0 con 0x7f5c2815e380 2023-11-26T07:11:04.889 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2f3e7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2806d4f0 con 0x7f5c281405e0 2023-11-26T07:11:04.889 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2d183700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c281405e0 0x7f5c2806cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.889 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2d183700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c281405e0 0x7f5c2806cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:48856/0 (socket says 172.21.15.203:48856) 2023-11-26T07:11:04.889 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2d183700 1 -- 172.21.15.203:0/2694562886 learned_addr learned my addr 172.21.15.203:0/2694562886 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:04.889 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c1ffff700 1 -- 172.21.15.203:0/2694562886 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469701045 0 0) 0x7f5c2806e100 con 0x7f5c2815e960 2023-11-26T07:11:04.889 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c1ffff700 1 -- 172.21.15.203:0/2694562886 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c140035b0 con 0x7f5c2815e960 2023-11-26T07:11:04.889 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2d183700 1 -- 172.21.15.203:0/2694562886 >> v2:172.21.15.83:3300/0 conn(0x7f5c2815e380 msgr2=0x7f5c28070e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-26T07:11:04.889 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2d183700 1 --2- 172.21.15.203:0/2694562886 >> v2:172.21.15.83:3300/0 conn(0x7f5c2815e380 0x7f5c28070e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.889 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2d183700 1 -- 172.21.15.203:0/2694562886 >> v1:172.21.15.83:6789/0 conn(0x7f5c2815e960 legacy=0x7f5c2815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.890 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.885+0000 7f5c2d183700 1 -- 172.21.15.203:0/2694562886 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2806d7d0 con 0x7f5c281405e0 2023-11-26T07:11:04.890 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.886+0000 7f5c2d183700 1 --2- 172.21.15.203:0/2694562886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c281405e0 0x7f5c2806cb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009b50 tx=0x7f5c24009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57efa9988f117061 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:04.890 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.886+0000 7f5c1ffff700 1 -- 172.21.15.203:0/2694562886 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2400b850 con 0x7f5c281405e0 2023-11-26T07:11:04.890 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.886+0000 7f5c1ffff700 1 -- 172.21.15.203:0/2694562886 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2400b9b0 con 0x7f5c281405e0 2023-11-26T07:11:04.891 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.886+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2694562886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c281405e0 msgr2=0x7f5c2806cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.891 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.886+0000 7f5c2f3e7700 1 --2- 172.21.15.203:0/2694562886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c281405e0 0x7f5c2806cb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009b50 tx=0x7f5c24009e60 comp rx=0 tx=0).stop 2023-11-26T07:11:04.891 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.887+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2694562886 shutdown_connections 2023-11-26T07:11:04.891 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.887+0000 7f5c2f3e7700 1 --2- 172.21.15.203:0/2694562886 >> v2:172.21.15.83:3300/0 conn(0x7f5c2815e380 0x7f5c28070e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.891 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.887+0000 7f5c2f3e7700 1 --2- 172.21.15.203:0/2694562886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c281405e0 0x7f5c2806cb80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.891 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.887+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2694562886 >> 172.21.15.203:0/2694562886 conn(0x7f5c2806b930 msgr2=0x7f5c280b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:04.891 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.887+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2694562886 shutdown_connections 2023-11-26T07:11:04.892 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.887+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2694562886 wait complete. 2023-11-26T07:11:04.892 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.888+0000 7f5c2f3e7700 1 Processor -- start 2023-11-26T07:11:04.892 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.889+0000 7f5c2f3e7700 1 -- start start 2023-11-26T07:11:04.893 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2f3e7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c2815e380 0x7f5c2813bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.893 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2f3e7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2815e960 0x7f5c28157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.893 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2f3e7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2813de30 0x7f5c2813e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:04.893 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2f3e7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28157460 con 0x7f5c2815e960 2023-11-26T07:11:04.894 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2f3e7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2813e800 con 0x7f5c2813de30 2023-11-26T07:11:04.894 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2f3e7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2813c090 con 0x7f5c2815e380 2023-11-26T07:11:04.894 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2d183700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c2815e380 0x7f5c2813bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.894 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2c982700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2815e960 0x7f5c28157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.894 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2d984700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2813de30 0x7f5c2813e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:04.894 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2c982700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2815e960 0x7f5c28157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.203:38162/0 (socket says 172.21.15.203:38162) 2023-11-26T07:11:04.894 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2c982700 1 -- 172.21.15.203:0/2923952185 learned_addr learned my addr 172.21.15.203:0/2923952185 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:04.894 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2d984700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2813de30 0x7f5c2813e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57724/0 (socket says 172.21.15.203:57724) 2023-11-26T07:11:04.894 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2c982700 1 -- 172.21.15.203:0/2923952185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c2815e380 msgr2=0x7f5c2813bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.894 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2c982700 1 --2- 172.21.15.203:0/2923952185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c2815e380 0x7f5c2813bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.895 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.890+0000 7f5c2c982700 1 -- 172.21.15.203:0/2923952185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2813de30 msgr2=0x7f5c2813e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:04.895 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c2c982700 1 --2- 172.21.15.203:0/2923952185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2813de30 0x7f5c2813e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:04.895 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c2c982700 1 -- 172.21.15.203:0/2923952185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24009550 con 0x7f5c2815e960 2023-11-26T07:11:04.895 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c2d984700 1 --2- 172.21.15.203:0/2923952185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2813de30 0x7f5c2813e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:04.895 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c2d183700 1 --2- 172.21.15.203:0/2923952185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c2815e380 0x7f5c2813bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:04.895 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c2c982700 1 --2- 172.21.15.203:0/2923952185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2815e960 0x7f5c28157880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800c530 tx=0x7f5c1800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:04.896 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c1dffb700 1 -- 172.21.15.203:0/2923952185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800cea0 con 0x7f5c2815e960 2023-11-26T07:11:04.896 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c1dffb700 1 -- 172.21.15.203:0/2923952185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c180052f0 con 0x7f5c2815e960 2023-11-26T07:11:04.896 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c1dffb700 1 -- 172.21.15.203:0/2923952185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18003680 con 0x7f5c2815e960 2023-11-26T07:11:04.896 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2923952185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2813cb00 con 0x7f5c2815e960 2023-11-26T07:11:04.896 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c1dffb700 1 -- 172.21.15.203:0/2923952185 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5c18003ae0 con 0x7f5c2815e960 2023-11-26T07:11:04.896 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2923952185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c28156ac0 con 0x7f5c2815e960 2023-11-26T07:11:04.896 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.891+0000 7f5c1dffb700 1 -- 172.21.15.203:0/2923952185 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f5c18011db0 con 0x7f5c2815e960 2023-11-26T07:11:04.896 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.892+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2923952185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c080052e0 con 0x7f5c2815e960 2023-11-26T07:11:04.897 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.895+0000 7f5c1dffb700 1 -- 172.21.15.203:0/2923952185 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5c18017070 con 0x7f5c2815e960 2023-11-26T07:11:04.983 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:04.981+0000 7f3809420200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-26T07:11:04.998 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.995+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2923952185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b639b30a-d697-4550-8610-09f074b23c40", "id": 6} v 0) v1 -- 0x7f5c08005630 con 0x7f5c2815e960 2023-11-26T07:11:05.001 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:04.999+0000 7f5c1dffb700 1 -- 172.21.15.203:0/2923952185 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b639b30a-d697-4550-8610-09f074b23c40", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5c18011590 con 0x7f5c2815e960 2023-11-26T07:11:05.002 INFO:teuthology.orchestra.run.smithi203.stdout:6 2023-11-26T07:11:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.000+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2923952185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2815e960 msgr2=0x7f5c28157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.000+0000 7f5c2f3e7700 1 --2- 172.21.15.203:0/2923952185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2815e960 0x7f5c28157880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800c530 tx=0x7f5c1800c840 comp rx=0 tx=0).stop 2023-11-26T07:11:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.000+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2923952185 shutdown_connections 2023-11-26T07:11:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.000+0000 7f5c2f3e7700 1 --2- 172.21.15.203:0/2923952185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c2813de30 0x7f5c2813e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.000+0000 7f5c2f3e7700 1 --2- 172.21.15.203:0/2923952185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c2815e380 0x7f5c2813bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.000+0000 7f5c2f3e7700 1 --2- 172.21.15.203:0/2923952185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c2815e960 0x7f5c28157880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.000+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2923952185 >> 172.21.15.203:0/2923952185 conn(0x7f5c2806b930 msgr2=0x7f5c2813f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.000+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2923952185 shutdown_connections 2023-11-26T07:11:05.003 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.000+0000 7f5c2f3e7700 1 -- 172.21.15.203:0/2923952185 wait complete. 2023-11-26T07:11:05.015 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new e1479b74-f049-477c-97c9-101df7533c7b 7 2023-11-26T07:11:05.051 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:05.049+0000 7f3809420200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-26T07:11:05.135 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:05.133+0000 7f317dfe1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-26T07:11:05.139 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.137+0000 7fa668721700 1 Processor -- start 2023-11-26T07:11:05.139 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.137+0000 7fa668721700 1 -- start start 2023-11-26T07:11:05.141 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.138+0000 7fa668721700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660073f90 0x7fa6600743e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:05.141 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.138+0000 7fa668721700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa660074920 con 0x7fa660145560 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.138+0000 7fa668721700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6600b5b60 con 0x7fa6600ade90 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.138+0000 7fa668721700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6600b6120 con 0x7fa660073f90 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.138+0000 7fa665cbc700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa6600ade90 0x7fa66006b990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56928/0 (socket says 172.21.15.203:56928) 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.138+0000 7fa665cbc700 1 -- 172.21.15.203:0/2148134127 learned_addr learned my addr 172.21.15.203:0/2148134127 (peer_addr_for_me v1:172.21.15.203:0/0) 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.138+0000 7fa666cbe700 1 --2- 172.21.15.203:0/2148134127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660073f90 0x7fa6600743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.139+0000 7fa6654bb700 1 -- 172.21.15.203:0/2148134127 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971015153 0 0) 0x7fa6600b5b60 con 0x7fa6600ade90 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.139+0000 7fa6654bb700 1 -- 172.21.15.203:0/2148134127 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa64c0035b0 con 0x7fa6600ade90 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.139+0000 7fa666cbe700 1 -- 172.21.15.203:0/2148134127 >> v1:172.21.15.203:6789/0 conn(0x7fa6600ade90 legacy=0x7fa66006b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.139+0000 7fa666cbe700 1 -- 172.21.15.203:0/2148134127 >> v1:172.21.15.83:6789/0 conn(0x7fa660145560 legacy=0x7fa6600b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.139+0000 7fa666cbe700 1 -- 172.21.15.203:0/2148134127 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6600b5e40 con 0x7fa660073f90 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.139+0000 7fa666cbe700 1 --2- 172.21.15.203:0/2148134127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660073f90 0x7fa6600743e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa658009dc0 tx=0x7fa65800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6496ba543529d1b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:05.142 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.139+0000 7fa6654bb700 1 -- 172.21.15.203:0/2148134127 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65800e070 con 0x7fa660073f90 2023-11-26T07:11:05.143 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.139+0000 7fa6654bb700 1 -- 172.21.15.203:0/2148134127 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65800bb20 con 0x7fa660073f90 2023-11-26T07:11:05.143 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.139+0000 7fa6654bb700 1 -- 172.21.15.203:0/2148134127 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658012430 con 0x7fa660073f90 2023-11-26T07:11:05.145 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.140+0000 7fa668721700 1 -- 172.21.15.203:0/2148134127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660073f90 msgr2=0x7fa6600743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.140+0000 7fa668721700 1 --2- 172.21.15.203:0/2148134127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660073f90 0x7fa6600743e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa658009dc0 tx=0x7fa65800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.140+0000 7fa668721700 1 -- 172.21.15.203:0/2148134127 shutdown_connections 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.140+0000 7fa668721700 1 --2- 172.21.15.203:0/2148134127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660073f90 0x7fa6600743e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.140+0000 7fa668721700 1 -- 172.21.15.203:0/2148134127 >> 172.21.15.203:0/2148134127 conn(0x7fa6600b28f0 msgr2=0x7fa6600b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.140+0000 7fa668721700 1 -- 172.21.15.203:0/2148134127 shutdown_connections 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.140+0000 7fa668721700 1 -- 172.21.15.203:0/2148134127 wait complete. 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.141+0000 7fa668721700 1 Processor -- start 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.141+0000 7fa668721700 1 -- start start 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa668721700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa660073f90 0x7fa6601f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa668721700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6600ade90 0x7fa66009e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa668721700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660145560 0x7fa6601400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa668721700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66009b1c0 con 0x7fa660073f90 2023-11-26T07:11:05.146 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa668721700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66009dfe0 con 0x7fa6600ade90 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa668721700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa660140600 con 0x7fa660145560 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa666cbe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660145560 0x7fa6601400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa666cbe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660145560 0x7fa6601400c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:48874/0 (socket says 172.21.15.203:48874) 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa666cbe700 1 -- 172.21.15.203:0/3376171276 learned_addr learned my addr 172.21.15.203:0/3376171276 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa665cbc700 1 --2- 172.21.15.203:0/3376171276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6600ade90 0x7fa66009e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa6664bd700 1 --2- 172.21.15.203:0/3376171276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa660073f90 0x7fa6601f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa666cbe700 1 -- 172.21.15.203:0/3376171276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6600ade90 msgr2=0x7fa66009e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa666cbe700 1 --2- 172.21.15.203:0/3376171276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6600ade90 0x7fa66009e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa666cbe700 1 -- 172.21.15.203:0/3376171276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa660073f90 msgr2=0x7fa6601f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa666cbe700 1 --2- 172.21.15.203:0/3376171276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa660073f90 0x7fa6601f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa666cbe700 1 -- 172.21.15.203:0/3376171276 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa658009a70 con 0x7fa660145560 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa665cbc700 1 --2- 172.21.15.203:0/3376171276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6600ade90 0x7fa66009e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa666cbe700 1 --2- 172.21.15.203:0/3376171276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660145560 0x7fa6601400c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa658014040 tx=0x7fa658004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:05.147 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa656ffd700 1 -- 172.21.15.203:0/3376171276 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658012430 con 0x7fa660145560 2023-11-26T07:11:05.148 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa656ffd700 1 -- 172.21.15.203:0/3376171276 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa658020070 con 0x7fa660145560 2023-11-26T07:11:05.148 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa656ffd700 1 -- 172.21.15.203:0/3376171276 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658004280 con 0x7fa660145560 2023-11-26T07:11:05.148 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.142+0000 7fa668721700 1 -- 172.21.15.203:0/3376171276 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66009d3c0 con 0x7fa660145560 2023-11-26T07:11:05.148 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.143+0000 7fa668721700 1 -- 172.21.15.203:0/3376171276 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66009d920 con 0x7fa660145560 2023-11-26T07:11:05.148 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.143+0000 7fa656ffd700 1 -- 172.21.15.203:0/3376171276 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa6580046e0 con 0x7fa660145560 2023-11-26T07:11:05.148 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.143+0000 7fa656ffd700 1 -- 172.21.15.203:0/3376171276 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fa658022070 con 0x7fa660145560 2023-11-26T07:11:05.148 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.143+0000 7fa668721700 1 -- 172.21.15.203:0/3376171276 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66009e120 con 0x7fa660145560 2023-11-26T07:11:05.148 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.146+0000 7fa656ffd700 1 -- 172.21.15.203:0/3376171276 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa66009e120 con 0x7fa660145560 2023-11-26T07:11:05.207 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:05.205+0000 7f317dfe1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-26T07:11:05.232 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.229+0000 7fa668721700 1 -- 172.21.15.203:0/3376171276 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e1479b74-f049-477c-97c9-101df7533c7b", "id": 7} v 0) v1 -- 0x7fa660068fb0 con 0x7fa660145560 2023-11-26T07:11:05.240 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.238+0000 7fa656ffd700 1 -- 172.21.15.203:0/3376171276 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e1479b74-f049-477c-97c9-101df7533c7b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa65802c060 con 0x7fa660145560 2023-11-26T07:11:05.240 INFO:teuthology.orchestra.run.smithi203.stdout:7 2023-11-26T07:11:05.241 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.239+0000 7fa668721700 1 -- 172.21.15.203:0/3376171276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660145560 msgr2=0x7fa6601400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.241 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.239+0000 7fa668721700 1 --2- 172.21.15.203:0/3376171276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660145560 0x7fa6601400c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa658014040 tx=0x7fa658004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.241 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.239+0000 7fa668721700 1 -- 172.21.15.203:0/3376171276 shutdown_connections 2023-11-26T07:11:05.242 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.239+0000 7fa668721700 1 --2- 172.21.15.203:0/3376171276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa660145560 0x7fa6601400c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.242 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.239+0000 7fa668721700 1 --2- 172.21.15.203:0/3376171276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa660073f90 0x7fa6601f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.242 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.239+0000 7fa668721700 1 --2- 172.21.15.203:0/3376171276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6600ade90 0x7fa66009e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.242 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.239+0000 7fa668721700 1 -- 172.21.15.203:0/3376171276 >> 172.21.15.203:0/3376171276 conn(0x7fa6600b28f0 msgr2=0x7fa6600b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:05.242 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.239+0000 7fa668721700 1 -- 172.21.15.203:0/3376171276 shutdown_connections 2023-11-26T07:11:05.242 INFO:teuthology.orchestra.run.smithi203.stderr:2023-11-26T07:11:05.239+0000 7fa668721700 1 -- 172.21.15.203:0/3376171276 wait complete. 2023-11-26T07:11:05.253 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-26T07:11:05.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-26T07:11:05.255 INFO:tasks.ceph.osd.0:Started 2023-11-26T07:11:05.255 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-26T07:11:05.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-26T07:11:05.257 INFO:tasks.ceph.osd.1:Started 2023-11-26T07:11:05.257 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-26T07:11:05.257 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-26T07:11:05.259 INFO:tasks.ceph.osd.2:Started 2023-11-26T07:11:05.259 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-26T07:11:05.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-26T07:11:05.261 INFO:tasks.ceph.osd.3:Started 2023-11-26T07:11:05.261 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-26T07:11:05.262 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-26T07:11:05.266 INFO:tasks.ceph.osd.4:Started 2023-11-26T07:11:05.266 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-26T07:11:05.267 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-26T07:11:05.268 INFO:tasks.ceph.osd.5:Started 2023-11-26T07:11:05.268 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-26T07:11:05.269 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-26T07:11:05.270 INFO:tasks.ceph.osd.6:Started 2023-11-26T07:11:05.270 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-26T07:11:05.271 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-26T07:11:05.272 INFO:tasks.ceph.osd.7:Started 2023-11-26T07:11:05.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:05.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.407+0000 7f65b549d700 1 Processor -- start 2023-11-26T07:11:05.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.407+0000 7f65b549d700 1 -- start start 2023-11-26T07:11:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65b549d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f65b015e1e0 0x7f65b00d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65b549d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b00d20b0 con 0x7f65b00d0f80 2023-11-26T07:11:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65b549d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b00d2390 con 0x7f65b00d3a00 2023-11-26T07:11:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65b549d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b00d3150 con 0x7f65b015e1e0 2023-11-26T07:11:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65af7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f65b00d0f80 0x7f65b00b6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53520/0 (socket says 172.21.15.83:53520) 2023-11-26T07:11:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65af7fe700 1 -- 172.21.15.83:0/2342492051 learned_addr learned my addr 172.21.15.83:0/2342492051 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65aeffd700 1 --2- 172.21.15.83:0/2342492051 >> v2:172.21.15.203:3300/0 conn(0x7f65b015e1e0 0x7f65b00d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65adffb700 1 -- 172.21.15.83:0/2342492051 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2740665553 0 0) 0x7f65b00d20b0 con 0x7f65b00d0f80 2023-11-26T07:11:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65adffb700 1 -- 172.21.15.83:0/2342492051 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65980035b0 con 0x7f65b00d0f80 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65aeffd700 1 -- 172.21.15.83:0/2342492051 >> v1:172.21.15.203:6789/0 conn(0x7f65b00d3a00 legacy=0x7f65b00d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65aeffd700 1 -- 172.21.15.83:0/2342492051 >> v1:172.21.15.83:6789/0 conn(0x7f65b00d0f80 legacy=0x7f65b00b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.408+0000 7f65aeffd700 1 -- 172.21.15.83:0/2342492051 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b00d2670 con 0x7f65b015e1e0 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.409+0000 7f65aeffd700 1 --2- 172.21.15.83:0/2342492051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65b015e1e0 0x7f65b00d34c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009840 tx=0x7f65a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70c67c972b230492 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.409+0000 7f65adffb700 1 -- 172.21.15.83:0/2342492051 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a4011830 con 0x7f65b015e1e0 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.409+0000 7f65adffb700 1 -- 172.21.15.83:0/2342492051 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a4011990 con 0x7f65b015e1e0 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.409+0000 7f65adffb700 1 -- 172.21.15.83:0/2342492051 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a4011d20 con 0x7f65b015e1e0 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.409+0000 7f65b549d700 1 -- 172.21.15.83:0/2342492051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65b015e1e0 msgr2=0x7f65b00d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.409+0000 7f65b549d700 1 --2- 172.21.15.83:0/2342492051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65b015e1e0 0x7f65b00d34c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009840 tx=0x7f65a4009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.409+0000 7f65b549d700 1 -- 172.21.15.83:0/2342492051 shutdown_connections 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.409+0000 7f65b549d700 1 --2- 172.21.15.83:0/2342492051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65b015e1e0 0x7f65b00d34c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.409+0000 7f65b549d700 1 -- 172.21.15.83:0/2342492051 >> 172.21.15.83:0/2342492051 conn(0x7f65b00b5b50 msgr2=0x7f65b00b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.409+0000 7f65b549d700 1 -- 172.21.15.83:0/2342492051 shutdown_connections 2023-11-26T07:11:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.409+0000 7f65b549d700 1 -- 172.21.15.83:0/2342492051 wait complete. 2023-11-26T07:11:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.411+0000 7f65b549d700 1 Processor -- start 2023-11-26T07:11:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.411+0000 7f65b549d700 1 -- start start 2023-11-26T07:11:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.411+0000 7f65b549d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65b00d0f80 0x7f65b00cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.411+0000 7f65b549d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b00d3a00 0x7f65b01f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.411+0000 7f65b549d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65b015e1e0 0x7f65b00ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.411+0000 7f65b549d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b00aabe0 con 0x7f65b00d3a00 2023-11-26T07:11:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.411+0000 7f65b549d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b00aa840 con 0x7f65b00d0f80 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.411+0000 7f65b549d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b00cc190 con 0x7f65b015e1e0 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65ae7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b00d3a00 0x7f65b01f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65ae7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b00d3a00 0x7f65b01f2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33484/0 (socket says 172.21.15.83:33484) 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65ae7fc700 1 -- 172.21.15.83:0/3438291775 learned_addr learned my addr 172.21.15.83:0/3438291775 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65aeffd700 1 --2- 172.21.15.83:0/3438291775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65b00d0f80 0x7f65b00cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65af7fe700 1 --2- 172.21.15.83:0/3438291775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65b015e1e0 0x7f65b00ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65ae7fc700 1 -- 172.21.15.83:0/3438291775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65b015e1e0 msgr2=0x7f65b00ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65ae7fc700 1 --2- 172.21.15.83:0/3438291775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65b015e1e0 0x7f65b00ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65ae7fc700 1 -- 172.21.15.83:0/3438291775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65b00d0f80 msgr2=0x7f65b00cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65ae7fc700 1 --2- 172.21.15.83:0/3438291775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65b00d0f80 0x7f65b00cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65ae7fc700 1 -- 172.21.15.83:0/3438291775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a40094a0 con 0x7f65b00d3a00 2023-11-26T07:11:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65ae7fc700 1 --2- 172.21.15.83:0/3438291775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b00d3a00 0x7f65b01f2e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65a000c4b0 tx=0x7f65a000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65977fe700 1 -- 172.21.15.83:0/3438291775 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a00052d0 con 0x7f65b00d3a00 2023-11-26T07:11:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65977fe700 1 -- 172.21.15.83:0/3438291775 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a0005430 con 0x7f65b00d3a00 2023-11-26T07:11:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65b549d700 1 -- 172.21.15.83:0/3438291775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b00cc470 con 0x7f65b00d3a00 2023-11-26T07:11:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65977fe700 1 -- 172.21.15.83:0/3438291775 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a00038c0 con 0x7f65b00d3a00 2023-11-26T07:11:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65b549d700 1 -- 172.21.15.83:0/3438291775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b00cbe80 con 0x7f65b00d3a00 2023-11-26T07:11:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65977fe700 1 -- 172.21.15.83:0/3438291775 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f65a0003d20 con 0x7f65b00d3a00 2023-11-26T07:11:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.412+0000 7f65977fe700 1 -- 172.21.15.83:0/3438291775 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f65a0011d10 con 0x7f65b00d3a00 2023-11-26T07:11:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.413+0000 7f65b549d700 1 -- 172.21.15.83:0/3438291775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b00aae50 con 0x7f65b00d3a00 2023-11-26T07:11:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.415+0000 7f65977fe700 1 -- 172.21.15.83:0/3438291775 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f65a0017070 con 0x7f65b00d3a00 2023-11-26T07:11:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.485+0000 7f65b549d700 1 -- 172.21.15.83:0/3438291775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65b00751e0 con 0x7f65b00d3a00 2023-11-26T07:11:05.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.485+0000 7f65977fe700 1 -- 172.21.15.83:0/3438291775 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f65a0011590 con 0x7f65b00d3a00 2023-11-26T07:11:05.488 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:05.232194+0000","last_up_change":"0.000000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.486+0000 7f65b549d700 1 -- 172.21.15.83:0/3438291775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b00d3a00 msgr2=0x7f65b01f2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.486+0000 7f65b549d700 1 --2- 172.21.15.83:0/3438291775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b00d3a00 0x7f65b01f2e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65a000c4b0 tx=0x7f65a000c7c0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.486+0000 7f65b549d700 1 -- 172.21.15.83:0/3438291775 shutdown_connections 2023-11-26T07:11:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.486+0000 7f65b549d700 1 --2- 172.21.15.83:0/3438291775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65b015e1e0 0x7f65b00ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.486+0000 7f65b549d700 1 --2- 172.21.15.83:0/3438291775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65b00d0f80 0x7f65b00cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.486+0000 7f65b549d700 1 --2- 172.21.15.83:0/3438291775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b00d3a00 0x7f65b01f2e30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.486+0000 7f65b549d700 1 -- 172.21.15.83:0/3438291775 >> 172.21.15.83:0/3438291775 conn(0x7f65b00b5b50 msgr2=0x7f65b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.486+0000 7f65b549d700 1 -- 172.21.15.83:0/3438291775 shutdown_connections 2023-11-26T07:11:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:05.486+0000 7f65b549d700 1 -- 172.21.15.83:0/3438291775 wait complete. 2023-11-26T07:11:05.501 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-26T07:11:05.501 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-26T07:11:05.794 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:05.792+0000 7f3809420200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:11:05.867 INFO:tasks.ceph.osd.0.smithi083.stderr:2023-11-26T07:11:05.865+0000 7fce55da5640 -1 Falling back to public interface 2023-11-26T07:11:05.868 INFO:tasks.ceph.osd.5.smithi203.stderr:2023-11-26T07:11:05.866+0000 7fe10e655640 -1 Falling back to public interface 2023-11-26T07:11:05.870 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:05.868+0000 7f3809420200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-26T07:11:05.882 INFO:tasks.ceph.osd.4.smithi203.stderr:2023-11-26T07:11:05.879+0000 7fb2bfdf7640 -1 Falling back to public interface 2023-11-26T07:11:05.885 INFO:tasks.ceph.osd.1.smithi083.stderr:2023-11-26T07:11:05.883+0000 7f52de267640 -1 Falling back to public interface 2023-11-26T07:11:05.892 INFO:tasks.ceph.osd.7.smithi203.stderr:2023-11-26T07:11:05.890+0000 7ff82ee45640 -1 Falling back to public interface 2023-11-26T07:11:05.897 INFO:tasks.ceph.osd.3.smithi083.stderr:2023-11-26T07:11:05.895+0000 7fb11cf78640 -1 Falling back to public interface 2023-11-26T07:11:05.898 INFO:tasks.ceph.osd.2.smithi083.stderr:2023-11-26T07:11:05.896+0000 7fe9abe65640 -1 Falling back to public interface 2023-11-26T07:11:05.905 INFO:tasks.ceph.osd.6.smithi203.stderr:2023-11-26T07:11:05.902+0000 7fa50f8ce640 -1 Falling back to public interface 2023-11-26T07:11:05.953 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:05.951+0000 7f317dfe1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:11:06.016 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:06.014+0000 7f3809420200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:11:06.031 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:06.029+0000 7f317dfe1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-26T07:11:06.175 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:06.173+0000 7f317dfe1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:11:06.253 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:06.250+0000 7f3809420200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-26T07:11:06.307 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:06.414 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:06.412+0000 7f317dfe1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-26T07:11:06.424 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.421+0000 7fc5b033d700 1 Processor -- start 2023-11-26T07:11:06.424 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.421+0000 7fc5b033d700 1 -- start start 2023-11-26T07:11:06.427 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5b033d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5a80aea10 0x7fc5a80d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:06.427 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5b033d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5a80d2040 0x7fc5a80b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5b033d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a80aee70 con 0x7fc5a815bf00 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5b033d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a80d2490 con 0x7fc5a80aea10 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5b033d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a80cc590 con 0x7fc5a80d2040 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ad8d8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5a80aea10 0x7fc5a80d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ad8d8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5a80aea10 0x7fc5a80d1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33490/0 (socket says 172.21.15.83:33490) 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ad8d8700 1 -- 172.21.15.83:0/256526512 learned_addr learned my addr 172.21.15.83:0/256526512 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ae8da700 1 --2- 172.21.15.83:0/256526512 >> v2:172.21.15.203:3300/0 conn(0x7fc5a80d2040 0x7fc5a80b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ad8d8700 1 -- 172.21.15.83:0/256526512 >> v2:172.21.15.203:3300/0 conn(0x7fc5a80d2040 msgr2=0x7fc5a80b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ad8d8700 1 --2- 172.21.15.83:0/256526512 >> v2:172.21.15.203:3300/0 conn(0x7fc5a80d2040 0x7fc5a80b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ad8d8700 1 -- 172.21.15.83:0/256526512 >> v1:172.21.15.203:6789/0 conn(0x7fc5a815bf00 legacy=0x7fc5a80d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ad8d8700 1 -- 172.21.15.83:0/256526512 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a815c2e0 con 0x7fc5a80aea10 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ad8d8700 1 --2- 172.21.15.83:0/256526512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a80aea10 0x7fc5a80d1b00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80d19f0 tx=0x7fc598009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2d84a290082f16 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:06.428 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ad0d7700 1 -- 172.21.15.83:0/256526512 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598011830 con 0x7fc5a80aea10 2023-11-26T07:11:06.429 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ad0d7700 1 -- 172.21.15.83:0/256526512 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598011990 con 0x7fc5a80aea10 2023-11-26T07:11:06.429 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.423+0000 7fc5ad0d7700 1 -- 172.21.15.83:0/256526512 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598011d20 con 0x7fc5a80aea10 2023-11-26T07:11:06.429 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.424+0000 7fc5b033d700 1 -- 172.21.15.83:0/256526512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a80aea10 msgr2=0x7fc5a80d1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:06.429 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.424+0000 7fc5b033d700 1 --2- 172.21.15.83:0/256526512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a80aea10 0x7fc5a80d1b00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80d19f0 tx=0x7fc598009bd0 comp rx=0 tx=0).stop 2023-11-26T07:11:06.429 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.424+0000 7fc5b033d700 1 -- 172.21.15.83:0/256526512 shutdown_connections 2023-11-26T07:11:06.429 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.424+0000 7fc5b033d700 1 --2- 172.21.15.83:0/256526512 >> v2:172.21.15.203:3300/0 conn(0x7fc5a80d2040 0x7fc5a80b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:06.429 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.424+0000 7fc5b033d700 1 --2- 172.21.15.83:0/256526512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a80aea10 0x7fc5a80d1b00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:06.429 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.424+0000 7fc5b033d700 1 -- 172.21.15.83:0/256526512 >> 172.21.15.83:0/256526512 conn(0x7fc5a809aa00 msgr2=0x7fc5a809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:06.429 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.424+0000 7fc5b033d700 1 -- 172.21.15.83:0/256526512 shutdown_connections 2023-11-26T07:11:06.429 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.424+0000 7fc5b033d700 1 -- 172.21.15.83:0/256526512 wait complete. 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.426+0000 7fc5b033d700 1 Processor -- start 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.426+0000 7fc5b033d700 1 -- start start 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.427+0000 7fc5b033d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5a80aea10 0x7fc5a815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.427+0000 7fc5b033d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a80d2040 0x7fc5a81f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.427+0000 7fc5b033d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5a815bf00 0x7fc5a8159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.427+0000 7fc5b033d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a815d8d0 con 0x7fc5a80d2040 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.427+0000 7fc5b033d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80b4880 con 0x7fc5a80aea10 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.427+0000 7fc5b033d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a815d320 con 0x7fc5a815bf00 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.427+0000 7fc5ad8d8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a80d2040 0x7fc5a81f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.427+0000 7fc5ad8d8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a80d2040 0x7fc5a81f0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33496/0 (socket says 172.21.15.83:33496) 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.427+0000 7fc5ad8d8700 1 -- 172.21.15.83:0/2663131871 learned_addr learned my addr 172.21.15.83:0/2663131871 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:06.432 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.427+0000 7fc5ae0d9700 1 --2- 172.21.15.83:0/2663131871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5a80aea10 0x7fc5a815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.427+0000 7fc5ae8da700 1 --2- 172.21.15.83:0/2663131871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5a815bf00 0x7fc5a8159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc5ad8d8700 1 -- 172.21.15.83:0/2663131871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5a815bf00 msgr2=0x7fc5a8159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc5ad8d8700 1 --2- 172.21.15.83:0/2663131871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5a815bf00 0x7fc5a8159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc5ad8d8700 1 -- 172.21.15.83:0/2663131871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5a80aea10 msgr2=0x7fc5a815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc5ad8d8700 1 --2- 172.21.15.83:0/2663131871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5a80aea10 0x7fc5a815cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc5ad8d8700 1 -- 172.21.15.83:0/2663131871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc598009550 con 0x7fc5a80d2040 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc5ad8d8700 1 --2- 172.21.15.83:0/2663131871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a80d2040 0x7fc5a81f0d40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc598009c10 tx=0x7fc598004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc59effd700 1 -- 172.21.15.83:0/2663131871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598004510 con 0x7fc5a80d2040 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc59effd700 1 -- 172.21.15.83:0/2663131871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59801e070 con 0x7fc5a80d2040 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc5b033d700 1 -- 172.21.15.83:0/2663131871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a815d5f0 con 0x7fc5a80d2040 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc59effd700 1 -- 172.21.15.83:0/2663131871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5980038e0 con 0x7fc5a80d2040 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc5b033d700 1 -- 172.21.15.83:0/2663131871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a80b4db0 con 0x7fc5a80d2040 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc59effd700 1 -- 172.21.15.83:0/2663131871 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc598003d40 con 0x7fc5a80d2040 2023-11-26T07:11:06.433 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.428+0000 7fc59effd700 1 -- 172.21.15.83:0/2663131871 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fc59802dc20 con 0x7fc5a80d2040 2023-11-26T07:11:06.434 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.429+0000 7fc5b033d700 1 -- 172.21.15.83:0/2663131871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a8068fb0 con 0x7fc5a80d2040 2023-11-26T07:11:06.434 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.431+0000 7fc59effd700 1 -- 172.21.15.83:0/2663131871 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc598020070 con 0x7fc5a80d2040 2023-11-26T07:11:06.469 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:06.467+0000 7f3809420200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-26T07:11:06.505 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.503+0000 7fc5b033d700 1 -- 172.21.15.83:0/2663131871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5a80c7510 con 0x7fc5a80d2040 2023-11-26T07:11:06.506 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.503+0000 7fc59effd700 1 -- 172.21.15.83:0/2663131871 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fc598015020 con 0x7fc5a80d2040 2023-11-26T07:11:06.506 INFO:teuthology.misc.health.smithi083.stdout: 2023-11-26T07:11:06.506 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":10,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:05.232194+0000","last_up_change":"0.000000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:06.507 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.504+0000 7fc5b033d700 1 -- 172.21.15.83:0/2663131871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a80d2040 msgr2=0x7fc5a81f0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:06.507 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.504+0000 7fc5b033d700 1 --2- 172.21.15.83:0/2663131871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a80d2040 0x7fc5a81f0d40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc598009c10 tx=0x7fc598004750 comp rx=0 tx=0).stop 2023-11-26T07:11:06.507 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.504+0000 7fc5b033d700 1 -- 172.21.15.83:0/2663131871 shutdown_connections 2023-11-26T07:11:06.507 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.504+0000 7fc5b033d700 1 --2- 172.21.15.83:0/2663131871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5a815bf00 0x7fc5a8159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:06.507 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.504+0000 7fc5b033d700 1 --2- 172.21.15.83:0/2663131871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5a80aea10 0x7fc5a815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:06.507 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.504+0000 7fc5b033d700 1 --2- 172.21.15.83:0/2663131871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a80d2040 0x7fc5a81f0d40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:06.508 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.504+0000 7fc5b033d700 1 -- 172.21.15.83:0/2663131871 >> 172.21.15.83:0/2663131871 conn(0x7fc5a809aa00 msgr2=0x7fc5a80cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:06.508 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.504+0000 7fc5b033d700 1 -- 172.21.15.83:0/2663131871 shutdown_connections 2023-11-26T07:11:06.508 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:06.504+0000 7fc5b033d700 1 -- 172.21.15.83:0/2663131871 wait complete. 2023-11-26T07:11:06.517 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-26T07:11:06.602 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:06.599+0000 7f3809420200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-26T07:11:06.630 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:06.627+0000 7f317dfe1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-26T07:11:06.767 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:06.765+0000 7f317dfe1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-26T07:11:06.819 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:06.817+0000 7f3809420200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-26T07:11:06.973 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:06.971+0000 7f3809420200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-26T07:11:06.988 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:06.985+0000 7f317dfe1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-26T07:11:07.089 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:07.087+0000 7f3809420200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-26T07:11:07.143 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:07.141+0000 7f317dfe1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-26T07:11:07.258 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:07.256+0000 7f317dfe1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-26T07:11:07.410 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:07.407+0000 7f3809420200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-26T07:11:07.508 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:07.506+0000 7f3809420200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-26T07:11:07.586 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:07.584+0000 7f317dfe1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-26T07:11:07.685 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:07.683+0000 7f317dfe1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-26T07:11:07.696 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:07.694+0000 7f3809420200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-26T07:11:07.874 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:07.872+0000 7f317dfe1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-26T07:11:08.287 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:08.284+0000 7f3809420200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-26T07:11:08.490 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:11:08.488+0000 7f317dfe1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-26T07:11:09.309 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:09.307+0000 7f37c1375700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-26T07:11:09.310 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:11:09.308+0000 7f37c1375700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-26T07:11:09.383 INFO:tasks.ceph.osd.0.smithi083.stderr:2023-11-26T07:11:09.381+0000 7fce55da5640 -1 osd.0 0 log_to_monitors true 2023-11-26T07:11:09.414 INFO:tasks.ceph.osd.5.smithi203.stderr:2023-11-26T07:11:09.412+0000 7fe10e655640 -1 osd.5 0 log_to_monitors true 2023-11-26T07:11:09.421 INFO:tasks.ceph.osd.4.smithi203.stderr:2023-11-26T07:11:09.419+0000 7fb2bfdf7640 -1 osd.4 0 log_to_monitors true 2023-11-26T07:11:09.426 INFO:tasks.ceph.osd.1.smithi083.stderr:2023-11-26T07:11:09.424+0000 7f52de267640 -1 osd.1 0 log_to_monitors true 2023-11-26T07:11:09.445 INFO:tasks.ceph.osd.3.smithi083.stderr:2023-11-26T07:11:09.443+0000 7fb11cf78640 -1 osd.3 0 log_to_monitors true 2023-11-26T07:11:09.445 INFO:tasks.ceph.osd.2.smithi083.stderr:2023-11-26T07:11:09.443+0000 7fe9abe65640 -1 osd.2 0 log_to_monitors true 2023-11-26T07:11:09.464 INFO:tasks.ceph.osd.7.smithi203.stderr:2023-11-26T07:11:09.461+0000 7ff82ee45640 -1 osd.7 0 log_to_monitors true 2023-11-26T07:11:09.480 INFO:tasks.ceph.osd.6.smithi203.stderr:2023-11-26T07:11:09.478+0000 7fa50f8ce640 -1 osd.6 0 log_to_monitors true 2023-11-26T07:11:13.323 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:13.339 INFO:tasks.ceph.osd.6.smithi203.stderr:2023-11-26T07:11:13.336+0000 7fa503485700 -1 osd.6 0 waiting for initial osdmap 2023-11-26T07:11:13.343 INFO:tasks.ceph.osd.6.smithi203.stderr:2023-11-26T07:11:13.341+0000 7fa4fb865700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:11:13.350 INFO:tasks.ceph.osd.0.smithi083.stderr:2023-11-26T07:11:13.348+0000 7fce4995c700 -1 osd.0 0 waiting for initial osdmap 2023-11-26T07:11:13.354 INFO:tasks.ceph.osd.1.smithi083.stderr:2023-11-26T07:11:13.349+0000 7f52d1e1e700 -1 osd.1 0 waiting for initial osdmap 2023-11-26T07:11:13.362 INFO:tasks.ceph.osd.4.smithi203.stderr:2023-11-26T07:11:13.360+0000 7fb2b39ae700 -1 osd.4 0 waiting for initial osdmap 2023-11-26T07:11:13.366 INFO:tasks.ceph.osd.1.smithi083.stderr:2023-11-26T07:11:13.364+0000 7f52ca1fe700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:11:13.366 INFO:tasks.ceph.osd.0.smithi083.stderr:2023-11-26T07:11:13.364+0000 7fce41d3c700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:11:13.373 INFO:tasks.ceph.osd.4.smithi203.stderr:2023-11-26T07:11:13.371+0000 7fb2abd8e700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:11:13.374 INFO:tasks.ceph.osd.3.smithi083.stderr:2023-11-26T07:11:13.372+0000 7fb110b2f700 -1 osd.3 0 waiting for initial osdmap 2023-11-26T07:11:13.375 INFO:tasks.ceph.osd.2.smithi083.stderr:2023-11-26T07:11:13.373+0000 7fe99fa1c700 -1 osd.2 0 waiting for initial osdmap 2023-11-26T07:11:13.379 INFO:tasks.ceph.osd.7.smithi203.stderr:2023-11-26T07:11:13.377+0000 7ff8229fc700 -1 osd.7 0 waiting for initial osdmap 2023-11-26T07:11:13.385 INFO:tasks.ceph.osd.3.smithi083.stderr:2023-11-26T07:11:13.383+0000 7fb108f0f700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:11:13.385 INFO:tasks.ceph.osd.5.smithi203.stderr:2023-11-26T07:11:13.383+0000 7fe10220c700 -1 osd.5 0 waiting for initial osdmap 2023-11-26T07:11:13.388 INFO:tasks.ceph.osd.2.smithi083.stderr:2023-11-26T07:11:13.386+0000 7fe997dfc700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:11:13.400 INFO:tasks.ceph.osd.7.smithi203.stderr:2023-11-26T07:11:13.398+0000 7ff81addc700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:11:13.405 INFO:tasks.ceph.osd.5.smithi203.stderr:2023-11-26T07:11:13.403+0000 7fe0fa5ec700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:11:13.457 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.454+0000 7f2d8f5a9700 1 Processor -- start 2023-11-26T07:11:13.457 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.455+0000 7f2d8f5a9700 1 -- start start 2023-11-26T07:11:13.459 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.455+0000 7f2d8f5a9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d880d02c0 0x7f2d880d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:13.460 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.455+0000 7f2d8f5a9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d880d0740 0x7f2d880d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:13.460 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.455+0000 7f2d8f5a9700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d880d0c40 con 0x7f2d8815bf60 2023-11-26T07:11:13.460 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.455+0000 7f2d8f5a9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8815c340 con 0x7f2d880d02c0 2023-11-26T07:11:13.460 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.455+0000 7f2d8f5a9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8806f220 con 0x7f2d880d0740 2023-11-26T07:11:13.460 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.455+0000 7f2d8cb44700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d880d02c0 0x7f2d880d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:13.460 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.455+0000 7f2d8cb44700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d880d02c0 0x7f2d880d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33600/0 (socket says 172.21.15.83:33600) 2023-11-26T07:11:13.460 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.455+0000 7f2d8d345700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2d8815bf60 0x7f2d880cc710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53638/0 (socket says 172.21.15.83:53638) 2023-11-26T07:11:13.460 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.455+0000 7f2d8cb44700 1 -- 172.21.15.83:0/547023496 learned_addr learned my addr 172.21.15.83:0/547023496 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:13.460 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d8db46700 1 --2- 172.21.15.83:0/547023496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d880d0740 0x7f2d880d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:13.460 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d8cb44700 1 -- 172.21.15.83:0/547023496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d880d0740 msgr2=0x7f2d880d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:13.460 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d8cb44700 1 --2- 172.21.15.83:0/547023496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d880d0740 0x7f2d880d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d8cb44700 1 -- 172.21.15.83:0/547023496 >> v1:172.21.15.83:6789/0 conn(0x7f2d8815bf60 legacy=0x7f2d880cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d8cb44700 1 -- 172.21.15.83:0/547023496 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d880d25d0 con 0x7f2d880d02c0 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d7ffff700 1 -- 172.21.15.83:0/547023496 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507223751 0 0) 0x7f2d880d0c40 con 0x7f2d8815bf60 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d8cb44700 1 --2- 172.21.15.83:0/547023496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d880d02c0 0x7f2d880d1b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2d780098f0 tx=0x7f2d78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb87f728a8f8111 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d7ffff700 1 -- 172.21.15.83:0/547023496 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800b8a0 con 0x7f2d880d02c0 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d7ffff700 1 -- 172.21.15.83:0/547023496 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7800ba00 con 0x7f2d880d02c0 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d7ffff700 1 -- 172.21.15.83:0/547023496 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800bd90 con 0x7f2d880d02c0 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/547023496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d880d02c0 msgr2=0x7f2d880d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.456+0000 7f2d8f5a9700 1 --2- 172.21.15.83:0/547023496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d880d02c0 0x7f2d880d1b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2d780098f0 tx=0x7f2d78009c00 comp rx=0 tx=0).stop 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.457+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/547023496 shutdown_connections 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.457+0000 7f2d8f5a9700 1 --2- 172.21.15.83:0/547023496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d880d0740 0x7f2d880d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:13.461 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.457+0000 7f2d8f5a9700 1 --2- 172.21.15.83:0/547023496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d880d02c0 0x7f2d880d1b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:13.462 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.457+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/547023496 >> 172.21.15.83:0/547023496 conn(0x7f2d8809aa00 msgr2=0x7f2d8809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:13.462 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.457+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/547023496 shutdown_connections 2023-11-26T07:11:13.462 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.457+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/547023496 wait complete. 2023-11-26T07:11:13.464 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.458+0000 7f2d8f5a9700 1 Processor -- start 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.458+0000 7f2d8f5a9700 1 -- start start 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8f5a9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d880d02c0 0x7f2d8815f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8f5a9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d880d0740 0x7f2d8815c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8f5a9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8815bf60 0x7f2d88153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8f5a9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d880b6530 con 0x7f2d880d02c0 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8f5a9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8815fa10 con 0x7f2d8815bf60 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8f5a9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d880b5b90 con 0x7f2d880d0740 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8d345700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d880d02c0 0x7f2d8815f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8d345700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d880d02c0 0x7f2d8815f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33604/0 (socket says 172.21.15.83:33604) 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8d345700 1 -- 172.21.15.83:0/3075460356 learned_addr learned my addr 172.21.15.83:0/3075460356 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8db46700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8815bf60 0x7f2d88153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:13.465 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8cb44700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d880d0740 0x7f2d8815c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8d345700 1 -- 172.21.15.83:0/3075460356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d880d0740 msgr2=0x7f2d8815c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8d345700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d880d0740 0x7f2d8815c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8d345700 1 -- 172.21.15.83:0/3075460356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8815bf60 msgr2=0x7f2d88153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8d345700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8815bf60 0x7f2d88153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8d345700 1 -- 172.21.15.83:0/3075460356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78009550 con 0x7f2d880d02c0 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8db46700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8815bf60 0x7f2d88153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8cb44700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d880d0740 0x7f2d8815c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8d345700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d880d02c0 0x7f2d8815f400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000c550 tx=0x7f2d8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d7dffb700 1 -- 172.21.15.83:0/3075460356 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80004c80 con 0x7f2d880d02c0 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d7dffb700 1 -- 172.21.15.83:0/3075460356 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d80004e00 con 0x7f2d880d02c0 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d7dffb700 1 -- 172.21.15.83:0/3075460356 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80007310 con 0x7f2d880d02c0 2023-11-26T07:11:13.466 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.459+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/3075460356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d880b5e70 con 0x7f2d880d02c0 2023-11-26T07:11:13.467 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.460+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/3075460356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8815b220 con 0x7f2d880d02c0 2023-11-26T07:11:13.467 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.460+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/3075460356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d880d0130 con 0x7f2d880d02c0 2023-11-26T07:11:13.467 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.461+0000 7f2d7dffb700 1 -- 172.21.15.83:0/3075460356 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f2d80007470 con 0x7f2d880d02c0 2023-11-26T07:11:13.467 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.461+0000 7f2d7dffb700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2d6c06ae40 0x7f2d6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:13.467 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.461+0000 7f2d8cb44700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2d6c06ae40 0x7f2d6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:13.467 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.461+0000 7f2d7dffb700 1 -- 172.21.15.83:0/3075460356 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f2d80088410 con 0x7f2d880d02c0 2023-11-26T07:11:13.467 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.462+0000 7f2d8cb44700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2d6c06ae40 0x7f2d6c06d300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2d8815c9c0 tx=0x7f2d78008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:13.468 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.466+0000 7f2d7dffb700 1 -- 172.21.15.83:0/3075460356 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d800543c0 con 0x7f2d880d02c0 2023-11-26T07:11:13.661 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.659+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/3075460356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d88068fb0 con 0x7f2d880d02c0 2023-11-26T07:11:13.662 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.659+0000 7f2d7dffb700 1 -- 172.21.15.83:0/3075460356 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2d800579e0 con 0x7f2d880d02c0 2023-11-26T07:11:13.662 INFO:teuthology.misc.health.smithi083.stdout: 2023-11-26T07:11:13.662 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":12,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:11.305716+0000","last_up_change":"0.000000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:13.664 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/3075460356 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2d6c06ae40 msgr2=0x7f2d6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:13.664 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2d6c06ae40 0x7f2d6c06d300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2d8815c9c0 tx=0x7f2d78008030 comp rx=0 tx=0).stop 2023-11-26T07:11:13.664 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/3075460356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d880d02c0 msgr2=0x7f2d8815f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:13.664 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d880d02c0 0x7f2d8815f400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000c550 tx=0x7f2d8000c910 comp rx=0 tx=0).stop 2023-11-26T07:11:13.664 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/3075460356 shutdown_connections 2023-11-26T07:11:13.664 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d8815bf60 0x7f2d88153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:13.665 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d880d02c0 0x7f2d8815f400 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:13.665 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2d6c06ae40 0x7f2d6c06d300 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:13.665 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 --2- 172.21.15.83:0/3075460356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d880d0740 0x7f2d8815c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:13.665 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/3075460356 >> 172.21.15.83:0/3075460356 conn(0x7f2d8809aa00 msgr2=0x7f2d880af970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:13.665 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/3075460356 shutdown_connections 2023-11-26T07:11:13.665 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:13.661+0000 7f2d8f5a9700 1 -- 172.21.15.83:0/3075460356 wait complete. 2023-11-26T07:11:13.675 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-26T07:11:20.480 DEBUG:teuthology.orchestra.run.smithi083:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:20.602 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.600+0000 7f1b7ee24700 1 Processor -- start 2023-11-26T07:11:20.603 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.600+0000 7f1b7ee24700 1 -- start start 2023-11-26T07:11:20.605 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7ee24700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1b7806c000 0x7f1b7806c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.605 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7ee24700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1b7815c990 0x7f1b7815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.605 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7ee24700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b78073f90 0x7f1b780743e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.605 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7ee24700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b78160bb0 con 0x7f1b7806c000 2023-11-26T07:11:20.605 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7ee24700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b7806c920 con 0x7f1b7815c990 2023-11-26T07:11:20.605 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7ee24700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7815fdf0 con 0x7f1b78073f90 2023-11-26T07:11:20.605 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7cbc0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1b7806c000 0x7f1b7806c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7cbc0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1b7806c000 0x7f1b7806c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33838/0 (socket says 172.21.15.83:33838) 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7cbc0700 1 -- 172.21.15.83:0/4133945805 learned_addr learned my addr 172.21.15.83:0/4133945805 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b77fff700 1 --2- 172.21.15.83:0/4133945805 >> v2:172.21.15.203:3300/0 conn(0x7f1b7815c990 0x7f1b7815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7d3c1700 1 --2- 172.21.15.83:0/4133945805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b78073f90 0x7f1b780743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7cbc0700 1 -- 172.21.15.83:0/4133945805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b78073f90 msgr2=0x7f1b780743e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7cbc0700 1 --2- 172.21.15.83:0/4133945805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b78073f90 0x7f1b780743e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7cbc0700 1 -- 172.21.15.83:0/4133945805 >> v2:172.21.15.203:3300/0 conn(0x7f1b7815c990 msgr2=0x7f1b7815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7cbc0700 1 --2- 172.21.15.83:0/4133945805 >> v2:172.21.15.203:3300/0 conn(0x7f1b7815c990 0x7f1b7815ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7cbc0700 1 -- 172.21.15.83:0/4133945805 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b781600d0 con 0x7f1b7806c000 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.601+0000 7f1b7cbc0700 1 --2- 172.21.15.83:0/4133945805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7806c000 0x7f1b7806c3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b780ce6d0 tx=0x7f1b6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2992e64832d8ef server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b777fe700 1 -- 172.21.15.83:0/4133945805 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00b870 con 0x7f1b7806c000 2023-11-26T07:11:20.606 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b777fe700 1 -- 172.21.15.83:0/4133945805 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6c00b9d0 con 0x7f1b7806c000 2023-11-26T07:11:20.607 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b777fe700 1 -- 172.21.15.83:0/4133945805 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00bd60 con 0x7f1b7806c000 2023-11-26T07:11:20.607 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b7ee24700 1 -- 172.21.15.83:0/4133945805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7806c000 msgr2=0x7f1b7806c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:20.607 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b7ee24700 1 --2- 172.21.15.83:0/4133945805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7806c000 0x7f1b7806c3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b780ce6d0 tx=0x7f1b6c009bd0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.607 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b7ee24700 1 -- 172.21.15.83:0/4133945805 shutdown_connections 2023-11-26T07:11:20.607 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b7ee24700 1 --2- 172.21.15.83:0/4133945805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b78073f90 0x7f1b780743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.607 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b7ee24700 1 --2- 172.21.15.83:0/4133945805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7806c000 0x7f1b7806c3e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.607 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b7ee24700 1 --2- 172.21.15.83:0/4133945805 >> v2:172.21.15.203:3300/0 conn(0x7f1b7815c990 0x7f1b7815ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.607 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b7ee24700 1 -- 172.21.15.83:0/4133945805 >> 172.21.15.83:0/4133945805 conn(0x7f1b78071ae0 msgr2=0x7f1b78071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:20.607 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b7ee24700 1 -- 172.21.15.83:0/4133945805 shutdown_connections 2023-11-26T07:11:20.607 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.602+0000 7f1b7ee24700 1 -- 172.21.15.83:0/4133945805 wait complete. 2023-11-26T07:11:20.610 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.603+0000 7f1b7ee24700 1 Processor -- start 2023-11-26T07:11:20.610 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.603+0000 7f1b7ee24700 1 -- start start 2023-11-26T07:11:20.610 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.604+0000 7f1b7ee24700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b7806c000 0x7f1b780affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.610 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.604+0000 7f1b7ee24700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b78073f90 0x7f1b78130df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.610 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.604+0000 7f1b7ee24700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7815c990 0x7f1b78158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.610 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.604+0000 7f1b7ee24700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b78131330 con 0x7f1b7815c990 2023-11-26T07:11:20.610 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.604+0000 7f1b7ee24700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7812e6d0 con 0x7f1b7806c000 2023-11-26T07:11:20.610 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.604+0000 7f1b7ee24700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7812e9b0 con 0x7f1b78073f90 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.604+0000 7f1b7d3c1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7815c990 0x7f1b78158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.604+0000 7f1b7d3c1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7815c990 0x7f1b78158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33848/0 (socket says 172.21.15.83:33848) 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b7d3c1700 1 -- 172.21.15.83:0/3986947569 learned_addr learned my addr 172.21.15.83:0/3986947569 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b77fff700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b78073f90 0x7f1b78130df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b7cbc0700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b7806c000 0x7f1b780affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b7d3c1700 1 -- 172.21.15.83:0/3986947569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b78073f90 msgr2=0x7f1b78130df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b7d3c1700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b78073f90 0x7f1b78130df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b7d3c1700 1 -- 172.21.15.83:0/3986947569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b7806c000 msgr2=0x7f1b780affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b7d3c1700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b7806c000 0x7f1b780affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b7d3c1700 1 -- 172.21.15.83:0/3986947569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c009550 con 0x7f1b7815c990 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b7d3c1700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7815c990 0x7f1b78158dd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b70007e60 tx=0x7f1b7000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b757fa700 1 -- 172.21.15.83:0/3986947569 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7000ede0 con 0x7f1b7815c990 2023-11-26T07:11:20.611 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b757fa700 1 -- 172.21.15.83:0/3986947569 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b70014420 con 0x7f1b7815c990 2023-11-26T07:11:20.612 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b7ee24700 1 -- 172.21.15.83:0/3986947569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7812ec90 con 0x7f1b7815c990 2023-11-26T07:11:20.612 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b757fa700 1 -- 172.21.15.83:0/3986947569 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b700147d0 con 0x7f1b7815c990 2023-11-26T07:11:20.612 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.605+0000 7f1b7ee24700 1 -- 172.21.15.83:0/3986947569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7812f250 con 0x7f1b7815c990 2023-11-26T07:11:20.612 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.606+0000 7f1b7ee24700 1 -- 172.21.15.83:0/3986947569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b78068fb0 con 0x7f1b7815c990 2023-11-26T07:11:20.612 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.607+0000 7f1b757fa700 1 -- 172.21.15.83:0/3986947569 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b70014930 con 0x7f1b7815c990 2023-11-26T07:11:20.612 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.607+0000 7f1b757fa700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b5c06aed0 0x7f1b5c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.612 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.607+0000 7f1b757fa700 1 -- 172.21.15.83:0/3986947569 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1b70089a80 con 0x7f1b7815c990 2023-11-26T07:11:20.612 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.607+0000 7f1b7cbc0700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b5c06aed0 0x7f1b5c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.612 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.607+0000 7f1b7cbc0700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b5c06aed0 0x7f1b5c06d390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c000c00 tx=0x7f1b6c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:20.614 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.611+0000 7f1b757fa700 1 -- 172.21.15.83:0/3986947569 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b70054e00 con 0x7f1b7815c990 2023-11-26T07:11:20.803 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.801+0000 7f1b7ee24700 1 -- 172.21.15.83:0/3986947569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b78158bb0 con 0x7f1b7815c990 2023-11-26T07:11:20.803 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.801+0000 7f1b757fa700 1 -- 172.21.15.83:0/3986947569 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7f1b70058420 con 0x7f1b7815c990 2023-11-26T07:11:20.804 INFO:teuthology.misc.health.smithi083.stdout: 2023-11-26T07:11:20.804 INFO:teuthology.misc.health.smithi083.stdout:{"epoch":17,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:18.378934+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:20.806 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.803+0000 7f1b7ee24700 1 -- 172.21.15.83:0/3986947569 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b5c06aed0 msgr2=0x7f1b5c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:20.806 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.803+0000 7f1b7ee24700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b5c06aed0 0x7f1b5c06d390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c000c00 tx=0x7f1b6c008030 comp rx=0 tx=0).stop 2023-11-26T07:11:20.806 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.803+0000 7f1b7ee24700 1 -- 172.21.15.83:0/3986947569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7815c990 msgr2=0x7f1b78158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:20.807 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.803+0000 7f1b7ee24700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7815c990 0x7f1b78158dd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b70007e60 tx=0x7f1b7000e3c0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.807 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.803+0000 7f1b7ee24700 1 -- 172.21.15.83:0/3986947569 shutdown_connections 2023-11-26T07:11:20.807 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.803+0000 7f1b7ee24700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b7815c990 0x7f1b78158dd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.807 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.803+0000 7f1b7ee24700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b7806c000 0x7f1b780affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.807 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.803+0000 7f1b7ee24700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b5c06aed0 0x7f1b5c06d390 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.807 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.803+0000 7f1b7ee24700 1 --2- 172.21.15.83:0/3986947569 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b78073f90 0x7f1b78130df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.807 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.803+0000 7f1b7ee24700 1 -- 172.21.15.83:0/3986947569 >> 172.21.15.83:0/3986947569 conn(0x7f1b78071ae0 msgr2=0x7f1b78141340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:20.807 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.804+0000 7f1b7ee24700 1 -- 172.21.15.83:0/3986947569 shutdown_connections 2023-11-26T07:11:20.807 INFO:teuthology.misc.health.smithi083.stderr:2023-11-26T07:11:20.804+0000 7f1b7ee24700 1 -- 172.21.15.83:0/3986947569 wait complete. 2023-11-26T07:11:20.817 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-26T07:11:20.817 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-26T07:11:20.818 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:11:20.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:11:20.820 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:11:20.820 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:11:20.820 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:11:20.822 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:11:20.822 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:11:20.823 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:11:20.825 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:11:20.826 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:11:20.826 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:11:20.829 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:11:20.829 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-26T07:11:20.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:20.899 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-26T07:11:20.899 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-26T07:11:20.915 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-11-26T07:11:20.915 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-11-26T07:11:20.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.954+0000 7fdf6477e700 1 Processor -- start 2023-11-26T07:11:20.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.954+0000 7fdf6477e700 1 -- start start 2023-11-26T07:11:20.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf6477e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c06ed40 0x7fdf5c0afa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf6477e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdf5c073f90 0x7fdf5c0743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf6477e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5c074900 con 0x7fdf5c15deb0 2023-11-26T07:11:20.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf6477e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5c0affa0 con 0x7fdf5c073f90 2023-11-26T07:11:20.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf6477e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c06dca0 con 0x7fdf5c06ed40 2023-11-26T07:11:20.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf61d19700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c06ed40 0x7fdf5c0afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf62d1b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdf5c073f90 0x7fdf5c0743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf62d1b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdf5c073f90 0x7fdf5c0743c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:33334/0 (socket says 172.21.15.83:33334) 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf62d1b700 1 -- 172.21.15.83:0/567140677 learned_addr learned my addr 172.21.15.83:0/567140677 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf61d19700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c06ed40 0x7fdf5c0afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44336/0 (socket says 172.21.15.83:44336) 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf62d1b700 1 -- 172.21.15.83:0/567140677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c06ed40 msgr2=0x7fdf5c0afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf62d1b700 1 --2- 172.21.15.83:0/567140677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c06ed40 0x7fdf5c0afa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf62d1b700 1 -- 172.21.15.83:0/567140677 >> v1:172.21.15.203:6789/0 conn(0x7fdf5c15deb0 legacy=0x7fdf5c0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf62d1b700 1 -- 172.21.15.83:0/567140677 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c06df80 con 0x7fdf5c073f90 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.955+0000 7fdf61518700 1 -- 172.21.15.83:0/567140677 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668381835 0 0) 0x7fdf5c074900 con 0x7fdf5c15deb0 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf61d19700 1 --2- 172.21.15.83:0/567140677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c06ed40 0x7fdf5c0afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf62d1b700 1 --2- 172.21.15.83:0/567140677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf5c073f90 0x7fdf5c0743c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009e50 tx=0x7fdf5400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a7d71b979757b01 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf61518700 1 -- 172.21.15.83:0/567140677 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400e070 con 0x7fdf5c073f90 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf61518700 1 -- 172.21.15.83:0/567140677 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5400b910 con 0x7fdf5c073f90 2023-11-26T07:11:20.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf61518700 1 -- 172.21.15.83:0/567140677 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5400bcc0 con 0x7fdf5c073f90 2023-11-26T07:11:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf6477e700 1 -- 172.21.15.83:0/567140677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf5c073f90 msgr2=0x7fdf5c0743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf6477e700 1 --2- 172.21.15.83:0/567140677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf5c073f90 0x7fdf5c0743c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009e50 tx=0x7fdf5400b330 comp rx=0 tx=0).stop 2023-11-26T07:11:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf6477e700 1 -- 172.21.15.83:0/567140677 shutdown_connections 2023-11-26T07:11:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf6477e700 1 --2- 172.21.15.83:0/567140677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf5c073f90 0x7fdf5c0743c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf6477e700 1 --2- 172.21.15.83:0/567140677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c06ed40 0x7fdf5c0afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf6477e700 1 -- 172.21.15.83:0/567140677 >> 172.21.15.83:0/567140677 conn(0x7fdf5c0d0ba0 msgr2=0x7fdf5c0d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf6477e700 1 -- 172.21.15.83:0/567140677 shutdown_connections 2023-11-26T07:11:20.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.956+0000 7fdf6477e700 1 -- 172.21.15.83:0/567140677 wait complete. 2023-11-26T07:11:20.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.958+0000 7fdf6477e700 1 Processor -- start 2023-11-26T07:11:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.958+0000 7fdf6477e700 1 -- start start 2023-11-26T07:11:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.958+0000 7fdf6477e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf5c06ed40 0x7fdf5c15bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.958+0000 7fdf6477e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c073f90 0x7fdf5c15c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.958+0000 7fdf6477e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf5c15deb0 0x7fdf5c148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.958+0000 7fdf6477e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c1f3ca0 con 0x7fdf5c06ed40 2023-11-26T07:11:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.958+0000 7fdf6477e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c149610 con 0x7fdf5c15deb0 2023-11-26T07:11:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.958+0000 7fdf6477e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c1f1750 con 0x7fdf5c073f90 2023-11-26T07:11:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.958+0000 7fdf6251a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf5c06ed40 0x7fdf5c15bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.958+0000 7fdf6251a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf5c06ed40 0x7fdf5c15bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33884/0 (socket says 172.21.15.83:33884) 2023-11-26T07:11:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.958+0000 7fdf6251a700 1 -- 172.21.15.83:0/2521245389 learned_addr learned my addr 172.21.15.83:0/2521245389 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:20.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf61d19700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c073f90 0x7fdf5c15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf62d1b700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf5c15deb0 0x7fdf5c148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf6251a700 1 -- 172.21.15.83:0/2521245389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c073f90 msgr2=0x7fdf5c15c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf6251a700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c073f90 0x7fdf5c15c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf6251a700 1 -- 172.21.15.83:0/2521245389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf5c15deb0 msgr2=0x7fdf5c148640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf6251a700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf5c15deb0 0x7fdf5c148640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf6251a700 1 -- 172.21.15.83:0/2521245389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf580094a0 con 0x7fdf5c06ed40 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf6251a700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf5c06ed40 0x7fdf5c15bec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c750 tx=0x7fdf4c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf52ffd700 1 -- 172.21.15.83:0/2521245389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0054e0 con 0x7fdf5c06ed40 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf6477e700 1 -- 172.21.15.83:0/2521245389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf54009b30 con 0x7fdf5c06ed40 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf6477e700 1 -- 172.21.15.83:0/2521245389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c1f44e0 con 0x7fdf5c06ed40 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf52ffd700 1 -- 172.21.15.83:0/2521245389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c003680 con 0x7fdf5c06ed40 2023-11-26T07:11:20.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.959+0000 7fdf52ffd700 1 -- 172.21.15.83:0/2521245389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0039f0 con 0x7fdf5c06ed40 2023-11-26T07:11:20.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.960+0000 7fdf6477e700 1 -- 172.21.15.83:0/2521245389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf440052e0 con 0x7fdf5c06ed40 2023-11-26T07:11:20.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.960+0000 7fdf52ffd700 1 -- 172.21.15.83:0/2521245389 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf4c003b50 con 0x7fdf5c06ed40 2023-11-26T07:11:20.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.961+0000 7fdf52ffd700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf4006aed0 0x7fdf4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:20.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.961+0000 7fdf61d19700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf4006aed0 0x7fdf4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:20.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.961+0000 7fdf52ffd700 1 -- 172.21.15.83:0/2521245389 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdf4c088540 con 0x7fdf5c06ed40 2023-11-26T07:11:20.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.961+0000 7fdf61d19700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf4006aed0 0x7fdf4006d390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c15a750 tx=0x7fdf58009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:20.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:20.965+0000 7fdf52ffd700 1 -- 172.21.15.83:0/2521245389 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf4c053ab0 con 0x7fdf5c06ed40 2023-11-26T07:11:21.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.157+0000 7fdf6477e700 1 -- 172.21.15.83:0/2521245389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf44005860 con 0x7fdf5c06ed40 2023-11-26T07:11:21.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.158+0000 7fdf52ffd700 1 -- 172.21.15.83:0/2521245389 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7fdf4c017090 con 0x7fdf5c06ed40 2023-11-26T07:11:21.161 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:21.161 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:18.378934+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:21.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 -- 172.21.15.83:0/2521245389 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf4006aed0 msgr2=0x7fdf4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf4006aed0 0x7fdf4006d390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c15a750 tx=0x7fdf58009210 comp rx=0 tx=0).stop 2023-11-26T07:11:21.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 -- 172.21.15.83:0/2521245389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf5c06ed40 msgr2=0x7fdf5c15bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf5c06ed40 0x7fdf5c15bec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c750 tx=0x7fdf4c00ca60 comp rx=0 tx=0).stop 2023-11-26T07:11:21.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 -- 172.21.15.83:0/2521245389 shutdown_connections 2023-11-26T07:11:21.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf5c15deb0 0x7fdf5c148640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf5c06ed40 0x7fdf5c15bec0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf4006aed0 0x7fdf4006d390 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 --2- 172.21.15.83:0/2521245389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf5c073f90 0x7fdf5c15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 -- 172.21.15.83:0/2521245389 >> 172.21.15.83:0/2521245389 conn(0x7fdf5c0d0ba0 msgr2=0x7fdf5c0743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:21.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 -- 172.21.15.83:0/2521245389 shutdown_connections 2023-11-26T07:11:21.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.160+0000 7fdf6477e700 1 -- 172.21.15.83:0/2521245389 wait complete. 2023-11-26T07:11:21.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:11:21.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:11:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.295+0000 7f98e895e700 1 Processor -- start 2023-11-26T07:11:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.295+0000 7f98e895e700 1 -- start start 2023-11-26T07:11:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e895e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e41587f0 0x7f98e406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e895e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f98e40b4370 0x7f98e415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e895e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e406b530 con 0x7f98e415b370 2023-11-26T07:11:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e895e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e40ae5e0 con 0x7f98e40b4370 2023-11-26T07:11:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e895e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e415be30 con 0x7f98e41587f0 2023-11-26T07:11:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e259c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e41587f0 0x7f98e406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e2d9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f98e40b4370 0x7f98e415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e259c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e41587f0 0x7f98e406aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44350/0 (socket says 172.21.15.83:44350) 2023-11-26T07:11:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e259c700 1 -- 172.21.15.83:0/2077489846 learned_addr learned my addr 172.21.15.83:0/2077489846 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e2d9d700 1 -- 172.21.15.83:0/2077489846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e41587f0 msgr2=0x7f98e406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e2d9d700 1 --2- 172.21.15.83:0/2077489846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e41587f0 0x7f98e406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e2d9d700 1 -- 172.21.15.83:0/2077489846 >> v1:172.21.15.203:6789/0 conn(0x7f98e415b370 legacy=0x7f98e40b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e2d9d700 1 -- 172.21.15.83:0/2077489846 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e415c110 con 0x7f98e40b4370 2023-11-26T07:11:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.296+0000 7f98e159a700 1 -- 172.21.15.83:0/2077489846 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509348534 0 0) 0x7f98e406b530 con 0x7f98e415b370 2023-11-26T07:11:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e2d9d700 1 --2- 172.21.15.83:0/2077489846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e40b4370 0x7f98e415b7f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009d90 tx=0x7f98dc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5eb4a0cfcebdd25 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e159a700 1 -- 172.21.15.83:0/2077489846 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc00e070 con 0x7f98e40b4370 2023-11-26T07:11:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e159a700 1 -- 172.21.15.83:0/2077489846 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98dc00bb20 con 0x7f98e40b4370 2023-11-26T07:11:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e159a700 1 -- 172.21.15.83:0/2077489846 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98dc012430 con 0x7f98e40b4370 2023-11-26T07:11:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e895e700 1 -- 172.21.15.83:0/2077489846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e40b4370 msgr2=0x7f98e415b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e895e700 1 --2- 172.21.15.83:0/2077489846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e40b4370 0x7f98e415b7f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009d90 tx=0x7f98dc00b330 comp rx=0 tx=0).stop 2023-11-26T07:11:21.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e895e700 1 -- 172.21.15.83:0/2077489846 shutdown_connections 2023-11-26T07:11:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e895e700 1 --2- 172.21.15.83:0/2077489846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e40b4370 0x7f98e415b7f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e895e700 1 --2- 172.21.15.83:0/2077489846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e41587f0 0x7f98e406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e895e700 1 -- 172.21.15.83:0/2077489846 >> 172.21.15.83:0/2077489846 conn(0x7f98e40d0bf0 msgr2=0x7f98e40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e895e700 1 -- 172.21.15.83:0/2077489846 shutdown_connections 2023-11-26T07:11:21.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.297+0000 7f98e895e700 1 -- 172.21.15.83:0/2077489846 wait complete. 2023-11-26T07:11:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.299+0000 7f98e895e700 1 Processor -- start 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.299+0000 7f98e895e700 1 -- start start 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.299+0000 7f98e895e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e40b4370 0x7f98e409bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.299+0000 7f98e895e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e41587f0 0x7f98e414f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.299+0000 7f98e895e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e415b370 0x7f98e40723d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.299+0000 7f98e895e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e414fc30 con 0x7f98e41587f0 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.299+0000 7f98e895e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e409ba80 con 0x7f98e415b370 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.299+0000 7f98e895e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4155ba0 con 0x7f98e40b4370 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e1d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e41587f0 0x7f98e414f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e1d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e41587f0 0x7f98e414f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33898/0 (socket says 172.21.15.83:33898) 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e1d9b700 1 -- 172.21.15.83:0/379941395 learned_addr learned my addr 172.21.15.83:0/379941395 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e259c700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e40b4370 0x7f98e409bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e2d9d700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e415b370 0x7f98e40723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e1d9b700 1 -- 172.21.15.83:0/379941395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e40b4370 msgr2=0x7f98e409bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e1d9b700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e40b4370 0x7f98e409bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e1d9b700 1 -- 172.21.15.83:0/379941395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e415b370 msgr2=0x7f98e40723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e1d9b700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e415b370 0x7f98e40723d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e1d9b700 1 -- 172.21.15.83:0/379941395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc009a70 con 0x7f98e41587f0 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e2d9d700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e415b370 0x7f98e40723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e259c700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e40b4370 0x7f98e409bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e1d9b700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e41587f0 0x7f98e414f6f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c550 tx=0x7f98d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98d2ffd700 1 -- 172.21.15.83:0/379941395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80054e0 con 0x7f98e41587f0 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98d2ffd700 1 -- 172.21.15.83:0/379941395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d8003680 con 0x7f98e41587f0 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e895e700 1 -- 172.21.15.83:0/379941395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e4072910 con 0x7f98e41587f0 2023-11-26T07:11:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98d2ffd700 1 -- 172.21.15.83:0/379941395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8003a30 con 0x7f98e41587f0 2023-11-26T07:11:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.300+0000 7f98e895e700 1 -- 172.21.15.83:0/379941395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e409b150 con 0x7f98e41587f0 2023-11-26T07:11:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.301+0000 7f98e895e700 1 -- 172.21.15.83:0/379941395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e40ae1c0 con 0x7f98e41587f0 2023-11-26T07:11:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.302+0000 7f98d2ffd700 1 -- 172.21.15.83:0/379941395 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98d8003b90 con 0x7f98e41587f0 2023-11-26T07:11:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.302+0000 7f98d2ffd700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f98c406aed0 0x7f98c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.302+0000 7f98d2ffd700 1 -- 172.21.15.83:0/379941395 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f98d80886a0 con 0x7f98e41587f0 2023-11-26T07:11:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.302+0000 7f98e259c700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f98c406aed0 0x7f98c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.302+0000 7f98e259c700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f98c406aed0 0x7f98c406d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009c20 tx=0x7f98d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:21.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.306+0000 7f98d2ffd700 1 -- 172.21.15.83:0/379941395 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98d8053b30 con 0x7f98e41587f0 2023-11-26T07:11:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.498+0000 7f98e895e700 1 -- 172.21.15.83:0/379941395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98e409bbf0 con 0x7f98e41587f0 2023-11-26T07:11:21.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.499+0000 7f98d2ffd700 1 -- 172.21.15.83:0/379941395 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98d8017020 con 0x7f98e41587f0 2023-11-26T07:11:21.501 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 -- 172.21.15.83:0/379941395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f98c406aed0 msgr2=0x7f98c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f98c406aed0 0x7f98c406d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009c20 tx=0x7f98d4008040 comp rx=0 tx=0).stop 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 -- 172.21.15.83:0/379941395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e41587f0 msgr2=0x7f98e414f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e41587f0 0x7f98e414f6f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c550 tx=0x7f98d800c910 comp rx=0 tx=0).stop 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 -- 172.21.15.83:0/379941395 shutdown_connections 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e415b370 0x7f98e40723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f98c406aed0 0x7f98c406d390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f98e40b4370 0x7f98e409bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 --2- 172.21.15.83:0/379941395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e41587f0 0x7f98e414f6f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 -- 172.21.15.83:0/379941395 >> 172.21.15.83:0/379941395 conn(0x7f98e40d0bf0 msgr2=0x7f98e415c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 -- 172.21.15.83:0/379941395 shutdown_connections 2023-11-26T07:11:21.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.501+0000 7f98e895e700 1 -- 172.21.15.83:0/379941395 wait complete. 2023-11-26T07:11:21.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:11:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.632+0000 7fb98d0f2700 1 Processor -- start 2023-11-26T07:11:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.632+0000 7fb98d0f2700 1 -- start start 2023-11-26T07:11:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.632+0000 7fb98d0f2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9880d2380 0x7fb9881604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.632+0000 7fb98d0f2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9880b22d0 con 0x7fb9880b3370 2023-11-26T07:11:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.632+0000 7fb98d0f2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9880b25b0 con 0x7fb98815bdd0 2023-11-26T07:11:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.632+0000 7fb98d0f2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880b2f50 con 0x7fb9880d2380 2023-11-26T07:11:21.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb98659c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb9880b3370 0x7fb9880d1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53948/0 (socket says 172.21.15.83:53948) 2023-11-26T07:11:21.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb98659c700 1 -- 172.21.15.83:0/907496657 learned_addr learned my addr 172.21.15.83:0/907496657 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:21.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb98759e700 1 --2- 172.21.15.83:0/907496657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9880d2380 0x7fb9881604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:21.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb985d9b700 1 -- 172.21.15.83:0/907496657 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576436275 0 0) 0x7fb9880b22d0 con 0x7fb9880b3370 2023-11-26T07:11:21.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb985d9b700 1 -- 172.21.15.83:0/907496657 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9700035b0 con 0x7fb9880b3370 2023-11-26T07:11:21.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb985d9b700 1 -- 172.21.15.83:0/907496657 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362449316 0 0) 0x7fb9880b25b0 con 0x7fb98815bdd0 2023-11-26T07:11:21.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb985d9b700 1 -- 172.21.15.83:0/907496657 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9880b22d0 con 0x7fb98815bdd0 2023-11-26T07:11:21.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb985d9b700 1 -- 172.21.15.83:0/907496657 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 954454629 0 0) 0x7fb9700035b0 con 0x7fb9880b3370 2023-11-26T07:11:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb98759e700 1 -- 172.21.15.83:0/907496657 >> v1:172.21.15.203:6789/0 conn(0x7fb98815bdd0 legacy=0x7fb9880b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb98759e700 1 -- 172.21.15.83:0/907496657 >> v1:172.21.15.83:6789/0 conn(0x7fb9880b3370 legacy=0x7fb9880d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb98759e700 1 -- 172.21.15.83:0/907496657 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9880b2890 con 0x7fb9880d2380 2023-11-26T07:11:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb98759e700 1 --2- 172.21.15.83:0/907496657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9880d2380 0x7fb9881604e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb980009dc0 tx=0x7fb98000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcce4fa9e66a781f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb985d9b700 1 -- 172.21.15.83:0/907496657 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98000e070 con 0x7fb9880d2380 2023-11-26T07:11:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb985d9b700 1 -- 172.21.15.83:0/907496657 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb980003a20 con 0x7fb9880d2380 2023-11-26T07:11:21.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.633+0000 7fb985d9b700 1 -- 172.21.15.83:0/907496657 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978001670 con 0x7fb9880d2380 2023-11-26T07:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.634+0000 7fb98d0f2700 1 -- 172.21.15.83:0/907496657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9880d2380 msgr2=0x7fb9881604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.634+0000 7fb98d0f2700 1 --2- 172.21.15.83:0/907496657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9880d2380 0x7fb9881604e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb980009dc0 tx=0x7fb98000b330 comp rx=0 tx=0).stop 2023-11-26T07:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.634+0000 7fb98d0f2700 1 -- 172.21.15.83:0/907496657 shutdown_connections 2023-11-26T07:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.634+0000 7fb98d0f2700 1 --2- 172.21.15.83:0/907496657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9880d2380 0x7fb9881604e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.634+0000 7fb98d0f2700 1 -- 172.21.15.83:0/907496657 >> 172.21.15.83:0/907496657 conn(0x7fb9880731a0 msgr2=0x7fb98809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.634+0000 7fb98d0f2700 1 -- 172.21.15.83:0/907496657 shutdown_connections 2023-11-26T07:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.634+0000 7fb98d0f2700 1 -- 172.21.15.83:0/907496657 wait complete. 2023-11-26T07:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.635+0000 7fb98d0f2700 1 Processor -- start 2023-11-26T07:11:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb98d0f2700 1 -- start start 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb98d0f2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9880b3370 0x7fb9880cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb98d0f2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9880d2380 0x7fb988149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb98d0f2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb98815bdd0 0x7fb988156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb98d0f2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb988155a90 con 0x7fb9880d2380 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb98d0f2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880b76b0 con 0x7fb98815bdd0 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb98d0f2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880b6e10 con 0x7fb9880b3370 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb98659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9880d2380 0x7fb988149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb98659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9880d2380 0x7fb988149a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33914/0 (socket says 172.21.15.83:33914) 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb98659c700 1 -- 172.21.15.83:0/1725203133 learned_addr learned my addr 172.21.15.83:0/1725203133 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb98759e700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb98815bdd0 0x7fb988156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.636+0000 7fb986d9d700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9880b3370 0x7fb9880cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.637+0000 7fb98659c700 1 -- 172.21.15.83:0/1725203133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9880b3370 msgr2=0x7fb9880cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.637+0000 7fb98659c700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9880b3370 0x7fb9880cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.637+0000 7fb98659c700 1 -- 172.21.15.83:0/1725203133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb98815bdd0 msgr2=0x7fb988156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.637+0000 7fb98659c700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb98815bdd0 0x7fb988156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.637+0000 7fb98659c700 1 -- 172.21.15.83:0/1725203133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb980009a70 con 0x7fb9880d2380 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.637+0000 7fb98659c700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9880d2380 0x7fb988149a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c780 tx=0x7fb97800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.637+0000 7fb96f7fe700 1 -- 172.21.15.83:0/1725203133 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978004d40 con 0x7fb9880d2380 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.637+0000 7fb96f7fe700 1 -- 172.21.15.83:0/1725203133 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9780070d0 con 0x7fb9880d2380 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.637+0000 7fb98d0f2700 1 -- 172.21.15.83:0/1725203133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9880b70f0 con 0x7fb9880d2380 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.637+0000 7fb96f7fe700 1 -- 172.21.15.83:0/1725203133 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978004ea0 con 0x7fb9880d2380 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.637+0000 7fb98d0f2700 1 -- 172.21.15.83:0/1725203133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98814b360 con 0x7fb9880d2380 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.638+0000 7fb98d0f2700 1 -- 172.21.15.83:0/1725203133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98814b6d0 con 0x7fb9880d2380 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.638+0000 7fb96f7fe700 1 -- 172.21.15.83:0/1725203133 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb978007480 con 0x7fb9880d2380 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.639+0000 7fb96f7fe700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb97406aed0 0x7fb97406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.639+0000 7fb96f7fe700 1 -- 172.21.15.83:0/1725203133 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb9780890c0 con 0x7fb9880d2380 2023-11-26T07:11:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.639+0000 7fb986d9d700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb97406aed0 0x7fb97406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.639+0000 7fb986d9d700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb97406aed0 0x7fb97406d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0011e0 tx=0x7fb97c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:21.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.643+0000 7fb96f7fe700 1 -- 172.21.15.83:0/1725203133 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9780544c0 con 0x7fb9880d2380 2023-11-26T07:11:21.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.872+0000 7fb98d0f2700 1 -- 172.21.15.83:0/1725203133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9880cf990 con 0x7fb9880d2380 2023-11-26T07:11:21.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.872+0000 7fb96f7fe700 1 -- 172.21.15.83:0/1725203133 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7fb978057ae0 con 0x7fb9880d2380 2023-11-26T07:11:21.875 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:21.875 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2802126359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2802126359},{"type":"v1","addr":"172.21.15.83:6836","nonce":2802126359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-26T07:11:21.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.874+0000 7fb98d0f2700 1 -- 172.21.15.83:0/1725203133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb97406aed0 msgr2=0x7fb97406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.874+0000 7fb98d0f2700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb97406aed0 0x7fb97406d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0011e0 tx=0x7fb97c009040 comp rx=0 tx=0).stop 2023-11-26T07:11:21.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.874+0000 7fb98d0f2700 1 -- 172.21.15.83:0/1725203133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9880d2380 msgr2=0x7fb988149a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:21.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.874+0000 7fb98d0f2700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9880d2380 0x7fb988149a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb97800c780 tx=0x7fb97800ca90 comp rx=0 tx=0).stop 2023-11-26T07:11:21.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.874+0000 7fb98d0f2700 1 -- 172.21.15.83:0/1725203133 shutdown_connections 2023-11-26T07:11:21.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.874+0000 7fb98d0f2700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb98815bdd0 0x7fb988156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.874+0000 7fb98d0f2700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb97406aed0 0x7fb97406d390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.874+0000 7fb98d0f2700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9880b3370 0x7fb9880cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.874+0000 7fb98d0f2700 1 --2- 172.21.15.83:0/1725203133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9880d2380 0x7fb988149a80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:21.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.874+0000 7fb98d0f2700 1 -- 172.21.15.83:0/1725203133 >> 172.21.15.83:0/1725203133 conn(0x7fb9880731a0 msgr2=0x7fb988153b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:21.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.874+0000 7fb98d0f2700 1 -- 172.21.15.83:0/1725203133 shutdown_connections 2023-11-26T07:11:21.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:21.875+0000 7fb98d0f2700 1 -- 172.21.15.83:0/1725203133 wait complete. 2023-11-26T07:11:21.878 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2023-11-26T07:11:23.889 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:11:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.020+0000 7fef895d5700 1 Processor -- start 2023-11-26T07:11:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.020+0000 7fef895d5700 1 -- start start 2023-11-26T07:11:24.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.020+0000 7fef895d5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fef840d0870 0x7fef840d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.020+0000 7fef895d5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef840b6ef0 0x7fef840b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.020+0000 7fef895d5700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef840d20b0 con 0x7fef8415e1e0 2023-11-26T07:11:24.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.020+0000 7fef895d5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef840d3150 con 0x7fef840b6ef0 2023-11-26T07:11:24.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.020+0000 7fef895d5700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef840d1240 con 0x7fef840d0870 2023-11-26T07:11:24.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef837fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef840b6ef0 0x7fef840b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef837fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef840b6ef0 0x7fef840b7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33926/0 (socket says 172.21.15.83:33926) 2023-11-26T07:11:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef837fe700 1 -- 172.21.15.83:0/2361873351 learned_addr learned my addr 172.21.15.83:0/2361873351 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef827fc700 1 --2- 172.21.15.83:0/2361873351 >> v2:172.21.15.203:3300/0 conn(0x7fef840d0870 0x7fef840d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef837fe700 1 -- 172.21.15.83:0/2361873351 >> v2:172.21.15.203:3300/0 conn(0x7fef840d0870 msgr2=0x7fef840d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef837fe700 1 --2- 172.21.15.83:0/2361873351 >> v2:172.21.15.203:3300/0 conn(0x7fef840d0870 0x7fef840d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef837fe700 1 -- 172.21.15.83:0/2361873351 >> v1:172.21.15.203:6789/0 conn(0x7fef8415e1e0 legacy=0x7fef840d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef837fe700 1 -- 172.21.15.83:0/2361873351 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840d2390 con 0x7fef840b6ef0 2023-11-26T07:11:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef837fe700 1 --2- 172.21.15.83:0/2361873351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef840b6ef0 0x7fef840b7360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef84161ba0 tx=0x7fef7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538f0835588d1c46 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef81ffb700 1 -- 172.21.15.83:0/2361873351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00e070 con 0x7fef840b6ef0 2023-11-26T07:11:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef81ffb700 1 -- 172.21.15.83:0/2361873351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7c00b9c0 con 0x7fef840b6ef0 2023-11-26T07:11:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.021+0000 7fef81ffb700 1 -- 172.21.15.83:0/2361873351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00bd70 con 0x7fef840b6ef0 2023-11-26T07:11:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.022+0000 7fef895d5700 1 -- 172.21.15.83:0/2361873351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef840b6ef0 msgr2=0x7fef840b7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.022+0000 7fef895d5700 1 --2- 172.21.15.83:0/2361873351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef840b6ef0 0x7fef840b7360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef84161ba0 tx=0x7fef7c00b330 comp rx=0 tx=0).stop 2023-11-26T07:11:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.022+0000 7fef895d5700 1 -- 172.21.15.83:0/2361873351 shutdown_connections 2023-11-26T07:11:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.022+0000 7fef895d5700 1 --2- 172.21.15.83:0/2361873351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef840b6ef0 0x7fef840b7360 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.022+0000 7fef895d5700 1 --2- 172.21.15.83:0/2361873351 >> v2:172.21.15.203:3300/0 conn(0x7fef840d0870 0x7fef840d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.022+0000 7fef895d5700 1 -- 172.21.15.83:0/2361873351 >> 172.21.15.83:0/2361873351 conn(0x7fef840b5b50 msgr2=0x7fef840b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.022+0000 7fef895d5700 1 -- 172.21.15.83:0/2361873351 shutdown_connections 2023-11-26T07:11:24.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.022+0000 7fef895d5700 1 -- 172.21.15.83:0/2361873351 wait complete. 2023-11-26T07:11:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.023+0000 7fef895d5700 1 Processor -- start 2023-11-26T07:11:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.023+0000 7fef895d5700 1 -- start start 2023-11-26T07:11:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef895d5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef840b6ef0 0x7fef840cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef895d5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef840d0870 0x7fef840cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef895d5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef8415e1e0 0x7fef841588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef895d5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef840cd7d0 con 0x7fef840d0870 2023-11-26T07:11:24.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef895d5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef840caf30 con 0x7fef8415e1e0 2023-11-26T07:11:24.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef895d5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef84159ad0 con 0x7fef840b6ef0 2023-11-26T07:11:24.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef827fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef840d0870 0x7fef840cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef827fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef840d0870 0x7fef840cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33932/0 (socket says 172.21.15.83:33932) 2023-11-26T07:11:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef827fc700 1 -- 172.21.15.83:0/2312798119 learned_addr learned my addr 172.21.15.83:0/2312798119 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef837fe700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef8415e1e0 0x7fef841588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef82ffd700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef840b6ef0 0x7fef840cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef827fc700 1 -- 172.21.15.83:0/2312798119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef840b6ef0 msgr2=0x7fef840cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef827fc700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef840b6ef0 0x7fef840cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef827fc700 1 -- 172.21.15.83:0/2312798119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef8415e1e0 msgr2=0x7fef841588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef827fc700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef8415e1e0 0x7fef841588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef827fc700 1 -- 172.21.15.83:0/2312798119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c009b30 con 0x7fef840d0870 2023-11-26T07:11:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.024+0000 7fef827fc700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef840d0870 0x7fef840cd950 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef7400e840 tx=0x7fef7400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.025+0000 7fef6b7fe700 1 -- 172.21.15.83:0/2312798119 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74013610 con 0x7fef840d0870 2023-11-26T07:11:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.025+0000 7fef6b7fe700 1 -- 172.21.15.83:0/2312798119 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef74013770 con 0x7fef840d0870 2023-11-26T07:11:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.025+0000 7fef895d5700 1 -- 172.21.15.83:0/2312798119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef840cde90 con 0x7fef840d0870 2023-11-26T07:11:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.025+0000 7fef6b7fe700 1 -- 172.21.15.83:0/2312798119 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74013b20 con 0x7fef840d0870 2023-11-26T07:11:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.025+0000 7fef895d5700 1 -- 172.21.15.83:0/2312798119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef840cb460 con 0x7fef840d0870 2023-11-26T07:11:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.025+0000 7fef895d5700 1 -- 172.21.15.83:0/2312798119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef840751e0 con 0x7fef840d0870 2023-11-26T07:11:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.026+0000 7fef6b7fe700 1 -- 172.21.15.83:0/2312798119 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef74013c80 con 0x7fef840d0870 2023-11-26T07:11:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.026+0000 7fef6b7fe700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7006aed0 0x7fef7006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.027+0000 7fef82ffd700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7006aed0 0x7fef7006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.027+0000 7fef6b7fe700 1 -- 172.21.15.83:0/2312798119 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fef74088ed0 con 0x7fef840d0870 2023-11-26T07:11:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.027+0000 7fef82ffd700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7006aed0 0x7fef7006d390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef78000f60 tx=0x7fef78009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.031+0000 7fef6b7fe700 1 -- 172.21.15.83:0/2312798119 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef74054360 con 0x7fef840d0870 2023-11-26T07:11:24.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.261+0000 7fef895d5700 1 -- 172.21.15.83:0/2312798119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef84159c10 con 0x7fef840d0870 2023-11-26T07:11:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.262+0000 7fef6b7fe700 1 -- 172.21.15.83:0/2312798119 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7fef74021e90 con 0x7fef840d0870 2023-11-26T07:11:24.264 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:24.264 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2208741688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2208741688},{"type":"v1","addr":"172.21.15.83:6837","nonce":2208741688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2802126359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2802126359},{"type":"v1","addr":"172.21.15.83:6836","nonce":2802126359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1258871616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1258871616},{"type":"v1","addr":"172.21.15.203:6834","nonce":1258871616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2957324266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":2957324266},{"type":"v1","addr":"172.21.15.203:6835","nonce":2957324266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-26T07:11:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 -- 172.21.15.83:0/2312798119 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7006aed0 msgr2=0x7fef7006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7006aed0 0x7fef7006d390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef78000f60 tx=0x7fef78009040 comp rx=0 tx=0).stop 2023-11-26T07:11:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 -- 172.21.15.83:0/2312798119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef840d0870 msgr2=0x7fef840cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef840d0870 0x7fef840cd950 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef7400e840 tx=0x7fef7400ec00 comp rx=0 tx=0).stop 2023-11-26T07:11:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 -- 172.21.15.83:0/2312798119 shutdown_connections 2023-11-26T07:11:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef8415e1e0 0x7fef841588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7006aed0 0x7fef7006d390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef840b6ef0 0x7fef840cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 --2- 172.21.15.83:0/2312798119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef840d0870 0x7fef840cd950 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 -- 172.21.15.83:0/2312798119 >> 172.21.15.83:0/2312798119 conn(0x7fef840b5b50 msgr2=0x7fef840b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 -- 172.21.15.83:0/2312798119 shutdown_connections 2023-11-26T07:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.264+0000 7fef895d5700 1 -- 172.21.15.83:0/2312798119 wait complete. 2023-11-26T07:11:24.268 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 3 2023-11-26T07:11:24.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.396+0000 7fb97bc67700 1 Processor -- start 2023-11-26T07:11:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.397+0000 7fb97bc67700 1 -- start start 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.397+0000 7fb97bc67700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb97415c160 0x7fb9740622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.397+0000 7fb97bc67700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9740b2870 0x7fb9740afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.397+0000 7fb97bc67700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9740b02e0 0x7fb9740ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.397+0000 7fb97bc67700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9740b2cf0 con 0x7fb9740b02e0 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.397+0000 7fb97bc67700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9740b0770 con 0x7fb97415c160 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.397+0000 7fb97bc67700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97415c540 con 0x7fb9740b2870 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb97a204700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9740b02e0 0x7fb9740ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb97a204700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb9740b02e0 0x7fb9740ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33938/0 (socket says 172.21.15.83:33938) 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb97a204700 1 -- 172.21.15.83:0/2218187838 learned_addr learned my addr 172.21.15.83:0/2218187838 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb979202700 1 --2- 172.21.15.83:0/2218187838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9740b2870 0x7fb9740afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb979a03700 1 --2- 172.21.15.83:0/2218187838 >> v2:172.21.15.203:3300/0 conn(0x7fb97415c160 0x7fb9740622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb97a204700 1 -- 172.21.15.83:0/2218187838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9740b2870 msgr2=0x7fb9740afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:24.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb97a204700 1 --2- 172.21.15.83:0/2218187838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9740b2870 0x7fb9740afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb97a204700 1 -- 172.21.15.83:0/2218187838 >> v2:172.21.15.203:3300/0 conn(0x7fb97415c160 msgr2=0x7fb9740622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:24.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb97a204700 1 --2- 172.21.15.83:0/2218187838 >> v2:172.21.15.203:3300/0 conn(0x7fb97415c160 0x7fb9740622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb97a204700 1 -- 172.21.15.83:0/2218187838 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9740b2ee0 con 0x7fb9740b02e0 2023-11-26T07:11:24.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb979202700 1 --2- 172.21.15.83:0/2218187838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9740b2870 0x7fb9740afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:24.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb97a204700 1 --2- 172.21.15.83:0/2218187838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9740b02e0 0x7fb9740ae1d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb97406fd60 tx=0x7fb96c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5895635c0d082e0e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:24.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb978a01700 1 -- 172.21.15.83:0/2218187838 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00e070 con 0x7fb9740b02e0 2023-11-26T07:11:24.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb978a01700 1 -- 172.21.15.83:0/2218187838 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c00b8d0 con 0x7fb9740b02e0 2023-11-26T07:11:24.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.398+0000 7fb978a01700 1 -- 172.21.15.83:0/2218187838 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00bc80 con 0x7fb9740b02e0 2023-11-26T07:11:24.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.399+0000 7fb97bc67700 1 -- 172.21.15.83:0/2218187838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9740b02e0 msgr2=0x7fb9740ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.399+0000 7fb97bc67700 1 --2- 172.21.15.83:0/2218187838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9740b02e0 0x7fb9740ae1d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb97406fd60 tx=0x7fb96c00b330 comp rx=0 tx=0).stop 2023-11-26T07:11:24.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.399+0000 7fb97bc67700 1 -- 172.21.15.83:0/2218187838 shutdown_connections 2023-11-26T07:11:24.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.399+0000 7fb97bc67700 1 --2- 172.21.15.83:0/2218187838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9740b02e0 0x7fb9740ae1d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.399+0000 7fb97bc67700 1 --2- 172.21.15.83:0/2218187838 >> v2:172.21.15.203:3300/0 conn(0x7fb97415c160 0x7fb9740622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.399+0000 7fb97bc67700 1 --2- 172.21.15.83:0/2218187838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9740b2870 0x7fb9740afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.399+0000 7fb97bc67700 1 -- 172.21.15.83:0/2218187838 >> 172.21.15.83:0/2218187838 conn(0x7fb9740b57b0 msgr2=0x7fb9740b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.399+0000 7fb97bc67700 1 -- 172.21.15.83:0/2218187838 shutdown_connections 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.399+0000 7fb97bc67700 1 -- 172.21.15.83:0/2218187838 wait complete. 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.401+0000 7fb97bc67700 1 Processor -- start 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.401+0000 7fb97bc67700 1 -- start start 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.401+0000 7fb97bc67700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9740b02e0 0x7fb97415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.401+0000 7fb97bc67700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9740b2870 0x7fb9740c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.401+0000 7fb97bc67700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb97415c160 0x7fb9740c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.401+0000 7fb97bc67700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97415d4f0 con 0x7fb9740b2870 2023-11-26T07:11:24.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.401+0000 7fb97bc67700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9741f3cf0 con 0x7fb97415c160 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.401+0000 7fb97bc67700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97415cc50 con 0x7fb9740b02e0 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.401+0000 7fb979202700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9740b2870 0x7fb9740c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.401+0000 7fb979202700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9740b2870 0x7fb9740c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33942/0 (socket says 172.21.15.83:33942) 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.401+0000 7fb979202700 1 -- 172.21.15.83:0/3754031003 learned_addr learned my addr 172.21.15.83:0/3754031003 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb979a03700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9740b02e0 0x7fb97415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb97a204700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb97415c160 0x7fb9740c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb979202700 1 -- 172.21.15.83:0/3754031003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9740b02e0 msgr2=0x7fb97415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb979202700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9740b02e0 0x7fb97415ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb979202700 1 -- 172.21.15.83:0/3754031003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb97415c160 msgr2=0x7fb9740c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb979202700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb97415c160 0x7fb9740c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb979202700 1 -- 172.21.15.83:0/3754031003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c009b30 con 0x7fb9740b2870 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb97a204700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb97415c160 0x7fb9740c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:24.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb979202700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9740b2870 0x7fb9740c9610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb97000d830 tx=0x7fb97000dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb96a7fc700 1 -- 172.21.15.83:0/3754031003 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970004030 con 0x7fb9740b2870 2023-11-26T07:11:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb96a7fc700 1 -- 172.21.15.83:0/3754031003 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb970004190 con 0x7fb9740b2870 2023-11-26T07:11:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb97bc67700 1 -- 172.21.15.83:0/3754031003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97415cf30 con 0x7fb9740b2870 2023-11-26T07:11:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb96a7fc700 1 -- 172.21.15.83:0/3754031003 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970004520 con 0x7fb9740b2870 2023-11-26T07:11:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.402+0000 7fb97bc67700 1 -- 172.21.15.83:0/3754031003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9740b3530 con 0x7fb9740b2870 2023-11-26T07:11:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.403+0000 7fb97bc67700 1 -- 172.21.15.83:0/3754031003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9740b3390 con 0x7fb9740b2870 2023-11-26T07:11:24.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.404+0000 7fb96a7fc700 1 -- 172.21.15.83:0/3754031003 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb9700036a0 con 0x7fb9740b2870 2023-11-26T07:11:24.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.404+0000 7fb96a7fc700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb95806aed0 0x7fb95806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.404+0000 7fb96a7fc700 1 -- 172.21.15.83:0/3754031003 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb970089890 con 0x7fb9740b2870 2023-11-26T07:11:24.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.404+0000 7fb979a03700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb95806aed0 0x7fb95806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.404+0000 7fb979a03700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb95806aed0 0x7fb95806d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb964009a10 tx=0x7fb964008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:24.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.408+0000 7fb96a7fc700 1 -- 172.21.15.83:0/3754031003 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9740b3390 con 0x7fb9740b2870 2023-11-26T07:11:24.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.602+0000 7fb97bc67700 1 -- 172.21.15.83:0/3754031003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9740b3390 con 0x7fb9740b2870 2023-11-26T07:11:24.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.602+0000 7fb96a7fc700 1 -- 172.21.15.83:0/3754031003 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7fb9740b3390 con 0x7fb9740b2870 2023-11-26T07:11:24.605 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:24.605 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:18.378934+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:24.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.604+0000 7fb97bc67700 1 -- 172.21.15.83:0/3754031003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb95806aed0 msgr2=0x7fb95806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.604+0000 7fb97bc67700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb95806aed0 0x7fb95806d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb964009a10 tx=0x7fb964008040 comp rx=0 tx=0).stop 2023-11-26T07:11:24.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.604+0000 7fb97bc67700 1 -- 172.21.15.83:0/3754031003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9740b2870 msgr2=0x7fb9740c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.604+0000 7fb97bc67700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9740b2870 0x7fb9740c9610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb97000d830 tx=0x7fb97000dbf0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.605+0000 7fb97bc67700 1 -- 172.21.15.83:0/3754031003 shutdown_connections 2023-11-26T07:11:24.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.605+0000 7fb97bc67700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb97415c160 0x7fb9740c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.605+0000 7fb97bc67700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb95806aed0 0x7fb95806d390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.605+0000 7fb97bc67700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9740b02e0 0x7fb97415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.605+0000 7fb97bc67700 1 --2- 172.21.15.83:0/3754031003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9740b2870 0x7fb9740c9610 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.605+0000 7fb97bc67700 1 -- 172.21.15.83:0/3754031003 >> 172.21.15.83:0/3754031003 conn(0x7fb9740b57b0 msgr2=0x7fb9740afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:24.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.605+0000 7fb97bc67700 1 -- 172.21.15.83:0/3754031003 shutdown_connections 2023-11-26T07:11:24.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.605+0000 7fb97bc67700 1 -- 172.21.15.83:0/3754031003 wait complete. 2023-11-26T07:11:24.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:11:24.619 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:11:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.739+0000 7f986268e700 1 Processor -- start 2023-11-26T07:11:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.739+0000 7f986268e700 1 -- start start 2023-11-26T07:11:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.740+0000 7f986268e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f985c15f9e0 0x7f985c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.740+0000 7f986268e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f985c073f90 0x7f985c074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.740+0000 7f986268e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c06b080 con 0x7f985c0cf610 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.740+0000 7f986268e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985c074960 con 0x7f985c073f90 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.740+0000 7f986268e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985c15c290 con 0x7f985c15f9e0 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.740+0000 7f9860c2b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f985c073f90 0x7f985c074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.740+0000 7f9860c2b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f985c073f90 0x7f985c074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33952/0 (socket says 172.21.15.83:33952) 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.740+0000 7f985bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f985c0cf610 0x7f985c0b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:53988/0 (socket says 172.21.15.83:53988) 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.740+0000 7f9860c2b700 1 -- 172.21.15.83:0/2031608018 learned_addr learned my addr 172.21.15.83:0/2031608018 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f985b7fe700 1 --2- 172.21.15.83:0/2031608018 >> v2:172.21.15.203:3300/0 conn(0x7f985c15f9e0 0x7f985c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f9860c2b700 1 -- 172.21.15.83:0/2031608018 >> v2:172.21.15.203:3300/0 conn(0x7f985c15f9e0 msgr2=0x7f985c0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f9860c2b700 1 --2- 172.21.15.83:0/2031608018 >> v2:172.21.15.203:3300/0 conn(0x7f985c15f9e0 0x7f985c0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f9860c2b700 1 -- 172.21.15.83:0/2031608018 >> v1:172.21.15.83:6789/0 conn(0x7f985c0cf610 legacy=0x7f985c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f9860c2b700 1 -- 172.21.15.83:0/2031608018 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c0cf9f0 con 0x7f985c073f90 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f985affd700 1 -- 172.21.15.83:0/2031608018 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517699654 0 0) 0x7f985c06b080 con 0x7f985c0cf610 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f985b7fe700 1 --2- 172.21.15.83:0/2031608018 >> v2:172.21.15.203:3300/0 conn(0x7f985c15f9e0 0x7f985c0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f9860c2b700 1 --2- 172.21.15.83:0/2031608018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c073f90 0x7f985c074420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9854009e50 tx=0x7f985400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567718fb70374a7d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f985affd700 1 -- 172.21.15.83:0/2031608018 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985400e070 con 0x7f985c073f90 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f985affd700 1 -- 172.21.15.83:0/2031608018 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985400b980 con 0x7f985c073f90 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f985affd700 1 -- 172.21.15.83:0/2031608018 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985400bd30 con 0x7f985c073f90 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f986268e700 1 -- 172.21.15.83:0/2031608018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c073f90 msgr2=0x7f985c074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f986268e700 1 --2- 172.21.15.83:0/2031608018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c073f90 0x7f985c074420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9854009e50 tx=0x7f985400b330 comp rx=0 tx=0).stop 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f986268e700 1 -- 172.21.15.83:0/2031608018 shutdown_connections 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f986268e700 1 --2- 172.21.15.83:0/2031608018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c073f90 0x7f985c074420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f986268e700 1 --2- 172.21.15.83:0/2031608018 >> v2:172.21.15.203:3300/0 conn(0x7f985c15f9e0 0x7f985c0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.741+0000 7f986268e700 1 -- 172.21.15.83:0/2031608018 >> 172.21.15.83:0/2031608018 conn(0x7f985c0d0bf0 msgr2=0x7f985c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.742+0000 7f986268e700 1 -- 172.21.15.83:0/2031608018 shutdown_connections 2023-11-26T07:11:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.742+0000 7f986268e700 1 -- 172.21.15.83:0/2031608018 wait complete. 2023-11-26T07:11:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.743+0000 7f986268e700 1 Processor -- start 2023-11-26T07:11:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.743+0000 7f986268e700 1 -- start start 2023-11-26T07:11:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f986268e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f985c073f90 0x7f985c14daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f986268e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cf610 0x7f985c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f986268e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f985c15f9e0 0x7f985c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f986268e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c14e030 con 0x7f985c0cf610 2023-11-26T07:11:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f986268e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c06f320 con 0x7f985c073f90 2023-11-26T07:11:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f986268e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c06e280 con 0x7f985c15f9e0 2023-11-26T07:11:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f985b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cf610 0x7f985c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f985b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cf610 0x7f985c06d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33956/0 (socket says 172.21.15.83:33956) 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f985b7fe700 1 -- 172.21.15.83:0/3657689335 learned_addr learned my addr 172.21.15.83:0/3657689335 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f985bfff700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f985c073f90 0x7f985c14daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f9860c2b700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f985c15f9e0 0x7f985c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f985b7fe700 1 -- 172.21.15.83:0/3657689335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f985c15f9e0 msgr2=0x7f985c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f985b7fe700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f985c15f9e0 0x7f985c15d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f985b7fe700 1 -- 172.21.15.83:0/3657689335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f985c073f90 msgr2=0x7f985c14daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f985b7fe700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f985c073f90 0x7f985c14daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f985b7fe700 1 -- 172.21.15.83:0/3657689335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9854009b30 con 0x7f985c0cf610 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f9860c2b700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f985c15f9e0 0x7f985c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f985b7fe700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cf610 0x7f985c06d930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c750 tx=0x7f984c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f9858ff9700 1 -- 172.21.15.83:0/3657689335 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c011800 con 0x7f985c0cf610 2023-11-26T07:11:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f9858ff9700 1 -- 172.21.15.83:0/3657689335 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c011960 con 0x7f985c0cf610 2023-11-26T07:11:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f9858ff9700 1 -- 172.21.15.83:0/3657689335 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c011cd0 con 0x7f985c0cf610 2023-11-26T07:11:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f986268e700 1 -- 172.21.15.83:0/3657689335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985c06e560 con 0x7f985c0cf610 2023-11-26T07:11:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.744+0000 7f986268e700 1 -- 172.21.15.83:0/3657689335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985c06eb20 con 0x7f985c0cf610 2023-11-26T07:11:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.745+0000 7f986268e700 1 -- 172.21.15.83:0/3657689335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985c068fb0 con 0x7f985c0cf610 2023-11-26T07:11:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.746+0000 7f9858ff9700 1 -- 172.21.15.83:0/3657689335 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f984c023480 con 0x7f985c0cf610 2023-11-26T07:11:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.746+0000 7f9858ff9700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f984806aed0 0x7f984806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.747+0000 7f9858ff9700 1 -- 172.21.15.83:0/3657689335 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f984c089600 con 0x7f985c0cf610 2023-11-26T07:11:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.747+0000 7f985bfff700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f984806aed0 0x7f984806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.747+0000 7f985bfff700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f984806aed0 0x7f984806d390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9850007fd0 tx=0x7f9850007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:24.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.750+0000 7f9858ff9700 1 -- 172.21.15.83:0/3657689335 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f984c0549e0 con 0x7f985c0cf610 2023-11-26T07:11:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.938+0000 7f986268e700 1 -- 172.21.15.83:0/3657689335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f985c160240 con 0x7f985c0cf610 2023-11-26T07:11:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.938+0000 7f9858ff9700 1 -- 172.21.15.83:0/3657689335 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f985c160240 con 0x7f985c0cf610 2023-11-26T07:11:24.941 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:11:24.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 -- 172.21.15.83:0/3657689335 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f984806aed0 msgr2=0x7f984806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f984806aed0 0x7f984806d390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9850007fd0 tx=0x7f9850007de0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 -- 172.21.15.83:0/3657689335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cf610 msgr2=0x7f985c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:24.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cf610 0x7f985c06d930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c750 tx=0x7f984c00ca60 comp rx=0 tx=0).stop 2023-11-26T07:11:24.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 -- 172.21.15.83:0/3657689335 shutdown_connections 2023-11-26T07:11:24.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f984806aed0 0x7f984806d390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f985c15f9e0 0x7f985c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f985c073f90 0x7f985c14daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 --2- 172.21.15.83:0/3657689335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f985c0cf610 0x7f985c06d930 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:24.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 -- 172.21.15.83:0/3657689335 >> 172.21.15.83:0/3657689335 conn(0x7f985c0d0bf0 msgr2=0x7f985c0b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:24.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 -- 172.21.15.83:0/3657689335 shutdown_connections 2023-11-26T07:11:24.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:24.940+0000 7f986268e700 1 -- 172.21.15.83:0/3657689335 wait complete. 2023-11-26T07:11:24.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:11:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.071+0000 7f59f1133700 1 Processor -- start 2023-11-26T07:11:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.071+0000 7f59f1133700 1 -- start start 2023-11-26T07:11:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.072+0000 7f59f1133700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f59ec15bdd0 0x7f59ec1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.072+0000 7f59f1133700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f59ec0d1c70 0x7f59ec0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.072+0000 7f59f1133700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec0b22d0 con 0x7f59ec0b3370 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.072+0000 7f59f1133700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec0b2f50 con 0x7f59ec0d1c70 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.072+0000 7f59f1133700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec0d2640 con 0x7f59ec15bdd0 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.072+0000 7f59eb59e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f59ec0d1c70 0x7f59ec0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.072+0000 7f59eb59e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f59ec0d1c70 0x7f59ec0d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33966/0 (socket says 172.21.15.83:33966) 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.072+0000 7f59eb59e700 1 -- 172.21.15.83:0/2779062423 learned_addr learned my addr 172.21.15.83:0/2779062423 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.072+0000 7f59ead9d700 1 --2- 172.21.15.83:0/2779062423 >> v2:172.21.15.203:3300/0 conn(0x7f59ec15bdd0 0x7f59ec1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.072+0000 7f59eb59e700 1 -- 172.21.15.83:0/2779062423 >> v2:172.21.15.203:3300/0 conn(0x7f59ec15bdd0 msgr2=0x7f59ec1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.072+0000 7f59eb59e700 1 --2- 172.21.15.83:0/2779062423 >> v2:172.21.15.203:3300/0 conn(0x7f59ec15bdd0 0x7f59ec1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59eb59e700 1 -- 172.21.15.83:0/2779062423 >> v1:172.21.15.203:6789/0 conn(0x7f59ec0b3370 legacy=0x7f59ec0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59eb59e700 1 -- 172.21.15.83:0/2779062423 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0b25b0 con 0x7f59ec0d1c70 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59eb59e700 1 --2- 172.21.15.83:0/2779062423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ec0d1c70 0x7f59ec0d2100 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59ec06ec10 tx=0x7f59e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340b14be36d23340 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59e9d9b700 1 -- 172.21.15.83:0/2779062423 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e400e070 con 0x7f59ec0d1c70 2023-11-26T07:11:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59e9d9b700 1 -- 172.21.15.83:0/2779062423 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e400b9c0 con 0x7f59ec0d1c70 2023-11-26T07:11:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59e9d9b700 1 -- 172.21.15.83:0/2779062423 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e400bd70 con 0x7f59ec0d1c70 2023-11-26T07:11:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59f1133700 1 -- 172.21.15.83:0/2779062423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ec0d1c70 msgr2=0x7f59ec0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59f1133700 1 --2- 172.21.15.83:0/2779062423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ec0d1c70 0x7f59ec0d2100 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59ec06ec10 tx=0x7f59e400b330 comp rx=0 tx=0).stop 2023-11-26T07:11:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59f1133700 1 -- 172.21.15.83:0/2779062423 shutdown_connections 2023-11-26T07:11:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59f1133700 1 --2- 172.21.15.83:0/2779062423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ec0d1c70 0x7f59ec0d2100 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59f1133700 1 --2- 172.21.15.83:0/2779062423 >> v2:172.21.15.203:3300/0 conn(0x7f59ec15bdd0 0x7f59ec1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59f1133700 1 -- 172.21.15.83:0/2779062423 >> 172.21.15.83:0/2779062423 conn(0x7f59ec0731a0 msgr2=0x7f59ec09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59f1133700 1 -- 172.21.15.83:0/2779062423 shutdown_connections 2023-11-26T07:11:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.073+0000 7f59f1133700 1 -- 172.21.15.83:0/2779062423 wait complete. 2023-11-26T07:11:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.075+0000 7f59f1133700 1 Processor -- start 2023-11-26T07:11:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.075+0000 7f59f1133700 1 -- start start 2023-11-26T07:11:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59f1133700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59ec0b3370 0x7f59ec156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59f1133700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ec0d1c70 0x7f59ec0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59f1133700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec15bdd0 0x7f59ec0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59f1133700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec0c7a50 con 0x7f59ec0d1c70 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59f1133700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec15f540 con 0x7f59ec15bdd0 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59f1133700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec1572c0 con 0x7f59ec0b3370 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59ea59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ec0d1c70 0x7f59ec0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59ea59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ec0d1c70 0x7f59ec0b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33970/0 (socket says 172.21.15.83:33970) 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59ea59c700 1 -- 172.21.15.83:0/1797958425 learned_addr learned my addr 172.21.15.83:0/1797958425 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59ead9d700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59ec0b3370 0x7f59ec156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59eb59e700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec15bdd0 0x7f59ec0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59ea59c700 1 -- 172.21.15.83:0/1797958425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59ec0b3370 msgr2=0x7f59ec156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59ea59c700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59ec0b3370 0x7f59ec156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59ea59c700 1 -- 172.21.15.83:0/1797958425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec15bdd0 msgr2=0x7f59ec0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59ea59c700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec15bdd0 0x7f59ec0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59ea59c700 1 -- 172.21.15.83:0/1797958425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e4009b30 con 0x7f59ec0d1c70 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59eb59e700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec15bdd0 0x7f59ec0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59ea59c700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ec0d1c70 0x7f59ec0b4c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c4e0 tx=0x7f59dc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59d37fe700 1 -- 172.21.15.83:0/1797958425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0052d0 con 0x7f59ec0d1c70 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59d37fe700 1 -- 172.21.15.83:0/1797958425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59dc005430 con 0x7f59ec0d1c70 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59f1133700 1 -- 172.21.15.83:0/1797958425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec0c5990 con 0x7f59ec0d1c70 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.076+0000 7f59d37fe700 1 -- 172.21.15.83:0/1797958425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0038c0 con 0x7f59ec0d1c70 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.077+0000 7f59f1133700 1 -- 172.21.15.83:0/1797958425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec15f260 con 0x7f59ec0d1c70 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.077+0000 7f59f1133700 1 -- 172.21.15.83:0/1797958425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec067f60 con 0x7f59ec0d1c70 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.078+0000 7f59d37fe700 1 -- 172.21.15.83:0/1797958425 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f59dc003a20 con 0x7f59ec0d1c70 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.078+0000 7f59d37fe700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f59d806aed0 0x7f59d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.078+0000 7f59d37fe700 1 -- 172.21.15.83:0/1797958425 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f59dc088770 con 0x7f59ec0d1c70 2023-11-26T07:11:25.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.078+0000 7f59ead9d700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f59d806aed0 0x7f59d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.078+0000 7f59ead9d700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f59d806aed0 0x7f59d806d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009770 tx=0x7f59e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:25.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.082+0000 7f59d37fe700 1 -- 172.21.15.83:0/1797958425 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f59dc053ae0 con 0x7f59ec0d1c70 2023-11-26T07:11:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.275+0000 7f59f1133700 1 -- 172.21.15.83:0/1797958425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f59ec0b7c60 con 0x7f59ec0d1c70 2023-11-26T07:11:25.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.276+0000 7f59d37fe700 1 -- 172.21.15.83:0/1797958425 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f59dc017090 con 0x7f59ec0d1c70 2023-11-26T07:11:25.278 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:25.278 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:11:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:11:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2023-11-26T07:11:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2023-11-26T07:11:25.279 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:11:25.279 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:11:25.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 -- 172.21.15.83:0/1797958425 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f59d806aed0 msgr2=0x7f59d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:25.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f59d806aed0 0x7f59d806d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009770 tx=0x7f59e0008040 comp rx=0 tx=0).stop 2023-11-26T07:11:25.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 -- 172.21.15.83:0/1797958425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ec0d1c70 msgr2=0x7f59ec0b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:25.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ec0d1c70 0x7f59ec0b4c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c4e0 tx=0x7f59dc00c8a0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 -- 172.21.15.83:0/1797958425 shutdown_connections 2023-11-26T07:11:25.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec15bdd0 0x7f59ec0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f59d806aed0 0x7f59d806d390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59ec0b3370 0x7f59ec156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 --2- 172.21.15.83:0/1797958425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ec0d1c70 0x7f59ec0b4c80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 -- 172.21.15.83:0/1797958425 >> 172.21.15.83:0/1797958425 conn(0x7f59ec0731a0 msgr2=0x7f59ec0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:25.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 -- 172.21.15.83:0/1797958425 shutdown_connections 2023-11-26T07:11:25.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.278+0000 7f59f1133700 1 -- 172.21.15.83:0/1797958425 wait complete. 2023-11-26T07:11:25.292 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:11:25.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:11:25.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.413+0000 7f96faeb0700 1 Processor -- start 2023-11-26T07:11:25.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.413+0000 7f96faeb0700 1 -- start start 2023-11-26T07:11:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.414+0000 7f96faeb0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f96f40b01e0 0x7f96f406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.414+0000 7f96faeb0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f96f40b59c0 0x7f96f40b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.414+0000 7f96faeb0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f40b05c0 con 0x7f96f4073f90 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.414+0000 7f96faeb0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f41470a0 con 0x7f96f40b59c0 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.414+0000 7f96faeb0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f40b5e70 con 0x7f96f40b01e0 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.414+0000 7f96f3fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f96f40b59c0 0x7f96f40b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.414+0000 7f96f3fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f96f40b59c0 0x7f96f40b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33976/0 (socket says 172.21.15.83:33976) 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.414+0000 7f96f944d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f96f4073f90 0x7f96f406aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54020/0 (socket says 172.21.15.83:54020) 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.414+0000 7f96f3fff700 1 -- 172.21.15.83:0/3168538635 learned_addr learned my addr 172.21.15.83:0/3168538635 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.415+0000 7f96f8c4c700 1 --2- 172.21.15.83:0/3168538635 >> v2:172.21.15.203:3300/0 conn(0x7f96f40b01e0 0x7f96f406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.415+0000 7f96f3fff700 1 -- 172.21.15.83:0/3168538635 >> v2:172.21.15.203:3300/0 conn(0x7f96f40b01e0 msgr2=0x7f96f406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.415+0000 7f96f3fff700 1 --2- 172.21.15.83:0/3168538635 >> v2:172.21.15.203:3300/0 conn(0x7f96f40b01e0 0x7f96f406e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.415+0000 7f96f3fff700 1 -- 172.21.15.83:0/3168538635 >> v1:172.21.15.83:6789/0 conn(0x7f96f4073f90 legacy=0x7f96f406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.415+0000 7f96f3fff700 1 -- 172.21.15.83:0/3168538635 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f40b4250 con 0x7f96f40b59c0 2023-11-26T07:11:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.415+0000 7f96f3fff700 1 --2- 172.21.15.83:0/3168538635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f40b59c0 0x7f96f40b3b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96f4072ce0 tx=0x7f96e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f861a694c47c36fc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.415+0000 7f96f37fe700 1 -- 172.21.15.83:0/3168538635 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0004230 con 0x7f96f40b59c0 2023-11-26T07:11:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.415+0000 7f96f37fe700 1 -- 172.21.15.83:0/3168538635 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0004390 con 0x7f96f40b59c0 2023-11-26T07:11:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.415+0000 7f96f37fe700 1 -- 172.21.15.83:0/3168538635 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e00036a0 con 0x7f96f40b59c0 2023-11-26T07:11:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.416+0000 7f96faeb0700 1 -- 172.21.15.83:0/3168538635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f40b59c0 msgr2=0x7f96f40b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.416+0000 7f96faeb0700 1 --2- 172.21.15.83:0/3168538635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f40b59c0 0x7f96f40b3b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f96f4072ce0 tx=0x7f96e0009c00 comp rx=0 tx=0).stop 2023-11-26T07:11:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.416+0000 7f96faeb0700 1 -- 172.21.15.83:0/3168538635 shutdown_connections 2023-11-26T07:11:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.416+0000 7f96faeb0700 1 --2- 172.21.15.83:0/3168538635 >> v2:172.21.15.203:3300/0 conn(0x7f96f40b01e0 0x7f96f406e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.416+0000 7f96faeb0700 1 --2- 172.21.15.83:0/3168538635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f40b59c0 0x7f96f40b3b50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.416+0000 7f96faeb0700 1 -- 172.21.15.83:0/3168538635 >> 172.21.15.83:0/3168538635 conn(0x7f96f40d0b40 msgr2=0x7f96f40d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.416+0000 7f96faeb0700 1 -- 172.21.15.83:0/3168538635 shutdown_connections 2023-11-26T07:11:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.416+0000 7f96faeb0700 1 -- 172.21.15.83:0/3168538635 wait complete. 2023-11-26T07:11:25.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.417+0000 7f96faeb0700 1 Processor -- start 2023-11-26T07:11:25.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.417+0000 7f96faeb0700 1 -- start start 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.418+0000 7f96faeb0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f4073f90 0x7f96f4144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.418+0000 7f96faeb0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f40b01e0 0x7f96f4141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.418+0000 7f96faeb0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f40b59c0 0x7f96f41420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.418+0000 7f96faeb0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f4152a30 con 0x7f96f40b01e0 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.418+0000 7f96faeb0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f41425f0 con 0x7f96f40b59c0 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.418+0000 7f96faeb0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f4151db0 con 0x7f96f4073f90 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.418+0000 7f96f3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f40b01e0 0x7f96f4141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.418+0000 7f96f3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f40b01e0 0x7f96f4141b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33984/0 (socket says 172.21.15.83:33984) 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.418+0000 7f96f3fff700 1 -- 172.21.15.83:0/352540581 learned_addr learned my addr 172.21.15.83:0/352540581 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96f8c4c700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f4073f90 0x7f96f4144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96f944d700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f40b59c0 0x7f96f41420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96f3fff700 1 -- 172.21.15.83:0/352540581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f4073f90 msgr2=0x7f96f4144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96f3fff700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f4073f90 0x7f96f4144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96f3fff700 1 -- 172.21.15.83:0/352540581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f40b59c0 msgr2=0x7f96f41420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96f3fff700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f40b59c0 0x7f96f41420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96f3fff700 1 -- 172.21.15.83:0/352540581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e0009550 con 0x7f96f40b01e0 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96f3fff700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f40b01e0 0x7f96f4141b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009870 tx=0x7f96e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96f17fa700 1 -- 172.21.15.83:0/352540581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0011630 con 0x7f96f40b01e0 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96f17fa700 1 -- 172.21.15.83:0/352540581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0011790 con 0x7f96f40b01e0 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96f17fa700 1 -- 172.21.15.83:0/352540581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0011b20 con 0x7f96f40b01e0 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96faeb0700 1 -- 172.21.15.83:0/352540581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f4152090 con 0x7f96f40b01e0 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.419+0000 7f96faeb0700 1 -- 172.21.15.83:0/352540581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f4152650 con 0x7f96f40b01e0 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.420+0000 7f96faeb0700 1 -- 172.21.15.83:0/352540581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f4152b70 con 0x7f96f40b01e0 2023-11-26T07:11:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.420+0000 7f96f17fa700 1 -- 172.21.15.83:0/352540581 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f96e0011c80 con 0x7f96f40b01e0 2023-11-26T07:11:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.421+0000 7f96f17fa700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f96d806aed0 0x7f96d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.421+0000 7f96f17fa700 1 -- 172.21.15.83:0/352540581 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f96e008aa60 con 0x7f96f40b01e0 2023-11-26T07:11:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.421+0000 7f96f8c4c700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f96d806aed0 0x7f96d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.421+0000 7f96f8c4c700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f96d806aed0 0x7f96d806d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f96f415ff90 tx=0x7f96e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.425+0000 7f96f17fa700 1 -- 172.21.15.83:0/352540581 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f96f4152b70 con 0x7f96f40b01e0 2023-11-26T07:11:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:25.616+0000 7f96faeb0700 1 -- 172.21.15.83:0/352540581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f96f4152b70 con 0x7f96f40b01e0 2023-11-26T07:11:26.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:11:26.400 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:11:26.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.502+0000 7f96f17fa700 1 -- 172.21.15.83:0/352540581 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f96f4152b70 con 0x7f96f40b01e0 2023-11-26T07:11:26.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.505+0000 7f96faeb0700 1 -- 172.21.15.83:0/352540581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f96d806aed0 msgr2=0x7f96d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:26.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.505+0000 7f96faeb0700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f96d806aed0 0x7f96d806d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f96f415ff90 tx=0x7f96e8008040 comp rx=0 tx=0).stop 2023-11-26T07:11:26.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.505+0000 7f96faeb0700 1 -- 172.21.15.83:0/352540581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f40b01e0 msgr2=0x7f96f4141b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:26.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.505+0000 7f96faeb0700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f40b01e0 0x7f96f4141b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009870 tx=0x7f96e0004750 comp rx=0 tx=0).stop 2023-11-26T07:11:26.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.506+0000 7f96faeb0700 1 -- 172.21.15.83:0/352540581 shutdown_connections 2023-11-26T07:11:26.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.506+0000 7f96faeb0700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96f40b59c0 0x7f96f41420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:26.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.506+0000 7f96faeb0700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f96d806aed0 0x7f96d806d390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:26.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.506+0000 7f96faeb0700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96f4073f90 0x7f96f4144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:26.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.506+0000 7f96faeb0700 1 --2- 172.21.15.83:0/352540581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f40b01e0 0x7f96f4141b70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:26.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.506+0000 7f96faeb0700 1 -- 172.21.15.83:0/352540581 >> 172.21.15.83:0/352540581 conn(0x7f96f40d0b40 msgr2=0x7f96f40743e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:26.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.508+0000 7f96faeb0700 1 -- 172.21.15.83:0/352540581 shutdown_connections 2023-11-26T07:11:26.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.508+0000 7f96faeb0700 1 -- 172.21.15.83:0/352540581 wait complete. 2023-11-26T07:11:26.514 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2023-11-26T07:11:26.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:11:26.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.656+0000 7fcaf5222700 1 Processor -- start 2023-11-26T07:11:26.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.656+0000 7fcaf5222700 1 -- start start 2023-11-26T07:11:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.656+0000 7fcaf5222700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcaf00b3bf0 0x7fcaf006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaf5222700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf00b0590 con 0x7fcaf0073f90 2023-11-26T07:11:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaf5222700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf00b4080 con 0x7fcaf015e150 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaf5222700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf00b4360 con 0x7fcaf00b3bf0 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaee59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcaf00b3bf0 0x7fcaf006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaef59e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcaf0073f90 0x7fcaf006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54034/0 (socket says 172.21.15.83:54034) 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaef59e700 1 -- 172.21.15.83:0/2597446304 learned_addr learned my addr 172.21.15.83:0/2597446304 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaee59c700 1 -- 172.21.15.83:0/2597446304 >> v1:172.21.15.203:6789/0 conn(0x7fcaf015e150 legacy=0x7fcaf006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaee59c700 1 -- 172.21.15.83:0/2597446304 >> v1:172.21.15.83:6789/0 conn(0x7fcaf0073f90 legacy=0x7fcaf006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaedd9b700 1 -- 172.21.15.83:0/2597446304 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660807528 0 0) 0x7fcaf00b0590 con 0x7fcaf0073f90 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaee59c700 1 -- 172.21.15.83:0/2597446304 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf006dae0 con 0x7fcaf00b3bf0 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaee59c700 1 --2- 172.21.15.83:0/2597446304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf00b3bf0 0x7fcaf006e5c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009840 tx=0x7fcae4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fb68ad833e55f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaedd9b700 1 -- 172.21.15.83:0/2597446304 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4011830 con 0x7fcaf00b3bf0 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaedd9b700 1 -- 172.21.15.83:0/2597446304 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae4011990 con 0x7fcaf00b3bf0 2023-11-26T07:11:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.657+0000 7fcaedd9b700 1 -- 172.21.15.83:0/2597446304 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4011d20 con 0x7fcaf00b3bf0 2023-11-26T07:11:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.658+0000 7fcaf5222700 1 -- 172.21.15.83:0/2597446304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf00b3bf0 msgr2=0x7fcaf006e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.658+0000 7fcaf5222700 1 --2- 172.21.15.83:0/2597446304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf00b3bf0 0x7fcaf006e5c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009840 tx=0x7fcae4009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.658+0000 7fcaf5222700 1 -- 172.21.15.83:0/2597446304 shutdown_connections 2023-11-26T07:11:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.658+0000 7fcaf5222700 1 --2- 172.21.15.83:0/2597446304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf00b3bf0 0x7fcaf006e5c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.658+0000 7fcaf5222700 1 -- 172.21.15.83:0/2597446304 >> 172.21.15.83:0/2597446304 conn(0x7fcaf00d0be0 msgr2=0x7fcaf00d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.658+0000 7fcaf5222700 1 -- 172.21.15.83:0/2597446304 shutdown_connections 2023-11-26T07:11:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.658+0000 7fcaf5222700 1 -- 172.21.15.83:0/2597446304 wait complete. 2023-11-26T07:11:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.659+0000 7fcaf5222700 1 Processor -- start 2023-11-26T07:11:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.659+0000 7fcaf5222700 1 -- start start 2023-11-26T07:11:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.660+0000 7fcaf5222700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf0073f90 0x7fcaf014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:26.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.660+0000 7fcaf5222700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf015e150 0x7fcaf014c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.660+0000 7fcaf5222700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf00c86e0 0x7fcaf00cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.660+0000 7fcaf5222700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00c5c00 con 0x7fcaf015e150 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.660+0000 7fcaf5222700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf01527f0 con 0x7fcaf00c86e0 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.660+0000 7fcaf5222700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf014f920 con 0x7fcaf0073f90 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.660+0000 7fcaee59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf015e150 0x7fcaf014c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.660+0000 7fcaee59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf015e150 0x7fcaf014c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33998/0 (socket says 172.21.15.83:33998) 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.660+0000 7fcaee59c700 1 -- 172.21.15.83:0/1337674234 learned_addr learned my addr 172.21.15.83:0/1337674234 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.660+0000 7fcaef59e700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf00c86e0 0x7fcaf00cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.660+0000 7fcaeed9d700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf0073f90 0x7fcaf014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcaee59c700 1 -- 172.21.15.83:0/1337674234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf0073f90 msgr2=0x7fcaf014f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcaee59c700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf0073f90 0x7fcaf014f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcaee59c700 1 -- 172.21.15.83:0/1337674234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf00c86e0 msgr2=0x7fcaf00cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcaee59c700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf00c86e0 0x7fcaf00cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcaee59c700 1 -- 172.21.15.83:0/1337674234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae40094a0 con 0x7fcaf015e150 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcaef59e700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf00c86e0 0x7fcaf00cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcaeed9d700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf0073f90 0x7fcaf014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcaee59c700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf015e150 0x7fcaf014c980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcae40090b0 tx=0x7fcae4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcad77fe700 1 -- 172.21.15.83:0/1337674234 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae40036a0 con 0x7fcaf015e150 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcad77fe700 1 -- 172.21.15.83:0/1337674234 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae401f070 con 0x7fcaf015e150 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcaf5222700 1 -- 172.21.15.83:0/1337674234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf00c5e70 con 0x7fcaf015e150 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcad77fe700 1 -- 172.21.15.83:0/1337674234 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4003a00 con 0x7fcaf015e150 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.661+0000 7fcaf5222700 1 -- 172.21.15.83:0/1337674234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf00c6430 con 0x7fcaf015e150 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.662+0000 7fcaf5222700 1 -- 172.21.15.83:0/1337674234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf0068fb0 con 0x7fcaf015e150 2023-11-26T07:11:26.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.662+0000 7fcad77fe700 1 -- 172.21.15.83:0/1337674234 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcae4003b60 con 0x7fcaf015e150 2023-11-26T07:11:26.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.663+0000 7fcad77fe700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcadc06aed0 0x7fcadc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:26.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.663+0000 7fcad77fe700 1 -- 172.21.15.83:0/1337674234 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fcae4021070 con 0x7fcaf015e150 2023-11-26T07:11:26.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.663+0000 7fcaeed9d700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcadc06aed0 0x7fcadc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:26.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.663+0000 7fcaeed9d700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcadc06aed0 0x7fcadc06d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcae0000f60 tx=0x7fcae0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.667+0000 7fcad77fe700 1 -- 172.21.15.83:0/1337674234 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcae408f070 con 0x7fcaf015e150 2023-11-26T07:11:26.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:26.858+0000 7fcaf5222700 1 -- 172.21.15.83:0/1337674234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcaf00c6620 con 0x7fcaf015e150 2023-11-26T07:11:27.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.507+0000 7fcad77fe700 1 -- 172.21.15.83:0/1337674234 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcae40150d0 con 0x7fcaf015e150 2023-11-26T07:11:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 -- 172.21.15.83:0/1337674234 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcadc06aed0 msgr2=0x7fcadc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcadc06aed0 0x7fcadc06d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcae0000f60 tx=0x7fcae0009040 comp rx=0 tx=0).stop 2023-11-26T07:11:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 -- 172.21.15.83:0/1337674234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf015e150 msgr2=0x7fcaf014c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf015e150 0x7fcaf014c980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcae40090b0 tx=0x7fcae4004750 comp rx=0 tx=0).stop 2023-11-26T07:11:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 -- 172.21.15.83:0/1337674234 shutdown_connections 2023-11-26T07:11:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf00c86e0 0x7fcaf00cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcadc06aed0 0x7fcadc06d390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf0073f90 0x7fcaf014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 --2- 172.21.15.83:0/1337674234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf015e150 0x7fcaf014c980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 -- 172.21.15.83:0/1337674234 >> 172.21.15.83:0/1337674234 conn(0x7fcaf00d0be0 msgr2=0x7fcaf006b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:27.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 -- 172.21.15.83:0/1337674234 shutdown_connections 2023-11-26T07:11:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.509+0000 7fcaf5222700 1 -- 172.21.15.83:0/1337674234 wait complete. 2023-11-26T07:11:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2023-11-26T07:11:27.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:11:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.662+0000 7f70661a6700 1 Processor -- start 2023-11-26T07:11:27.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.662+0000 7f70661a6700 1 -- start start 2023-11-26T07:11:27.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f70661a6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f706015bf50 0x7f70600ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:27.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f70661a6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f70600ce830 0x7f70600cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:27.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f70661a6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70600d0140 con 0x7f70600cf1f0 2023-11-26T07:11:27.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f70661a6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706015c330 con 0x7f70600ce830 2023-11-26T07:11:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f70661a6700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70600d03e0 con 0x7f706015bf50 2023-11-26T07:11:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f705effd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f70600ce830 0x7f70600cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f705ffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f70600cf1f0 0x7f70600c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54048/0 (socket says 172.21.15.83:54048) 2023-11-26T07:11:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f705ffff700 1 -- 172.21.15.83:0/2155472734 learned_addr learned my addr 172.21.15.83:0/2155472734 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f705f7fe700 1 --2- 172.21.15.83:0/2155472734 >> v2:172.21.15.203:3300/0 conn(0x7f706015bf50 0x7f70600ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:27.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f705e7fc700 1 -- 172.21.15.83:0/2155472734 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207956659 0 0) 0x7f70600d0140 con 0x7f70600cf1f0 2023-11-26T07:11:27.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f705e7fc700 1 -- 172.21.15.83:0/2155472734 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70480035b0 con 0x7f70600cf1f0 2023-11-26T07:11:27.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f705effd700 1 -- 172.21.15.83:0/2155472734 >> v2:172.21.15.203:3300/0 conn(0x7f706015bf50 msgr2=0x7f70600ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:27.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f705effd700 1 --2- 172.21.15.83:0/2155472734 >> v2:172.21.15.203:3300/0 conn(0x7f706015bf50 0x7f70600ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:27.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f705effd700 1 -- 172.21.15.83:0/2155472734 >> v1:172.21.15.83:6789/0 conn(0x7f70600cf1f0 legacy=0x7f70600c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.663+0000 7f705effd700 1 -- 172.21.15.83:0/2155472734 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70600d06c0 con 0x7f70600ce830 2023-11-26T07:11:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f705effd700 1 --2- 172.21.15.83:0/2155472734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70600ce830 0x7f70600cecb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70600715e0 tx=0x7f7050009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeae7ce8486818 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f705e7fc700 1 -- 172.21.15.83:0/2155472734 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050011830 con 0x7f70600ce830 2023-11-26T07:11:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f705e7fc700 1 -- 172.21.15.83:0/2155472734 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7050011990 con 0x7f70600ce830 2023-11-26T07:11:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f705e7fc700 1 -- 172.21.15.83:0/2155472734 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050011d20 con 0x7f70600ce830 2023-11-26T07:11:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f70661a6700 1 -- 172.21.15.83:0/2155472734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70600ce830 msgr2=0x7f70600cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f70661a6700 1 --2- 172.21.15.83:0/2155472734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70600ce830 0x7f70600cecb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70600715e0 tx=0x7f7050009b20 comp rx=0 tx=0).stop 2023-11-26T07:11:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f70661a6700 1 -- 172.21.15.83:0/2155472734 shutdown_connections 2023-11-26T07:11:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f70661a6700 1 --2- 172.21.15.83:0/2155472734 >> v2:172.21.15.203:3300/0 conn(0x7f706015bf50 0x7f70600ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f70661a6700 1 --2- 172.21.15.83:0/2155472734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70600ce830 0x7f70600cecb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f70661a6700 1 -- 172.21.15.83:0/2155472734 >> 172.21.15.83:0/2155472734 conn(0x7f70600b3780 msgr2=0x7f70600b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f70661a6700 1 -- 172.21.15.83:0/2155472734 shutdown_connections 2023-11-26T07:11:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.664+0000 7f70661a6700 1 -- 172.21.15.83:0/2155472734 wait complete. 2023-11-26T07:11:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.665+0000 7f70661a6700 1 Processor -- start 2023-11-26T07:11:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.666+0000 7f70661a6700 1 -- start start 2023-11-26T07:11:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.666+0000 7f70661a6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70600ce830 0x7f70600cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.666+0000 7f70661a6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70600cf1f0 0x7f7060155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.666+0000 7f70661a6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f706015bf50 0x7f7060143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.666+0000 7f70661a6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601549e0 con 0x7f70600cf1f0 2023-11-26T07:11:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.666+0000 7f70661a6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600b5a80 con 0x7f70600ce830 2023-11-26T07:11:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.666+0000 7f70661a6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70601584a0 con 0x7f706015bf50 2023-11-26T07:11:27.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f705effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70600cf1f0 0x7f7060155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f705effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70600cf1f0 0x7f7060155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34012/0 (socket says 172.21.15.83:34012) 2023-11-26T07:11:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f705effd700 1 -- 172.21.15.83:0/755257448 learned_addr learned my addr 172.21.15.83:0/755257448 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f705f7fe700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70600ce830 0x7f70600cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f705ffff700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f706015bf50 0x7f7060143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:27.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f705effd700 1 -- 172.21.15.83:0/755257448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f706015bf50 msgr2=0x7f7060143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f705effd700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f706015bf50 0x7f7060143060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f705effd700 1 -- 172.21.15.83:0/755257448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70600ce830 msgr2=0x7f70600cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f705effd700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70600ce830 0x7f70600cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f705effd700 1 -- 172.21.15.83:0/755257448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70500094a0 con 0x7f70600cf1f0 2023-11-26T07:11:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f705effd700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70600cf1f0 0x7f7060155130 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7050009b60 tx=0x7f7050004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f7047fff700 1 -- 172.21.15.83:0/755257448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70500036a0 con 0x7f70600cf1f0 2023-11-26T07:11:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f7047fff700 1 -- 172.21.15.83:0/755257448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705001f070 con 0x7f70600cf1f0 2023-11-26T07:11:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f70661a6700 1 -- 172.21.15.83:0/755257448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70600cbf70 con 0x7f70600cf1f0 2023-11-26T07:11:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f7047fff700 1 -- 172.21.15.83:0/755257448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050003a40 con 0x7f70600cf1f0 2023-11-26T07:11:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.667+0000 7f70661a6700 1 -- 172.21.15.83:0/755257448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7060157c00 con 0x7f70600cf1f0 2023-11-26T07:11:27.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.668+0000 7f70661a6700 1 -- 172.21.15.83:0/755257448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7060068fb0 con 0x7f70600cf1f0 2023-11-26T07:11:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.669+0000 7f7047fff700 1 -- 172.21.15.83:0/755257448 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7050003ba0 con 0x7f70600cf1f0 2023-11-26T07:11:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.669+0000 7f7047fff700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f704c06aed0 0x7f704c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.669+0000 7f705f7fe700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f704c06aed0 0x7f704c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.669+0000 7f7047fff700 1 -- 172.21.15.83:0/755257448 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f7050021070 con 0x7f70600cf1f0 2023-11-26T07:11:27.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.669+0000 7f705f7fe700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f704c06aed0 0x7f704c06d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7054006b20 tx=0x7f7054008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.675+0000 7f7047fff700 1 -- 172.21.15.83:0/755257448 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7050055f40 con 0x7f70600cf1f0 2023-11-26T07:11:27.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:27.909+0000 7f70661a6700 1 -- 172.21.15.83:0/755257448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f70600c7e40 con 0x7f70600cf1f0 2023-11-26T07:11:28.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.530+0000 7f7047fff700 1 -- 172.21.15.83:0/755257448 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f70500150c0 con 0x7f70600cf1f0 2023-11-26T07:11:28.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.535+0000 7f70661a6700 1 -- 172.21.15.83:0/755257448 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f704c06aed0 msgr2=0x7f704c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:28.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.535+0000 7f70661a6700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f704c06aed0 0x7f704c06d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7054006b20 tx=0x7f7054008040 comp rx=0 tx=0).stop 2023-11-26T07:11:28.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.535+0000 7f70661a6700 1 -- 172.21.15.83:0/755257448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70600cf1f0 msgr2=0x7f7060155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:28.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.535+0000 7f70661a6700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70600cf1f0 0x7f7060155130 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7050009b60 tx=0x7f7050004750 comp rx=0 tx=0).stop 2023-11-26T07:11:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.538+0000 7f70661a6700 1 -- 172.21.15.83:0/755257448 shutdown_connections 2023-11-26T07:11:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.538+0000 7f70661a6700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f704c06aed0 0x7f704c06d390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.538+0000 7f70661a6700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f706015bf50 0x7f7060143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.538+0000 7f70661a6700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70600ce830 0x7f70600cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.538+0000 7f70661a6700 1 --2- 172.21.15.83:0/755257448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70600cf1f0 0x7f7060155130 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.538+0000 7f70661a6700 1 -- 172.21.15.83:0/755257448 >> 172.21.15.83:0/755257448 conn(0x7f70600b3780 msgr2=0x7f70600c5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:28.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.539+0000 7f70661a6700 1 -- 172.21.15.83:0/755257448 shutdown_connections 2023-11-26T07:11:28.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.550+0000 7f70661a6700 1 -- 172.21.15.83:0/755257448 wait complete. 2023-11-26T07:11:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 2 and data pool 3 2023-11-26T07:11:28.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:28.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.689+0000 7f6f179cb700 1 Processor -- start 2023-11-26T07:11:28.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.689+0000 7f6f179cb700 1 -- start start 2023-11-26T07:11:28.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.690+0000 7f6f179cb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6f1006c710 0x7f6f100b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:28.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.690+0000 7f6f179cb700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f100ae430 con 0x7f6f1015e080 2023-11-26T07:11:28.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.690+0000 7f6f179cb700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f100ae710 con 0x7f6f100af4d0 2023-11-26T07:11:28.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.690+0000 7f6f179cb700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f100af0b0 con 0x7f6f1006c710 2023-11-26T07:11:28.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.690+0000 7f6f14f66700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6f1006c710 0x7f6f100b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:28.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.690+0000 7f6f15767700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6f1015e080 0x7f6f1006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54058/0 (socket says 172.21.15.83:54058) 2023-11-26T07:11:28.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.690+0000 7f6f15767700 1 -- 172.21.15.83:0/1820082069 learned_addr learned my addr 172.21.15.83:0/1820082069 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:28.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.690+0000 7f6f14f66700 1 -- 172.21.15.83:0/1820082069 >> v1:172.21.15.203:6789/0 conn(0x7f6f100af4d0 legacy=0x7f6f1006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:28.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.690+0000 7f6f14f66700 1 -- 172.21.15.83:0/1820082069 >> v1:172.21.15.83:6789/0 conn(0x7f6f1015e080 legacy=0x7f6f1006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:28.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f14f66700 1 -- 172.21.15.83:0/1820082069 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f100ae9f0 con 0x7f6f1006c710 2023-11-26T07:11:28.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.690+0000 7f6f07fff700 1 -- 172.21.15.83:0/1820082069 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125341312 0 0) 0x7f6f100ae430 con 0x7f6f1015e080 2023-11-26T07:11:28.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f14f66700 1 --2- 172.21.15.83:0/1820082069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f1006c710 0x7f6f100b5050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009840 tx=0x7f6f00009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81bf79327c9273e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:28.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f07fff700 1 -- 172.21.15.83:0/1820082069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00011830 con 0x7f6f1006c710 2023-11-26T07:11:28.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f07fff700 1 -- 172.21.15.83:0/1820082069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f00011990 con 0x7f6f1006c710 2023-11-26T07:11:28.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f07fff700 1 -- 172.21.15.83:0/1820082069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f00011d20 con 0x7f6f1006c710 2023-11-26T07:11:28.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f179cb700 1 -- 172.21.15.83:0/1820082069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f1006c710 msgr2=0x7f6f100b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:28.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f179cb700 1 --2- 172.21.15.83:0/1820082069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f1006c710 0x7f6f100b5050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009840 tx=0x7f6f00009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:28.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f179cb700 1 -- 172.21.15.83:0/1820082069 shutdown_connections 2023-11-26T07:11:28.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f179cb700 1 --2- 172.21.15.83:0/1820082069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f1006c710 0x7f6f100b5050 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f179cb700 1 -- 172.21.15.83:0/1820082069 >> 172.21.15.83:0/1820082069 conn(0x7f6f10071b90 msgr2=0x7f6f10071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:28.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f179cb700 1 -- 172.21.15.83:0/1820082069 shutdown_connections 2023-11-26T07:11:28.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.691+0000 7f6f179cb700 1 -- 172.21.15.83:0/1820082069 wait complete. 2023-11-26T07:11:28.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f179cb700 1 Processor -- start 2023-11-26T07:11:28.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f179cb700 1 -- start start 2023-11-26T07:11:28.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f179cb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f1006c710 0x7f6f100ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:28.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f179cb700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f100af4d0 0x7f6f10160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:28.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f179cb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f1015e080 0x7f6f10144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:28.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f179cb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f100cf8f0 con 0x7f6f1006c710 2023-11-26T07:11:28.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f179cb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f101613f0 con 0x7f6f1015e080 2023-11-26T07:11:28.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f179cb700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f100cece0 con 0x7f6f100af4d0 2023-11-26T07:11:28.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f15767700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f1006c710 0x7f6f100ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:28.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f15767700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f1006c710 0x7f6f100ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34024/0 (socket says 172.21.15.83:34024) 2023-11-26T07:11:28.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f15767700 1 -- 172.21.15.83:0/2223041785 learned_addr learned my addr 172.21.15.83:0/2223041785 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:28.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.693+0000 7f6f14f66700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f100af4d0 0x7f6f10160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:28.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f15f68700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f1015e080 0x7f6f10144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:28.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f15767700 1 -- 172.21.15.83:0/2223041785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f100af4d0 msgr2=0x7f6f10160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:28.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f15767700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f100af4d0 0x7f6f10160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f15767700 1 -- 172.21.15.83:0/2223041785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f1015e080 msgr2=0x7f6f10144220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:28.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f15767700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f1015e080 0x7f6f10144220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f15767700 1 -- 172.21.15.83:0/2223041785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f000094a0 con 0x7f6f1006c710 2023-11-26T07:11:28.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f15767700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f1006c710 0x7f6f100ce670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00c520 tx=0x7f6f0c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:28.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f05ffb700 1 -- 172.21.15.83:0/2223041785 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c004c80 con 0x7f6f1006c710 2023-11-26T07:11:28.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f05ffb700 1 -- 172.21.15.83:0/2223041785 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c004e20 con 0x7f6f1006c710 2023-11-26T07:11:28.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f179cb700 1 -- 172.21.15.83:0/2223041785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f100cefc0 con 0x7f6f1006c710 2023-11-26T07:11:28.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f05ffb700 1 -- 172.21.15.83:0/2223041785 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c007310 con 0x7f6f1006c710 2023-11-26T07:11:28.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.694+0000 7f6f179cb700 1 -- 172.21.15.83:0/2223041785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f100cf580 con 0x7f6f1006c710 2023-11-26T07:11:28.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.695+0000 7f6f179cb700 1 -- 172.21.15.83:0/2223041785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f100753f0 con 0x7f6f1006c710 2023-11-26T07:11:28.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.695+0000 7f6f05ffb700 1 -- 172.21.15.83:0/2223041785 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f0c007470 con 0x7f6f1006c710 2023-11-26T07:11:28.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.696+0000 7f6f05ffb700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ef406aed0 0x7f6ef406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:28.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.696+0000 7f6f14f66700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ef406aed0 0x7f6ef406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:28.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.696+0000 7f6f05ffb700 1 -- 172.21.15.83:0/2223041785 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f0c0891f0 con 0x7f6f1006c710 2023-11-26T07:11:28.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.696+0000 7f6f14f66700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ef406aed0 0x7f6ef406d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f100b5560 tx=0x7f6f0001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:28.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.700+0000 7f6f05ffb700 1 -- 172.21.15.83:0/2223041785 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f0c054330 con 0x7f6f1006c710 2023-11-26T07:11:28.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.896+0000 7f6f179cb700 1 -- 172.21.15.83:0/2223041785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f10067f60 con 0x7f6f1006c710 2023-11-26T07:11:28.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.897+0000 7f6f05ffb700 1 -- 172.21.15.83:0/2223041785 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16506 (secure 0 0 0) 0x7f6f0c057950 con 0x7f6f1006c710 2023-11-26T07:11:28.898 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:28.899 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:27.911126+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 -- 172.21.15.83:0/2223041785 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ef406aed0 msgr2=0x7f6ef406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ef406aed0 0x7f6ef406d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f100b5560 tx=0x7f6f0001c040 comp rx=0 tx=0).stop 2023-11-26T07:11:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 -- 172.21.15.83:0/2223041785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f1006c710 msgr2=0x7f6f100ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:28.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f1006c710 0x7f6f100ce670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00c520 tx=0x7f6f0c00c8e0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 -- 172.21.15.83:0/2223041785 shutdown_connections 2023-11-26T07:11:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f1015e080 0x7f6f10144220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f1006c710 0x7f6f100ce670 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ef406aed0 0x7f6ef406d390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 --2- 172.21.15.83:0/2223041785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f100af4d0 0x7f6f10160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 -- 172.21.15.83:0/2223041785 >> 172.21.15.83:0/2223041785 conn(0x7f6f10071b90 msgr2=0x7f6f1006bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 -- 172.21.15.83:0/2223041785 shutdown_connections 2023-11-26T07:11:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:28.899+0000 7f6f179cb700 1 -- 172.21.15.83:0/2223041785 wait complete. 2023-11-26T07:11:28.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:29.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.031+0000 7f62a78a5700 1 Processor -- start 2023-11-26T07:11:29.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.031+0000 7f62a78a5700 1 -- start start 2023-11-26T07:11:29.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.032+0000 7f62a78a5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a00b3370 0x7f62a01604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.032+0000 7f62a78a5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f62a00d1c70 0x7f62a00d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.032+0000 7f62a78a5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a00b22d0 con 0x7f62a015bdd0 2023-11-26T07:11:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.032+0000 7f62a78a5700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a00b2f50 con 0x7f62a00d1c70 2023-11-26T07:11:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.032+0000 7f62a78a5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a00d2640 con 0x7f62a00b3370 2023-11-26T07:11:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.032+0000 7f62a5641700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f62a015bdd0 0x7f62a00b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54072/0 (socket says 172.21.15.83:54072) 2023-11-26T07:11:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.032+0000 7f62a5641700 1 -- 172.21.15.83:0/2968820675 learned_addr learned my addr 172.21.15.83:0/2968820675 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:29.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a4e40700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a00b3370 0x7f62a01604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968064574 0 0) 0x7f62a00b22d0 con 0x7f62a015bdd0 2023-11-26T07:11:29.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a5e42700 1 --2- 172.21.15.83:0/2968820675 >> v2:172.21.15.203:3300/0 conn(0x7f62a00d1c70 0x7f62a00d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f628c0035b0 con 0x7f62a015bdd0 2023-11-26T07:11:29.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 732866119 0 0) 0x7f628c0035b0 con 0x7f62a015bdd0 2023-11-26T07:11:29.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a00b3370 msgr2=0x7f62a01604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:29.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a00b3370 0x7f62a01604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 >> v2:172.21.15.203:3300/0 conn(0x7f62a00d1c70 msgr2=0x7f62a00d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:29.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 --2- 172.21.15.83:0/2968820675 >> v2:172.21.15.203:3300/0 conn(0x7f62a00d1c70 0x7f62a00d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a00b25b0 con 0x7f62a015bdd0 2023-11-26T07:11:29.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f6290001670 con 0x7f62a015bdd0 2023-11-26T07:11:29.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 >> v1:172.21.15.83:6789/0 conn(0x7f62a015bdd0 legacy=0x7f62a00b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:29.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a00b3370 0x7f628c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 0x7f628c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f628c004d90 0x7f628c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a00b2f50 con 0x7f628c003f80 2023-11-26T07:11:29.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a00d2640 con 0x7f628c004d90 2023-11-26T07:11:29.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c00ba10 con 0x7f62a00b3370 2023-11-26T07:11:29.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a4e40700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 0x7f628c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a5641700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a00b3370 0x7f628c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a5e42700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f628c004d90 0x7f628c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a4e40700 1 -- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a00b3370 msgr2=0x7f628c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:29.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a4e40700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a00b3370 0x7f628c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a4e40700 1 -- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f628c004d90 msgr2=0x7f628c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:29.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a4e40700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f628c004d90 0x7f628c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a4e40700 1 -- 172.21.15.83:0/2968820675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0035b0 con 0x7f628c003f80 2023-11-26T07:11:29.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a5641700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a00b3370 0x7f628c003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:29.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.033+0000 7f62a4e40700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 0x7f628c004850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f629c009b60 tx=0x7f628c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41b54c16c47f84f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:29.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.034+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c00e660 con 0x7f628c003f80 2023-11-26T07:11:29.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.034+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c00e7c0 con 0x7f628c003f80 2023-11-26T07:11:29.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.034+0000 7f6297fff700 1 -- 172.21.15.83:0/2968820675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c00eb30 con 0x7f628c003f80 2023-11-26T07:11:29.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.034+0000 7f62a78a5700 1 -- 172.21.15.83:0/2968820675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 msgr2=0x7f628c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:29.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.034+0000 7f62a78a5700 1 --2- 172.21.15.83:0/2968820675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 0x7f628c004850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f629c009b60 tx=0x7f628c005880 comp rx=0 tx=0).stop 2023-11-26T07:11:29.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.034+0000 7f62a5e42700 1 -- 172.21.15.83:0/2968820675 reap_dead start 2023-11-26T07:11:29.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.034+0000 7f62a78a5700 1 -- 172.21.15.83:0/2968820675 shutdown_connections 2023-11-26T07:11:29.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.034+0000 7f62a78a5700 1 -- 172.21.15.83:0/2968820675 >> 172.21.15.83:0/2968820675 conn(0x7f62a00731a0 msgr2=0x7f62a009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:29.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.034+0000 7f62a78a5700 1 -- 172.21.15.83:0/2968820675 shutdown_connections 2023-11-26T07:11:29.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.034+0000 7f62a78a5700 1 -- 172.21.15.83:0/2968820675 wait complete. 2023-11-26T07:11:29.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.035+0000 7f62a78a5700 1 Processor -- start 2023-11-26T07:11:29.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.035+0000 7f62a78a5700 1 -- start start 2023-11-26T07:11:29.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a78a5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 0x7f62a01edc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a78a5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a015c8d0 0x7f62a01f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a78a5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62a015d280 0x7f62a015d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a78a5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a006b5a0 con 0x7f628c003f80 2023-11-26T07:11:29.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a78a5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a015dc50 con 0x7f62a015d280 2023-11-26T07:11:29.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a78a5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a0158ab0 con 0x7f62a015c8d0 2023-11-26T07:11:29.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a5641700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 0x7f62a01edc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a5641700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 0x7f62a01edc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34044/0 (socket says 172.21.15.83:34044) 2023-11-26T07:11:29.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a5641700 1 -- 172.21.15.83:0/823374814 learned_addr learned my addr 172.21.15.83:0/823374814 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:29.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a4e40700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a015c8d0 0x7f62a01f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a5e42700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62a015d280 0x7f62a015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a5641700 1 -- 172.21.15.83:0/823374814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a015c8d0 msgr2=0x7f62a01f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.036+0000 7f62a5641700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a015c8d0 0x7f62a01f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.037+0000 7f62a5641700 1 -- 172.21.15.83:0/823374814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62a015d280 msgr2=0x7f62a015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.037+0000 7f62a5641700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62a015d280 0x7f62a015d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.037+0000 7f62a5641700 1 -- 172.21.15.83:0/823374814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a00c6840 con 0x7f628c003f80 2023-11-26T07:11:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.037+0000 7f62a4e40700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a015c8d0 0x7f62a01f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:29.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.037+0000 7f62a5e42700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62a015d280 0x7f62a015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:29.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.037+0000 7f62a5641700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 0x7f62a01edc80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6290000fb0 tx=0x7f629000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:29.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.037+0000 7f6295ffb700 1 -- 172.21.15.83:0/823374814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6290005e00 con 0x7f628c003f80 2023-11-26T07:11:29.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.037+0000 7f6295ffb700 1 -- 172.21.15.83:0/823374814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629000c750 con 0x7f628c003f80 2023-11-26T07:11:29.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.037+0000 7f62a78a5700 1 -- 172.21.15.83:0/823374814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a01f4600 con 0x7f628c003f80 2023-11-26T07:11:29.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.037+0000 7f6295ffb700 1 -- 172.21.15.83:0/823374814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629000cac0 con 0x7f628c003f80 2023-11-26T07:11:29.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.037+0000 7f62a78a5700 1 -- 172.21.15.83:0/823374814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a01f4bc0 con 0x7f628c003f80 2023-11-26T07:11:29.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.038+0000 7f62a78a5700 1 -- 172.21.15.83:0/823374814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a0067f60 con 0x7f628c003f80 2023-11-26T07:11:29.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.038+0000 7f6295ffb700 1 -- 172.21.15.83:0/823374814 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f629000cc20 con 0x7f628c003f80 2023-11-26T07:11:29.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.039+0000 7f6295ffb700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f628406aed0 0x7f628406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.039+0000 7f6295ffb700 1 -- 172.21.15.83:0/823374814 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6290089430 con 0x7f628c003f80 2023-11-26T07:11:29.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.039+0000 7f62a4e40700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f628406aed0 0x7f628406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.039+0000 7f62a4e40700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f628406aed0 0x7f628406d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f629c0090b0 tx=0x7f629c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:29.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.043+0000 7f6295ffb700 1 -- 172.21.15.83:0/823374814 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62900544e0 con 0x7f628c003f80 2023-11-26T07:11:29.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.234+0000 7f62a78a5700 1 -- 172.21.15.83:0/823374814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62a00b7c60 con 0x7f628c003f80 2023-11-26T07:11:29.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.235+0000 7f6295ffb700 1 -- 172.21.15.83:0/823374814 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16506 (secure 0 0 0) 0x7f6290057b00 con 0x7f628c003f80 2023-11-26T07:11:29.237 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:29.237 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:27.911126+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:29.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 -- 172.21.15.83:0/823374814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f628406aed0 msgr2=0x7f628406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:29.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f628406aed0 0x7f628406d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f629c0090b0 tx=0x7f629c008030 comp rx=0 tx=0).stop 2023-11-26T07:11:29.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 -- 172.21.15.83:0/823374814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 msgr2=0x7f62a01edc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:29.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 0x7f62a01edc80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6290000fb0 tx=0x7f629000ee80 comp rx=0 tx=0).stop 2023-11-26T07:11:29.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 -- 172.21.15.83:0/823374814 shutdown_connections 2023-11-26T07:11:29.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62a015d280 0x7f62a015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f628c003f80 0x7f62a01edc80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f628406aed0 0x7f628406d390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 --2- 172.21.15.83:0/823374814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62a015c8d0 0x7f62a01f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 -- 172.21.15.83:0/823374814 >> 172.21.15.83:0/823374814 conn(0x7f62a00731a0 msgr2=0x7f62a00cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:29.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 -- 172.21.15.83:0/823374814 shutdown_connections 2023-11-26T07:11:29.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.237+0000 7f62a78a5700 1 -- 172.21.15.83:0/823374814 wait complete. 2023-11-26T07:11:29.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:11:29.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.372+0000 7f9fe56e4700 1 Processor -- start 2023-11-26T07:11:29.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.373+0000 7f9fe56e4700 1 -- start start 2023-11-26T07:11:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.373+0000 7f9fe56e4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fe015e140 0x7f9fe00d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.373+0000 7f9fe56e4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9fe00b98c0 0x7f9fe00d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.373+0000 7f9fe56e4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe00b80e0 con 0x7f9fe00d39d0 2023-11-26T07:11:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.373+0000 7f9fe56e4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe015e520 con 0x7f9fe00b98c0 2023-11-26T07:11:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.373+0000 7f9fe56e4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe00d2930 con 0x7f9fe015e140 2023-11-26T07:11:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.373+0000 7f9fde7fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9fe00d39d0 0x7f9fe00b59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54092/0 (socket says 172.21.15.83:54092) 2023-11-26T07:11:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fde7fc700 1 -- 172.21.15.83:0/2644219720 learned_addr learned my addr 172.21.15.83:0/2644219720 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161335743 0 0) 0x7f9fd0001110 con 0x7f9fe00d39d0 2023-11-26T07:11:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fdeffd700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fe015e140 0x7f9fe00d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fdf7fe700 1 --2- 172.21.15.83:0/2644219720 >> v2:172.21.15.203:3300/0 conn(0x7f9fe00b98c0 0x7f9fe00d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc80035b0 con 0x7f9fe00d39d0 2023-11-26T07:11:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2956548629 0 0) 0x7f9fc80035b0 con 0x7f9fe00d39d0 2023-11-26T07:11:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fe015e140 msgr2=0x7f9fe00d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fe015e140 0x7f9fe00d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 >> v2:172.21.15.203:3300/0 conn(0x7f9fe00b98c0 msgr2=0x7f9fe00d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 --2- 172.21.15.83:0/2644219720 >> v2:172.21.15.203:3300/0 conn(0x7f9fe00b98c0 0x7f9fe00d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe00d2c10 con 0x7f9fe00d39d0 2023-11-26T07:11:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f9fd0001800 con 0x7f9fe00d39d0 2023-11-26T07:11:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 >> v1:172.21.15.83:6789/0 conn(0x7f9fe00d39d0 legacy=0x7f9fe00b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fc8003a00 0x7f9fc8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fc8004850 0x7f9fc8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fe015e140 0x7f9fc8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe015e520 con 0x7f9fc8004850 2023-11-26T07:11:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe00d2930 con 0x7f9fc8003a00 2023-11-26T07:11:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc800ba10 con 0x7f9fe015e140 2023-11-26T07:11:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fde7fc700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fc8004850 0x7f9fc8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fdeffd700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fc8003a00 0x7f9fc8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fdf7fe700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fe015e140 0x7f9fc8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fde7fc700 1 -- 172.21.15.83:0/2644219720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fe015e140 msgr2=0x7f9fc8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fde7fc700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fe015e140 0x7f9fc8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fde7fc700 1 -- 172.21.15.83:0/2644219720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fc8003a00 msgr2=0x7f9fc8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fde7fc700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fc8003a00 0x7f9fc8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.374+0000 7f9fde7fc700 1 -- 172.21.15.83:0/2644219720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc80035b0 con 0x7f9fc8004850 2023-11-26T07:11:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.375+0000 7f9fde7fc700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fc8004850 0x7f9fc8004cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000c2f0 tx=0x7f9fc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b739cd3f8c3b66 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.375+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd000ce30 con 0x7f9fc8004850 2023-11-26T07:11:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.375+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd0005960 con 0x7f9fc8004850 2023-11-26T07:11:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.375+0000 7f9fddffb700 1 -- 172.21.15.83:0/2644219720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0005d10 con 0x7f9fc8004850 2023-11-26T07:11:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.375+0000 7f9fe56e4700 1 -- 172.21.15.83:0/2644219720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fc8004850 msgr2=0x7f9fc8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.375+0000 7f9fe56e4700 1 --2- 172.21.15.83:0/2644219720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fc8004850 0x7f9fc8004cd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000c2f0 tx=0x7f9fc8005880 comp rx=0 tx=0).stop 2023-11-26T07:11:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.375+0000 7f9fdf7fe700 1 -- 172.21.15.83:0/2644219720 reap_dead start 2023-11-26T07:11:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.375+0000 7f9fe56e4700 1 -- 172.21.15.83:0/2644219720 shutdown_connections 2023-11-26T07:11:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.375+0000 7f9fe56e4700 1 -- 172.21.15.83:0/2644219720 >> 172.21.15.83:0/2644219720 conn(0x7f9fe006b410 msgr2=0x7f9fe006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:29.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.375+0000 7f9fe56e4700 1 -- 172.21.15.83:0/2644219720 shutdown_connections 2023-11-26T07:11:29.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.375+0000 7f9fe56e4700 1 -- 172.21.15.83:0/2644219720 wait complete. 2023-11-26T07:11:29.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.377+0000 7f9fe56e4700 1 Processor -- start 2023-11-26T07:11:29.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.377+0000 7f9fe56e4700 1 -- start start 2023-11-26T07:11:29.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.377+0000 7f9fe56e4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fc8004850 0x7f9fe00b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.377+0000 7f9fe56e4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe00b42a0 0x7f9fe015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.377+0000 7f9fe56e4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe00c9660 0x7f9fe00c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.377+0000 7f9fe56e4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe00b2720 con 0x7f9fe00c9660 2023-11-26T07:11:29.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.377+0000 7f9fe56e4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe00ca020 con 0x7f9fe00b42a0 2023-11-26T07:11:29.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.377+0000 7f9fe56e4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe00b6660 con 0x7f9fc8004850 2023-11-26T07:11:29.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.377+0000 7f9fdf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe00c9660 0x7f9fe00c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.377+0000 7f9fdf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe00c9660 0x7f9fe00c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34068/0 (socket says 172.21.15.83:34068) 2023-11-26T07:11:29.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.377+0000 7f9fdf7fe700 1 -- 172.21.15.83:0/1019496416 learned_addr learned my addr 172.21.15.83:0/1019496416 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:29.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fde7fc700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe00b42a0 0x7f9fe015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fdeffd700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fc8004850 0x7f9fe00b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fdf7fe700 1 -- 172.21.15.83:0/1019496416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fc8004850 msgr2=0x7f9fe00b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:29.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fdf7fe700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fc8004850 0x7f9fe00b6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fdf7fe700 1 -- 172.21.15.83:0/1019496416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe00b42a0 msgr2=0x7f9fe015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:29.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fdf7fe700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe00b42a0 0x7f9fe015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:29.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fdf7fe700 1 -- 172.21.15.83:0/1019496416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe00c78b0 con 0x7f9fe00c9660 2023-11-26T07:11:29.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fde7fc700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe00b42a0 0x7f9fe015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:29.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fdf7fe700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe00c9660 0x7f9fe00c9ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00b6df0 tx=0x7f9fd80114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:29.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fc77fe700 1 -- 172.21.15.83:0/1019496416 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd800e070 con 0x7f9fe00c9660 2023-11-26T07:11:29.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fc77fe700 1 -- 172.21.15.83:0/1019496416 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd8003cf0 con 0x7f9fe00c9660 2023-11-26T07:11:29.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fe56e4700 1 -- 172.21.15.83:0/1019496416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe00b47d0 con 0x7f9fe00c9660 2023-11-26T07:11:29.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fc77fe700 1 -- 172.21.15.83:0/1019496416 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8003ea0 con 0x7f9fe00c9660 2023-11-26T07:11:29.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.378+0000 7f9fe56e4700 1 -- 172.21.15.83:0/1019496416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe00b4d90 con 0x7f9fe00c9660 2023-11-26T07:11:29.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.379+0000 7f9fe56e4700 1 -- 172.21.15.83:0/1019496416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe00c7a40 con 0x7f9fe00c9660 2023-11-26T07:11:29.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.379+0000 7f9fc77fe700 1 -- 172.21.15.83:0/1019496416 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9fd8011a10 con 0x7f9fe00c9660 2023-11-26T07:11:29.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.380+0000 7f9fc77fe700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9fcc06aed0 0x7f9fcc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:29.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.380+0000 7f9fc77fe700 1 -- 172.21.15.83:0/1019496416 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fd8089bf0 con 0x7f9fe00c9660 2023-11-26T07:11:29.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.380+0000 7f9fdeffd700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9fcc06aed0 0x7f9fcc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:29.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.380+0000 7f9fdeffd700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9fcc06aed0 0x7f9fcc06d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009940 tx=0x7f9fd4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:29.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.384+0000 7f9fc77fe700 1 -- 172.21.15.83:0/1019496416 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9fd8054d30 con 0x7f9fe00c9660 2023-11-26T07:11:29.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:29.623+0000 7f9fe56e4700 1 -- 172.21.15.83:0/1019496416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9fe00d1bb0 con 0x7f9fe00c9660 2023-11-26T07:11:30.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.541+0000 7f9fc77fe700 1 -- 172.21.15.83:0/1019496416 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9fd8014350 con 0x7f9fe00c9660 2023-11-26T07:11:30.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 -- 172.21.15.83:0/1019496416 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9fcc06aed0 msgr2=0x7f9fcc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:30.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9fcc06aed0 0x7f9fcc06d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009940 tx=0x7f9fd4008040 comp rx=0 tx=0).stop 2023-11-26T07:11:30.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 -- 172.21.15.83:0/1019496416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe00c9660 msgr2=0x7f9fe00c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:30.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe00c9660 0x7f9fe00c9ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00b6df0 tx=0x7f9fd80114b0 comp rx=0 tx=0).stop 2023-11-26T07:11:30.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 -- 172.21.15.83:0/1019496416 shutdown_connections 2023-11-26T07:11:30.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe00c9660 0x7f9fe00c9ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:30.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9fcc06aed0 0x7f9fcc06d390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:30.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fc8004850 0x7f9fe00b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:30.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 --2- 172.21.15.83:0/1019496416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe00b42a0 0x7f9fe015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:30.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 -- 172.21.15.83:0/1019496416 >> 172.21.15.83:0/1019496416 conn(0x7f9fe006b410 msgr2=0x7f9fe00ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:30.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 -- 172.21.15.83:0/1019496416 shutdown_connections 2023-11-26T07:11:30.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.544+0000 7f9fe56e4700 1 -- 172.21.15.83:0/1019496416 wait complete. 2023-11-26T07:11:30.560 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:11:30.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:11:30.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.688+0000 7f6ec2824700 1 Processor -- start 2023-11-26T07:11:30.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.688+0000 7f6ec2824700 1 -- start start 2023-11-26T07:11:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.688+0000 7f6ec2824700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6ebc0746a0 0x7f6ebc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ec2824700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc0b3c60 con 0x7f6ebc0afc90 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ec2824700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc0b58c0 con 0x7f6ebc15e160 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ec2824700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc0b4500 con 0x7f6ebc0746a0 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ec0dc1700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6ebc0afc90 0x7f6ebc06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54116/0 (socket says 172.21.15.83:54116) 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ec0dc1700 1 -- 172.21.15.83:0/2618960024 learned_addr learned my addr 172.21.15.83:0/2618960024 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426567522 0 0) 0x7f6ebc0b3c60 con 0x7f6ebc0afc90 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebb7fe700 1 --2- 172.21.15.83:0/2618960024 >> v2:172.21.15.203:3300/0 conn(0x7f6ebc0746a0 0x7f6ebc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea40035b0 con 0x7f6ebc0afc90 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2536700618 0 0) 0x7f6ea40035b0 con 0x7f6ebc0afc90 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 >> v2:172.21.15.203:3300/0 conn(0x7f6ebc0746a0 msgr2=0x7f6ebc0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 --2- 172.21.15.83:0/2618960024 >> v2:172.21.15.203:3300/0 conn(0x7f6ebc0746a0 0x7f6ebc0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 >> v1:172.21.15.203:6789/0 conn(0x7f6ebc15e160 legacy=0x7f6ebc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc06bdc0 con 0x7f6ebc0afc90 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f6eb40019f0 con 0x7f6ebc0afc90 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 >> v1:172.21.15.83:6789/0 conn(0x7f6ebc0afc90 legacy=0x7f6ebc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 --2- 172.21.15.83:0/2618960024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4003a00 0x7f6ea4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 --2- 172.21.15.83:0/2618960024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea4004850 0x7f6ea4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 --2- 172.21.15.83:0/2618960024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ea4005210 0x7f6ea400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0b4500 con 0x7f6ea4003a00 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea400c0b0 con 0x7f6ea4004850 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea400c270 con 0x7f6ea4005210 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebbfff700 1 --2- 172.21.15.83:0/2618960024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4003a00 0x7f6ea4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ebb7fe700 1 --2- 172.21.15.83:0/2618960024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea4004850 0x7f6ea4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.689+0000 7f6ec0dc1700 1 --2- 172.21.15.83:0/2618960024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ea4005210 0x7f6ea400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ebbfff700 1 -- 172.21.15.83:0/2618960024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ea4005210 msgr2=0x7f6ea400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ebbfff700 1 --2- 172.21.15.83:0/2618960024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ea4005210 0x7f6ea400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ebbfff700 1 -- 172.21.15.83:0/2618960024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea4004850 msgr2=0x7f6ea4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ebbfff700 1 --2- 172.21.15.83:0/2618960024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea4004850 0x7f6ea4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ec0dc1700 1 -- 172.21.15.83:0/2618960024 reap_dead start 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ebbfff700 1 -- 172.21.15.83:0/2618960024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea40035b0 con 0x7f6ea4003a00 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ebbfff700 1 --2- 172.21.15.83:0/2618960024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4003a00 0x7f6ea4003e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0017c0 tx=0x7f6ea4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c47a53a16447a23 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0052d0 con 0x7f6ea4003a00 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac005430 con 0x7f6ea4003a00 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ebaffd700 1 -- 172.21.15.83:0/2618960024 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0038c0 con 0x7f6ea4003a00 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ec2824700 1 -- 172.21.15.83:0/2618960024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4003a00 msgr2=0x7f6ea4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:30.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ec2824700 1 --2- 172.21.15.83:0/2618960024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4003a00 0x7f6ea4003e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0017c0 tx=0x7f6ea4005880 comp rx=0 tx=0).stop 2023-11-26T07:11:30.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ec2824700 1 -- 172.21.15.83:0/2618960024 shutdown_connections 2023-11-26T07:11:30.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ec2824700 1 --2- 172.21.15.83:0/2618960024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ea4003a00 0x7f6ea4003e20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:30.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ec2824700 1 -- 172.21.15.83:0/2618960024 >> 172.21.15.83:0/2618960024 conn(0x7f6ebc0d0bf0 msgr2=0x7f6ebc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:30.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.690+0000 7f6ec2824700 1 -- 172.21.15.83:0/2618960024 shutdown_connections 2023-11-26T07:11:30.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.691+0000 7f6ec2824700 1 -- 172.21.15.83:0/2618960024 wait complete. 2023-11-26T07:11:30.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.692+0000 7f6ec2824700 1 Processor -- start 2023-11-26T07:11:30.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.692+0000 7f6ec2824700 1 -- start start 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec2824700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea4003a00 0x7f6ebc14e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec2824700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ebc161120 0x7f6ebc0c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec2824700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15a880 0x7f6ebc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec2824700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc15b250 con 0x7f6ebc15a880 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec2824700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc14ec70 con 0x7f6ea4003a00 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec2824700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0c6ee0 con 0x7f6ebc161120 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec0dc1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15a880 0x7f6ebc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec0dc1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15a880 0x7f6ebc15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34088/0 (socket says 172.21.15.83:34088) 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec0dc1700 1 -- 172.21.15.83:0/2469644296 learned_addr learned my addr 172.21.15.83:0/2469644296 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ebb7fe700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ebc161120 0x7f6ebc0c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ebbfff700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea4003a00 0x7f6ebc14e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:30.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec0dc1700 1 -- 172.21.15.83:0/2469644296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ebc161120 msgr2=0x7f6ebc0c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec0dc1700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ebc161120 0x7f6ebc0c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec0dc1700 1 -- 172.21.15.83:0/2469644296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea4003a00 msgr2=0x7f6ebc14e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec0dc1700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea4003a00 0x7f6ebc14e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec0dc1700 1 -- 172.21.15.83:0/2469644296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc0c71c0 con 0x7f6ebc15a880 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ebbfff700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea4003a00 0x7f6ebc14e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec0dc1700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15a880 0x7f6ebc15ad10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc075050 tx=0x7f6eb400a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6eb8ff9700 1 -- 172.21.15.83:0/2469644296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb400ad50 con 0x7f6ebc15a880 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6eb8ff9700 1 -- 172.21.15.83:0/2469644296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb40133f0 con 0x7f6ebc15a880 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6eb8ff9700 1 -- 172.21.15.83:0/2469644296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb40137a0 con 0x7f6ebc15a880 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec2824700 1 -- 172.21.15.83:0/2469644296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc0c74a0 con 0x7f6ebc15a880 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.693+0000 7f6ec2824700 1 -- 172.21.15.83:0/2469644296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc0c7a60 con 0x7f6ebc15a880 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.694+0000 7f6ec2824700 1 -- 172.21.15.83:0/2469644296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc0c7bf0 con 0x7f6ebc15a880 2023-11-26T07:11:30.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.695+0000 7f6eb8ff9700 1 -- 172.21.15.83:0/2469644296 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6eb4013900 con 0x7f6ebc15a880 2023-11-26T07:11:30.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.695+0000 7f6eb8ff9700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ea806aed0 0x7f6ea806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:30.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.695+0000 7f6eb8ff9700 1 -- 172.21.15.83:0/2469644296 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6eb408a0d0 con 0x7f6ebc15a880 2023-11-26T07:11:30.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.695+0000 7f6ebbfff700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ea806aed0 0x7f6ea806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:30.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.695+0000 7f6ebbfff700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ea806aed0 0x7f6ea806d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6eac005040 tx=0x7f6eac009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:30.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.699+0000 7f6eb8ff9700 1 -- 172.21.15.83:0/2469644296 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6eb4055210 con 0x7f6ebc15a880 2023-11-26T07:11:30.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:30.936+0000 7f6ec2824700 1 -- 172.21.15.83:0/2469644296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6ebc068fb0 con 0x7f6ebc15a880 2023-11-26T07:11:31.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.542+0000 7f6eb8ff9700 1 -- 172.21.15.83:0/2469644296 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6eb4058830 con 0x7f6ebc15a880 2023-11-26T07:11:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 -- 172.21.15.83:0/2469644296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ea806aed0 msgr2=0x7f6ea806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ea806aed0 0x7f6ea806d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6eac005040 tx=0x7f6eac009030 comp rx=0 tx=0).stop 2023-11-26T07:11:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 -- 172.21.15.83:0/2469644296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15a880 msgr2=0x7f6ebc15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15a880 0x7f6ebc15ad10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc075050 tx=0x7f6eb400a5e0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 -- 172.21.15.83:0/2469644296 shutdown_connections 2023-11-26T07:11:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ebc15a880 0x7f6ebc15ad10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ea4003a00 0x7f6ebc14e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6ea806aed0 0x7f6ea806d390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 --2- 172.21.15.83:0/2469644296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ebc161120 0x7f6ebc0c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 -- 172.21.15.83:0/2469644296 >> 172.21.15.83:0/2469644296 conn(0x7f6ebc0d0bf0 msgr2=0x7f6ebc0add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 -- 172.21.15.83:0/2469644296 shutdown_connections 2023-11-26T07:11:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.544+0000 7f6ec2824700 1 -- 172.21.15.83:0/2469644296 wait complete. 2023-11-26T07:11:31.559 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:11:31.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:11:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.681+0000 7fc02689d700 1 Processor -- start 2023-11-26T07:11:31.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.681+0000 7fc02689d700 1 -- start start 2023-11-26T07:11:31.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.682+0000 7fc02689d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc020152cc0 0x7fc02006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:31.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.682+0000 7fc02689d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0200b49a0 0x7fc02006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:31.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.682+0000 7fc02689d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0200cff20 0x7fc0200d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:31.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.682+0000 7fc02689d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02015c4b0 con 0x7fc0200b49a0 2023-11-26T07:11:31.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.682+0000 7fc02689d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02015c790 con 0x7fc020152cc0 2023-11-26T07:11:31.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.682+0000 7fc02689d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02015ca70 con 0x7fc0200cff20 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.682+0000 7fc01f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0200b49a0 0x7fc02006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.682+0000 7fc01f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0200b49a0 0x7fc02006c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34096/0 (socket says 172.21.15.83:34096) 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.682+0000 7fc01f7fe700 1 -- 172.21.15.83:0/7133632 learned_addr learned my addr 172.21.15.83:0/7133632 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc024e3a700 1 --2- 172.21.15.83:0/7133632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0200cff20 0x7fc0200d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc01ffff700 1 --2- 172.21.15.83:0/7133632 >> v2:172.21.15.203:3300/0 conn(0x7fc020152cc0 0x7fc02006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc01f7fe700 1 -- 172.21.15.83:0/7133632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0200cff20 msgr2=0x7fc0200d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc01f7fe700 1 --2- 172.21.15.83:0/7133632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0200cff20 0x7fc0200d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc01f7fe700 1 -- 172.21.15.83:0/7133632 >> v2:172.21.15.203:3300/0 conn(0x7fc020152cc0 msgr2=0x7fc02006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc01f7fe700 1 --2- 172.21.15.83:0/7133632 >> v2:172.21.15.203:3300/0 conn(0x7fc020152cc0 0x7fc02006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc01f7fe700 1 -- 172.21.15.83:0/7133632 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02015cd50 con 0x7fc0200b49a0 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc01f7fe700 1 --2- 172.21.15.83:0/7133632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0200b49a0 0x7fc02006c540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc020075030 tx=0x7fc014009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169f9cda37677f85 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:31.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc01effd700 1 -- 172.21.15.83:0/7133632 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01400b870 con 0x7fc0200b49a0 2023-11-26T07:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc01effd700 1 -- 172.21.15.83:0/7133632 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01400b9d0 con 0x7fc0200b49a0 2023-11-26T07:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc01effd700 1 -- 172.21.15.83:0/7133632 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01400bd60 con 0x7fc0200b49a0 2023-11-26T07:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc02689d700 1 -- 172.21.15.83:0/7133632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0200b49a0 msgr2=0x7fc02006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.683+0000 7fc02689d700 1 --2- 172.21.15.83:0/7133632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0200b49a0 0x7fc02006c540 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc020075030 tx=0x7fc014009bd0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.684+0000 7fc02689d700 1 -- 172.21.15.83:0/7133632 shutdown_connections 2023-11-26T07:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.684+0000 7fc02689d700 1 --2- 172.21.15.83:0/7133632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0200cff20 0x7fc0200d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.684+0000 7fc02689d700 1 --2- 172.21.15.83:0/7133632 >> v2:172.21.15.203:3300/0 conn(0x7fc020152cc0 0x7fc02006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.684+0000 7fc02689d700 1 --2- 172.21.15.83:0/7133632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0200b49a0 0x7fc02006c540 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.684+0000 7fc02689d700 1 -- 172.21.15.83:0/7133632 >> 172.21.15.83:0/7133632 conn(0x7fc020071b90 msgr2=0x7fc020071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.684+0000 7fc02689d700 1 -- 172.21.15.83:0/7133632 shutdown_connections 2023-11-26T07:11:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.684+0000 7fc02689d700 1 -- 172.21.15.83:0/7133632 wait complete. 2023-11-26T07:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.685+0000 7fc02689d700 1 Processor -- start 2023-11-26T07:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.685+0000 7fc02689d700 1 -- start start 2023-11-26T07:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc02689d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0200b49a0 0x7fc02013f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc02689d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0200cff20 0x7fc020159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc02689d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc020152cc0 0x7fc02015e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc02689d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc020158ce0 con 0x7fc0200b49a0 2023-11-26T07:11:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc02689d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc020158060 con 0x7fc020152cc0 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc02689d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc020158340 con 0x7fc0200cff20 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0200b49a0 0x7fc02013f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0200b49a0 0x7fc02013f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34098/0 (socket says 172.21.15.83:34098) 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01ffff700 1 -- 172.21.15.83:0/1037044797 learned_addr learned my addr 172.21.15.83:0/1037044797 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01f7fe700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0200cff20 0x7fc020159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc024e3a700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc020152cc0 0x7fc02015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01ffff700 1 -- 172.21.15.83:0/1037044797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0200cff20 msgr2=0x7fc020159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01ffff700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0200cff20 0x7fc020159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01ffff700 1 -- 172.21.15.83:0/1037044797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc020152cc0 msgr2=0x7fc02015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01ffff700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc020152cc0 0x7fc02015e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01ffff700 1 -- 172.21.15.83:0/1037044797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc014009550 con 0x7fc0200b49a0 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01f7fe700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0200cff20 0x7fc020159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc024e3a700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc020152cc0 0x7fc02015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01ffff700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0200b49a0 0x7fc02013f2d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d800 tx=0x7fc01000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:31.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01cff9700 1 -- 172.21.15.83:0/1037044797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0100125f0 con 0x7fc0200b49a0 2023-11-26T07:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01cff9700 1 -- 172.21.15.83:0/1037044797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc010012750 con 0x7fc0200b49a0 2023-11-26T07:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc02689d700 1 -- 172.21.15.83:0/1037044797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc020158620 con 0x7fc0200b49a0 2023-11-26T07:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc01cff9700 1 -- 172.21.15.83:0/1037044797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010012b00 con 0x7fc0200b49a0 2023-11-26T07:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.686+0000 7fc02689d700 1 -- 172.21.15.83:0/1037044797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02013f810 con 0x7fc0200b49a0 2023-11-26T07:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.687+0000 7fc02689d700 1 -- 172.21.15.83:0/1037044797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0200753f0 con 0x7fc0200b49a0 2023-11-26T07:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.688+0000 7fc01cff9700 1 -- 172.21.15.83:0/1037044797 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc010012c60 con 0x7fc0200b49a0 2023-11-26T07:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.688+0000 7fc01cff9700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc00c06aed0 0x7fc00c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.688+0000 7fc01cff9700 1 -- 172.21.15.83:0/1037044797 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc010089910 con 0x7fc0200b49a0 2023-11-26T07:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.688+0000 7fc01f7fe700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc00c06aed0 0x7fc00c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:31.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.689+0000 7fc01f7fe700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc00c06aed0 0x7fc00c06d390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc014000c00 tx=0x7fc014008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:31.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.693+0000 7fc01cff9700 1 -- 172.21.15.83:0/1037044797 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc010054a50 con 0x7fc0200b49a0 2023-11-26T07:11:31.913 INFO:tasks.ceph.mds.b.smithi203.stderr:2023-11-26T07:11:31.911+0000 7fd6aaa59700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:11:31.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.927+0000 7fc02689d700 1 -- 172.21.15.83:0/1037044797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc020067f60 con 0x7fc0200b49a0 2023-11-26T07:11:31.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.928+0000 7fc01cff9700 1 -- 172.21.15.83:0/1037044797 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fc010058070 con 0x7fc0200b49a0 2023-11-26T07:11:31.929 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:31.929 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2208741688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2208741688},{"type":"v1","addr":"172.21.15.83:6837","nonce":2208741688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2802126359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2802126359},{"type":"v1","addr":"172.21.15.83:6836","nonce":2802126359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1258871616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1258871616},{"type":"v1","addr":"172.21.15.203:6834","nonce":1258871616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:11:27.911475+0000","modified":"2023-11-26T07:11:30.937105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/2957324266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":2957324266},{"type":"v1","addr":"172.21.15.203:6835","nonce":2957324266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:11:31.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 -- 172.21.15.83:0/1037044797 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc00c06aed0 msgr2=0x7fc00c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:31.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc00c06aed0 0x7fc00c06d390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc014000c00 tx=0x7fc014008030 comp rx=0 tx=0).stop 2023-11-26T07:11:31.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 -- 172.21.15.83:0/1037044797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0200b49a0 msgr2=0x7fc02013f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:31.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0200b49a0 0x7fc02013f2d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d800 tx=0x7fc01000dbc0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 -- 172.21.15.83:0/1037044797 shutdown_connections 2023-11-26T07:11:31.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc020152cc0 0x7fc02015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0200b49a0 0x7fc02013f2d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc00c06aed0 0x7fc00c06d390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 --2- 172.21.15.83:0/1037044797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0200cff20 0x7fc020159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:31.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 -- 172.21.15.83:0/1037044797 >> 172.21.15.83:0/1037044797 conn(0x7fc020071b90 msgr2=0x7fc02006c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:31.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 -- 172.21.15.83:0/1037044797 shutdown_connections 2023-11-26T07:11:31.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:31.930+0000 7fc02689d700 1 -- 172.21.15.83:0/1037044797 wait complete. 2023-11-26T07:11:31.933 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2023-11-26T07:11:31.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.061+0000 7fc29f19b700 1 Processor -- start 2023-11-26T07:11:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.061+0000 7fc29f19b700 1 -- start start 2023-11-26T07:11:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29f19b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc29815bdd0 0x7fc2981604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29f19b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2980d1c70 0x7fc2980d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29f19b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2980b22d0 con 0x7fc2980b3370 2023-11-26T07:11:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29f19b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2980b2f50 con 0x7fc2980d1c70 2023-11-26T07:11:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29f19b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2980d2640 con 0x7fc29815bdd0 2023-11-26T07:11:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29d738700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2980d1c70 0x7fc2980d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29d738700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2980d1c70 0x7fc2980d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34110/0 (socket says 172.21.15.83:34110) 2023-11-26T07:11:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29d738700 1 -- 172.21.15.83:0/629553442 learned_addr learned my addr 172.21.15.83:0/629553442 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:32.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29cf37700 1 --2- 172.21.15.83:0/629553442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc29815bdd0 0x7fc2981604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29d738700 1 -- 172.21.15.83:0/629553442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc29815bdd0 msgr2=0x7fc2981604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:32.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29d738700 1 --2- 172.21.15.83:0/629553442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc29815bdd0 0x7fc2981604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29d738700 1 -- 172.21.15.83:0/629553442 >> v1:172.21.15.203:6789/0 conn(0x7fc2980b3370 legacy=0x7fc2980b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29d738700 1 -- 172.21.15.83:0/629553442 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2980b25b0 con 0x7fc2980d1c70 2023-11-26T07:11:32.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc29d738700 1 --2- 172.21.15.83:0/629553442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2980d1c70 0x7fc2980d2100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc290009e50 tx=0x7fc29000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52171cab0e5b253b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:32.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc2977fe700 1 -- 172.21.15.83:0/629553442 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29000e070 con 0x7fc2980d1c70 2023-11-26T07:11:32.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc2977fe700 1 -- 172.21.15.83:0/629553442 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29000b9c0 con 0x7fc2980d1c70 2023-11-26T07:11:32.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.062+0000 7fc2977fe700 1 -- 172.21.15.83:0/629553442 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29000bd70 con 0x7fc2980d1c70 2023-11-26T07:11:32.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.063+0000 7fc29f19b700 1 -- 172.21.15.83:0/629553442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2980d1c70 msgr2=0x7fc2980d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.063+0000 7fc29f19b700 1 --2- 172.21.15.83:0/629553442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2980d1c70 0x7fc2980d2100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc290009e50 tx=0x7fc29000b330 comp rx=0 tx=0).stop 2023-11-26T07:11:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.063+0000 7fc29f19b700 1 -- 172.21.15.83:0/629553442 shutdown_connections 2023-11-26T07:11:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.063+0000 7fc29f19b700 1 --2- 172.21.15.83:0/629553442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2980d1c70 0x7fc2980d2100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.063+0000 7fc29f19b700 1 --2- 172.21.15.83:0/629553442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc29815bdd0 0x7fc2981604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.063+0000 7fc29f19b700 1 -- 172.21.15.83:0/629553442 >> 172.21.15.83:0/629553442 conn(0x7fc2980731a0 msgr2=0x7fc29809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.063+0000 7fc29f19b700 1 -- 172.21.15.83:0/629553442 shutdown_connections 2023-11-26T07:11:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.063+0000 7fc29f19b700 1 -- 172.21.15.83:0/629553442 wait complete. 2023-11-26T07:11:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.064+0000 7fc29f19b700 1 Processor -- start 2023-11-26T07:11:32.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.064+0000 7fc29f19b700 1 -- start start 2023-11-26T07:11:32.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.065+0000 7fc29f19b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2980b3370 0x7fc298148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.065+0000 7fc29f19b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2980d1c70 0x7fc2980b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.065+0000 7fc29f19b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc29815bdd0 0x7fc298149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.065+0000 7fc29f19b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2980b5f80 con 0x7fc2980b3370 2023-11-26T07:11:32.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.065+0000 7fc29f19b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29815cd30 con 0x7fc29815bdd0 2023-11-26T07:11:32.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.065+0000 7fc29f19b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc298148fb0 con 0x7fc2980d1c70 2023-11-26T07:11:32.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.065+0000 7fc29cf37700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2980b3370 0x7fc298148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.065+0000 7fc29cf37700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2980b3370 0x7fc298148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34112/0 (socket says 172.21.15.83:34112) 2023-11-26T07:11:32.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.065+0000 7fc29cf37700 1 -- 172.21.15.83:0/1247589795 learned_addr learned my addr 172.21.15.83:0/1247589795 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.065+0000 7fc29d738700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc29815bdd0 0x7fc298149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.065+0000 7fc297fff700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2980d1c70 0x7fc2980b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc29cf37700 1 -- 172.21.15.83:0/1247589795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2980d1c70 msgr2=0x7fc2980b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc29cf37700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2980d1c70 0x7fc2980b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc29cf37700 1 -- 172.21.15.83:0/1247589795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc29815bdd0 msgr2=0x7fc298149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc29cf37700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc29815bdd0 0x7fc298149230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc29cf37700 1 -- 172.21.15.83:0/1247589795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc290009b30 con 0x7fc2980b3370 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc29d738700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc29815bdd0 0x7fc298149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc29cf37700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2980b3370 0x7fc298148a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00d830 tx=0x7fc28c00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc2957fa700 1 -- 172.21.15.83:0/1247589795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0125f0 con 0x7fc2980b3370 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc2957fa700 1 -- 172.21.15.83:0/1247589795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c012750 con 0x7fc2980b3370 2023-11-26T07:11:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc29f19b700 1 -- 172.21.15.83:0/1247589795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29815c510 con 0x7fc2980b3370 2023-11-26T07:11:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc2957fa700 1 -- 172.21.15.83:0/1247589795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c012b00 con 0x7fc2980b3370 2023-11-26T07:11:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.066+0000 7fc29f19b700 1 -- 172.21.15.83:0/1247589795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2980b5ba0 con 0x7fc2980b3370 2023-11-26T07:11:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.067+0000 7fc29f19b700 1 -- 172.21.15.83:0/1247589795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc298067f60 con 0x7fc2980b3370 2023-11-26T07:11:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.067+0000 7fc2957fa700 1 -- 172.21.15.83:0/1247589795 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc28c012c60 con 0x7fc2980b3370 2023-11-26T07:11:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.067+0000 7fc2957fa700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc27c06aed0 0x7fc27c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.068+0000 7fc2957fa700 1 -- 172.21.15.83:0/1247589795 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc28c0899b0 con 0x7fc2980b3370 2023-11-26T07:11:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.068+0000 7fc297fff700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc27c06aed0 0x7fc27c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.068+0000 7fc297fff700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc27c06aed0 0x7fc27c06d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc298070bf0 tx=0x7fc284009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:32.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.072+0000 7fc2957fa700 1 -- 172.21.15.83:0/1247589795 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc28c054a60 con 0x7fc2980b3370 2023-11-26T07:11:32.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.264+0000 7fc29f19b700 1 -- 172.21.15.83:0/1247589795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2980b60c0 con 0x7fc2980b3370 2023-11-26T07:11:32.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.265+0000 7fc2957fa700 1 -- 172.21.15.83:0/1247589795 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fc28c058080 con 0x7fc2980b3370 2023-11-26T07:11:32.267 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:32.267 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 -- 172.21.15.83:0/1247589795 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc27c06aed0 msgr2=0x7fc27c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc27c06aed0 0x7fc27c06d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc298070bf0 tx=0x7fc284009040 comp rx=0 tx=0).stop 2023-11-26T07:11:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 -- 172.21.15.83:0/1247589795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2980b3370 msgr2=0x7fc298148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2980b3370 0x7fc298148a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00d830 tx=0x7fc28c00dbf0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 -- 172.21.15.83:0/1247589795 shutdown_connections 2023-11-26T07:11:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc29815bdd0 0x7fc298149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2980b3370 0x7fc298148a70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc27c06aed0 0x7fc27c06d390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 --2- 172.21.15.83:0/1247589795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2980d1c70 0x7fc2980b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 -- 172.21.15.83:0/1247589795 >> 172.21.15.83:0/1247589795 conn(0x7fc2980731a0 msgr2=0x7fc2980ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 -- 172.21.15.83:0/1247589795 shutdown_connections 2023-11-26T07:11:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.267+0000 7fc29f19b700 1 -- 172.21.15.83:0/1247589795 wait complete. 2023-11-26T07:11:32.281 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:11:32.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:11:32.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.398+0000 7f7718087700 1 Processor -- start 2023-11-26T07:11:32.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.399+0000 7f7718087700 1 -- start start 2023-11-26T07:11:32.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.399+0000 7f7718087700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f771006c710 0x7f77100b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.399+0000 7f7718087700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77100af4d0 0x7f77100afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7718087700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77100ae430 con 0x7f771015e080 2023-11-26T07:11:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7718087700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77100af0b0 con 0x7f771006c710 2023-11-26T07:11:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7718087700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77100b05a0 con 0x7f77100af4d0 2023-11-26T07:11:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7715622700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f771006c710 0x7f77100b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7715e23700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f771015e080 0x7f771006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54160/0 (socket says 172.21.15.83:54160) 2023-11-26T07:11:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7715e23700 1 -- 172.21.15.83:0/2904722002 learned_addr learned my addr 172.21.15.83:0/2904722002 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7716624700 1 --2- 172.21.15.83:0/2904722002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77100af4d0 0x7f77100afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7715622700 1 -- 172.21.15.83:0/2904722002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77100af4d0 msgr2=0x7f77100afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7715622700 1 --2- 172.21.15.83:0/2904722002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77100af4d0 0x7f77100afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7715622700 1 -- 172.21.15.83:0/2904722002 >> v1:172.21.15.83:6789/0 conn(0x7f771015e080 legacy=0x7f771006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7715622700 1 -- 172.21.15.83:0/2904722002 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77100ae710 con 0x7f771006c710 2023-11-26T07:11:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7715622700 1 --2- 172.21.15.83:0/2904722002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f771006c710 0x7f77100b5050 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f770c009840 tx=0x7f770c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97680fad841cb780 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7714e21700 1 -- 172.21.15.83:0/2904722002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c011830 con 0x7f771006c710 2023-11-26T07:11:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7714e21700 1 -- 172.21.15.83:0/2904722002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c011990 con 0x7f771006c710 2023-11-26T07:11:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.400+0000 7f7714e21700 1 -- 172.21.15.83:0/2904722002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c011d20 con 0x7f771006c710 2023-11-26T07:11:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.401+0000 7f7718087700 1 -- 172.21.15.83:0/2904722002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f771006c710 msgr2=0x7f77100b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.401+0000 7f7718087700 1 --2- 172.21.15.83:0/2904722002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f771006c710 0x7f77100b5050 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f770c009840 tx=0x7f770c009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.401+0000 7f7718087700 1 -- 172.21.15.83:0/2904722002 shutdown_connections 2023-11-26T07:11:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.401+0000 7f7718087700 1 --2- 172.21.15.83:0/2904722002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77100af4d0 0x7f77100afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.401+0000 7f7718087700 1 --2- 172.21.15.83:0/2904722002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f771006c710 0x7f77100b5050 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.401+0000 7f7718087700 1 -- 172.21.15.83:0/2904722002 >> 172.21.15.83:0/2904722002 conn(0x7f7710071b90 msgr2=0x7f7710071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:32.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.401+0000 7f7718087700 1 -- 172.21.15.83:0/2904722002 shutdown_connections 2023-11-26T07:11:32.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.401+0000 7f7718087700 1 -- 172.21.15.83:0/2904722002 wait complete. 2023-11-26T07:11:32.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.402+0000 7f7718087700 1 Processor -- start 2023-11-26T07:11:32.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.402+0000 7f7718087700 1 -- start start 2023-11-26T07:11:32.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7718087700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771006c710 0x7f7710141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7718087700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77100af4d0 0x7f7710155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7718087700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771015e080 0x7f7710144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7718087700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7710157210 con 0x7f77100af4d0 2023-11-26T07:11:32.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7718087700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7710156590 con 0x7f771015e080 2023-11-26T07:11:32.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7718087700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7710156870 con 0x7f771006c710 2023-11-26T07:11:32.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7715622700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77100af4d0 0x7f7710155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7715622700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77100af4d0 0x7f7710155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34128/0 (socket says 172.21.15.83:34128) 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7715622700 1 -- 172.21.15.83:0/1973483132 learned_addr learned my addr 172.21.15.83:0/1973483132 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7716624700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771015e080 0x7f7710144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7715e23700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771006c710 0x7f7710141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7715622700 1 -- 172.21.15.83:0/1973483132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771006c710 msgr2=0x7f7710141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7715622700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771006c710 0x7f7710141e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7715622700 1 -- 172.21.15.83:0/1973483132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771015e080 msgr2=0x7f7710144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7715622700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771015e080 0x7f7710144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7715622700 1 -- 172.21.15.83:0/1973483132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c0094a0 con 0x7f77100af4d0 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7716624700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771015e080 0x7f7710144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7715622700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77100af4d0 0x7f7710155cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f770c0090b0 tx=0x7f770c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f77067fc700 1 -- 172.21.15.83:0/1973483132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c0038a0 con 0x7f77100af4d0 2023-11-26T07:11:32.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f77067fc700 1 -- 172.21.15.83:0/1973483132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c01f070 con 0x7f77100af4d0 2023-11-26T07:11:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7718087700 1 -- 172.21.15.83:0/1973483132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7710156b50 con 0x7f77100af4d0 2023-11-26T07:11:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f77067fc700 1 -- 172.21.15.83:0/1973483132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c003c60 con 0x7f77100af4d0 2023-11-26T07:11:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.403+0000 7f7718087700 1 -- 172.21.15.83:0/1973483132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7710156230 con 0x7f77100af4d0 2023-11-26T07:11:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.404+0000 7f7718087700 1 -- 172.21.15.83:0/1973483132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77100753f0 con 0x7f77100af4d0 2023-11-26T07:11:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.405+0000 7f77067fc700 1 -- 172.21.15.83:0/1973483132 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f770c01a430 con 0x7f77100af4d0 2023-11-26T07:11:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.405+0000 7f77067fc700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f76f406aed0 0x7f76f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.405+0000 7f7715e23700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f76f406aed0 0x7f76f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.405+0000 7f77067fc700 1 -- 172.21.15.83:0/1973483132 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f770c021070 con 0x7f77100af4d0 2023-11-26T07:11:32.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.406+0000 7f7715e23700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f76f406aed0 0x7f76f406d390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7700005910 tx=0x7f7700009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.410+0000 7f77067fc700 1 -- 172.21.15.83:0/1973483132 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f770c056170 con 0x7f77100af4d0 2023-11-26T07:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.598+0000 7f7718087700 1 -- 172.21.15.83:0/1973483132 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f77100aecd0 con 0x7f76f406aed0 2023-11-26T07:11:32.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.606+0000 7f77067fc700 1 -- 172.21.15.83:0/1973483132 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264636 (secure 0 0 0) 0x7f7700005940 con 0x7f76f406aed0 2023-11-26T07:11:32.610 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 -- 172.21.15.83:0/1973483132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f76f406aed0 msgr2=0x7f76f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f76f406aed0 0x7f76f406d390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7700005910 tx=0x7f7700009040 comp rx=0 tx=0).stop 2023-11-26T07:11:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 -- 172.21.15.83:0/1973483132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77100af4d0 msgr2=0x7f7710155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77100af4d0 0x7f7710155cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f770c0090b0 tx=0x7f770c004750 comp rx=0 tx=0).stop 2023-11-26T07:11:32.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 -- 172.21.15.83:0/1973483132 shutdown_connections 2023-11-26T07:11:32.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f771015e080 0x7f7710144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f76f406aed0 0x7f76f406d390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771006c710 0x7f7710141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 --2- 172.21.15.83:0/1973483132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77100af4d0 0x7f7710155cf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 -- 172.21.15.83:0/1973483132 >> 172.21.15.83:0/1973483132 conn(0x7f7710071b90 msgr2=0x7f77100b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:32.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 -- 172.21.15.83:0/1973483132 shutdown_connections 2023-11-26T07:11:32.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.609+0000 7f7718087700 1 -- 172.21.15.83:0/1973483132 wait complete. 2023-11-26T07:11:32.613 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:11:32.622 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-26T07:11:32.307880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219040,"kb_used_data":2848,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755680,"statfs":{"total":773094113280,"available":772869816320,"internally_reserved":0,"allocated":2916352,"data_stored":1752404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.803952"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540665+0000","last_change":"2023-11-26T07:11:28.537635+0000","last_active":"2023-11-26T07:11:29.540665+0000","last_peered":"2023-11-26T07:11:29.540665+0000","last_clean":"2023-11-26T07:11:29.540665+0000","last_became_active":"2023-11-26T07:11:28.537327+0000","last_became_peered":"2023-11-26T07:11:28.537327+0000","last_unstale":"2023-11-26T07:11:29.540665+0000","last_undegraded":"2023-11-26T07:11:29.540665+0000","last_fullsized":"2023-11-26T07:11:29.540665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:48:59.913650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537894+0000","last_change":"2023-11-26T07:11:28.541611+0000","last_active":"2023-11-26T07:11:29.537894+0000","last_peered":"2023-11-26T07:11:29.537894+0000","last_clean":"2023-11-26T07:11:29.537894+0000","last_became_active":"2023-11-26T07:11:28.541267+0000","last_became_peered":"2023-11-26T07:11:28.541267+0000","last_unstale":"2023-11-26T07:11:29.537894+0000","last_undegraded":"2023-11-26T07:11:29.537894+0000","last_fullsized":"2023-11-26T07:11:29.537894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189176+0000","last_change":"2023-11-26T07:11:28.543463+0000","last_active":"2023-11-26T07:11:30.189176+0000","last_peered":"2023-11-26T07:11:30.189176+0000","last_clean":"2023-11-26T07:11:30.189176+0000","last_became_active":"2023-11-26T07:11:28.542718+0000","last_became_peered":"2023-11-26T07:11:28.542718+0000","last_unstale":"2023-11-26T07:11:30.189176+0000","last_undegraded":"2023-11-26T07:11:30.189176+0000","last_fullsized":"2023-11-26T07:11:30.189176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.553883+0000","last_change":"2023-11-26T07:11:28.544447+0000","last_active":"2023-11-26T07:11:28.553883+0000","last_peered":"2023-11-26T07:11:28.553883+0000","last_clean":"2023-11-26T07:11:28.553883+0000","last_became_active":"2023-11-26T07:11:28.543689+0000","last_became_peered":"2023-11-26T07:11:28.543689+0000","last_unstale":"2023-11-26T07:11:28.553883+0000","last_undegraded":"2023-11-26T07:11:28.553883+0000","last_fullsized":"2023-11-26T07:11:28.553883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:53.475994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540289+0000","last_change":"2023-11-26T07:11:28.542518+0000","last_active":"2023-11-26T07:11:29.540289+0000","last_peered":"2023-11-26T07:11:29.540289+0000","last_clean":"2023-11-26T07:11:29.540289+0000","last_became_active":"2023-11-26T07:11:28.542255+0000","last_became_peered":"2023-11-26T07:11:28.542255+0000","last_unstale":"2023-11-26T07:11:29.540289+0000","last_undegraded":"2023-11-26T07:11:29.540289+0000","last_fullsized":"2023-11-26T07:11:29.540289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:48.717411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555860+0000","last_change":"2023-11-26T07:11:28.550053+0000","last_active":"2023-11-26T07:11:28.555860+0000","last_peered":"2023-11-26T07:11:28.555860+0000","last_clean":"2023-11-26T07:11:28.555860+0000","last_became_active":"2023-11-26T07:11:28.549624+0000","last_became_peered":"2023-11-26T07:11:28.549624+0000","last_unstale":"2023-11-26T07:11:28.555860+0000","last_undegraded":"2023-11-26T07:11:28.555860+0000","last_fullsized":"2023-11-26T07:11:28.555860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539979+0000","last_change":"2023-11-26T07:11:28.543140+0000","last_active":"2023-11-26T07:11:29.539979+0000","last_peered":"2023-11-26T07:11:29.539979+0000","last_clean":"2023-11-26T07:11:29.539979+0000","last_became_active":"2023-11-26T07:11:28.542835+0000","last_became_peered":"2023-11-26T07:11:28.542835+0000","last_unstale":"2023-11-26T07:11:29.539979+0000","last_undegraded":"2023-11-26T07:11:29.539979+0000","last_fullsized":"2023-11-26T07:11:29.539979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:14.369275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.562130+0000","last_change":"2023-11-26T07:11:28.554561+0000","last_active":"2023-11-26T07:11:28.562130+0000","last_peered":"2023-11-26T07:11:28.562130+0000","last_clean":"2023-11-26T07:11:28.562130+0000","last_became_active":"2023-11-26T07:11:28.554273+0000","last_became_peered":"2023-11-26T07:11:28.554273+0000","last_unstale":"2023-11-26T07:11:28.562130+0000","last_undegraded":"2023-11-26T07:11:28.562130+0000","last_fullsized":"2023-11-26T07:11:28.562130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:51.244423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540623+0000","last_change":"2023-11-26T07:11:28.539323+0000","last_active":"2023-11-26T07:11:29.540623+0000","last_peered":"2023-11-26T07:11:29.540623+0000","last_clean":"2023-11-26T07:11:29.540623+0000","last_became_active":"2023-11-26T07:11:28.539047+0000","last_became_peered":"2023-11-26T07:11:28.539047+0000","last_unstale":"2023-11-26T07:11:29.540623+0000","last_undegraded":"2023-11-26T07:11:29.540623+0000","last_fullsized":"2023-11-26T07:11:29.540623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:01:36.280118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.543260+0000","last_change":"2023-11-26T07:11:28.538383+0000","last_active":"2023-11-26T07:11:28.543260+0000","last_peered":"2023-11-26T07:11:28.543260+0000","last_clean":"2023-11-26T07:11:28.543260+0000","last_became_active":"2023-11-26T07:11:28.537726+0000","last_became_peered":"2023-11-26T07:11:28.537726+0000","last_unstale":"2023-11-26T07:11:28.543260+0000","last_undegraded":"2023-11-26T07:11:28.543260+0000","last_fullsized":"2023-11-26T07:11:28.543260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:32.075747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.546268+0000","last_change":"2023-11-26T07:11:28.538846+0000","last_active":"2023-11-26T07:11:28.546268+0000","last_peered":"2023-11-26T07:11:28.546268+0000","last_clean":"2023-11-26T07:11:28.546268+0000","last_became_active":"2023-11-26T07:11:28.537769+0000","last_became_peered":"2023-11-26T07:11:28.537769+0000","last_unstale":"2023-11-26T07:11:28.546268+0000","last_undegraded":"2023-11-26T07:11:28.546268+0000","last_fullsized":"2023-11-26T07:11:28.546268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:19.228362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539796+0000","last_change":"2023-11-26T07:11:28.541985+0000","last_active":"2023-11-26T07:11:29.539796+0000","last_peered":"2023-11-26T07:11:29.539796+0000","last_clean":"2023-11-26T07:11:29.539796+0000","last_became_active":"2023-11-26T07:11:28.541404+0000","last_became_peered":"2023-11-26T07:11:28.541404+0000","last_unstale":"2023-11-26T07:11:29.539796+0000","last_undegraded":"2023-11-26T07:11:29.539796+0000","last_fullsized":"2023-11-26T07:11:29.539796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:11.545490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.561691+0000","last_change":"2023-11-26T07:11:28.551585+0000","last_active":"2023-11-26T07:11:28.561691+0000","last_peered":"2023-11-26T07:11:28.561691+0000","last_clean":"2023-11-26T07:11:28.561691+0000","last_became_active":"2023-11-26T07:11:28.551296+0000","last_became_peered":"2023-11-26T07:11:28.551296+0000","last_unstale":"2023-11-26T07:11:28.561691+0000","last_undegraded":"2023-11-26T07:11:28.561691+0000","last_fullsized":"2023-11-26T07:11:28.561691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:20.471456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189126+0000","last_change":"2023-11-26T07:11:28.541346+0000","last_active":"2023-11-26T07:11:30.189126+0000","last_peered":"2023-11-26T07:11:30.189126+0000","last_clean":"2023-11-26T07:11:30.189126+0000","last_became_active":"2023-11-26T07:11:28.540961+0000","last_became_peered":"2023-11-26T07:11:28.540961+0000","last_unstale":"2023-11-26T07:11:30.189126+0000","last_undegraded":"2023-11-26T07:11:30.189126+0000","last_fullsized":"2023-11-26T07:11:30.189126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:48.999647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537528+0000","last_change":"2023-11-26T07:11:28.536348+0000","last_active":"2023-11-26T07:11:29.537528+0000","last_peered":"2023-11-26T07:11:29.537528+0000","last_clean":"2023-11-26T07:11:29.537528+0000","last_became_active":"2023-11-26T07:11:28.535391+0000","last_became_peered":"2023-11-26T07:11:28.535391+0000","last_unstale":"2023-11-26T07:11:29.537528+0000","last_undegraded":"2023-11-26T07:11:29.537528+0000","last_fullsized":"2023-11-26T07:11:29.537528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.553977+0000","last_change":"2023-11-26T07:11:28.544579+0000","last_active":"2023-11-26T07:11:28.553977+0000","last_peered":"2023-11-26T07:11:28.553977+0000","last_clean":"2023-11-26T07:11:28.553977+0000","last_became_active":"2023-11-26T07:11:28.542480+0000","last_became_peered":"2023-11-26T07:11:28.542480+0000","last_unstale":"2023-11-26T07:11:28.553977+0000","last_undegraded":"2023-11-26T07:11:28.553977+0000","last_fullsized":"2023-11-26T07:11:28.553977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:22.511330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.556798+0000","last_change":"2023-11-26T07:11:28.554495+0000","last_active":"2023-11-26T07:11:28.556798+0000","last_peered":"2023-11-26T07:11:28.556798+0000","last_clean":"2023-11-26T07:11:28.556798+0000","last_became_active":"2023-11-26T07:11:28.554114+0000","last_became_peered":"2023-11-26T07:11:28.554114+0000","last_unstale":"2023-11-26T07:11:28.556798+0000","last_undegraded":"2023-11-26T07:11:28.556798+0000","last_fullsized":"2023-11-26T07:11:28.556798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:26.403994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.551941+0000","last_change":"2023-11-26T07:11:28.544698+0000","last_active":"2023-11-26T07:11:28.551941+0000","last_peered":"2023-11-26T07:11:28.551941+0000","last_clean":"2023-11-26T07:11:28.551941+0000","last_became_active":"2023-11-26T07:11:28.544347+0000","last_became_peered":"2023-11-26T07:11:28.544347+0000","last_unstale":"2023-11-26T07:11:28.551941+0000","last_undegraded":"2023-11-26T07:11:28.551941+0000","last_fullsized":"2023-11-26T07:11:28.551941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:46.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540518+0000","last_change":"2023-11-26T07:11:28.550524+0000","last_active":"2023-11-26T07:11:29.540518+0000","last_peered":"2023-11-26T07:11:29.540518+0000","last_clean":"2023-11-26T07:11:29.540518+0000","last_became_active":"2023-11-26T07:11:28.543803+0000","last_became_peered":"2023-11-26T07:11:28.543803+0000","last_unstale":"2023-11-26T07:11:29.540518+0000","last_undegraded":"2023-11-26T07:11:29.540518+0000","last_fullsized":"2023-11-26T07:11:29.540518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:36:40.660123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189251+0000","last_change":"2023-11-26T07:11:27.524720+0000","last_active":"2023-11-26T07:11:30.189251+0000","last_peered":"2023-11-26T07:11:30.189251+0000","last_clean":"2023-11-26T07:11:30.189251+0000","last_became_active":"2023-11-26T07:11:27.524424+0000","last_became_peered":"2023-11-26T07:11:27.524424+0000","last_unstale":"2023-11-26T07:11:30.189251+0000","last_undegraded":"2023-11-26T07:11:30.189251+0000","last_fullsized":"2023-11-26T07:11:30.189251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.542197+0000","last_change":"2023-11-26T07:11:28.540843+0000","last_active":"2023-11-26T07:11:28.542197+0000","last_peered":"2023-11-26T07:11:28.542197+0000","last_clean":"2023-11-26T07:11:28.542197+0000","last_became_active":"2023-11-26T07:11:28.540579+0000","last_became_peered":"2023-11-26T07:11:28.540579+0000","last_unstale":"2023-11-26T07:11:28.542197+0000","last_undegraded":"2023-11-26T07:11:28.542197+0000","last_fullsized":"2023-11-26T07:11:28.542197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:33.426915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.561729+0000","last_change":"2023-11-26T07:11:27.528053+0000","last_active":"2023-11-26T07:11:28.561729+0000","last_peered":"2023-11-26T07:11:28.561729+0000","last_clean":"2023-11-26T07:11:28.561729+0000","last_became_active":"2023-11-26T07:11:27.526741+0000","last_became_peered":"2023-11-26T07:11:27.526741+0000","last_unstale":"2023-11-26T07:11:28.561729+0000","last_undegraded":"2023-11-26T07:11:28.561729+0000","last_fullsized":"2023-11-26T07:11:28.561729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:31.071342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189238+0000","last_change":"2023-11-26T07:11:28.551779+0000","last_active":"2023-11-26T07:11:30.189238+0000","last_peered":"2023-11-26T07:11:30.189238+0000","last_clean":"2023-11-26T07:11:30.189238+0000","last_became_active":"2023-11-26T07:11:28.551489+0000","last_became_peered":"2023-11-26T07:11:28.551489+0000","last_unstale":"2023-11-26T07:11:30.189238+0000","last_undegraded":"2023-11-26T07:11:30.189238+0000","last_fullsized":"2023-11-26T07:11:30.189238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540841+0000","last_change":"2023-11-26T07:11:27.530905+0000","last_active":"2023-11-26T07:11:29.540841+0000","last_peered":"2023-11-26T07:11:29.540841+0000","last_clean":"2023-11-26T07:11:29.540841+0000","last_became_active":"2023-11-26T07:11:27.529568+0000","last_became_peered":"2023-11-26T07:11:27.529568+0000","last_unstale":"2023-11-26T07:11:29.540841+0000","last_undegraded":"2023-11-26T07:11:29.540841+0000","last_fullsized":"2023-11-26T07:11:29.540841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:12.389909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555768+0000","last_change":"2023-11-26T07:11:28.547553+0000","last_active":"2023-11-26T07:11:28.555768+0000","last_peered":"2023-11-26T07:11:28.555768+0000","last_clean":"2023-11-26T07:11:28.555768+0000","last_became_active":"2023-11-26T07:11:28.547052+0000","last_became_peered":"2023-11-26T07:11:28.547052+0000","last_unstale":"2023-11-26T07:11:28.555768+0000","last_undegraded":"2023-11-26T07:11:28.555768+0000","last_fullsized":"2023-11-26T07:11:28.555768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:11.545490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540344+0000","last_change":"2023-11-26T07:11:27.528392+0000","last_active":"2023-11-26T07:11:29.540344+0000","last_peered":"2023-11-26T07:11:29.540344+0000","last_clean":"2023-11-26T07:11:29.540344+0000","last_became_active":"2023-11-26T07:11:27.528039+0000","last_became_peered":"2023-11-26T07:11:27.528039+0000","last_unstale":"2023-11-26T07:11:29.540344+0000","last_undegraded":"2023-11-26T07:11:29.540344+0000","last_fullsized":"2023-11-26T07:11:29.540344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.554133+0000","last_change":"2023-11-26T07:11:28.546262+0000","last_active":"2023-11-26T07:11:28.554133+0000","last_peered":"2023-11-26T07:11:28.554133+0000","last_clean":"2023-11-26T07:11:28.554133+0000","last_became_active":"2023-11-26T07:11:28.545998+0000","last_became_peered":"2023-11-26T07:11:28.545998+0000","last_unstale":"2023-11-26T07:11:28.554133+0000","last_undegraded":"2023-11-26T07:11:28.554133+0000","last_fullsized":"2023-11-26T07:11:28.554133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:17.562000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.561753+0000","last_change":"2023-11-26T07:11:27.528591+0000","last_active":"2023-11-26T07:11:28.561753+0000","last_peered":"2023-11-26T07:11:28.561753+0000","last_clean":"2023-11-26T07:11:28.561753+0000","last_became_active":"2023-11-26T07:11:27.528327+0000","last_became_peered":"2023-11-26T07:11:27.528327+0000","last_unstale":"2023-11-26T07:11:28.561753+0000","last_undegraded":"2023-11-26T07:11:28.561753+0000","last_fullsized":"2023-11-26T07:11:28.561753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:21.506925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539218+0000","last_change":"2023-11-26T07:11:28.541089+0000","last_active":"2023-11-26T07:11:29.539218+0000","last_peered":"2023-11-26T07:11:29.539218+0000","last_clean":"2023-11-26T07:11:29.539218+0000","last_became_active":"2023-11-26T07:11:28.534262+0000","last_became_peered":"2023-11-26T07:11:28.534262+0000","last_unstale":"2023-11-26T07:11:29.539218+0000","last_undegraded":"2023-11-26T07:11:29.539218+0000","last_fullsized":"2023-11-26T07:11:29.539218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:33.426915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549517+0000","last_change":"2023-11-26T07:11:27.530892+0000","last_active":"2023-11-26T07:11:28.549517+0000","last_peered":"2023-11-26T07:11:28.549517+0000","last_clean":"2023-11-26T07:11:28.549517+0000","last_became_active":"2023-11-26T07:11:27.529853+0000","last_became_peered":"2023-11-26T07:11:27.529853+0000","last_unstale":"2023-11-26T07:11:28.549517+0000","last_undegraded":"2023-11-26T07:11:28.549517+0000","last_fullsized":"2023-11-26T07:11:28.549517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189422+0000","last_change":"2023-11-26T07:11:28.548912+0000","last_active":"2023-11-26T07:11:30.189422+0000","last_peered":"2023-11-26T07:11:30.189422+0000","last_clean":"2023-11-26T07:11:30.189422+0000","last_became_active":"2023-11-26T07:11:28.547915+0000","last_became_peered":"2023-11-26T07:11:28.547915+0000","last_unstale":"2023-11-26T07:11:30.189422+0000","last_undegraded":"2023-11-26T07:11:30.189422+0000","last_fullsized":"2023-11-26T07:11:30.189422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:42.974584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.537113+0000","last_change":"2023-11-26T07:11:27.529609+0000","last_active":"2023-11-26T07:11:28.537113+0000","last_peered":"2023-11-26T07:11:28.537113+0000","last_clean":"2023-11-26T07:11:28.537113+0000","last_became_active":"2023-11-26T07:11:27.529045+0000","last_became_peered":"2023-11-26T07:11:27.529045+0000","last_unstale":"2023-11-26T07:11:28.537113+0000","last_undegraded":"2023-11-26T07:11:28.537113+0000","last_fullsized":"2023-11-26T07:11:28.537113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:18.223957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.544959+0000","last_change":"2023-11-26T07:11:28.543403+0000","last_active":"2023-11-26T07:11:28.544959+0000","last_peered":"2023-11-26T07:11:28.544959+0000","last_clean":"2023-11-26T07:11:28.544959+0000","last_became_active":"2023-11-26T07:11:28.543119+0000","last_became_peered":"2023-11-26T07:11:28.543119+0000","last_unstale":"2023-11-26T07:11:28.544959+0000","last_undegraded":"2023-11-26T07:11:28.544959+0000","last_fullsized":"2023-11-26T07:11:28.544959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:36.121867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.533451+0000","last_change":"2023-11-26T07:11:27.528106+0000","last_active":"2023-11-26T07:11:28.533451+0000","last_peered":"2023-11-26T07:11:28.533451+0000","last_clean":"2023-11-26T07:11:28.533451+0000","last_became_active":"2023-11-26T07:11:27.527832+0000","last_became_peered":"2023-11-26T07:11:27.527832+0000","last_unstale":"2023-11-26T07:11:28.533451+0000","last_undegraded":"2023-11-26T07:11:28.533451+0000","last_fullsized":"2023-11-26T07:11:28.533451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:50.240018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537690+0000","last_change":"2023-11-26T07:11:28.543380+0000","last_active":"2023-11-26T07:11:29.537690+0000","last_peered":"2023-11-26T07:11:29.537690+0000","last_clean":"2023-11-26T07:11:29.537690+0000","last_became_active":"2023-11-26T07:11:28.543083+0000","last_became_peered":"2023-11-26T07:11:28.543083+0000","last_unstale":"2023-11-26T07:11:29.537690+0000","last_undegraded":"2023-11-26T07:11:29.537690+0000","last_fullsized":"2023-11-26T07:11:29.537690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.531315+0000","last_change":"2023-11-26T07:11:27.531555+0000","last_active":"2023-11-26T07:11:28.531315+0000","last_peered":"2023-11-26T07:11:28.531315+0000","last_clean":"2023-11-26T07:11:28.531315+0000","last_became_active":"2023-11-26T07:11:27.531241+0000","last_became_peered":"2023-11-26T07:11:27.531241+0000","last_unstale":"2023-11-26T07:11:28.531315+0000","last_undegraded":"2023-11-26T07:11:28.531315+0000","last_fullsized":"2023-11-26T07:11:28.531315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555943+0000","last_change":"2023-11-26T07:11:28.552112+0000","last_active":"2023-11-26T07:11:28.555943+0000","last_peered":"2023-11-26T07:11:28.555943+0000","last_clean":"2023-11-26T07:11:28.555943+0000","last_became_active":"2023-11-26T07:11:28.551827+0000","last_became_peered":"2023-11-26T07:11:28.551827+0000","last_unstale":"2023-11-26T07:11:28.555943+0000","last_undegraded":"2023-11-26T07:11:28.555943+0000","last_fullsized":"2023-11-26T07:11:28.555943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:33.426915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.531252+0000","last_change":"2023-11-26T07:11:27.531668+0000","last_active":"2023-11-26T07:11:28.531252+0000","last_peered":"2023-11-26T07:11:28.531252+0000","last_clean":"2023-11-26T07:11:28.531252+0000","last_became_active":"2023-11-26T07:11:27.531403+0000","last_became_peered":"2023-11-26T07:11:27.531403+0000","last_unstale":"2023-11-26T07:11:28.531252+0000","last_undegraded":"2023-11-26T07:11:28.531252+0000","last_fullsized":"2023-11-26T07:11:28.531252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189378+0000","last_change":"2023-11-26T07:11:28.542823+0000","last_active":"2023-11-26T07:11:30.189378+0000","last_peered":"2023-11-26T07:11:30.189378+0000","last_clean":"2023-11-26T07:11:30.189378+0000","last_became_active":"2023-11-26T07:11:28.542535+0000","last_became_peered":"2023-11-26T07:11:28.542535+0000","last_unstale":"2023-11-26T07:11:30.189378+0000","last_undegraded":"2023-11-26T07:11:30.189378+0000","last_fullsized":"2023-11-26T07:11:30.189378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537747+0000","last_change":"2023-11-26T07:11:27.531081+0000","last_active":"2023-11-26T07:11:29.537747+0000","last_peered":"2023-11-26T07:11:29.537747+0000","last_clean":"2023-11-26T07:11:29.537747+0000","last_became_active":"2023-11-26T07:11:27.530823+0000","last_became_peered":"2023-11-26T07:11:27.530823+0000","last_unstale":"2023-11-26T07:11:29.537747+0000","last_undegraded":"2023-11-26T07:11:29.537747+0000","last_fullsized":"2023-11-26T07:11:29.537747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:21.506925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539660+0000","last_change":"2023-11-26T07:11:28.529675+0000","last_active":"2023-11-26T07:11:29.539660+0000","last_peered":"2023-11-26T07:11:29.539660+0000","last_clean":"2023-11-26T07:11:29.539660+0000","last_became_active":"2023-11-26T07:11:28.529388+0000","last_became_peered":"2023-11-26T07:11:28.529388+0000","last_unstale":"2023-11-26T07:11:29.539660+0000","last_undegraded":"2023-11-26T07:11:29.539660+0000","last_fullsized":"2023-11-26T07:11:29.539660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:32.075747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.563480+0000","last_change":"2023-11-26T07:11:27.534932+0000","last_active":"2023-11-26T07:11:28.563480+0000","last_peered":"2023-11-26T07:11:28.563480+0000","last_clean":"2023-11-26T07:11:28.563480+0000","last_became_active":"2023-11-26T07:11:27.534626+0000","last_became_peered":"2023-11-26T07:11:27.534626+0000","last_unstale":"2023-11-26T07:11:28.563480+0000","last_undegraded":"2023-11-26T07:11:28.563480+0000","last_fullsized":"2023-11-26T07:11:28.563480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:25.399589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540678+0000","last_change":"2023-11-26T07:11:28.538363+0000","last_active":"2023-11-26T07:11:29.540678+0000","last_peered":"2023-11-26T07:11:29.540678+0000","last_clean":"2023-11-26T07:11:29.540678+0000","last_became_active":"2023-11-26T07:11:28.538098+0000","last_became_peered":"2023-11-26T07:11:28.538098+0000","last_unstale":"2023-11-26T07:11:29.540678+0000","last_undegraded":"2023-11-26T07:11:29.540678+0000","last_fullsized":"2023-11-26T07:11:29.540678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:00.284371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189378+0000","last_change":"2023-11-26T07:11:27.522269+0000","last_active":"2023-11-26T07:11:30.189378+0000","last_peered":"2023-11-26T07:11:30.189378+0000","last_clean":"2023-11-26T07:11:30.189378+0000","last_became_active":"2023-11-26T07:11:27.521852+0000","last_became_peered":"2023-11-26T07:11:27.521852+0000","last_unstale":"2023-11-26T07:11:30.189378+0000","last_undegraded":"2023-11-26T07:11:30.189378+0000","last_fullsized":"2023-11-26T07:11:30.189378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189106+0000","last_change":"2023-11-26T07:11:28.552836+0000","last_active":"2023-11-26T07:11:30.189106+0000","last_peered":"2023-11-26T07:11:30.189106+0000","last_clean":"2023-11-26T07:11:30.189106+0000","last_became_active":"2023-11-26T07:11:28.552532+0000","last_became_peered":"2023-11-26T07:11:28.552532+0000","last_unstale":"2023-11-26T07:11:30.189106+0000","last_undegraded":"2023-11-26T07:11:30.189106+0000","last_fullsized":"2023-11-26T07:11:30.189106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:51.244423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.536282+0000","last_change":"2023-11-26T07:11:27.534390+0000","last_active":"2023-11-26T07:11:28.536282+0000","last_peered":"2023-11-26T07:11:28.536282+0000","last_clean":"2023-11-26T07:11:28.536282+0000","last_became_active":"2023-11-26T07:11:27.534100+0000","last_became_peered":"2023-11-26T07:11:27.534100+0000","last_unstale":"2023-11-26T07:11:28.536282+0000","last_undegraded":"2023-11-26T07:11:28.536282+0000","last_fullsized":"2023-11-26T07:11:28.536282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:15.199340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540363+0000","last_change":"2023-11-26T07:11:28.530599+0000","last_active":"2023-11-26T07:11:29.540363+0000","last_peered":"2023-11-26T07:11:29.540363+0000","last_clean":"2023-11-26T07:11:29.540363+0000","last_became_active":"2023-11-26T07:11:28.530091+0000","last_became_peered":"2023-11-26T07:11:28.530091+0000","last_unstale":"2023-11-26T07:11:29.540363+0000","last_undegraded":"2023-11-26T07:11:29.540363+0000","last_fullsized":"2023-11-26T07:11:29.540363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:21:12.239221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.530812+0000","last_change":"2023-11-26T07:11:27.530816+0000","last_active":"2023-11-26T07:11:28.530812+0000","last_peered":"2023-11-26T07:11:28.530812+0000","last_clean":"2023-11-26T07:11:28.530812+0000","last_became_active":"2023-11-26T07:11:27.530026+0000","last_became_peered":"2023-11-26T07:11:27.530026+0000","last_unstale":"2023-11-26T07:11:28.530812+0000","last_undegraded":"2023-11-26T07:11:28.530812+0000","last_fullsized":"2023-11-26T07:11:28.530812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:32.422510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.546897+0000","last_change":"2023-11-26T07:11:28.542511+0000","last_active":"2023-11-26T07:11:28.546897+0000","last_peered":"2023-11-26T07:11:28.546897+0000","last_clean":"2023-11-26T07:11:28.546897+0000","last_became_active":"2023-11-26T07:11:28.542221+0000","last_became_peered":"2023-11-26T07:11:28.542221+0000","last_unstale":"2023-11-26T07:11:28.546897+0000","last_undegraded":"2023-11-26T07:11:28.546897+0000","last_fullsized":"2023-11-26T07:11:28.546897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:53.475994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.556270+0000","last_change":"2023-11-26T07:11:27.524724+0000","last_active":"2023-11-26T07:11:28.556270+0000","last_peered":"2023-11-26T07:11:28.556270+0000","last_clean":"2023-11-26T07:11:28.556270+0000","last_became_active":"2023-11-26T07:11:27.524453+0000","last_became_peered":"2023-11-26T07:11:27.524453+0000","last_unstale":"2023-11-26T07:11:28.556270+0000","last_undegraded":"2023-11-26T07:11:28.556270+0000","last_fullsized":"2023-11-26T07:11:28.556270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:32.422510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540122+0000","last_change":"2023-11-26T07:11:28.530396+0000","last_active":"2023-11-26T07:11:29.540122+0000","last_peered":"2023-11-26T07:11:29.540122+0000","last_clean":"2023-11-26T07:11:29.540122+0000","last_became_active":"2023-11-26T07:11:28.530134+0000","last_became_peered":"2023-11-26T07:11:28.530134+0000","last_unstale":"2023-11-26T07:11:29.540122+0000","last_undegraded":"2023-11-26T07:11:29.540122+0000","last_fullsized":"2023-11-26T07:11:29.540122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:58.985237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539626+0000","last_change":"2023-11-26T07:11:27.526573+0000","last_active":"2023-11-26T07:11:29.539626+0000","last_peered":"2023-11-26T07:11:29.539626+0000","last_clean":"2023-11-26T07:11:29.539626+0000","last_became_active":"2023-11-26T07:11:27.526278+0000","last_became_peered":"2023-11-26T07:11:27.526278+0000","last_unstale":"2023-11-26T07:11:29.539626+0000","last_undegraded":"2023-11-26T07:11:29.539626+0000","last_fullsized":"2023-11-26T07:11:29.539626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:35.117462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540624+0000","last_change":"2023-11-26T07:11:28.530307+0000","last_active":"2023-11-26T07:11:29.540624+0000","last_peered":"2023-11-26T07:11:29.540624+0000","last_clean":"2023-11-26T07:11:29.540624+0000","last_became_active":"2023-11-26T07:11:28.530012+0000","last_became_peered":"2023-11-26T07:11:28.530012+0000","last_unstale":"2023-11-26T07:11:29.540624+0000","last_undegraded":"2023-11-26T07:11:29.540624+0000","last_fullsized":"2023-11-26T07:11:29.540624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:51.665872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537662+0000","last_change":"2023-11-26T07:11:27.525592+0000","last_active":"2023-11-26T07:11:29.537662+0000","last_peered":"2023-11-26T07:11:29.537662+0000","last_clean":"2023-11-26T07:11:29.537662+0000","last_became_active":"2023-11-26T07:11:27.525304+0000","last_became_peered":"2023-11-26T07:11:27.525304+0000","last_unstale":"2023-11-26T07:11:29.537662+0000","last_undegraded":"2023-11-26T07:11:29.537662+0000","last_fullsized":"2023-11-26T07:11:29.537662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:52.471590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.547994+0000","last_change":"2023-11-26T07:11:28.538490+0000","last_active":"2023-11-26T07:11:28.547994+0000","last_peered":"2023-11-26T07:11:28.547994+0000","last_clean":"2023-11-26T07:11:28.547994+0000","last_became_active":"2023-11-26T07:11:28.538185+0000","last_became_peered":"2023-11-26T07:11:28.538185+0000","last_unstale":"2023-11-26T07:11:28.547994+0000","last_undegraded":"2023-11-26T07:11:28.547994+0000","last_fullsized":"2023-11-26T07:11:28.547994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:32.075747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.536144+0000","last_change":"2023-11-26T07:11:27.524225+0000","last_active":"2023-11-26T07:11:28.536144+0000","last_peered":"2023-11-26T07:11:28.536144+0000","last_clean":"2023-11-26T07:11:28.536144+0000","last_became_active":"2023-11-26T07:11:27.523610+0000","last_became_peered":"2023-11-26T07:11:27.523610+0000","last_unstale":"2023-11-26T07:11:28.536144+0000","last_undegraded":"2023-11-26T07:11:28.536144+0000","last_fullsized":"2023-11-26T07:11:28.536144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:19.248482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539423+0000","last_change":"2023-11-26T07:11:28.547971+0000","last_active":"2023-11-26T07:11:29.539423+0000","last_peered":"2023-11-26T07:11:29.539423+0000","last_clean":"2023-11-26T07:11:29.539423+0000","last_became_active":"2023-11-26T07:11:28.547756+0000","last_became_peered":"2023-11-26T07:11:28.547756+0000","last_unstale":"2023-11-26T07:11:29.539423+0000","last_undegraded":"2023-11-26T07:11:29.539423+0000","last_fullsized":"2023-11-26T07:11:29.539423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:57.098847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537553+0000","last_change":"2023-11-26T07:11:28.542253+0000","last_active":"2023-11-26T07:11:29.537553+0000","last_peered":"2023-11-26T07:11:29.537553+0000","last_clean":"2023-11-26T07:11:29.537553+0000","last_became_active":"2023-11-26T07:11:28.541977+0000","last_became_peered":"2023-11-26T07:11:28.541977+0000","last_unstale":"2023-11-26T07:11:29.537553+0000","last_undegraded":"2023-11-26T07:11:29.537553+0000","last_fullsized":"2023-11-26T07:11:29.537553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.535254+0000","last_change":"2023-11-26T07:11:27.534895+0000","last_active":"2023-11-26T07:11:28.535254+0000","last_peered":"2023-11-26T07:11:28.535254+0000","last_clean":"2023-11-26T07:11:28.535254+0000","last_became_active":"2023-11-26T07:11:27.534654+0000","last_became_peered":"2023-11-26T07:11:27.534654+0000","last_unstale":"2023-11-26T07:11:28.535254+0000","last_undegraded":"2023-11-26T07:11:28.535254+0000","last_fullsized":"2023-11-26T07:11:28.535254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:18.223957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.545545+0000","last_change":"2023-11-26T07:11:28.544575+0000","last_active":"2023-11-26T07:11:28.545545+0000","last_peered":"2023-11-26T07:11:28.545545+0000","last_clean":"2023-11-26T07:11:28.545545+0000","last_became_active":"2023-11-26T07:11:28.543760+0000","last_became_peered":"2023-11-26T07:11:28.543760+0000","last_unstale":"2023-11-26T07:11:28.545545+0000","last_undegraded":"2023-11-26T07:11:28.545545+0000","last_fullsized":"2023-11-26T07:11:28.545545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:46.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.530620+0000","last_change":"2023-11-26T07:11:27.530887+0000","last_active":"2023-11-26T07:11:28.530620+0000","last_peered":"2023-11-26T07:11:28.530620+0000","last_clean":"2023-11-26T07:11:28.530620+0000","last_became_active":"2023-11-26T07:11:27.530613+0000","last_became_peered":"2023-11-26T07:11:27.530613+0000","last_unstale":"2023-11-26T07:11:28.530620+0000","last_undegraded":"2023-11-26T07:11:28.530620+0000","last_fullsized":"2023-11-26T07:11:28.530620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:35.117462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.525772+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:11:28.525772+0000","last_peered":"2023-11-26T07:11:28.525772+0000","last_clean":"2023-11-26T07:11:28.525772+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:11:28.525772+0000","last_undegraded":"2023-11-26T07:11:28.525772+0000","last_fullsized":"2023-11-26T07:11:28.525772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189105+0000","last_change":"2023-11-26T07:11:28.548666+0000","last_active":"2023-11-26T07:11:30.189105+0000","last_peered":"2023-11-26T07:11:30.189105+0000","last_clean":"2023-11-26T07:11:30.189105+0000","last_became_active":"2023-11-26T07:11:28.548390+0000","last_became_peered":"2023-11-26T07:11:28.548390+0000","last_unstale":"2023-11-26T07:11:30.189105+0000","last_undegraded":"2023-11-26T07:11:30.189105+0000","last_fullsized":"2023-11-26T07:11:30.189105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:26.403994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539972+0000","last_change":"2023-11-26T07:11:27.525655+0000","last_active":"2023-11-26T07:11:29.539972+0000","last_peered":"2023-11-26T07:11:29.539972+0000","last_clean":"2023-11-26T07:11:29.539972+0000","last_became_active":"2023-11-26T07:11:27.525321+0000","last_became_peered":"2023-11-26T07:11:27.525321+0000","last_unstale":"2023-11-26T07:11:29.539972+0000","last_undegraded":"2023-11-26T07:11:29.539972+0000","last_fullsized":"2023-11-26T07:11:29.539972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:47.995242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540819+0000","last_change":"2023-11-26T07:11:27.528743+0000","last_active":"2023-11-26T07:11:29.540819+0000","last_peered":"2023-11-26T07:11:29.540819+0000","last_clean":"2023-11-26T07:11:29.540819+0000","last_became_active":"2023-11-26T07:11:27.527803+0000","last_became_peered":"2023-11-26T07:11:27.527803+0000","last_unstale":"2023-11-26T07:11:29.540819+0000","last_undegraded":"2023-11-26T07:11:29.540819+0000","last_fullsized":"2023-11-26T07:11:29.540819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:16:53.693262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537674+0000","last_change":"2023-11-26T07:11:28.542614+0000","last_active":"2023-11-26T07:11:29.537674+0000","last_peered":"2023-11-26T07:11:29.537674+0000","last_clean":"2023-11-26T07:11:29.537674+0000","last_became_active":"2023-11-26T07:11:28.542356+0000","last_became_peered":"2023-11-26T07:11:28.542356+0000","last_unstale":"2023-11-26T07:11:29.537674+0000","last_undegraded":"2023-11-26T07:11:29.537674+0000","last_fullsized":"2023-11-26T07:11:29.537674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537749+0000","last_change":"2023-11-26T07:11:27.527868+0000","last_active":"2023-11-26T07:11:29.537749+0000","last_peered":"2023-11-26T07:11:29.537749+0000","last_clean":"2023-11-26T07:11:29.537749+0000","last_became_active":"2023-11-26T07:11:27.527482+0000","last_became_peered":"2023-11-26T07:11:27.527482+0000","last_unstale":"2023-11-26T07:11:29.537749+0000","last_undegraded":"2023-11-26T07:11:29.537749+0000","last_fullsized":"2023-11-26T07:11:29.537749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.562118+0000","last_change":"2023-11-26T07:11:28.554487+0000","last_active":"2023-11-26T07:11:28.562118+0000","last_peered":"2023-11-26T07:11:28.562118+0000","last_clean":"2023-11-26T07:11:28.562118+0000","last_became_active":"2023-11-26T07:11:28.554210+0000","last_became_peered":"2023-11-26T07:11:28.554210+0000","last_unstale":"2023-11-26T07:11:28.562118+0000","last_undegraded":"2023-11-26T07:11:28.562118+0000","last_fullsized":"2023-11-26T07:11:28.562118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:53.475994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555204+0000","last_change":"2023-11-26T07:11:27.531628+0000","last_active":"2023-11-26T07:11:28.555204+0000","last_peered":"2023-11-26T07:11:28.555204+0000","last_clean":"2023-11-26T07:11:28.555204+0000","last_became_active":"2023-11-26T07:11:27.531336+0000","last_became_peered":"2023-11-26T07:11:27.531336+0000","last_unstale":"2023-11-26T07:11:28.555204+0000","last_undegraded":"2023-11-26T07:11:28.555204+0000","last_fullsized":"2023-11-26T07:11:28.555204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:10.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189372+0000","last_change":"2023-11-26T07:11:28.540197+0000","last_active":"2023-11-26T07:11:30.189372+0000","last_peered":"2023-11-26T07:11:30.189372+0000","last_clean":"2023-11-26T07:11:30.189372+0000","last_became_active":"2023-11-26T07:11:28.539930+0000","last_became_peered":"2023-11-26T07:11:28.539930+0000","last_unstale":"2023-11-26T07:11:30.189372+0000","last_undegraded":"2023-11-26T07:11:30.189372+0000","last_fullsized":"2023-11-26T07:11:30.189372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539876+0000","last_change":"2023-11-26T07:11:27.528055+0000","last_active":"2023-11-26T07:11:29.539876+0000","last_peered":"2023-11-26T07:11:29.539876+0000","last_clean":"2023-11-26T07:11:29.539876+0000","last_became_active":"2023-11-26T07:11:27.527760+0000","last_became_peered":"2023-11-26T07:11:27.527760+0000","last_unstale":"2023-11-26T07:11:29.539876+0000","last_undegraded":"2023-11-26T07:11:29.539876+0000","last_fullsized":"2023-11-26T07:11:29.539876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:10.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539630+0000","last_change":"2023-11-26T07:11:28.544985+0000","last_active":"2023-11-26T07:11:29.539630+0000","last_peered":"2023-11-26T07:11:29.539630+0000","last_clean":"2023-11-26T07:11:29.539630+0000","last_became_active":"2023-11-26T07:11:28.544546+0000","last_became_peered":"2023-11-26T07:11:28.544546+0000","last_unstale":"2023-11-26T07:11:29.539630+0000","last_undegraded":"2023-11-26T07:11:29.539630+0000","last_fullsized":"2023-11-26T07:11:29.539630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:19.228362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537514+0000","last_change":"2023-11-26T07:11:27.527741+0000","last_active":"2023-11-26T07:11:29.537514+0000","last_peered":"2023-11-26T07:11:29.537514+0000","last_clean":"2023-11-26T07:11:29.537514+0000","last_became_active":"2023-11-26T07:11:27.526479+0000","last_became_peered":"2023-11-26T07:11:27.526479+0000","last_unstale":"2023-11-26T07:11:29.537514+0000","last_undegraded":"2023-11-26T07:11:29.537514+0000","last_fullsized":"2023-11-26T07:11:29.537514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:31.071342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.543414+0000","last_change":"2023-11-26T07:11:28.540204+0000","last_active":"2023-11-26T07:11:28.543414+0000","last_peered":"2023-11-26T07:11:28.543414+0000","last_clean":"2023-11-26T07:11:28.543414+0000","last_became_active":"2023-11-26T07:11:28.539891+0000","last_became_peered":"2023-11-26T07:11:28.539891+0000","last_unstale":"2023-11-26T07:11:28.543414+0000","last_undegraded":"2023-11-26T07:11:28.543414+0000","last_fullsized":"2023-11-26T07:11:28.543414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:57.098847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537841+0000","last_change":"2023-11-26T07:11:27.534469+0000","last_active":"2023-11-26T07:11:29.537841+0000","last_peered":"2023-11-26T07:11:29.537841+0000","last_clean":"2023-11-26T07:11:29.537841+0000","last_became_active":"2023-11-26T07:11:27.534116+0000","last_became_peered":"2023-11-26T07:11:27.534116+0000","last_unstale":"2023-11-26T07:11:29.537841+0000","last_undegraded":"2023-11-26T07:11:29.537841+0000","last_fullsized":"2023-11-26T07:11:29.537841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:25.399589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549011+0000","last_change":"2023-11-26T07:11:28.547277+0000","last_active":"2023-11-26T07:11:28.549011+0000","last_peered":"2023-11-26T07:11:28.549011+0000","last_clean":"2023-11-26T07:11:28.549011+0000","last_became_active":"2023-11-26T07:11:28.546992+0000","last_became_peered":"2023-11-26T07:11:28.546992+0000","last_unstale":"2023-11-26T07:11:28.549011+0000","last_undegraded":"2023-11-26T07:11:28.549011+0000","last_fullsized":"2023-11-26T07:11:28.549011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:17.562000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540403+0000","last_change":"2023-11-26T07:11:27.529785+0000","last_active":"2023-11-26T07:11:29.540403+0000","last_peered":"2023-11-26T07:11:29.540403+0000","last_clean":"2023-11-26T07:11:29.540403+0000","last_became_active":"2023-11-26T07:11:27.528686+0000","last_became_peered":"2023-11-26T07:11:27.528686+0000","last_unstale":"2023-11-26T07:11:29.540403+0000","last_undegraded":"2023-11-26T07:11:29.540403+0000","last_fullsized":"2023-11-26T07:11:29.540403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:21.053006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549487+0000","last_change":"2023-11-26T07:11:28.546713+0000","last_active":"2023-11-26T07:11:28.549487+0000","last_peered":"2023-11-26T07:11:28.549487+0000","last_clean":"2023-11-26T07:11:28.549487+0000","last_became_active":"2023-11-26T07:11:28.546453+0000","last_became_peered":"2023-11-26T07:11:28.546453+0000","last_unstale":"2023-11-26T07:11:28.549487+0000","last_undegraded":"2023-11-26T07:11:28.549487+0000","last_fullsized":"2023-11-26T07:11:28.549487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:57.098847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.533987+0000","last_change":"2023-11-26T07:11:27.531109+0000","last_active":"2023-11-26T07:11:28.533987+0000","last_peered":"2023-11-26T07:11:28.533987+0000","last_clean":"2023-11-26T07:11:28.533987+0000","last_became_active":"2023-11-26T07:11:27.530786+0000","last_became_peered":"2023-11-26T07:11:27.530786+0000","last_unstale":"2023-11-26T07:11:28.533987+0000","last_undegraded":"2023-11-26T07:11:28.533987+0000","last_fullsized":"2023-11-26T07:11:28.533987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:52.471590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540761+0000","last_change":"2023-11-26T07:11:28.539795+0000","last_active":"2023-11-26T07:11:29.540761+0000","last_peered":"2023-11-26T07:11:29.540761+0000","last_clean":"2023-11-26T07:11:29.540761+0000","last_became_active":"2023-11-26T07:11:28.539562+0000","last_became_peered":"2023-11-26T07:11:28.539562+0000","last_unstale":"2023-11-26T07:11:29.540761+0000","last_undegraded":"2023-11-26T07:11:29.540761+0000","last_fullsized":"2023-11-26T07:11:29.540761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:02:05.837198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537674+0000","last_change":"2023-11-26T07:11:27.522266+0000","last_active":"2023-11-26T07:11:29.537674+0000","last_peered":"2023-11-26T07:11:29.537674+0000","last_clean":"2023-11-26T07:11:29.537674+0000","last_became_active":"2023-11-26T07:11:27.521972+0000","last_became_peered":"2023-11-26T07:11:27.521972+0000","last_unstale":"2023-11-26T07:11:29.537674+0000","last_undegraded":"2023-11-26T07:11:29.537674+0000","last_fullsized":"2023-11-26T07:11:29.537674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.553910+0000","last_change":"2023-11-26T07:11:28.544499+0000","last_active":"2023-11-26T07:11:28.553910+0000","last_peered":"2023-11-26T07:11:28.553910+0000","last_clean":"2023-11-26T07:11:28.553910+0000","last_became_active":"2023-11-26T07:11:28.544154+0000","last_became_peered":"2023-11-26T07:11:28.544154+0000","last_unstale":"2023-11-26T07:11:28.553910+0000","last_undegraded":"2023-11-26T07:11:28.553910+0000","last_fullsized":"2023-11-26T07:11:28.553910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:19.228362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.562209+0000","last_change":"2023-11-26T07:11:27.535061+0000","last_active":"2023-11-26T07:11:28.562209+0000","last_peered":"2023-11-26T07:11:28.562209+0000","last_clean":"2023-11-26T07:11:28.562209+0000","last_became_active":"2023-11-26T07:11:27.534792+0000","last_became_peered":"2023-11-26T07:11:27.534792+0000","last_unstale":"2023-11-26T07:11:28.562209+0000","last_undegraded":"2023-11-26T07:11:28.562209+0000","last_fullsized":"2023-11-26T07:11:28.562209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540720+0000","last_change":"2023-11-26T07:11:28.532153+0000","last_active":"2023-11-26T07:11:29.540720+0000","last_peered":"2023-11-26T07:11:29.540720+0000","last_clean":"2023-11-26T07:11:29.540720+0000","last_became_active":"2023-11-26T07:11:28.531880+0000","last_became_peered":"2023-11-26T07:11:28.531880+0000","last_unstale":"2023-11-26T07:11:29.540720+0000","last_undegraded":"2023-11-26T07:11:29.540720+0000","last_fullsized":"2023-11-26T07:11:29.540720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:02.252744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549518+0000","last_change":"2023-11-26T07:11:28.544445+0000","last_active":"2023-11-26T07:11:28.549518+0000","last_peered":"2023-11-26T07:11:28.549518+0000","last_clean":"2023-11-26T07:11:28.549518+0000","last_became_active":"2023-11-26T07:11:28.544146+0000","last_became_peered":"2023-11-26T07:11:28.544146+0000","last_unstale":"2023-11-26T07:11:28.549518+0000","last_undegraded":"2023-11-26T07:11:28.549518+0000","last_fullsized":"2023-11-26T07:11:28.549518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:48.999647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549521+0000","last_change":"2023-11-26T07:11:28.542940+0000","last_active":"2023-11-26T07:11:28.549521+0000","last_peered":"2023-11-26T07:11:28.549521+0000","last_clean":"2023-11-26T07:11:28.549521+0000","last_became_active":"2023-11-26T07:11:28.542607+0000","last_became_peered":"2023-11-26T07:11:28.542607+0000","last_unstale":"2023-11-26T07:11:28.549521+0000","last_undegraded":"2023-11-26T07:11:28.549521+0000","last_fullsized":"2023-11-26T07:11:28.549521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:36.121867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555845+0000","last_change":"2023-11-26T07:11:28.549728+0000","last_active":"2023-11-26T07:11:28.555845+0000","last_peered":"2023-11-26T07:11:28.555845+0000","last_clean":"2023-11-26T07:11:28.555845+0000","last_became_active":"2023-11-26T07:11:28.543537+0000","last_became_peered":"2023-11-26T07:11:28.543537+0000","last_unstale":"2023-11-26T07:11:28.555845+0000","last_undegraded":"2023-11-26T07:11:28.555845+0000","last_fullsized":"2023-11-26T07:11:28.555845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:48.999647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189278+0000","last_change":"2023-11-26T07:11:28.546921+0000","last_active":"2023-11-26T07:11:30.189278+0000","last_peered":"2023-11-26T07:11:30.189278+0000","last_clean":"2023-11-26T07:11:30.189278+0000","last_became_active":"2023-11-26T07:11:28.546543+0000","last_became_peered":"2023-11-26T07:11:28.546543+0000","last_unstale":"2023-11-26T07:11:30.189278+0000","last_undegraded":"2023-11-26T07:11:30.189278+0000","last_fullsized":"2023-11-26T07:11:30.189278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.551018+0000","last_change":"2023-11-26T07:11:28.549516+0000","last_active":"2023-11-26T07:11:28.551018+0000","last_peered":"2023-11-26T07:11:28.551018+0000","last_clean":"2023-11-26T07:11:28.551018+0000","last_became_active":"2023-11-26T07:11:28.548917+0000","last_became_peered":"2023-11-26T07:11:28.548917+0000","last_unstale":"2023-11-26T07:11:28.551018+0000","last_undegraded":"2023-11-26T07:11:28.551018+0000","last_fullsized":"2023-11-26T07:11:28.551018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:11.545490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537692+0000","last_change":"2023-11-26T07:11:28.541342+0000","last_active":"2023-11-26T07:11:29.537692+0000","last_peered":"2023-11-26T07:11:29.537692+0000","last_clean":"2023-11-26T07:11:29.537692+0000","last_became_active":"2023-11-26T07:11:28.540919+0000","last_became_peered":"2023-11-26T07:11:28.540919+0000","last_unstale":"2023-11-26T07:11:29.537692+0000","last_undegraded":"2023-11-26T07:11:29.537692+0000","last_fullsized":"2023-11-26T07:11:29.537692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.554271+0000","last_change":"2023-11-26T07:11:28.551636+0000","last_active":"2023-11-26T07:11:28.554271+0000","last_peered":"2023-11-26T07:11:28.554271+0000","last_clean":"2023-11-26T07:11:28.554271+0000","last_became_active":"2023-11-26T07:11:28.551335+0000","last_became_peered":"2023-11-26T07:11:28.551335+0000","last_unstale":"2023-11-26T07:11:28.554271+0000","last_undegraded":"2023-11-26T07:11:28.554271+0000","last_fullsized":"2023-11-26T07:11:28.554271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:22.511330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.548898+0000","last_change":"2023-11-26T07:11:28.546402+0000","last_active":"2023-11-26T07:11:28.548898+0000","last_peered":"2023-11-26T07:11:28.548898+0000","last_clean":"2023-11-26T07:11:28.548898+0000","last_became_active":"2023-11-26T07:11:28.546132+0000","last_became_peered":"2023-11-26T07:11:28.546132+0000","last_unstale":"2023-11-26T07:11:28.548898+0000","last_undegraded":"2023-11-26T07:11:28.548898+0000","last_fullsized":"2023-11-26T07:11:28.548898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:26.403994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555807+0000","last_change":"2023-11-26T07:11:28.548148+0000","last_active":"2023-11-26T07:11:28.555807+0000","last_peered":"2023-11-26T07:11:28.555807+0000","last_clean":"2023-11-26T07:11:28.555807+0000","last_became_active":"2023-11-26T07:11:28.546933+0000","last_became_peered":"2023-11-26T07:11:28.546933+0000","last_unstale":"2023-11-26T07:11:28.555807+0000","last_undegraded":"2023-11-26T07:11:28.555807+0000","last_fullsized":"2023-11-26T07:11:28.555807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:20.471456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.550683+0000","last_change":"2023-11-26T07:11:28.549377+0000","last_active":"2023-11-26T07:11:28.550683+0000","last_peered":"2023-11-26T07:11:28.550683+0000","last_clean":"2023-11-26T07:11:28.550683+0000","last_became_active":"2023-11-26T07:11:28.549093+0000","last_became_peered":"2023-11-26T07:11:28.549093+0000","last_unstale":"2023-11-26T07:11:28.550683+0000","last_undegraded":"2023-11-26T07:11:28.550683+0000","last_fullsized":"2023-11-26T07:11:28.550683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:51.544284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.556832+0000","last_change":"2023-11-26T07:11:28.554891+0000","last_active":"2023-11-26T07:11:28.556832+0000","last_peered":"2023-11-26T07:11:28.556832+0000","last_clean":"2023-11-26T07:11:28.556832+0000","last_became_active":"2023-11-26T07:11:28.554617+0000","last_became_peered":"2023-11-26T07:11:28.554617+0000","last_unstale":"2023-11-26T07:11:28.556832+0000","last_undegraded":"2023-11-26T07:11:28.556832+0000","last_fullsized":"2023-11-26T07:11:28.556832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189270+0000","last_change":"2023-11-26T07:11:28.552756+0000","last_active":"2023-11-26T07:11:30.189270+0000","last_peered":"2023-11-26T07:11:30.189270+0000","last_clean":"2023-11-26T07:11:30.189270+0000","last_became_active":"2023-11-26T07:11:28.552548+0000","last_became_peered":"2023-11-26T07:11:28.552548+0000","last_unstale":"2023-11-26T07:11:30.189270+0000","last_undegraded":"2023-11-26T07:11:30.189270+0000","last_fullsized":"2023-11-26T07:11:30.189270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549607+0000","last_change":"2023-11-26T07:11:28.547396+0000","last_active":"2023-11-26T07:11:28.549607+0000","last_peered":"2023-11-26T07:11:28.549607+0000","last_clean":"2023-11-26T07:11:28.549607+0000","last_became_active":"2023-11-26T07:11:28.547176+0000","last_became_peered":"2023-11-26T07:11:28.547176+0000","last_unstale":"2023-11-26T07:11:28.549607+0000","last_undegraded":"2023-11-26T07:11:28.549607+0000","last_fullsized":"2023-11-26T07:11:28.549607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:51.244423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":660287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":69281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":69407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":69371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:11:32.628 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:11:32.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.781+0000 7f0797866700 1 Processor -- start 2023-11-26T07:11:32.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.781+0000 7f0797866700 1 -- start start 2023-11-26T07:11:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.782+0000 7f0797866700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f07900d2380 0x7f07901604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.782+0000 7f0797866700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07900b22d0 con 0x7f07900b3370 2023-11-26T07:11:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.782+0000 7f0797866700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07900b25b0 con 0x7f079015bdd0 2023-11-26T07:11:32.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.782+0000 7f0797866700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07900b2f50 con 0x7f07900d2380 2023-11-26T07:11:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.782+0000 7f0794e01700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f07900b3370 0x7f07900d1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54174/0 (socket says 172.21.15.83:54174) 2023-11-26T07:11:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.782+0000 7f0794e01700 1 -- 172.21.15.83:0/4172824385 learned_addr learned my addr 172.21.15.83:0/4172824385 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.782+0000 7f0795e03700 1 --2- 172.21.15.83:0/4172824385 >> v2:172.21.15.83:3300/0 conn(0x7f07900d2380 0x7f07901604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.782+0000 7f0787fff700 1 -- 172.21.15.83:0/4172824385 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462665841 0 0) 0x7f07900b22d0 con 0x7f07900b3370 2023-11-26T07:11:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.783+0000 7f0787fff700 1 -- 172.21.15.83:0/4172824385 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f077c0035b0 con 0x7f07900b3370 2023-11-26T07:11:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.783+0000 7f0787fff700 1 -- 172.21.15.83:0/4172824385 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393142859 0 0) 0x7f07900b25b0 con 0x7f079015bdd0 2023-11-26T07:11:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.783+0000 7f0787fff700 1 -- 172.21.15.83:0/4172824385 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07900b22d0 con 0x7f079015bdd0 2023-11-26T07:11:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.783+0000 7f0795e03700 1 -- 172.21.15.83:0/4172824385 >> v1:172.21.15.203:6789/0 conn(0x7f079015bdd0 legacy=0x7f07900b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.783+0000 7f0795e03700 1 -- 172.21.15.83:0/4172824385 >> v1:172.21.15.83:6789/0 conn(0x7f07900b3370 legacy=0x7f07900d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.783+0000 7f0795e03700 1 -- 172.21.15.83:0/4172824385 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07900b2890 con 0x7f07900d2380 2023-11-26T07:11:32.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.783+0000 7f0795e03700 1 --2- 172.21.15.83:0/4172824385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07900d2380 0x7f07901604e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0788009d90 tx=0x7f078800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b652a75affb130d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.783+0000 7f0787fff700 1 -- 172.21.15.83:0/4172824385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078800e070 con 0x7f07900d2380 2023-11-26T07:11:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.783+0000 7f0787fff700 1 -- 172.21.15.83:0/4172824385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078800bb20 con 0x7f07900d2380 2023-11-26T07:11:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.783+0000 7f0787fff700 1 -- 172.21.15.83:0/4172824385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788012430 con 0x7f07900d2380 2023-11-26T07:11:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.784+0000 7f0797866700 1 -- 172.21.15.83:0/4172824385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07900d2380 msgr2=0x7f07901604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.784+0000 7f0797866700 1 --2- 172.21.15.83:0/4172824385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07900d2380 0x7f07901604e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0788009d90 tx=0x7f078800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.784+0000 7f0797866700 1 -- 172.21.15.83:0/4172824385 shutdown_connections 2023-11-26T07:11:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.784+0000 7f0797866700 1 --2- 172.21.15.83:0/4172824385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07900d2380 0x7f07901604e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.784+0000 7f0797866700 1 -- 172.21.15.83:0/4172824385 >> 172.21.15.83:0/4172824385 conn(0x7f07900731a0 msgr2=0x7f079009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:32.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.784+0000 7f0797866700 1 -- 172.21.15.83:0/4172824385 shutdown_connections 2023-11-26T07:11:32.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.784+0000 7f0797866700 1 -- 172.21.15.83:0/4172824385 wait complete. 2023-11-26T07:11:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.785+0000 7f0797866700 1 Processor -- start 2023-11-26T07:11:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.785+0000 7f0797866700 1 -- start start 2023-11-26T07:11:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0797866700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07900b3370 0x7f0790152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0797866700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07900d2380 0x7f079014a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0797866700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f079015bdd0 0x7f079014d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0797866700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07900c8db0 con 0x7f07900b3370 2023-11-26T07:11:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0797866700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901517e0 con 0x7f07900d2380 2023-11-26T07:11:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0797866700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0790151ac0 con 0x7f079015bdd0 2023-11-26T07:11:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0795602700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07900b3370 0x7f0790152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0795602700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07900b3370 0x7f0790152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34140/0 (socket says 172.21.15.83:34140) 2023-11-26T07:11:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0795602700 1 -- 172.21.15.83:0/1292740332 learned_addr learned my addr 172.21.15.83:0/1292740332 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0794e01700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07900d2380 0x7f079014a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0795e03700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f079015bdd0 0x7f079014d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0795602700 1 -- 172.21.15.83:0/1292740332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f079015bdd0 msgr2=0x7f079014d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0795602700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f079015bdd0 0x7f079014d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0795602700 1 -- 172.21.15.83:0/1292740332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07900d2380 msgr2=0x7f079014a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0795602700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07900d2380 0x7f079014a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0795602700 1 -- 172.21.15.83:0/1292740332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0788009a70 con 0x7f07900b3370 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0794e01700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07900d2380 0x7f079014a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0795602700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07900b3370 0x7f0790152910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f078000c500 tx=0x7f078000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0785ffb700 1 -- 172.21.15.83:0/1292740332 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07800052d0 con 0x7f07900b3370 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0785ffb700 1 -- 172.21.15.83:0/1292740332 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0780005450 con 0x7f07900b3370 2023-11-26T07:11:32.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0797866700 1 -- 172.21.15.83:0/1292740332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0790151da0 con 0x7f07900b3370 2023-11-26T07:11:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.786+0000 7f0785ffb700 1 -- 172.21.15.83:0/1292740332 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07800038c0 con 0x7f07900b3370 2023-11-26T07:11:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.787+0000 7f0797866700 1 -- 172.21.15.83:0/1292740332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0790152360 con 0x7f07900b3370 2023-11-26T07:11:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.787+0000 7f0797866700 1 -- 172.21.15.83:0/1292740332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0790067f60 con 0x7f07900b3370 2023-11-26T07:11:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.788+0000 7f0785ffb700 1 -- 172.21.15.83:0/1292740332 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0780003a20 con 0x7f07900b3370 2023-11-26T07:11:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.788+0000 7f0785ffb700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f077406aed0 0x7f077406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.788+0000 7f0785ffb700 1 -- 172.21.15.83:0/1292740332 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0780088a60 con 0x7f07900b3370 2023-11-26T07:11:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.788+0000 7f0794e01700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f077406aed0 0x7f077406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.789+0000 7f0794e01700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f077406aed0 0x7f077406d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07900705d0 tx=0x7f078c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.793+0000 7f0785ffb700 1 -- 172.21.15.83:0/1292740332 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0780053ba0 con 0x7f07900b3370 2023-11-26T07:11:32.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.980+0000 7f0797866700 1 -- 172.21.15.83:0/1292740332 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0790166cf0 con 0x7f077406aed0 2023-11-26T07:11:32.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.987+0000 7f0785ffb700 1 -- 172.21.15.83:0/1292740332 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264636 (secure 0 0 0) 0x7f078c0014f0 con 0x7f077406aed0 2023-11-26T07:11:32.992 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 -- 172.21.15.83:0/1292740332 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f077406aed0 msgr2=0x7f077406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f077406aed0 0x7f077406d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07900705d0 tx=0x7f078c009040 comp rx=0 tx=0).stop 2023-11-26T07:11:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 -- 172.21.15.83:0/1292740332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07900b3370 msgr2=0x7f0790152910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07900b3370 0x7f0790152910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f078000c500 tx=0x7f078000c8c0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 -- 172.21.15.83:0/1292740332 shutdown_connections 2023-11-26T07:11:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f077406aed0 0x7f077406d390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f079015bdd0 0x7f079014d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07900b3370 0x7f0790152910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 --2- 172.21.15.83:0/1292740332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07900d2380 0x7f079014a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 -- 172.21.15.83:0/1292740332 >> 172.21.15.83:0/1292740332 conn(0x7f07900731a0 msgr2=0x7f07900b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 -- 172.21.15.83:0/1292740332 shutdown_connections 2023-11-26T07:11:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:32.990+0000 7f0797866700 1 -- 172.21.15.83:0/1292740332 wait complete. 2023-11-26T07:11:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:11:33.004 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-26T07:11:32.307880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219040,"kb_used_data":2848,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755680,"statfs":{"total":773094113280,"available":772869816320,"internally_reserved":0,"allocated":2916352,"data_stored":1752404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.803952"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540665+0000","last_change":"2023-11-26T07:11:28.537635+0000","last_active":"2023-11-26T07:11:29.540665+0000","last_peered":"2023-11-26T07:11:29.540665+0000","last_clean":"2023-11-26T07:11:29.540665+0000","last_became_active":"2023-11-26T07:11:28.537327+0000","last_became_peered":"2023-11-26T07:11:28.537327+0000","last_unstale":"2023-11-26T07:11:29.540665+0000","last_undegraded":"2023-11-26T07:11:29.540665+0000","last_fullsized":"2023-11-26T07:11:29.540665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:48:59.913650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537894+0000","last_change":"2023-11-26T07:11:28.541611+0000","last_active":"2023-11-26T07:11:29.537894+0000","last_peered":"2023-11-26T07:11:29.537894+0000","last_clean":"2023-11-26T07:11:29.537894+0000","last_became_active":"2023-11-26T07:11:28.541267+0000","last_became_peered":"2023-11-26T07:11:28.541267+0000","last_unstale":"2023-11-26T07:11:29.537894+0000","last_undegraded":"2023-11-26T07:11:29.537894+0000","last_fullsized":"2023-11-26T07:11:29.537894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189176+0000","last_change":"2023-11-26T07:11:28.543463+0000","last_active":"2023-11-26T07:11:30.189176+0000","last_peered":"2023-11-26T07:11:30.189176+0000","last_clean":"2023-11-26T07:11:30.189176+0000","last_became_active":"2023-11-26T07:11:28.542718+0000","last_became_peered":"2023-11-26T07:11:28.542718+0000","last_unstale":"2023-11-26T07:11:30.189176+0000","last_undegraded":"2023-11-26T07:11:30.189176+0000","last_fullsized":"2023-11-26T07:11:30.189176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.553883+0000","last_change":"2023-11-26T07:11:28.544447+0000","last_active":"2023-11-26T07:11:28.553883+0000","last_peered":"2023-11-26T07:11:28.553883+0000","last_clean":"2023-11-26T07:11:28.553883+0000","last_became_active":"2023-11-26T07:11:28.543689+0000","last_became_peered":"2023-11-26T07:11:28.543689+0000","last_unstale":"2023-11-26T07:11:28.553883+0000","last_undegraded":"2023-11-26T07:11:28.553883+0000","last_fullsized":"2023-11-26T07:11:28.553883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:53.475994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540289+0000","last_change":"2023-11-26T07:11:28.542518+0000","last_active":"2023-11-26T07:11:29.540289+0000","last_peered":"2023-11-26T07:11:29.540289+0000","last_clean":"2023-11-26T07:11:29.540289+0000","last_became_active":"2023-11-26T07:11:28.542255+0000","last_became_peered":"2023-11-26T07:11:28.542255+0000","last_unstale":"2023-11-26T07:11:29.540289+0000","last_undegraded":"2023-11-26T07:11:29.540289+0000","last_fullsized":"2023-11-26T07:11:29.540289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:48.717411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555860+0000","last_change":"2023-11-26T07:11:28.550053+0000","last_active":"2023-11-26T07:11:28.555860+0000","last_peered":"2023-11-26T07:11:28.555860+0000","last_clean":"2023-11-26T07:11:28.555860+0000","last_became_active":"2023-11-26T07:11:28.549624+0000","last_became_peered":"2023-11-26T07:11:28.549624+0000","last_unstale":"2023-11-26T07:11:28.555860+0000","last_undegraded":"2023-11-26T07:11:28.555860+0000","last_fullsized":"2023-11-26T07:11:28.555860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539979+0000","last_change":"2023-11-26T07:11:28.543140+0000","last_active":"2023-11-26T07:11:29.539979+0000","last_peered":"2023-11-26T07:11:29.539979+0000","last_clean":"2023-11-26T07:11:29.539979+0000","last_became_active":"2023-11-26T07:11:28.542835+0000","last_became_peered":"2023-11-26T07:11:28.542835+0000","last_unstale":"2023-11-26T07:11:29.539979+0000","last_undegraded":"2023-11-26T07:11:29.539979+0000","last_fullsized":"2023-11-26T07:11:29.539979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:14.369275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.562130+0000","last_change":"2023-11-26T07:11:28.554561+0000","last_active":"2023-11-26T07:11:28.562130+0000","last_peered":"2023-11-26T07:11:28.562130+0000","last_clean":"2023-11-26T07:11:28.562130+0000","last_became_active":"2023-11-26T07:11:28.554273+0000","last_became_peered":"2023-11-26T07:11:28.554273+0000","last_unstale":"2023-11-26T07:11:28.562130+0000","last_undegraded":"2023-11-26T07:11:28.562130+0000","last_fullsized":"2023-11-26T07:11:28.562130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:51.244423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540623+0000","last_change":"2023-11-26T07:11:28.539323+0000","last_active":"2023-11-26T07:11:29.540623+0000","last_peered":"2023-11-26T07:11:29.540623+0000","last_clean":"2023-11-26T07:11:29.540623+0000","last_became_active":"2023-11-26T07:11:28.539047+0000","last_became_peered":"2023-11-26T07:11:28.539047+0000","last_unstale":"2023-11-26T07:11:29.540623+0000","last_undegraded":"2023-11-26T07:11:29.540623+0000","last_fullsized":"2023-11-26T07:11:29.540623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:01:36.280118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.543260+0000","last_change":"2023-11-26T07:11:28.538383+0000","last_active":"2023-11-26T07:11:28.543260+0000","last_peered":"2023-11-26T07:11:28.543260+0000","last_clean":"2023-11-26T07:11:28.543260+0000","last_became_active":"2023-11-26T07:11:28.537726+0000","last_became_peered":"2023-11-26T07:11:28.537726+0000","last_unstale":"2023-11-26T07:11:28.543260+0000","last_undegraded":"2023-11-26T07:11:28.543260+0000","last_fullsized":"2023-11-26T07:11:28.543260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:32.075747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.546268+0000","last_change":"2023-11-26T07:11:28.538846+0000","last_active":"2023-11-26T07:11:28.546268+0000","last_peered":"2023-11-26T07:11:28.546268+0000","last_clean":"2023-11-26T07:11:28.546268+0000","last_became_active":"2023-11-26T07:11:28.537769+0000","last_became_peered":"2023-11-26T07:11:28.537769+0000","last_unstale":"2023-11-26T07:11:28.546268+0000","last_undegraded":"2023-11-26T07:11:28.546268+0000","last_fullsized":"2023-11-26T07:11:28.546268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:19.228362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539796+0000","last_change":"2023-11-26T07:11:28.541985+0000","last_active":"2023-11-26T07:11:29.539796+0000","last_peered":"2023-11-26T07:11:29.539796+0000","last_clean":"2023-11-26T07:11:29.539796+0000","last_became_active":"2023-11-26T07:11:28.541404+0000","last_became_peered":"2023-11-26T07:11:28.541404+0000","last_unstale":"2023-11-26T07:11:29.539796+0000","last_undegraded":"2023-11-26T07:11:29.539796+0000","last_fullsized":"2023-11-26T07:11:29.539796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:11.545490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.561691+0000","last_change":"2023-11-26T07:11:28.551585+0000","last_active":"2023-11-26T07:11:28.561691+0000","last_peered":"2023-11-26T07:11:28.561691+0000","last_clean":"2023-11-26T07:11:28.561691+0000","last_became_active":"2023-11-26T07:11:28.551296+0000","last_became_peered":"2023-11-26T07:11:28.551296+0000","last_unstale":"2023-11-26T07:11:28.561691+0000","last_undegraded":"2023-11-26T07:11:28.561691+0000","last_fullsized":"2023-11-26T07:11:28.561691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:20.471456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189126+0000","last_change":"2023-11-26T07:11:28.541346+0000","last_active":"2023-11-26T07:11:30.189126+0000","last_peered":"2023-11-26T07:11:30.189126+0000","last_clean":"2023-11-26T07:11:30.189126+0000","last_became_active":"2023-11-26T07:11:28.540961+0000","last_became_peered":"2023-11-26T07:11:28.540961+0000","last_unstale":"2023-11-26T07:11:30.189126+0000","last_undegraded":"2023-11-26T07:11:30.189126+0000","last_fullsized":"2023-11-26T07:11:30.189126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:48.999647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537528+0000","last_change":"2023-11-26T07:11:28.536348+0000","last_active":"2023-11-26T07:11:29.537528+0000","last_peered":"2023-11-26T07:11:29.537528+0000","last_clean":"2023-11-26T07:11:29.537528+0000","last_became_active":"2023-11-26T07:11:28.535391+0000","last_became_peered":"2023-11-26T07:11:28.535391+0000","last_unstale":"2023-11-26T07:11:29.537528+0000","last_undegraded":"2023-11-26T07:11:29.537528+0000","last_fullsized":"2023-11-26T07:11:29.537528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.553977+0000","last_change":"2023-11-26T07:11:28.544579+0000","last_active":"2023-11-26T07:11:28.553977+0000","last_peered":"2023-11-26T07:11:28.553977+0000","last_clean":"2023-11-26T07:11:28.553977+0000","last_became_active":"2023-11-26T07:11:28.542480+0000","last_became_peered":"2023-11-26T07:11:28.542480+0000","last_unstale":"2023-11-26T07:11:28.553977+0000","last_undegraded":"2023-11-26T07:11:28.553977+0000","last_fullsized":"2023-11-26T07:11:28.553977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:22.511330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.556798+0000","last_change":"2023-11-26T07:11:28.554495+0000","last_active":"2023-11-26T07:11:28.556798+0000","last_peered":"2023-11-26T07:11:28.556798+0000","last_clean":"2023-11-26T07:11:28.556798+0000","last_became_active":"2023-11-26T07:11:28.554114+0000","last_became_peered":"2023-11-26T07:11:28.554114+0000","last_unstale":"2023-11-26T07:11:28.556798+0000","last_undegraded":"2023-11-26T07:11:28.556798+0000","last_fullsized":"2023-11-26T07:11:28.556798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:26.403994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.551941+0000","last_change":"2023-11-26T07:11:28.544698+0000","last_active":"2023-11-26T07:11:28.551941+0000","last_peered":"2023-11-26T07:11:28.551941+0000","last_clean":"2023-11-26T07:11:28.551941+0000","last_became_active":"2023-11-26T07:11:28.544347+0000","last_became_peered":"2023-11-26T07:11:28.544347+0000","last_unstale":"2023-11-26T07:11:28.551941+0000","last_undegraded":"2023-11-26T07:11:28.551941+0000","last_fullsized":"2023-11-26T07:11:28.551941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:46.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540518+0000","last_change":"2023-11-26T07:11:28.550524+0000","last_active":"2023-11-26T07:11:29.540518+0000","last_peered":"2023-11-26T07:11:29.540518+0000","last_clean":"2023-11-26T07:11:29.540518+0000","last_became_active":"2023-11-26T07:11:28.543803+0000","last_became_peered":"2023-11-26T07:11:28.543803+0000","last_unstale":"2023-11-26T07:11:29.540518+0000","last_undegraded":"2023-11-26T07:11:29.540518+0000","last_fullsized":"2023-11-26T07:11:29.540518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:36:40.660123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189251+0000","last_change":"2023-11-26T07:11:27.524720+0000","last_active":"2023-11-26T07:11:30.189251+0000","last_peered":"2023-11-26T07:11:30.189251+0000","last_clean":"2023-11-26T07:11:30.189251+0000","last_became_active":"2023-11-26T07:11:27.524424+0000","last_became_peered":"2023-11-26T07:11:27.524424+0000","last_unstale":"2023-11-26T07:11:30.189251+0000","last_undegraded":"2023-11-26T07:11:30.189251+0000","last_fullsized":"2023-11-26T07:11:30.189251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.542197+0000","last_change":"2023-11-26T07:11:28.540843+0000","last_active":"2023-11-26T07:11:28.542197+0000","last_peered":"2023-11-26T07:11:28.542197+0000","last_clean":"2023-11-26T07:11:28.542197+0000","last_became_active":"2023-11-26T07:11:28.540579+0000","last_became_peered":"2023-11-26T07:11:28.540579+0000","last_unstale":"2023-11-26T07:11:28.542197+0000","last_undegraded":"2023-11-26T07:11:28.542197+0000","last_fullsized":"2023-11-26T07:11:28.542197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:33.426915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.561729+0000","last_change":"2023-11-26T07:11:27.528053+0000","last_active":"2023-11-26T07:11:28.561729+0000","last_peered":"2023-11-26T07:11:28.561729+0000","last_clean":"2023-11-26T07:11:28.561729+0000","last_became_active":"2023-11-26T07:11:27.526741+0000","last_became_peered":"2023-11-26T07:11:27.526741+0000","last_unstale":"2023-11-26T07:11:28.561729+0000","last_undegraded":"2023-11-26T07:11:28.561729+0000","last_fullsized":"2023-11-26T07:11:28.561729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:31.071342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189238+0000","last_change":"2023-11-26T07:11:28.551779+0000","last_active":"2023-11-26T07:11:30.189238+0000","last_peered":"2023-11-26T07:11:30.189238+0000","last_clean":"2023-11-26T07:11:30.189238+0000","last_became_active":"2023-11-26T07:11:28.551489+0000","last_became_peered":"2023-11-26T07:11:28.551489+0000","last_unstale":"2023-11-26T07:11:30.189238+0000","last_undegraded":"2023-11-26T07:11:30.189238+0000","last_fullsized":"2023-11-26T07:11:30.189238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540841+0000","last_change":"2023-11-26T07:11:27.530905+0000","last_active":"2023-11-26T07:11:29.540841+0000","last_peered":"2023-11-26T07:11:29.540841+0000","last_clean":"2023-11-26T07:11:29.540841+0000","last_became_active":"2023-11-26T07:11:27.529568+0000","last_became_peered":"2023-11-26T07:11:27.529568+0000","last_unstale":"2023-11-26T07:11:29.540841+0000","last_undegraded":"2023-11-26T07:11:29.540841+0000","last_fullsized":"2023-11-26T07:11:29.540841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:12.389909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555768+0000","last_change":"2023-11-26T07:11:28.547553+0000","last_active":"2023-11-26T07:11:28.555768+0000","last_peered":"2023-11-26T07:11:28.555768+0000","last_clean":"2023-11-26T07:11:28.555768+0000","last_became_active":"2023-11-26T07:11:28.547052+0000","last_became_peered":"2023-11-26T07:11:28.547052+0000","last_unstale":"2023-11-26T07:11:28.555768+0000","last_undegraded":"2023-11-26T07:11:28.555768+0000","last_fullsized":"2023-11-26T07:11:28.555768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:11.545490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540344+0000","last_change":"2023-11-26T07:11:27.528392+0000","last_active":"2023-11-26T07:11:29.540344+0000","last_peered":"2023-11-26T07:11:29.540344+0000","last_clean":"2023-11-26T07:11:29.540344+0000","last_became_active":"2023-11-26T07:11:27.528039+0000","last_became_peered":"2023-11-26T07:11:27.528039+0000","last_unstale":"2023-11-26T07:11:29.540344+0000","last_undegraded":"2023-11-26T07:11:29.540344+0000","last_fullsized":"2023-11-26T07:11:29.540344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.554133+0000","last_change":"2023-11-26T07:11:28.546262+0000","last_active":"2023-11-26T07:11:28.554133+0000","last_peered":"2023-11-26T07:11:28.554133+0000","last_clean":"2023-11-26T07:11:28.554133+0000","last_became_active":"2023-11-26T07:11:28.545998+0000","last_became_peered":"2023-11-26T07:11:28.545998+0000","last_unstale":"2023-11-26T07:11:28.554133+0000","last_undegraded":"2023-11-26T07:11:28.554133+0000","last_fullsized":"2023-11-26T07:11:28.554133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:17.562000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.561753+0000","last_change":"2023-11-26T07:11:27.528591+0000","last_active":"2023-11-26T07:11:28.561753+0000","last_peered":"2023-11-26T07:11:28.561753+0000","last_clean":"2023-11-26T07:11:28.561753+0000","last_became_active":"2023-11-26T07:11:27.528327+0000","last_became_peered":"2023-11-26T07:11:27.528327+0000","last_unstale":"2023-11-26T07:11:28.561753+0000","last_undegraded":"2023-11-26T07:11:28.561753+0000","last_fullsized":"2023-11-26T07:11:28.561753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:21.506925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539218+0000","last_change":"2023-11-26T07:11:28.541089+0000","last_active":"2023-11-26T07:11:29.539218+0000","last_peered":"2023-11-26T07:11:29.539218+0000","last_clean":"2023-11-26T07:11:29.539218+0000","last_became_active":"2023-11-26T07:11:28.534262+0000","last_became_peered":"2023-11-26T07:11:28.534262+0000","last_unstale":"2023-11-26T07:11:29.539218+0000","last_undegraded":"2023-11-26T07:11:29.539218+0000","last_fullsized":"2023-11-26T07:11:29.539218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:33.426915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549517+0000","last_change":"2023-11-26T07:11:27.530892+0000","last_active":"2023-11-26T07:11:28.549517+0000","last_peered":"2023-11-26T07:11:28.549517+0000","last_clean":"2023-11-26T07:11:28.549517+0000","last_became_active":"2023-11-26T07:11:27.529853+0000","last_became_peered":"2023-11-26T07:11:27.529853+0000","last_unstale":"2023-11-26T07:11:28.549517+0000","last_undegraded":"2023-11-26T07:11:28.549517+0000","last_fullsized":"2023-11-26T07:11:28.549517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189422+0000","last_change":"2023-11-26T07:11:28.548912+0000","last_active":"2023-11-26T07:11:30.189422+0000","last_peered":"2023-11-26T07:11:30.189422+0000","last_clean":"2023-11-26T07:11:30.189422+0000","last_became_active":"2023-11-26T07:11:28.547915+0000","last_became_peered":"2023-11-26T07:11:28.547915+0000","last_unstale":"2023-11-26T07:11:30.189422+0000","last_undegraded":"2023-11-26T07:11:30.189422+0000","last_fullsized":"2023-11-26T07:11:30.189422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:42.974584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.537113+0000","last_change":"2023-11-26T07:11:27.529609+0000","last_active":"2023-11-26T07:11:28.537113+0000","last_peered":"2023-11-26T07:11:28.537113+0000","last_clean":"2023-11-26T07:11:28.537113+0000","last_became_active":"2023-11-26T07:11:27.529045+0000","last_became_peered":"2023-11-26T07:11:27.529045+0000","last_unstale":"2023-11-26T07:11:28.537113+0000","last_undegraded":"2023-11-26T07:11:28.537113+0000","last_fullsized":"2023-11-26T07:11:28.537113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:18.223957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.544959+0000","last_change":"2023-11-26T07:11:28.543403+0000","last_active":"2023-11-26T07:11:28.544959+0000","last_peered":"2023-11-26T07:11:28.544959+0000","last_clean":"2023-11-26T07:11:28.544959+0000","last_became_active":"2023-11-26T07:11:28.543119+0000","last_became_peered":"2023-11-26T07:11:28.543119+0000","last_unstale":"2023-11-26T07:11:28.544959+0000","last_undegraded":"2023-11-26T07:11:28.544959+0000","last_fullsized":"2023-11-26T07:11:28.544959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:36.121867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.533451+0000","last_change":"2023-11-26T07:11:27.528106+0000","last_active":"2023-11-26T07:11:28.533451+0000","last_peered":"2023-11-26T07:11:28.533451+0000","last_clean":"2023-11-26T07:11:28.533451+0000","last_became_active":"2023-11-26T07:11:27.527832+0000","last_became_peered":"2023-11-26T07:11:27.527832+0000","last_unstale":"2023-11-26T07:11:28.533451+0000","last_undegraded":"2023-11-26T07:11:28.533451+0000","last_fullsized":"2023-11-26T07:11:28.533451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:50.240018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537690+0000","last_change":"2023-11-26T07:11:28.543380+0000","last_active":"2023-11-26T07:11:29.537690+0000","last_peered":"2023-11-26T07:11:29.537690+0000","last_clean":"2023-11-26T07:11:29.537690+0000","last_became_active":"2023-11-26T07:11:28.543083+0000","last_became_peered":"2023-11-26T07:11:28.543083+0000","last_unstale":"2023-11-26T07:11:29.537690+0000","last_undegraded":"2023-11-26T07:11:29.537690+0000","last_fullsized":"2023-11-26T07:11:29.537690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.531315+0000","last_change":"2023-11-26T07:11:27.531555+0000","last_active":"2023-11-26T07:11:28.531315+0000","last_peered":"2023-11-26T07:11:28.531315+0000","last_clean":"2023-11-26T07:11:28.531315+0000","last_became_active":"2023-11-26T07:11:27.531241+0000","last_became_peered":"2023-11-26T07:11:27.531241+0000","last_unstale":"2023-11-26T07:11:28.531315+0000","last_undegraded":"2023-11-26T07:11:28.531315+0000","last_fullsized":"2023-11-26T07:11:28.531315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555943+0000","last_change":"2023-11-26T07:11:28.552112+0000","last_active":"2023-11-26T07:11:28.555943+0000","last_peered":"2023-11-26T07:11:28.555943+0000","last_clean":"2023-11-26T07:11:28.555943+0000","last_became_active":"2023-11-26T07:11:28.551827+0000","last_became_peered":"2023-11-26T07:11:28.551827+0000","last_unstale":"2023-11-26T07:11:28.555943+0000","last_undegraded":"2023-11-26T07:11:28.555943+0000","last_fullsized":"2023-11-26T07:11:28.555943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:33.426915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.531252+0000","last_change":"2023-11-26T07:11:27.531668+0000","last_active":"2023-11-26T07:11:28.531252+0000","last_peered":"2023-11-26T07:11:28.531252+0000","last_clean":"2023-11-26T07:11:28.531252+0000","last_became_active":"2023-11-26T07:11:27.531403+0000","last_became_peered":"2023-11-26T07:11:27.531403+0000","last_unstale":"2023-11-26T07:11:28.531252+0000","last_undegraded":"2023-11-26T07:11:28.531252+0000","last_fullsized":"2023-11-26T07:11:28.531252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189378+0000","last_change":"2023-11-26T07:11:28.542823+0000","last_active":"2023-11-26T07:11:30.189378+0000","last_peered":"2023-11-26T07:11:30.189378+0000","last_clean":"2023-11-26T07:11:30.189378+0000","last_became_active":"2023-11-26T07:11:28.542535+0000","last_became_peered":"2023-11-26T07:11:28.542535+0000","last_unstale":"2023-11-26T07:11:30.189378+0000","last_undegraded":"2023-11-26T07:11:30.189378+0000","last_fullsized":"2023-11-26T07:11:30.189378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537747+0000","last_change":"2023-11-26T07:11:27.531081+0000","last_active":"2023-11-26T07:11:29.537747+0000","last_peered":"2023-11-26T07:11:29.537747+0000","last_clean":"2023-11-26T07:11:29.537747+0000","last_became_active":"2023-11-26T07:11:27.530823+0000","last_became_peered":"2023-11-26T07:11:27.530823+0000","last_unstale":"2023-11-26T07:11:29.537747+0000","last_undegraded":"2023-11-26T07:11:29.537747+0000","last_fullsized":"2023-11-26T07:11:29.537747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:21.506925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539660+0000","last_change":"2023-11-26T07:11:28.529675+0000","last_active":"2023-11-26T07:11:29.539660+0000","last_peered":"2023-11-26T07:11:29.539660+0000","last_clean":"2023-11-26T07:11:29.539660+0000","last_became_active":"2023-11-26T07:11:28.529388+0000","last_became_peered":"2023-11-26T07:11:28.529388+0000","last_unstale":"2023-11-26T07:11:29.539660+0000","last_undegraded":"2023-11-26T07:11:29.539660+0000","last_fullsized":"2023-11-26T07:11:29.539660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:32.075747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.563480+0000","last_change":"2023-11-26T07:11:27.534932+0000","last_active":"2023-11-26T07:11:28.563480+0000","last_peered":"2023-11-26T07:11:28.563480+0000","last_clean":"2023-11-26T07:11:28.563480+0000","last_became_active":"2023-11-26T07:11:27.534626+0000","last_became_peered":"2023-11-26T07:11:27.534626+0000","last_unstale":"2023-11-26T07:11:28.563480+0000","last_undegraded":"2023-11-26T07:11:28.563480+0000","last_fullsized":"2023-11-26T07:11:28.563480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:25.399589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540678+0000","last_change":"2023-11-26T07:11:28.538363+0000","last_active":"2023-11-26T07:11:29.540678+0000","last_peered":"2023-11-26T07:11:29.540678+0000","last_clean":"2023-11-26T07:11:29.540678+0000","last_became_active":"2023-11-26T07:11:28.538098+0000","last_became_peered":"2023-11-26T07:11:28.538098+0000","last_unstale":"2023-11-26T07:11:29.540678+0000","last_undegraded":"2023-11-26T07:11:29.540678+0000","last_fullsized":"2023-11-26T07:11:29.540678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:00.284371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189378+0000","last_change":"2023-11-26T07:11:27.522269+0000","last_active":"2023-11-26T07:11:30.189378+0000","last_peered":"2023-11-26T07:11:30.189378+0000","last_clean":"2023-11-26T07:11:30.189378+0000","last_became_active":"2023-11-26T07:11:27.521852+0000","last_became_peered":"2023-11-26T07:11:27.521852+0000","last_unstale":"2023-11-26T07:11:30.189378+0000","last_undegraded":"2023-11-26T07:11:30.189378+0000","last_fullsized":"2023-11-26T07:11:30.189378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189106+0000","last_change":"2023-11-26T07:11:28.552836+0000","last_active":"2023-11-26T07:11:30.189106+0000","last_peered":"2023-11-26T07:11:30.189106+0000","last_clean":"2023-11-26T07:11:30.189106+0000","last_became_active":"2023-11-26T07:11:28.552532+0000","last_became_peered":"2023-11-26T07:11:28.552532+0000","last_unstale":"2023-11-26T07:11:30.189106+0000","last_undegraded":"2023-11-26T07:11:30.189106+0000","last_fullsized":"2023-11-26T07:11:30.189106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:51.244423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.536282+0000","last_change":"2023-11-26T07:11:27.534390+0000","last_active":"2023-11-26T07:11:28.536282+0000","last_peered":"2023-11-26T07:11:28.536282+0000","last_clean":"2023-11-26T07:11:28.536282+0000","last_became_active":"2023-11-26T07:11:27.534100+0000","last_became_peered":"2023-11-26T07:11:27.534100+0000","last_unstale":"2023-11-26T07:11:28.536282+0000","last_undegraded":"2023-11-26T07:11:28.536282+0000","last_fullsized":"2023-11-26T07:11:28.536282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:15.199340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540363+0000","last_change":"2023-11-26T07:11:28.530599+0000","last_active":"2023-11-26T07:11:29.540363+0000","last_peered":"2023-11-26T07:11:29.540363+0000","last_clean":"2023-11-26T07:11:29.540363+0000","last_became_active":"2023-11-26T07:11:28.530091+0000","last_became_peered":"2023-11-26T07:11:28.530091+0000","last_unstale":"2023-11-26T07:11:29.540363+0000","last_undegraded":"2023-11-26T07:11:29.540363+0000","last_fullsized":"2023-11-26T07:11:29.540363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:21:12.239221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.530812+0000","last_change":"2023-11-26T07:11:27.530816+0000","last_active":"2023-11-26T07:11:28.530812+0000","last_peered":"2023-11-26T07:11:28.530812+0000","last_clean":"2023-11-26T07:11:28.530812+0000","last_became_active":"2023-11-26T07:11:27.530026+0000","last_became_peered":"2023-11-26T07:11:27.530026+0000","last_unstale":"2023-11-26T07:11:28.530812+0000","last_undegraded":"2023-11-26T07:11:28.530812+0000","last_fullsized":"2023-11-26T07:11:28.530812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:32.422510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.546897+0000","last_change":"2023-11-26T07:11:28.542511+0000","last_active":"2023-11-26T07:11:28.546897+0000","last_peered":"2023-11-26T07:11:28.546897+0000","last_clean":"2023-11-26T07:11:28.546897+0000","last_became_active":"2023-11-26T07:11:28.542221+0000","last_became_peered":"2023-11-26T07:11:28.542221+0000","last_unstale":"2023-11-26T07:11:28.546897+0000","last_undegraded":"2023-11-26T07:11:28.546897+0000","last_fullsized":"2023-11-26T07:11:28.546897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:53.475994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.556270+0000","last_change":"2023-11-26T07:11:27.524724+0000","last_active":"2023-11-26T07:11:28.556270+0000","last_peered":"2023-11-26T07:11:28.556270+0000","last_clean":"2023-11-26T07:11:28.556270+0000","last_became_active":"2023-11-26T07:11:27.524453+0000","last_became_peered":"2023-11-26T07:11:27.524453+0000","last_unstale":"2023-11-26T07:11:28.556270+0000","last_undegraded":"2023-11-26T07:11:28.556270+0000","last_fullsized":"2023-11-26T07:11:28.556270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:32.422510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540122+0000","last_change":"2023-11-26T07:11:28.530396+0000","last_active":"2023-11-26T07:11:29.540122+0000","last_peered":"2023-11-26T07:11:29.540122+0000","last_clean":"2023-11-26T07:11:29.540122+0000","last_became_active":"2023-11-26T07:11:28.530134+0000","last_became_peered":"2023-11-26T07:11:28.530134+0000","last_unstale":"2023-11-26T07:11:29.540122+0000","last_undegraded":"2023-11-26T07:11:29.540122+0000","last_fullsized":"2023-11-26T07:11:29.540122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:58.985237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539626+0000","last_change":"2023-11-26T07:11:27.526573+0000","last_active":"2023-11-26T07:11:29.539626+0000","last_peered":"2023-11-26T07:11:29.539626+0000","last_clean":"2023-11-26T07:11:29.539626+0000","last_became_active":"2023-11-26T07:11:27.526278+0000","last_became_peered":"2023-11-26T07:11:27.526278+0000","last_unstale":"2023-11-26T07:11:29.539626+0000","last_undegraded":"2023-11-26T07:11:29.539626+0000","last_fullsized":"2023-11-26T07:11:29.539626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:35.117462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540624+0000","last_change":"2023-11-26T07:11:28.530307+0000","last_active":"2023-11-26T07:11:29.540624+0000","last_peered":"2023-11-26T07:11:29.540624+0000","last_clean":"2023-11-26T07:11:29.540624+0000","last_became_active":"2023-11-26T07:11:28.530012+0000","last_became_peered":"2023-11-26T07:11:28.530012+0000","last_unstale":"2023-11-26T07:11:29.540624+0000","last_undegraded":"2023-11-26T07:11:29.540624+0000","last_fullsized":"2023-11-26T07:11:29.540624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:51.665872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537662+0000","last_change":"2023-11-26T07:11:27.525592+0000","last_active":"2023-11-26T07:11:29.537662+0000","last_peered":"2023-11-26T07:11:29.537662+0000","last_clean":"2023-11-26T07:11:29.537662+0000","last_became_active":"2023-11-26T07:11:27.525304+0000","last_became_peered":"2023-11-26T07:11:27.525304+0000","last_unstale":"2023-11-26T07:11:29.537662+0000","last_undegraded":"2023-11-26T07:11:29.537662+0000","last_fullsized":"2023-11-26T07:11:29.537662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:52.471590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.547994+0000","last_change":"2023-11-26T07:11:28.538490+0000","last_active":"2023-11-26T07:11:28.547994+0000","last_peered":"2023-11-26T07:11:28.547994+0000","last_clean":"2023-11-26T07:11:28.547994+0000","last_became_active":"2023-11-26T07:11:28.538185+0000","last_became_peered":"2023-11-26T07:11:28.538185+0000","last_unstale":"2023-11-26T07:11:28.547994+0000","last_undegraded":"2023-11-26T07:11:28.547994+0000","last_fullsized":"2023-11-26T07:11:28.547994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:32.075747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.536144+0000","last_change":"2023-11-26T07:11:27.524225+0000","last_active":"2023-11-26T07:11:28.536144+0000","last_peered":"2023-11-26T07:11:28.536144+0000","last_clean":"2023-11-26T07:11:28.536144+0000","last_became_active":"2023-11-26T07:11:27.523610+0000","last_became_peered":"2023-11-26T07:11:27.523610+0000","last_unstale":"2023-11-26T07:11:28.536144+0000","last_undegraded":"2023-11-26T07:11:28.536144+0000","last_fullsized":"2023-11-26T07:11:28.536144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:19.248482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539423+0000","last_change":"2023-11-26T07:11:28.547971+0000","last_active":"2023-11-26T07:11:29.539423+0000","last_peered":"2023-11-26T07:11:29.539423+0000","last_clean":"2023-11-26T07:11:29.539423+0000","last_became_active":"2023-11-26T07:11:28.547756+0000","last_became_peered":"2023-11-26T07:11:28.547756+0000","last_unstale":"2023-11-26T07:11:29.539423+0000","last_undegraded":"2023-11-26T07:11:29.539423+0000","last_fullsized":"2023-11-26T07:11:29.539423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:57.098847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537553+0000","last_change":"2023-11-26T07:11:28.542253+0000","last_active":"2023-11-26T07:11:29.537553+0000","last_peered":"2023-11-26T07:11:29.537553+0000","last_clean":"2023-11-26T07:11:29.537553+0000","last_became_active":"2023-11-26T07:11:28.541977+0000","last_became_peered":"2023-11-26T07:11:28.541977+0000","last_unstale":"2023-11-26T07:11:29.537553+0000","last_undegraded":"2023-11-26T07:11:29.537553+0000","last_fullsized":"2023-11-26T07:11:29.537553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.535254+0000","last_change":"2023-11-26T07:11:27.534895+0000","last_active":"2023-11-26T07:11:28.535254+0000","last_peered":"2023-11-26T07:11:28.535254+0000","last_clean":"2023-11-26T07:11:28.535254+0000","last_became_active":"2023-11-26T07:11:27.534654+0000","last_became_peered":"2023-11-26T07:11:27.534654+0000","last_unstale":"2023-11-26T07:11:28.535254+0000","last_undegraded":"2023-11-26T07:11:28.535254+0000","last_fullsized":"2023-11-26T07:11:28.535254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:18.223957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.545545+0000","last_change":"2023-11-26T07:11:28.544575+0000","last_active":"2023-11-26T07:11:28.545545+0000","last_peered":"2023-11-26T07:11:28.545545+0000","last_clean":"2023-11-26T07:11:28.545545+0000","last_became_active":"2023-11-26T07:11:28.543760+0000","last_became_peered":"2023-11-26T07:11:28.543760+0000","last_unstale":"2023-11-26T07:11:28.545545+0000","last_undegraded":"2023-11-26T07:11:28.545545+0000","last_fullsized":"2023-11-26T07:11:28.545545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:46.756916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.530620+0000","last_change":"2023-11-26T07:11:27.530887+0000","last_active":"2023-11-26T07:11:28.530620+0000","last_peered":"2023-11-26T07:11:28.530620+0000","last_clean":"2023-11-26T07:11:28.530620+0000","last_became_active":"2023-11-26T07:11:27.530613+0000","last_became_peered":"2023-11-26T07:11:27.530613+0000","last_unstale":"2023-11-26T07:11:28.530620+0000","last_undegraded":"2023-11-26T07:11:28.530620+0000","last_fullsized":"2023-11-26T07:11:28.530620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:35.117462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.525772+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:11:28.525772+0000","last_peered":"2023-11-26T07:11:28.525772+0000","last_clean":"2023-11-26T07:11:28.525772+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:11:28.525772+0000","last_undegraded":"2023-11-26T07:11:28.525772+0000","last_fullsized":"2023-11-26T07:11:28.525772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189105+0000","last_change":"2023-11-26T07:11:28.548666+0000","last_active":"2023-11-26T07:11:30.189105+0000","last_peered":"2023-11-26T07:11:30.189105+0000","last_clean":"2023-11-26T07:11:30.189105+0000","last_became_active":"2023-11-26T07:11:28.548390+0000","last_became_peered":"2023-11-26T07:11:28.548390+0000","last_unstale":"2023-11-26T07:11:30.189105+0000","last_undegraded":"2023-11-26T07:11:30.189105+0000","last_fullsized":"2023-11-26T07:11:30.189105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:26.403994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539972+0000","last_change":"2023-11-26T07:11:27.525655+0000","last_active":"2023-11-26T07:11:29.539972+0000","last_peered":"2023-11-26T07:11:29.539972+0000","last_clean":"2023-11-26T07:11:29.539972+0000","last_became_active":"2023-11-26T07:11:27.525321+0000","last_became_peered":"2023-11-26T07:11:27.525321+0000","last_unstale":"2023-11-26T07:11:29.539972+0000","last_undegraded":"2023-11-26T07:11:29.539972+0000","last_fullsized":"2023-11-26T07:11:29.539972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:47.995242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540819+0000","last_change":"2023-11-26T07:11:27.528743+0000","last_active":"2023-11-26T07:11:29.540819+0000","last_peered":"2023-11-26T07:11:29.540819+0000","last_clean":"2023-11-26T07:11:29.540819+0000","last_became_active":"2023-11-26T07:11:27.527803+0000","last_became_peered":"2023-11-26T07:11:27.527803+0000","last_unstale":"2023-11-26T07:11:29.540819+0000","last_undegraded":"2023-11-26T07:11:29.540819+0000","last_fullsized":"2023-11-26T07:11:29.540819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:16:53.693262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537674+0000","last_change":"2023-11-26T07:11:28.542614+0000","last_active":"2023-11-26T07:11:29.537674+0000","last_peered":"2023-11-26T07:11:29.537674+0000","last_clean":"2023-11-26T07:11:29.537674+0000","last_became_active":"2023-11-26T07:11:28.542356+0000","last_became_peered":"2023-11-26T07:11:28.542356+0000","last_unstale":"2023-11-26T07:11:29.537674+0000","last_undegraded":"2023-11-26T07:11:29.537674+0000","last_fullsized":"2023-11-26T07:11:29.537674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537749+0000","last_change":"2023-11-26T07:11:27.527868+0000","last_active":"2023-11-26T07:11:29.537749+0000","last_peered":"2023-11-26T07:11:29.537749+0000","last_clean":"2023-11-26T07:11:29.537749+0000","last_became_active":"2023-11-26T07:11:27.527482+0000","last_became_peered":"2023-11-26T07:11:27.527482+0000","last_unstale":"2023-11-26T07:11:29.537749+0000","last_undegraded":"2023-11-26T07:11:29.537749+0000","last_fullsized":"2023-11-26T07:11:29.537749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.562118+0000","last_change":"2023-11-26T07:11:28.554487+0000","last_active":"2023-11-26T07:11:28.562118+0000","last_peered":"2023-11-26T07:11:28.562118+0000","last_clean":"2023-11-26T07:11:28.562118+0000","last_became_active":"2023-11-26T07:11:28.554210+0000","last_became_peered":"2023-11-26T07:11:28.554210+0000","last_unstale":"2023-11-26T07:11:28.562118+0000","last_undegraded":"2023-11-26T07:11:28.562118+0000","last_fullsized":"2023-11-26T07:11:28.562118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:53.475994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555204+0000","last_change":"2023-11-26T07:11:27.531628+0000","last_active":"2023-11-26T07:11:28.555204+0000","last_peered":"2023-11-26T07:11:28.555204+0000","last_clean":"2023-11-26T07:11:28.555204+0000","last_became_active":"2023-11-26T07:11:27.531336+0000","last_became_peered":"2023-11-26T07:11:27.531336+0000","last_unstale":"2023-11-26T07:11:28.555204+0000","last_undegraded":"2023-11-26T07:11:28.555204+0000","last_fullsized":"2023-11-26T07:11:28.555204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:10.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189372+0000","last_change":"2023-11-26T07:11:28.540197+0000","last_active":"2023-11-26T07:11:30.189372+0000","last_peered":"2023-11-26T07:11:30.189372+0000","last_clean":"2023-11-26T07:11:30.189372+0000","last_became_active":"2023-11-26T07:11:28.539930+0000","last_became_peered":"2023-11-26T07:11:28.539930+0000","last_unstale":"2023-11-26T07:11:30.189372+0000","last_undegraded":"2023-11-26T07:11:30.189372+0000","last_fullsized":"2023-11-26T07:11:30.189372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539876+0000","last_change":"2023-11-26T07:11:27.528055+0000","last_active":"2023-11-26T07:11:29.539876+0000","last_peered":"2023-11-26T07:11:29.539876+0000","last_clean":"2023-11-26T07:11:29.539876+0000","last_became_active":"2023-11-26T07:11:27.527760+0000","last_became_peered":"2023-11-26T07:11:27.527760+0000","last_unstale":"2023-11-26T07:11:29.539876+0000","last_undegraded":"2023-11-26T07:11:29.539876+0000","last_fullsized":"2023-11-26T07:11:29.539876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:10.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539630+0000","last_change":"2023-11-26T07:11:28.544985+0000","last_active":"2023-11-26T07:11:29.539630+0000","last_peered":"2023-11-26T07:11:29.539630+0000","last_clean":"2023-11-26T07:11:29.539630+0000","last_became_active":"2023-11-26T07:11:28.544546+0000","last_became_peered":"2023-11-26T07:11:28.544546+0000","last_unstale":"2023-11-26T07:11:29.539630+0000","last_undegraded":"2023-11-26T07:11:29.539630+0000","last_fullsized":"2023-11-26T07:11:29.539630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:19.228362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537514+0000","last_change":"2023-11-26T07:11:27.527741+0000","last_active":"2023-11-26T07:11:29.537514+0000","last_peered":"2023-11-26T07:11:29.537514+0000","last_clean":"2023-11-26T07:11:29.537514+0000","last_became_active":"2023-11-26T07:11:27.526479+0000","last_became_peered":"2023-11-26T07:11:27.526479+0000","last_unstale":"2023-11-26T07:11:29.537514+0000","last_undegraded":"2023-11-26T07:11:29.537514+0000","last_fullsized":"2023-11-26T07:11:29.537514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:31.071342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.543414+0000","last_change":"2023-11-26T07:11:28.540204+0000","last_active":"2023-11-26T07:11:28.543414+0000","last_peered":"2023-11-26T07:11:28.543414+0000","last_clean":"2023-11-26T07:11:28.543414+0000","last_became_active":"2023-11-26T07:11:28.539891+0000","last_became_peered":"2023-11-26T07:11:28.539891+0000","last_unstale":"2023-11-26T07:11:28.543414+0000","last_undegraded":"2023-11-26T07:11:28.543414+0000","last_fullsized":"2023-11-26T07:11:28.543414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:57.098847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537841+0000","last_change":"2023-11-26T07:11:27.534469+0000","last_active":"2023-11-26T07:11:29.537841+0000","last_peered":"2023-11-26T07:11:29.537841+0000","last_clean":"2023-11-26T07:11:29.537841+0000","last_became_active":"2023-11-26T07:11:27.534116+0000","last_became_peered":"2023-11-26T07:11:27.534116+0000","last_unstale":"2023-11-26T07:11:29.537841+0000","last_undegraded":"2023-11-26T07:11:29.537841+0000","last_fullsized":"2023-11-26T07:11:29.537841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:25.399589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549011+0000","last_change":"2023-11-26T07:11:28.547277+0000","last_active":"2023-11-26T07:11:28.549011+0000","last_peered":"2023-11-26T07:11:28.549011+0000","last_clean":"2023-11-26T07:11:28.549011+0000","last_became_active":"2023-11-26T07:11:28.546992+0000","last_became_peered":"2023-11-26T07:11:28.546992+0000","last_unstale":"2023-11-26T07:11:28.549011+0000","last_undegraded":"2023-11-26T07:11:28.549011+0000","last_fullsized":"2023-11-26T07:11:28.549011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:17.562000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540403+0000","last_change":"2023-11-26T07:11:27.529785+0000","last_active":"2023-11-26T07:11:29.540403+0000","last_peered":"2023-11-26T07:11:29.540403+0000","last_clean":"2023-11-26T07:11:29.540403+0000","last_became_active":"2023-11-26T07:11:27.528686+0000","last_became_peered":"2023-11-26T07:11:27.528686+0000","last_unstale":"2023-11-26T07:11:29.540403+0000","last_undegraded":"2023-11-26T07:11:29.540403+0000","last_fullsized":"2023-11-26T07:11:29.540403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:21.053006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549487+0000","last_change":"2023-11-26T07:11:28.546713+0000","last_active":"2023-11-26T07:11:28.549487+0000","last_peered":"2023-11-26T07:11:28.549487+0000","last_clean":"2023-11-26T07:11:28.549487+0000","last_became_active":"2023-11-26T07:11:28.546453+0000","last_became_peered":"2023-11-26T07:11:28.546453+0000","last_unstale":"2023-11-26T07:11:28.549487+0000","last_undegraded":"2023-11-26T07:11:28.549487+0000","last_fullsized":"2023-11-26T07:11:28.549487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:57.098847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.533987+0000","last_change":"2023-11-26T07:11:27.531109+0000","last_active":"2023-11-26T07:11:28.533987+0000","last_peered":"2023-11-26T07:11:28.533987+0000","last_clean":"2023-11-26T07:11:28.533987+0000","last_became_active":"2023-11-26T07:11:27.530786+0000","last_became_peered":"2023-11-26T07:11:27.530786+0000","last_unstale":"2023-11-26T07:11:28.533987+0000","last_undegraded":"2023-11-26T07:11:28.533987+0000","last_fullsized":"2023-11-26T07:11:28.533987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:52.471590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540761+0000","last_change":"2023-11-26T07:11:28.539795+0000","last_active":"2023-11-26T07:11:29.540761+0000","last_peered":"2023-11-26T07:11:29.540761+0000","last_clean":"2023-11-26T07:11:29.540761+0000","last_became_active":"2023-11-26T07:11:28.539562+0000","last_became_peered":"2023-11-26T07:11:28.539562+0000","last_unstale":"2023-11-26T07:11:29.540761+0000","last_undegraded":"2023-11-26T07:11:29.540761+0000","last_fullsized":"2023-11-26T07:11:29.540761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:02:05.837198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537674+0000","last_change":"2023-11-26T07:11:27.522266+0000","last_active":"2023-11-26T07:11:29.537674+0000","last_peered":"2023-11-26T07:11:29.537674+0000","last_clean":"2023-11-26T07:11:29.537674+0000","last_became_active":"2023-11-26T07:11:27.521972+0000","last_became_peered":"2023-11-26T07:11:27.521972+0000","last_unstale":"2023-11-26T07:11:29.537674+0000","last_undegraded":"2023-11-26T07:11:29.537674+0000","last_fullsized":"2023-11-26T07:11:29.537674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.553910+0000","last_change":"2023-11-26T07:11:28.544499+0000","last_active":"2023-11-26T07:11:28.553910+0000","last_peered":"2023-11-26T07:11:28.553910+0000","last_clean":"2023-11-26T07:11:28.553910+0000","last_became_active":"2023-11-26T07:11:28.544154+0000","last_became_peered":"2023-11-26T07:11:28.544154+0000","last_unstale":"2023-11-26T07:11:28.553910+0000","last_undegraded":"2023-11-26T07:11:28.553910+0000","last_fullsized":"2023-11-26T07:11:28.553910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:19.228362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.562209+0000","last_change":"2023-11-26T07:11:27.535061+0000","last_active":"2023-11-26T07:11:28.562209+0000","last_peered":"2023-11-26T07:11:28.562209+0000","last_clean":"2023-11-26T07:11:28.562209+0000","last_became_active":"2023-11-26T07:11:27.534792+0000","last_became_peered":"2023-11-26T07:11:27.534792+0000","last_unstale":"2023-11-26T07:11:28.562209+0000","last_undegraded":"2023-11-26T07:11:28.562209+0000","last_fullsized":"2023-11-26T07:11:28.562209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540720+0000","last_change":"2023-11-26T07:11:28.532153+0000","last_active":"2023-11-26T07:11:29.540720+0000","last_peered":"2023-11-26T07:11:29.540720+0000","last_clean":"2023-11-26T07:11:29.540720+0000","last_became_active":"2023-11-26T07:11:28.531880+0000","last_became_peered":"2023-11-26T07:11:28.531880+0000","last_unstale":"2023-11-26T07:11:29.540720+0000","last_undegraded":"2023-11-26T07:11:29.540720+0000","last_fullsized":"2023-11-26T07:11:29.540720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:02.252744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549518+0000","last_change":"2023-11-26T07:11:28.544445+0000","last_active":"2023-11-26T07:11:28.549518+0000","last_peered":"2023-11-26T07:11:28.549518+0000","last_clean":"2023-11-26T07:11:28.549518+0000","last_became_active":"2023-11-26T07:11:28.544146+0000","last_became_peered":"2023-11-26T07:11:28.544146+0000","last_unstale":"2023-11-26T07:11:28.549518+0000","last_undegraded":"2023-11-26T07:11:28.549518+0000","last_fullsized":"2023-11-26T07:11:28.549518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:48.999647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549521+0000","last_change":"2023-11-26T07:11:28.542940+0000","last_active":"2023-11-26T07:11:28.549521+0000","last_peered":"2023-11-26T07:11:28.549521+0000","last_clean":"2023-11-26T07:11:28.549521+0000","last_became_active":"2023-11-26T07:11:28.542607+0000","last_became_peered":"2023-11-26T07:11:28.542607+0000","last_unstale":"2023-11-26T07:11:28.549521+0000","last_undegraded":"2023-11-26T07:11:28.549521+0000","last_fullsized":"2023-11-26T07:11:28.549521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:36.121867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555845+0000","last_change":"2023-11-26T07:11:28.549728+0000","last_active":"2023-11-26T07:11:28.555845+0000","last_peered":"2023-11-26T07:11:28.555845+0000","last_clean":"2023-11-26T07:11:28.555845+0000","last_became_active":"2023-11-26T07:11:28.543537+0000","last_became_peered":"2023-11-26T07:11:28.543537+0000","last_unstale":"2023-11-26T07:11:28.555845+0000","last_undegraded":"2023-11-26T07:11:28.555845+0000","last_fullsized":"2023-11-26T07:11:28.555845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:48.999647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189278+0000","last_change":"2023-11-26T07:11:28.546921+0000","last_active":"2023-11-26T07:11:30.189278+0000","last_peered":"2023-11-26T07:11:30.189278+0000","last_clean":"2023-11-26T07:11:30.189278+0000","last_became_active":"2023-11-26T07:11:28.546543+0000","last_became_peered":"2023-11-26T07:11:28.546543+0000","last_unstale":"2023-11-26T07:11:30.189278+0000","last_undegraded":"2023-11-26T07:11:30.189278+0000","last_fullsized":"2023-11-26T07:11:30.189278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.551018+0000","last_change":"2023-11-26T07:11:28.549516+0000","last_active":"2023-11-26T07:11:28.551018+0000","last_peered":"2023-11-26T07:11:28.551018+0000","last_clean":"2023-11-26T07:11:28.551018+0000","last_became_active":"2023-11-26T07:11:28.548917+0000","last_became_peered":"2023-11-26T07:11:28.548917+0000","last_unstale":"2023-11-26T07:11:28.551018+0000","last_undegraded":"2023-11-26T07:11:28.551018+0000","last_fullsized":"2023-11-26T07:11:28.551018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:11.545490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537692+0000","last_change":"2023-11-26T07:11:28.541342+0000","last_active":"2023-11-26T07:11:29.537692+0000","last_peered":"2023-11-26T07:11:29.537692+0000","last_clean":"2023-11-26T07:11:29.537692+0000","last_became_active":"2023-11-26T07:11:28.540919+0000","last_became_peered":"2023-11-26T07:11:28.540919+0000","last_unstale":"2023-11-26T07:11:29.537692+0000","last_undegraded":"2023-11-26T07:11:29.537692+0000","last_fullsized":"2023-11-26T07:11:29.537692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.554271+0000","last_change":"2023-11-26T07:11:28.551636+0000","last_active":"2023-11-26T07:11:28.554271+0000","last_peered":"2023-11-26T07:11:28.554271+0000","last_clean":"2023-11-26T07:11:28.554271+0000","last_became_active":"2023-11-26T07:11:28.551335+0000","last_became_peered":"2023-11-26T07:11:28.551335+0000","last_unstale":"2023-11-26T07:11:28.554271+0000","last_undegraded":"2023-11-26T07:11:28.554271+0000","last_fullsized":"2023-11-26T07:11:28.554271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:22.511330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.548898+0000","last_change":"2023-11-26T07:11:28.546402+0000","last_active":"2023-11-26T07:11:28.548898+0000","last_peered":"2023-11-26T07:11:28.548898+0000","last_clean":"2023-11-26T07:11:28.548898+0000","last_became_active":"2023-11-26T07:11:28.546132+0000","last_became_peered":"2023-11-26T07:11:28.546132+0000","last_unstale":"2023-11-26T07:11:28.548898+0000","last_undegraded":"2023-11-26T07:11:28.548898+0000","last_fullsized":"2023-11-26T07:11:28.548898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:26.403994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.555807+0000","last_change":"2023-11-26T07:11:28.548148+0000","last_active":"2023-11-26T07:11:28.555807+0000","last_peered":"2023-11-26T07:11:28.555807+0000","last_clean":"2023-11-26T07:11:28.555807+0000","last_became_active":"2023-11-26T07:11:28.546933+0000","last_became_peered":"2023-11-26T07:11:28.546933+0000","last_unstale":"2023-11-26T07:11:28.555807+0000","last_undegraded":"2023-11-26T07:11:28.555807+0000","last_fullsized":"2023-11-26T07:11:28.555807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:20.471456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.550683+0000","last_change":"2023-11-26T07:11:28.549377+0000","last_active":"2023-11-26T07:11:28.550683+0000","last_peered":"2023-11-26T07:11:28.550683+0000","last_clean":"2023-11-26T07:11:28.550683+0000","last_became_active":"2023-11-26T07:11:28.549093+0000","last_became_peered":"2023-11-26T07:11:28.549093+0000","last_unstale":"2023-11-26T07:11:28.550683+0000","last_undegraded":"2023-11-26T07:11:28.550683+0000","last_fullsized":"2023-11-26T07:11:28.550683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:51.544284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.556832+0000","last_change":"2023-11-26T07:11:28.554891+0000","last_active":"2023-11-26T07:11:28.556832+0000","last_peered":"2023-11-26T07:11:28.556832+0000","last_clean":"2023-11-26T07:11:28.556832+0000","last_became_active":"2023-11-26T07:11:28.554617+0000","last_became_peered":"2023-11-26T07:11:28.554617+0000","last_unstale":"2023-11-26T07:11:28.556832+0000","last_undegraded":"2023-11-26T07:11:28.556832+0000","last_fullsized":"2023-11-26T07:11:28.556832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189270+0000","last_change":"2023-11-26T07:11:28.552756+0000","last_active":"2023-11-26T07:11:30.189270+0000","last_peered":"2023-11-26T07:11:30.189270+0000","last_clean":"2023-11-26T07:11:30.189270+0000","last_became_active":"2023-11-26T07:11:28.552548+0000","last_became_peered":"2023-11-26T07:11:28.552548+0000","last_unstale":"2023-11-26T07:11:30.189270+0000","last_undegraded":"2023-11-26T07:11:30.189270+0000","last_fullsized":"2023-11-26T07:11:30.189270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.549607+0000","last_change":"2023-11-26T07:11:28.547396+0000","last_active":"2023-11-26T07:11:28.549607+0000","last_peered":"2023-11-26T07:11:28.549607+0000","last_clean":"2023-11-26T07:11:28.549607+0000","last_became_active":"2023-11-26T07:11:28.547176+0000","last_became_peered":"2023-11-26T07:11:28.547176+0000","last_unstale":"2023-11-26T07:11:28.549607+0000","last_undegraded":"2023-11-26T07:11:28.549607+0000","last_fullsized":"2023-11-26T07:11:28.549607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:51.244423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":660287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":69281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":69407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":69371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:11:33.008 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:11:33.008 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-26T07:11:33.008 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-26T07:11:33.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-26T07:11:33.009 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-26T07:11:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.162+0000 7f9548e04700 1 Processor -- start 2023-11-26T07:11:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.162+0000 7f9548e04700 1 -- start start 2023-11-26T07:11:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f9548e04700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f954415ddd0 0x7f95440b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f9548e04700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f954415e1b0 0x7f95440cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f9548e04700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95440b1d40 con 0x7f95440d1b30 2023-11-26T07:11:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f9548e04700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95440b28c0 con 0x7f954415ddd0 2023-11-26T07:11:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f9548e04700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954406c000 con 0x7f954415e1b0 2023-11-26T07:11:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f9542d9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f95440d1b30 0x7f9544161c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54192/0 (socket says 172.21.15.83:54192) 2023-11-26T07:11:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f9542d9d700 1 -- 172.21.15.83:0/3013312966 learned_addr learned my addr 172.21.15.83:0/3013312966 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f9541d9b700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f954415e1b0 0x7f95440cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954259c700 1 --2- 172.21.15.83:0/3013312966 >> v2:172.21.15.203:3300/0 conn(0x7f954415ddd0 0x7f95440b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265859686 0 0) 0x7f95440b1d40 con 0x7f95440d1b30 2023-11-26T07:11:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f952c0035b0 con 0x7f95440d1b30 2023-11-26T07:11:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 88670277 0 0) 0x7f952c0035b0 con 0x7f95440d1b30 2023-11-26T07:11:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f954415e1b0 msgr2=0x7f95440cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f954415e1b0 0x7f95440cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 >> v2:172.21.15.203:3300/0 conn(0x7f954415ddd0 msgr2=0x7f95440b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 --2- 172.21.15.83:0/3013312966 >> v2:172.21.15.203:3300/0 conn(0x7f954415ddd0 0x7f95440b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440b1f90 con 0x7f95440d1b30 2023-11-26T07:11:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f953c0019f0 con 0x7f95440d1b30 2023-11-26T07:11:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 >> v1:172.21.15.83:6789/0 conn(0x7f95440d1b30 legacy=0x7f9544161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f954415e1b0 0x7f952c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f952c003f80 0x7f952c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f952c004d90 0x7f952c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95440b28c0 con 0x7f952c004d90 2023-11-26T07:11:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954406c000 con 0x7f952c003f80 2023-11-26T07:11:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.163+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c00baf0 con 0x7f954415e1b0 2023-11-26T07:11:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f9542d9d700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f952c004d90 0x7f952c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f954259c700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f954415e1b0 0x7f952c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f9541d9b700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f952c003f80 0x7f952c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f9542d9d700 1 -- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f954415e1b0 msgr2=0x7f952c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f9542d9d700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f954415e1b0 0x7f952c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f9542d9d700 1 -- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f952c003f80 msgr2=0x7f952c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f9542d9d700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f952c003f80 0x7f952c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f9542d9d700 1 -- 172.21.15.83:0/3013312966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c0035b0 con 0x7f952c004d90 2023-11-26T07:11:33.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f9541d9b700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f952c003f80 0x7f952c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:33.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f9542d9d700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f952c004d90 0x7f952c005210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f953c00a860 tx=0x7f952c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593b5e632c779564 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:33.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c012600 con 0x7f952c004d90 2023-11-26T07:11:33.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c012780 con 0x7f952c004d90 2023-11-26T07:11:33.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f954159a700 1 -- 172.21.15.83:0/3013312966 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c012b30 con 0x7f952c004d90 2023-11-26T07:11:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f9548e04700 1 -- 172.21.15.83:0/3013312966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f952c004d90 msgr2=0x7f952c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.164+0000 7f9548e04700 1 --2- 172.21.15.83:0/3013312966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f952c004d90 0x7f952c005210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f953c00a860 tx=0x7f952c005880 comp rx=0 tx=0).stop 2023-11-26T07:11:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.165+0000 7f9542d9d700 1 -- 172.21.15.83:0/3013312966 reap_dead start 2023-11-26T07:11:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.165+0000 7f9548e04700 1 -- 172.21.15.83:0/3013312966 shutdown_connections 2023-11-26T07:11:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.165+0000 7f9548e04700 1 -- 172.21.15.83:0/3013312966 >> 172.21.15.83:0/3013312966 conn(0x7f95440731a0 msgr2=0x7f954409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.165+0000 7f9548e04700 1 -- 172.21.15.83:0/3013312966 shutdown_connections 2023-11-26T07:11:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.165+0000 7f9548e04700 1 -- 172.21.15.83:0/3013312966 wait complete. 2023-11-26T07:11:33.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.166+0000 7f9548e04700 1 Processor -- start 2023-11-26T07:11:33.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.166+0000 7f9548e04700 1 -- start start 2023-11-26T07:11:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9548e04700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f952c004d90 0x7f9544160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9548e04700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9544152640 0x7f9544155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9548e04700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95441560c0 0x7f95440b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9548e04700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954415f0d0 con 0x7f95441560c0 2023-11-26T07:11:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9548e04700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95441537e0 con 0x7f9544152640 2023-11-26T07:11:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9548e04700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954415f2e0 con 0x7f952c004d90 2023-11-26T07:11:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9542d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95441560c0 0x7f95440b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9542d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95441560c0 0x7f95440b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34164/0 (socket says 172.21.15.83:34164) 2023-11-26T07:11:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9542d9d700 1 -- 172.21.15.83:0/605773449 learned_addr learned my addr 172.21.15.83:0/605773449 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f954259c700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f952c004d90 0x7f9544160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9541d9b700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9544152640 0x7f9544155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9542d9d700 1 -- 172.21.15.83:0/605773449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f952c004d90 msgr2=0x7f9544160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9542d9d700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f952c004d90 0x7f9544160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9542d9d700 1 -- 172.21.15.83:0/605773449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9544152640 msgr2=0x7f9544155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9542d9d700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9544152640 0x7f9544155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9542d9d700 1 -- 172.21.15.83:0/605773449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440b2ed0 con 0x7f95441560c0 2023-11-26T07:11:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9542d9d700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95441560c0 0x7f95440b4df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f953c012000 tx=0x7f953c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9532ffd700 1 -- 172.21.15.83:0/605773449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0019c0 con 0x7f95441560c0 2023-11-26T07:11:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9532ffd700 1 -- 172.21.15.83:0/605773449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c01e400 con 0x7f95441560c0 2023-11-26T07:11:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9548e04700 1 -- 172.21.15.83:0/605773449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954415f5c0 con 0x7f95441560c0 2023-11-26T07:11:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9532ffd700 1 -- 172.21.15.83:0/605773449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c01e7b0 con 0x7f95441560c0 2023-11-26T07:11:33.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.167+0000 7f9548e04700 1 -- 172.21.15.83:0/605773449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954415fb80 con 0x7f95441560c0 2023-11-26T07:11:33.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.168+0000 7f9548e04700 1 -- 172.21.15.83:0/605773449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95441601d0 con 0x7f95441560c0 2023-11-26T07:11:33.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.169+0000 7f9532ffd700 1 -- 172.21.15.83:0/605773449 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f953c019030 con 0x7f95441560c0 2023-11-26T07:11:33.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.169+0000 7f9532ffd700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f952406aed0 0x7f952406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.169+0000 7f9532ffd700 1 -- 172.21.15.83:0/605773449 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f953c08a7a0 con 0x7f95441560c0 2023-11-26T07:11:33.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.169+0000 7f954259c700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f952406aed0 0x7f952406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.169+0000 7f954259c700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f952406aed0 0x7f952406d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9538009c20 tx=0x7f9538008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:33.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.173+0000 7f9532ffd700 1 -- 172.21.15.83:0/605773449 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f953c0558e0 con 0x7f95441560c0 2023-11-26T07:11:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.420+0000 7f9548e04700 1 -- 172.21.15.83:0/605773449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9544067f60 con 0x7f95441560c0 2023-11-26T07:11:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.423+0000 7f9532ffd700 1 -- 172.21.15.83:0/605773449 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171031 (secure 0 0 0) 0x7f953c026090 con 0x7f95441560c0 2023-11-26T07:11:33.427 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.425+0000 7f9548e04700 1 -- 172.21.15.83:0/605773449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f952406aed0 msgr2=0x7f952406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.425+0000 7f9548e04700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f952406aed0 0x7f952406d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9538009c20 tx=0x7f9538008040 comp rx=0 tx=0).stop 2023-11-26T07:11:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.426+0000 7f9548e04700 1 -- 172.21.15.83:0/605773449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95441560c0 msgr2=0x7f95440b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.426+0000 7f9548e04700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95441560c0 0x7f95440b4df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f953c012000 tx=0x7f953c001d80 comp rx=0 tx=0).stop 2023-11-26T07:11:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.426+0000 7f9548e04700 1 -- 172.21.15.83:0/605773449 shutdown_connections 2023-11-26T07:11:33.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.426+0000 7f9548e04700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95441560c0 0x7f95440b4df0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.426+0000 7f9548e04700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f952406aed0 0x7f952406d390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.426+0000 7f9548e04700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f952c004d90 0x7f9544160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.426+0000 7f9548e04700 1 --2- 172.21.15.83:0/605773449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9544152640 0x7f9544155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.426+0000 7f9548e04700 1 -- 172.21.15.83:0/605773449 >> 172.21.15.83:0/605773449 conn(0x7f95440731a0 msgr2=0x7f95440b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:33.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.426+0000 7f9548e04700 1 -- 172.21.15.83:0/605773449 shutdown_connections 2023-11-26T07:11:33.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.426+0000 7f9548e04700 1 -- 172.21.15.83:0/605773449 wait complete. 2023-11-26T07:11:33.439 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":1483230064},{"type":"v1","addr":"172.21.15.83:6833","nonce":1483230064}]},"active_addr":"172.21.15.83:6833/1483230064","active_change":"2023-11-26T07:11:08.299320+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1792598028}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1164473704}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":304865544}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2981312946}]}]} 2023-11-26T07:11:33.442 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-26T07:11:33.442 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-26T07:11:33.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:33.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.562+0000 7f5db1da5700 1 Processor -- start 2023-11-26T07:11:33.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.562+0000 7f5db1da5700 1 -- start start 2023-11-26T07:11:33.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.563+0000 7f5db1da5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5dac0af4d0 0x7f5dac0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.563+0000 7f5db1da5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dac0ae430 con 0x7f5dac06c710 2023-11-26T07:11:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.563+0000 7f5db1da5700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dac0ae710 con 0x7f5dac15e080 2023-11-26T07:11:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.563+0000 7f5db1da5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dac0af0b0 con 0x7f5dac0af4d0 2023-11-26T07:11:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.563+0000 7f5dabfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5dac0af4d0 0x7f5dac0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.563+0000 7f5daaffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5dac06c710 0x7f5dac06e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54210/0 (socket says 172.21.15.83:54210) 2023-11-26T07:11:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.563+0000 7f5daaffd700 1 -- 172.21.15.83:0/3590981578 learned_addr learned my addr 172.21.15.83:0/3590981578 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5daa7fc700 1 -- 172.21.15.83:0/3590981578 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450889869 0 0) 0x7f5dac0ae430 con 0x7f5dac06c710 2023-11-26T07:11:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5daa7fc700 1 -- 172.21.15.83:0/3590981578 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d940035b0 con 0x7f5dac06c710 2023-11-26T07:11:33.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5dabfff700 1 -- 172.21.15.83:0/3590981578 >> v1:172.21.15.203:6789/0 conn(0x7f5dac15e080 legacy=0x7f5dac06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:33.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5dabfff700 1 -- 172.21.15.83:0/3590981578 >> v1:172.21.15.83:6789/0 conn(0x7f5dac06c710 legacy=0x7f5dac06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5dabfff700 1 -- 172.21.15.83:0/3590981578 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac0ae9f0 con 0x7f5dac0af4d0 2023-11-26T07:11:33.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5dabfff700 1 --2- 172.21.15.83:0/3590981578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dac0af4d0 0x7f5dac0b5050 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009d90 tx=0x7f5da400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befb4badcc60bb63 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:33.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5daa7fc700 1 -- 172.21.15.83:0/3590981578 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da400e070 con 0x7f5dac0af4d0 2023-11-26T07:11:33.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5daa7fc700 1 -- 172.21.15.83:0/3590981578 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da400bae0 con 0x7f5dac0af4d0 2023-11-26T07:11:33.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5daa7fc700 1 -- 172.21.15.83:0/3590981578 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da400be90 con 0x7f5dac0af4d0 2023-11-26T07:11:33.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5db1da5700 1 -- 172.21.15.83:0/3590981578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dac0af4d0 msgr2=0x7f5dac0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5db1da5700 1 --2- 172.21.15.83:0/3590981578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dac0af4d0 0x7f5dac0b5050 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009d90 tx=0x7f5da400b330 comp rx=0 tx=0).stop 2023-11-26T07:11:33.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5db1da5700 1 -- 172.21.15.83:0/3590981578 shutdown_connections 2023-11-26T07:11:33.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5db1da5700 1 --2- 172.21.15.83:0/3590981578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dac0af4d0 0x7f5dac0b5050 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.564+0000 7f5db1da5700 1 -- 172.21.15.83:0/3590981578 >> 172.21.15.83:0/3590981578 conn(0x7f5dac071b90 msgr2=0x7f5dac071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:33.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.565+0000 7f5db1da5700 1 -- 172.21.15.83:0/3590981578 shutdown_connections 2023-11-26T07:11:33.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.565+0000 7f5db1da5700 1 -- 172.21.15.83:0/3590981578 wait complete. 2023-11-26T07:11:33.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.566+0000 7f5db1da5700 1 Processor -- start 2023-11-26T07:11:33.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.566+0000 7f5db1da5700 1 -- start start 2023-11-26T07:11:33.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.566+0000 7f5db1da5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5dac06c710 0x7f5dac1419e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5db1da5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dac0af4d0 0x7f5dac0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5db1da5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dac15e080 0x7f5dac0b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5db1da5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac0d0630 con 0x7f5dac15e080 2023-11-26T07:11:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5db1da5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac141f20 con 0x7f5dac0af4d0 2023-11-26T07:11:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5db1da5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac0cdb60 con 0x7f5dac06c710 2023-11-26T07:11:33.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5dabfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dac15e080 0x7f5dac0b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5dabfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dac15e080 0x7f5dac0b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34178/0 (socket says 172.21.15.83:34178) 2023-11-26T07:11:33.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5dabfff700 1 -- 172.21.15.83:0/1947851453 learned_addr learned my addr 172.21.15.83:0/1947851453 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:33.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5daaffd700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dac0af4d0 0x7f5dac0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5dab7fe700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5dac06c710 0x7f5dac1419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5dabfff700 1 -- 172.21.15.83:0/1947851453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5dac06c710 msgr2=0x7f5dac1419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5dabfff700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5dac06c710 0x7f5dac1419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5dabfff700 1 -- 172.21.15.83:0/1947851453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dac0af4d0 msgr2=0x7f5dac0cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5dabfff700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dac0af4d0 0x7f5dac0cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5dabfff700 1 -- 172.21.15.83:0/1947851453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da4009a70 con 0x7f5dac15e080 2023-11-26T07:11:33.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5dab7fe700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5dac06c710 0x7f5dac1419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:33.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5dabfff700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dac15e080 0x7f5dac0b1360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5dac09f040 tx=0x7f5da4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:33.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5d93fff700 1 -- 172.21.15.83:0/1947851453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4004720 con 0x7f5dac15e080 2023-11-26T07:11:33.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5d93fff700 1 -- 172.21.15.83:0/1947851453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da40048a0 con 0x7f5dac15e080 2023-11-26T07:11:33.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5db1da5700 1 -- 172.21.15.83:0/1947851453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dac0d0350 con 0x7f5dac15e080 2023-11-26T07:11:33.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5d93fff700 1 -- 172.21.15.83:0/1947851453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4012670 con 0x7f5dac15e080 2023-11-26T07:11:33.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.567+0000 7f5db1da5700 1 -- 172.21.15.83:0/1947851453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dac15b540 con 0x7f5dac15e080 2023-11-26T07:11:33.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.568+0000 7f5db1da5700 1 -- 172.21.15.83:0/1947851453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dac0753f0 con 0x7f5dac15e080 2023-11-26T07:11:33.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.569+0000 7f5d93fff700 1 -- 172.21.15.83:0/1947851453 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5da40127d0 con 0x7f5dac15e080 2023-11-26T07:11:33.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.569+0000 7f5d93fff700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5d9806aed0 0x7f5d9806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.569+0000 7f5d93fff700 1 -- 172.21.15.83:0/1947851453 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5da408ad00 con 0x7f5dac15e080 2023-11-26T07:11:33.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.569+0000 7f5dab7fe700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5d9806aed0 0x7f5d9806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.570+0000 7f5dab7fe700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5d9806aed0 0x7f5d9806d390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5da0006fd0 tx=0x7f5da0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:33.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.574+0000 7f5d93fff700 1 -- 172.21.15.83:0/1947851453 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5da4055db0 con 0x7f5dac15e080 2023-11-26T07:11:33.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.764+0000 7f5db1da5700 1 -- 172.21.15.83:0/1947851453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dac15b0c0 con 0x7f5dac15e080 2023-11-26T07:11:33.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.765+0000 7f5d93fff700 1 -- 172.21.15.83:0/1947851453 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f5dac15b0c0 con 0x7f5dac15e080 2023-11-26T07:11:33.767 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:33.767 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:33.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 -- 172.21.15.83:0/1947851453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5d9806aed0 msgr2=0x7f5d9806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5d9806aed0 0x7f5d9806d390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5da0006fd0 tx=0x7f5da0008040 comp rx=0 tx=0).stop 2023-11-26T07:11:33.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 -- 172.21.15.83:0/1947851453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dac15e080 msgr2=0x7f5dac0b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dac15e080 0x7f5dac0b1360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5dac09f040 tx=0x7f5da4004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 -- 172.21.15.83:0/1947851453 shutdown_connections 2023-11-26T07:11:33.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dac15e080 0x7f5dac0b1360 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5d9806aed0 0x7f5d9806d390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5dac06c710 0x7f5dac1419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 --2- 172.21.15.83:0/1947851453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dac0af4d0 0x7f5dac0cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 -- 172.21.15.83:0/1947851453 >> 172.21.15.83:0/1947851453 conn(0x7f5dac071b90 msgr2=0x7f5dac0b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:33.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 -- 172.21.15.83:0/1947851453 shutdown_connections 2023-11-26T07:11:33.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.767+0000 7f5db1da5700 1 -- 172.21.15.83:0/1947851453 wait complete. 2023-11-26T07:11:33.781 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-26T07:11:33.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:33.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.899+0000 7f33462d9700 1 Processor -- start 2023-11-26T07:11:33.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.899+0000 7f33462d9700 1 -- start start 2023-11-26T07:11:33.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f33462d9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f334015bdd0 0x7f33401604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f33462d9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33400b3370 0x7f33400b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f33462d9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f33400d1c70 0x7f33400d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f33462d9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33400b2f50 con 0x7f334015bdd0 2023-11-26T07:11:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f33462d9700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33400d2660 con 0x7f33400d1c70 2023-11-26T07:11:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f33462d9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33400b2030 con 0x7f33400b3370 2023-11-26T07:11:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f334015bdd0 0x7f33401604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f334015bdd0 0x7f33401604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34182/0 (socket says 172.21.15.83:34182) 2023-11-26T07:11:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333ffff700 1 -- 172.21.15.83:0/3990915235 learned_addr learned my addr 172.21.15.83:0/3990915235 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333f7fe700 1 --2- 172.21.15.83:0/3990915235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33400b3370 0x7f33400b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f3344876700 1 --2- 172.21.15.83:0/3990915235 >> v2:172.21.15.203:3300/0 conn(0x7f33400d1c70 0x7f33400d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333ffff700 1 -- 172.21.15.83:0/3990915235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33400b3370 msgr2=0x7f33400b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333ffff700 1 --2- 172.21.15.83:0/3990915235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33400b3370 0x7f33400b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333ffff700 1 -- 172.21.15.83:0/3990915235 >> v2:172.21.15.203:3300/0 conn(0x7f33400d1c70 msgr2=0x7f33400d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333ffff700 1 --2- 172.21.15.83:0/3990915235 >> v2:172.21.15.203:3300/0 conn(0x7f33400d1c70 0x7f33400d2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333ffff700 1 -- 172.21.15.83:0/3990915235 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33400b22d0 con 0x7f334015bdd0 2023-11-26T07:11:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333ffff700 1 --2- 172.21.15.83:0/3990915235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f334015bdd0 0x7f33401604e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3340160a20 tx=0x7f3334009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4997fb838ba45099 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:33.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333effd700 1 -- 172.21.15.83:0/3990915235 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333400b870 con 0x7f334015bdd0 2023-11-26T07:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333effd700 1 -- 172.21.15.83:0/3990915235 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333400b9d0 con 0x7f334015bdd0 2023-11-26T07:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.900+0000 7f333effd700 1 -- 172.21.15.83:0/3990915235 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333400bd60 con 0x7f334015bdd0 2023-11-26T07:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.901+0000 7f33462d9700 1 -- 172.21.15.83:0/3990915235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f334015bdd0 msgr2=0x7f33401604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.901+0000 7f33462d9700 1 --2- 172.21.15.83:0/3990915235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f334015bdd0 0x7f33401604e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3340160a20 tx=0x7f3334009bd0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.901+0000 7f33462d9700 1 -- 172.21.15.83:0/3990915235 shutdown_connections 2023-11-26T07:11:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.901+0000 7f33462d9700 1 --2- 172.21.15.83:0/3990915235 >> v2:172.21.15.203:3300/0 conn(0x7f33400d1c70 0x7f33400d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.901+0000 7f33462d9700 1 --2- 172.21.15.83:0/3990915235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f334015bdd0 0x7f33401604e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.901+0000 7f33462d9700 1 --2- 172.21.15.83:0/3990915235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f33400b3370 0x7f33400b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.901+0000 7f33462d9700 1 -- 172.21.15.83:0/3990915235 >> 172.21.15.83:0/3990915235 conn(0x7f33400731a0 msgr2=0x7f334009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.901+0000 7f33462d9700 1 -- 172.21.15.83:0/3990915235 shutdown_connections 2023-11-26T07:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.901+0000 7f33462d9700 1 -- 172.21.15.83:0/3990915235 wait complete. 2023-11-26T07:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.902+0000 7f33462d9700 1 Processor -- start 2023-11-26T07:11:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.902+0000 7f33462d9700 1 -- start start 2023-11-26T07:11:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f33462d9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33400b3370 0x7f33400cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f33462d9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33400d1c70 0x7f33400b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f33462d9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f334015bdd0 0x7f33400cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f33462d9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400c57b0 con 0x7f33400b3370 2023-11-26T07:11:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f33462d9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400ca2f0 con 0x7f33400d1c70 2023-11-26T07:11:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f33462d9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33400ca5d0 con 0x7f334015bdd0 2023-11-26T07:11:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f333ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33400b3370 0x7f33400cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f333ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33400b3370 0x7f33400cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34188/0 (socket says 172.21.15.83:34188) 2023-11-26T07:11:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f333ffff700 1 -- 172.21.15.83:0/206564806 learned_addr learned my addr 172.21.15.83:0/206564806 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f333ffff700 1 -- 172.21.15.83:0/206564806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f334015bdd0 msgr2=0x7f33400cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f333ffff700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f334015bdd0 0x7f33400cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f333ffff700 1 -- 172.21.15.83:0/206564806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33400d1c70 msgr2=0x7f33400b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f333f7fe700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33400d1c70 0x7f33400b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f333ffff700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33400d1c70 0x7f33400b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.903+0000 7f333ffff700 1 -- 172.21.15.83:0/206564806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3334009550 con 0x7f33400b3370 2023-11-26T07:11:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.904+0000 7f333ffff700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33400b3370 0x7f33400cdad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3334000c00 tx=0x7f3334004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.904+0000 7f333cff9700 1 -- 172.21.15.83:0/206564806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334004540 con 0x7f33400b3370 2023-11-26T07:11:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.904+0000 7f333cff9700 1 -- 172.21.15.83:0/206564806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33340036a0 con 0x7f33400b3370 2023-11-26T07:11:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.904+0000 7f33462d9700 1 -- 172.21.15.83:0/206564806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33400ca8b0 con 0x7f33400b3370 2023-11-26T07:11:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.904+0000 7f333cff9700 1 -- 172.21.15.83:0/206564806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334003a50 con 0x7f33400b3370 2023-11-26T07:11:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.904+0000 7f33462d9700 1 -- 172.21.15.83:0/206564806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33400cae70 con 0x7f33400b3370 2023-11-26T07:11:33.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.905+0000 7f33462d9700 1 -- 172.21.15.83:0/206564806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3340067f60 con 0x7f33400b3370 2023-11-26T07:11:33.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.906+0000 7f333cff9700 1 -- 172.21.15.83:0/206564806 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3334021070 con 0x7f33400b3370 2023-11-26T07:11:33.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.906+0000 7f333cff9700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f332c06aed0 0x7f332c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:33.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.906+0000 7f333cff9700 1 -- 172.21.15.83:0/206564806 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f333408b140 con 0x7f33400b3370 2023-11-26T07:11:33.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.906+0000 7f333f7fe700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f332c06aed0 0x7f332c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:33.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.907+0000 7f333f7fe700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f332c06aed0 0x7f332c06d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33400cf740 tx=0x7f3330008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:33.910+0000 7f333cff9700 1 -- 172.21.15.83:0/206564806 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3334056360 con 0x7f33400b3370 2023-11-26T07:11:34.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.100+0000 7f33462d9700 1 -- 172.21.15.83:0/206564806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33400cb090 con 0x7f33400b3370 2023-11-26T07:11:34.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.101+0000 7f333cff9700 1 -- 172.21.15.83:0/206564806 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f3334059980 con 0x7f33400b3370 2023-11-26T07:11:34.103 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:34.103 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:34.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 -- 172.21.15.83:0/206564806 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f332c06aed0 msgr2=0x7f332c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f332c06aed0 0x7f332c06d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33400cf740 tx=0x7f3330008040 comp rx=0 tx=0).stop 2023-11-26T07:11:34.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 -- 172.21.15.83:0/206564806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33400b3370 msgr2=0x7f33400cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33400b3370 0x7f33400cdad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3334000c00 tx=0x7f3334004750 comp rx=0 tx=0).stop 2023-11-26T07:11:34.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 -- 172.21.15.83:0/206564806 shutdown_connections 2023-11-26T07:11:34.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f332c06aed0 0x7f332c06d390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f334015bdd0 0x7f33400cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33400b3370 0x7f33400cdad0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 --2- 172.21.15.83:0/206564806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33400d1c70 0x7f33400b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 -- 172.21.15.83:0/206564806 >> 172.21.15.83:0/206564806 conn(0x7f33400731a0 msgr2=0x7f3340159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 -- 172.21.15.83:0/206564806 shutdown_connections 2023-11-26T07:11:34.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.103+0000 7f33462d9700 1 -- 172.21.15.83:0/206564806 wait complete. 2023-11-26T07:11:34.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-26T07:11:34.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-26T07:11:34.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-26T07:11:34.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-26T07:11:34.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-26T07:11:34.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-26T07:11:34.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-26T07:11:34.120 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-26T07:11:34.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.310+0000 7fc377276700 1 Processor -- start 2023-11-26T07:11:34.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.310+0000 7fc377276700 1 -- start start 2023-11-26T07:11:34.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.311+0000 7fc377276700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3700d0520 0x7fc3700b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.311+0000 7fc377276700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc370071be0 0x7fc3700b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.311+0000 7fc377276700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3700720e0 con 0x7fc37015d610 2023-11-26T07:11:34.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.311+0000 7fc377276700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc370072d60 con 0x7fc370071be0 2023-11-26T07:11:34.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.311+0000 7fc377276700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37006de80 con 0x7fc3700d0520 2023-11-26T07:11:34.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.312+0000 7fc375012700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3700d0520 0x7fc3700b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.312+0000 7fc374811700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc37015d610 0x7fc3700714d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54238/0 (socket says 172.21.15.83:54238) 2023-11-26T07:11:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.312+0000 7fc374811700 1 -- 172.21.15.83:0/3505215022 learned_addr learned my addr 172.21.15.83:0/3505215022 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:34.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.313+0000 7fc375813700 1 --2- 172.21.15.83:0/3505215022 >> v2:172.21.15.83:3300/0 conn(0x7fc370071be0 0x7fc3700b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.313+0000 7fc367fff700 1 -- 172.21.15.83:0/3505215022 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500352941 0 0) 0x7fc3700720e0 con 0x7fc37015d610 2023-11-26T07:11:34.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.313+0000 7fc367fff700 1 -- 172.21.15.83:0/3505215022 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc35c0035b0 con 0x7fc37015d610 2023-11-26T07:11:34.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.313+0000 7fc375813700 1 -- 172.21.15.83:0/3505215022 >> v2:172.21.15.203:3300/0 conn(0x7fc3700d0520 msgr2=0x7fc3700b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.313+0000 7fc375813700 1 --2- 172.21.15.83:0/3505215022 >> v2:172.21.15.203:3300/0 conn(0x7fc3700d0520 0x7fc3700b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.313+0000 7fc375813700 1 -- 172.21.15.83:0/3505215022 >> v1:172.21.15.83:6789/0 conn(0x7fc37015d610 legacy=0x7fc3700714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.313+0000 7fc367fff700 1 -- 172.21.15.83:0/3505215022 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3677208983 0 0) 0x7fc35c0035b0 con 0x7fc37015d610 2023-11-26T07:11:34.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.313+0000 7fc375813700 1 -- 172.21.15.83:0/3505215022 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3700723c0 con 0x7fc370071be0 2023-11-26T07:11:34.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.314+0000 7fc375813700 1 --2- 172.21.15.83:0/3505215022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc370071be0 0x7fc3700b4090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3700cfa00 tx=0x7fc360009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54c068c2f563f36 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.314+0000 7fc367fff700 1 -- 172.21.15.83:0/3505215022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3680019f0 con 0x7fc370071be0 2023-11-26T07:11:34.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.314+0000 7fc367fff700 1 -- 172.21.15.83:0/3505215022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc360004030 con 0x7fc370071be0 2023-11-26T07:11:34.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.314+0000 7fc367fff700 1 -- 172.21.15.83:0/3505215022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3600043a0 con 0x7fc370071be0 2023-11-26T07:11:34.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.315+0000 7fc377276700 1 -- 172.21.15.83:0/3505215022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc370071be0 msgr2=0x7fc3700b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.315+0000 7fc377276700 1 --2- 172.21.15.83:0/3505215022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc370071be0 0x7fc3700b4090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc3700cfa00 tx=0x7fc360009b20 comp rx=0 tx=0).stop 2023-11-26T07:11:34.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.315+0000 7fc377276700 1 -- 172.21.15.83:0/3505215022 shutdown_connections 2023-11-26T07:11:34.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.315+0000 7fc377276700 1 --2- 172.21.15.83:0/3505215022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc370071be0 0x7fc3700b4090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.315+0000 7fc377276700 1 --2- 172.21.15.83:0/3505215022 >> v2:172.21.15.203:3300/0 conn(0x7fc3700d0520 0x7fc3700b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.315+0000 7fc377276700 1 -- 172.21.15.83:0/3505215022 >> 172.21.15.83:0/3505215022 conn(0x7fc3700cfa30 msgr2=0x7fc3700cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.316+0000 7fc377276700 1 -- 172.21.15.83:0/3505215022 shutdown_connections 2023-11-26T07:11:34.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.316+0000 7fc377276700 1 -- 172.21.15.83:0/3505215022 wait complete. 2023-11-26T07:11:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.317+0000 7fc377276700 1 Processor -- start 2023-11-26T07:11:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.318+0000 7fc377276700 1 -- start start 2023-11-26T07:11:34.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc377276700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc370071be0 0x7fc3700cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc377276700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc37015d610 0x7fc37015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc377276700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37013ba60 0x7fc37015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc377276700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37015c430 con 0x7fc37013ba60 2023-11-26T07:11:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc377276700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700cd5a0 con 0x7fc370071be0 2023-11-26T07:11:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc377276700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700cc990 con 0x7fc37015d610 2023-11-26T07:11:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc375813700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37013ba60 0x7fc37015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc375813700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37013ba60 0x7fc37015b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34206/0 (socket says 172.21.15.83:34206) 2023-11-26T07:11:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc375813700 1 -- 172.21.15.83:0/1657735462 learned_addr learned my addr 172.21.15.83:0/1657735462 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc375012700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc370071be0 0x7fc3700cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc375813700 1 -- 172.21.15.83:0/1657735462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc37015d610 msgr2=0x7fc37015bef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:34.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc375813700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc37015d610 0x7fc37015bef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc375813700 1 -- 172.21.15.83:0/1657735462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc370071be0 msgr2=0x7fc3700cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc375813700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc370071be0 0x7fc3700cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.319+0000 7fc375813700 1 -- 172.21.15.83:0/1657735462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3600094a0 con 0x7fc37013ba60 2023-11-26T07:11:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.320+0000 7fc375813700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37013ba60 0x7fc37015b1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc360014040 tx=0x7fc360004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.320+0000 7fc365ffb700 1 -- 172.21.15.83:0/1657735462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360004590 con 0x7fc37013ba60 2023-11-26T07:11:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.320+0000 7fc377276700 1 -- 172.21.15.83:0/1657735462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3700ccc70 con 0x7fc37013ba60 2023-11-26T07:11:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.320+0000 7fc377276700 1 -- 172.21.15.83:0/1657735462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3700cd230 con 0x7fc37013ba60 2023-11-26T07:11:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.320+0000 7fc365ffb700 1 -- 172.21.15.83:0/1657735462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3600107c0 con 0x7fc37013ba60 2023-11-26T07:11:34.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.320+0000 7fc365ffb700 1 -- 172.21.15.83:0/1657735462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360010b30 con 0x7fc37013ba60 2023-11-26T07:11:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.322+0000 7fc377276700 1 -- 172.21.15.83:0/1657735462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc3701570b0 con 0x7fc37013ba60 2023-11-26T07:11:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.323+0000 7fc365ffb700 1 -- 172.21.15.83:0/1657735462 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc360018070 con 0x7fc37013ba60 2023-11-26T07:11:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.323+0000 7fc365ffb700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc35c06b1b0 0x7fc35c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.323+0000 7fc365ffb700 1 -- 172.21.15.83:0/1657735462 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc36008ad00 con 0x7fc37013ba60 2023-11-26T07:11:34.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.323+0000 7fc365ffb700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] conn(0x7fc35c073350 0x7fc35c075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.323+0000 7fc365ffb700 1 -- 172.21.15.83:0/1657735462 --> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc35c075e20 con 0x7fc35c073350 2023-11-26T07:11:34.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.323+0000 7fc365ffb700 1 -- 172.21.15.83:0/1657735462 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc36008d320 con 0x7fc37013ba60 2023-11-26T07:11:34.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.324+0000 7fc374811700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] conn(0x7fc35c073350 0x7fc35c075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.324+0000 7fc374811700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] conn(0x7fc35c073350 0x7fc35c075770 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.325+0000 7fc365ffb700 1 -- 172.21.15.83:0/1657735462 <== osd.6 v2:172.21.15.203:6824/2898154732 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc35c075e20 con 0x7fc35c073350 2023-11-26T07:11:34.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.330+0000 7f49c37d6700 1 Processor -- start 2023-11-26T07:11:34.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.331+0000 7f49c37d6700 1 -- start start 2023-11-26T07:11:34.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.332+0000 7f49c37d6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc15d6a0 0x7f49bc0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.332+0000 7f49c37d6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc0720e0 con 0x7f49bc0b4090 2023-11-26T07:11:34.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.332+0000 7f49c37d6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc0723c0 con 0x7f49bc071be0 2023-11-26T07:11:34.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.332+0000 7f49c37d6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc072d60 con 0x7f49bc15d6a0 2023-11-26T07:11:34.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.332+0000 7f49c0d71700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f49bc0b4090 0x7f49bc0714d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54254/0 (socket says 172.21.15.83:54254) 2023-11-26T07:11:34.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.332+0000 7f49c0d71700 1 -- 172.21.15.83:0/594900124 learned_addr learned my addr 172.21.15.83:0/594900124 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:34.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.332+0000 7f49c1572700 1 --2- 172.21.15.83:0/594900124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc15d6a0 0x7f49bc0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.332+0000 7ffbc6d96700 1 Processor -- start 2023-11-26T07:11:34.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.333+0000 7f49c1572700 1 -- 172.21.15.83:0/594900124 >> v1:172.21.15.203:6789/0 conn(0x7f49bc071be0 legacy=0x7f49bc140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.333+0000 7f49c1572700 1 -- 172.21.15.83:0/594900124 >> v1:172.21.15.83:6789/0 conn(0x7f49bc0b4090 legacy=0x7f49bc0714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.333+0000 7f49b3fff700 1 -- 172.21.15.83:0/594900124 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319214040 0 0) 0x7f49bc0720e0 con 0x7f49bc0b4090 2023-11-26T07:11:34.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.333+0000 7f49c1572700 1 -- 172.21.15.83:0/594900124 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc0726a0 con 0x7f49bc15d6a0 2023-11-26T07:11:34.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.333+0000 7f49c1572700 1 --2- 172.21.15.83:0/594900124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc15d6a0 0x7f49bc0b3b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f49b80091e0 tx=0x7f49b80094f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cf9efd3c5fb3bd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.333+0000 7f49b3fff700 1 -- 172.21.15.83:0/594900124 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8004230 con 0x7f49bc15d6a0 2023-11-26T07:11:34.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.333+0000 7f49b3fff700 1 -- 172.21.15.83:0/594900124 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b8004390 con 0x7f49bc15d6a0 2023-11-26T07:11:34.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.333+0000 7f49b3fff700 1 -- 172.21.15.83:0/594900124 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b800c700 con 0x7f49bc15d6a0 2023-11-26T07:11:34.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.333+0000 7ffbc6d96700 1 -- start start 2023-11-26T07:11:34.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7f49c37d6700 1 -- 172.21.15.83:0/594900124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc15d6a0 msgr2=0x7f49bc0b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7f49c37d6700 1 --2- 172.21.15.83:0/594900124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc15d6a0 0x7f49bc0b3b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f49b80091e0 tx=0x7f49b80094f0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc6d96700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ffbc00b4260 0x7ffbc00714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc6d96700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffbc0071a10 0x7ffbc0140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc6d96700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc00720e0 con 0x7ffbc015d6a0 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc6d96700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc0072d60 con 0x7ffbc0071a10 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc6d96700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc006de80 con 0x7ffbc00b4260 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc5333700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffbc0071a10 0x7ffbc0140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc4b32700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ffbc015d6a0 0x7ffbc00b3b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54258/0 (socket says 172.21.15.83:54258) 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc5333700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffbc0071a10 0x7ffbc0140fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34222/0 (socket says 172.21.15.83:34222) 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc4b32700 1 -- 172.21.15.83:0/1186997128 learned_addr learned my addr 172.21.15.83:0/1186997128 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbbffff700 1 --2- 172.21.15.83:0/1186997128 >> v2:172.21.15.203:3300/0 conn(0x7ffbc00b4260 0x7ffbc00714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbbf7fe700 1 -- 172.21.15.83:0/1186997128 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871585570 0 0) 0x7ffbc00720e0 con 0x7ffbc015d6a0 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbbf7fe700 1 -- 172.21.15.83:0/1186997128 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba80035b0 con 0x7ffbc015d6a0 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc5333700 1 -- 172.21.15.83:0/1186997128 >> v2:172.21.15.203:3300/0 conn(0x7ffbc00b4260 msgr2=0x7ffbc00714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc5333700 1 --2- 172.21.15.83:0/1186997128 >> v2:172.21.15.203:3300/0 conn(0x7ffbc00b4260 0x7ffbc00714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc5333700 1 -- 172.21.15.83:0/1186997128 >> v1:172.21.15.83:6789/0 conn(0x7ffbc015d6a0 legacy=0x7ffbc00b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.334+0000 7ffbc5333700 1 -- 172.21.15.83:0/1186997128 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc00723c0 con 0x7ffbc0071a10 2023-11-26T07:11:34.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.335+0000 7ffbbffff700 1 --2- 172.21.15.83:0/1186997128 >> v2:172.21.15.203:3300/0 conn(0x7ffbc00b4260 0x7ffbc00714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:34.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.335+0000 7ffbc5333700 1 --2- 172.21.15.83:0/1186997128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbc0071a10 0x7ffbc0140fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0062350 tx=0x7ffbb800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba2a8b9d95fb79b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.335+0000 7ffbbf7fe700 1 -- 172.21.15.83:0/1186997128 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb800f070 con 0x7ffbc0071a10 2023-11-26T07:11:34.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.335+0000 7ffbbf7fe700 1 -- 172.21.15.83:0/1186997128 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb800a2e0 con 0x7ffbc0071a10 2023-11-26T07:11:34.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.335+0000 7ffbbf7fe700 1 -- 172.21.15.83:0/1186997128 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb800cb10 con 0x7ffbc0071a10 2023-11-26T07:11:34.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.335+0000 7ffbc6d96700 1 -- 172.21.15.83:0/1186997128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbc0071a10 msgr2=0x7ffbc0140fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.335+0000 7ffbc6d96700 1 --2- 172.21.15.83:0/1186997128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbc0071a10 0x7ffbc0140fd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0062350 tx=0x7ffbb800c330 comp rx=0 tx=0).stop 2023-11-26T07:11:34.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.329+0000 7fc375012700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc35c06b1b0 0x7fc35c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.329+0000 7fc375012700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc35c06b1b0 0x7fc35c06d670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc370140e60 tx=0x7fc36c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.336+0000 7ffbc6d96700 1 -- 172.21.15.83:0/1186997128 shutdown_connections 2023-11-26T07:11:34.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.336+0000 7ffbc6d96700 1 --2- 172.21.15.83:0/1186997128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbc0071a10 0x7ffbc0140fd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.336+0000 7ffbc6d96700 1 --2- 172.21.15.83:0/1186997128 >> v2:172.21.15.203:3300/0 conn(0x7ffbc00b4260 0x7ffbc00714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.336+0000 7ffbc6d96700 1 -- 172.21.15.83:0/1186997128 >> 172.21.15.83:0/1186997128 conn(0x7ffbc00cfa30 msgr2=0x7ffbc00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.336+0000 7ffbc6d96700 1 -- 172.21.15.83:0/1186997128 shutdown_connections 2023-11-26T07:11:34.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.336+0000 7ffbc6d96700 1 -- 172.21.15.83:0/1186997128 wait complete. 2023-11-26T07:11:34.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.337+0000 7f49c37d6700 1 -- 172.21.15.83:0/594900124 shutdown_connections 2023-11-26T07:11:34.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.337+0000 7f49c37d6700 1 --2- 172.21.15.83:0/594900124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc15d6a0 0x7f49bc0b3b50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.337+0000 7f49c37d6700 1 -- 172.21.15.83:0/594900124 >> 172.21.15.83:0/594900124 conn(0x7f49bc0cfa30 msgr2=0x7f49bc0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.337+0000 7f49c37d6700 1 -- 172.21.15.83:0/594900124 shutdown_connections 2023-11-26T07:11:34.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.337+0000 7f49c37d6700 1 -- 172.21.15.83:0/594900124 wait complete. 2023-11-26T07:11:34.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.337+0000 7ffbc6d96700 1 Processor -- start 2023-11-26T07:11:34.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.337+0000 7ffbc6d96700 1 -- start start 2023-11-26T07:11:34.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.338+0000 7ffbc6d96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbc0071a10 0x7ffbc0154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.338+0000 7ffbc6d96700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbc00b4260 0x7ffbc0156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.338+0000 7ffbc6d96700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbc015d6a0 0x7ffbc00cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.338+0000 7ffbc6d96700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc01e67a0 con 0x7ffbc015d6a0 2023-11-26T07:11:34.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.338+0000 7ffbc6d96700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc01e5700 con 0x7ffbc0071a10 2023-11-26T07:11:34.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.338+0000 7ffbc6d96700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc01e59e0 con 0x7ffbc00b4260 2023-11-26T07:11:34.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7ffbbffff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbc00b4260 0x7ffbc0156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7ffbbffff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbc00b4260 0x7ffbc0156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44684/0 (socket says 172.21.15.83:44684) 2023-11-26T07:11:34.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7ffbbffff700 1 -- 172.21.15.83:0/3942368641 learned_addr learned my addr 172.21.15.83:0/3942368641 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7fec07588700 1 Processor -- start 2023-11-26T07:11:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.338+0000 7f41adb1b700 1 Processor -- start 2023-11-26T07:11:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.338+0000 7f41adb1b700 1 -- start start 2023-11-26T07:11:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.338+0000 7f41adb1b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f41a8072bb0 0x7f41a806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.338+0000 7f41adb1b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 0x7f41a8070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7f41adb1b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a808bdb0 con 0x7f41a816ae60 2023-11-26T07:11:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7f41adb1b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a806ce30 con 0x7f41a8072bb0 2023-11-26T07:11:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7f41adb1b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a806d110 con 0x7f41a80b4260 2023-11-26T07:11:34.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7f41a6ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f41a8072bb0 0x7f41a806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7f41a6ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f41a8072bb0 0x7f41a806efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34230/0 (socket says 172.21.15.83:34230) 2023-11-26T07:11:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7f41a6ffd700 1 -- 172.21.15.83:0/1273420019 learned_addr learned my addr 172.21.15.83:0/1273420019 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589863445 0 0) 0x7f41a808bdb0 con 0x7f41a816ae60 2023-11-26T07:11:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41900035b0 con 0x7f41a816ae60 2023-11-26T07:11:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7f41a7fff700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 0x7f41a8070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1367074187 0 0) 0x7f41900035b0 con 0x7f41a816ae60 2023-11-26T07:11:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 msgr2=0x7f41a8070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 0x7f41a8070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 >> v2:172.21.15.83:3300/0 conn(0x7f41a8072bb0 msgr2=0x7f41a806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 --2- 172.21.15.83:0/1273420019 >> v2:172.21.15.83:3300/0 conn(0x7f41a8072bb0 0x7f41a806efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a806d3f0 con 0x7f41a816ae60 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f4198001670 con 0x7f41a816ae60 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 >> v1:172.21.15.203:6789/0 conn(0x7f41a816ae60 legacy=0x7f41a80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 0x7f4190003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4190003f80 0x7f4190004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4190004d90 0x7f419000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a806ce30 con 0x7f4190004d90 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a806d110 con 0x7f4190003f80 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41900052f0 con 0x7f41a80b4260 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a77fe700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 0x7f4190003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f41a6ffd700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4190003f80 0x7f4190004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f41a77fe700 1 -- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4190003f80 msgr2=0x7f4190004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f41a77fe700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4190003f80 0x7f4190004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f41a77fe700 1 -- 172.21.15.83:0/1273420019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4190004d90 msgr2=0x7f419000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f41a77fe700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4190004d90 0x7f419000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f41a77fe700 1 -- 172.21.15.83:0/1273420019 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41900035b0 con 0x7f41a80b4260 2023-11-26T07:11:34.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f41a6ffd700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4190003f80 0x7f4190004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:34.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7fec07588700 1 -- start start 2023-11-26T07:11:34.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7fec07588700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 0x7fec000b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7fec07588700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fec00163860 0x7fec000b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7fec07588700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec000b2e80 con 0x7fec00163480 2023-11-26T07:11:34.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7fec07588700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec00164510 con 0x7fec00163860 2023-11-26T07:11:34.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7fec07588700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec00160610 con 0x7fec000b08b0 2023-11-26T07:11:34.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7fd2957e1700 1 Processor -- start 2023-11-26T07:11:34.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7fd2957e1700 1 -- start start 2023-11-26T07:11:34.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f41a77fe700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 0x7f4190003a00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f419801d220 tx=0x7f4190005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39bda72a3d5dd050 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7fec05324700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fec00163480 0x7fec000b01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:38904/0 (socket says 172.21.15.83:38904) 2023-11-26T07:11:34.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7fec05324700 1 -- 172.21.15.83:0/2181338715 learned_addr learned my addr 172.21.15.83:0/2181338715 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:34.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7fec04b23700 1 --2- 172.21.15.83:0/2181338715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 0x7fec000b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7fec05b25700 1 --2- 172.21.15.83:0/2181338715 >> v2:172.21.15.203:3300/0 conn(0x7fec00163860 0x7fec000b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7ffbc4b32700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbc0071a10 0x7ffbc0154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7ffbbffff700 1 -- 172.21.15.83:0/3942368641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbc0071a10 msgr2=0x7ffbc0154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7ffbbffff700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbc0071a10 0x7ffbc0154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7ffbbffff700 1 -- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbc015d6a0 msgr2=0x7ffbc00cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7f49c37d6700 1 Processor -- start 2023-11-26T07:11:34.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.339+0000 7f49c37d6700 1 -- start start 2023-11-26T07:11:34.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f49c37d6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc071be0 0x7f49bc1e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f49c37d6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc0b4090 0x7f49bc1e3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f49c37d6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49bc15d6a0 0x7f49bc1e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f49c37d6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc204910 con 0x7f49bc15d6a0 2023-11-26T07:11:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f49c37d6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc204d30 con 0x7f49bc0b4090 2023-11-26T07:11:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f49c37d6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc203f90 con 0x7f49bc071be0 2023-11-26T07:11:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f49c1d73700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49bc15d6a0 0x7f49bc1e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f49c0d71700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc0b4090 0x7f49bc1e3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f49c1d73700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49bc15d6a0 0x7f49bc1e3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34242/0 (socket says 172.21.15.83:34242) 2023-11-26T07:11:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f49c1d73700 1 -- 172.21.15.83:0/3338918799 learned_addr learned my addr 172.21.15.83:0/3338918799 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.340+0000 7f49c0d71700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc0b4090 0x7f49bc1e3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:33692/0 (socket says 172.21.15.83:33692) 2023-11-26T07:11:34.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7fec04b23700 1 -- 172.21.15.83:0/2181338715 >> v2:172.21.15.203:3300/0 conn(0x7fec00163860 msgr2=0x7fec000b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7fec04b23700 1 --2- 172.21.15.83:0/2181338715 >> v2:172.21.15.203:3300/0 conn(0x7fec00163860 0x7fec000b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7fec04b23700 1 -- 172.21.15.83:0/2181338715 >> v1:172.21.15.203:6789/0 conn(0x7fec00163480 legacy=0x7fec000b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f49c1d73700 1 -- 172.21.15.83:0/3338918799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc071be0 msgr2=0x7f49bc1e6480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:34.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f49c1d73700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49bc071be0 0x7f49bc1e6480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f49c1d73700 1 -- 172.21.15.83:0/3338918799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc0b4090 msgr2=0x7f49bc1e3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f49c1d73700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49bc0b4090 0x7f49bc1e3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f49c1d73700 1 -- 172.21.15.83:0/3338918799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b8009d30 con 0x7f49bc15d6a0 2023-11-26T07:11:34.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.341+0000 7f49c1d73700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49bc15d6a0 0x7f49bc1e3eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f49b402e900 tx=0x7f49b402ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7ffbbffff700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbc015d6a0 0x7ffbc00cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7ffbbffff700 1 -- 172.21.15.83:0/3942368641 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb00094a0 con 0x7ffbc00b4260 2023-11-26T07:11:34.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7ffbc4b32700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbc0071a10 0x7ffbc0154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7febf7fff700 1 -- 172.21.15.83:0/2181338715 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537194257 0 0) 0x7fec000b2e80 con 0x7fec00163480 2023-11-26T07:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7fec04b23700 1 -- 172.21.15.83:0/2181338715 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec000b2ba0 con 0x7fec000b08b0 2023-11-26T07:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7fec05b25700 1 --2- 172.21.15.83:0/2181338715 >> v2:172.21.15.203:3300/0 conn(0x7fec00163860 0x7fec000b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7f49b1ffb700 1 -- 172.21.15.83:0/3338918799 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4006ba0 con 0x7f49bc15d6a0 2023-11-26T07:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7f49b1ffb700 1 -- 172.21.15.83:0/3338918799 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b4006d00 con 0x7f49bc15d6a0 2023-11-26T07:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7f49b1ffb700 1 -- 172.21.15.83:0/3338918799 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4003c40 con 0x7f49bc15d6a0 2023-11-26T07:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7f49c37d6700 1 -- 172.21.15.83:0/3338918799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc204250 con 0x7f49bc15d6a0 2023-11-26T07:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7f49c37d6700 1 -- 172.21.15.83:0/3338918799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc1e69c0 con 0x7f49bc15d6a0 2023-11-26T07:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7fd2957e1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd2900cf360 0x7fd2900b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7fd2957e1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29015a790 con 0x7fd29015deb0 2023-11-26T07:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7fd2957e1700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2900b4c40 con 0x7fd29015a3b0 2023-11-26T07:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7fd2957e1700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2900cf8b0 con 0x7fd2900cf360 2023-11-26T07:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7fd28f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd29015deb0 0x7fd29015e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54296/0 (socket says 172.21.15.83:54296) 2023-11-26T07:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7fd28f7fe700 1 -- 172.21.15.83:0/2148785366 learned_addr learned my addr 172.21.15.83:0/2148785366 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7fd28e7fc700 1 --2- 172.21.15.83:0/2148785366 >> v2:172.21.15.203:3300/0 conn(0x7fd2900cf360 0x7fd2900b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560582248 0 0) 0x7fd29015a790 con 0x7fd29015deb0 2023-11-26T07:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2780035b0 con 0x7fd29015deb0 2023-11-26T07:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551445704 0 0) 0x7fd2900b4c40 con 0x7fd29015a3b0 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd29015a790 con 0x7fd29015a3b0 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1055150511 0 0) 0x7fd2780035b0 con 0x7fd29015deb0 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 >> v2:172.21.15.203:3300/0 conn(0x7fd2900cf360 msgr2=0x7fd2900b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 --2- 172.21.15.83:0/2148785366 >> v2:172.21.15.203:3300/0 conn(0x7fd2900cf360 0x7fd2900b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 >> v1:172.21.15.203:6789/0 conn(0x7fd29015a3b0 legacy=0x7fd2900b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd290070c60 con 0x7fd29015deb0 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fd2880019f0 con 0x7fd29015deb0 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 >> v1:172.21.15.83:6789/0 conn(0x7fd29015deb0 legacy=0x7fd29015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 --2- 172.21.15.83:0/2148785366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd278003bc0 0x7fd278003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 --2- 172.21.15.83:0/2148785366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd278004850 0x7fd278004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 --2- 172.21.15.83:0/2148785366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2780052e0 0x7fd27800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd284001230 con 0x7fd278004850 2023-11-26T07:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2900cf8b0 con 0x7fd278003bc0 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27800c2b0 con 0x7fd2780052e0 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fd28f7fe700 1 --2- 172.21.15.83:0/2148785366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2780052e0 0x7fd27800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7fd28effd700 1 --2- 172.21.15.83:0/2148785366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd278003bc0 0x7fd278003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7fd28e7fc700 1 --2- 172.21.15.83:0/2148785366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd278004850 0x7fd278004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7fd28e7fc700 1 -- 172.21.15.83:0/2148785366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2780052e0 msgr2=0x7fd27800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7fd28e7fc700 1 --2- 172.21.15.83:0/2148785366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2780052e0 0x7fd27800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7fd28e7fc700 1 -- 172.21.15.83:0/2148785366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd278003bc0 msgr2=0x7fd278003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7fd28e7fc700 1 --2- 172.21.15.83:0/2148785366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd278003bc0 0x7fd278003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7fd28e7fc700 1 -- 172.21.15.83:0/2148785366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780035b0 con 0x7fd278004850 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7fd28e7fc700 1 --2- 172.21.15.83:0/2148785366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd278004850 0x7fd278004cd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd28000aa30 tx=0x7fd278005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b5d6c0d376354e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280004220 con 0x7fd278004850 2023-11-26T07:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280004380 con 0x7fd278004850 2023-11-26T07:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fd28dffb700 1 -- 172.21.15.83:0/2148785366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28000ae60 con 0x7fd278004850 2023-11-26T07:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fd28f7fe700 1 -- 172.21.15.83:0/2148785366 reap_dead start 2023-11-26T07:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fd2957e1700 1 -- 172.21.15.83:0/2148785366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd278004850 msgr2=0x7fd278004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fd2957e1700 1 --2- 172.21.15.83:0/2148785366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd278004850 0x7fd278004cd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd28000aa30 tx=0x7fd278005880 comp rx=0 tx=0).stop 2023-11-26T07:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fd2957e1700 1 -- 172.21.15.83:0/2148785366 shutdown_connections 2023-11-26T07:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fd2957e1700 1 --2- 172.21.15.83:0/2148785366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd278004850 0x7fd278004cd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fd2957e1700 1 -- 172.21.15.83:0/2148785366 >> 172.21.15.83:0/2148785366 conn(0x7fd2900b1740 msgr2=0x7fd2900b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.349+0000 7fd2957e1700 1 -- 172.21.15.83:0/2148785366 shutdown_connections 2023-11-26T07:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.349+0000 7fd2957e1700 1 -- 172.21.15.83:0/2148785366 wait complete. 2023-11-26T07:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7fd2957e1700 1 Processor -- start 2023-11-26T07:11:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7f49c37d6700 1 -- 172.21.15.83:0/3338918799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49bc1416e0 con 0x7f49bc15d6a0 2023-11-26T07:11:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7fec04b23700 1 --2- 172.21.15.83:0/2181338715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 0x7fec000b1e40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7febfc009840 tx=0x7febfc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=725a6c565395520c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7febf7fff700 1 -- 172.21.15.83:0/2181338715 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc011830 con 0x7fec000b08b0 2023-11-26T07:11:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7febf7fff700 1 -- 172.21.15.83:0/2181338715 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc011990 con 0x7fec000b08b0 2023-11-26T07:11:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7febf7fff700 1 -- 172.21.15.83:0/2181338715 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc011d20 con 0x7fec000b08b0 2023-11-26T07:11:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7fec07588700 1 -- 172.21.15.83:0/2181338715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 msgr2=0x7fec000b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7fec07588700 1 --2- 172.21.15.83:0/2181338715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 0x7fec000b1e40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7febfc009840 tx=0x7febfc009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7fec07588700 1 -- 172.21.15.83:0/2181338715 shutdown_connections 2023-11-26T07:11:34.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7fec07588700 1 --2- 172.21.15.83:0/2181338715 >> v2:172.21.15.203:3300/0 conn(0x7fec00163860 0x7fec000b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7fec07588700 1 --2- 172.21.15.83:0/2181338715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 0x7fec000b1e40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7fec07588700 1 -- 172.21.15.83:0/2181338715 >> 172.21.15.83:0/2181338715 conn(0x7fec000b57b0 msgr2=0x7fec000b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7fec07588700 1 -- 172.21.15.83:0/2181338715 shutdown_connections 2023-11-26T07:11:34.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7fec07588700 1 -- 172.21.15.83:0/2181338715 wait complete. 2023-11-26T07:11:34.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.345+0000 7fec07588700 1 Processor -- start 2023-11-26T07:11:34.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7fec07588700 1 -- start start 2023-11-26T07:11:34.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec07588700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 0x7fec0014f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec07588700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec00163480 0x7fec000b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec07588700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec00163860 0x7fec0015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec07588700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec000cdba0 con 0x7fec00163480 2023-11-26T07:11:34.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec07588700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec000b3d30 con 0x7fec00163860 2023-11-26T07:11:34.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec07588700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0014fc80 con 0x7fec000b08b0 2023-11-26T07:11:34.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec05324700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 0x7fec0014f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec05324700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 0x7fec0014f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44724/0 (socket says 172.21.15.83:44724) 2023-11-26T07:11:34.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec05324700 1 -- 172.21.15.83:0/3418836513 learned_addr learned my addr 172.21.15.83:0/3418836513 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec05b25700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec00163860 0x7fec0015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec04b23700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec00163480 0x7fec000b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec05324700 1 -- 172.21.15.83:0/3418836513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec00163860 msgr2=0x7fec0015f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec05324700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec00163860 0x7fec0015f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec05324700 1 -- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec00163480 msgr2=0x7fec000b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec05324700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec00163480 0x7fec000b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fec05324700 1 -- 172.21.15.83:0/3418836513 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf8009a70 con 0x7fec000b08b0 2023-11-26T07:11:34.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fec04b23700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec00163480 0x7fec000b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:34.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fec05324700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 0x7fec0014f6a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7febf000c750 tx=0x7febf000cb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7febf5ffb700 1 -- 172.21.15.83:0/3418836513 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0004c80 con 0x7fec000b08b0 2023-11-26T07:11:34.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7febf5ffb700 1 -- 172.21.15.83:0/3418836513 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf0004e00 con 0x7fec000b08b0 2023-11-26T07:11:34.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fec07588700 1 -- 172.21.15.83:0/3418836513 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febfc0094a0 con 0x7fec000b08b0 2023-11-26T07:11:34.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fec07588700 1 -- 172.21.15.83:0/3418836513 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec00150240 con 0x7fec000b08b0 2023-11-26T07:11:34.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7febf5ffb700 1 -- 172.21.15.83:0/3418836513 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0007310 con 0x7fec000b08b0 2023-11-26T07:11:34.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.350+0000 7febf5ffb700 1 -- 172.21.15.83:0/3418836513 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7febf0007470 con 0x7fec000b08b0 2023-11-26T07:11:34.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7ffbbffff700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbc00b4260 0x7ffbc0156970 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000ec10 tx=0x7ffbb0009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198007bf0 con 0x7f41a80b4260 2023-11-26T07:11:34.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419801dbc0 con 0x7f41a80b4260 2023-11-26T07:11:34.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.342+0000 7f41a67fc700 1 -- 172.21.15.83:0/1273420019 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198023430 con 0x7f41a80b4260 2023-11-26T07:11:34.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7f41adb1b700 1 -- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 msgr2=0x7f4190003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7f41adb1b700 1 --2- 172.21.15.83:0/1273420019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 0x7f4190003a00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f419801d220 tx=0x7f4190005880 comp rx=0 tx=0).stop 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7f41a7fff700 1 -- 172.21.15.83:0/1273420019 reap_dead start 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7f41adb1b700 1 -- 172.21.15.83:0/1273420019 shutdown_connections 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.343+0000 7f41adb1b700 1 -- 172.21.15.83:0/1273420019 >> 172.21.15.83:0/1273420019 conn(0x7f41a80cfa30 msgr2=0x7f41a80cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7f41adb1b700 1 -- 172.21.15.83:0/1273420019 shutdown_connections 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.344+0000 7f41adb1b700 1 -- 172.21.15.83:0/1273420019 wait complete. 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7f41adb1b700 1 Processor -- start 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7f41adb1b700 1 -- start start 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7f41adb1b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 0x7f41a8151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7f41adb1b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41a81683a0 0x7f41a8152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41a6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41a81683a0 0x7f41a8152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41a6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41a81683a0 0x7f41a8152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34276/0 (socket says 172.21.15.83:34276) 2023-11-26T07:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41adb1b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a816b720 0x7f41a816bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41adb1b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a8151b50 con 0x7f41a81683a0 2023-11-26T07:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41adb1b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a8151d80 con 0x7f41a816b720 2023-11-26T07:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41adb1b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a8152060 con 0x7f41a80b4260 2023-11-26T07:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41a77fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 0x7f41a8151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41a6ffd700 1 -- 172.21.15.83:0/408123067 learned_addr learned my addr 172.21.15.83:0/408123067 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41a7fff700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a816b720 0x7f41a816bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41a6ffd700 1 -- 172.21.15.83:0/408123067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 msgr2=0x7f41a8151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41a6ffd700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 0x7f41a8151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41a6ffd700 1 -- 172.21.15.83:0/408123067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a816b720 msgr2=0x7f41a816bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41a6ffd700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a816b720 0x7f41a816bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41a6ffd700 1 -- 172.21.15.83:0/408123067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a8152340 con 0x7f41a81683a0 2023-11-26T07:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7f41a77fe700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41a80b4260 0x7f41a8151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.349+0000 7f41a6ffd700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41a81683a0 0x7f41a8152e20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f419c007940 tx=0x7f419c007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.349+0000 7f418ffff700 1 -- 172.21.15.83:0/408123067 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c00f050 con 0x7f41a81683a0 2023-11-26T07:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.349+0000 7f418ffff700 1 -- 172.21.15.83:0/408123067 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c010040 con 0x7f41a81683a0 2023-11-26T07:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.349+0000 7f418ffff700 1 -- 172.21.15.83:0/408123067 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c004230 con 0x7f41a81683a0 2023-11-26T07:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.349+0000 7f41adb1b700 1 -- 172.21.15.83:0/408123067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a8152620 con 0x7f41a81683a0 2023-11-26T07:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.349+0000 7f41adb1b700 1 -- 172.21.15.83:0/408123067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a8153360 con 0x7f41a81683a0 2023-11-26T07:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.350+0000 7f41adb1b700 1 -- 172.21.15.83:0/408123067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41a81521d0 con 0x7f41a81683a0 2023-11-26T07:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7fe485753700 1 Processor -- start 2023-11-26T07:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.346+0000 7fe485753700 1 -- start start 2023-11-26T07:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fe485753700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe48006eb70 0x7fe480070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fe485753700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe48015b140 0x7fe480071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fe485753700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4800b3000 con 0x7fe480158770 2023-11-26T07:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fe485753700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe48006e300 con 0x7fe48006eb70 2023-11-26T07:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fe485753700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4800b32e0 con 0x7fe48015b140 2023-11-26T07:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fe47f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe48015b140 0x7fe480071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fe47f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe48015b140 0x7fe480071100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:33724/0 (socket says 172.21.15.83:33724) 2023-11-26T07:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.347+0000 7fe47f7fe700 1 -- 172.21.15.83:0/1749328602 learned_addr learned my addr 172.21.15.83:0/1749328602 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fe47f7fe700 1 -- 172.21.15.83:0/1749328602 >> v2:172.21.15.83:3300/0 conn(0x7fe48006eb70 msgr2=0x7fe480070bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-26T07:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fe47f7fe700 1 --2- 172.21.15.83:0/1749328602 >> v2:172.21.15.83:3300/0 conn(0x7fe48006eb70 0x7fe480070bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fe47f7fe700 1 -- 172.21.15.83:0/1749328602 >> v1:172.21.15.83:6789/0 conn(0x7fe480158770 legacy=0x7fe48006e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fe47f7fe700 1 -- 172.21.15.83:0/1749328602 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4800b35c0 con 0x7fe48015b140 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fe47f7fe700 1 --2- 172.21.15.83:0/1749328602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe48015b140 0x7fe480071100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4800b26a0 tx=0x7fe47800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54cc812829362b24 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fe47dffb700 1 -- 172.21.15.83:0/1749328602 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47800e070 con 0x7fe48015b140 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fe47dffb700 1 -- 172.21.15.83:0/1749328602 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe478003a20 con 0x7fe48015b140 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.348+0000 7fe47dffb700 1 -- 172.21.15.83:0/1749328602 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478003dd0 con 0x7fe48015b140 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.349+0000 7fe485753700 1 -- 172.21.15.83:0/1749328602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe48015b140 msgr2=0x7fe480071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.349+0000 7fe485753700 1 --2- 172.21.15.83:0/1749328602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe48015b140 0x7fe480071100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4800b26a0 tx=0x7fe47800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.350+0000 7fe485753700 1 -- 172.21.15.83:0/1749328602 shutdown_connections 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.350+0000 7fe485753700 1 --2- 172.21.15.83:0/1749328602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe48015b140 0x7fe480071100 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.350+0000 7fe485753700 1 --2- 172.21.15.83:0/1749328602 >> v2:172.21.15.83:3300/0 conn(0x7fe48006eb70 0x7fe480070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.350+0000 7fe485753700 1 -- 172.21.15.83:0/1749328602 >> 172.21.15.83:0/1749328602 conn(0x7fe4800b1740 msgr2=0x7fe4800b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.350+0000 7fe485753700 1 -- 172.21.15.83:0/1749328602 shutdown_connections 2023-11-26T07:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.350+0000 7fe485753700 1 -- 172.21.15.83:0/1749328602 wait complete. 2023-11-26T07:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7f49b1ffb700 1 -- 172.21.15.83:0/3338918799 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49b4005070 con 0x7f49bc15d6a0 2023-11-26T07:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7f49b1ffb700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f49a006afa0 0x7f49a006d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7f49b1ffb700 1 -- 172.21.15.83:0/3338918799 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49b40aad80 con 0x7f49bc15d6a0 2023-11-26T07:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7f49b1ffb700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] conn(0x7f49a00731c0 0x7f49a00755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7f49b1ffb700 1 -- 172.21.15.83:0/3338918799 --> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49a0075c90 con 0x7f49a00731c0 2023-11-26T07:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7f49b1ffb700 1 -- 172.21.15.83:0/3338918799 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b40ad3a0 con 0x7f49bc15d6a0 2023-11-26T07:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.352+0000 7f49c0d71700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] conn(0x7f49a00731c0 0x7f49a00755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.352+0000 7f49c1572700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f49a006afa0 0x7f49a006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7febf5ffb700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7febe406aed0 0x7febe406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7febf5ffb700 1 -- 172.21.15.83:0/3418836513 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febf0089250 con 0x7fec000b08b0 2023-11-26T07:11:34.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7fec04b23700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7febe406aed0 0x7febe406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.352+0000 7fec04b23700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7febe406aed0 0x7febe406d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febfc0090b0 tx=0x7febfc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7f418ffff700 1 -- 172.21.15.83:0/408123067 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f419c004390 con 0x7f41a81683a0 2023-11-26T07:11:34.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.352+0000 7f418ffff700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f419006c450 0x7f419006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.352+0000 7f418ffff700 1 -- 172.21.15.83:0/408123067 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f419c089fe0 con 0x7f41a81683a0 2023-11-26T07:11:34.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.352+0000 7f418ffff700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] conn(0x7f4190074670 0x7f4190076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.352+0000 7f418ffff700 1 -- 172.21.15.83:0/408123067 --> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4190077140 con 0x7f4190074670 2023-11-26T07:11:34.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.352+0000 7f418ffff700 1 -- 172.21.15.83:0/408123067 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f419c08a390 con 0x7f41a81683a0 2023-11-26T07:11:34.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.352+0000 7f41a77fe700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f419006c450 0x7f419006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.352+0000 7fe485753700 1 Processor -- start 2023-11-26T07:11:34.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.351+0000 7fd2957e1700 1 -- start start 2023-11-26T07:11:34.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd2957e1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd278004850 0x7fd290154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd2957e1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd290151af0 0x7fd290151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd2957e1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd29015b9b0 0x7fd29015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd2957e1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd290149da0 con 0x7fd278004850 2023-11-26T07:11:34.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd2957e1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2901524b0 con 0x7fd290151af0 2023-11-26T07:11:34.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd2957e1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd290149f30 con 0x7fd29015b9b0 2023-11-26T07:11:34.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fe485753700 1 -- start start 2023-11-26T07:11:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7f41a7fff700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] conn(0x7f4190074670 0x7f4190076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd28effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd278004850 0x7fd290154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd28effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd278004850 0x7fd290154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34292/0 (socket says 172.21.15.83:34292) 2023-11-26T07:11:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd28effd700 1 -- 172.21.15.83:0/3253792374 learned_addr learned my addr 172.21.15.83:0/3253792374 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd28f7fe700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd29015b9b0 0x7fd29015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7f49c1572700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f49a006afa0 0x7f49a006d460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f49bc069230 tx=0x7f49b8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7f49c0d71700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] conn(0x7f49a00731c0 0x7f49a00755e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd28f7fe700 1 -- 172.21.15.83:0/3253792374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd290151af0 msgr2=0x7fd290151f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-26T07:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd28f7fe700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd290151af0 0x7fd290151f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd28f7fe700 1 -- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd278004850 msgr2=0x7fd290154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd28f7fe700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd278004850 0x7fd290154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7fd28f7fe700 1 -- 172.21.15.83:0/3253792374 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd290155380 con 0x7fd29015b9b0 2023-11-26T07:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7f49b1ffb700 1 -- 172.21.15.83:0/3338918799 <== osd.1 v2:172.21.15.83:6808/438950548 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f49a0075c90 con 0x7f49a00731c0 2023-11-26T07:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7ffbbd7fa700 1 -- 172.21.15.83:0/3942368641 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000c940 con 0x7ffbc00b4260 2023-11-26T07:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7ffbbd7fa700 1 -- 172.21.15.83:0/3942368641 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb000caa0 con 0x7ffbc00b4260 2023-11-26T07:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7ffbbd7fa700 1 -- 172.21.15.83:0/3942368641 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000ce10 con 0x7ffbc00b4260 2023-11-26T07:11:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7ffbc6d96700 1 -- 172.21.15.83:0/3942368641 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb800ab80 con 0x7ffbc00b4260 2023-11-26T07:11:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7ffbc6d96700 1 -- 172.21.15.83:0/3942368641 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc01e5fa0 con 0x7ffbc00b4260 2023-11-26T07:11:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7ffbc6d96700 1 -- 172.21.15.83:0/3942368641 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffba0000fb0 con 0x7ffbc00b4260 2023-11-26T07:11:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.353+0000 7f41a77fe700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f419006c450 0x7f419006e910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4198025040 tx=0x7f419801b000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7fe485753700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe48006eb70 0x7fe4800cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7fe485753700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe480158770 0x7fe4800cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7fe485753700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48015b140 0x7fe48014aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7fe485753700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48014a5d0 con 0x7fe48015b140 2023-11-26T07:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7fe485753700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4800cf190 con 0x7fe48006eb70 2023-11-26T07:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7fe485753700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe480155f00 con 0x7fe480158770 2023-11-26T07:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.355+0000 7f41a7fff700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] conn(0x7f4190074670 0x7f4190076a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.355+0000 7ffbbd7fa700 1 -- 172.21.15.83:0/3942368641 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffbb0005a20 con 0x7ffbc00b4260 2023-11-26T07:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7fe47effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe48006eb70 0x7fe4800cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7fe47effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe48006eb70 0x7fe4800cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:33748/0 (socket says 172.21.15.83:33748) 2023-11-26T07:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7fe47effd700 1 -- 172.21.15.83:0/346110885 learned_addr learned my addr 172.21.15.83:0/346110885 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.354+0000 7fe47f7fe700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48015b140 0x7fe48014aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.355+0000 7fe47f7fe700 1 -- 172.21.15.83:0/346110885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe480158770 msgr2=0x7fe4800cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.355+0000 7fe47f7fe700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe480158770 0x7fe4800cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.355+0000 7fe47f7fe700 1 -- 172.21.15.83:0/346110885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe48006eb70 msgr2=0x7fe4800cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.355+0000 7fe47f7fe700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe48006eb70 0x7fe4800cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.355+0000 7fe47f7fe700 1 -- 172.21.15.83:0/346110885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe478009b30 con 0x7fe48015b140 2023-11-26T07:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.355+0000 7fe47f7fe700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48015b140 0x7fe48014aff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe4800b5c80 tx=0x7fe478004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.355+0000 7fd28f7fe700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd29015b9b0 0x7fd29015be60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd29015e290 tx=0x7fd28800a510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.355+0000 7f418ffff700 1 -- 172.21.15.83:0/408123067 <== osd.4 v2:172.21.15.203:6808/1845411432 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f4190077140 con 0x7f4190074670 2023-11-26T07:11:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.356+0000 7ffbbd7fa700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffbac06af20 0x7ffbac06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.356+0000 7ffbc4b32700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffbac06af20 0x7ffbac06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.356+0000 7ffbbd7fa700 1 -- 172.21.15.83:0/3942368641 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbb00895e0 con 0x7ffbc00b4260 2023-11-26T07:11:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.356+0000 7ffbbd7fa700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] conn(0x7ffbac073140 0x7ffbac075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.356+0000 7ffbbd7fa700 1 -- 172.21.15.83:0/3942368641 --> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffbac075c10 con 0x7ffbac073140 2023-11-26T07:11:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.356+0000 7ffbbd7fa700 1 -- 172.21.15.83:0/3942368641 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbb0036ea0 con 0x7ffbc00b4260 2023-11-26T07:11:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.356+0000 7fc377276700 1 -- 172.21.15.83:0/1657735462 --> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc350000f80 con 0x7fc35c073350 2023-11-26T07:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.356+0000 7ffbc5333700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] conn(0x7ffbac073140 0x7ffbac075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.356+0000 7ffbc4b32700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffbac06af20 0x7ffbac06d3e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4004930 tx=0x7ffbb4004dd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.357+0000 7fec07588700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] conn(0x7febe00015d0 0x7febe0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.357+0000 7fec07588700 1 -- 172.21.15.83:0/3418836513 --> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7febe0006bc0 con 0x7febe00015d0 2023-11-26T07:11:34.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.357+0000 7ffbc5333700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] conn(0x7ffbac073140 0x7ffbac075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.357+0000 7fec05b25700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] conn(0x7febe00015d0 0x7febe0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.357+0000 7fec05b25700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] conn(0x7febe00015d0 0x7febe0003a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.358+0000 7fd2777fe700 1 -- 172.21.15.83:0/3253792374 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28800ac20 con 0x7fd29015b9b0 2023-11-26T07:11:34.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.357+0000 7fc365ffb700 1 -- 172.21.15.83:0/1657735462 <== osd.6 v2:172.21.15.203:6824/2898154732 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc350000f80 con 0x7fc35c073350 2023-11-26T07:11:34.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.358+0000 7ffbbd7fa700 1 -- 172.21.15.83:0/3942368641 <== osd.3 v2:172.21.15.83:6816/1108190663 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ffbac075c10 con 0x7ffbac073140 2023-11-26T07:11:34.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.358+0000 7febf5ffb700 1 -- 172.21.15.83:0/3418836513 <== osd.2 v2:172.21.15.83:6818/913771869 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7febe0006bc0 con 0x7febe00015d0 2023-11-26T07:11:34.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.358+0000 7fd2777fe700 1 -- 172.21.15.83:0/3253792374 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28800ad80 con 0x7fd29015b9b0 2023-11-26T07:11:34.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.358+0000 7fd2777fe700 1 -- 172.21.15.83:0/3253792374 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288013630 con 0x7fd29015b9b0 2023-11-26T07:11:34.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.358+0000 7fd2957e1700 1 -- 172.21.15.83:0/3253792374 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29015c720 con 0x7fd29015b9b0 2023-11-26T07:11:34.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.358+0000 7fd2957e1700 1 -- 172.21.15.83:0/3253792374 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29015cce0 con 0x7fd29015b9b0 2023-11-26T07:11:34.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.360+0000 7fe4677fe700 1 -- 172.21.15.83:0/346110885 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478004380 con 0x7fe48015b140 2023-11-26T07:11:34.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.360+0000 7fd2957e1700 1 -- 172.21.15.83:0/3253792374 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd270000fb0 con 0x7fd29015b9b0 2023-11-26T07:11:34.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.360+0000 7fe4677fe700 1 -- 172.21.15.83:0/346110885 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4780044e0 con 0x7fe48015b140 2023-11-26T07:11:34.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.360+0000 7fe4677fe700 1 -- 172.21.15.83:0/346110885 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478004890 con 0x7fe48015b140 2023-11-26T07:11:34.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.360+0000 7fe485753700 1 -- 172.21.15.83:0/346110885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4801561e0 con 0x7fe48015b140 2023-11-26T07:11:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.360+0000 7fe485753700 1 -- 172.21.15.83:0/346110885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4801567a0 con 0x7fe48015b140 2023-11-26T07:11:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.362+0000 7fe4677fe700 1 -- 172.21.15.83:0/346110885 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe478025460 con 0x7fe48015b140 2023-11-26T07:11:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.362+0000 7fe485753700 1 -- 172.21.15.83:0/346110885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4800b41d0 con 0x7fe48015b140 2023-11-26T07:11:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fc377276700 1 -- 172.21.15.83:0/1657735462 >> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] conn(0x7fc35c073350 msgr2=0x7fc35c075770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fc377276700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] conn(0x7fc35c073350 0x7fc35c075770 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fc377276700 1 -- 172.21.15.83:0/1657735462 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc35c06b1b0 msgr2=0x7fc35c06d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fc377276700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc35c06b1b0 0x7fc35c06d670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc370140e60 tx=0x7fc36c008040 comp rx=0 tx=0).stop 2023-11-26T07:11:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fc377276700 1 -- 172.21.15.83:0/1657735462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37013ba60 msgr2=0x7fc37015b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fc377276700 1 --2- 172.21.15.83:0/1657735462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc37013ba60 0x7fc37015b1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc360014040 tx=0x7fc360004750 comp rx=0 tx=0).stop 2023-11-26T07:11:34.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fc375813700 1 -- 172.21.15.83:0/1657735462 reap_dead start 2023-11-26T07:11:34.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fe4677fe700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe46c06af20 0x7fe46c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fe4677fe700 1 -- 172.21.15.83:0/346110885 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe47808a3b0 con 0x7fe48015b140 2023-11-26T07:11:34.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fe4677fe700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] conn(0x7fe46c073140 0x7fe46c075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fc377276700 1 -- 172.21.15.83:0/1657735462 shutdown_connections 2023-11-26T07:11:34.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fc377276700 1 -- 172.21.15.83:0/1657735462 >> 172.21.15.83:0/1657735462 conn(0x7fc3700cfa30 msgr2=0x7fc3701569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.364+0000 7fc377276700 1 -- 172.21.15.83:0/1657735462 shutdown_connections 2023-11-26T07:11:34.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fe4677fe700 1 -- 172.21.15.83:0/346110885 --> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe46c075c10 con 0x7fe46c073140 2023-11-26T07:11:34.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.363+0000 7fe4677fe700 1 -- 172.21.15.83:0/346110885 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe47808ca40 con 0x7fe48015b140 2023-11-26T07:11:34.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.364+0000 7fe47e7fc700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] conn(0x7fe46c073140 0x7fe46c075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.364+0000 7fe47e7fc700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] conn(0x7fe46c073140 0x7fe46c075560 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.365+0000 7fc377276700 1 -- 172.21.15.83:0/1657735462 wait complete. 2023-11-26T07:11:34.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.366+0000 7fd2777fe700 1 -- 172.21.15.83:0/3253792374 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd288013790 con 0x7fd29015b9b0 2023-11-26T07:11:34.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.366+0000 7fd2777fe700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd27c06af50 0x7fd27c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.366+0000 7fd2777fe700 1 -- 172.21.15.83:0/3253792374 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd288089e40 con 0x7fd29015b9b0 2023-11-26T07:11:34.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.367+0000 7fd2777fe700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] conn(0x7fd27c073170 0x7fd27c075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.367+0000 7fd2777fe700 1 -- 172.21.15.83:0/3253792374 --> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd27c075c40 con 0x7fd27c073170 2023-11-26T07:11:34.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.367+0000 7fd2777fe700 1 -- 172.21.15.83:0/3253792374 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd28808c550 con 0x7fd29015b9b0 2023-11-26T07:11:34.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.367+0000 7fd28e7fc700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] conn(0x7fd27c073170 0x7fd27c075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.367+0000 7fd28e7fc700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] conn(0x7fd27c073170 0x7fd27c075590 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.368+0000 7fd2777fe700 1 -- 172.21.15.83:0/3253792374 <== osd.0 v2:172.21.15.83:6800/3940814281 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd27c075c40 con 0x7fd27c073170 2023-11-26T07:11:34.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.366+0000 7fe4677fe700 1 -- 172.21.15.83:0/346110885 <== osd.5 v2:172.21.15.203:6800/2181588544 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe46c075c10 con 0x7fe46c073140 2023-11-26T07:11:34.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.372+0000 7fe47effd700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe46c06af20 0x7fe46c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.373+0000 7fe47effd700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe46c06af20 0x7fe46c06d3e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4700017f0 tx=0x7fe470009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.380+0000 7fd28effd700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd27c06af50 0x7fd27c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.381+0000 7fd28effd700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd27c06af50 0x7fd27c06d410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd284001720 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.393 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-26T07:11:34.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-26T07:11:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.384+0000 7f49c37d6700 1 -- 172.21.15.83:0/3338918799 --> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f49bc204a50 con 0x7f49a00731c0 2023-11-26T07:11:34.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.384+0000 7f49b1ffb700 1 -- 172.21.15.83:0/3338918799 <== osd.1 v2:172.21.15.83:6808/438950548 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f49bc204a50 con 0x7f49a00731c0 2023-11-26T07:11:34.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.385+0000 7f49c37d6700 1 -- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] conn(0x7f49a00731c0 msgr2=0x7f49a00755e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.385+0000 7f49c37d6700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] conn(0x7f49a00731c0 0x7f49a00755e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.386+0000 7f49c37d6700 1 -- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f49a006afa0 msgr2=0x7f49a006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.386+0000 7f49c37d6700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f49a006afa0 0x7f49a006d460 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f49bc069230 tx=0x7f49b8008030 comp rx=0 tx=0).stop 2023-11-26T07:11:34.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.386+0000 7f49c37d6700 1 -- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49bc15d6a0 msgr2=0x7f49bc1e3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.386+0000 7f49c37d6700 1 --2- 172.21.15.83:0/3338918799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49bc15d6a0 0x7f49bc1e3eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f49b402e900 tx=0x7f49b402ec10 comp rx=0 tx=0).stop 2023-11-26T07:11:34.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.386+0000 7f49c1d73700 1 -- 172.21.15.83:0/3338918799 reap_dead start 2023-11-26T07:11:34.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.386+0000 7f49c37d6700 1 -- 172.21.15.83:0/3338918799 shutdown_connections 2023-11-26T07:11:34.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.386+0000 7f49c37d6700 1 -- 172.21.15.83:0/3338918799 >> 172.21.15.83:0/3338918799 conn(0x7f49bc0cfa30 msgr2=0x7f49bc0cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.386+0000 7f49c37d6700 1 -- 172.21.15.83:0/3338918799 shutdown_connections 2023-11-26T07:11:34.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.386+0000 7f49c37d6700 1 -- 172.21.15.83:0/3338918799 wait complete. 2023-11-26T07:11:34.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.387+0000 7ff08e2c2700 1 Processor -- start 2023-11-26T07:11:34.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.387+0000 7f41adb1b700 1 -- 172.21.15.83:0/408123067 --> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f41a806cac0 con 0x7f4190074670 2023-11-26T07:11:34.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7fec07588700 1 -- 172.21.15.83:0/3418836513 --> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7febe0005ca0 con 0x7febe00015d0 2023-11-26T07:11:34.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.388+0000 7ff08e2c2700 1 -- start start 2023-11-26T07:11:34.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7ff08e2c2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08815d6a0 0x7ff0880b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7ff08e2c2700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0880b4090 0x7ff0880714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7ff08e2c2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff088071e70 con 0x7ff088071a10 2023-11-26T07:11:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7ff08e2c2700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff088072d60 con 0x7ff0880b4090 2023-11-26T07:11:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7ff08e2c2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08806de80 con 0x7ff08815d6a0 2023-11-26T07:11:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7f418ffff700 1 -- 172.21.15.83:0/408123067 <== osd.4 v2:172.21.15.203:6808/1845411432 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f41a0015370 con 0x7f4190074670 2023-11-26T07:11:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7febf5ffb700 1 -- 172.21.15.83:0/3418836513 <== osd.2 v2:172.21.15.83:6818/913771869 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7febe0005ca0 con 0x7febe00015d0 2023-11-26T07:11:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7ff0877fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0880b4090 0x7ff0880714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7ff087fff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08815d6a0 0x7ff0880b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7ff0877fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff0880b4090 0x7ff0880714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:33770/0 (socket says 172.21.15.83:33770) 2023-11-26T07:11:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7ff0877fe700 1 -- 172.21.15.83:0/1894272400 learned_addr learned my addr 172.21.15.83:0/1894272400 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.389+0000 7ff087fff700 1 --2- 172.21.15.83:0/1894272400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08815d6a0 0x7ff0880b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44774/0 (socket says 172.21.15.83:44774) 2023-11-26T07:11:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7ff0877fe700 1 -- 172.21.15.83:0/1894272400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08815d6a0 msgr2=0x7ff0880b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7ff0877fe700 1 --2- 172.21.15.83:0/1894272400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08815d6a0 0x7ff0880b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7ff0877fe700 1 -- 172.21.15.83:0/1894272400 >> v1:172.21.15.83:6789/0 conn(0x7ff088071a10 legacy=0x7ff08806cac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-26T07:11:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7ff0877fe700 1 -- 172.21.15.83:0/1894272400 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0880720e0 con 0x7ff0880b4090 2023-11-26T07:11:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7ff087fff700 1 --2- 172.21.15.83:0/1894272400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08815d6a0 0x7ff0880b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7ff0877fe700 1 --2- 172.21.15.83:0/1894272400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0880b4090 0x7ff0880714d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff078009920 tx=0x7ff078009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64c0e4d854de5e07 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7f41adb1b700 1 -- 172.21.15.83:0/408123067 >> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] conn(0x7f4190074670 msgr2=0x7f4190076a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7f41adb1b700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] conn(0x7f4190074670 0x7f4190076a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7f41adb1b700 1 -- 172.21.15.83:0/408123067 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f419006c450 msgr2=0x7f419006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7f41adb1b700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f419006c450 0x7f419006e910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4198025040 tx=0x7f419801b000 comp rx=0 tx=0).stop 2023-11-26T07:11:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7f41adb1b700 1 -- 172.21.15.83:0/408123067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41a81683a0 msgr2=0x7f41a8152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7f41adb1b700 1 --2- 172.21.15.83:0/408123067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41a81683a0 0x7f41a8152e20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f419c007940 tx=0x7f419c007c50 comp rx=0 tx=0).stop 2023-11-26T07:11:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7f41a7fff700 1 -- 172.21.15.83:0/408123067 reap_dead start 2023-11-26T07:11:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7ff086ffd700 1 -- 172.21.15.83:0/1894272400 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078004230 con 0x7ff0880b4090 2023-11-26T07:11:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7ff086ffd700 1 -- 172.21.15.83:0/1894272400 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff078004390 con 0x7ff0880b4090 2023-11-26T07:11:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.390+0000 7ff086ffd700 1 -- 172.21.15.83:0/1894272400 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078009e70 con 0x7ff0880b4090 2023-11-26T07:11:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7fec07588700 1 -- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] conn(0x7febe00015d0 msgr2=0x7febe0003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7fec07588700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] conn(0x7febe00015d0 0x7febe0003a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1894272400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0880b4090 msgr2=0x7ff0880714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7ff08e2c2700 1 --2- 172.21.15.83:0/1894272400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0880b4090 0x7ff0880714d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff078009920 tx=0x7ff078009c30 comp rx=0 tx=0).stop 2023-11-26T07:11:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1894272400 shutdown_connections 2023-11-26T07:11:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7ff08e2c2700 1 --2- 172.21.15.83:0/1894272400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff08815d6a0 0x7ff0880b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7ff08e2c2700 1 --2- 172.21.15.83:0/1894272400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0880b4090 0x7ff0880714d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1894272400 >> 172.21.15.83:0/1894272400 conn(0x7ff0880cfa30 msgr2=0x7ff0880cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1894272400 shutdown_connections 2023-11-26T07:11:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7fec07588700 1 -- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7febe406aed0 msgr2=0x7febe406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7fec07588700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7febe406aed0 0x7febe406d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febfc0090b0 tx=0x7febfc008030 comp rx=0 tx=0).stop 2023-11-26T07:11:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7fec07588700 1 -- 172.21.15.83:0/3418836513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 msgr2=0x7fec0014f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7fec07588700 1 --2- 172.21.15.83:0/3418836513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec000b08b0 0x7fec0014f6a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7febf000c750 tx=0x7febf000cb10 comp rx=0 tx=0).stop 2023-11-26T07:11:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7fec05b25700 1 -- 172.21.15.83:0/3418836513 reap_dead start 2023-11-26T07:11:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1894272400 wait complete. 2023-11-26T07:11:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7fec07588700 1 -- 172.21.15.83:0/3418836513 shutdown_connections 2023-11-26T07:11:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.391+0000 7fec07588700 1 -- 172.21.15.83:0/3418836513 >> 172.21.15.83:0/3418836513 conn(0x7fec000b57b0 msgr2=0x7fec0006e960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.393+0000 7ff08e2c2700 1 Processor -- start 2023-11-26T07:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.393+0000 7fec07588700 1 -- 172.21.15.83:0/3418836513 shutdown_connections 2023-11-26T07:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.393+0000 7fec07588700 1 -- 172.21.15.83:0/3418836513 wait complete. 2023-11-26T07:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.394+0000 7ff08e2c2700 1 -- start start 2023-11-26T07:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7f41adb1b700 1 -- 172.21.15.83:0/408123067 shutdown_connections 2023-11-26T07:11:34.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7f41adb1b700 1 -- 172.21.15.83:0/408123067 >> 172.21.15.83:0/408123067 conn(0x7f41a80cfa30 msgr2=0x7f41a816a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7f41adb1b700 1 -- 172.21.15.83:0/408123067 shutdown_connections 2023-11-26T07:11:34.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7f41adb1b700 1 -- 172.21.15.83:0/408123067 wait complete. 2023-11-26T07:11:34.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff08e2c2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff088071a10 0x7ff0880aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff08e2c2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0880b4090 0x7ff08813fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff08e2c2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff08815d6a0 0x7ff0881402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff08e2c2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff088156260 con 0x7ff08815d6a0 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff08e2c2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880ad870 con 0x7ff088071a10 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff08e2c2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880ae3f0 con 0x7ff0880b4090 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff0877fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0880b4090 0x7ff08813fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff087fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff088071a10 0x7ff0880aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff0877fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0880b4090 0x7ff08813fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44778/0 (socket says 172.21.15.83:44778) 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff0877fe700 1 -- 172.21.15.83:0/1703124254 learned_addr learned my addr 172.21.15.83:0/1703124254 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff087fff700 1 -- 172.21.15.83:0/1703124254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0880b4090 msgr2=0x7ff08813fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff087fff700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0880b4090 0x7ff08813fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff087fff700 1 -- 172.21.15.83:0/1703124254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff08815d6a0 msgr2=0x7ff0881402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff087fff700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff08815d6a0 0x7ff0881402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff087fff700 1 -- 172.21.15.83:0/1703124254 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff078009550 con 0x7ff088071a10 2023-11-26T07:11:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.395+0000 7ff0877fe700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0880b4090 0x7ff08813fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.396+0000 7ff087fff700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff088071a10 0x7ff0880aebd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff08000d910 tx=0x7ff08000dc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.396+0000 7ff084ff9700 1 -- 172.21.15.83:0/1703124254 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08000ba10 con 0x7ff088071a10 2023-11-26T07:11:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.396+0000 7ff084ff9700 1 -- 172.21.15.83:0/1703124254 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08000bb70 con 0x7ff088071a10 2023-11-26T07:11:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.396+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1703124254 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0880adac0 con 0x7ff088071a10 2023-11-26T07:11:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.396+0000 7ff084ff9700 1 -- 172.21.15.83:0/1703124254 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08000de60 con 0x7ff088071a10 2023-11-26T07:11:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.396+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1703124254 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0880ae080 con 0x7ff088071a10 2023-11-26T07:11:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.397+0000 7ffbc6d96700 1 -- 172.21.15.83:0/3942368641 --> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffba0002d30 con 0x7ffbac073140 2023-11-26T07:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.397+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1703124254 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff068000fb0 con 0x7ff088071a10 2023-11-26T07:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.397+0000 7ff084ff9700 1 -- 172.21.15.83:0/1703124254 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff080012410 con 0x7ff088071a10 2023-11-26T07:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.398+0000 7fd2957e1700 1 -- 172.21.15.83:0/3253792374 --> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd270002d30 con 0x7fd27c073170 2023-11-26T07:11:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.398+0000 7ff084ff9700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff07406aed0 0x7ff07406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.398+0000 7ff084ff9700 1 -- 172.21.15.83:0/1703124254 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff080089b50 con 0x7ff088071a10 2023-11-26T07:11:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.398+0000 7ff0877fe700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff07406aed0 0x7ff07406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.398+0000 7ff084ff9700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] conn(0x7ff074073070 0x7ff074075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.398+0000 7ff084ff9700 1 -- 172.21.15.83:0/1703124254 --> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff074075b40 con 0x7ff074073070 2023-11-26T07:11:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.398+0000 7ff084ff9700 1 -- 172.21.15.83:0/1703124254 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff08008e2c0 con 0x7ff088071a10 2023-11-26T07:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.398+0000 7fd2777fe700 1 -- 172.21.15.83:0/3253792374 <== osd.0 v2:172.21.15.83:6800/3940814281 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd270002d30 con 0x7fd27c073170 2023-11-26T07:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.399+0000 7ff08c85f700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] conn(0x7ff074073070 0x7ff074075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.399+0000 7ff0877fe700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff07406aed0 0x7ff07406d390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff08806f540 tx=0x7ff078008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.399+0000 7fd2957e1700 1 -- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] conn(0x7fd27c073170 msgr2=0x7fd27c075590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.399+0000 7fd2957e1700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] conn(0x7fd27c073170 0x7fd27c075590 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.399+0000 7ff08c85f700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] conn(0x7ff074073070 0x7ff074075490 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.400+0000 7ff084ff9700 1 -- 172.21.15.83:0/1703124254 <== osd.7 v2:172.21.15.203:6816/2979691223 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff074075b40 con 0x7ff074073070 2023-11-26T07:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.400+0000 7ffbbd7fa700 1 -- 172.21.15.83:0/3942368641 <== osd.3 v2:172.21.15.83:6816/1108190663 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffba0002d30 con 0x7ffbac073140 2023-11-26T07:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.401+0000 7fe485753700 1 -- 172.21.15.83:0/346110885 --> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe4800b3b80 con 0x7fe46c073140 2023-11-26T07:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.402+0000 7fd2957e1700 1 -- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd27c06af50 msgr2=0x7fd27c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.402+0000 7fd2957e1700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd27c06af50 0x7fd27c06d410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd284001720 tx=0x7fd284009040 comp rx=0 tx=0).stop 2023-11-26T07:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.402+0000 7fd2957e1700 1 -- 172.21.15.83:0/3253792374 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd29015b9b0 msgr2=0x7fd29015be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.402+0000 7fd2957e1700 1 --2- 172.21.15.83:0/3253792374 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd29015b9b0 0x7fd29015be60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd29015e290 tx=0x7fd28800a510 comp rx=0 tx=0).stop 2023-11-26T07:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.402+0000 7fd28f7fe700 1 -- 172.21.15.83:0/3253792374 reap_dead start 2023-11-26T07:11:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.402+0000 7fd2957e1700 1 -- 172.21.15.83:0/3253792374 shutdown_connections 2023-11-26T07:11:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.402+0000 7fd2957e1700 1 -- 172.21.15.83:0/3253792374 >> 172.21.15.83:0/3253792374 conn(0x7fd2900b1740 msgr2=0x7fd290159390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.415 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-26T07:11:34.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-26T07:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.404+0000 7fd2957e1700 1 -- 172.21.15.83:0/3253792374 shutdown_connections 2023-11-26T07:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.404+0000 7fd2957e1700 1 -- 172.21.15.83:0/3253792374 wait complete. 2023-11-26T07:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.405+0000 7fe4677fe700 1 -- 172.21.15.83:0/346110885 <== osd.5 v2:172.21.15.203:6800/2181588544 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe474009880 con 0x7fe46c073140 2023-11-26T07:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.405+0000 7fe485753700 1 -- 172.21.15.83:0/346110885 >> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] conn(0x7fe46c073140 msgr2=0x7fe46c075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.405+0000 7fe485753700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] conn(0x7fe46c073140 0x7fe46c075560 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.406+0000 7fe485753700 1 -- 172.21.15.83:0/346110885 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe46c06af20 msgr2=0x7fe46c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.406+0000 7fe485753700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe46c06af20 0x7fe46c06d3e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe4700017f0 tx=0x7fe470009040 comp rx=0 tx=0).stop 2023-11-26T07:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.406+0000 7fe485753700 1 -- 172.21.15.83:0/346110885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48015b140 msgr2=0x7fe48014aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.406+0000 7fe485753700 1 --2- 172.21.15.83:0/346110885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48015b140 0x7fe48014aff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe4800b5c80 tx=0x7fe478004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.406+0000 7fe47f7fe700 1 -- 172.21.15.83:0/346110885 reap_dead start 2023-11-26T07:11:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.406+0000 7fe485753700 1 -- 172.21.15.83:0/346110885 shutdown_connections 2023-11-26T07:11:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.406+0000 7fe485753700 1 -- 172.21.15.83:0/346110885 >> 172.21.15.83:0/346110885 conn(0x7fe4800b1740 msgr2=0x7fe4800b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.406+0000 7fe485753700 1 -- 172.21.15.83:0/346110885 shutdown_connections 2023-11-26T07:11:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.406+0000 7fe485753700 1 -- 172.21.15.83:0/346110885 wait complete. 2023-11-26T07:11:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.408+0000 7ffbc6d96700 1 -- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] conn(0x7ffbac073140 msgr2=0x7ffbac075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.408+0000 7ffbc6d96700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] conn(0x7ffbac073140 0x7ffbac075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.410+0000 7ffbc6d96700 1 -- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffbac06af20 msgr2=0x7ffbac06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.410+0000 7ffbc6d96700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffbac06af20 0x7ffbac06d3e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4004930 tx=0x7ffbb4004dd0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.410+0000 7ffbc6d96700 1 -- 172.21.15.83:0/3942368641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbc00b4260 msgr2=0x7ffbc0156970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.410+0000 7ffbc6d96700 1 --2- 172.21.15.83:0/3942368641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbc00b4260 0x7ffbc0156970 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000ec10 tx=0x7ffbb0009ec0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.410+0000 7ffbc5333700 1 -- 172.21.15.83:0/3942368641 reap_dead start 2023-11-26T07:11:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.410+0000 7ffbc6d96700 1 -- 172.21.15.83:0/3942368641 shutdown_connections 2023-11-26T07:11:34.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.410+0000 7ffbc6d96700 1 -- 172.21.15.83:0/3942368641 >> 172.21.15.83:0/3942368641 conn(0x7ffbc00cfa30 msgr2=0x7ffbc015c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.420 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-26T07:11:34.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-26T07:11:34.421 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-26T07:11:34.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-26T07:11:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.415+0000 7ffbc6d96700 1 -- 172.21.15.83:0/3942368641 shutdown_connections 2023-11-26T07:11:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.415+0000 7ffbc6d96700 1 -- 172.21.15.83:0/3942368641 wait complete. 2023-11-26T07:11:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.421+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1703124254 --> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff068002d30 con 0x7ff074073070 2023-11-26T07:11:34.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.421+0000 7ff084ff9700 1 -- 172.21.15.83:0/1703124254 <== osd.7 v2:172.21.15.203:6816/2979691223 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff068002d30 con 0x7ff074073070 2023-11-26T07:11:34.426 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-26T07:11:34.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.422+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1703124254 >> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] conn(0x7ff074073070 msgr2=0x7ff074075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.422+0000 7ff08e2c2700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] conn(0x7ff074073070 0x7ff074075490 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.422+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1703124254 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff07406aed0 msgr2=0x7ff07406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.422+0000 7ff08e2c2700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff07406aed0 0x7ff07406d390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff08806f540 tx=0x7ff078008030 comp rx=0 tx=0).stop 2023-11-26T07:11:34.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.422+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1703124254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff088071a10 msgr2=0x7ff0880aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.422+0000 7ff08e2c2700 1 --2- 172.21.15.83:0/1703124254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff088071a10 0x7ff0880aebd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff08000d910 tx=0x7ff08000dc20 comp rx=0 tx=0).stop 2023-11-26T07:11:34.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.423+0000 7ff08c85f700 1 -- 172.21.15.83:0/1703124254 reap_dead start 2023-11-26T07:11:34.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-26T07:11:34.428 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-26T07:11:34.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-26T07:11:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.424+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1703124254 shutdown_connections 2023-11-26T07:11:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.424+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1703124254 >> 172.21.15.83:0/1703124254 conn(0x7ff0880cfa30 msgr2=0x7ff0880b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.427+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1703124254 shutdown_connections 2023-11-26T07:11:34.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.427+0000 7ff08e2c2700 1 -- 172.21.15.83:0/1703124254 wait complete. 2023-11-26T07:11:34.436 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-26T07:11:34.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-26T07:11:34.451 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-26T07:11:34.451 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-26T07:11:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.599+0000 7f8835ec1700 1 Processor -- start 2023-11-26T07:11:34.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.599+0000 7f8835ec1700 1 -- start start 2023-11-26T07:11:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.601+0000 7f8835ec1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88300b3b50 0x7f88300b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.601+0000 7f8835ec1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f883006d4d0 0x7f883006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.601+0000 7f8835ec1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8830163a80 0x7f8830163f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.601+0000 7f8835ec1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883006ced0 con 0x7f8830163a80 2023-11-26T07:11:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.601+0000 7f8835ec1700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883006da70 con 0x7f883006d4d0 2023-11-26T07:11:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.601+0000 7f8835ec1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88300b4470 con 0x7f88300b3b50 2023-11-26T07:11:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.601+0000 7f882ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8830163a80 0x7f8830163f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.601+0000 7f882ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8830163a80 0x7f8830163f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34332/0 (socket says 172.21.15.83:34332) 2023-11-26T07:11:34.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.601+0000 7f882ffff700 1 -- 172.21.15.83:0/3565761723 learned_addr learned my addr 172.21.15.83:0/3565761723 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.602+0000 7f585304b700 1 Processor -- start 2023-11-26T07:11:34.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.602+0000 7f882ffff700 1 -- 172.21.15.83:0/3565761723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88300b3b50 msgr2=0x7f88300b3f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:11:34.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.602+0000 7f882ffff700 1 --2- 172.21.15.83:0/3565761723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88300b3b50 0x7f88300b3f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.602+0000 7f882ffff700 1 -- 172.21.15.83:0/3565761723 >> v2:172.21.15.203:3300/0 conn(0x7f883006d4d0 msgr2=0x7f883006eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:11:34.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.602+0000 7f882ffff700 1 --2- 172.21.15.83:0/3565761723 >> v2:172.21.15.203:3300/0 conn(0x7f883006d4d0 0x7f883006eee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.602+0000 7f882ffff700 1 -- 172.21.15.83:0/3565761723 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88301634e0 con 0x7f8830163a80 2023-11-26T07:11:34.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.602+0000 7f882ffff700 1 --2- 172.21.15.83:0/3565761723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8830163a80 0x7f8830163f10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f883006c450 tx=0x7f882800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5bc8c98a498243 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.603+0000 7f882e7fc700 1 -- 172.21.15.83:0/3565761723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800e070 con 0x7f8830163a80 2023-11-26T07:11:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.602+0000 7f585304b700 1 -- start start 2023-11-26T07:11:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.603+0000 7f585304b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f584c15d6a0 0x7f584c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.603+0000 7f585304b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584c071be0 0x7f584c140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.603+0000 7f585304b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584c0720e0 con 0x7f584c0b4090 2023-11-26T07:11:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.603+0000 7f585304b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584c072d60 con 0x7f584c15d6a0 2023-11-26T07:11:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.603+0000 7f585304b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c06de80 con 0x7f584c071be0 2023-11-26T07:11:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.604+0000 7f58515e8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584c071be0 0x7f584c140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.604+0000 7f5850de7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f584c15d6a0 0x7f584c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.604+0000 7f58515e8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584c071be0 0x7f584c140fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44796/0 (socket says 172.21.15.83:44796) 2023-11-26T07:11:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.604+0000 7f58515e8700 1 -- 172.21.15.83:0/546876080 learned_addr learned my addr 172.21.15.83:0/546876080 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.604+0000 7f58515e8700 1 -- 172.21.15.83:0/546876080 >> v2:172.21.15.203:3300/0 conn(0x7f584c15d6a0 msgr2=0x7f584c0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.604+0000 7f58515e8700 1 --2- 172.21.15.83:0/546876080 >> v2:172.21.15.203:3300/0 conn(0x7f584c15d6a0 0x7f584c0b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.604+0000 7f58515e8700 1 -- 172.21.15.83:0/546876080 >> v1:172.21.15.203:6789/0 conn(0x7f584c0b4090 legacy=0x7f584c0714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.604+0000 7f58515e8700 1 -- 172.21.15.83:0/546876080 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c0723c0 con 0x7f584c071be0 2023-11-26T07:11:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.604+0000 7f5850de7700 1 --2- 172.21.15.83:0/546876080 >> v2:172.21.15.203:3300/0 conn(0x7f584c15d6a0 0x7f584c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.604+0000 7f58515e8700 1 --2- 172.21.15.83:0/546876080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584c071be0 0x7f584c140fd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f583c00b9a0 tx=0x7f583c00bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef4ac2da6d6b74c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.604+0000 7f584b7fe700 1 -- 172.21.15.83:0/546876080 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c00cb60 con 0x7f584c071be0 2023-11-26T07:11:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.605+0000 7f584b7fe700 1 -- 172.21.15.83:0/546876080 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c00ccc0 con 0x7f584c071be0 2023-11-26T07:11:34.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.605+0000 7f584b7fe700 1 -- 172.21.15.83:0/546876080 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c004270 con 0x7f584c071be0 2023-11-26T07:11:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.606+0000 7f585304b700 1 -- 172.21.15.83:0/546876080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584c071be0 msgr2=0x7f584c140fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.606+0000 7f585304b700 1 --2- 172.21.15.83:0/546876080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584c071be0 0x7f584c140fd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f583c00b9a0 tx=0x7f583c00bcb0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.606+0000 7f585304b700 1 -- 172.21.15.83:0/546876080 shutdown_connections 2023-11-26T07:11:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.606+0000 7f585304b700 1 --2- 172.21.15.83:0/546876080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584c071be0 0x7f584c140fd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.606+0000 7f585304b700 1 --2- 172.21.15.83:0/546876080 >> v2:172.21.15.203:3300/0 conn(0x7f584c15d6a0 0x7f584c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.606+0000 7f585304b700 1 -- 172.21.15.83:0/546876080 >> 172.21.15.83:0/546876080 conn(0x7f584c0cfa30 msgr2=0x7f584c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.606+0000 7f585304b700 1 -- 172.21.15.83:0/546876080 shutdown_connections 2023-11-26T07:11:34.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.606+0000 7f585304b700 1 -- 172.21.15.83:0/546876080 wait complete. 2023-11-26T07:11:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.606+0000 7f882e7fc700 1 -- 172.21.15.83:0/3565761723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882800b9c0 con 0x7f8830163a80 2023-11-26T07:11:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.606+0000 7f882e7fc700 1 -- 172.21.15.83:0/3565761723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800bd70 con 0x7f8830163a80 2023-11-26T07:11:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.607+0000 7f8835ec1700 1 -- 172.21.15.83:0/3565761723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8830163a80 msgr2=0x7f8830163f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.607+0000 7f8835ec1700 1 --2- 172.21.15.83:0/3565761723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8830163a80 0x7f8830163f10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f883006c450 tx=0x7f882800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:34.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.608+0000 7f8835ec1700 1 -- 172.21.15.83:0/3565761723 shutdown_connections 2023-11-26T07:11:34.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.608+0000 7f8835ec1700 1 --2- 172.21.15.83:0/3565761723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8830163a80 0x7f8830163f10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.608+0000 7f8835ec1700 1 --2- 172.21.15.83:0/3565761723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88300b3b50 0x7f88300b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.608+0000 7f8835ec1700 1 --2- 172.21.15.83:0/3565761723 >> v2:172.21.15.203:3300/0 conn(0x7f883006d4d0 0x7f883006eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.608+0000 7f8835ec1700 1 -- 172.21.15.83:0/3565761723 >> 172.21.15.83:0/3565761723 conn(0x7f88300cfa30 msgr2=0x7f88300cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.608+0000 7f8835ec1700 1 -- 172.21.15.83:0/3565761723 shutdown_connections 2023-11-26T07:11:34.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.608+0000 7f8835ec1700 1 -- 172.21.15.83:0/3565761723 wait complete. 2023-11-26T07:11:34.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.610+0000 7f8835ec1700 1 Processor -- start 2023-11-26T07:11:34.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.610+0000 7f8835ec1700 1 -- start start 2023-11-26T07:11:34.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.611+0000 7f8835ec1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f883006d4d0 0x7f88300cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.611+0000 7f8835ec1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88300b3b50 0x7f8830157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.611+0000 7f8835ec1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8830163a80 0x7f8830152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.611+0000 7f8835ec1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88300cd3a0 con 0x7f883006d4d0 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.611+0000 7f8835ec1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8830156cb0 con 0x7f88300b3b50 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.611+0000 7f8835ec1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8830153150 con 0x7f8830163a80 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.612+0000 7f882f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f883006d4d0 0x7f88300cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.612+0000 7f882f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f883006d4d0 0x7f88300cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34346/0 (socket says 172.21.15.83:34346) 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.612+0000 7f882f7fe700 1 -- 172.21.15.83:0/1193379285 learned_addr learned my addr 172.21.15.83:0/1193379285 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.612+0000 7f882effd700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88300b3b50 0x7f8830157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.612+0000 7f882f7fe700 1 -- 172.21.15.83:0/1193379285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8830163a80 msgr2=0x7f8830152090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.612+0000 7f882f7fe700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8830163a80 0x7f8830152090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.612+0000 7f882f7fe700 1 -- 172.21.15.83:0/1193379285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88300b3b50 msgr2=0x7f8830157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.612+0000 7f882f7fe700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88300b3b50 0x7f8830157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.612+0000 7f882f7fe700 1 -- 172.21.15.83:0/1193379285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8828009b30 con 0x7f883006d4d0 2023-11-26T07:11:34.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.612+0000 7f882effd700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88300b3b50 0x7f8830157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:34.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.612+0000 7f882f7fe700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f883006d4d0 0x7f88300cce60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f882000b470 tx=0x7f882000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.613+0000 7f8817fff700 1 -- 172.21.15.83:0/1193379285 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820011830 con 0x7f883006d4d0 2023-11-26T07:11:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.613+0000 7f8817fff700 1 -- 172.21.15.83:0/1193379285 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8820011990 con 0x7f883006d4d0 2023-11-26T07:11:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.613+0000 7f8817fff700 1 -- 172.21.15.83:0/1193379285 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820011d40 con 0x7f883006d4d0 2023-11-26T07:11:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.613+0000 7f8835ec1700 1 -- 172.21.15.83:0/1193379285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8830157c40 con 0x7f883006d4d0 2023-11-26T07:11:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.613+0000 7f8835ec1700 1 -- 172.21.15.83:0/1193379285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88300cd860 con 0x7f883006d4d0 2023-11-26T07:11:34.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.615+0000 7f8835ec1700 1 -- 172.21.15.83:0/1193379285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88100052e0 con 0x7f883006d4d0 2023-11-26T07:11:34.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.617+0000 7f8817fff700 1 -- 172.21.15.83:0/1193379285 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f882001a400 con 0x7f883006d4d0 2023-11-26T07:11:34.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.617+0000 7f8817fff700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f881806b0b0 0x7f881806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.618+0000 7f882effd700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f881806b0b0 0x7f881806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.619+0000 7f882effd700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f881806b0b0 0x7f881806d570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8830162340 tx=0x7f8824008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.620+0000 7f8817fff700 1 -- 172.21.15.83:0/1193379285 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8820059060 con 0x7f883006d4d0 2023-11-26T07:11:34.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.608+0000 7f585304b700 1 Processor -- start 2023-11-26T07:11:34.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.608+0000 7f585304b700 1 -- start start 2023-11-26T07:11:34.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.609+0000 7f585304b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f584c071be0 0x7f584c1f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.609+0000 7f585304b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584c0b4090 0x7f584c156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.609+0000 7f585304b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f584c15d6a0 0x7f584c1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.609+0000 7f585304b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c1f8cf0 con 0x7f584c071be0 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.609+0000 7f585304b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c1f7f10 con 0x7f584c15d6a0 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.609+0000 7f585304b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c159130 con 0x7f584c0b4090 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.609+0000 7f5850de7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f584c071be0 0x7f584c1f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.609+0000 7f5850de7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f584c071be0 0x7f584c1f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34342/0 (socket says 172.21.15.83:34342) 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.609+0000 7f5850de7700 1 -- 172.21.15.83:0/3525863177 learned_addr learned my addr 172.21.15.83:0/3525863177 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.610+0000 7f5850de7700 1 -- 172.21.15.83:0/3525863177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584c0b4090 msgr2=0x7f584c156480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.610+0000 7f5850de7700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584c0b4090 0x7f584c156480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.610+0000 7f5850de7700 1 -- 172.21.15.83:0/3525863177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f584c15d6a0 msgr2=0x7f584c1569c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.610+0000 7f5850de7700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f584c15d6a0 0x7f584c1569c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.610+0000 7f5850de7700 1 -- 172.21.15.83:0/3525863177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5844005e80 con 0x7f584c071be0 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.610+0000 7f5850de7700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f584c071be0 0x7f584c1f4f30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f584400cec0 tx=0x7f58440100b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.611+0000 7f58497fa700 1 -- 172.21.15.83:0/3525863177 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844003c20 con 0x7f584c071be0 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.611+0000 7f58497fa700 1 -- 172.21.15.83:0/3525863177 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5844003d80 con 0x7f584c071be0 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.611+0000 7f58497fa700 1 -- 172.21.15.83:0/3525863177 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58440109a0 con 0x7f584c071be0 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.613+0000 7f585304b700 1 -- 172.21.15.83:0/3525863177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583c00b600 con 0x7f584c071be0 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.613+0000 7f585304b700 1 -- 172.21.15.83:0/3525863177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584c1596f0 con 0x7f584c071be0 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.615+0000 7f58497fa700 1 -- 172.21.15.83:0/3525863177 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5844015070 con 0x7f584c071be0 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.616+0000 7f58497fa700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f583808cf30 0x7f583808f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.616+0000 7f58497fa700 1 -- 172.21.15.83:0/3525863177 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5844088ec0 con 0x7f584c071be0 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.616+0000 7f584bfff700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f583808cf30 0x7f583808f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.618+0000 7f585304b700 1 -- 172.21.15.83:0/3525863177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584c0751e0 con 0x7f584c071be0 2023-11-26T07:11:34.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.619+0000 7f584bfff700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f583808cf30 0x7f583808f3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f584c0af500 tx=0x7f5840007a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.625+0000 7f8817fff700 1 -- 172.21.15.83:0/1193379285 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8820054ca0 con 0x7f883006d4d0 2023-11-26T07:11:34.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.627+0000 7f58497fa700 1 -- 172.21.15.83:0/3525863177 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5844053fd0 con 0x7f584c071be0 2023-11-26T07:11:34.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.629+0000 7fd1856b0700 1 Processor -- start 2023-11-26T07:11:34.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.629+0000 7fd1856b0700 1 -- start start 2023-11-26T07:11:34.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.630+0000 7fd1856b0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd180164a90 0x7fd18006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.630+0000 7fd1856b0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd18006d4d0 0x7fd18016bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.630+0000 7fd1856b0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18006ced0 con 0x7fd1800b3b50 2023-11-26T07:11:34.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.630+0000 7fd1856b0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18006da70 con 0x7fd180164a90 2023-11-26T07:11:34.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.630+0000 7fd1856b0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801634e0 con 0x7fd18006d4d0 2023-11-26T07:11:34.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.631+0000 7fd17effd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd1800b3b50 0x7fd1800b3f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:39012/0 (socket says 172.21.15.83:39012) 2023-11-26T07:11:34.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.631+0000 7fd17effd700 1 -- 172.21.15.83:0/2495740577 learned_addr learned my addr 172.21.15.83:0/2495740577 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:34.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.631+0000 7fd17e7fc700 1 --2- 172.21.15.83:0/2495740577 >> v2:172.21.15.83:3300/0 conn(0x7fd180164a90 0x7fd18006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.631+0000 7fd17dffb700 1 -- 172.21.15.83:0/2495740577 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839312600 0 0) 0x7fd18006ced0 con 0x7fd1800b3b50 2023-11-26T07:11:34.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.631+0000 7fd17dffb700 1 -- 172.21.15.83:0/2495740577 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1680035b0 con 0x7fd1800b3b50 2023-11-26T07:11:34.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.631+0000 7fd17e7fc700 1 -- 172.21.15.83:0/2495740577 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd18006d4d0 msgr2=0x7fd18016bee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:11:34.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.631+0000 7fd17e7fc700 1 --2- 172.21.15.83:0/2495740577 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd18006d4d0 0x7fd18016bee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.631+0000 7fd17e7fc700 1 -- 172.21.15.83:0/2495740577 >> v1:172.21.15.203:6789/0 conn(0x7fd1800b3b50 legacy=0x7fd1800b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.631+0000 7fd17e7fc700 1 -- 172.21.15.83:0/2495740577 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1801637c0 con 0x7fd180164a90 2023-11-26T07:11:34.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.632+0000 7fd17e7fc700 1 --2- 172.21.15.83:0/2495740577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd180164a90 0x7fd18006eee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd174009840 tx=0x7fd174009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da096bddd90859e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.632+0000 7fd17dffb700 1 -- 172.21.15.83:0/2495740577 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174011830 con 0x7fd180164a90 2023-11-26T07:11:34.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.632+0000 7fd17dffb700 1 -- 172.21.15.83:0/2495740577 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd174011990 con 0x7fd180164a90 2023-11-26T07:11:34.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.632+0000 7fd17dffb700 1 -- 172.21.15.83:0/2495740577 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174011d20 con 0x7fd180164a90 2023-11-26T07:11:34.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.633+0000 7fd1856b0700 1 -- 172.21.15.83:0/2495740577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd180164a90 msgr2=0x7fd18006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.633+0000 7fd1856b0700 1 --2- 172.21.15.83:0/2495740577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd180164a90 0x7fd18006eee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd174009840 tx=0x7fd174009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:34.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.634+0000 7fd1856b0700 1 -- 172.21.15.83:0/2495740577 shutdown_connections 2023-11-26T07:11:34.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.634+0000 7fd1856b0700 1 --2- 172.21.15.83:0/2495740577 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd18006d4d0 0x7fd18016bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.634+0000 7fd1856b0700 1 --2- 172.21.15.83:0/2495740577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd180164a90 0x7fd18006eee0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.634+0000 7fd1856b0700 1 -- 172.21.15.83:0/2495740577 >> 172.21.15.83:0/2495740577 conn(0x7fd1800cfa30 msgr2=0x7fd1800cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.634+0000 7fd1856b0700 1 -- 172.21.15.83:0/2495740577 shutdown_connections 2023-11-26T07:11:34.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.634+0000 7fd1856b0700 1 -- 172.21.15.83:0/2495740577 wait complete. 2023-11-26T07:11:34.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.636+0000 7fd1856b0700 1 Processor -- start 2023-11-26T07:11:34.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.636+0000 7fd1856b0700 1 -- start start 2023-11-26T07:11:34.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.637+0000 7fd1856b0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd18006d4d0 0x7fd1800b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.637+0000 7fd1856b0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1800b3b50 0x7fd180140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.637+0000 7fd1856b0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180164a90 0x7fd1801411f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.637+0000 7fd1856b0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180141a10 con 0x7fd1800b3b50 2023-11-26T07:11:34.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.637+0000 7fd1856b0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180071520 con 0x7fd180164a90 2023-11-26T07:11:34.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.637+0000 7fd1856b0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd180070bf0 con 0x7fd18006d4d0 2023-11-26T07:11:34.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.638+0000 7fd17e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1800b3b50 0x7fd180140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.638+0000 7fd17e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1800b3b50 0x7fd180140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34364/0 (socket says 172.21.15.83:34364) 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.638+0000 7fd17e7fc700 1 -- 172.21.15.83:0/999059439 learned_addr learned my addr 172.21.15.83:0/999059439 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.638+0000 7fd17f7fe700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180164a90 0x7fd1801411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.638+0000 7fd17e7fc700 1 -- 172.21.15.83:0/999059439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd18006d4d0 msgr2=0x7fd1800b0570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.638+0000 7fd17e7fc700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd18006d4d0 0x7fd1800b0570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.638+0000 7fd17e7fc700 1 -- 172.21.15.83:0/999059439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180164a90 msgr2=0x7fd1801411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.638+0000 7fd17e7fc700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180164a90 0x7fd1801411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.638+0000 7fd17e7fc700 1 -- 172.21.15.83:0/999059439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1740094a0 con 0x7fd1800b3b50 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.638+0000 7fd17e7fc700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1800b3b50 0x7fd180140cb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd1740090b0 tx=0x7fd174004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.639+0000 7fd16f7fe700 1 -- 172.21.15.83:0/999059439 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1740036a0 con 0x7fd1800b3b50 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.639+0000 7fd1856b0700 1 -- 172.21.15.83:0/999059439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd180070ed0 con 0x7fd1800b3b50 2023-11-26T07:11:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.639+0000 7fd16f7fe700 1 -- 172.21.15.83:0/999059439 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17401f070 con 0x7fd1800b3b50 2023-11-26T07:11:34.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.639+0000 7fd16f7fe700 1 -- 172.21.15.83:0/999059439 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174003a00 con 0x7fd1800b3b50 2023-11-26T07:11:34.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.639+0000 7fd1856b0700 1 -- 172.21.15.83:0/999059439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd180141730 con 0x7fd1800b3b50 2023-11-26T07:11:34.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.641+0000 7fd16f7fe700 1 -- 172.21.15.83:0/999059439 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd174003b60 con 0x7fd1800b3b50 2023-11-26T07:11:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.642+0000 7fd16f7fe700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd16406aed0 0x7fd16406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.642+0000 7fd16f7fe700 1 -- 172.21.15.83:0/999059439 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd174021070 con 0x7fd1800b3b50 2023-11-26T07:11:34.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.645+0000 7fd17effd700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd16406aed0 0x7fd16406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.645+0000 7fd17effd700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd16406aed0 0x7fd16406d390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1800cf770 tx=0x7fd178001650 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.646+0000 7fde72b9d700 1 Processor -- start 2023-11-26T07:11:34.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.646+0000 7fcf59d5d700 1 Processor -- start 2023-11-26T07:11:34.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.646+0000 7fde72b9d700 1 -- start start 2023-11-26T07:11:34.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.647+0000 7fde72b9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fde6c0d1570 0x7fde6c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.647+0000 7fde72b9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 0x7fde6c0d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.646+0000 7fcf59d5d700 1 -- start start 2023-11-26T07:11:34.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.647+0000 7fcf59d5d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0ba0 0x7fcf540af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.647+0000 7fcf59d5d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcf5415ee90 0x7fcf5415b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.647+0000 7fcf59d5d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf540b2bd0 con 0x7fcf540b0f80 2023-11-26T07:11:34.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.647+0000 7fcf59d5d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf540b1fc0 con 0x7fcf5415ee90 2023-11-26T07:11:34.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.647+0000 7fcf59d5d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540b22a0 con 0x7fcf540b0ba0 2023-11-26T07:11:34.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.647+0000 7fde72b9d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6c0d2d10 con 0x7fde6c15dfc0 2023-11-26T07:11:34.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.647+0000 7fde72b9d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6c0cefd0 con 0x7fde6c0d1570 2023-11-26T07:11:34.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.647+0000 7fde72b9d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c0cf2b0 con 0x7fde6c0b2fc0 2023-11-26T07:11:34.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0ba0 0x7fcf540af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0ba0 0x7fcf540af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44826/0 (socket says 172.21.15.83:44826) 2023-11-26T07:11:34.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fcf537fe700 1 -- 172.21.15.83:0/3289351797 learned_addr learned my addr 172.21.15.83:0/3289351797 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fde7113a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 0x7fde6c0d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fde7113a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 0x7fde6c0d3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44832/0 (socket says 172.21.15.83:44832) 2023-11-26T07:11:34.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fcf53fff700 1 --2- 172.21.15.83:0/3289351797 >> v2:172.21.15.83:3300/0 conn(0x7fcf5415ee90 0x7fcf5415b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fde7113a700 1 -- 172.21.15.83:0/2371272368 learned_addr learned my addr 172.21.15.83:0/2371272368 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fde6bfff700 1 --2- 172.21.15.83:0/2371272368 >> v2:172.21.15.203:3300/0 conn(0x7fde6c0d1570 0x7fde6c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fde7113a700 1 -- 172.21.15.83:0/2371272368 >> v2:172.21.15.203:3300/0 conn(0x7fde6c0d1570 msgr2=0x7fde6c0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fde7113a700 1 --2- 172.21.15.83:0/2371272368 >> v2:172.21.15.203:3300/0 conn(0x7fde6c0d1570 0x7fde6c0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fde7113a700 1 -- 172.21.15.83:0/2371272368 >> v1:172.21.15.203:6789/0 conn(0x7fde6c15dfc0 legacy=0x7fde6c0d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fde7113a700 1 -- 172.21.15.83:0/2371272368 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c0d2fb0 con 0x7fde6c0b2fc0 2023-11-26T07:11:34.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fcf53fff700 1 -- 172.21.15.83:0/3289351797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0ba0 msgr2=0x7fcf540af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fcf53fff700 1 --2- 172.21.15.83:0/3289351797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0ba0 0x7fcf540af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fcf53fff700 1 -- 172.21.15.83:0/3289351797 >> v1:172.21.15.83:6789/0 conn(0x7fcf540b0f80 legacy=0x7fcf540cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fcf53fff700 1 -- 172.21.15.83:0/3289351797 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf540b2580 con 0x7fcf5415ee90 2023-11-26T07:11:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fde7113a700 1 --2- 172.21.15.83:0/2371272368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 0x7fde6c0d3a10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde64009e50 tx=0x7fde6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30ad17c52533632c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fcf537fe700 1 --2- 172.21.15.83:0/3289351797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0ba0 0x7fcf540af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.648+0000 7fde6b7fe700 1 -- 172.21.15.83:0/2371272368 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6400e070 con 0x7fde6c0b2fc0 2023-11-26T07:11:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.649+0000 7fcf53fff700 1 --2- 172.21.15.83:0/3289351797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf5415ee90 0x7fcf5415b450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009840 tx=0x7fcf44009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a231f87c74a743f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.649+0000 7fde6b7fe700 1 -- 172.21.15.83:0/2371272368 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde64003a20 con 0x7fde6c0b2fc0 2023-11-26T07:11:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.649+0000 7fde6b7fe700 1 -- 172.21.15.83:0/2371272368 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64003db0 con 0x7fde6c0b2fc0 2023-11-26T07:11:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.649+0000 7fde72b9d700 1 -- 172.21.15.83:0/2371272368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 msgr2=0x7fde6c0d3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.650+0000 7fde72b9d700 1 --2- 172.21.15.83:0/2371272368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 0x7fde6c0d3a10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde64009e50 tx=0x7fde6400b330 comp rx=0 tx=0).stop 2023-11-26T07:11:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.650+0000 7fcf527fc700 1 -- 172.21.15.83:0/3289351797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44004230 con 0x7fcf5415ee90 2023-11-26T07:11:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.650+0000 7fcf527fc700 1 -- 172.21.15.83:0/3289351797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf44004390 con 0x7fcf5415ee90 2023-11-26T07:11:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.650+0000 7fcf527fc700 1 -- 172.21.15.83:0/3289351797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4400f740 con 0x7fcf5415ee90 2023-11-26T07:11:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.650+0000 7fd1856b0700 1 -- 172.21.15.83:0/999059439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1800751e0 con 0x7fd1800b3b50 2023-11-26T07:11:34.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.651+0000 7fcf59d5d700 1 -- 172.21.15.83:0/3289351797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf5415ee90 msgr2=0x7fcf5415b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.651+0000 7fcf59d5d700 1 --2- 172.21.15.83:0/3289351797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf5415ee90 0x7fcf5415b450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009840 tx=0x7fcf44009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:34.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.652+0000 7fde72b9d700 1 -- 172.21.15.83:0/2371272368 shutdown_connections 2023-11-26T07:11:34.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.652+0000 7fde72b9d700 1 --2- 172.21.15.83:0/2371272368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 0x7fde6c0d3a10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.652+0000 7fde72b9d700 1 --2- 172.21.15.83:0/2371272368 >> v2:172.21.15.203:3300/0 conn(0x7fde6c0d1570 0x7fde6c0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.652+0000 7fde72b9d700 1 -- 172.21.15.83:0/2371272368 >> 172.21.15.83:0/2371272368 conn(0x7fde6c06b410 msgr2=0x7fde6c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.652+0000 7fcf59d5d700 1 -- 172.21.15.83:0/3289351797 shutdown_connections 2023-11-26T07:11:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.652+0000 7fcf59d5d700 1 --2- 172.21.15.83:0/3289351797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf5415ee90 0x7fcf5415b450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.652+0000 7fcf59d5d700 1 --2- 172.21.15.83:0/3289351797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0ba0 0x7fcf540af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.652+0000 7fcf59d5d700 1 -- 172.21.15.83:0/3289351797 >> 172.21.15.83:0/3289351797 conn(0x7fcf540b57b0 msgr2=0x7fcf540b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.652+0000 7fcf59d5d700 1 -- 172.21.15.83:0/3289351797 shutdown_connections 2023-11-26T07:11:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.652+0000 7fcf59d5d700 1 -- 172.21.15.83:0/3289351797 wait complete. 2023-11-26T07:11:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.654+0000 7fcf59d5d700 1 Processor -- start 2023-11-26T07:11:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.654+0000 7fcf59d5d700 1 -- start start 2023-11-26T07:11:34.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.659+0000 7fde72b9d700 1 -- 172.21.15.83:0/2371272368 shutdown_connections 2023-11-26T07:11:34.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.655+0000 7fcf59d5d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf540b0ba0 0x7fcf540af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.655+0000 7fcf59d5d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0f80 0x7fcf540b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.655+0000 7fcf59d5d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf5415ee90 0x7fcf541f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.655+0000 7fcf59d5d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540b4990 con 0x7fcf540b0ba0 2023-11-26T07:11:34.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.655+0000 7fcf59d5d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540b4480 con 0x7fcf5415ee90 2023-11-26T07:11:34.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.655+0000 7fcf59d5d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf541f8950 con 0x7fcf540b0f80 2023-11-26T07:11:34.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.655+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf540b0ba0 0x7fcf540af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.656+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf540b0ba0 0x7fcf540af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34380/0 (socket says 172.21.15.83:34380) 2023-11-26T07:11:34.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.656+0000 7fcf537fe700 1 -- 172.21.15.83:0/157492132 learned_addr learned my addr 172.21.15.83:0/157492132 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.656+0000 7fcf52ffd700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0f80 0x7fcf540b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.656+0000 7fcf53fff700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf5415ee90 0x7fcf541f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.656+0000 7fcf537fe700 1 -- 172.21.15.83:0/157492132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0f80 msgr2=0x7fcf540b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.656+0000 7fcf537fe700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0f80 0x7fcf540b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.656+0000 7fcf537fe700 1 -- 172.21.15.83:0/157492132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf5415ee90 msgr2=0x7fcf541f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.656+0000 7fcf537fe700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf5415ee90 0x7fcf541f27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.656+0000 7fcf537fe700 1 -- 172.21.15.83:0/157492132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440094a0 con 0x7fcf540b0ba0 2023-11-26T07:11:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.656+0000 7fcf537fe700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf540b0ba0 0x7fcf540af4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c007e00 tx=0x7fcf4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.659+0000 7fd16f7fe700 1 -- 172.21.15.83:0/999059439 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd174055f70 con 0x7fd1800b3b50 2023-11-26T07:11:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.659+0000 7fcf3bfff700 1 -- 172.21.15.83:0/157492132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00dcc0 con 0x7fcf540b0ba0 2023-11-26T07:11:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.659+0000 7fcf3bfff700 1 -- 172.21.15.83:0/157492132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c00f040 con 0x7fcf540b0ba0 2023-11-26T07:11:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.659+0000 7fcf3bfff700 1 -- 172.21.15.83:0/157492132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c009e10 con 0x7fcf540b0ba0 2023-11-26T07:11:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.659+0000 7fcf59d5d700 1 -- 172.21.15.83:0/157492132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf540b46b0 con 0x7fcf540b0ba0 2023-11-26T07:11:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.659+0000 7fcf59d5d700 1 -- 172.21.15.83:0/157492132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf541f8e70 con 0x7fcf540b0ba0 2023-11-26T07:11:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.659+0000 7fde72b9d700 1 -- 172.21.15.83:0/2371272368 wait complete. 2023-11-26T07:11:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.661+0000 7fde72b9d700 1 Processor -- start 2023-11-26T07:11:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.661+0000 7fde72b9d700 1 -- start start 2023-11-26T07:11:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.662+0000 7fde72b9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 0x7fde6c1fad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.663+0000 7fde72b9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0d1570 0x7fde6c0c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.663+0000 7fde72b9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c15dfc0 0x7fde6c0ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.663+0000 7fde72b9d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c15bcb0 con 0x7fde6c0d1570 2023-11-26T07:11:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.663+0000 7fde72b9d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1f8470 con 0x7fde6c15dfc0 2023-11-26T07:11:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.663+0000 7fde72b9d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c0c8610 con 0x7fde6c0b2fc0 2023-11-26T07:11:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.662+0000 7fcf3bfff700 1 -- 172.21.15.83:0/157492132 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcf4c003a20 con 0x7fcf540b0ba0 2023-11-26T07:11:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.662+0000 7fcf3bfff700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcf4006af20 0x7fcf4006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.663+0000 7fcf3bfff700 1 -- 172.21.15.83:0/157492132 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf4c0890c0 con 0x7fcf540b0ba0 2023-11-26T07:11:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.663+0000 7fcf52ffd700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcf4006af20 0x7fcf4006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.663+0000 7fde6bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0d1570 0x7fde6c0c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.663+0000 7fde6bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0d1570 0x7fde6c0c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34390/0 (socket says 172.21.15.83:34390) 2023-11-26T07:11:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.663+0000 7fde6bfff700 1 -- 172.21.15.83:0/2889615656 learned_addr learned my addr 172.21.15.83:0/2889615656 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.663+0000 7fcf52ffd700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcf4006af20 0x7fcf4006d3e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf540745c0 tx=0x7fcf48007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde6bfff700 1 -- 172.21.15.83:0/2889615656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 msgr2=0x7fde6c1fad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde70939700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 0x7fde6c1fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde6bfff700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 0x7fde6c1fad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde6bfff700 1 -- 172.21.15.83:0/2889615656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c15dfc0 msgr2=0x7fde6c0ca200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde6bfff700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c15dfc0 0x7fde6c0ca200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde6bfff700 1 -- 172.21.15.83:0/2889615656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde64009b30 con 0x7fde6c0d1570 2023-11-26T07:11:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde6bfff700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0d1570 0x7fde6c0c9cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fde5800e810 tx=0x7fde5800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde697fa700 1 -- 172.21.15.83:0/2889615656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58013610 con 0x7fde6c0d1570 2023-11-26T07:11:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde697fa700 1 -- 172.21.15.83:0/2889615656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58013770 con 0x7fde6c0d1570 2023-11-26T07:11:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde697fa700 1 -- 172.21.15.83:0/2889615656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58013b00 con 0x7fde6c0d1570 2023-11-26T07:11:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde72b9d700 1 -- 172.21.15.83:0/2889615656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c0c88f0 con 0x7fde6c0d1570 2023-11-26T07:11:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.664+0000 7fde72b9d700 1 -- 172.21.15.83:0/2889615656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c0c8eb0 con 0x7fde6c0d1570 2023-11-26T07:11:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.666+0000 7f22ec264700 1 Processor -- start 2023-11-26T07:11:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.666+0000 7fde697fa700 1 -- 172.21.15.83:0/2889615656 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde58013c60 con 0x7fde6c0d1570 2023-11-26T07:11:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.667+0000 7fde697fa700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5c06b080 0x7fde5c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.667+0000 7fde697fa700 1 -- 172.21.15.83:0/2889615656 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde580893b0 con 0x7fde6c0d1570 2023-11-26T07:11:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.666+0000 7f22ec264700 1 -- start start 2023-11-26T07:11:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.667+0000 7f22ec264700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22e40d0810 0x7f22e4166b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.667+0000 7f22ec264700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40afe30 con 0x7f22e415bf50 2023-11-26T07:11:34.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.667+0000 7f22ec264700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40af220 con 0x7f22e40b31c0 2023-11-26T07:11:34.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.667+0000 7f22ec264700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e40b3640 con 0x7f22e40d0810 2023-11-26T07:11:34.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.668+0000 7fcf59d5d700 1 -- 172.21.15.83:0/157492132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf54067f60 con 0x7fcf540b0ba0 2023-11-26T07:11:34.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.668+0000 7f22e97ff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f22e40b31c0 0x7f22e40d0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:39056/0 (socket says 172.21.15.83:39056) 2023-11-26T07:11:34.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.668+0000 7f22e97ff700 1 -- 172.21.15.83:0/122891866 learned_addr learned my addr 172.21.15.83:0/122891866 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:34.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.668+0000 7fde70939700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5c06b080 0x7fde5c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.668+0000 7fde70939700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5c06b080 0x7fde5c06d540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde60007fd0 tx=0x7fde60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.670+0000 7fde72b9d700 1 -- 172.21.15.83:0/2889615656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c0052e0 con 0x7fde6c0d1570 2023-11-26T07:11:34.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.677+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193318626 0 0) 0x7f22e40af220 con 0x7f22e40b31c0 2023-11-26T07:11:34.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.677+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d00035b0 con 0x7f22e40b31c0 2023-11-26T07:11:34.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 825229405 0 0) 0x7f22d00035b0 con 0x7f22e40b31c0 2023-11-26T07:11:34.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 >> v2:172.21.15.83:3300/0 conn(0x7f22e40d0810 msgr2=0x7f22e4166b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22ea801700 1 --2- 172.21.15.83:0/122891866 >> v2:172.21.15.83:3300/0 conn(0x7f22e40d0810 0x7f22e4166b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 --2- 172.21.15.83:0/122891866 >> v2:172.21.15.83:3300/0 conn(0x7f22e40d0810 0x7f22e4166b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 >> v1:172.21.15.83:6789/0 conn(0x7f22e415bf50 legacy=0x7f22e40aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40af500 con 0x7f22e40b31c0 2023-11-26T07:11:34.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f22e0001670 con 0x7f22e40b31c0 2023-11-26T07:11:34.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 >> v1:172.21.15.203:6789/0 conn(0x7f22e40b31c0 legacy=0x7f22e40d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 --2- 172.21.15.83:0/122891866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d0003a50 0x7f22d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 --2- 172.21.15.83:0/122891866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22d0004850 0x7f22d0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 --2- 172.21.15.83:0/122891866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22d0005410 0x7f22d000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40b3640 con 0x7f22d0003a50 2023-11-26T07:11:34.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d000c270 con 0x7f22d0005410 2023-11-26T07:11:34.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d000c430 con 0x7f22d0004850 2023-11-26T07:11:34.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22ea801700 1 --2- 172.21.15.83:0/122891866 >> v2:172.21.15.83:3300/0 conn(0x7f22e40d0810 0x7f22e4166b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:34.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7fcf3bfff700 1 -- 172.21.15.83:0/157492132 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf4c054300 con 0x7fcf540b0ba0 2023-11-26T07:11:34.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.678+0000 7f22ea000700 1 --2- 172.21.15.83:0/122891866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d0003a50 0x7f22d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.679+0000 7f22ea000700 1 -- 172.21.15.83:0/122891866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22d0004850 msgr2=0x7f22d0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.679+0000 7f22ea000700 1 --2- 172.21.15.83:0/122891866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22d0004850 0x7f22d0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.679+0000 7f22ea000700 1 -- 172.21.15.83:0/122891866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22d0005410 msgr2=0x7f22d000ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:11:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.679+0000 7f22ea000700 1 --2- 172.21.15.83:0/122891866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22d0005410 0x7f22d000ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.679+0000 7f22ea000700 1 -- 172.21.15.83:0/122891866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40af500 con 0x7f22d0003a50 2023-11-26T07:11:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.679+0000 7f22ea000700 1 --2- 172.21.15.83:0/122891866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d0003a50 0x7f22d0003e70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22dc002220 tx=0x7f22d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21e4ef25f2c6207 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.680+0000 7f22ea801700 1 -- 172.21.15.83:0/122891866 reap_dead start 2023-11-26T07:11:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.687+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc003c00 con 0x7f22d0003a50 2023-11-26T07:11:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.687+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc003d60 con 0x7f22d0003a50 2023-11-26T07:11:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.687+0000 7f22e8ffe700 1 -- 172.21.15.83:0/122891866 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00cbd0 con 0x7f22d0003a50 2023-11-26T07:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.687+0000 7fde697fa700 1 -- 172.21.15.83:0/2889615656 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde580543d0 con 0x7fde6c0d1570 2023-11-26T07:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.688+0000 7f22ec264700 1 -- 172.21.15.83:0/122891866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d0003a50 msgr2=0x7f22d0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.688+0000 7f22ec264700 1 --2- 172.21.15.83:0/122891866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d0003a50 0x7f22d0003e70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22dc002220 tx=0x7f22d0005880 comp rx=0 tx=0).stop 2023-11-26T07:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.688+0000 7f22ec264700 1 -- 172.21.15.83:0/122891866 shutdown_connections 2023-11-26T07:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.688+0000 7f22ec264700 1 --2- 172.21.15.83:0/122891866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d0003a50 0x7f22d0003e70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.688+0000 7f22ec264700 1 -- 172.21.15.83:0/122891866 >> 172.21.15.83:0/122891866 conn(0x7f22e409aa00 msgr2=0x7f22e409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.689+0000 7f22ec264700 1 -- 172.21.15.83:0/122891866 shutdown_connections 2023-11-26T07:11:34.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.689+0000 7f22ec264700 1 -- 172.21.15.83:0/122891866 wait complete. 2023-11-26T07:11:34.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.691+0000 7f22ec264700 1 Processor -- start 2023-11-26T07:11:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.691+0000 7f22ec264700 1 -- start start 2023-11-26T07:11:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.692+0000 7f22ec264700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22d0003a50 0x7f22e40b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.692+0000 7f22ec264700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e40cee70 0x7f22e4154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.692+0000 7f22ec264700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e4154a40 0x7f22e406b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.692+0000 7f22ec264700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40ce9e0 con 0x7f22e40cee70 2023-11-26T07:11:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.692+0000 7f22ec264700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40cde40 con 0x7f22e4154a40 2023-11-26T07:11:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.692+0000 7f22ec264700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40ce0b0 con 0x7f22d0003a50 2023-11-26T07:11:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.692+0000 7f22e97ff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e40cee70 0x7f22e4154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.692+0000 7f22e97ff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e40cee70 0x7f22e4154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34410/0 (socket says 172.21.15.83:34410) 2023-11-26T07:11:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.692+0000 7f22e97ff700 1 -- 172.21.15.83:0/2536628433 learned_addr learned my addr 172.21.15.83:0/2536628433 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.692+0000 7f22ea801700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e4154a40 0x7f22e406b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.693+0000 7f22ea000700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22d0003a50 0x7f22e40b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.693+0000 7f22ea000700 1 -- 172.21.15.83:0/2536628433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e4154a40 msgr2=0x7f22e406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.693+0000 7f22ea000700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e4154a40 0x7f22e406b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.693+0000 7f22ea000700 1 -- 172.21.15.83:0/2536628433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e40cee70 msgr2=0x7f22e4154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.693+0000 7f22ea000700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e40cee70 0x7f22e4154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.693+0000 7f22ea000700 1 -- 172.21.15.83:0/2536628433 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40ce390 con 0x7f22d0003a50 2023-11-26T07:11:34.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.693+0000 7f22ea000700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22d0003a50 0x7f22e40b70b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f22dc018000 tx=0x7f22dc0013d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.706+0000 7f22da7fc700 1 -- 172.21.15.83:0/2536628433 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0053a0 con 0x7f22d0003a50 2023-11-26T07:11:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.706+0000 7f22da7fc700 1 -- 172.21.15.83:0/2536628433 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc005500 con 0x7f22d0003a50 2023-11-26T07:11:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.706+0000 7f22da7fc700 1 -- 172.21.15.83:0/2536628433 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0058b0 con 0x7f22d0003a50 2023-11-26T07:11:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.707+0000 7f22ec264700 1 -- 172.21.15.83:0/2536628433 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e40ce670 con 0x7f22d0003a50 2023-11-26T07:11:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.707+0000 7f22ec264700 1 -- 172.21.15.83:0/2536628433 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e40cf2f0 con 0x7f22d0003a50 2023-11-26T07:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.713+0000 7f22da7fc700 1 -- 172.21.15.83:0/2536628433 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f22dc024460 con 0x7f22d0003a50 2023-11-26T07:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.713+0000 7f22ec264700 1 -- 172.21.15.83:0/2536628433 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22cc0052e0 con 0x7f22d0003a50 2023-11-26T07:11:34.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.713+0000 7f22da7fc700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f22c806afa0 0x7f22c806d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.713+0000 7f22da7fc700 1 -- 172.21.15.83:0/2536628433 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22dc08a910 con 0x7f22d0003a50 2023-11-26T07:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.717+0000 7f22e97ff700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f22c806afa0 0x7f22c806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.715+0000 7fb55d7fa700 1 Processor -- start 2023-11-26T07:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.715+0000 7fb55d7fa700 1 -- start start 2023-11-26T07:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.717+0000 7fb55d7fa700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb558164a90 0x7fb55806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.717+0000 7fb55d7fa700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55806ced0 con 0x7fb55806d4d0 2023-11-26T07:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.717+0000 7fb55d7fa700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55806da70 con 0x7fb5580b3b50 2023-11-26T07:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.717+0000 7fb55d7fa700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5581634e0 con 0x7fb558164a90 2023-11-26T07:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.717+0000 7fb5577fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb55806d4d0 0x7fb55816bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54458/0 (socket says 172.21.15.83:54458) 2023-11-26T07:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.717+0000 7fb5577fe700 1 -- 172.21.15.83:0/2928226591 learned_addr learned my addr 172.21.15.83:0/2928226591 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.717+0000 7fb5567fc700 1 --2- 172.21.15.83:0/2928226591 >> v2:172.21.15.203:3300/0 conn(0x7fb558164a90 0x7fb55806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836759049 0 0) 0x7fb55806ced0 con 0x7fb55806d4d0 2023-11-26T07:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5400035b0 con 0x7fb55806d4d0 2023-11-26T07:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1073365976 0 0) 0x7fb5400035b0 con 0x7fb55806d4d0 2023-11-26T07:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 >> v2:172.21.15.203:3300/0 conn(0x7fb558164a90 msgr2=0x7fb55806eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 --2- 172.21.15.83:0/2928226591 >> v2:172.21.15.203:3300/0 conn(0x7fb558164a90 0x7fb55806eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 >> v1:172.21.15.203:6789/0 conn(0x7fb5580b3b50 legacy=0x7fb5580b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5581637c0 con 0x7fb55806d4d0 2023-11-26T07:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fb5500019f0 con 0x7fb55806d4d0 2023-11-26T07:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 >> v1:172.21.15.83:6789/0 conn(0x7fb55806d4d0 legacy=0x7fb55816bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 --2- 172.21.15.83:0/2928226591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb540003a50 0x7fb540003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 --2- 172.21.15.83:0/2928226591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb540004850 0x7fb540004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 --2- 172.21.15.83:0/2928226591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5400052e0 0x7fb54000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5581634e0 con 0x7fb540004850 2023-11-26T07:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54000c230 con 0x7fb5400052e0 2023-11-26T07:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.718+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54000c3f0 con 0x7fb540003a50 2023-11-26T07:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.719+0000 7fb5577fe700 1 --2- 172.21.15.83:0/2928226591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5400052e0 0x7fb54000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.719+0000 7fb5567fc700 1 --2- 172.21.15.83:0/2928226591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb540004850 0x7fb540004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.719+0000 7fb5567fc700 1 -- 172.21.15.83:0/2928226591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb540003a50 msgr2=0x7fb540003e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.719+0000 7fb5567fc700 1 --2- 172.21.15.83:0/2928226591 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb540003a50 0x7fb540003e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.719+0000 7fb5567fc700 1 -- 172.21.15.83:0/2928226591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5400052e0 msgr2=0x7fb54000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.719+0000 7fb5567fc700 1 --2- 172.21.15.83:0/2928226591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5400052e0 0x7fb54000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.719+0000 7fb5567fc700 1 -- 172.21.15.83:0/2928226591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5400035b0 con 0x7fb540004850 2023-11-26T07:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.719+0000 7f22da7fc700 1 -- 172.21.15.83:0/2536628433 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f22dc055ae0 con 0x7f22d0003a50 2023-11-26T07:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.720+0000 7f22e97ff700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f22c806afa0 0x7f22c806d460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f22e40aefa0 tx=0x7f22e0001a30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.720+0000 7fb5567fc700 1 --2- 172.21.15.83:0/2928226591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb540004850 0x7fb540004cd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb548009850 tx=0x7fb540005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fad2de806a1c25 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.720+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54800c070 con 0x7fb540004850 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.720+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb548013610 con 0x7fb540004850 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.720+0000 7fb555ffb700 1 -- 172.21.15.83:0/2928226591 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5480139c0 con 0x7fb540004850 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.721+0000 7fb55d7fa700 1 -- 172.21.15.83:0/2928226591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb540004850 msgr2=0x7fb540004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.721+0000 7fb55d7fa700 1 --2- 172.21.15.83:0/2928226591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb540004850 0x7fb540004cd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb548009850 tx=0x7fb540005880 comp rx=0 tx=0).stop 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.721+0000 7fb5577fe700 1 -- 172.21.15.83:0/2928226591 reap_dead start 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.721+0000 7fb5577fe700 1 -- 172.21.15.83:0/2928226591 reap_dead start 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.721+0000 7fb55d7fa700 1 -- 172.21.15.83:0/2928226591 shutdown_connections 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.721+0000 7fb55d7fa700 1 -- 172.21.15.83:0/2928226591 >> 172.21.15.83:0/2928226591 conn(0x7fb5580cfa30 msgr2=0x7fb5580cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.722+0000 7fb55d7fa700 1 -- 172.21.15.83:0/2928226591 shutdown_connections 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.722+0000 7fb55d7fa700 1 -- 172.21.15.83:0/2928226591 wait complete. 2023-11-26T07:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.724+0000 7fb55d7fa700 1 Processor -- start 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.725+0000 7fb55d7fa700 1 -- start start 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.726+0000 7fb55d7fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb55814a440 0x7fb558147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.726+0000 7fb55d7fa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5580cc8d0 0x7fb55815bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.726+0000 7fb55d7fa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb55814a820 0x7fb55815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.726+0000 7fb55d7fa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb558072710 con 0x7fb5580cc8d0 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.726+0000 7fb55d7fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55814cf60 con 0x7fb55814a440 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.726+0000 7fb55d7fa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5581479a0 con 0x7fb55814a820 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.727+0000 7fb5567fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5580cc8d0 0x7fb55815bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.727+0000 7fb5567fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5580cc8d0 0x7fb55815bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34428/0 (socket says 172.21.15.83:34428) 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.727+0000 7fb5567fc700 1 -- 172.21.15.83:0/898805554 learned_addr learned my addr 172.21.15.83:0/898805554 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.727+0000 7fb5567fc700 1 -- 172.21.15.83:0/898805554 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb55814a820 msgr2=0x7fb55815c0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.727+0000 7fb5567fc700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb55814a820 0x7fb55815c0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.727+0000 7fb5567fc700 1 -- 172.21.15.83:0/898805554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb55814a440 msgr2=0x7fb558147460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.727+0000 7fb5567fc700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb55814a440 0x7fb558147460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.727+0000 7fb5567fc700 1 -- 172.21.15.83:0/898805554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5580cc440 con 0x7fb5580cc8d0 2023-11-26T07:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.727+0000 7fb5567fc700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5580cc8d0 0x7fb55815bb80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb548013e90 tx=0x7fb5480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.728+0000 7fb53f7fe700 1 -- 172.21.15.83:0/898805554 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54801d840 con 0x7fb5580cc8d0 2023-11-26T07:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.728+0000 7fb53f7fe700 1 -- 172.21.15.83:0/898805554 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54801d9a0 con 0x7fb5580cc8d0 2023-11-26T07:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.728+0000 7fb53f7fe700 1 -- 172.21.15.83:0/898805554 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54801dd30 con 0x7fb5580cc8d0 2023-11-26T07:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.728+0000 7fb55d7fa700 1 -- 172.21.15.83:0/898805554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5581395b0 con 0x7fb5580cc8d0 2023-11-26T07:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.728+0000 7fb55d7fa700 1 -- 172.21.15.83:0/898805554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb558072c00 con 0x7fb5580cc8d0 2023-11-26T07:11:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.732+0000 7fb53f7fe700 1 -- 172.21.15.83:0/898805554 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb548030440 con 0x7fb5580cc8d0 2023-11-26T07:11:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.733+0000 7fb53f7fe700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb54006cc70 0x7fb54006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.733+0000 7fb53f7fe700 1 -- 172.21.15.83:0/898805554 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb54808a9b0 con 0x7fb5580cc8d0 2023-11-26T07:11:34.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.736+0000 7fb55d7fa700 1 -- 172.21.15.83:0/898805554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5380052e0 con 0x7fb5580cc8d0 2023-11-26T07:11:34.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.738+0000 7fb556ffd700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb54006cc70 0x7fb54006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.745+0000 7fb53f7fe700 1 -- 172.21.15.83:0/898805554 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb548055a50 con 0x7fb5580cc8d0 2023-11-26T07:11:34.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.751+0000 7fb556ffd700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb54006cc70 0x7fb54006f130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb54c007fd0 tx=0x7fb54c007a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.763+0000 7febf5ce3700 1 Processor -- start 2023-11-26T07:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.763+0000 7febf5ce3700 1 -- start start 2023-11-26T07:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.765+0000 7febf5ce3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7febf00d0520 0x7febf00b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.765+0000 7febf5ce3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf00720e0 con 0x7febf015d610 2023-11-26T07:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.765+0000 7febf5ce3700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf00723c0 con 0x7febf0071be0 2023-11-26T07:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.765+0000 7febf5ce3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf0072d60 con 0x7febf00d0520 2023-11-26T07:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.766+0000 7febef7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7febf00d0520 0x7febf00b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.766+0000 7febef7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7febf00d0520 0x7febf00b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34436/0 (socket says 172.21.15.83:34436) 2023-11-26T07:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.766+0000 7febef7fe700 1 -- 172.21.15.83:0/1225166583 learned_addr learned my addr 172.21.15.83:0/1225166583 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.766+0000 7febef7fe700 1 -- 172.21.15.83:0/1225166583 >> v1:172.21.15.203:6789/0 conn(0x7febf0071be0 legacy=0x7febf0140fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.766+0000 7febef7fe700 1 -- 172.21.15.83:0/1225166583 >> v1:172.21.15.83:6789/0 conn(0x7febf015d610 legacy=0x7febf00714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.766+0000 7febef7fe700 1 -- 172.21.15.83:0/1225166583 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf00726a0 con 0x7febf00d0520 2023-11-26T07:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.766+0000 7febef7fe700 1 --2- 172.21.15.83:0/1225166583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf00d0520 0x7febf00b3b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7febe8009e50 tx=0x7febe800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9877da6be76ccf28 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.767+0000 7febee7fc700 1 -- 172.21.15.83:0/1225166583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe800e070 con 0x7febf00d0520 2023-11-26T07:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.767+0000 7febee7fc700 1 -- 172.21.15.83:0/1225166583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe8003a20 con 0x7febf00d0520 2023-11-26T07:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.767+0000 7febee7fc700 1 -- 172.21.15.83:0/1225166583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8003dd0 con 0x7febf00d0520 2023-11-26T07:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.768+0000 7febf5ce3700 1 -- 172.21.15.83:0/1225166583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf00d0520 msgr2=0x7febf00b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.768+0000 7febf5ce3700 1 --2- 172.21.15.83:0/1225166583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf00d0520 0x7febf00b3b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7febe8009e50 tx=0x7febe800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:34.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.769+0000 7febf5ce3700 1 -- 172.21.15.83:0/1225166583 shutdown_connections 2023-11-26T07:11:34.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.769+0000 7febf5ce3700 1 --2- 172.21.15.83:0/1225166583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf00d0520 0x7febf00b3b50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.769+0000 7febf5ce3700 1 -- 172.21.15.83:0/1225166583 >> 172.21.15.83:0/1225166583 conn(0x7febf00cfa30 msgr2=0x7febf00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:34.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.769+0000 7febf5ce3700 1 -- 172.21.15.83:0/1225166583 shutdown_connections 2023-11-26T07:11:34.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.769+0000 7febf5ce3700 1 -- 172.21.15.83:0/1225166583 wait complete. 2023-11-26T07:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.771+0000 7febf5ce3700 1 Processor -- start 2023-11-26T07:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.772+0000 7febf5ce3700 1 -- start start 2023-11-26T07:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.773+0000 7febf5ce3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf0071be0 0x7febf013cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.773+0000 7febf5ce3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febf00d0520 0x7febf00ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.773+0000 7febf5ce3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febf015d610 0x7febf0153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.773+0000 7febf5ce3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf00cecb0 con 0x7febf0071be0 2023-11-26T07:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.773+0000 7febf5ce3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf00cef90 con 0x7febf015d610 2023-11-26T07:11:34.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.773+0000 7febf5ce3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf00cf270 con 0x7febf00d0520 2023-11-26T07:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.773+0000 7febef7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf0071be0 0x7febf013cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.773+0000 7febef7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf0071be0 0x7febf013cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34444/0 (socket says 172.21.15.83:34444) 2023-11-26T07:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.773+0000 7febef7fe700 1 -- 172.21.15.83:0/562318360 learned_addr learned my addr 172.21.15.83:0/562318360 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.774+0000 7febef7fe700 1 -- 172.21.15.83:0/562318360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febf00d0520 msgr2=0x7febf00ce3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.774+0000 7febef7fe700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febf00d0520 0x7febf00ce3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.774+0000 7febef7fe700 1 -- 172.21.15.83:0/562318360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febf015d610 msgr2=0x7febf0153050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.774+0000 7febef7fe700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febf015d610 0x7febf0153050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.774+0000 7febef7fe700 1 -- 172.21.15.83:0/562318360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8009b30 con 0x7febf0071be0 2023-11-26T07:11:34.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.774+0000 7febef7fe700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf0071be0 0x7febf013cfc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7febf00737d0 tx=0x7febe8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.776+0000 7febdffff700 1 -- 172.21.15.83:0/562318360 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8004500 con 0x7febf0071be0 2023-11-26T07:11:34.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.776+0000 7febdffff700 1 -- 172.21.15.83:0/562318360 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe80046a0 con 0x7febf0071be0 2023-11-26T07:11:34.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.776+0000 7febdffff700 1 -- 172.21.15.83:0/562318360 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8012430 con 0x7febf0071be0 2023-11-26T07:11:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.776+0000 7febf5ce3700 1 -- 172.21.15.83:0/562318360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf00cf550 con 0x7febf0071be0 2023-11-26T07:11:34.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.776+0000 7febf5ce3700 1 -- 172.21.15.83:0/562318360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf00ce910 con 0x7febf0071be0 2023-11-26T07:11:34.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.781+0000 7febdffff700 1 -- 172.21.15.83:0/562318360 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7febe8019030 con 0x7febf0071be0 2023-11-26T07:11:34.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.781+0000 7febdffff700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7febd806b130 0x7febd806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:34.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.781+0000 7febdffff700 1 -- 172.21.15.83:0/562318360 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febe808a900 con 0x7febf0071be0 2023-11-26T07:11:34.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.782+0000 7febeeffd700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7febd806b130 0x7febd806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.783+0000 7febeeffd700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7febd806b130 0x7febd806d5f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7febe0006fd0 tx=0x7febe0006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:34.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.783+0000 7febf5ce3700 1 -- 172.21.15.83:0/562318360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd40052e0 con 0x7febf0071be0 2023-11-26T07:11:34.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:34.793+0000 7febdffff700 1 -- 172.21.15.83:0/562318360 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7febe8055a20 con 0x7febf0071be0 2023-11-26T07:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.008+0000 7f8835ec1700 1 -- 172.21.15.83:0/1193379285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8810005860 con 0x7f883006d4d0 2023-11-26T07:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.008+0000 7f8817fff700 1 -- 172.21.15.83:0/1193379285 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8820015020 con 0x7f883006d4d0 2023-11-26T07:11:35.013 INFO:teuthology.orchestra.run.smithi083.stdout:55834574854 2023-11-26T07:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.012+0000 7f8835ec1700 1 -- 172.21.15.83:0/1193379285 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f881806b0b0 msgr2=0x7f881806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.012+0000 7f8835ec1700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f881806b0b0 0x7f881806d570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8830162340 tx=0x7f8824008040 comp rx=0 tx=0).stop 2023-11-26T07:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.012+0000 7f8835ec1700 1 -- 172.21.15.83:0/1193379285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f883006d4d0 msgr2=0x7f88300cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.012+0000 7f8835ec1700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f883006d4d0 0x7f88300cce60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f882000b470 tx=0x7f882000b830 comp rx=0 tx=0).stop 2023-11-26T07:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.012+0000 7f8835ec1700 1 -- 172.21.15.83:0/1193379285 shutdown_connections 2023-11-26T07:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.012+0000 7f8835ec1700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f881806b0b0 0x7f881806d570 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.012+0000 7f8835ec1700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8830163a80 0x7f8830152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.012+0000 7f8835ec1700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f883006d4d0 0x7f88300cce60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.012+0000 7f8835ec1700 1 --2- 172.21.15.83:0/1193379285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88300b3b50 0x7f8830157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.012+0000 7f8835ec1700 1 -- 172.21.15.83:0/1193379285 >> 172.21.15.83:0/1193379285 conn(0x7f88300cfa30 msgr2=0x7f88300711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.013+0000 7f8835ec1700 1 -- 172.21.15.83:0/1193379285 shutdown_connections 2023-11-26T07:11:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.013+0000 7f8835ec1700 1 -- 172.21.15.83:0/1193379285 wait complete. 2023-11-26T07:11:35.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.017+0000 7f585304b700 1 -- 172.21.15.83:0/3525863177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f584c15c930 con 0x7f584c071be0 2023-11-26T07:11:35.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.017+0000 7f58497fa700 1 -- 172.21.15.83:0/3525863177 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f58440575f0 con 0x7f584c071be0 2023-11-26T07:11:35.022 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-26T07:11:35.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.021+0000 7f585304b700 1 -- 172.21.15.83:0/3525863177 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f583808cf30 msgr2=0x7f583808f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.021+0000 7f585304b700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f583808cf30 0x7f583808f3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f584c0af500 tx=0x7f5840007a20 comp rx=0 tx=0).stop 2023-11-26T07:11:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.021+0000 7f585304b700 1 -- 172.21.15.83:0/3525863177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f584c071be0 msgr2=0x7f584c1f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.021+0000 7f585304b700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f584c071be0 0x7f584c1f4f30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f584400cec0 tx=0x7f58440100b0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.022+0000 7f585304b700 1 -- 172.21.15.83:0/3525863177 shutdown_connections 2023-11-26T07:11:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.022+0000 7f585304b700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f584c15d6a0 0x7f584c1569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.022+0000 7f585304b700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f584c071be0 0x7f584c1f4f30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.022+0000 7f585304b700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f583808cf30 0x7f583808f3f0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f584c0af500 tx=0x7f5840007a20 comp rx=0 tx=0).stop 2023-11-26T07:11:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.022+0000 7f585304b700 1 --2- 172.21.15.83:0/3525863177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584c0b4090 0x7f584c156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.022+0000 7f585304b700 1 -- 172.21.15.83:0/3525863177 >> 172.21.15.83:0/3525863177 conn(0x7f584c0cfa30 msgr2=0x7f584c140150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:35.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.023+0000 7f585304b700 1 -- 172.21.15.83:0/3525863177 shutdown_connections 2023-11-26T07:11:35.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.023+0000 7f585304b700 1 -- 172.21.15.83:0/3525863177 wait complete. 2023-11-26T07:11:35.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-11-26T07:11:35.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2023-11-26T07:11:35.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.052+0000 7fd1856b0700 1 -- 172.21.15.83:0/999059439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd1800b4180 con 0x7fd1800b3b50 2023-11-26T07:11:35.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.052+0000 7fd16f7fe700 1 -- 172.21.15.83:0/999059439 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd174059590 con 0x7fd1800b3b50 2023-11-26T07:11:35.054 INFO:teuthology.orchestra.run.smithi083.stdout:55834574854 2023-11-26T07:11:35.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.056+0000 7fd1856b0700 1 -- 172.21.15.83:0/999059439 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd16406aed0 msgr2=0x7fd16406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.056+0000 7fd1856b0700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd16406aed0 0x7fd16406d390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1800cf770 tx=0x7fd178001650 comp rx=0 tx=0).stop 2023-11-26T07:11:35.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.056+0000 7fd1856b0700 1 -- 172.21.15.83:0/999059439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1800b3b50 msgr2=0x7fd180140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.056+0000 7fd1856b0700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1800b3b50 0x7fd180140cb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd1740090b0 tx=0x7fd174004750 comp rx=0 tx=0).stop 2023-11-26T07:11:35.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.057+0000 7fd1856b0700 1 -- 172.21.15.83:0/999059439 shutdown_connections 2023-11-26T07:11:35.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.057+0000 7fd1856b0700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd180164a90 0x7fd1801411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.057+0000 7fd1856b0700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd16406aed0 0x7fd16406d390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.057+0000 7fd1856b0700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd18006d4d0 0x7fd1800b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.057+0000 7fd1856b0700 1 --2- 172.21.15.83:0/999059439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1800b3b50 0x7fd180140cb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.057+0000 7fd1856b0700 1 -- 172.21.15.83:0/999059439 >> 172.21.15.83:0/999059439 conn(0x7fd1800cfa30 msgr2=0x7fd18016c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.060+0000 7fd1856b0700 1 -- 172.21.15.83:0/999059439 shutdown_connections 2023-11-26T07:11:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.060+0000 7fd1856b0700 1 -- 172.21.15.83:0/999059439 wait complete. 2023-11-26T07:11:35.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.060+0000 7fcf59d5d700 1 -- 172.21.15.83:0/157492132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcf54071f00 con 0x7fcf540b0ba0 2023-11-26T07:11:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.060+0000 7fcf3bfff700 1 -- 172.21.15.83:0/157492132 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcf4c01a020 con 0x7fcf540b0ba0 2023-11-26T07:11:35.062 INFO:teuthology.orchestra.run.smithi083.stdout:55834574854 2023-11-26T07:11:35.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.062+0000 7fcf59d5d700 1 -- 172.21.15.83:0/157492132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcf4006af20 msgr2=0x7fcf4006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.062+0000 7fcf59d5d700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcf4006af20 0x7fcf4006d3e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf540745c0 tx=0x7fcf48007ec0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.062+0000 7fcf59d5d700 1 -- 172.21.15.83:0/157492132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf540b0ba0 msgr2=0x7fcf540af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.062+0000 7fcf59d5d700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf540b0ba0 0x7fcf540af4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c007e00 tx=0x7fcf4c00d3b0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.068+0000 7fcf59d5d700 1 -- 172.21.15.83:0/157492132 shutdown_connections 2023-11-26T07:11:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.068+0000 7fcf59d5d700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf5415ee90 0x7fcf541f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.068+0000 7fcf59d5d700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf540b0ba0 0x7fcf540af4b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.068+0000 7fcf59d5d700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcf4006af20 0x7fcf4006d3e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.068+0000 7fcf59d5d700 1 --2- 172.21.15.83:0/157492132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf540b0f80 0x7fcf540b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.068+0000 7fcf59d5d700 1 -- 172.21.15.83:0/157492132 >> 172.21.15.83:0/157492132 conn(0x7fcf540b57b0 msgr2=0x7fcf54158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.068+0000 7fcf59d5d700 1 -- 172.21.15.83:0/157492132 shutdown_connections 2023-11-26T07:11:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.068+0000 7fcf59d5d700 1 -- 172.21.15.83:0/157492132 wait complete. 2023-11-26T07:11:35.079 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-11-26T07:11:35.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-11-26T07:11:35.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.085+0000 7f22ec264700 1 -- 172.21.15.83:0/2536628433 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f22cc005590 con 0x7f22d0003a50 2023-11-26T07:11:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.086+0000 7f22da7fc700 1 -- 172.21.15.83:0/2536628433 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f22dc059100 con 0x7f22d0003a50 2023-11-26T07:11:35.087 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-26T07:11:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.086+0000 7fde72b9d700 1 -- 172.21.15.83:0/2889615656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fde4c005590 con 0x7fde6c0d1570 2023-11-26T07:11:35.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.086+0000 7fde697fa700 1 -- 172.21.15.83:0/2889615656 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fde580579f0 con 0x7fde6c0d1570 2023-11-26T07:11:35.088 INFO:teuthology.orchestra.run.smithi083.stdout:55834574854 2023-11-26T07:11:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.088+0000 7f22ec264700 1 -- 172.21.15.83:0/2536628433 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f22c806afa0 msgr2=0x7f22c806d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.088+0000 7f22ec264700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f22c806afa0 0x7f22c806d460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f22e40aefa0 tx=0x7f22e0001a30 comp rx=0 tx=0).stop 2023-11-26T07:11:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.088+0000 7f22ec264700 1 -- 172.21.15.83:0/2536628433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22d0003a50 msgr2=0x7f22e40b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.088+0000 7f22ec264700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22d0003a50 0x7f22e40b70b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f22dc018000 tx=0x7f22dc0013d0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7f22ec264700 1 -- 172.21.15.83:0/2536628433 shutdown_connections 2023-11-26T07:11:35.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7f22ec264700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e4154a40 0x7f22e406b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7f22ec264700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f22c806afa0 0x7f22c806d460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7f22ec264700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22d0003a50 0x7f22e40b70b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7f22ec264700 1 --2- 172.21.15.83:0/2536628433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e40cee70 0x7f22e4154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7f22ec264700 1 -- 172.21.15.83:0/2536628433 >> 172.21.15.83:0/2536628433 conn(0x7f22e409aa00 msgr2=0x7f22e40d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:35.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7f22ec264700 1 -- 172.21.15.83:0/2536628433 shutdown_connections 2023-11-26T07:11:35.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7f22ec264700 1 -- 172.21.15.83:0/2536628433 wait complete. 2023-11-26T07:11:35.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7fde72b9d700 1 -- 172.21.15.83:0/2889615656 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5c06b080 msgr2=0x7fde5c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7fde72b9d700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5c06b080 0x7fde5c06d540 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde60007fd0 tx=0x7fde60009040 comp rx=0 tx=0).stop 2023-11-26T07:11:35.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7fde72b9d700 1 -- 172.21.15.83:0/2889615656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0d1570 msgr2=0x7fde6c0c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.089+0000 7fde72b9d700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0d1570 0x7fde6c0c9cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fde5800e810 tx=0x7fde5800ebd0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.090+0000 7fde72b9d700 1 -- 172.21.15.83:0/2889615656 shutdown_connections 2023-11-26T07:11:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.090+0000 7fde72b9d700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c15dfc0 0x7fde6c0ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.090+0000 7fde72b9d700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5c06b080 0x7fde5c06d540 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.090+0000 7fde72b9d700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c0b2fc0 0x7fde6c1fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.090+0000 7fde72b9d700 1 --2- 172.21.15.83:0/2889615656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0d1570 0x7fde6c0c9cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.090+0000 7fde72b9d700 1 -- 172.21.15.83:0/2889615656 >> 172.21.15.83:0/2889615656 conn(0x7fde6c06b410 msgr2=0x7fde6c0b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.090+0000 7fde72b9d700 1 -- 172.21.15.83:0/2889615656 shutdown_connections 2023-11-26T07:11:35.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.090+0000 7fde72b9d700 1 -- 172.21.15.83:0/2889615656 wait complete. 2023-11-26T07:11:35.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2023-11-26T07:11:35.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-11-26T07:11:35.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.131+0000 7fb55d7fa700 1 -- 172.21.15.83:0/898805554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb538005590 con 0x7fb5580cc8d0 2023-11-26T07:11:35.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.131+0000 7fb53f7fe700 1 -- 172.21.15.83:0/898805554 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb548017030 con 0x7fb5580cc8d0 2023-11-26T07:11:35.133 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-26T07:11:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.133+0000 7fb55d7fa700 1 -- 172.21.15.83:0/898805554 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb54006cc70 msgr2=0x7fb54006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.133+0000 7fb55d7fa700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb54006cc70 0x7fb54006f130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb54c007fd0 tx=0x7fb54c007a20 comp rx=0 tx=0).stop 2023-11-26T07:11:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.133+0000 7fb55d7fa700 1 -- 172.21.15.83:0/898805554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5580cc8d0 msgr2=0x7fb55815bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.133+0000 7fb55d7fa700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5580cc8d0 0x7fb55815bb80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb548013e90 tx=0x7fb5480047c0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.133+0000 7fb55d7fa700 1 -- 172.21.15.83:0/898805554 shutdown_connections 2023-11-26T07:11:35.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.133+0000 7fb55d7fa700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb54006cc70 0x7fb54006f130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.133+0000 7fb55d7fa700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb55814a820 0x7fb55815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.133+0000 7fb55d7fa700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb55814a440 0x7fb558147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.133+0000 7fb55d7fa700 1 --2- 172.21.15.83:0/898805554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5580cc8d0 0x7fb55815bb80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.133+0000 7fb55d7fa700 1 -- 172.21.15.83:0/898805554 >> 172.21.15.83:0/898805554 conn(0x7fb5580cfa30 msgr2=0x7fb55816ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:35.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.133+0000 7fb55d7fa700 1 -- 172.21.15.83:0/898805554 shutdown_connections 2023-11-26T07:11:35.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.134+0000 7fb55d7fa700 1 -- 172.21.15.83:0/898805554 wait complete. 2023-11-26T07:11:35.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.135+0000 7febf5ce3700 1 -- 172.21.15.83:0/562318360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7febd4005590 con 0x7febf0071be0 2023-11-26T07:11:35.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.135+0000 7febdffff700 1 -- 172.21.15.83:0/562318360 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7febe8017020 con 0x7febf0071be0 2023-11-26T07:11:35.137 INFO:teuthology.orchestra.run.smithi083.stdout:55834574855 2023-11-26T07:11:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 -- 172.21.15.83:0/562318360 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7febd806b130 msgr2=0x7febd806d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7febd806b130 0x7febd806d5f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7febe0006fd0 tx=0x7febe0006bc0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 -- 172.21.15.83:0/562318360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf0071be0 msgr2=0x7febf013cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf0071be0 0x7febf013cfc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7febf00737d0 tx=0x7febe8004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 -- 172.21.15.83:0/562318360 shutdown_connections 2023-11-26T07:11:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febf015d610 0x7febf0153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febf0071be0 0x7febf013cfc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7febd806b130 0x7febd806d5f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 --2- 172.21.15.83:0/562318360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febf00d0520 0x7febf00ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:35.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 -- 172.21.15.83:0/562318360 >> 172.21.15.83:0/562318360 conn(0x7febf00cfa30 msgr2=0x7febf015c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:35.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 -- 172.21.15.83:0/562318360 shutdown_connections 2023-11-26T07:11:35.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:35.137+0000 7febf5ce3700 1 -- 172.21.15.83:0/562318360 wait complete. 2023-11-26T07:11:35.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2023-11-26T07:11:35.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-11-26T07:11:36.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-26T07:11:36.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-26T07:11:36.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-26T07:11:36.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-26T07:11:36.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-26T07:11:36.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-26T07:11:36.150 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-26T07:11:36.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-26T07:11:36.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.212+0000 7fef95c9f700 1 Processor -- start 2023-11-26T07:11:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.212+0000 7fef95c9f700 1 -- start start 2023-11-26T07:11:36.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.213+0000 7fef95c9f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fef900b4260 0x7fef900714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.213+0000 7fef95c9f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef90071a10 0x7fef90140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.213+0000 7fef95c9f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef900720e0 con 0x7fef9015d6a0 2023-11-26T07:11:36.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.213+0000 7fef95c9f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef90072d60 con 0x7fef90071a10 2023-11-26T07:11:36.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.213+0000 7fef95c9f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9006de80 con 0x7fef900b4260 2023-11-26T07:11:36.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.213+0000 7fef8ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef90071a10 0x7fef90140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.214+0000 7fef8ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef90071a10 0x7fef90140fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34452/0 (socket says 172.21.15.83:34452) 2023-11-26T07:11:36.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.214+0000 7fef8ffff700 1 -- 172.21.15.83:0/1153599065 learned_addr learned my addr 172.21.15.83:0/1153599065 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.214+0000 7fef8effd700 1 --2- 172.21.15.83:0/1153599065 >> v2:172.21.15.203:3300/0 conn(0x7fef900b4260 0x7fef900714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.214+0000 7fef8ffff700 1 -- 172.21.15.83:0/1153599065 >> v2:172.21.15.203:3300/0 conn(0x7fef900b4260 msgr2=0x7fef900714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:36.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.214+0000 7fef8ffff700 1 --2- 172.21.15.83:0/1153599065 >> v2:172.21.15.203:3300/0 conn(0x7fef900b4260 0x7fef900714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.214+0000 7fef8ffff700 1 -- 172.21.15.83:0/1153599065 >> v1:172.21.15.203:6789/0 conn(0x7fef9015d6a0 legacy=0x7fef900b3b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-26T07:11:36.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.214+0000 7fef8ffff700 1 -- 172.21.15.83:0/1153599065 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900723c0 con 0x7fef90071a10 2023-11-26T07:11:36.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.214+0000 7fef8ffff700 1 --2- 172.21.15.83:0/1153599065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef90071a10 0x7fef90140fd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fef90062350 tx=0x7fef8800d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab90aaf29f8b88c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.218+0000 7fef8e7fc700 1 -- 172.21.15.83:0/1153599065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88010070 con 0x7fef90071a10 2023-11-26T07:11:36.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.218+0000 7fef8e7fc700 1 -- 172.21.15.83:0/1153599065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8800dca0 con 0x7fef90071a10 2023-11-26T07:11:36.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.218+0000 7fef8e7fc700 1 -- 172.21.15.83:0/1153599065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88007d50 con 0x7fef90071a10 2023-11-26T07:11:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.219+0000 7fef95c9f700 1 -- 172.21.15.83:0/1153599065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef90071a10 msgr2=0x7fef90140fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.219+0000 7fef95c9f700 1 --2- 172.21.15.83:0/1153599065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef90071a10 0x7fef90140fd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fef90062350 tx=0x7fef8800d650 comp rx=0 tx=0).stop 2023-11-26T07:11:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.220+0000 7fef95c9f700 1 -- 172.21.15.83:0/1153599065 shutdown_connections 2023-11-26T07:11:36.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.220+0000 7fef95c9f700 1 --2- 172.21.15.83:0/1153599065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef90071a10 0x7fef90140fd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.220+0000 7fef95c9f700 1 --2- 172.21.15.83:0/1153599065 >> v2:172.21.15.203:3300/0 conn(0x7fef900b4260 0x7fef900714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.220+0000 7fef95c9f700 1 -- 172.21.15.83:0/1153599065 >> 172.21.15.83:0/1153599065 conn(0x7fef900cfa30 msgr2=0x7fef900cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.220+0000 7fef95c9f700 1 -- 172.21.15.83:0/1153599065 shutdown_connections 2023-11-26T07:11:36.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.220+0000 7fef95c9f700 1 -- 172.21.15.83:0/1153599065 wait complete. 2023-11-26T07:11:36.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.221+0000 7fef95c9f700 1 Processor -- start 2023-11-26T07:11:36.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.221+0000 7fef95c9f700 1 -- start start 2023-11-26T07:11:36.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.222+0000 7fef95c9f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef90071a10 0x7fef901404f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.222+0000 7fef95c9f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef900b4260 0x7fef901539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.222+0000 7fef95c9f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9015d6a0 0x7fef901e3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.222+0000 7fef95c9f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901fc610 con 0x7fef9015d6a0 2023-11-26T07:11:36.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.222+0000 7fef95c9f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901541b0 con 0x7fef90071a10 2023-11-26T07:11:36.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.222+0000 7fef95c9f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef90140a30 con 0x7fef900b4260 2023-11-26T07:11:36.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.222+0000 7fef8ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9015d6a0 0x7fef901e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.222+0000 7fef8ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9015d6a0 0x7fef901e3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34456/0 (socket says 172.21.15.83:34456) 2023-11-26T07:11:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.222+0000 7fef8ffff700 1 -- 172.21.15.83:0/1320143174 learned_addr learned my addr 172.21.15.83:0/1320143174 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.223+0000 7fef8ffff700 1 -- 172.21.15.83:0/1320143174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef900b4260 msgr2=0x7fef901539d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-26T07:11:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.223+0000 7fef8ffff700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef900b4260 0x7fef901539d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.223+0000 7fef8ffff700 1 -- 172.21.15.83:0/1320143174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef90071a10 msgr2=0x7fef901404f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.223+0000 7fef8ffff700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef90071a10 0x7fef901404f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.223+0000 7fef8ffff700 1 -- 172.21.15.83:0/1320143174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8800d050 con 0x7fef9015d6a0 2023-11-26T07:11:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.223+0000 7fef8ffff700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9015d6a0 0x7fef901e3840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fef8800d690 tx=0x7fef88004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.225+0000 7fef77fff700 1 -- 172.21.15.83:0/1320143174 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88004450 con 0x7fef9015d6a0 2023-11-26T07:11:36.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.225+0000 7fef77fff700 1 -- 172.21.15.83:0/1320143174 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef880045f0 con 0x7fef9015d6a0 2023-11-26T07:11:36.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.225+0000 7fef77fff700 1 -- 172.21.15.83:0/1320143174 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88008030 con 0x7fef9015d6a0 2023-11-26T07:11:36.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.226+0000 7fef95c9f700 1 -- 172.21.15.83:0/1320143174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef90153f10 con 0x7fef9015d6a0 2023-11-26T07:11:36.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.226+0000 7fef95c9f700 1 -- 172.21.15.83:0/1320143174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef901fbb30 con 0x7fef9015d6a0 2023-11-26T07:11:36.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.228+0000 7fef77fff700 1 -- 172.21.15.83:0/1320143174 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef88008190 con 0x7fef9015d6a0 2023-11-26T07:11:36.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.229+0000 7fef95c9f700 1 -- 172.21.15.83:0/1320143174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef700052e0 con 0x7fef9015d6a0 2023-11-26T07:11:36.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.229+0000 7fef77fff700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06afa0 0x7fef7c06d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.229+0000 7fef77fff700 1 -- 172.21.15.83:0/1320143174 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef8808ab20 con 0x7fef9015d6a0 2023-11-26T07:11:36.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.233+0000 7fef8f7fe700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06afa0 0x7fef7c06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.233+0000 7fef8f7fe700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06afa0 0x7fef7c06d460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fef80006fd0 tx=0x7fef80006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.239+0000 7fef77fff700 1 -- 172.21.15.83:0/1320143174 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef88055e10 con 0x7fef9015d6a0 2023-11-26T07:11:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.249+0000 7f474e01b700 1 Processor -- start 2023-11-26T07:11:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.249+0000 7f474e01b700 1 -- start start 2023-11-26T07:11:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.250+0000 7f474e01b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4748164a90 0x7f474806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.250+0000 7f474e01b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f474806d4d0 0x7f474816bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.250+0000 7f474e01b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f474806ced0 con 0x7f47480b3b50 2023-11-26T07:11:36.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.250+0000 7f474e01b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474806da70 con 0x7f474806d4d0 2023-11-26T07:11:36.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.250+0000 7f474e01b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47481634e0 con 0x7f4748164a90 2023-11-26T07:11:36.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.251+0000 7f4747fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f474806d4d0 0x7f474816bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.251+0000 7f4746ffd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4748164a90 0x7f474806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.251+0000 7f4746ffd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4748164a90 0x7f474806eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44922/0 (socket says 172.21.15.83:44922) 2023-11-26T07:11:36.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.251+0000 7f4746ffd700 1 -- 172.21.15.83:0/1189743067 learned_addr learned my addr 172.21.15.83:0/1189743067 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.252+0000 7f4747fff700 1 -- 172.21.15.83:0/1189743067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4748164a90 msgr2=0x7f474806eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:36.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.252+0000 7f4747fff700 1 --2- 172.21.15.83:0/1189743067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4748164a90 0x7f474806eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.252+0000 7f4747fff700 1 -- 172.21.15.83:0/1189743067 >> v1:172.21.15.83:6789/0 conn(0x7f47480b3b50 legacy=0x7f47480b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.252+0000 7f4747fff700 1 -- 172.21.15.83:0/1189743067 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47481637c0 con 0x7f474806d4d0 2023-11-26T07:11:36.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.253+0000 7f4747fff700 1 --2- 172.21.15.83:0/1189743067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474806d4d0 0x7f474816bee0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4738009840 tx=0x7f4738009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac51fd1e443c429e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.253+0000 7f47467fc700 1 -- 172.21.15.83:0/1189743067 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738004230 con 0x7f474806d4d0 2023-11-26T07:11:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.254+0000 7f47467fc700 1 -- 172.21.15.83:0/1189743067 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4738004390 con 0x7f474806d4d0 2023-11-26T07:11:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.254+0000 7f47467fc700 1 -- 172.21.15.83:0/1189743067 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47380036a0 con 0x7f474806d4d0 2023-11-26T07:11:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.254+0000 7f474e01b700 1 -- 172.21.15.83:0/1189743067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474806d4d0 msgr2=0x7f474816bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.254+0000 7f474e01b700 1 --2- 172.21.15.83:0/1189743067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474806d4d0 0x7f474816bee0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4738009840 tx=0x7f4738009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.255+0000 7f474e01b700 1 -- 172.21.15.83:0/1189743067 shutdown_connections 2023-11-26T07:11:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.255+0000 7f474e01b700 1 --2- 172.21.15.83:0/1189743067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474806d4d0 0x7f474816bee0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.255+0000 7f474e01b700 1 --2- 172.21.15.83:0/1189743067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4748164a90 0x7f474806eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.255+0000 7f474e01b700 1 -- 172.21.15.83:0/1189743067 >> 172.21.15.83:0/1189743067 conn(0x7f47480cfa30 msgr2=0x7f47480cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.255+0000 7f474e01b700 1 -- 172.21.15.83:0/1189743067 shutdown_connections 2023-11-26T07:11:36.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.255+0000 7f474e01b700 1 -- 172.21.15.83:0/1189743067 wait complete. 2023-11-26T07:11:36.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.257+0000 7f474e01b700 1 Processor -- start 2023-11-26T07:11:36.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.257+0000 7f474e01b700 1 -- start start 2023-11-26T07:11:36.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.258+0000 7f474e01b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474806d4d0 0x7f4748161b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.258+0000 7f474e01b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47480b3b50 0x7f474815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.258+0000 7f474e01b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4748164a90 0x7f47481f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.258+0000 7f474e01b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474815eeb0 con 0x7f47480b3b50 2023-11-26T07:11:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.258+0000 7f474e01b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474815f100 con 0x7f474806d4d0 2023-11-26T07:11:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.258+0000 7f474e01b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474815f3e0 con 0x7f4748164a90 2023-11-26T07:11:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.258+0000 7f4746ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47480b3b50 0x7f474815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.258+0000 7f4746ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47480b3b50 0x7f474815c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34468/0 (socket says 172.21.15.83:34468) 2023-11-26T07:11:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.258+0000 7f4746ffd700 1 -- 172.21.15.83:0/1914197129 learned_addr learned my addr 172.21.15.83:0/1914197129 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.259+0000 7f4746ffd700 1 -- 172.21.15.83:0/1914197129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4748164a90 msgr2=0x7f47481f2840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-26T07:11:36.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.259+0000 7f4746ffd700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4748164a90 0x7f47481f2840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.259+0000 7f4746ffd700 1 -- 172.21.15.83:0/1914197129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474806d4d0 msgr2=0x7f4748161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.259+0000 7f4746ffd700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474806d4d0 0x7f4748161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.259+0000 7f4746ffd700 1 -- 172.21.15.83:0/1914197129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47380094a0 con 0x7f47480b3b50 2023-11-26T07:11:36.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.259+0000 7f4746ffd700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47480b3b50 0x7f474815c630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f474000eed0 tx=0x7f474000b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.259+0000 7f472ffff700 1 -- 172.21.15.83:0/1914197129 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740010040 con 0x7f47480b3b50 2023-11-26T07:11:36.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.260+0000 7f472ffff700 1 -- 172.21.15.83:0/1914197129 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474000b980 con 0x7f47480b3b50 2023-11-26T07:11:36.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.260+0000 7f472ffff700 1 -- 172.21.15.83:0/1914197129 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474000bd10 con 0x7f47480b3b50 2023-11-26T07:11:36.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.260+0000 7f474e01b700 1 -- 172.21.15.83:0/1914197129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474815f6c0 con 0x7f47480b3b50 2023-11-26T07:11:36.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.260+0000 7f474e01b700 1 -- 172.21.15.83:0/1914197129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47481620c0 con 0x7f47480b3b50 2023-11-26T07:11:36.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.262+0000 7f474e01b700 1 -- 172.21.15.83:0/1914197129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47480751e0 con 0x7f47480b3b50 2023-11-26T07:11:36.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.262+0000 7f472ffff700 1 -- 172.21.15.83:0/1914197129 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f474001d410 con 0x7f47480b3b50 2023-11-26T07:11:36.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.263+0000 7f472ffff700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f473406af20 0x7f473406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.263+0000 7f472ffff700 1 -- 172.21.15.83:0/1914197129 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47400894b0 con 0x7f47480b3b50 2023-11-26T07:11:36.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.263+0000 7fedb8167700 1 Processor -- start 2023-11-26T07:11:36.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.263+0000 7f47477fe700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f473406af20 0x7f473406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.264+0000 7f47477fe700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f473406af20 0x7f473406d3e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f473c0017a0 tx=0x7f473c007a40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.264+0000 7fedb8167700 1 -- start start 2023-11-26T07:11:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.265+0000 7fedb8167700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedb00746a0 0x7fedb016d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.265+0000 7fedb8167700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb01667f0 con 0x7fedb014b250 2023-11-26T07:11:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.265+0000 7fedb8167700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb006ec10 con 0x7fedb006c620 2023-11-26T07:11:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.265+0000 7fedb8167700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb0165f30 con 0x7fedb00746a0 2023-11-26T07:11:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.266+0000 7fedb5702700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fedb006c620 0x7fedb0073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:39132/0 (socket says 172.21.15.83:39132) 2023-11-26T07:11:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.266+0000 7fedb5702700 1 -- 172.21.15.83:0/829038988 learned_addr learned my addr 172.21.15.83:0/829038988 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.266+0000 7fedb6704700 1 --2- 172.21.15.83:0/829038988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedb00746a0 0x7fedb016d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.266+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068002711 0 0) 0x7fedb006ec10 con 0x7fedb006c620 2023-11-26T07:11:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.267+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed9c0035b0 con 0x7fedb006c620 2023-11-26T07:11:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2216630468 0 0) 0x7fed9c0035b0 con 0x7fedb006c620 2023-11-26T07:11:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedb00746a0 msgr2=0x7fedb016d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 --2- 172.21.15.83:0/829038988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedb00746a0 0x7fedb016d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 >> v1:172.21.15.83:6789/0 conn(0x7fedb014b250 legacy=0x7fedb00b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb01661a0 con 0x7fedb006c620 2023-11-26T07:11:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fedac0016b0 con 0x7fedb006c620 2023-11-26T07:11:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 >> v1:172.21.15.203:6789/0 conn(0x7fedb006c620 legacy=0x7fedb0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 --2- 172.21.15.83:0/829038988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 0x7fed9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 --2- 172.21.15.83:0/829038988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed9c004850 0x7fed9c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 --2- 172.21.15.83:0/829038988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed9c005260 0x7fed9c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb0165f30 con 0x7fed9c003a00 2023-11-26T07:11:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c00c190 con 0x7fed9c005260 2023-11-26T07:11:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c00c350 con 0x7fed9c004850 2023-11-26T07:11:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.268+0000 7fedb5702700 1 --2- 172.21.15.83:0/829038988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed9c004850 0x7fed9c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.269+0000 7fedb5f03700 1 --2- 172.21.15.83:0/829038988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 0x7fed9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.269+0000 7fedb5f03700 1 -- 172.21.15.83:0/829038988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed9c004850 msgr2=0x7fed9c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.269+0000 7fedb5f03700 1 --2- 172.21.15.83:0/829038988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed9c004850 0x7fed9c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.269+0000 7fedb5f03700 1 -- 172.21.15.83:0/829038988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed9c005260 msgr2=0x7fed9c00ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:11:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.269+0000 7fedb5f03700 1 --2- 172.21.15.83:0/829038988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed9c005260 0x7fed9c00ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.269+0000 7fedb5f03700 1 -- 172.21.15.83:0/829038988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb01661a0 con 0x7fed9c003a00 2023-11-26T07:11:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.269+0000 7fedb5f03700 1 --2- 172.21.15.83:0/829038988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 0x7fed9c003e20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feda00017a0 tx=0x7fed9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72d84031db86a8a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.271+0000 7f472ffff700 1 -- 172.21.15.83:0/1914197129 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47400546d0 con 0x7f47480b3b50 2023-11-26T07:11:36.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.272+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda00052d0 con 0x7fed9c003a00 2023-11-26T07:11:36.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.272+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda0005430 con 0x7fed9c003a00 2023-11-26T07:11:36.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.272+0000 7fedb4f01700 1 -- 172.21.15.83:0/829038988 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0003880 con 0x7fed9c003a00 2023-11-26T07:11:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.273+0000 7fedb8167700 1 -- 172.21.15.83:0/829038988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 msgr2=0x7fed9c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.273+0000 7fedb8167700 1 --2- 172.21.15.83:0/829038988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 0x7fed9c003e20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feda00017a0 tx=0x7fed9c005880 comp rx=0 tx=0).stop 2023-11-26T07:11:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.273+0000 7fedb6704700 1 -- 172.21.15.83:0/829038988 reap_dead start 2023-11-26T07:11:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.273+0000 7fedb8167700 1 -- 172.21.15.83:0/829038988 shutdown_connections 2023-11-26T07:11:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.273+0000 7fedb8167700 1 -- 172.21.15.83:0/829038988 >> 172.21.15.83:0/829038988 conn(0x7fedb00d0ac0 msgr2=0x7fedb00d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.273+0000 7fedb8167700 1 -- 172.21.15.83:0/829038988 shutdown_connections 2023-11-26T07:11:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.273+0000 7fedb8167700 1 -- 172.21.15.83:0/829038988 wait complete. 2023-11-26T07:11:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.275+0000 7fedb8167700 1 Processor -- start 2023-11-26T07:11:36.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.275+0000 7fedb8167700 1 -- start start 2023-11-26T07:11:36.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.276+0000 7fedb8167700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 0x7fedb01403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.276+0000 7fedb8167700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedb0141320 0x7fedb01417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.276+0000 7fedb8167700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedb00b1890 0x7fedb00b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.276+0000 7fedb8167700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb00b2260 con 0x7fed9c003a00 2023-11-26T07:11:36.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.276+0000 7fedb8167700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb01482e0 con 0x7fedb00b1890 2023-11-26T07:11:36.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.276+0000 7fedb8167700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb01485c0 con 0x7fedb0141320 2023-11-26T07:11:36.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.277+0000 7fedb5f03700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 0x7fedb01403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.277+0000 7fedb5f03700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 0x7fedb01403a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34484/0 (socket says 172.21.15.83:34484) 2023-11-26T07:11:36.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.277+0000 7fedb5f03700 1 -- 172.21.15.83:0/1851040021 learned_addr learned my addr 172.21.15.83:0/1851040021 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.277+0000 7fedb5f03700 1 -- 172.21.15.83:0/1851040021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedb0141320 msgr2=0x7fedb01417a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:36.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.277+0000 7fedb5f03700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedb0141320 0x7fedb01417a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.277+0000 7fedb5f03700 1 -- 172.21.15.83:0/1851040021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedb00b1890 msgr2=0x7fedb00b1d20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.277+0000 7fedb5f03700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedb00b1890 0x7fedb00b1d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.277+0000 7fedb5f03700 1 -- 172.21.15.83:0/1851040021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb0166e00 con 0x7fed9c003a00 2023-11-26T07:11:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.277+0000 7fedb5f03700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 0x7fedb01403a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feda0001d90 tx=0x7feda0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.278+0000 7feda67fc700 1 -- 172.21.15.83:0/1851040021 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0007240 con 0x7fed9c003a00 2023-11-26T07:11:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.278+0000 7feda67fc700 1 -- 172.21.15.83:0/1851040021 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda00073a0 con 0x7fed9c003a00 2023-11-26T07:11:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.278+0000 7feda67fc700 1 -- 172.21.15.83:0/1851040021 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0007730 con 0x7fed9c003a00 2023-11-26T07:11:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.278+0000 7fedb8167700 1 -- 172.21.15.83:0/1851040021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb01488a0 con 0x7fed9c003a00 2023-11-26T07:11:36.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.278+0000 7fedb8167700 1 -- 172.21.15.83:0/1851040021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb0148e60 con 0x7fed9c003a00 2023-11-26T07:11:36.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.280+0000 7feda67fc700 1 -- 172.21.15.83:0/1851040021 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feda0007960 con 0x7fed9c003a00 2023-11-26T07:11:36.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.281+0000 7feda67fc700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fed9406af20 0x7fed9406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.281+0000 7feda67fc700 1 -- 172.21.15.83:0/1851040021 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feda008a040 con 0x7fed9c003a00 2023-11-26T07:11:36.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.283+0000 7fedb8167700 1 -- 172.21.15.83:0/1851040021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed980052e0 con 0x7fed9c003a00 2023-11-26T07:11:36.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.289+0000 7fedb5702700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fed9406af20 0x7fed9406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.290+0000 7fedb5702700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fed9406af20 0x7fed9406d3e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fedac001610 tx=0x7fedac000fb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.293+0000 7feda67fc700 1 -- 172.21.15.83:0/1851040021 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feda0055380 con 0x7fed9c003a00 2023-11-26T07:11:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.316+0000 7f5a880ca700 1 Processor -- start 2023-11-26T07:11:36.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.316+0000 7f5a880ca700 1 -- start start 2023-11-26T07:11:36.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.318+0000 7f5a880ca700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a8015d6a0 0x7f5a800b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.318+0000 7f5a880ca700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5a80071be0 0x7f5a80140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.318+0000 7f5a880ca700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a800720e0 con 0x7f5a800b4090 2023-11-26T07:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.318+0000 7f5a880ca700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a80072d60 con 0x7f5a8015d6a0 2023-11-26T07:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.318+0000 7f5a880ca700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8006de80 con 0x7f5a80071be0 2023-11-26T07:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.318+0000 7f5a85e66700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a8015d6a0 0x7f5a800b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.318+0000 7f5a85e66700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5a8015d6a0 0x7f5a800b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34492/0 (socket says 172.21.15.83:34492) 2023-11-26T07:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.318+0000 7f5a85e66700 1 -- 172.21.15.83:0/1752433389 learned_addr learned my addr 172.21.15.83:0/1752433389 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.318+0000 7f5a85e66700 1 -- 172.21.15.83:0/1752433389 >> v2:172.21.15.203:3300/0 conn(0x7f5a80071be0 msgr2=0x7f5a80140fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:11:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.318+0000 7f5a85e66700 1 --2- 172.21.15.83:0/1752433389 >> v2:172.21.15.203:3300/0 conn(0x7f5a80071be0 0x7f5a80140fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.319+0000 7f5a85e66700 1 -- 172.21.15.83:0/1752433389 >> v1:172.21.15.83:6789/0 conn(0x7f5a800b4090 legacy=0x7f5a800714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.319+0000 7f5a85e66700 1 -- 172.21.15.83:0/1752433389 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a800723c0 con 0x7f5a8015d6a0 2023-11-26T07:11:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.319+0000 7f5a85e66700 1 --2- 172.21.15.83:0/1752433389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a8015d6a0 0x7f5a800b3b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5a80070a50 tx=0x7f5a78009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ece5ba1c08b268d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.322+0000 7fc8e6a14700 1 Processor -- start 2023-11-26T07:11:36.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.322+0000 7fc8e6a14700 1 -- start start 2023-11-26T07:11:36.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.323+0000 7fc8e6a14700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8e00b3b50 0x7fc8e00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.323+0000 7fc8e6a14700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc8e0164a90 0x7fc8e016bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.323+0000 7fc8e6a14700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e006ced0 con 0x7fc8e006d4d0 2023-11-26T07:11:36.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.323+0000 7fc8e6a14700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e006da70 con 0x7fc8e00b3b50 2023-11-26T07:11:36.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.323+0000 7fc8e6a14700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e00b4470 con 0x7fc8e0164a90 2023-11-26T07:11:36.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.323+0000 7fc8e4fb1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc8e0164a90 0x7fc8e016bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.324+0000 7fc8e4fb1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc8e0164a90 0x7fc8e016bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:33952/0 (socket says 172.21.15.83:33952) 2023-11-26T07:11:36.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.324+0000 7fc8e4fb1700 1 -- 172.21.15.83:0/2040917460 learned_addr learned my addr 172.21.15.83:0/2040917460 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.324+0000 7fc8e4fb1700 1 -- 172.21.15.83:0/2040917460 >> v2:172.21.15.83:3300/0 conn(0x7fc8e00b3b50 msgr2=0x7fc8e00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:36.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.324+0000 7fc8e4fb1700 1 --2- 172.21.15.83:0/2040917460 >> v2:172.21.15.83:3300/0 conn(0x7fc8e00b3b50 0x7fc8e00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.324+0000 7fc8e4fb1700 1 -- 172.21.15.83:0/2040917460 >> v1:172.21.15.83:6789/0 conn(0x7fc8e006d4d0 legacy=0x7fc8e006eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:36.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.324+0000 7fc8e4fb1700 1 -- 172.21.15.83:0/2040917460 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e01634e0 con 0x7fc8e0164a90 2023-11-26T07:11:36.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.324+0000 7fc8e4fb1700 1 --2- 172.21.15.83:0/2040917460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e0164a90 0x7fc8e016bee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00cf9e0 tx=0x7fc8d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9dc706b87435401 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.328+0000 7f5a84e64700 1 -- 172.21.15.83:0/1752433389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78003c20 con 0x7f5a8015d6a0 2023-11-26T07:11:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.328+0000 7f5a84e64700 1 -- 172.21.15.83:0/1752433389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a78003d80 con 0x7f5a8015d6a0 2023-11-26T07:11:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.328+0000 7f5a84e64700 1 -- 172.21.15.83:0/1752433389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800c930 con 0x7f5a8015d6a0 2023-11-26T07:11:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.329+0000 7f5a880ca700 1 -- 172.21.15.83:0/1752433389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a8015d6a0 msgr2=0x7f5a800b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.329+0000 7f5a880ca700 1 --2- 172.21.15.83:0/1752433389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a8015d6a0 0x7f5a800b3b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5a80070a50 tx=0x7f5a78009730 comp rx=0 tx=0).stop 2023-11-26T07:11:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.329+0000 7fc8deffd700 1 -- 172.21.15.83:0/2040917460 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800e070 con 0x7fc8e0164a90 2023-11-26T07:11:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.329+0000 7fc8deffd700 1 -- 172.21.15.83:0/2040917460 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d8003a20 con 0x7fc8e0164a90 2023-11-26T07:11:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.329+0000 7fc8deffd700 1 -- 172.21.15.83:0/2040917460 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8003dd0 con 0x7fc8e0164a90 2023-11-26T07:11:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.329+0000 7f5a880ca700 1 -- 172.21.15.83:0/1752433389 shutdown_connections 2023-11-26T07:11:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.329+0000 7f5a880ca700 1 --2- 172.21.15.83:0/1752433389 >> v2:172.21.15.203:3300/0 conn(0x7f5a80071be0 0x7f5a80140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.329+0000 7f5a880ca700 1 --2- 172.21.15.83:0/1752433389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a8015d6a0 0x7f5a800b3b50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.329+0000 7f5a880ca700 1 -- 172.21.15.83:0/1752433389 >> 172.21.15.83:0/1752433389 conn(0x7f5a800cfa30 msgr2=0x7f5a800cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.329+0000 7f5a880ca700 1 -- 172.21.15.83:0/1752433389 shutdown_connections 2023-11-26T07:11:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.329+0000 7f5a880ca700 1 -- 172.21.15.83:0/1752433389 wait complete. 2023-11-26T07:11:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.330+0000 7fc8e6a14700 1 -- 172.21.15.83:0/2040917460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e0164a90 msgr2=0x7fc8e016bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.330+0000 7fc8e6a14700 1 --2- 172.21.15.83:0/2040917460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e0164a90 0x7fc8e016bee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00cf9e0 tx=0x7fc8d800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.331+0000 7f5a880ca700 1 Processor -- start 2023-11-26T07:11:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.331+0000 7fc8e6a14700 1 -- 172.21.15.83:0/2040917460 shutdown_connections 2023-11-26T07:11:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.331+0000 7fc8e6a14700 1 --2- 172.21.15.83:0/2040917460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e0164a90 0x7fc8e016bee0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.331+0000 7fc8e6a14700 1 --2- 172.21.15.83:0/2040917460 >> v2:172.21.15.83:3300/0 conn(0x7fc8e00b3b50 0x7fc8e00b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.331+0000 7fc8e6a14700 1 -- 172.21.15.83:0/2040917460 >> 172.21.15.83:0/2040917460 conn(0x7fc8e00cfa30 msgr2=0x7fc8e00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.331+0000 7f5a880ca700 1 -- start start 2023-11-26T07:11:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.332+0000 7f5a880ca700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80071be0 0x7f5a800b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.332+0000 7f5a880ca700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a800b4090 0x7f5a800ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.332+0000 7f5a880ca700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8015d6a0 0x7f5a800ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.332+0000 7f5a880ca700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a800cec30 con 0x7f5a80071be0 2023-11-26T07:11:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.332+0000 7f5a880ca700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a801592c0 con 0x7f5a8015d6a0 2023-11-26T07:11:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.332+0000 7f5a880ca700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a800cc550 con 0x7f5a800b4090 2023-11-26T07:11:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.332+0000 7fc8e6a14700 1 -- 172.21.15.83:0/2040917460 shutdown_connections 2023-11-26T07:11:36.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.333+0000 7fc8e6a14700 1 -- 172.21.15.83:0/2040917460 wait complete. 2023-11-26T07:11:36.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.334+0000 7fc8e6a14700 1 Processor -- start 2023-11-26T07:11:36.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.335+0000 7fc8e6a14700 1 -- start start 2023-11-26T07:11:36.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7fc8e6a14700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e006d4d0 0x7fc8e015c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7fc8e6a14700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8e00b3b50 0x7fc8e013d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7fc8e6a14700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8e0164a90 0x7fc8e013d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7fc8e6a14700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0149c20 con 0x7fc8e00b3b50 2023-11-26T07:11:36.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7fc8e6a14700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e013bc50 con 0x7fc8e006d4d0 2023-11-26T07:11:36.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7fc8e6a14700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e013ac30 con 0x7fc8e0164a90 2023-11-26T07:11:36.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7fc8dffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e006d4d0 0x7fc8e015c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7fc8dffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e006d4d0 0x7fc8e015c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:33962/0 (socket says 172.21.15.83:33962) 2023-11-26T07:11:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7fc8dffff700 1 -- 172.21.15.83:0/913879278 learned_addr learned my addr 172.21.15.83:0/913879278 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7fc8e4fb1700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8e0164a90 0x7fc8e013d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7f5a85665700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a800b4090 0x7f5a800ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7f5a85665700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a800b4090 0x7f5a800ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44964/0 (socket says 172.21.15.83:44964) 2023-11-26T07:11:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7f5a85665700 1 -- 172.21.15.83:0/2167816791 learned_addr learned my addr 172.21.15.83:0/2167816791 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.336+0000 7f5a86667700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8015d6a0 0x7f5a800ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7f5a85665700 1 -- 172.21.15.83:0/2167816791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8015d6a0 msgr2=0x7f5a800ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7f5a85665700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8015d6a0 0x7f5a800ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7f5a85665700 1 -- 172.21.15.83:0/2167816791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80071be0 msgr2=0x7f5a800b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7f5a85665700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80071be0 0x7f5a800b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7f5a85665700 1 -- 172.21.15.83:0/2167816791 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7800c3d0 con 0x7f5a800b4090 2023-11-26T07:11:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7fc8dffff700 1 -- 172.21.15.83:0/913879278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8e0164a90 msgr2=0x7fc8e013d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7fc8dffff700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8e0164a90 0x7fc8e013d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7fc8dffff700 1 -- 172.21.15.83:0/913879278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8e00b3b50 msgr2=0x7fc8e013d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7fc8dffff700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8e00b3b50 0x7fc8e013d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7fc8dffff700 1 -- 172.21.15.83:0/913879278 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d8009b30 con 0x7fc8e006d4d0 2023-11-26T07:11:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7f5a85665700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a800b4090 0x7f5a800ce0c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00c4a0 tx=0x7f5a7c00c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7f5a767fc700 1 -- 172.21.15.83:0/2167816791 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c0054e0 con 0x7f5a800b4090 2023-11-26T07:11:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7f5a767fc700 1 -- 172.21.15.83:0/2167816791 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7c003680 con 0x7f5a800b4090 2023-11-26T07:11:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7f5a880ca700 1 -- 172.21.15.83:0/2167816791 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a800cc830 con 0x7f5a800b4090 2023-11-26T07:11:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7f5a767fc700 1 -- 172.21.15.83:0/2167816791 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c003a10 con 0x7f5a800b4090 2023-11-26T07:11:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.337+0000 7f5a880ca700 1 -- 172.21.15.83:0/2167816791 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a800ccdf0 con 0x7f5a800b4090 2023-11-26T07:11:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.339+0000 7f5a767fc700 1 -- 172.21.15.83:0/2167816791 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a7c003b70 con 0x7f5a800b4090 2023-11-26T07:11:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.339+0000 7f5a767fc700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5a6406af20 0x7f5a6406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.339+0000 7f5a767fc700 1 -- 172.21.15.83:0/2167816791 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a7c088790 con 0x7f5a800b4090 2023-11-26T07:11:36.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.340+0000 7f5a85e66700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5a6406af20 0x7f5a6406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.340+0000 7f5a85e66700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5a6406af20 0x7f5a6406d3e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009770 tx=0x7f5a78013000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.340+0000 7fc8dffff700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e006d4d0 0x7fc8e015c600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000b530 tx=0x7fc8d000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.341+0000 7f5a880ca700 1 -- 172.21.15.83:0/2167816791 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a800cd010 con 0x7f5a800b4090 2023-11-26T07:11:36.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.341+0000 7f5f49104700 1 Processor -- start 2023-11-26T07:11:36.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.342+0000 7f5f49104700 1 -- start start 2023-11-26T07:11:36.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f49104700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5f440cc550 0x7f5f4415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f49104700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc930 0x7f5f4415e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f49104700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5f4415e550 0x7f5f440ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f49104700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4415ea00 con 0x7f5f4415e550 2023-11-26T07:11:36.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f49104700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4415de40 con 0x7f5f440cc550 2023-11-26T07:11:36.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f49104700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440cf6e0 con 0x7f5f440cc930 2023-11-26T07:11:36.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f4359e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5f4415e550 0x7f5f440ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f4259c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc930 0x7f5f4415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f4359e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5f4415e550 0x7f5f440ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34522/0 (socket says 172.21.15.83:34522) 2023-11-26T07:11:36.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f4259c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc930 0x7f5f4415e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:44978/0 (socket says 172.21.15.83:44978) 2023-11-26T07:11:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f4359e700 1 -- 172.21.15.83:0/723358278 learned_addr learned my addr 172.21.15.83:0/723358278 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.343+0000 7f5f42d9d700 1 --2- 172.21.15.83:0/723358278 >> v2:172.21.15.203:3300/0 conn(0x7f5f440cc550 0x7f5f4415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.344+0000 7f5f4359e700 1 -- 172.21.15.83:0/723358278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc930 msgr2=0x7f5f4415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.344+0000 7f5f4359e700 1 --2- 172.21.15.83:0/723358278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc930 0x7f5f4415e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.344+0000 7f5f4359e700 1 -- 172.21.15.83:0/723358278 >> v2:172.21.15.203:3300/0 conn(0x7f5f440cc550 msgr2=0x7f5f4415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.344+0000 7f5f4359e700 1 --2- 172.21.15.83:0/723358278 >> v2:172.21.15.203:3300/0 conn(0x7f5f440cc550 0x7f5f4415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.344+0000 7f5f4359e700 1 -- 172.21.15.83:0/723358278 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f44071870 con 0x7f5f4415e550 2023-11-26T07:11:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.344+0000 7fc8dcff9700 1 -- 172.21.15.83:0/913879278 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0011830 con 0x7fc8e006d4d0 2023-11-26T07:11:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.344+0000 7fc8dcff9700 1 -- 172.21.15.83:0/913879278 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d0011990 con 0x7fc8e006d4d0 2023-11-26T07:11:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.344+0000 7fc8dcff9700 1 -- 172.21.15.83:0/913879278 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0011d20 con 0x7fc8e006d4d0 2023-11-26T07:11:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.344+0000 7f5f4359e700 1 --2- 172.21.15.83:0/723358278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f4415e550 0x7f5f440ce970 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5f4406c900 tx=0x7f5f3c00bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2778591dfe654bd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.345+0000 7f5f41d9b700 1 -- 172.21.15.83:0/723358278 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00e210 con 0x7f5f4415e550 2023-11-26T07:11:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.345+0000 7f5f41d9b700 1 -- 172.21.15.83:0/723358278 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c00e370 con 0x7f5f4415e550 2023-11-26T07:11:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.345+0000 7fc8e6a14700 1 -- 172.21.15.83:0/913879278 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e013b9b0 con 0x7fc8e006d4d0 2023-11-26T07:11:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.345+0000 7f5f41d9b700 1 -- 172.21.15.83:0/723358278 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00e720 con 0x7f5f4415e550 2023-11-26T07:11:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.345+0000 7fc8e6a14700 1 -- 172.21.15.83:0/913879278 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e013b1b0 con 0x7fc8e006d4d0 2023-11-26T07:11:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.346+0000 7f5f49104700 1 -- 172.21.15.83:0/723358278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f4415e550 msgr2=0x7f5f440ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.346+0000 7f5f49104700 1 --2- 172.21.15.83:0/723358278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f4415e550 0x7f5f440ce970 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5f4406c900 tx=0x7f5f3c00bb50 comp rx=0 tx=0).stop 2023-11-26T07:11:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.346+0000 7f5f49104700 1 -- 172.21.15.83:0/723358278 shutdown_connections 2023-11-26T07:11:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.346+0000 7f5f49104700 1 --2- 172.21.15.83:0/723358278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f4415e550 0x7f5f440ce970 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.346+0000 7f5f49104700 1 --2- 172.21.15.83:0/723358278 >> v2:172.21.15.203:3300/0 conn(0x7f5f440cc550 0x7f5f4415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.346+0000 7f5f49104700 1 --2- 172.21.15.83:0/723358278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc930 0x7f5f4415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.346+0000 7f5f49104700 1 -- 172.21.15.83:0/723358278 >> 172.21.15.83:0/723358278 conn(0x7f5f4406cec0 msgr2=0x7f5f4406d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.346+0000 7f5a767fc700 1 -- 172.21.15.83:0/2167816791 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a7c053a60 con 0x7f5a800b4090 2023-11-26T07:11:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.347+0000 7fc8dcff9700 1 -- 172.21.15.83:0/913879278 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc8d001a400 con 0x7fc8e006d4d0 2023-11-26T07:11:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.348+0000 7fc8dcff9700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8cc06af20 0x7fc8cc06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.348+0000 7fc8dcff9700 1 -- 172.21.15.83:0/913879278 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8d00895c0 con 0x7fc8e006d4d0 2023-11-26T07:11:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.347+0000 7f5f49104700 1 -- 172.21.15.83:0/723358278 shutdown_connections 2023-11-26T07:11:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.348+0000 7fc8df7fe700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8cc06af20 0x7fc8cc06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.349+0000 7fc8df7fe700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8cc06af20 0x7fc8cc06d3e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc8e01506d0 tx=0x7fc8d4007a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.351+0000 7f5f49104700 1 -- 172.21.15.83:0/723358278 wait complete. 2023-11-26T07:11:36.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.353+0000 7f5f49104700 1 Processor -- start 2023-11-26T07:11:36.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.353+0000 7f5f49104700 1 -- start start 2023-11-26T07:11:36.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.354+0000 7f5f49104700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc550 0x7f5f4414dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.354+0000 7f5f49104700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f440cc930 0x7f5f44149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.354+0000 7f5f49104700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f4415e550 0x7f5f441507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.354+0000 7f5f49104700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440b42a0 con 0x7f5f440cc930 2023-11-26T07:11:36.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.354+0000 7f5f49104700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4414e520 con 0x7f5f4415e550 2023-11-26T07:11:36.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.354+0000 7f5f49104700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f44149fe0 con 0x7f5f440cc550 2023-11-26T07:11:36.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.355+0000 7f5f42d9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc550 0x7f5f4414dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.355+0000 7f5f4259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f440cc930 0x7f5f44149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.355+0000 7f5f4259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f440cc930 0x7f5f44149aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34528/0 (socket says 172.21.15.83:34528) 2023-11-26T07:11:36.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.355+0000 7f5f4259c700 1 -- 172.21.15.83:0/283811240 learned_addr learned my addr 172.21.15.83:0/283811240 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.355+0000 7f5f42d9d700 1 -- 172.21.15.83:0/283811240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f4415e550 msgr2=0x7f5f441507f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:36.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.355+0000 7f5f42d9d700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f4415e550 0x7f5f441507f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.355+0000 7f5f42d9d700 1 -- 172.21.15.83:0/283811240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f440cc930 msgr2=0x7f5f44149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.355+0000 7f5f42d9d700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f440cc930 0x7f5f44149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.355+0000 7f5f42d9d700 1 -- 172.21.15.83:0/283811240 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c00b580 con 0x7f5f440cc550 2023-11-26T07:11:36.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.355+0000 7f5f42d9d700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc550 0x7f5f4414dfe0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00bf30 tx=0x7f5f2c013390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.363+0000 7fc8e6a14700 1 -- 172.21.15.83:0/913879278 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e00751e0 con 0x7fc8e006d4d0 2023-11-26T07:11:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.363+0000 7f5f3b7fe700 1 -- 172.21.15.83:0/283811240 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c013c60 con 0x7f5f440cc550 2023-11-26T07:11:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.363+0000 7f5f3b7fe700 1 -- 172.21.15.83:0/283811240 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c013dc0 con 0x7f5f440cc550 2023-11-26T07:11:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.363+0000 7f5f3b7fe700 1 -- 172.21.15.83:0/283811240 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c010580 con 0x7f5f440cc550 2023-11-26T07:11:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.363+0000 7f5f49104700 1 -- 172.21.15.83:0/283811240 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4414ea20 con 0x7f5f440cc550 2023-11-26T07:11:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.363+0000 7f5f49104700 1 -- 172.21.15.83:0/283811240 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4414efe0 con 0x7f5f440cc550 2023-11-26T07:11:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.366+0000 7f5f3b7fe700 1 -- 172.21.15.83:0/283811240 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f2c0106e0 con 0x7f5f440cc550 2023-11-26T07:11:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.366+0000 7f5f3b7fe700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f2406af20 0x7f5f2406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.366+0000 7f5f49104700 1 -- 172.21.15.83:0/283811240 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f280052e0 con 0x7f5f440cc550 2023-11-26T07:11:36.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.366+0000 7f5f4259c700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f2406af20 0x7f5f2406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.367+0000 7f5f3b7fe700 1 -- 172.21.15.83:0/283811240 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f2c088c80 con 0x7f5f440cc550 2023-11-26T07:11:36.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.367+0000 7f5f4259c700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f2406af20 0x7f5f2406d3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5f440b0020 tx=0x7f5f34009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.372+0000 7f5f3b7fe700 1 -- 172.21.15.83:0/283811240 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f2c053da0 con 0x7f5f440cc550 2023-11-26T07:11:36.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.374+0000 7fc8dcff9700 1 -- 172.21.15.83:0/913879278 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8d0054750 con 0x7fc8e006d4d0 2023-11-26T07:11:36.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.388+0000 7fb7d570d700 1 Processor -- start 2023-11-26T07:11:36.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.388+0000 7fb7d570d700 1 -- start start 2023-11-26T07:11:36.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.389+0000 7fb7d570d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7d00b3b50 0x7fb7d00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.389+0000 7fb7d570d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d006d4d0 0x7fb7d006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.389+0000 7fb7d570d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb7d0163a80 0x7fb7d0163f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.389+0000 7fb7d570d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d006ced0 con 0x7fb7d00b3b50 2023-11-26T07:11:36.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.389+0000 7fb7d570d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d006da70 con 0x7fb7d0163a80 2023-11-26T07:11:36.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.389+0000 7fb7d570d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d00b4470 con 0x7fb7d006d4d0 2023-11-26T07:11:36.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.390+0000 7fb7ceffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7d00b3b50 0x7fb7d00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.390+0000 7fb7ceffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7d00b3b50 0x7fb7d00b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34538/0 (socket says 172.21.15.83:34538) 2023-11-26T07:11:36.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.390+0000 7fb7ceffd700 1 -- 172.21.15.83:0/3459217792 learned_addr learned my addr 172.21.15.83:0/3459217792 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.391+0000 7fb7ceffd700 1 -- 172.21.15.83:0/3459217792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d006d4d0 msgr2=0x7fb7d006eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:11:36.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.391+0000 7fb7ceffd700 1 --2- 172.21.15.83:0/3459217792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d006d4d0 0x7fb7d006eee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.391+0000 7fb7ceffd700 1 -- 172.21.15.83:0/3459217792 >> v2:172.21.15.203:3300/0 conn(0x7fb7d0163a80 msgr2=0x7fb7d0163f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:11:36.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.391+0000 7fb7ceffd700 1 --2- 172.21.15.83:0/3459217792 >> v2:172.21.15.203:3300/0 conn(0x7fb7d0163a80 0x7fb7d0163f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.391+0000 7fb7ceffd700 1 -- 172.21.15.83:0/3459217792 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d01634e0 con 0x7fb7d00b3b50 2023-11-26T07:11:36.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.391+0000 7fb7ceffd700 1 --2- 172.21.15.83:0/3459217792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d00b3b50 0x7fb7d00b3f30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00706f0 tx=0x7fb7c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72a0889d488937f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.392+0000 7fb7cdffb700 1 -- 172.21.15.83:0/3459217792 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c400b870 con 0x7fb7d00b3b50 2023-11-26T07:11:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.392+0000 7fb7cdffb700 1 -- 172.21.15.83:0/3459217792 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c400b9d0 con 0x7fb7d00b3b50 2023-11-26T07:11:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.392+0000 7fb7cdffb700 1 -- 172.21.15.83:0/3459217792 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c400bd60 con 0x7fb7d00b3b50 2023-11-26T07:11:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.393+0000 7fb7d570d700 1 -- 172.21.15.83:0/3459217792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d00b3b50 msgr2=0x7fb7d00b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.393+0000 7fb7d570d700 1 --2- 172.21.15.83:0/3459217792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d00b3b50 0x7fb7d00b3f30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00706f0 tx=0x7fb7c4009bd0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.394+0000 7fb7d570d700 1 -- 172.21.15.83:0/3459217792 shutdown_connections 2023-11-26T07:11:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.394+0000 7fb7d570d700 1 --2- 172.21.15.83:0/3459217792 >> v2:172.21.15.203:3300/0 conn(0x7fb7d0163a80 0x7fb7d0163f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.394+0000 7fb7d570d700 1 --2- 172.21.15.83:0/3459217792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d00b3b50 0x7fb7d00b3f30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.394+0000 7fb7d570d700 1 --2- 172.21.15.83:0/3459217792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d006d4d0 0x7fb7d006eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.394+0000 7fb7d570d700 1 -- 172.21.15.83:0/3459217792 >> 172.21.15.83:0/3459217792 conn(0x7fb7d00cfa30 msgr2=0x7fb7d00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.394+0000 7fb7d570d700 1 -- 172.21.15.83:0/3459217792 shutdown_connections 2023-11-26T07:11:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.394+0000 7fb7d570d700 1 -- 172.21.15.83:0/3459217792 wait complete. 2023-11-26T07:11:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.396+0000 7fb7d570d700 1 Processor -- start 2023-11-26T07:11:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.397+0000 7fb7d570d700 1 -- start start 2023-11-26T07:11:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.398+0000 7fb7d570d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d006d4d0 0x7fb7d00ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.398+0000 7fb7d570d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d00b3b50 0x7fb7d00cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.398+0000 7fb7d570d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d0163a80 0x7fb7d013f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.398+0000 7fb7d570d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d00cce90 con 0x7fb7d0163a80 2023-11-26T07:11:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.398+0000 7fb7d570d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d00cd170 con 0x7fb7d00b3b50 2023-11-26T07:11:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.398+0000 7fb7d570d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d00cd450 con 0x7fb7d006d4d0 2023-11-26T07:11:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.398+0000 7fb7cf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d0163a80 0x7fb7d013f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.398+0000 7fb7cf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d0163a80 0x7fb7d013f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34544/0 (socket says 172.21.15.83:34544) 2023-11-26T07:11:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.398+0000 7fb7cf7fe700 1 -- 172.21.15.83:0/2147774563 learned_addr learned my addr 172.21.15.83:0/2147774563 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.399+0000 7fb7cf7fe700 1 -- 172.21.15.83:0/2147774563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d006d4d0 msgr2=0x7fb7d00ce750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:36.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.399+0000 7fb7cf7fe700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d006d4d0 0x7fb7d00ce750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.399+0000 7fb7cf7fe700 1 -- 172.21.15.83:0/2147774563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d00b3b50 msgr2=0x7fb7d00cc440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:36.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.399+0000 7fb7cf7fe700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d00b3b50 0x7fb7d00cc440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.399+0000 7fb7cf7fe700 1 -- 172.21.15.83:0/2147774563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4009550 con 0x7fb7d0163a80 2023-11-26T07:11:36.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.399+0000 7fb7cf7fe700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d0163a80 0x7fb7d013f270 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800b780 tx=0x7fb7c800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.401+0000 7fb7bf7fe700 1 -- 172.21.15.83:0/2147774563 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800f840 con 0x7fb7d0163a80 2023-11-26T07:11:36.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.401+0000 7fb7bf7fe700 1 -- 172.21.15.83:0/2147774563 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c800f9a0 con 0x7fb7d0163a80 2023-11-26T07:11:36.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.402+0000 7fb7bf7fe700 1 -- 172.21.15.83:0/2147774563 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800fd50 con 0x7fb7d0163a80 2023-11-26T07:11:36.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.402+0000 7fb7d570d700 1 -- 172.21.15.83:0/2147774563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d00cd730 con 0x7fb7d0163a80 2023-11-26T07:11:36.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.402+0000 7fb7d570d700 1 -- 172.21.15.83:0/2147774563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d014ef70 con 0x7fb7d0163a80 2023-11-26T07:11:36.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.407+0000 7fb7bf7fe700 1 -- 172.21.15.83:0/2147774563 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb7c80203f0 con 0x7fb7d0163a80 2023-11-26T07:11:36.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.407+0000 7fb7bf7fe700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7b406afa0 0x7fb7b406d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.407+0000 7fb7bf7fe700 1 -- 172.21.15.83:0/2147774563 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7c8089910 con 0x7fb7d0163a80 2023-11-26T07:11:36.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.408+0000 7fb7d570d700 1 -- 172.21.15.83:0/2147774563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ac0052e0 con 0x7fb7d0163a80 2023-11-26T07:11:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.410+0000 7fb7ceffd700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7b406afa0 0x7fb7b406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.410+0000 7fb7ceffd700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7b406afa0 0x7fb7b406d460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009c10 tx=0x7fb7c4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.418+0000 7fb7bf7fe700 1 -- 172.21.15.83:0/2147774563 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7c8054a30 con 0x7fb7d0163a80 2023-11-26T07:11:36.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.467+0000 7ffa6056f700 1 Processor -- start 2023-11-26T07:11:36.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.467+0000 7ffa6056f700 1 -- start start 2023-11-26T07:11:36.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.469+0000 7ffa6056f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffa58164a90 0x7ffa5806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.469+0000 7ffa6056f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ffa5806d4d0 0x7ffa5816bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.469+0000 7ffa6056f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5806ced0 con 0x7ffa580b3b50 2023-11-26T07:11:36.471 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.469+0000 7ffa6056f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5806da70 con 0x7ffa58164a90 2023-11-26T07:11:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.469+0000 7ffa6056f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa581634e0 con 0x7ffa5806d4d0 2023-11-26T07:11:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.470+0000 7ffa5eb0c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ffa5806d4d0 0x7ffa5816bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.470+0000 7ffa5db0a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffa58164a90 0x7ffa5806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.470+0000 7ffa5db0a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffa58164a90 0x7ffa5806eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34552/0 (socket says 172.21.15.83:34552) 2023-11-26T07:11:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.470+0000 7ffa5db0a700 1 -- 172.21.15.83:0/3806468721 learned_addr learned my addr 172.21.15.83:0/3806468721 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.470+0000 7ffa5e30b700 1 --1- 172.21.15.83:0/3806468721 >> v1:172.21.15.203:6789/0 conn(0x7ffa580b3b50 0x7ffa580b3f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:39206/0 (socket says 172.21.15.83:39206) 2023-11-26T07:11:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.470+0000 7ffa5db0a700 1 -- 172.21.15.83:0/3806468721 >> v2:172.21.15.203:3300/0 conn(0x7ffa5806d4d0 msgr2=0x7ffa5816bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:36.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.470+0000 7ffa5db0a700 1 --2- 172.21.15.83:0/3806468721 >> v2:172.21.15.203:3300/0 conn(0x7ffa5806d4d0 0x7ffa5816bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.470+0000 7ffa5db0a700 1 -- 172.21.15.83:0/3806468721 >> v1:172.21.15.203:6789/0 conn(0x7ffa580b3b50 legacy=0x7ffa580b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.470+0000 7ffa5db0a700 1 -- 172.21.15.83:0/3806468721 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa581637c0 con 0x7ffa58164a90 2023-11-26T07:11:36.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.470+0000 7ffa5db0a700 1 --2- 172.21.15.83:0/3806468721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa58164a90 0x7ffa5806eee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa58070a50 tx=0x7ffa48009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e7e2b3dc0b05d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.471+0000 7ffa5d309700 1 -- 172.21.15.83:0/3806468721 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa48011830 con 0x7ffa58164a90 2023-11-26T07:11:36.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.471+0000 7ffa5d309700 1 -- 172.21.15.83:0/3806468721 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa48011990 con 0x7ffa58164a90 2023-11-26T07:11:36.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.471+0000 7ffa5d309700 1 -- 172.21.15.83:0/3806468721 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa48011d20 con 0x7ffa58164a90 2023-11-26T07:11:36.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.472+0000 7ffa6056f700 1 -- 172.21.15.83:0/3806468721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa58164a90 msgr2=0x7ffa5806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.472+0000 7ffa6056f700 1 --2- 172.21.15.83:0/3806468721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa58164a90 0x7ffa5806eee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa58070a50 tx=0x7ffa48009bd0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.473+0000 7ffa6056f700 1 -- 172.21.15.83:0/3806468721 shutdown_connections 2023-11-26T07:11:36.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.473+0000 7ffa6056f700 1 --2- 172.21.15.83:0/3806468721 >> v2:172.21.15.203:3300/0 conn(0x7ffa5806d4d0 0x7ffa5816bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.473+0000 7ffa6056f700 1 --2- 172.21.15.83:0/3806468721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa58164a90 0x7ffa5806eee0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.473+0000 7ffa6056f700 1 -- 172.21.15.83:0/3806468721 >> 172.21.15.83:0/3806468721 conn(0x7ffa580cfa30 msgr2=0x7ffa580cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.473+0000 7ffa6056f700 1 -- 172.21.15.83:0/3806468721 shutdown_connections 2023-11-26T07:11:36.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.473+0000 7ffa6056f700 1 -- 172.21.15.83:0/3806468721 wait complete. 2023-11-26T07:11:36.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.475+0000 7ffa6056f700 1 Processor -- start 2023-11-26T07:11:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.476+0000 7ffa6056f700 1 -- start start 2023-11-26T07:11:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.477+0000 7ffa6056f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa5806d4d0 0x7ffa580cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.477+0000 7ffa6056f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa580b3b50 0x7ffa581482d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.477+0000 7ffa6056f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa58164a90 0x7ffa5813e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.477+0000 7ffa6056f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58162480 con 0x7ffa5806d4d0 2023-11-26T07:11:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.477+0000 7ffa6056f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58148810 con 0x7ffa58164a90 2023-11-26T07:11:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.477+0000 7ffa6056f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5813e7c0 con 0x7ffa580b3b50 2023-11-26T07:11:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.477+0000 7ffa5e30b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa5806d4d0 0x7ffa580cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.477+0000 7ffa5e30b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa5806d4d0 0x7ffa580cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34554/0 (socket says 172.21.15.83:34554) 2023-11-26T07:11:36.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.477+0000 7ffa5e30b700 1 -- 172.21.15.83:0/2864957415 learned_addr learned my addr 172.21.15.83:0/2864957415 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.478+0000 7ffa5e30b700 1 -- 172.21.15.83:0/2864957415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa580b3b50 msgr2=0x7ffa581482d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.478+0000 7ffa5e30b700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa580b3b50 0x7ffa581482d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.478+0000 7ffa5e30b700 1 -- 172.21.15.83:0/2864957415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa58164a90 msgr2=0x7ffa5813e280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:11:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.478+0000 7ffa5e30b700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa58164a90 0x7ffa5813e280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.478+0000 7ffa5e30b700 1 -- 172.21.15.83:0/2864957415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa48009550 con 0x7ffa5806d4d0 2023-11-26T07:11:36.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.478+0000 7ffa5e30b700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa5806d4d0 0x7ffa580cc440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000cc50 tx=0x7ffa5000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.479+0000 7ffa4effd700 1 -- 172.21.15.83:0/2864957415 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5000ade0 con 0x7ffa5806d4d0 2023-11-26T07:11:36.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.479+0000 7ffa4effd700 1 -- 172.21.15.83:0/2864957415 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5000ce50 con 0x7ffa5806d4d0 2023-11-26T07:11:36.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.479+0000 7ffa4effd700 1 -- 172.21.15.83:0/2864957415 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50007270 con 0x7ffa5806d4d0 2023-11-26T07:11:36.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.479+0000 7ffa6056f700 1 -- 172.21.15.83:0/2864957415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa581621a0 con 0x7ffa5806d4d0 2023-11-26T07:11:36.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.479+0000 7ffa6056f700 1 -- 172.21.15.83:0/2864957415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5814a470 con 0x7ffa5806d4d0 2023-11-26T07:11:36.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.484+0000 7ffa4effd700 1 -- 172.21.15.83:0/2864957415 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa500073d0 con 0x7ffa5806d4d0 2023-11-26T07:11:36.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.485+0000 7ffa4effd700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffa4406b130 0x7ffa4406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:36.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.485+0000 7ffa4effd700 1 -- 172.21.15.83:0/2864957415 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa500893e0 con 0x7ffa5806d4d0 2023-11-26T07:11:36.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.486+0000 7ffa5db0a700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffa4406b130 0x7ffa4406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:36.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.486+0000 7ffa5db0a700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffa4406b130 0x7ffa4406d5f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffa58164f20 tx=0x7ffa48008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.487+0000 7ffa6056f700 1 -- 172.21.15.83:0/2864957415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa400052e0 con 0x7ffa5806d4d0 2023-11-26T07:11:36.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.497+0000 7ffa4effd700 1 -- 172.21.15.83:0/2864957415 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa50054520 con 0x7ffa5806d4d0 2023-11-26T07:11:36.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.635+0000 7fef95c9f700 1 -- 172.21.15.83:0/1320143174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fef70005590 con 0x7fef9015d6a0 2023-11-26T07:11:36.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.636+0000 7fef77fff700 1 -- 172.21.15.83:0/1320143174 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fef88025030 con 0x7fef9015d6a0 2023-11-26T07:11:36.640 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-26T07:11:36.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.640+0000 7fef95c9f700 1 -- 172.21.15.83:0/1320143174 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06afa0 msgr2=0x7fef7c06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.640+0000 7fef95c9f700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06afa0 0x7fef7c06d460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fef80006fd0 tx=0x7fef80006bc0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.640+0000 7fef95c9f700 1 -- 172.21.15.83:0/1320143174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9015d6a0 msgr2=0x7fef901e3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.640+0000 7fef95c9f700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9015d6a0 0x7fef901e3840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fef8800d690 tx=0x7fef88004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.640+0000 7fef95c9f700 1 -- 172.21.15.83:0/1320143174 shutdown_connections 2023-11-26T07:11:36.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.640+0000 7fef95c9f700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9015d6a0 0x7fef901e3840 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.640+0000 7fef95c9f700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef90071a10 0x7fef901404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.640+0000 7fef95c9f700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06afa0 0x7fef7c06d460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.640+0000 7fef95c9f700 1 --2- 172.21.15.83:0/1320143174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef900b4260 0x7fef901539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.640+0000 7fef95c9f700 1 -- 172.21.15.83:0/1320143174 >> 172.21.15.83:0/1320143174 conn(0x7fef900cfa30 msgr2=0x7fef900cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.641+0000 7fef95c9f700 1 -- 172.21.15.83:0/1320143174 shutdown_connections 2023-11-26T07:11:36.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.641+0000 7fef95c9f700 1 -- 172.21.15.83:0/1320143174 wait complete. 2023-11-26T07:11:36.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.2 2023-11-26T07:11:36.661 DEBUG:teuthology.parallel:result is None 2023-11-26T07:11:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.680+0000 7f474e01b700 1 -- 172.21.15.83:0/1914197129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f474815fbe0 con 0x7f47480b3b50 2023-11-26T07:11:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.681+0000 7f472ffff700 1 -- 172.21.15.83:0/1914197129 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f474815fbe0 con 0x7f47480b3b50 2023-11-26T07:11:36.682 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-26T07:11:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.685+0000 7f474e01b700 1 -- 172.21.15.83:0/1914197129 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f473406af20 msgr2=0x7f473406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.685+0000 7f474e01b700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f473406af20 0x7f473406d3e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f473c0017a0 tx=0x7f473c007a40 comp rx=0 tx=0).stop 2023-11-26T07:11:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.685+0000 7f474e01b700 1 -- 172.21.15.83:0/1914197129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47480b3b50 msgr2=0x7f474815c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.685+0000 7f474e01b700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47480b3b50 0x7f474815c630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f474000eed0 tx=0x7f474000b0b0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.685+0000 7f474e01b700 1 -- 172.21.15.83:0/1914197129 shutdown_connections 2023-11-26T07:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.685+0000 7f474e01b700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f473406af20 0x7f473406d3e0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f473c0017a0 tx=0x7f473c007a40 comp rx=0 tx=0).stop 2023-11-26T07:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.685+0000 7f474e01b700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4748164a90 0x7f47481f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.685+0000 7f474e01b700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474806d4d0 0x7f4748161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.685+0000 7f474e01b700 1 --2- 172.21.15.83:0/1914197129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47480b3b50 0x7f474815c630 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.685+0000 7f474e01b700 1 -- 172.21.15.83:0/1914197129 >> 172.21.15.83:0/1914197129 conn(0x7f47480cfa30 msgr2=0x7f47480cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.686+0000 7f474e01b700 1 -- 172.21.15.83:0/1914197129 shutdown_connections 2023-11-26T07:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.686+0000 7f474e01b700 1 -- 172.21.15.83:0/1914197129 wait complete. 2023-11-26T07:11:36.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.4 2023-11-26T07:11:36.705 DEBUG:teuthology.parallel:result is None 2023-11-26T07:11:36.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.704+0000 7fedb8167700 1 -- 172.21.15.83:0/1851040021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fed98005860 con 0x7fed9c003a00 2023-11-26T07:11:36.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.704+0000 7feda67fc700 1 -- 172.21.15.83:0/1851040021 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feda0017070 con 0x7fed9c003a00 2023-11-26T07:11:36.706 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-26T07:11:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.706+0000 7fedb8167700 1 -- 172.21.15.83:0/1851040021 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fed9406af20 msgr2=0x7fed9406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.706+0000 7fedb8167700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fed9406af20 0x7fed9406d3e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fedac001610 tx=0x7fedac000fb0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.706+0000 7fedb8167700 1 -- 172.21.15.83:0/1851040021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 msgr2=0x7fedb01403a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.706+0000 7fedb8167700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 0x7fedb01403a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feda0001d90 tx=0x7feda0001120 comp rx=0 tx=0).stop 2023-11-26T07:11:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.707+0000 7fedb8167700 1 -- 172.21.15.83:0/1851040021 shutdown_connections 2023-11-26T07:11:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.707+0000 7fedb8167700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedb00b1890 0x7fedb00b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.707+0000 7fedb8167700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed9c003a00 0x7fedb01403a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.707+0000 7fedb8167700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fed9406af20 0x7fed9406d3e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.707+0000 7fedb8167700 1 --2- 172.21.15.83:0/1851040021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedb0141320 0x7fedb01417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.707+0000 7fedb8167700 1 -- 172.21.15.83:0/1851040021 >> 172.21.15.83:0/1851040021 conn(0x7fedb00d0ac0 msgr2=0x7fedb006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.709+0000 7fedb8167700 1 -- 172.21.15.83:0/1851040021 shutdown_connections 2023-11-26T07:11:36.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.709+0000 7fedb8167700 1 -- 172.21.15.83:0/1851040021 wait complete. 2023-11-26T07:11:36.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.6 2023-11-26T07:11:36.722 DEBUG:teuthology.parallel:result is None 2023-11-26T07:11:36.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.728+0000 7fc8e6a14700 1 -- 172.21.15.83:0/913879278 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc8e0162200 con 0x7fc8e006d4d0 2023-11-26T07:11:36.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.728+0000 7fc8dcff9700 1 -- 172.21.15.83:0/913879278 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc8d0015030 con 0x7fc8e006d4d0 2023-11-26T07:11:36.730 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-26T07:11:36.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.730+0000 7fc8e6a14700 1 -- 172.21.15.83:0/913879278 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8cc06af20 msgr2=0x7fc8cc06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.730+0000 7fc8e6a14700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8cc06af20 0x7fc8cc06d3e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc8e01506d0 tx=0x7fc8d4007a20 comp rx=0 tx=0).stop 2023-11-26T07:11:36.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.730+0000 7fc8e6a14700 1 -- 172.21.15.83:0/913879278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e006d4d0 msgr2=0x7fc8e015c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.730+0000 7fc8e6a14700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e006d4d0 0x7fc8e015c600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000b530 tx=0x7fc8d000b840 comp rx=0 tx=0).stop 2023-11-26T07:11:36.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.730+0000 7fc8e6a14700 1 -- 172.21.15.83:0/913879278 shutdown_connections 2023-11-26T07:11:36.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.730+0000 7fc8e6a14700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8cc06af20 0x7fc8cc06d3e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.730+0000 7fc8e6a14700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8e0164a90 0x7fc8e013d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.730+0000 7fc8e6a14700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8e006d4d0 0x7fc8e015c600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.730+0000 7fc8e6a14700 1 --2- 172.21.15.83:0/913879278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8e00b3b50 0x7fc8e013d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.730+0000 7fc8e6a14700 1 -- 172.21.15.83:0/913879278 >> 172.21.15.83:0/913879278 conn(0x7fc8e00cfa30 msgr2=0x7fc8e00b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.731+0000 7fc8e6a14700 1 -- 172.21.15.83:0/913879278 shutdown_connections 2023-11-26T07:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.731+0000 7fc8e6a14700 1 -- 172.21.15.83:0/913879278 wait complete. 2023-11-26T07:11:36.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.734+0000 7f5a880ca700 1 -- 172.21.15.83:0/2167816791 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5a800751e0 con 0x7f5a800b4090 2023-11-26T07:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.735+0000 7f5a767fc700 1 -- 172.21.15.83:0/2167816791 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5a7c017020 con 0x7f5a800b4090 2023-11-26T07:11:36.736 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-26T07:11:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 -- 172.21.15.83:0/2167816791 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5a6406af20 msgr2=0x7f5a6406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5a6406af20 0x7f5a6406d3e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009770 tx=0x7f5a78013000 comp rx=0 tx=0).stop 2023-11-26T07:11:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 -- 172.21.15.83:0/2167816791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a800b4090 msgr2=0x7f5a800ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a800b4090 0x7f5a800ce0c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00c4a0 tx=0x7f5a7c00c7b0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 -- 172.21.15.83:0/2167816791 shutdown_connections 2023-11-26T07:11:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a8015d6a0 0x7f5a800ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80071be0 0x7f5a800b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5a6406af20 0x7f5a6406d3e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 --2- 172.21.15.83:0/2167816791 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a800b4090 0x7f5a800ce0c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 -- 172.21.15.83:0/2167816791 >> 172.21.15.83:0/2167816791 conn(0x7f5a800cfa30 msgr2=0x7f5a80141870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 -- 172.21.15.83:0/2167816791 shutdown_connections 2023-11-26T07:11:36.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.737+0000 7f5a880ca700 1 -- 172.21.15.83:0/2167816791 wait complete. 2023-11-26T07:11:36.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.0 2023-11-26T07:11:36.745 DEBUG:teuthology.parallel:result is None 2023-11-26T07:11:36.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.1 2023-11-26T07:11:36.752 DEBUG:teuthology.parallel:result is None 2023-11-26T07:11:36.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.761+0000 7f5f49104700 1 -- 172.21.15.83:0/283811240 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5f28005590 con 0x7f5f440cc550 2023-11-26T07:11:36.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.762+0000 7f5f3b7fe700 1 -- 172.21.15.83:0/283811240 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f2c019070 con 0x7f5f440cc550 2023-11-26T07:11:36.763 INFO:teuthology.orchestra.run.smithi083.stdout:55834574856 2023-11-26T07:11:36.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 -- 172.21.15.83:0/283811240 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f2406af20 msgr2=0x7f5f2406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f2406af20 0x7f5f2406d3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5f440b0020 tx=0x7f5f34009210 comp rx=0 tx=0).stop 2023-11-26T07:11:36.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 -- 172.21.15.83:0/283811240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc550 msgr2=0x7f5f4414dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc550 0x7f5f4414dfe0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00bf30 tx=0x7f5f2c013390 comp rx=0 tx=0).stop 2023-11-26T07:11:36.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 -- 172.21.15.83:0/283811240 shutdown_connections 2023-11-26T07:11:36.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f4415e550 0x7f5f441507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f2406af20 0x7f5f2406d3e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f440cc550 0x7f5f4414dfe0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 --2- 172.21.15.83:0/283811240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f440cc930 0x7f5f44149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 -- 172.21.15.83:0/283811240 >> 172.21.15.83:0/283811240 conn(0x7f5f4406cec0 msgr2=0x7f5f4406d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 -- 172.21.15.83:0/283811240 shutdown_connections 2023-11-26T07:11:36.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.764+0000 7f5f49104700 1 -- 172.21.15.83:0/283811240 wait complete. 2023-11-26T07:11:36.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.5 2023-11-26T07:11:36.778 DEBUG:teuthology.parallel:result is None 2023-11-26T07:11:36.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.802+0000 7fb7d570d700 1 -- 172.21.15.83:0/2147774563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb7ac005590 con 0x7fb7d0163a80 2023-11-26T07:11:36.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.802+0000 7fb7bf7fe700 1 -- 172.21.15.83:0/2147774563 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb7c8058050 con 0x7fb7d0163a80 2023-11-26T07:11:36.803 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-26T07:11:36.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 -- 172.21.15.83:0/2147774563 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7b406afa0 msgr2=0x7fb7b406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7b406afa0 0x7fb7b406d460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009c10 tx=0x7fb7c4008030 comp rx=0 tx=0).stop 2023-11-26T07:11:36.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 -- 172.21.15.83:0/2147774563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d0163a80 msgr2=0x7fb7d013f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d0163a80 0x7fb7d013f270 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800b780 tx=0x7fb7c800bb40 comp rx=0 tx=0).stop 2023-11-26T07:11:36.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 -- 172.21.15.83:0/2147774563 shutdown_connections 2023-11-26T07:11:36.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7d0163a80 0x7fb7d013f270 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7b406afa0 0x7fb7b406d460 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7d006d4d0 0x7fb7d00ce750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 --2- 172.21.15.83:0/2147774563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7d00b3b50 0x7fb7d00cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 -- 172.21.15.83:0/2147774563 >> 172.21.15.83:0/2147774563 conn(0x7fb7d00cfa30 msgr2=0x7fb7d0070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 -- 172.21.15.83:0/2147774563 shutdown_connections 2023-11-26T07:11:36.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.804+0000 7fb7d570d700 1 -- 172.21.15.83:0/2147774563 wait complete. 2023-11-26T07:11:36.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.7 2023-11-26T07:11:36.818 DEBUG:teuthology.parallel:result is None 2023-11-26T07:11:36.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.847+0000 7ffa6056f700 1 -- 172.21.15.83:0/2864957415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffa40005590 con 0x7ffa5806d4d0 2023-11-26T07:11:36.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.847+0000 7ffa4effd700 1 -- 172.21.15.83:0/2864957415 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffa50017070 con 0x7ffa5806d4d0 2023-11-26T07:11:36.849 INFO:teuthology.orchestra.run.smithi083.stdout:55834574857 2023-11-26T07:11:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.849+0000 7ffa6056f700 1 -- 172.21.15.83:0/2864957415 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffa4406b130 msgr2=0x7ffa4406d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.849+0000 7ffa6056f700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffa4406b130 0x7ffa4406d5f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffa58164f20 tx=0x7ffa48008030 comp rx=0 tx=0).stop 2023-11-26T07:11:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.849+0000 7ffa6056f700 1 -- 172.21.15.83:0/2864957415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa5806d4d0 msgr2=0x7ffa580cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.849+0000 7ffa6056f700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa5806d4d0 0x7ffa580cc440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000cc50 tx=0x7ffa5000a340 comp rx=0 tx=0).stop 2023-11-26T07:11:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.849+0000 7ffa6056f700 1 -- 172.21.15.83:0/2864957415 shutdown_connections 2023-11-26T07:11:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.849+0000 7ffa6056f700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa58164a90 0x7ffa5813e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.849+0000 7ffa6056f700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa5806d4d0 0x7ffa580cc440 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.849+0000 7ffa6056f700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffa4406b130 0x7ffa4406d5f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.849+0000 7ffa6056f700 1 --2- 172.21.15.83:0/2864957415 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa580b3b50 0x7ffa581482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.849+0000 7ffa6056f700 1 -- 172.21.15.83:0/2864957415 >> 172.21.15.83:0/2864957415 conn(0x7ffa580cfa30 msgr2=0x7ffa5814cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.849+0000 7ffa6056f700 1 -- 172.21.15.83:0/2864957415 shutdown_connections 2023-11-26T07:11:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:36.850+0000 7ffa6056f700 1 -- 172.21.15.83:0/2864957415 wait complete. 2023-11-26T07:11:36.863 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574857 for osd.3 2023-11-26T07:11:36.863 DEBUG:teuthology.parallel:result is None 2023-11-26T07:11:36.863 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-26T07:11:36.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:11:37.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.022+0000 7f566619c700 1 Processor -- start 2023-11-26T07:11:37.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.022+0000 7f566619c700 1 -- start start 2023-11-26T07:11:37.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.023+0000 7f566619c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f56600af4d0 0x7f56600b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.023+0000 7f566619c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56600ae430 con 0x7f566015e080 2023-11-26T07:11:37.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.023+0000 7f566619c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56600ae710 con 0x7f566006c710 2023-11-26T07:11:37.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.023+0000 7f566619c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56600af0b0 con 0x7f56600af4d0 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.023+0000 7f565ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f56600af4d0 0x7f56600b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.023+0000 7f565f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f566015e080 0x7f566006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54602/0 (socket says 172.21.15.83:54602) 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.023+0000 7f565f7fe700 1 -- 172.21.15.83:0/2217491348 learned_addr learned my addr 172.21.15.83:0/2217491348 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.023+0000 7f565ffff700 1 -- 172.21.15.83:0/2217491348 >> v1:172.21.15.203:6789/0 conn(0x7f566006c710 legacy=0x7f566006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.023+0000 7f565ffff700 1 -- 172.21.15.83:0/2217491348 >> v1:172.21.15.83:6789/0 conn(0x7f566015e080 legacy=0x7f566006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.023+0000 7f565e7fc700 1 -- 172.21.15.83:0/2217491348 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741239037 0 0) 0x7f56600ae430 con 0x7f566015e080 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.023+0000 7f565ffff700 1 -- 172.21.15.83:0/2217491348 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56600ae9f0 con 0x7f56600af4d0 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.024+0000 7f565ffff700 1 --2- 172.21.15.83:0/2217491348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56600af4d0 0x7f56600b5050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5658009d90 tx=0x7f565800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0edaef0254e8a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.024+0000 7f565e7fc700 1 -- 172.21.15.83:0/2217491348 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565800e070 con 0x7f56600af4d0 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.024+0000 7f565e7fc700 1 -- 172.21.15.83:0/2217491348 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565800bb90 con 0x7f56600af4d0 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.024+0000 7f565e7fc700 1 -- 172.21.15.83:0/2217491348 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658003a20 con 0x7f56600af4d0 2023-11-26T07:11:37.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.024+0000 7f566619c700 1 -- 172.21.15.83:0/2217491348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56600af4d0 msgr2=0x7f56600b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.024+0000 7f566619c700 1 --2- 172.21.15.83:0/2217491348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56600af4d0 0x7f56600b5050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5658009d90 tx=0x7f565800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.024+0000 7f566619c700 1 -- 172.21.15.83:0/2217491348 shutdown_connections 2023-11-26T07:11:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.024+0000 7f566619c700 1 --2- 172.21.15.83:0/2217491348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56600af4d0 0x7f56600b5050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.024+0000 7f566619c700 1 -- 172.21.15.83:0/2217491348 >> 172.21.15.83:0/2217491348 conn(0x7f5660071b90 msgr2=0x7f5660071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.024+0000 7f566619c700 1 -- 172.21.15.83:0/2217491348 shutdown_connections 2023-11-26T07:11:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.024+0000 7f566619c700 1 -- 172.21.15.83:0/2217491348 wait complete. 2023-11-26T07:11:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.025+0000 7f566619c700 1 Processor -- start 2023-11-26T07:11:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.026+0000 7f566619c700 1 -- start start 2023-11-26T07:11:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.026+0000 7f566619c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f566006c710 0x7f56600b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.026+0000 7f566619c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56600af4d0 0x7f56600cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.026+0000 7f566619c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566015e080 0x7f56601520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.026+0000 7f566619c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56600c9950 con 0x7f56600af4d0 2023-11-26T07:11:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.026+0000 7f566619c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56600c8dd0 con 0x7f566015e080 2023-11-26T07:11:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.026+0000 7f566619c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56600c90b0 con 0x7f566006c710 2023-11-26T07:11:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56600af4d0 0x7f56600cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56600af4d0 0x7f56600cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34570/0 (socket says 172.21.15.83:34570) 2023-11-26T07:11:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565effd700 1 -- 172.21.15.83:0/1915048694 learned_addr learned my addr 172.21.15.83:0/1915048694 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565ffff700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566015e080 0x7f56601520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565f7fe700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f566006c710 0x7f56600b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565effd700 1 -- 172.21.15.83:0/1915048694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f566006c710 msgr2=0x7f56600b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565effd700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f566006c710 0x7f56600b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565effd700 1 -- 172.21.15.83:0/1915048694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566015e080 msgr2=0x7f56601520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565effd700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566015e080 0x7f56601520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565effd700 1 -- 172.21.15.83:0/1915048694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5658009a70 con 0x7f56600af4d0 2023-11-26T07:11:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565ffff700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566015e080 0x7f56601520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f565effd700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56600af4d0 0x7f56600cfde0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f565000c4d0 tx=0x7f565000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f5647fff700 1 -- 172.21.15.83:0/1915048694 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650011790 con 0x7f56600af4d0 2023-11-26T07:11:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f5647fff700 1 -- 172.21.15.83:0/1915048694 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56500118f0 con 0x7f56600af4d0 2023-11-26T07:11:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f566619c700 1 -- 172.21.15.83:0/1915048694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56600c9390 con 0x7f56600af4d0 2023-11-26T07:11:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f5647fff700 1 -- 172.21.15.83:0/1915048694 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650011ca0 con 0x7f56600af4d0 2023-11-26T07:11:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.027+0000 7f566619c700 1 -- 172.21.15.83:0/1915048694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566015b9f0 con 0x7f56600af4d0 2023-11-26T07:11:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.028+0000 7f566619c700 1 -- 172.21.15.83:0/1915048694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56600753f0 con 0x7f56600af4d0 2023-11-26T07:11:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.029+0000 7f5647fff700 1 -- 172.21.15.83:0/1915048694 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f565001c460 con 0x7f56600af4d0 2023-11-26T07:11:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.029+0000 7f5647fff700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f564c06aed0 0x7f564c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.029+0000 7f565f7fe700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f564c06aed0 0x7f564c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.029+0000 7f5647fff700 1 -- 172.21.15.83:0/1915048694 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56500893e0 con 0x7f56600af4d0 2023-11-26T07:11:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.030+0000 7f565f7fe700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f564c06aed0 0x7f564c06d390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5654000f60 tx=0x7f5654007c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:37.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.034+0000 7f5647fff700 1 -- 172.21.15.83:0/1915048694 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5650054490 con 0x7f56600af4d0 2023-11-26T07:11:37.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.220+0000 7f566619c700 1 -- 172.21.15.83:0/1915048694 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56600b05a0 con 0x7f564c06aed0 2023-11-26T07:11:37.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.227+0000 7f5647fff700 1 -- 172.21.15.83:0/1915048694 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264521 (secure 0 0 0) 0x7f5654012380 con 0x7f564c06aed0 2023-11-26T07:11:37.231 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:37.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 -- 172.21.15.83:0/1915048694 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f564c06aed0 msgr2=0x7f564c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f564c06aed0 0x7f564c06d390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5654000f60 tx=0x7f5654007c90 comp rx=0 tx=0).stop 2023-11-26T07:11:37.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 -- 172.21.15.83:0/1915048694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56600af4d0 msgr2=0x7f56600cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56600af4d0 0x7f56600cfde0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f565000c4d0 tx=0x7f565000c890 comp rx=0 tx=0).stop 2023-11-26T07:11:37.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 -- 172.21.15.83:0/1915048694 shutdown_connections 2023-11-26T07:11:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f566015e080 0x7f56601520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f564c06aed0 0x7f564c06d390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f566006c710 0x7f56600b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 --2- 172.21.15.83:0/1915048694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56600af4d0 0x7f56600cfde0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 -- 172.21.15.83:0/1915048694 >> 172.21.15.83:0/1915048694 conn(0x7f5660071b90 msgr2=0x7f566006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 -- 172.21.15.83:0/1915048694 shutdown_connections 2023-11-26T07:11:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.230+0000 7f566619c700 1 -- 172.21.15.83:0/1915048694 wait complete. 2023-11-26T07:11:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:11:37.243 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-26T07:11:36.309392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219108,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755612,"statfs":{"total":773094113280,"available":772869746688,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002297"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540665+0000","last_change":"2023-11-26T07:11:28.537635+0000","last_active":"2023-11-26T07:11:29.540665+0000","last_peered":"2023-11-26T07:11:29.540665+0000","last_clean":"2023-11-26T07:11:29.540665+0000","last_became_active":"2023-11-26T07:11:28.537327+0000","last_became_peered":"2023-11-26T07:11:28.537327+0000","last_unstale":"2023-11-26T07:11:29.540665+0000","last_undegraded":"2023-11-26T07:11:29.540665+0000","last_fullsized":"2023-11-26T07:11:29.540665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:48:59.913650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537894+0000","last_change":"2023-11-26T07:11:28.541611+0000","last_active":"2023-11-26T07:11:29.537894+0000","last_peered":"2023-11-26T07:11:29.537894+0000","last_clean":"2023-11-26T07:11:29.537894+0000","last_became_active":"2023-11-26T07:11:28.541267+0000","last_became_peered":"2023-11-26T07:11:28.541267+0000","last_unstale":"2023-11-26T07:11:29.537894+0000","last_undegraded":"2023-11-26T07:11:29.537894+0000","last_fullsized":"2023-11-26T07:11:29.537894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189176+0000","last_change":"2023-11-26T07:11:28.543463+0000","last_active":"2023-11-26T07:11:30.189176+0000","last_peered":"2023-11-26T07:11:30.189176+0000","last_clean":"2023-11-26T07:11:30.189176+0000","last_became_active":"2023-11-26T07:11:28.542718+0000","last_became_peered":"2023-11-26T07:11:28.542718+0000","last_unstale":"2023-11-26T07:11:30.189176+0000","last_undegraded":"2023-11-26T07:11:30.189176+0000","last_fullsized":"2023-11-26T07:11:30.189176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719807+0000","last_change":"2023-11-26T07:11:28.544447+0000","last_active":"2023-11-26T07:11:30.719807+0000","last_peered":"2023-11-26T07:11:30.719807+0000","last_clean":"2023-11-26T07:11:30.719807+0000","last_became_active":"2023-11-26T07:11:28.543689+0000","last_became_peered":"2023-11-26T07:11:28.543689+0000","last_unstale":"2023-11-26T07:11:30.719807+0000","last_undegraded":"2023-11-26T07:11:30.719807+0000","last_fullsized":"2023-11-26T07:11:30.719807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:36:40.660123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540289+0000","last_change":"2023-11-26T07:11:28.542518+0000","last_active":"2023-11-26T07:11:29.540289+0000","last_peered":"2023-11-26T07:11:29.540289+0000","last_clean":"2023-11-26T07:11:29.540289+0000","last_became_active":"2023-11-26T07:11:28.542255+0000","last_became_peered":"2023-11-26T07:11:28.542255+0000","last_unstale":"2023-11-26T07:11:29.540289+0000","last_undegraded":"2023-11-26T07:11:29.540289+0000","last_fullsized":"2023-11-26T07:11:29.540289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:48.717411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719213+0000","last_change":"2023-11-26T07:11:28.550053+0000","last_active":"2023-11-26T07:11:30.719213+0000","last_peered":"2023-11-26T07:11:30.719213+0000","last_clean":"2023-11-26T07:11:30.719213+0000","last_became_active":"2023-11-26T07:11:28.549624+0000","last_became_peered":"2023-11-26T07:11:28.549624+0000","last_unstale":"2023-11-26T07:11:30.719213+0000","last_undegraded":"2023-11-26T07:11:30.719213+0000","last_fullsized":"2023-11-26T07:11:30.719213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:42.974584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539979+0000","last_change":"2023-11-26T07:11:28.543140+0000","last_active":"2023-11-26T07:11:29.539979+0000","last_peered":"2023-11-26T07:11:29.539979+0000","last_clean":"2023-11-26T07:11:29.539979+0000","last_became_active":"2023-11-26T07:11:28.542835+0000","last_became_peered":"2023-11-26T07:11:28.542835+0000","last_unstale":"2023-11-26T07:11:29.539979+0000","last_undegraded":"2023-11-26T07:11:29.539979+0000","last_fullsized":"2023-11-26T07:11:29.539979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:14.369275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850216+0000","last_change":"2023-11-26T07:11:28.554561+0000","last_active":"2023-11-26T07:11:30.850216+0000","last_peered":"2023-11-26T07:11:30.850216+0000","last_clean":"2023-11-26T07:11:30.850216+0000","last_became_active":"2023-11-26T07:11:28.554273+0000","last_became_peered":"2023-11-26T07:11:28.554273+0000","last_unstale":"2023-11-26T07:11:30.850216+0000","last_undegraded":"2023-11-26T07:11:30.850216+0000","last_fullsized":"2023-11-26T07:11:30.850216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540623+0000","last_change":"2023-11-26T07:11:28.539323+0000","last_active":"2023-11-26T07:11:29.540623+0000","last_peered":"2023-11-26T07:11:29.540623+0000","last_clean":"2023-11-26T07:11:29.540623+0000","last_became_active":"2023-11-26T07:11:28.539047+0000","last_became_peered":"2023-11-26T07:11:28.539047+0000","last_unstale":"2023-11-26T07:11:29.540623+0000","last_undegraded":"2023-11-26T07:11:29.540623+0000","last_fullsized":"2023-11-26T07:11:29.540623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:01:36.280118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.921372+0000","last_change":"2023-11-26T07:11:28.538383+0000","last_active":"2023-11-26T07:11:30.921372+0000","last_peered":"2023-11-26T07:11:30.921372+0000","last_clean":"2023-11-26T07:11:30.921372+0000","last_became_active":"2023-11-26T07:11:28.537726+0000","last_became_peered":"2023-11-26T07:11:28.537726+0000","last_unstale":"2023-11-26T07:11:30.921372+0000","last_undegraded":"2023-11-26T07:11:30.921372+0000","last_fullsized":"2023-11-26T07:11:30.921372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:42.974584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.921271+0000","last_change":"2023-11-26T07:11:28.538846+0000","last_active":"2023-11-26T07:11:30.921271+0000","last_peered":"2023-11-26T07:11:30.921271+0000","last_clean":"2023-11-26T07:11:30.921271+0000","last_became_active":"2023-11-26T07:11:28.537769+0000","last_became_peered":"2023-11-26T07:11:28.537769+0000","last_unstale":"2023-11-26T07:11:30.921271+0000","last_undegraded":"2023-11-26T07:11:30.921271+0000","last_fullsized":"2023-11-26T07:11:30.921271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539796+0000","last_change":"2023-11-26T07:11:28.541985+0000","last_active":"2023-11-26T07:11:29.539796+0000","last_peered":"2023-11-26T07:11:29.539796+0000","last_clean":"2023-11-26T07:11:29.539796+0000","last_became_active":"2023-11-26T07:11:28.541404+0000","last_became_peered":"2023-11-26T07:11:28.541404+0000","last_unstale":"2023-11-26T07:11:29.539796+0000","last_undegraded":"2023-11-26T07:11:29.539796+0000","last_fullsized":"2023-11-26T07:11:29.539796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:11.545490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850419+0000","last_change":"2023-11-26T07:11:28.551585+0000","last_active":"2023-11-26T07:11:30.850419+0000","last_peered":"2023-11-26T07:11:30.850419+0000","last_clean":"2023-11-26T07:11:30.850419+0000","last_became_active":"2023-11-26T07:11:28.551296+0000","last_became_peered":"2023-11-26T07:11:28.551296+0000","last_unstale":"2023-11-26T07:11:30.850419+0000","last_undegraded":"2023-11-26T07:11:30.850419+0000","last_fullsized":"2023-11-26T07:11:30.850419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189126+0000","last_change":"2023-11-26T07:11:28.541346+0000","last_active":"2023-11-26T07:11:30.189126+0000","last_peered":"2023-11-26T07:11:30.189126+0000","last_clean":"2023-11-26T07:11:30.189126+0000","last_became_active":"2023-11-26T07:11:28.540961+0000","last_became_peered":"2023-11-26T07:11:28.540961+0000","last_unstale":"2023-11-26T07:11:30.189126+0000","last_undegraded":"2023-11-26T07:11:30.189126+0000","last_fullsized":"2023-11-26T07:11:30.189126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:48.999647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537528+0000","last_change":"2023-11-26T07:11:28.536348+0000","last_active":"2023-11-26T07:11:29.537528+0000","last_peered":"2023-11-26T07:11:29.537528+0000","last_clean":"2023-11-26T07:11:29.537528+0000","last_became_active":"2023-11-26T07:11:28.535391+0000","last_became_peered":"2023-11-26T07:11:28.535391+0000","last_unstale":"2023-11-26T07:11:29.537528+0000","last_undegraded":"2023-11-26T07:11:29.537528+0000","last_fullsized":"2023-11-26T07:11:29.537528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719167+0000","last_change":"2023-11-26T07:11:28.544579+0000","last_active":"2023-11-26T07:11:30.719167+0000","last_peered":"2023-11-26T07:11:30.719167+0000","last_clean":"2023-11-26T07:11:30.719167+0000","last_became_active":"2023-11-26T07:11:28.542480+0000","last_became_peered":"2023-11-26T07:11:28.542480+0000","last_unstale":"2023-11-26T07:11:30.719167+0000","last_undegraded":"2023-11-26T07:11:30.719167+0000","last_fullsized":"2023-11-26T07:11:30.719167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719934+0000","last_change":"2023-11-26T07:11:28.554495+0000","last_active":"2023-11-26T07:11:30.719934+0000","last_peered":"2023-11-26T07:11:30.719934+0000","last_clean":"2023-11-26T07:11:30.719934+0000","last_became_active":"2023-11-26T07:11:28.554114+0000","last_became_peered":"2023-11-26T07:11:28.554114+0000","last_unstale":"2023-11-26T07:11:30.719934+0000","last_undegraded":"2023-11-26T07:11:30.719934+0000","last_fullsized":"2023-11-26T07:11:30.719934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:01:36.280118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:31.219855+0000","last_change":"2023-11-26T07:11:28.544698+0000","last_active":"2023-11-26T07:11:31.219855+0000","last_peered":"2023-11-26T07:11:31.219855+0000","last_clean":"2023-11-26T07:11:31.219855+0000","last_became_active":"2023-11-26T07:11:28.544347+0000","last_became_peered":"2023-11-26T07:11:28.544347+0000","last_unstale":"2023-11-26T07:11:31.219855+0000","last_undegraded":"2023-11-26T07:11:31.219855+0000","last_fullsized":"2023-11-26T07:11:31.219855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:11.545490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540518+0000","last_change":"2023-11-26T07:11:28.550524+0000","last_active":"2023-11-26T07:11:29.540518+0000","last_peered":"2023-11-26T07:11:29.540518+0000","last_clean":"2023-11-26T07:11:29.540518+0000","last_became_active":"2023-11-26T07:11:28.543803+0000","last_became_peered":"2023-11-26T07:11:28.543803+0000","last_unstale":"2023-11-26T07:11:29.540518+0000","last_undegraded":"2023-11-26T07:11:29.540518+0000","last_fullsized":"2023-11-26T07:11:29.540518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:36:40.660123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189251+0000","last_change":"2023-11-26T07:11:27.524720+0000","last_active":"2023-11-26T07:11:30.189251+0000","last_peered":"2023-11-26T07:11:30.189251+0000","last_clean":"2023-11-26T07:11:30.189251+0000","last_became_active":"2023-11-26T07:11:27.524424+0000","last_became_peered":"2023-11-26T07:11:27.524424+0000","last_unstale":"2023-11-26T07:11:30.189251+0000","last_undegraded":"2023-11-26T07:11:30.189251+0000","last_fullsized":"2023-11-26T07:11:30.189251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850692+0000","last_change":"2023-11-26T07:11:28.540843+0000","last_active":"2023-11-26T07:11:30.850692+0000","last_peered":"2023-11-26T07:11:30.850692+0000","last_clean":"2023-11-26T07:11:30.850692+0000","last_became_active":"2023-11-26T07:11:28.540579+0000","last_became_peered":"2023-11-26T07:11:28.540579+0000","last_unstale":"2023-11-26T07:11:30.850692+0000","last_undegraded":"2023-11-26T07:11:30.850692+0000","last_fullsized":"2023-11-26T07:11:30.850692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:13.394314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.320406+0000","last_change":"2023-11-26T07:11:27.528053+0000","last_active":"2023-11-26T07:11:30.320406+0000","last_peered":"2023-11-26T07:11:30.320406+0000","last_clean":"2023-11-26T07:11:30.320406+0000","last_became_active":"2023-11-26T07:11:27.526741+0000","last_became_peered":"2023-11-26T07:11:27.526741+0000","last_unstale":"2023-11-26T07:11:30.320406+0000","last_undegraded":"2023-11-26T07:11:30.320406+0000","last_fullsized":"2023-11-26T07:11:30.320406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:31.071342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189238+0000","last_change":"2023-11-26T07:11:28.551779+0000","last_active":"2023-11-26T07:11:30.189238+0000","last_peered":"2023-11-26T07:11:30.189238+0000","last_clean":"2023-11-26T07:11:30.189238+0000","last_became_active":"2023-11-26T07:11:28.551489+0000","last_became_peered":"2023-11-26T07:11:28.551489+0000","last_unstale":"2023-11-26T07:11:30.189238+0000","last_undegraded":"2023-11-26T07:11:30.189238+0000","last_fullsized":"2023-11-26T07:11:30.189238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540841+0000","last_change":"2023-11-26T07:11:27.530905+0000","last_active":"2023-11-26T07:11:29.540841+0000","last_peered":"2023-11-26T07:11:29.540841+0000","last_clean":"2023-11-26T07:11:29.540841+0000","last_became_active":"2023-11-26T07:11:27.529568+0000","last_became_peered":"2023-11-26T07:11:27.529568+0000","last_unstale":"2023-11-26T07:11:29.540841+0000","last_undegraded":"2023-11-26T07:11:29.540841+0000","last_fullsized":"2023-11-26T07:11:29.540841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:12.389909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719541+0000","last_change":"2023-11-26T07:11:28.547553+0000","last_active":"2023-11-26T07:11:30.719541+0000","last_peered":"2023-11-26T07:11:30.719541+0000","last_clean":"2023-11-26T07:11:30.719541+0000","last_became_active":"2023-11-26T07:11:28.547052+0000","last_became_peered":"2023-11-26T07:11:28.547052+0000","last_unstale":"2023-11-26T07:11:30.719541+0000","last_undegraded":"2023-11-26T07:11:30.719541+0000","last_fullsized":"2023-11-26T07:11:30.719541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:58.985237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540344+0000","last_change":"2023-11-26T07:11:27.528392+0000","last_active":"2023-11-26T07:11:29.540344+0000","last_peered":"2023-11-26T07:11:29.540344+0000","last_clean":"2023-11-26T07:11:29.540344+0000","last_became_active":"2023-11-26T07:11:27.528039+0000","last_became_peered":"2023-11-26T07:11:27.528039+0000","last_unstale":"2023-11-26T07:11:29.540344+0000","last_undegraded":"2023-11-26T07:11:29.540344+0000","last_fullsized":"2023-11-26T07:11:29.540344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719504+0000","last_change":"2023-11-26T07:11:28.546262+0000","last_active":"2023-11-26T07:11:30.719504+0000","last_peered":"2023-11-26T07:11:30.719504+0000","last_clean":"2023-11-26T07:11:30.719504+0000","last_became_active":"2023-11-26T07:11:28.545998+0000","last_became_peered":"2023-11-26T07:11:28.545998+0000","last_unstale":"2023-11-26T07:11:30.719504+0000","last_undegraded":"2023-11-26T07:11:30.719504+0000","last_fullsized":"2023-11-26T07:11:30.719504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:13.394314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.320372+0000","last_change":"2023-11-26T07:11:27.528591+0000","last_active":"2023-11-26T07:11:30.320372+0000","last_peered":"2023-11-26T07:11:30.320372+0000","last_clean":"2023-11-26T07:11:30.320372+0000","last_became_active":"2023-11-26T07:11:27.528327+0000","last_became_peered":"2023-11-26T07:11:27.528327+0000","last_unstale":"2023-11-26T07:11:30.320372+0000","last_undegraded":"2023-11-26T07:11:30.320372+0000","last_fullsized":"2023-11-26T07:11:30.320372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:21.506925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539218+0000","last_change":"2023-11-26T07:11:28.541089+0000","last_active":"2023-11-26T07:11:29.539218+0000","last_peered":"2023-11-26T07:11:29.539218+0000","last_clean":"2023-11-26T07:11:29.539218+0000","last_became_active":"2023-11-26T07:11:28.534262+0000","last_became_peered":"2023-11-26T07:11:28.534262+0000","last_unstale":"2023-11-26T07:11:29.539218+0000","last_undegraded":"2023-11-26T07:11:29.539218+0000","last_fullsized":"2023-11-26T07:11:29.539218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:33.426915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.331849+0000","last_change":"2023-11-26T07:11:27.530892+0000","last_active":"2023-11-26T07:11:30.331849+0000","last_peered":"2023-11-26T07:11:30.331849+0000","last_clean":"2023-11-26T07:11:30.331849+0000","last_became_active":"2023-11-26T07:11:27.529853+0000","last_became_peered":"2023-11-26T07:11:27.529853+0000","last_unstale":"2023-11-26T07:11:30.331849+0000","last_undegraded":"2023-11-26T07:11:30.331849+0000","last_fullsized":"2023-11-26T07:11:30.331849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189422+0000","last_change":"2023-11-26T07:11:28.548912+0000","last_active":"2023-11-26T07:11:30.189422+0000","last_peered":"2023-11-26T07:11:30.189422+0000","last_clean":"2023-11-26T07:11:30.189422+0000","last_became_active":"2023-11-26T07:11:28.547915+0000","last_became_peered":"2023-11-26T07:11:28.547915+0000","last_unstale":"2023-11-26T07:11:30.189422+0000","last_undegraded":"2023-11-26T07:11:30.189422+0000","last_fullsized":"2023-11-26T07:11:30.189422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:42.974584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.320300+0000","last_change":"2023-11-26T07:11:27.529609+0000","last_active":"2023-11-26T07:11:30.320300+0000","last_peered":"2023-11-26T07:11:30.320300+0000","last_clean":"2023-11-26T07:11:30.320300+0000","last_became_active":"2023-11-26T07:11:27.529045+0000","last_became_peered":"2023-11-26T07:11:27.529045+0000","last_unstale":"2023-11-26T07:11:30.320300+0000","last_undegraded":"2023-11-26T07:11:30.320300+0000","last_fullsized":"2023-11-26T07:11:30.320300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:18.223957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850312+0000","last_change":"2023-11-26T07:11:28.543403+0000","last_active":"2023-11-26T07:11:30.850312+0000","last_peered":"2023-11-26T07:11:30.850312+0000","last_clean":"2023-11-26T07:11:30.850312+0000","last_became_active":"2023-11-26T07:11:28.543119+0000","last_became_peered":"2023-11-26T07:11:28.543119+0000","last_unstale":"2023-11-26T07:11:30.850312+0000","last_undegraded":"2023-11-26T07:11:30.850312+0000","last_fullsized":"2023-11-26T07:11:30.850312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.319713+0000","last_change":"2023-11-26T07:11:27.528106+0000","last_active":"2023-11-26T07:11:30.319713+0000","last_peered":"2023-11-26T07:11:30.319713+0000","last_clean":"2023-11-26T07:11:30.319713+0000","last_became_active":"2023-11-26T07:11:27.527832+0000","last_became_peered":"2023-11-26T07:11:27.527832+0000","last_unstale":"2023-11-26T07:11:30.319713+0000","last_undegraded":"2023-11-26T07:11:30.319713+0000","last_fullsized":"2023-11-26T07:11:30.319713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:50.240018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537690+0000","last_change":"2023-11-26T07:11:28.543380+0000","last_active":"2023-11-26T07:11:29.537690+0000","last_peered":"2023-11-26T07:11:29.537690+0000","last_clean":"2023-11-26T07:11:29.537690+0000","last_became_active":"2023-11-26T07:11:28.543083+0000","last_became_peered":"2023-11-26T07:11:28.543083+0000","last_unstale":"2023-11-26T07:11:29.537690+0000","last_undegraded":"2023-11-26T07:11:29.537690+0000","last_fullsized":"2023-11-26T07:11:29.537690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.331947+0000","last_change":"2023-11-26T07:11:27.531555+0000","last_active":"2023-11-26T07:11:30.331947+0000","last_peered":"2023-11-26T07:11:30.331947+0000","last_clean":"2023-11-26T07:11:30.331947+0000","last_became_active":"2023-11-26T07:11:27.531241+0000","last_became_peered":"2023-11-26T07:11:27.531241+0000","last_unstale":"2023-11-26T07:11:30.331947+0000","last_undegraded":"2023-11-26T07:11:30.331947+0000","last_fullsized":"2023-11-26T07:11:30.331947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719839+0000","last_change":"2023-11-26T07:11:28.552112+0000","last_active":"2023-11-26T07:11:30.719839+0000","last_peered":"2023-11-26T07:11:30.719839+0000","last_clean":"2023-11-26T07:11:30.719839+0000","last_became_active":"2023-11-26T07:11:28.551827+0000","last_became_peered":"2023-11-26T07:11:28.551827+0000","last_unstale":"2023-11-26T07:11:30.719839+0000","last_undegraded":"2023-11-26T07:11:30.719839+0000","last_fullsized":"2023-11-26T07:11:30.719839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:51.665872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.319891+0000","last_change":"2023-11-26T07:11:27.531668+0000","last_active":"2023-11-26T07:11:30.319891+0000","last_peered":"2023-11-26T07:11:30.319891+0000","last_clean":"2023-11-26T07:11:30.319891+0000","last_became_active":"2023-11-26T07:11:27.531403+0000","last_became_peered":"2023-11-26T07:11:27.531403+0000","last_unstale":"2023-11-26T07:11:30.319891+0000","last_undegraded":"2023-11-26T07:11:30.319891+0000","last_fullsized":"2023-11-26T07:11:30.319891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189378+0000","last_change":"2023-11-26T07:11:28.542823+0000","last_active":"2023-11-26T07:11:30.189378+0000","last_peered":"2023-11-26T07:11:30.189378+0000","last_clean":"2023-11-26T07:11:30.189378+0000","last_became_active":"2023-11-26T07:11:28.542535+0000","last_became_peered":"2023-11-26T07:11:28.542535+0000","last_unstale":"2023-11-26T07:11:30.189378+0000","last_undegraded":"2023-11-26T07:11:30.189378+0000","last_fullsized":"2023-11-26T07:11:30.189378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537747+0000","last_change":"2023-11-26T07:11:27.531081+0000","last_active":"2023-11-26T07:11:29.537747+0000","last_peered":"2023-11-26T07:11:29.537747+0000","last_clean":"2023-11-26T07:11:29.537747+0000","last_became_active":"2023-11-26T07:11:27.530823+0000","last_became_peered":"2023-11-26T07:11:27.530823+0000","last_unstale":"2023-11-26T07:11:29.537747+0000","last_undegraded":"2023-11-26T07:11:29.537747+0000","last_fullsized":"2023-11-26T07:11:29.537747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:21.506925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539660+0000","last_change":"2023-11-26T07:11:28.529675+0000","last_active":"2023-11-26T07:11:29.539660+0000","last_peered":"2023-11-26T07:11:29.539660+0000","last_clean":"2023-11-26T07:11:29.539660+0000","last_became_active":"2023-11-26T07:11:28.529388+0000","last_became_peered":"2023-11-26T07:11:28.529388+0000","last_unstale":"2023-11-26T07:11:29.539660+0000","last_undegraded":"2023-11-26T07:11:29.539660+0000","last_fullsized":"2023-11-26T07:11:29.539660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:32.075747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.331864+0000","last_change":"2023-11-26T07:11:27.534932+0000","last_active":"2023-11-26T07:11:30.331864+0000","last_peered":"2023-11-26T07:11:30.331864+0000","last_clean":"2023-11-26T07:11:30.331864+0000","last_became_active":"2023-11-26T07:11:27.534626+0000","last_became_peered":"2023-11-26T07:11:27.534626+0000","last_unstale":"2023-11-26T07:11:30.331864+0000","last_undegraded":"2023-11-26T07:11:30.331864+0000","last_fullsized":"2023-11-26T07:11:30.331864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:25.399589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540678+0000","last_change":"2023-11-26T07:11:28.538363+0000","last_active":"2023-11-26T07:11:29.540678+0000","last_peered":"2023-11-26T07:11:29.540678+0000","last_clean":"2023-11-26T07:11:29.540678+0000","last_became_active":"2023-11-26T07:11:28.538098+0000","last_became_peered":"2023-11-26T07:11:28.538098+0000","last_unstale":"2023-11-26T07:11:29.540678+0000","last_undegraded":"2023-11-26T07:11:29.540678+0000","last_fullsized":"2023-11-26T07:11:29.540678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:00.284371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189378+0000","last_change":"2023-11-26T07:11:27.522269+0000","last_active":"2023-11-26T07:11:30.189378+0000","last_peered":"2023-11-26T07:11:30.189378+0000","last_clean":"2023-11-26T07:11:30.189378+0000","last_became_active":"2023-11-26T07:11:27.521852+0000","last_became_peered":"2023-11-26T07:11:27.521852+0000","last_unstale":"2023-11-26T07:11:30.189378+0000","last_undegraded":"2023-11-26T07:11:30.189378+0000","last_fullsized":"2023-11-26T07:11:30.189378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189106+0000","last_change":"2023-11-26T07:11:28.552836+0000","last_active":"2023-11-26T07:11:30.189106+0000","last_peered":"2023-11-26T07:11:30.189106+0000","last_clean":"2023-11-26T07:11:30.189106+0000","last_became_active":"2023-11-26T07:11:28.552532+0000","last_became_peered":"2023-11-26T07:11:28.552532+0000","last_unstale":"2023-11-26T07:11:30.189106+0000","last_undegraded":"2023-11-26T07:11:30.189106+0000","last_fullsized":"2023-11-26T07:11:30.189106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:51.244423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.319798+0000","last_change":"2023-11-26T07:11:27.534390+0000","last_active":"2023-11-26T07:11:30.319798+0000","last_peered":"2023-11-26T07:11:30.319798+0000","last_clean":"2023-11-26T07:11:30.319798+0000","last_became_active":"2023-11-26T07:11:27.534100+0000","last_became_peered":"2023-11-26T07:11:27.534100+0000","last_unstale":"2023-11-26T07:11:30.319798+0000","last_undegraded":"2023-11-26T07:11:30.319798+0000","last_fullsized":"2023-11-26T07:11:30.319798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:15.199340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540363+0000","last_change":"2023-11-26T07:11:28.530599+0000","last_active":"2023-11-26T07:11:29.540363+0000","last_peered":"2023-11-26T07:11:29.540363+0000","last_clean":"2023-11-26T07:11:29.540363+0000","last_became_active":"2023-11-26T07:11:28.530091+0000","last_became_peered":"2023-11-26T07:11:28.530091+0000","last_unstale":"2023-11-26T07:11:29.540363+0000","last_undegraded":"2023-11-26T07:11:29.540363+0000","last_fullsized":"2023-11-26T07:11:29.540363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:21:12.239221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.331823+0000","last_change":"2023-11-26T07:11:27.530816+0000","last_active":"2023-11-26T07:11:30.331823+0000","last_peered":"2023-11-26T07:11:30.331823+0000","last_clean":"2023-11-26T07:11:30.331823+0000","last_became_active":"2023-11-26T07:11:27.530026+0000","last_became_peered":"2023-11-26T07:11:27.530026+0000","last_unstale":"2023-11-26T07:11:30.331823+0000","last_undegraded":"2023-11-26T07:11:30.331823+0000","last_fullsized":"2023-11-26T07:11:30.331823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:32.422510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.920929+0000","last_change":"2023-11-26T07:11:28.542511+0000","last_active":"2023-11-26T07:11:30.920929+0000","last_peered":"2023-11-26T07:11:30.920929+0000","last_clean":"2023-11-26T07:11:30.920929+0000","last_became_active":"2023-11-26T07:11:28.542221+0000","last_became_peered":"2023-11-26T07:11:28.542221+0000","last_unstale":"2023-11-26T07:11:30.920929+0000","last_undegraded":"2023-11-26T07:11:30.920929+0000","last_fullsized":"2023-11-26T07:11:30.920929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.321930+0000","last_change":"2023-11-26T07:11:27.524724+0000","last_active":"2023-11-26T07:11:30.321930+0000","last_peered":"2023-11-26T07:11:30.321930+0000","last_clean":"2023-11-26T07:11:30.321930+0000","last_became_active":"2023-11-26T07:11:27.524453+0000","last_became_peered":"2023-11-26T07:11:27.524453+0000","last_unstale":"2023-11-26T07:11:30.321930+0000","last_undegraded":"2023-11-26T07:11:30.321930+0000","last_fullsized":"2023-11-26T07:11:30.321930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:32.422510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540122+0000","last_change":"2023-11-26T07:11:28.530396+0000","last_active":"2023-11-26T07:11:29.540122+0000","last_peered":"2023-11-26T07:11:29.540122+0000","last_clean":"2023-11-26T07:11:29.540122+0000","last_became_active":"2023-11-26T07:11:28.530134+0000","last_became_peered":"2023-11-26T07:11:28.530134+0000","last_unstale":"2023-11-26T07:11:29.540122+0000","last_undegraded":"2023-11-26T07:11:29.540122+0000","last_fullsized":"2023-11-26T07:11:29.540122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:58.985237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539626+0000","last_change":"2023-11-26T07:11:27.526573+0000","last_active":"2023-11-26T07:11:29.539626+0000","last_peered":"2023-11-26T07:11:29.539626+0000","last_clean":"2023-11-26T07:11:29.539626+0000","last_became_active":"2023-11-26T07:11:27.526278+0000","last_became_peered":"2023-11-26T07:11:27.526278+0000","last_unstale":"2023-11-26T07:11:29.539626+0000","last_undegraded":"2023-11-26T07:11:29.539626+0000","last_fullsized":"2023-11-26T07:11:29.539626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:35.117462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540624+0000","last_change":"2023-11-26T07:11:28.530307+0000","last_active":"2023-11-26T07:11:29.540624+0000","last_peered":"2023-11-26T07:11:29.540624+0000","last_clean":"2023-11-26T07:11:29.540624+0000","last_became_active":"2023-11-26T07:11:28.530012+0000","last_became_peered":"2023-11-26T07:11:28.530012+0000","last_unstale":"2023-11-26T07:11:29.540624+0000","last_undegraded":"2023-11-26T07:11:29.540624+0000","last_fullsized":"2023-11-26T07:11:29.540624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:51.665872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537662+0000","last_change":"2023-11-26T07:11:27.525592+0000","last_active":"2023-11-26T07:11:29.537662+0000","last_peered":"2023-11-26T07:11:29.537662+0000","last_clean":"2023-11-26T07:11:29.537662+0000","last_became_active":"2023-11-26T07:11:27.525304+0000","last_became_peered":"2023-11-26T07:11:27.525304+0000","last_unstale":"2023-11-26T07:11:29.537662+0000","last_undegraded":"2023-11-26T07:11:29.537662+0000","last_fullsized":"2023-11-26T07:11:29.537662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:52.471590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719811+0000","last_change":"2023-11-26T07:11:28.538490+0000","last_active":"2023-11-26T07:11:30.719811+0000","last_peered":"2023-11-26T07:11:30.719811+0000","last_clean":"2023-11-26T07:11:30.719811+0000","last_became_active":"2023-11-26T07:11:28.538185+0000","last_became_peered":"2023-11-26T07:11:28.538185+0000","last_unstale":"2023-11-26T07:11:30.719811+0000","last_undegraded":"2023-11-26T07:11:30.719811+0000","last_fullsized":"2023-11-26T07:11:30.719811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:21:12.239221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.319783+0000","last_change":"2023-11-26T07:11:27.524225+0000","last_active":"2023-11-26T07:11:30.319783+0000","last_peered":"2023-11-26T07:11:30.319783+0000","last_clean":"2023-11-26T07:11:30.319783+0000","last_became_active":"2023-11-26T07:11:27.523610+0000","last_became_peered":"2023-11-26T07:11:27.523610+0000","last_unstale":"2023-11-26T07:11:30.319783+0000","last_undegraded":"2023-11-26T07:11:30.319783+0000","last_fullsized":"2023-11-26T07:11:30.319783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:19.248482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539423+0000","last_change":"2023-11-26T07:11:28.547971+0000","last_active":"2023-11-26T07:11:29.539423+0000","last_peered":"2023-11-26T07:11:29.539423+0000","last_clean":"2023-11-26T07:11:29.539423+0000","last_became_active":"2023-11-26T07:11:28.547756+0000","last_became_peered":"2023-11-26T07:11:28.547756+0000","last_unstale":"2023-11-26T07:11:29.539423+0000","last_undegraded":"2023-11-26T07:11:29.539423+0000","last_fullsized":"2023-11-26T07:11:29.539423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:57.098847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537553+0000","last_change":"2023-11-26T07:11:28.542253+0000","last_active":"2023-11-26T07:11:29.537553+0000","last_peered":"2023-11-26T07:11:29.537553+0000","last_clean":"2023-11-26T07:11:29.537553+0000","last_became_active":"2023-11-26T07:11:28.541977+0000","last_became_peered":"2023-11-26T07:11:28.541977+0000","last_unstale":"2023-11-26T07:11:29.537553+0000","last_undegraded":"2023-11-26T07:11:29.537553+0000","last_fullsized":"2023-11-26T07:11:29.537553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.321970+0000","last_change":"2023-11-26T07:11:27.534895+0000","last_active":"2023-11-26T07:11:30.321970+0000","last_peered":"2023-11-26T07:11:30.321970+0000","last_clean":"2023-11-26T07:11:30.321970+0000","last_became_active":"2023-11-26T07:11:27.534654+0000","last_became_peered":"2023-11-26T07:11:27.534654+0000","last_unstale":"2023-11-26T07:11:30.321970+0000","last_undegraded":"2023-11-26T07:11:30.321970+0000","last_fullsized":"2023-11-26T07:11:30.321970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:18.223957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850465+0000","last_change":"2023-11-26T07:11:28.544575+0000","last_active":"2023-11-26T07:11:30.850465+0000","last_peered":"2023-11-26T07:11:30.850465+0000","last_clean":"2023-11-26T07:11:30.850465+0000","last_became_active":"2023-11-26T07:11:28.543760+0000","last_became_peered":"2023-11-26T07:11:28.543760+0000","last_unstale":"2023-11-26T07:11:30.850465+0000","last_undegraded":"2023-11-26T07:11:30.850465+0000","last_fullsized":"2023-11-26T07:11:30.850465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.319818+0000","last_change":"2023-11-26T07:11:27.530887+0000","last_active":"2023-11-26T07:11:30.319818+0000","last_peered":"2023-11-26T07:11:30.319818+0000","last_clean":"2023-11-26T07:11:30.319818+0000","last_became_active":"2023-11-26T07:11:27.530613+0000","last_became_peered":"2023-11-26T07:11:27.530613+0000","last_unstale":"2023-11-26T07:11:30.319818+0000","last_undegraded":"2023-11-26T07:11:30.319818+0000","last_fullsized":"2023-11-26T07:11:30.319818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:35.117462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.525772+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:11:28.525772+0000","last_peered":"2023-11-26T07:11:28.525772+0000","last_clean":"2023-11-26T07:11:28.525772+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:11:28.525772+0000","last_undegraded":"2023-11-26T07:11:28.525772+0000","last_fullsized":"2023-11-26T07:11:28.525772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189105+0000","last_change":"2023-11-26T07:11:28.548666+0000","last_active":"2023-11-26T07:11:30.189105+0000","last_peered":"2023-11-26T07:11:30.189105+0000","last_clean":"2023-11-26T07:11:30.189105+0000","last_became_active":"2023-11-26T07:11:28.548390+0000","last_became_peered":"2023-11-26T07:11:28.548390+0000","last_unstale":"2023-11-26T07:11:30.189105+0000","last_undegraded":"2023-11-26T07:11:30.189105+0000","last_fullsized":"2023-11-26T07:11:30.189105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:26.403994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539972+0000","last_change":"2023-11-26T07:11:27.525655+0000","last_active":"2023-11-26T07:11:29.539972+0000","last_peered":"2023-11-26T07:11:29.539972+0000","last_clean":"2023-11-26T07:11:29.539972+0000","last_became_active":"2023-11-26T07:11:27.525321+0000","last_became_peered":"2023-11-26T07:11:27.525321+0000","last_unstale":"2023-11-26T07:11:29.539972+0000","last_undegraded":"2023-11-26T07:11:29.539972+0000","last_fullsized":"2023-11-26T07:11:29.539972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:47.995242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540819+0000","last_change":"2023-11-26T07:11:27.528743+0000","last_active":"2023-11-26T07:11:29.540819+0000","last_peered":"2023-11-26T07:11:29.540819+0000","last_clean":"2023-11-26T07:11:29.540819+0000","last_became_active":"2023-11-26T07:11:27.527803+0000","last_became_peered":"2023-11-26T07:11:27.527803+0000","last_unstale":"2023-11-26T07:11:29.540819+0000","last_undegraded":"2023-11-26T07:11:29.540819+0000","last_fullsized":"2023-11-26T07:11:29.540819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:16:53.693262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537674+0000","last_change":"2023-11-26T07:11:28.542614+0000","last_active":"2023-11-26T07:11:29.537674+0000","last_peered":"2023-11-26T07:11:29.537674+0000","last_clean":"2023-11-26T07:11:29.537674+0000","last_became_active":"2023-11-26T07:11:28.542356+0000","last_became_peered":"2023-11-26T07:11:28.542356+0000","last_unstale":"2023-11-26T07:11:29.537674+0000","last_undegraded":"2023-11-26T07:11:29.537674+0000","last_fullsized":"2023-11-26T07:11:29.537674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.318105+0000","last_change":"2023-11-26T07:11:27.527868+0000","last_active":"2023-11-26T07:11:30.318105+0000","last_peered":"2023-11-26T07:11:30.318105+0000","last_clean":"2023-11-26T07:11:30.318105+0000","last_became_active":"2023-11-26T07:11:27.527482+0000","last_became_peered":"2023-11-26T07:11:27.527482+0000","last_unstale":"2023-11-26T07:11:30.318105+0000","last_undegraded":"2023-11-26T07:11:30.318105+0000","last_fullsized":"2023-11-26T07:11:30.318105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850552+0000","last_change":"2023-11-26T07:11:28.554487+0000","last_active":"2023-11-26T07:11:30.850552+0000","last_peered":"2023-11-26T07:11:30.850552+0000","last_clean":"2023-11-26T07:11:30.850552+0000","last_became_active":"2023-11-26T07:11:28.554210+0000","last_became_peered":"2023-11-26T07:11:28.554210+0000","last_unstale":"2023-11-26T07:11:30.850552+0000","last_undegraded":"2023-11-26T07:11:30.850552+0000","last_fullsized":"2023-11-26T07:11:30.850552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:24:47.640100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.331934+0000","last_change":"2023-11-26T07:11:27.531628+0000","last_active":"2023-11-26T07:11:30.331934+0000","last_peered":"2023-11-26T07:11:30.331934+0000","last_clean":"2023-11-26T07:11:30.331934+0000","last_became_active":"2023-11-26T07:11:27.531336+0000","last_became_peered":"2023-11-26T07:11:27.531336+0000","last_unstale":"2023-11-26T07:11:30.331934+0000","last_undegraded":"2023-11-26T07:11:30.331934+0000","last_fullsized":"2023-11-26T07:11:30.331934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:10.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189372+0000","last_change":"2023-11-26T07:11:28.540197+0000","last_active":"2023-11-26T07:11:30.189372+0000","last_peered":"2023-11-26T07:11:30.189372+0000","last_clean":"2023-11-26T07:11:30.189372+0000","last_became_active":"2023-11-26T07:11:28.539930+0000","last_became_peered":"2023-11-26T07:11:28.539930+0000","last_unstale":"2023-11-26T07:11:30.189372+0000","last_undegraded":"2023-11-26T07:11:30.189372+0000","last_fullsized":"2023-11-26T07:11:30.189372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539876+0000","last_change":"2023-11-26T07:11:27.528055+0000","last_active":"2023-11-26T07:11:29.539876+0000","last_peered":"2023-11-26T07:11:29.539876+0000","last_clean":"2023-11-26T07:11:29.539876+0000","last_became_active":"2023-11-26T07:11:27.527760+0000","last_became_peered":"2023-11-26T07:11:27.527760+0000","last_unstale":"2023-11-26T07:11:29.539876+0000","last_undegraded":"2023-11-26T07:11:29.539876+0000","last_fullsized":"2023-11-26T07:11:29.539876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:10.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539630+0000","last_change":"2023-11-26T07:11:28.544985+0000","last_active":"2023-11-26T07:11:29.539630+0000","last_peered":"2023-11-26T07:11:29.539630+0000","last_clean":"2023-11-26T07:11:29.539630+0000","last_became_active":"2023-11-26T07:11:28.544546+0000","last_became_peered":"2023-11-26T07:11:28.544546+0000","last_unstale":"2023-11-26T07:11:29.539630+0000","last_undegraded":"2023-11-26T07:11:29.539630+0000","last_fullsized":"2023-11-26T07:11:29.539630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:19.228362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537514+0000","last_change":"2023-11-26T07:11:27.527741+0000","last_active":"2023-11-26T07:11:29.537514+0000","last_peered":"2023-11-26T07:11:29.537514+0000","last_clean":"2023-11-26T07:11:29.537514+0000","last_became_active":"2023-11-26T07:11:27.526479+0000","last_became_peered":"2023-11-26T07:11:27.526479+0000","last_unstale":"2023-11-26T07:11:29.537514+0000","last_undegraded":"2023-11-26T07:11:29.537514+0000","last_fullsized":"2023-11-26T07:11:29.537514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:31.071342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850638+0000","last_change":"2023-11-26T07:11:28.540204+0000","last_active":"2023-11-26T07:11:30.850638+0000","last_peered":"2023-11-26T07:11:30.850638+0000","last_clean":"2023-11-26T07:11:30.850638+0000","last_became_active":"2023-11-26T07:11:28.539891+0000","last_became_peered":"2023-11-26T07:11:28.539891+0000","last_unstale":"2023-11-26T07:11:30.850638+0000","last_undegraded":"2023-11-26T07:11:30.850638+0000","last_fullsized":"2023-11-26T07:11:30.850638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:16:54.697667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537841+0000","last_change":"2023-11-26T07:11:27.534469+0000","last_active":"2023-11-26T07:11:29.537841+0000","last_peered":"2023-11-26T07:11:29.537841+0000","last_clean":"2023-11-26T07:11:29.537841+0000","last_became_active":"2023-11-26T07:11:27.534116+0000","last_became_peered":"2023-11-26T07:11:27.534116+0000","last_unstale":"2023-11-26T07:11:29.537841+0000","last_undegraded":"2023-11-26T07:11:29.537841+0000","last_fullsized":"2023-11-26T07:11:29.537841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:25.399589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850461+0000","last_change":"2023-11-26T07:11:28.547277+0000","last_active":"2023-11-26T07:11:30.850461+0000","last_peered":"2023-11-26T07:11:30.850461+0000","last_clean":"2023-11-26T07:11:30.850461+0000","last_became_active":"2023-11-26T07:11:28.546992+0000","last_became_peered":"2023-11-26T07:11:28.546992+0000","last_unstale":"2023-11-26T07:11:30.850461+0000","last_undegraded":"2023-11-26T07:11:30.850461+0000","last_fullsized":"2023-11-26T07:11:30.850461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:42.974584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540403+0000","last_change":"2023-11-26T07:11:27.529785+0000","last_active":"2023-11-26T07:11:29.540403+0000","last_peered":"2023-11-26T07:11:29.540403+0000","last_clean":"2023-11-26T07:11:29.540403+0000","last_became_active":"2023-11-26T07:11:27.528686+0000","last_became_peered":"2023-11-26T07:11:27.528686+0000","last_unstale":"2023-11-26T07:11:29.540403+0000","last_undegraded":"2023-11-26T07:11:29.540403+0000","last_fullsized":"2023-11-26T07:11:29.540403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:21.053006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.921146+0000","last_change":"2023-11-26T07:11:28.546713+0000","last_active":"2023-11-26T07:11:30.921146+0000","last_peered":"2023-11-26T07:11:30.921146+0000","last_clean":"2023-11-26T07:11:30.921146+0000","last_became_active":"2023-11-26T07:11:28.546453+0000","last_became_peered":"2023-11-26T07:11:28.546453+0000","last_unstale":"2023-11-26T07:11:30.921146+0000","last_undegraded":"2023-11-26T07:11:30.921146+0000","last_fullsized":"2023-11-26T07:11:30.921146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.321984+0000","last_change":"2023-11-26T07:11:27.531109+0000","last_active":"2023-11-26T07:11:30.321984+0000","last_peered":"2023-11-26T07:11:30.321984+0000","last_clean":"2023-11-26T07:11:30.321984+0000","last_became_active":"2023-11-26T07:11:27.530786+0000","last_became_peered":"2023-11-26T07:11:27.530786+0000","last_unstale":"2023-11-26T07:11:30.321984+0000","last_undegraded":"2023-11-26T07:11:30.321984+0000","last_fullsized":"2023-11-26T07:11:30.321984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:52.471590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540761+0000","last_change":"2023-11-26T07:11:28.539795+0000","last_active":"2023-11-26T07:11:29.540761+0000","last_peered":"2023-11-26T07:11:29.540761+0000","last_clean":"2023-11-26T07:11:29.540761+0000","last_became_active":"2023-11-26T07:11:28.539562+0000","last_became_peered":"2023-11-26T07:11:28.539562+0000","last_unstale":"2023-11-26T07:11:29.540761+0000","last_undegraded":"2023-11-26T07:11:29.540761+0000","last_fullsized":"2023-11-26T07:11:29.540761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:02:05.837198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537674+0000","last_change":"2023-11-26T07:11:27.522266+0000","last_active":"2023-11-26T07:11:29.537674+0000","last_peered":"2023-11-26T07:11:29.537674+0000","last_clean":"2023-11-26T07:11:29.537674+0000","last_became_active":"2023-11-26T07:11:27.521972+0000","last_became_peered":"2023-11-26T07:11:27.521972+0000","last_unstale":"2023-11-26T07:11:29.537674+0000","last_undegraded":"2023-11-26T07:11:29.537674+0000","last_fullsized":"2023-11-26T07:11:29.537674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719435+0000","last_change":"2023-11-26T07:11:28.544499+0000","last_active":"2023-11-26T07:11:30.719435+0000","last_peered":"2023-11-26T07:11:30.719435+0000","last_clean":"2023-11-26T07:11:30.719435+0000","last_became_active":"2023-11-26T07:11:28.544154+0000","last_became_peered":"2023-11-26T07:11:28.544154+0000","last_unstale":"2023-11-26T07:11:30.719435+0000","last_undegraded":"2023-11-26T07:11:30.719435+0000","last_fullsized":"2023-11-26T07:11:30.719435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:16:54.697667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.322016+0000","last_change":"2023-11-26T07:11:27.535061+0000","last_active":"2023-11-26T07:11:30.322016+0000","last_peered":"2023-11-26T07:11:30.322016+0000","last_clean":"2023-11-26T07:11:30.322016+0000","last_became_active":"2023-11-26T07:11:27.534792+0000","last_became_peered":"2023-11-26T07:11:27.534792+0000","last_unstale":"2023-11-26T07:11:30.322016+0000","last_undegraded":"2023-11-26T07:11:30.322016+0000","last_fullsized":"2023-11-26T07:11:30.322016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540720+0000","last_change":"2023-11-26T07:11:28.532153+0000","last_active":"2023-11-26T07:11:29.540720+0000","last_peered":"2023-11-26T07:11:29.540720+0000","last_clean":"2023-11-26T07:11:29.540720+0000","last_became_active":"2023-11-26T07:11:28.531880+0000","last_became_peered":"2023-11-26T07:11:28.531880+0000","last_unstale":"2023-11-26T07:11:29.540720+0000","last_undegraded":"2023-11-26T07:11:29.540720+0000","last_fullsized":"2023-11-26T07:11:29.540720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:02.252744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.920884+0000","last_change":"2023-11-26T07:11:28.544445+0000","last_active":"2023-11-26T07:11:30.920884+0000","last_peered":"2023-11-26T07:11:30.920884+0000","last_clean":"2023-11-26T07:11:30.920884+0000","last_became_active":"2023-11-26T07:11:28.544146+0000","last_became_peered":"2023-11-26T07:11:28.544146+0000","last_unstale":"2023-11-26T07:11:30.920884+0000","last_undegraded":"2023-11-26T07:11:30.920884+0000","last_fullsized":"2023-11-26T07:11:30.920884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:31.219688+0000","last_change":"2023-11-26T07:11:28.542940+0000","last_active":"2023-11-26T07:11:31.219688+0000","last_peered":"2023-11-26T07:11:31.219688+0000","last_clean":"2023-11-26T07:11:31.219688+0000","last_became_active":"2023-11-26T07:11:28.542607+0000","last_became_peered":"2023-11-26T07:11:28.542607+0000","last_unstale":"2023-11-26T07:11:31.219688+0000","last_undegraded":"2023-11-26T07:11:31.219688+0000","last_fullsized":"2023-11-26T07:11:31.219688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:32.075747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719379+0000","last_change":"2023-11-26T07:11:28.549728+0000","last_active":"2023-11-26T07:11:30.719379+0000","last_peered":"2023-11-26T07:11:30.719379+0000","last_clean":"2023-11-26T07:11:30.719379+0000","last_became_active":"2023-11-26T07:11:28.543537+0000","last_became_peered":"2023-11-26T07:11:28.543537+0000","last_unstale":"2023-11-26T07:11:30.719379+0000","last_undegraded":"2023-11-26T07:11:30.719379+0000","last_fullsized":"2023-11-26T07:11:30.719379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:24:47.640100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189278+0000","last_change":"2023-11-26T07:11:28.546921+0000","last_active":"2023-11-26T07:11:30.189278+0000","last_peered":"2023-11-26T07:11:30.189278+0000","last_clean":"2023-11-26T07:11:30.189278+0000","last_became_active":"2023-11-26T07:11:28.546543+0000","last_became_peered":"2023-11-26T07:11:28.546543+0000","last_unstale":"2023-11-26T07:11:30.189278+0000","last_undegraded":"2023-11-26T07:11:30.189278+0000","last_fullsized":"2023-11-26T07:11:30.189278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850701+0000","last_change":"2023-11-26T07:11:28.549516+0000","last_active":"2023-11-26T07:11:30.850701+0000","last_peered":"2023-11-26T07:11:30.850701+0000","last_clean":"2023-11-26T07:11:30.850701+0000","last_became_active":"2023-11-26T07:11:28.548917+0000","last_became_peered":"2023-11-26T07:11:28.548917+0000","last_unstale":"2023-11-26T07:11:30.850701+0000","last_undegraded":"2023-11-26T07:11:30.850701+0000","last_fullsized":"2023-11-26T07:11:30.850701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:14.369275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537692+0000","last_change":"2023-11-26T07:11:28.541342+0000","last_active":"2023-11-26T07:11:29.537692+0000","last_peered":"2023-11-26T07:11:29.537692+0000","last_clean":"2023-11-26T07:11:29.537692+0000","last_became_active":"2023-11-26T07:11:28.540919+0000","last_became_peered":"2023-11-26T07:11:28.540919+0000","last_unstale":"2023-11-26T07:11:29.537692+0000","last_undegraded":"2023-11-26T07:11:29.537692+0000","last_fullsized":"2023-11-26T07:11:29.537692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.921212+0000","last_change":"2023-11-26T07:11:28.551636+0000","last_active":"2023-11-26T07:11:30.921212+0000","last_peered":"2023-11-26T07:11:30.921212+0000","last_clean":"2023-11-26T07:11:30.921212+0000","last_became_active":"2023-11-26T07:11:28.551335+0000","last_became_peered":"2023-11-26T07:11:28.551335+0000","last_unstale":"2023-11-26T07:11:30.921212+0000","last_undegraded":"2023-11-26T07:11:30.921212+0000","last_fullsized":"2023-11-26T07:11:30.921212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850489+0000","last_change":"2023-11-26T07:11:28.546402+0000","last_active":"2023-11-26T07:11:30.850489+0000","last_peered":"2023-11-26T07:11:30.850489+0000","last_clean":"2023-11-26T07:11:30.850489+0000","last_became_active":"2023-11-26T07:11:28.546132+0000","last_became_peered":"2023-11-26T07:11:28.546132+0000","last_unstale":"2023-11-26T07:11:30.850489+0000","last_undegraded":"2023-11-26T07:11:30.850489+0000","last_fullsized":"2023-11-26T07:11:30.850489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719685+0000","last_change":"2023-11-26T07:11:28.548148+0000","last_active":"2023-11-26T07:11:30.719685+0000","last_peered":"2023-11-26T07:11:30.719685+0000","last_clean":"2023-11-26T07:11:30.719685+0000","last_became_active":"2023-11-26T07:11:28.546933+0000","last_became_peered":"2023-11-26T07:11:28.546933+0000","last_unstale":"2023-11-26T07:11:30.719685+0000","last_undegraded":"2023-11-26T07:11:30.719685+0000","last_fullsized":"2023-11-26T07:11:30.719685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:48.717411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850378+0000","last_change":"2023-11-26T07:11:28.549377+0000","last_active":"2023-11-26T07:11:30.850378+0000","last_peered":"2023-11-26T07:11:30.850378+0000","last_clean":"2023-11-26T07:11:30.850378+0000","last_became_active":"2023-11-26T07:11:28.549093+0000","last_became_peered":"2023-11-26T07:11:28.549093+0000","last_unstale":"2023-11-26T07:11:30.850378+0000","last_undegraded":"2023-11-26T07:11:30.850378+0000","last_fullsized":"2023-11-26T07:11:30.850378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719517+0000","last_change":"2023-11-26T07:11:28.554891+0000","last_active":"2023-11-26T07:11:30.719517+0000","last_peered":"2023-11-26T07:11:30.719517+0000","last_clean":"2023-11-26T07:11:30.719517+0000","last_became_active":"2023-11-26T07:11:28.554617+0000","last_became_peered":"2023-11-26T07:11:28.554617+0000","last_unstale":"2023-11-26T07:11:30.719517+0000","last_undegraded":"2023-11-26T07:11:30.719517+0000","last_fullsized":"2023-11-26T07:11:30.719517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:14.369275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189270+0000","last_change":"2023-11-26T07:11:28.552756+0000","last_active":"2023-11-26T07:11:30.189270+0000","last_peered":"2023-11-26T07:11:30.189270+0000","last_clean":"2023-11-26T07:11:30.189270+0000","last_became_active":"2023-11-26T07:11:28.552548+0000","last_became_peered":"2023-11-26T07:11:28.552548+0000","last_unstale":"2023-11-26T07:11:30.189270+0000","last_undegraded":"2023-11-26T07:11:30.189270+0000","last_fullsized":"2023-11-26T07:11:30.189270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.921199+0000","last_change":"2023-11-26T07:11:28.547396+0000","last_active":"2023-11-26T07:11:30.921199+0000","last_peered":"2023-11-26T07:11:30.921199+0000","last_clean":"2023-11-26T07:11:30.921199+0000","last_became_active":"2023-11-26T07:11:28.547176+0000","last_became_peered":"2023-11-26T07:11:28.547176+0000","last_unstale":"2023-11-26T07:11:30.921199+0000","last_undegraded":"2023-11-26T07:11:30.921199+0000","last_fullsized":"2023-11-26T07:11:30.921199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:11:37.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:11:37.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.401+0000 7f2cc0363700 1 Processor -- start 2023-11-26T07:11:37.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.401+0000 7f2cc0363700 1 -- start start 2023-11-26T07:11:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cc0363700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cb815bdd0 0x7f2cb81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cc0363700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2cb80b3370 0x7f2cb80b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cc0363700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb80b2030 con 0x7f2cb80d1c70 2023-11-26T07:11:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cc0363700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb80b2f50 con 0x7f2cb80b3370 2023-11-26T07:11:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cc0363700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80b22d0 con 0x7f2cb815bdd0 2023-11-26T07:11:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cbd8fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2cb80b3370 0x7f2cb80b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cbd8fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2cb80b3370 0x7f2cb80b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34576/0 (socket says 172.21.15.83:34576) 2023-11-26T07:11:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cbd8fe700 1 -- 172.21.15.83:0/3522511081 learned_addr learned my addr 172.21.15.83:0/3522511081 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cbe0ff700 1 --2- 172.21.15.83:0/3522511081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cb815bdd0 0x7f2cb81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cbd8fe700 1 -- 172.21.15.83:0/3522511081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cb815bdd0 msgr2=0x7f2cb81604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:37.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cbd8fe700 1 --2- 172.21.15.83:0/3522511081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cb815bdd0 0x7f2cb81604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cbd8fe700 1 -- 172.21.15.83:0/3522511081 >> v1:172.21.15.203:6789/0 conn(0x7f2cb80d1c70 legacy=0x7f2cb80ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cbd8fe700 1 -- 172.21.15.83:0/3522511081 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb80b25b0 con 0x7f2cb80b3370 2023-11-26T07:11:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cbd8fe700 1 --2- 172.21.15.83:0/3522511081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80b3370 0x7f2cb80b1af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80098f0 tx=0x7f2ca8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421e6ee92f8e94c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.402+0000 7f2cbd0fd700 1 -- 172.21.15.83:0/3522511081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8004230 con 0x7f2cb80b3370 2023-11-26T07:11:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.403+0000 7f2cbd0fd700 1 -- 172.21.15.83:0/3522511081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca8004390 con 0x7f2cb80b3370 2023-11-26T07:11:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.403+0000 7f2cbd0fd700 1 -- 172.21.15.83:0/3522511081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80036a0 con 0x7f2cb80b3370 2023-11-26T07:11:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.403+0000 7f2cc0363700 1 -- 172.21.15.83:0/3522511081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80b3370 msgr2=0x7f2cb80b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.403+0000 7f2cc0363700 1 --2- 172.21.15.83:0/3522511081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80b3370 0x7f2cb80b1af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80098f0 tx=0x7f2ca8009c00 comp rx=0 tx=0).stop 2023-11-26T07:11:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.403+0000 7f2cc0363700 1 -- 172.21.15.83:0/3522511081 shutdown_connections 2023-11-26T07:11:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.403+0000 7f2cc0363700 1 --2- 172.21.15.83:0/3522511081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cb815bdd0 0x7f2cb81604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.403+0000 7f2cc0363700 1 --2- 172.21.15.83:0/3522511081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80b3370 0x7f2cb80b1af0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.403+0000 7f2cc0363700 1 -- 172.21.15.83:0/3522511081 >> 172.21.15.83:0/3522511081 conn(0x7f2cb80731a0 msgr2=0x7f2cb809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:37.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.403+0000 7f2cc0363700 1 -- 172.21.15.83:0/3522511081 shutdown_connections 2023-11-26T07:11:37.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.403+0000 7f2cc0363700 1 -- 172.21.15.83:0/3522511081 wait complete. 2023-11-26T07:11:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.404+0000 7f2cc0363700 1 Processor -- start 2023-11-26T07:11:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.404+0000 7f2cc0363700 1 -- start start 2023-11-26T07:11:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cc0363700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80b3370 0x7f2cb81e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cc0363700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cb80d1c70 0x7f2cb80cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cc0363700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cb815bdd0 0x7f2cb80cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cc0363700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb81e60a0 con 0x7f2cb80b3370 2023-11-26T07:11:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cc0363700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80cae10 con 0x7f2cb815bdd0 2023-11-26T07:11:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cc0363700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80cecb0 con 0x7f2cb80d1c70 2023-11-26T07:11:37.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cbe0ff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80b3370 0x7f2cb81e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cbe0ff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80b3370 0x7f2cb81e5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34582/0 (socket says 172.21.15.83:34582) 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cbe0ff700 1 -- 172.21.15.83:0/4210141600 learned_addr learned my addr 172.21.15.83:0/4210141600 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cbe0ff700 1 -- 172.21.15.83:0/4210141600 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cb80d1c70 msgr2=0x7f2cb80cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cbe0ff700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cb80d1c70 0x7f2cb80cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cbe0ff700 1 -- 172.21.15.83:0/4210141600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cb815bdd0 msgr2=0x7f2cb80cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cbe0ff700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cb815bdd0 0x7f2cb80cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cbe0ff700 1 -- 172.21.15.83:0/4210141600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca8009550 con 0x7f2cb80b3370 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.405+0000 7f2cbe0ff700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80b3370 0x7f2cb81e5b60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400c7f0 tx=0x7f2cb400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.406+0000 7f2caeffd700 1 -- 172.21.15.83:0/4210141600 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4011800 con 0x7f2cb80b3370 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.406+0000 7f2caeffd700 1 -- 172.21.15.83:0/4210141600 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb4011960 con 0x7f2cb80b3370 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.406+0000 7f2cc0363700 1 -- 172.21.15.83:0/4210141600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb806b190 con 0x7f2cb80b3370 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.406+0000 7f2caeffd700 1 -- 172.21.15.83:0/4210141600 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4011d10 con 0x7f2cb80b3370 2023-11-26T07:11:37.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.406+0000 7f2cc0363700 1 -- 172.21.15.83:0/4210141600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb806b750 con 0x7f2cb80b3370 2023-11-26T07:11:37.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.407+0000 7f2cc0363700 1 -- 172.21.15.83:0/4210141600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb8067f60 con 0x7f2cb80b3370 2023-11-26T07:11:37.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.407+0000 7f2caeffd700 1 -- 172.21.15.83:0/4210141600 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2cb4018360 con 0x7f2cb80b3370 2023-11-26T07:11:37.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.407+0000 7f2caeffd700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2c9c06aed0 0x7f2c9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.407+0000 7f2cbd8fe700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2c9c06aed0 0x7f2c9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.407+0000 7f2caeffd700 1 -- 172.21.15.83:0/4210141600 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cb4089590 con 0x7f2cb80b3370 2023-11-26T07:11:37.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.408+0000 7f2cbd8fe700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2c9c06aed0 0x7f2c9c06d390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009c40 tx=0x7f2ca8010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:37.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.412+0000 7f2caeffd700 1 -- 172.21.15.83:0/4210141600 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2cb4054640 con 0x7f2cb80b3370 2023-11-26T07:11:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.597+0000 7f2cc0363700 1 -- 172.21.15.83:0/4210141600 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2cb80b2b70 con 0x7f2c9c06aed0 2023-11-26T07:11:37.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.604+0000 7f2caeffd700 1 -- 172.21.15.83:0/4210141600 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264521 (secure 0 0 0) 0x7f2ca801f070 con 0x7f2c9c06aed0 2023-11-26T07:11:37.609 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:37.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 -- 172.21.15.83:0/4210141600 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2c9c06aed0 msgr2=0x7f2c9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2c9c06aed0 0x7f2c9c06d390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009c40 tx=0x7f2ca8010000 comp rx=0 tx=0).stop 2023-11-26T07:11:37.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 -- 172.21.15.83:0/4210141600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80b3370 msgr2=0x7f2cb81e5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80b3370 0x7f2cb81e5b60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400c7f0 tx=0x7f2cb400cbb0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 -- 172.21.15.83:0/4210141600 shutdown_connections 2023-11-26T07:11:37.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cb815bdd0 0x7f2cb80cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80b3370 0x7f2cb81e5b60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2c9c06aed0 0x7f2c9c06d390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 --2- 172.21.15.83:0/4210141600 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cb80d1c70 0x7f2cb80cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 -- 172.21.15.83:0/4210141600 >> 172.21.15.83:0/4210141600 conn(0x7f2cb80731a0 msgr2=0x7f2cb80b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 -- 172.21.15.83:0/4210141600 shutdown_connections 2023-11-26T07:11:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.608+0000 7f2cc0363700 1 -- 172.21.15.83:0/4210141600 wait complete. 2023-11-26T07:11:37.612 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:11:37.621 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-26T07:11:36.309392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219108,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755612,"statfs":{"total":773094113280,"available":772869746688,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002297"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540665+0000","last_change":"2023-11-26T07:11:28.537635+0000","last_active":"2023-11-26T07:11:29.540665+0000","last_peered":"2023-11-26T07:11:29.540665+0000","last_clean":"2023-11-26T07:11:29.540665+0000","last_became_active":"2023-11-26T07:11:28.537327+0000","last_became_peered":"2023-11-26T07:11:28.537327+0000","last_unstale":"2023-11-26T07:11:29.540665+0000","last_undegraded":"2023-11-26T07:11:29.540665+0000","last_fullsized":"2023-11-26T07:11:29.540665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:48:59.913650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537894+0000","last_change":"2023-11-26T07:11:28.541611+0000","last_active":"2023-11-26T07:11:29.537894+0000","last_peered":"2023-11-26T07:11:29.537894+0000","last_clean":"2023-11-26T07:11:29.537894+0000","last_became_active":"2023-11-26T07:11:28.541267+0000","last_became_peered":"2023-11-26T07:11:28.541267+0000","last_unstale":"2023-11-26T07:11:29.537894+0000","last_undegraded":"2023-11-26T07:11:29.537894+0000","last_fullsized":"2023-11-26T07:11:29.537894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189176+0000","last_change":"2023-11-26T07:11:28.543463+0000","last_active":"2023-11-26T07:11:30.189176+0000","last_peered":"2023-11-26T07:11:30.189176+0000","last_clean":"2023-11-26T07:11:30.189176+0000","last_became_active":"2023-11-26T07:11:28.542718+0000","last_became_peered":"2023-11-26T07:11:28.542718+0000","last_unstale":"2023-11-26T07:11:30.189176+0000","last_undegraded":"2023-11-26T07:11:30.189176+0000","last_fullsized":"2023-11-26T07:11:30.189176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719807+0000","last_change":"2023-11-26T07:11:28.544447+0000","last_active":"2023-11-26T07:11:30.719807+0000","last_peered":"2023-11-26T07:11:30.719807+0000","last_clean":"2023-11-26T07:11:30.719807+0000","last_became_active":"2023-11-26T07:11:28.543689+0000","last_became_peered":"2023-11-26T07:11:28.543689+0000","last_unstale":"2023-11-26T07:11:30.719807+0000","last_undegraded":"2023-11-26T07:11:30.719807+0000","last_fullsized":"2023-11-26T07:11:30.719807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:36:40.660123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540289+0000","last_change":"2023-11-26T07:11:28.542518+0000","last_active":"2023-11-26T07:11:29.540289+0000","last_peered":"2023-11-26T07:11:29.540289+0000","last_clean":"2023-11-26T07:11:29.540289+0000","last_became_active":"2023-11-26T07:11:28.542255+0000","last_became_peered":"2023-11-26T07:11:28.542255+0000","last_unstale":"2023-11-26T07:11:29.540289+0000","last_undegraded":"2023-11-26T07:11:29.540289+0000","last_fullsized":"2023-11-26T07:11:29.540289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:48.717411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719213+0000","last_change":"2023-11-26T07:11:28.550053+0000","last_active":"2023-11-26T07:11:30.719213+0000","last_peered":"2023-11-26T07:11:30.719213+0000","last_clean":"2023-11-26T07:11:30.719213+0000","last_became_active":"2023-11-26T07:11:28.549624+0000","last_became_peered":"2023-11-26T07:11:28.549624+0000","last_unstale":"2023-11-26T07:11:30.719213+0000","last_undegraded":"2023-11-26T07:11:30.719213+0000","last_fullsized":"2023-11-26T07:11:30.719213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:42.974584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539979+0000","last_change":"2023-11-26T07:11:28.543140+0000","last_active":"2023-11-26T07:11:29.539979+0000","last_peered":"2023-11-26T07:11:29.539979+0000","last_clean":"2023-11-26T07:11:29.539979+0000","last_became_active":"2023-11-26T07:11:28.542835+0000","last_became_peered":"2023-11-26T07:11:28.542835+0000","last_unstale":"2023-11-26T07:11:29.539979+0000","last_undegraded":"2023-11-26T07:11:29.539979+0000","last_fullsized":"2023-11-26T07:11:29.539979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:14.369275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850216+0000","last_change":"2023-11-26T07:11:28.554561+0000","last_active":"2023-11-26T07:11:30.850216+0000","last_peered":"2023-11-26T07:11:30.850216+0000","last_clean":"2023-11-26T07:11:30.850216+0000","last_became_active":"2023-11-26T07:11:28.554273+0000","last_became_peered":"2023-11-26T07:11:28.554273+0000","last_unstale":"2023-11-26T07:11:30.850216+0000","last_undegraded":"2023-11-26T07:11:30.850216+0000","last_fullsized":"2023-11-26T07:11:30.850216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540623+0000","last_change":"2023-11-26T07:11:28.539323+0000","last_active":"2023-11-26T07:11:29.540623+0000","last_peered":"2023-11-26T07:11:29.540623+0000","last_clean":"2023-11-26T07:11:29.540623+0000","last_became_active":"2023-11-26T07:11:28.539047+0000","last_became_peered":"2023-11-26T07:11:28.539047+0000","last_unstale":"2023-11-26T07:11:29.540623+0000","last_undegraded":"2023-11-26T07:11:29.540623+0000","last_fullsized":"2023-11-26T07:11:29.540623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:01:36.280118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.921372+0000","last_change":"2023-11-26T07:11:28.538383+0000","last_active":"2023-11-26T07:11:30.921372+0000","last_peered":"2023-11-26T07:11:30.921372+0000","last_clean":"2023-11-26T07:11:30.921372+0000","last_became_active":"2023-11-26T07:11:28.537726+0000","last_became_peered":"2023-11-26T07:11:28.537726+0000","last_unstale":"2023-11-26T07:11:30.921372+0000","last_undegraded":"2023-11-26T07:11:30.921372+0000","last_fullsized":"2023-11-26T07:11:30.921372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:42.974584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.921271+0000","last_change":"2023-11-26T07:11:28.538846+0000","last_active":"2023-11-26T07:11:30.921271+0000","last_peered":"2023-11-26T07:11:30.921271+0000","last_clean":"2023-11-26T07:11:30.921271+0000","last_became_active":"2023-11-26T07:11:28.537769+0000","last_became_peered":"2023-11-26T07:11:28.537769+0000","last_unstale":"2023-11-26T07:11:30.921271+0000","last_undegraded":"2023-11-26T07:11:30.921271+0000","last_fullsized":"2023-11-26T07:11:30.921271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539796+0000","last_change":"2023-11-26T07:11:28.541985+0000","last_active":"2023-11-26T07:11:29.539796+0000","last_peered":"2023-11-26T07:11:29.539796+0000","last_clean":"2023-11-26T07:11:29.539796+0000","last_became_active":"2023-11-26T07:11:28.541404+0000","last_became_peered":"2023-11-26T07:11:28.541404+0000","last_unstale":"2023-11-26T07:11:29.539796+0000","last_undegraded":"2023-11-26T07:11:29.539796+0000","last_fullsized":"2023-11-26T07:11:29.539796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:11.545490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850419+0000","last_change":"2023-11-26T07:11:28.551585+0000","last_active":"2023-11-26T07:11:30.850419+0000","last_peered":"2023-11-26T07:11:30.850419+0000","last_clean":"2023-11-26T07:11:30.850419+0000","last_became_active":"2023-11-26T07:11:28.551296+0000","last_became_peered":"2023-11-26T07:11:28.551296+0000","last_unstale":"2023-11-26T07:11:30.850419+0000","last_undegraded":"2023-11-26T07:11:30.850419+0000","last_fullsized":"2023-11-26T07:11:30.850419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189126+0000","last_change":"2023-11-26T07:11:28.541346+0000","last_active":"2023-11-26T07:11:30.189126+0000","last_peered":"2023-11-26T07:11:30.189126+0000","last_clean":"2023-11-26T07:11:30.189126+0000","last_became_active":"2023-11-26T07:11:28.540961+0000","last_became_peered":"2023-11-26T07:11:28.540961+0000","last_unstale":"2023-11-26T07:11:30.189126+0000","last_undegraded":"2023-11-26T07:11:30.189126+0000","last_fullsized":"2023-11-26T07:11:30.189126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:48.999647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537528+0000","last_change":"2023-11-26T07:11:28.536348+0000","last_active":"2023-11-26T07:11:29.537528+0000","last_peered":"2023-11-26T07:11:29.537528+0000","last_clean":"2023-11-26T07:11:29.537528+0000","last_became_active":"2023-11-26T07:11:28.535391+0000","last_became_peered":"2023-11-26T07:11:28.535391+0000","last_unstale":"2023-11-26T07:11:29.537528+0000","last_undegraded":"2023-11-26T07:11:29.537528+0000","last_fullsized":"2023-11-26T07:11:29.537528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719167+0000","last_change":"2023-11-26T07:11:28.544579+0000","last_active":"2023-11-26T07:11:30.719167+0000","last_peered":"2023-11-26T07:11:30.719167+0000","last_clean":"2023-11-26T07:11:30.719167+0000","last_became_active":"2023-11-26T07:11:28.542480+0000","last_became_peered":"2023-11-26T07:11:28.542480+0000","last_unstale":"2023-11-26T07:11:30.719167+0000","last_undegraded":"2023-11-26T07:11:30.719167+0000","last_fullsized":"2023-11-26T07:11:30.719167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719934+0000","last_change":"2023-11-26T07:11:28.554495+0000","last_active":"2023-11-26T07:11:30.719934+0000","last_peered":"2023-11-26T07:11:30.719934+0000","last_clean":"2023-11-26T07:11:30.719934+0000","last_became_active":"2023-11-26T07:11:28.554114+0000","last_became_peered":"2023-11-26T07:11:28.554114+0000","last_unstale":"2023-11-26T07:11:30.719934+0000","last_undegraded":"2023-11-26T07:11:30.719934+0000","last_fullsized":"2023-11-26T07:11:30.719934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:01:36.280118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:31.219855+0000","last_change":"2023-11-26T07:11:28.544698+0000","last_active":"2023-11-26T07:11:31.219855+0000","last_peered":"2023-11-26T07:11:31.219855+0000","last_clean":"2023-11-26T07:11:31.219855+0000","last_became_active":"2023-11-26T07:11:28.544347+0000","last_became_peered":"2023-11-26T07:11:28.544347+0000","last_unstale":"2023-11-26T07:11:31.219855+0000","last_undegraded":"2023-11-26T07:11:31.219855+0000","last_fullsized":"2023-11-26T07:11:31.219855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:11.545490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540518+0000","last_change":"2023-11-26T07:11:28.550524+0000","last_active":"2023-11-26T07:11:29.540518+0000","last_peered":"2023-11-26T07:11:29.540518+0000","last_clean":"2023-11-26T07:11:29.540518+0000","last_became_active":"2023-11-26T07:11:28.543803+0000","last_became_peered":"2023-11-26T07:11:28.543803+0000","last_unstale":"2023-11-26T07:11:29.540518+0000","last_undegraded":"2023-11-26T07:11:29.540518+0000","last_fullsized":"2023-11-26T07:11:29.540518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:36:40.660123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189251+0000","last_change":"2023-11-26T07:11:27.524720+0000","last_active":"2023-11-26T07:11:30.189251+0000","last_peered":"2023-11-26T07:11:30.189251+0000","last_clean":"2023-11-26T07:11:30.189251+0000","last_became_active":"2023-11-26T07:11:27.524424+0000","last_became_peered":"2023-11-26T07:11:27.524424+0000","last_unstale":"2023-11-26T07:11:30.189251+0000","last_undegraded":"2023-11-26T07:11:30.189251+0000","last_fullsized":"2023-11-26T07:11:30.189251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850692+0000","last_change":"2023-11-26T07:11:28.540843+0000","last_active":"2023-11-26T07:11:30.850692+0000","last_peered":"2023-11-26T07:11:30.850692+0000","last_clean":"2023-11-26T07:11:30.850692+0000","last_became_active":"2023-11-26T07:11:28.540579+0000","last_became_peered":"2023-11-26T07:11:28.540579+0000","last_unstale":"2023-11-26T07:11:30.850692+0000","last_undegraded":"2023-11-26T07:11:30.850692+0000","last_fullsized":"2023-11-26T07:11:30.850692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:13.394314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.320406+0000","last_change":"2023-11-26T07:11:27.528053+0000","last_active":"2023-11-26T07:11:30.320406+0000","last_peered":"2023-11-26T07:11:30.320406+0000","last_clean":"2023-11-26T07:11:30.320406+0000","last_became_active":"2023-11-26T07:11:27.526741+0000","last_became_peered":"2023-11-26T07:11:27.526741+0000","last_unstale":"2023-11-26T07:11:30.320406+0000","last_undegraded":"2023-11-26T07:11:30.320406+0000","last_fullsized":"2023-11-26T07:11:30.320406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:31.071342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189238+0000","last_change":"2023-11-26T07:11:28.551779+0000","last_active":"2023-11-26T07:11:30.189238+0000","last_peered":"2023-11-26T07:11:30.189238+0000","last_clean":"2023-11-26T07:11:30.189238+0000","last_became_active":"2023-11-26T07:11:28.551489+0000","last_became_peered":"2023-11-26T07:11:28.551489+0000","last_unstale":"2023-11-26T07:11:30.189238+0000","last_undegraded":"2023-11-26T07:11:30.189238+0000","last_fullsized":"2023-11-26T07:11:30.189238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540841+0000","last_change":"2023-11-26T07:11:27.530905+0000","last_active":"2023-11-26T07:11:29.540841+0000","last_peered":"2023-11-26T07:11:29.540841+0000","last_clean":"2023-11-26T07:11:29.540841+0000","last_became_active":"2023-11-26T07:11:27.529568+0000","last_became_peered":"2023-11-26T07:11:27.529568+0000","last_unstale":"2023-11-26T07:11:29.540841+0000","last_undegraded":"2023-11-26T07:11:29.540841+0000","last_fullsized":"2023-11-26T07:11:29.540841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:12.389909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719541+0000","last_change":"2023-11-26T07:11:28.547553+0000","last_active":"2023-11-26T07:11:30.719541+0000","last_peered":"2023-11-26T07:11:30.719541+0000","last_clean":"2023-11-26T07:11:30.719541+0000","last_became_active":"2023-11-26T07:11:28.547052+0000","last_became_peered":"2023-11-26T07:11:28.547052+0000","last_unstale":"2023-11-26T07:11:30.719541+0000","last_undegraded":"2023-11-26T07:11:30.719541+0000","last_fullsized":"2023-11-26T07:11:30.719541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:58.985237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540344+0000","last_change":"2023-11-26T07:11:27.528392+0000","last_active":"2023-11-26T07:11:29.540344+0000","last_peered":"2023-11-26T07:11:29.540344+0000","last_clean":"2023-11-26T07:11:29.540344+0000","last_became_active":"2023-11-26T07:11:27.528039+0000","last_became_peered":"2023-11-26T07:11:27.528039+0000","last_unstale":"2023-11-26T07:11:29.540344+0000","last_undegraded":"2023-11-26T07:11:29.540344+0000","last_fullsized":"2023-11-26T07:11:29.540344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719504+0000","last_change":"2023-11-26T07:11:28.546262+0000","last_active":"2023-11-26T07:11:30.719504+0000","last_peered":"2023-11-26T07:11:30.719504+0000","last_clean":"2023-11-26T07:11:30.719504+0000","last_became_active":"2023-11-26T07:11:28.545998+0000","last_became_peered":"2023-11-26T07:11:28.545998+0000","last_unstale":"2023-11-26T07:11:30.719504+0000","last_undegraded":"2023-11-26T07:11:30.719504+0000","last_fullsized":"2023-11-26T07:11:30.719504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:13.394314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.320372+0000","last_change":"2023-11-26T07:11:27.528591+0000","last_active":"2023-11-26T07:11:30.320372+0000","last_peered":"2023-11-26T07:11:30.320372+0000","last_clean":"2023-11-26T07:11:30.320372+0000","last_became_active":"2023-11-26T07:11:27.528327+0000","last_became_peered":"2023-11-26T07:11:27.528327+0000","last_unstale":"2023-11-26T07:11:30.320372+0000","last_undegraded":"2023-11-26T07:11:30.320372+0000","last_fullsized":"2023-11-26T07:11:30.320372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:21.506925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539218+0000","last_change":"2023-11-26T07:11:28.541089+0000","last_active":"2023-11-26T07:11:29.539218+0000","last_peered":"2023-11-26T07:11:29.539218+0000","last_clean":"2023-11-26T07:11:29.539218+0000","last_became_active":"2023-11-26T07:11:28.534262+0000","last_became_peered":"2023-11-26T07:11:28.534262+0000","last_unstale":"2023-11-26T07:11:29.539218+0000","last_undegraded":"2023-11-26T07:11:29.539218+0000","last_fullsized":"2023-11-26T07:11:29.539218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:33.426915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.331849+0000","last_change":"2023-11-26T07:11:27.530892+0000","last_active":"2023-11-26T07:11:30.331849+0000","last_peered":"2023-11-26T07:11:30.331849+0000","last_clean":"2023-11-26T07:11:30.331849+0000","last_became_active":"2023-11-26T07:11:27.529853+0000","last_became_peered":"2023-11-26T07:11:27.529853+0000","last_unstale":"2023-11-26T07:11:30.331849+0000","last_undegraded":"2023-11-26T07:11:30.331849+0000","last_fullsized":"2023-11-26T07:11:30.331849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189422+0000","last_change":"2023-11-26T07:11:28.548912+0000","last_active":"2023-11-26T07:11:30.189422+0000","last_peered":"2023-11-26T07:11:30.189422+0000","last_clean":"2023-11-26T07:11:30.189422+0000","last_became_active":"2023-11-26T07:11:28.547915+0000","last_became_peered":"2023-11-26T07:11:28.547915+0000","last_unstale":"2023-11-26T07:11:30.189422+0000","last_undegraded":"2023-11-26T07:11:30.189422+0000","last_fullsized":"2023-11-26T07:11:30.189422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:42.974584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.320300+0000","last_change":"2023-11-26T07:11:27.529609+0000","last_active":"2023-11-26T07:11:30.320300+0000","last_peered":"2023-11-26T07:11:30.320300+0000","last_clean":"2023-11-26T07:11:30.320300+0000","last_became_active":"2023-11-26T07:11:27.529045+0000","last_became_peered":"2023-11-26T07:11:27.529045+0000","last_unstale":"2023-11-26T07:11:30.320300+0000","last_undegraded":"2023-11-26T07:11:30.320300+0000","last_fullsized":"2023-11-26T07:11:30.320300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:18.223957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850312+0000","last_change":"2023-11-26T07:11:28.543403+0000","last_active":"2023-11-26T07:11:30.850312+0000","last_peered":"2023-11-26T07:11:30.850312+0000","last_clean":"2023-11-26T07:11:30.850312+0000","last_became_active":"2023-11-26T07:11:28.543119+0000","last_became_peered":"2023-11-26T07:11:28.543119+0000","last_unstale":"2023-11-26T07:11:30.850312+0000","last_undegraded":"2023-11-26T07:11:30.850312+0000","last_fullsized":"2023-11-26T07:11:30.850312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.319713+0000","last_change":"2023-11-26T07:11:27.528106+0000","last_active":"2023-11-26T07:11:30.319713+0000","last_peered":"2023-11-26T07:11:30.319713+0000","last_clean":"2023-11-26T07:11:30.319713+0000","last_became_active":"2023-11-26T07:11:27.527832+0000","last_became_peered":"2023-11-26T07:11:27.527832+0000","last_unstale":"2023-11-26T07:11:30.319713+0000","last_undegraded":"2023-11-26T07:11:30.319713+0000","last_fullsized":"2023-11-26T07:11:30.319713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:50.240018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537690+0000","last_change":"2023-11-26T07:11:28.543380+0000","last_active":"2023-11-26T07:11:29.537690+0000","last_peered":"2023-11-26T07:11:29.537690+0000","last_clean":"2023-11-26T07:11:29.537690+0000","last_became_active":"2023-11-26T07:11:28.543083+0000","last_became_peered":"2023-11-26T07:11:28.543083+0000","last_unstale":"2023-11-26T07:11:29.537690+0000","last_undegraded":"2023-11-26T07:11:29.537690+0000","last_fullsized":"2023-11-26T07:11:29.537690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.331947+0000","last_change":"2023-11-26T07:11:27.531555+0000","last_active":"2023-11-26T07:11:30.331947+0000","last_peered":"2023-11-26T07:11:30.331947+0000","last_clean":"2023-11-26T07:11:30.331947+0000","last_became_active":"2023-11-26T07:11:27.531241+0000","last_became_peered":"2023-11-26T07:11:27.531241+0000","last_unstale":"2023-11-26T07:11:30.331947+0000","last_undegraded":"2023-11-26T07:11:30.331947+0000","last_fullsized":"2023-11-26T07:11:30.331947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719839+0000","last_change":"2023-11-26T07:11:28.552112+0000","last_active":"2023-11-26T07:11:30.719839+0000","last_peered":"2023-11-26T07:11:30.719839+0000","last_clean":"2023-11-26T07:11:30.719839+0000","last_became_active":"2023-11-26T07:11:28.551827+0000","last_became_peered":"2023-11-26T07:11:28.551827+0000","last_unstale":"2023-11-26T07:11:30.719839+0000","last_undegraded":"2023-11-26T07:11:30.719839+0000","last_fullsized":"2023-11-26T07:11:30.719839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:51.665872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.319891+0000","last_change":"2023-11-26T07:11:27.531668+0000","last_active":"2023-11-26T07:11:30.319891+0000","last_peered":"2023-11-26T07:11:30.319891+0000","last_clean":"2023-11-26T07:11:30.319891+0000","last_became_active":"2023-11-26T07:11:27.531403+0000","last_became_peered":"2023-11-26T07:11:27.531403+0000","last_unstale":"2023-11-26T07:11:30.319891+0000","last_undegraded":"2023-11-26T07:11:30.319891+0000","last_fullsized":"2023-11-26T07:11:30.319891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189378+0000","last_change":"2023-11-26T07:11:28.542823+0000","last_active":"2023-11-26T07:11:30.189378+0000","last_peered":"2023-11-26T07:11:30.189378+0000","last_clean":"2023-11-26T07:11:30.189378+0000","last_became_active":"2023-11-26T07:11:28.542535+0000","last_became_peered":"2023-11-26T07:11:28.542535+0000","last_unstale":"2023-11-26T07:11:30.189378+0000","last_undegraded":"2023-11-26T07:11:30.189378+0000","last_fullsized":"2023-11-26T07:11:30.189378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537747+0000","last_change":"2023-11-26T07:11:27.531081+0000","last_active":"2023-11-26T07:11:29.537747+0000","last_peered":"2023-11-26T07:11:29.537747+0000","last_clean":"2023-11-26T07:11:29.537747+0000","last_became_active":"2023-11-26T07:11:27.530823+0000","last_became_peered":"2023-11-26T07:11:27.530823+0000","last_unstale":"2023-11-26T07:11:29.537747+0000","last_undegraded":"2023-11-26T07:11:29.537747+0000","last_fullsized":"2023-11-26T07:11:29.537747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:31:21.506925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539660+0000","last_change":"2023-11-26T07:11:28.529675+0000","last_active":"2023-11-26T07:11:29.539660+0000","last_peered":"2023-11-26T07:11:29.539660+0000","last_clean":"2023-11-26T07:11:29.539660+0000","last_became_active":"2023-11-26T07:11:28.529388+0000","last_became_peered":"2023-11-26T07:11:28.529388+0000","last_unstale":"2023-11-26T07:11:29.539660+0000","last_undegraded":"2023-11-26T07:11:29.539660+0000","last_fullsized":"2023-11-26T07:11:29.539660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:32.075747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.331864+0000","last_change":"2023-11-26T07:11:27.534932+0000","last_active":"2023-11-26T07:11:30.331864+0000","last_peered":"2023-11-26T07:11:30.331864+0000","last_clean":"2023-11-26T07:11:30.331864+0000","last_became_active":"2023-11-26T07:11:27.534626+0000","last_became_peered":"2023-11-26T07:11:27.534626+0000","last_unstale":"2023-11-26T07:11:30.331864+0000","last_undegraded":"2023-11-26T07:11:30.331864+0000","last_fullsized":"2023-11-26T07:11:30.331864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:25.399589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540678+0000","last_change":"2023-11-26T07:11:28.538363+0000","last_active":"2023-11-26T07:11:29.540678+0000","last_peered":"2023-11-26T07:11:29.540678+0000","last_clean":"2023-11-26T07:11:29.540678+0000","last_became_active":"2023-11-26T07:11:28.538098+0000","last_became_peered":"2023-11-26T07:11:28.538098+0000","last_unstale":"2023-11-26T07:11:29.540678+0000","last_undegraded":"2023-11-26T07:11:29.540678+0000","last_fullsized":"2023-11-26T07:11:29.540678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:00.284371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189378+0000","last_change":"2023-11-26T07:11:27.522269+0000","last_active":"2023-11-26T07:11:30.189378+0000","last_peered":"2023-11-26T07:11:30.189378+0000","last_clean":"2023-11-26T07:11:30.189378+0000","last_became_active":"2023-11-26T07:11:27.521852+0000","last_became_peered":"2023-11-26T07:11:27.521852+0000","last_unstale":"2023-11-26T07:11:30.189378+0000","last_undegraded":"2023-11-26T07:11:30.189378+0000","last_fullsized":"2023-11-26T07:11:30.189378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:16.557595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189106+0000","last_change":"2023-11-26T07:11:28.552836+0000","last_active":"2023-11-26T07:11:30.189106+0000","last_peered":"2023-11-26T07:11:30.189106+0000","last_clean":"2023-11-26T07:11:30.189106+0000","last_became_active":"2023-11-26T07:11:28.552532+0000","last_became_peered":"2023-11-26T07:11:28.552532+0000","last_unstale":"2023-11-26T07:11:30.189106+0000","last_undegraded":"2023-11-26T07:11:30.189106+0000","last_fullsized":"2023-11-26T07:11:30.189106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:44:51.244423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.319798+0000","last_change":"2023-11-26T07:11:27.534390+0000","last_active":"2023-11-26T07:11:30.319798+0000","last_peered":"2023-11-26T07:11:30.319798+0000","last_clean":"2023-11-26T07:11:30.319798+0000","last_became_active":"2023-11-26T07:11:27.534100+0000","last_became_peered":"2023-11-26T07:11:27.534100+0000","last_unstale":"2023-11-26T07:11:30.319798+0000","last_undegraded":"2023-11-26T07:11:30.319798+0000","last_fullsized":"2023-11-26T07:11:30.319798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:15.199340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540363+0000","last_change":"2023-11-26T07:11:28.530599+0000","last_active":"2023-11-26T07:11:29.540363+0000","last_peered":"2023-11-26T07:11:29.540363+0000","last_clean":"2023-11-26T07:11:29.540363+0000","last_became_active":"2023-11-26T07:11:28.530091+0000","last_became_peered":"2023-11-26T07:11:28.530091+0000","last_unstale":"2023-11-26T07:11:29.540363+0000","last_undegraded":"2023-11-26T07:11:29.540363+0000","last_fullsized":"2023-11-26T07:11:29.540363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:21:12.239221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.331823+0000","last_change":"2023-11-26T07:11:27.530816+0000","last_active":"2023-11-26T07:11:30.331823+0000","last_peered":"2023-11-26T07:11:30.331823+0000","last_clean":"2023-11-26T07:11:30.331823+0000","last_became_active":"2023-11-26T07:11:27.530026+0000","last_became_peered":"2023-11-26T07:11:27.530026+0000","last_unstale":"2023-11-26T07:11:30.331823+0000","last_undegraded":"2023-11-26T07:11:30.331823+0000","last_fullsized":"2023-11-26T07:11:30.331823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:32.422510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.920929+0000","last_change":"2023-11-26T07:11:28.542511+0000","last_active":"2023-11-26T07:11:30.920929+0000","last_peered":"2023-11-26T07:11:30.920929+0000","last_clean":"2023-11-26T07:11:30.920929+0000","last_became_active":"2023-11-26T07:11:28.542221+0000","last_became_peered":"2023-11-26T07:11:28.542221+0000","last_unstale":"2023-11-26T07:11:30.920929+0000","last_undegraded":"2023-11-26T07:11:30.920929+0000","last_fullsized":"2023-11-26T07:11:30.920929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.321930+0000","last_change":"2023-11-26T07:11:27.524724+0000","last_active":"2023-11-26T07:11:30.321930+0000","last_peered":"2023-11-26T07:11:30.321930+0000","last_clean":"2023-11-26T07:11:30.321930+0000","last_became_active":"2023-11-26T07:11:27.524453+0000","last_became_peered":"2023-11-26T07:11:27.524453+0000","last_unstale":"2023-11-26T07:11:30.321930+0000","last_undegraded":"2023-11-26T07:11:30.321930+0000","last_fullsized":"2023-11-26T07:11:30.321930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:32.422510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540122+0000","last_change":"2023-11-26T07:11:28.530396+0000","last_active":"2023-11-26T07:11:29.540122+0000","last_peered":"2023-11-26T07:11:29.540122+0000","last_clean":"2023-11-26T07:11:29.540122+0000","last_became_active":"2023-11-26T07:11:28.530134+0000","last_became_peered":"2023-11-26T07:11:28.530134+0000","last_unstale":"2023-11-26T07:11:29.540122+0000","last_undegraded":"2023-11-26T07:11:29.540122+0000","last_fullsized":"2023-11-26T07:11:29.540122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:58.985237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539626+0000","last_change":"2023-11-26T07:11:27.526573+0000","last_active":"2023-11-26T07:11:29.539626+0000","last_peered":"2023-11-26T07:11:29.539626+0000","last_clean":"2023-11-26T07:11:29.539626+0000","last_became_active":"2023-11-26T07:11:27.526278+0000","last_became_peered":"2023-11-26T07:11:27.526278+0000","last_unstale":"2023-11-26T07:11:29.539626+0000","last_undegraded":"2023-11-26T07:11:29.539626+0000","last_fullsized":"2023-11-26T07:11:29.539626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:35.117462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540624+0000","last_change":"2023-11-26T07:11:28.530307+0000","last_active":"2023-11-26T07:11:29.540624+0000","last_peered":"2023-11-26T07:11:29.540624+0000","last_clean":"2023-11-26T07:11:29.540624+0000","last_became_active":"2023-11-26T07:11:28.530012+0000","last_became_peered":"2023-11-26T07:11:28.530012+0000","last_unstale":"2023-11-26T07:11:29.540624+0000","last_undegraded":"2023-11-26T07:11:29.540624+0000","last_fullsized":"2023-11-26T07:11:29.540624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:51.665872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537662+0000","last_change":"2023-11-26T07:11:27.525592+0000","last_active":"2023-11-26T07:11:29.537662+0000","last_peered":"2023-11-26T07:11:29.537662+0000","last_clean":"2023-11-26T07:11:29.537662+0000","last_became_active":"2023-11-26T07:11:27.525304+0000","last_became_peered":"2023-11-26T07:11:27.525304+0000","last_unstale":"2023-11-26T07:11:29.537662+0000","last_undegraded":"2023-11-26T07:11:29.537662+0000","last_fullsized":"2023-11-26T07:11:29.537662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:52.471590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719811+0000","last_change":"2023-11-26T07:11:28.538490+0000","last_active":"2023-11-26T07:11:30.719811+0000","last_peered":"2023-11-26T07:11:30.719811+0000","last_clean":"2023-11-26T07:11:30.719811+0000","last_became_active":"2023-11-26T07:11:28.538185+0000","last_became_peered":"2023-11-26T07:11:28.538185+0000","last_unstale":"2023-11-26T07:11:30.719811+0000","last_undegraded":"2023-11-26T07:11:30.719811+0000","last_fullsized":"2023-11-26T07:11:30.719811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:21:12.239221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.319783+0000","last_change":"2023-11-26T07:11:27.524225+0000","last_active":"2023-11-26T07:11:30.319783+0000","last_peered":"2023-11-26T07:11:30.319783+0000","last_clean":"2023-11-26T07:11:30.319783+0000","last_became_active":"2023-11-26T07:11:27.523610+0000","last_became_peered":"2023-11-26T07:11:27.523610+0000","last_unstale":"2023-11-26T07:11:30.319783+0000","last_undegraded":"2023-11-26T07:11:30.319783+0000","last_fullsized":"2023-11-26T07:11:30.319783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:19.248482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539423+0000","last_change":"2023-11-26T07:11:28.547971+0000","last_active":"2023-11-26T07:11:29.539423+0000","last_peered":"2023-11-26T07:11:29.539423+0000","last_clean":"2023-11-26T07:11:29.539423+0000","last_became_active":"2023-11-26T07:11:28.547756+0000","last_became_peered":"2023-11-26T07:11:28.547756+0000","last_unstale":"2023-11-26T07:11:29.539423+0000","last_undegraded":"2023-11-26T07:11:29.539423+0000","last_fullsized":"2023-11-26T07:11:29.539423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:57.098847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537553+0000","last_change":"2023-11-26T07:11:28.542253+0000","last_active":"2023-11-26T07:11:29.537553+0000","last_peered":"2023-11-26T07:11:29.537553+0000","last_clean":"2023-11-26T07:11:29.537553+0000","last_became_active":"2023-11-26T07:11:28.541977+0000","last_became_peered":"2023-11-26T07:11:28.541977+0000","last_unstale":"2023-11-26T07:11:29.537553+0000","last_undegraded":"2023-11-26T07:11:29.537553+0000","last_fullsized":"2023-11-26T07:11:29.537553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:50.521188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.321970+0000","last_change":"2023-11-26T07:11:27.534895+0000","last_active":"2023-11-26T07:11:30.321970+0000","last_peered":"2023-11-26T07:11:30.321970+0000","last_clean":"2023-11-26T07:11:30.321970+0000","last_became_active":"2023-11-26T07:11:27.534654+0000","last_became_peered":"2023-11-26T07:11:27.534654+0000","last_unstale":"2023-11-26T07:11:30.321970+0000","last_undegraded":"2023-11-26T07:11:30.321970+0000","last_fullsized":"2023-11-26T07:11:30.321970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:18.223957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850465+0000","last_change":"2023-11-26T07:11:28.544575+0000","last_active":"2023-11-26T07:11:30.850465+0000","last_peered":"2023-11-26T07:11:30.850465+0000","last_clean":"2023-11-26T07:11:30.850465+0000","last_became_active":"2023-11-26T07:11:28.543760+0000","last_became_peered":"2023-11-26T07:11:28.543760+0000","last_unstale":"2023-11-26T07:11:30.850465+0000","last_undegraded":"2023-11-26T07:11:30.850465+0000","last_fullsized":"2023-11-26T07:11:30.850465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.319818+0000","last_change":"2023-11-26T07:11:27.530887+0000","last_active":"2023-11-26T07:11:30.319818+0000","last_peered":"2023-11-26T07:11:30.319818+0000","last_clean":"2023-11-26T07:11:30.319818+0000","last_became_active":"2023-11-26T07:11:27.530613+0000","last_became_peered":"2023-11-26T07:11:27.530613+0000","last_unstale":"2023-11-26T07:11:30.319818+0000","last_undegraded":"2023-11-26T07:11:30.319818+0000","last_fullsized":"2023-11-26T07:11:30.319818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:35.117462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:28.525772+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:11:28.525772+0000","last_peered":"2023-11-26T07:11:28.525772+0000","last_clean":"2023-11-26T07:11:28.525772+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:11:28.525772+0000","last_undegraded":"2023-11-26T07:11:28.525772+0000","last_fullsized":"2023-11-26T07:11:28.525772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189105+0000","last_change":"2023-11-26T07:11:28.548666+0000","last_active":"2023-11-26T07:11:30.189105+0000","last_peered":"2023-11-26T07:11:30.189105+0000","last_clean":"2023-11-26T07:11:30.189105+0000","last_became_active":"2023-11-26T07:11:28.548390+0000","last_became_peered":"2023-11-26T07:11:28.548390+0000","last_unstale":"2023-11-26T07:11:30.189105+0000","last_undegraded":"2023-11-26T07:11:30.189105+0000","last_fullsized":"2023-11-26T07:11:30.189105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:26.403994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539972+0000","last_change":"2023-11-26T07:11:27.525655+0000","last_active":"2023-11-26T07:11:29.539972+0000","last_peered":"2023-11-26T07:11:29.539972+0000","last_clean":"2023-11-26T07:11:29.539972+0000","last_became_active":"2023-11-26T07:11:27.525321+0000","last_became_peered":"2023-11-26T07:11:27.525321+0000","last_unstale":"2023-11-26T07:11:29.539972+0000","last_undegraded":"2023-11-26T07:11:29.539972+0000","last_fullsized":"2023-11-26T07:11:29.539972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:34:47.995242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540819+0000","last_change":"2023-11-26T07:11:27.528743+0000","last_active":"2023-11-26T07:11:29.540819+0000","last_peered":"2023-11-26T07:11:29.540819+0000","last_clean":"2023-11-26T07:11:29.540819+0000","last_became_active":"2023-11-26T07:11:27.527803+0000","last_became_peered":"2023-11-26T07:11:27.527803+0000","last_unstale":"2023-11-26T07:11:29.540819+0000","last_undegraded":"2023-11-26T07:11:29.540819+0000","last_fullsized":"2023-11-26T07:11:29.540819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:16:53.693262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537674+0000","last_change":"2023-11-26T07:11:28.542614+0000","last_active":"2023-11-26T07:11:29.537674+0000","last_peered":"2023-11-26T07:11:29.537674+0000","last_clean":"2023-11-26T07:11:29.537674+0000","last_became_active":"2023-11-26T07:11:28.542356+0000","last_became_peered":"2023-11-26T07:11:28.542356+0000","last_unstale":"2023-11-26T07:11:29.537674+0000","last_undegraded":"2023-11-26T07:11:29.537674+0000","last_fullsized":"2023-11-26T07:11:29.537674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.318105+0000","last_change":"2023-11-26T07:11:27.527868+0000","last_active":"2023-11-26T07:11:30.318105+0000","last_peered":"2023-11-26T07:11:30.318105+0000","last_clean":"2023-11-26T07:11:30.318105+0000","last_became_active":"2023-11-26T07:11:27.527482+0000","last_became_peered":"2023-11-26T07:11:27.527482+0000","last_unstale":"2023-11-26T07:11:30.318105+0000","last_undegraded":"2023-11-26T07:11:30.318105+0000","last_fullsized":"2023-11-26T07:11:30.318105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:19.467051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850552+0000","last_change":"2023-11-26T07:11:28.554487+0000","last_active":"2023-11-26T07:11:30.850552+0000","last_peered":"2023-11-26T07:11:30.850552+0000","last_clean":"2023-11-26T07:11:30.850552+0000","last_became_active":"2023-11-26T07:11:28.554210+0000","last_became_peered":"2023-11-26T07:11:28.554210+0000","last_unstale":"2023-11-26T07:11:30.850552+0000","last_undegraded":"2023-11-26T07:11:30.850552+0000","last_fullsized":"2023-11-26T07:11:30.850552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:24:47.640100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.331934+0000","last_change":"2023-11-26T07:11:27.531628+0000","last_active":"2023-11-26T07:11:30.331934+0000","last_peered":"2023-11-26T07:11:30.331934+0000","last_clean":"2023-11-26T07:11:30.331934+0000","last_became_active":"2023-11-26T07:11:27.531336+0000","last_became_peered":"2023-11-26T07:11:27.531336+0000","last_unstale":"2023-11-26T07:11:30.331934+0000","last_undegraded":"2023-11-26T07:11:30.331934+0000","last_fullsized":"2023-11-26T07:11:30.331934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:10.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189372+0000","last_change":"2023-11-26T07:11:28.540197+0000","last_active":"2023-11-26T07:11:30.189372+0000","last_peered":"2023-11-26T07:11:30.189372+0000","last_clean":"2023-11-26T07:11:30.189372+0000","last_became_active":"2023-11-26T07:11:28.539930+0000","last_became_peered":"2023-11-26T07:11:28.539930+0000","last_unstale":"2023-11-26T07:11:30.189372+0000","last_undegraded":"2023-11-26T07:11:30.189372+0000","last_fullsized":"2023-11-26T07:11:30.189372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539876+0000","last_change":"2023-11-26T07:11:27.528055+0000","last_active":"2023-11-26T07:11:29.539876+0000","last_peered":"2023-11-26T07:11:29.539876+0000","last_clean":"2023-11-26T07:11:29.539876+0000","last_became_active":"2023-11-26T07:11:27.527760+0000","last_became_peered":"2023-11-26T07:11:27.527760+0000","last_unstale":"2023-11-26T07:11:29.539876+0000","last_undegraded":"2023-11-26T07:11:29.539876+0000","last_fullsized":"2023-11-26T07:11:29.539876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:10.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.539630+0000","last_change":"2023-11-26T07:11:28.544985+0000","last_active":"2023-11-26T07:11:29.539630+0000","last_peered":"2023-11-26T07:11:29.539630+0000","last_clean":"2023-11-26T07:11:29.539630+0000","last_became_active":"2023-11-26T07:11:28.544546+0000","last_became_peered":"2023-11-26T07:11:28.544546+0000","last_unstale":"2023-11-26T07:11:29.539630+0000","last_undegraded":"2023-11-26T07:11:29.539630+0000","last_fullsized":"2023-11-26T07:11:29.539630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:19.228362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537514+0000","last_change":"2023-11-26T07:11:27.527741+0000","last_active":"2023-11-26T07:11:29.537514+0000","last_peered":"2023-11-26T07:11:29.537514+0000","last_clean":"2023-11-26T07:11:29.537514+0000","last_became_active":"2023-11-26T07:11:27.526479+0000","last_became_peered":"2023-11-26T07:11:27.526479+0000","last_unstale":"2023-11-26T07:11:29.537514+0000","last_undegraded":"2023-11-26T07:11:29.537514+0000","last_fullsized":"2023-11-26T07:11:29.537514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:31.071342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850638+0000","last_change":"2023-11-26T07:11:28.540204+0000","last_active":"2023-11-26T07:11:30.850638+0000","last_peered":"2023-11-26T07:11:30.850638+0000","last_clean":"2023-11-26T07:11:30.850638+0000","last_became_active":"2023-11-26T07:11:28.539891+0000","last_became_peered":"2023-11-26T07:11:28.539891+0000","last_unstale":"2023-11-26T07:11:30.850638+0000","last_undegraded":"2023-11-26T07:11:30.850638+0000","last_fullsized":"2023-11-26T07:11:30.850638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:16:54.697667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537841+0000","last_change":"2023-11-26T07:11:27.534469+0000","last_active":"2023-11-26T07:11:29.537841+0000","last_peered":"2023-11-26T07:11:29.537841+0000","last_clean":"2023-11-26T07:11:29.537841+0000","last_became_active":"2023-11-26T07:11:27.534116+0000","last_became_peered":"2023-11-26T07:11:27.534116+0000","last_unstale":"2023-11-26T07:11:29.537841+0000","last_undegraded":"2023-11-26T07:11:29.537841+0000","last_fullsized":"2023-11-26T07:11:29.537841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:18:25.399589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850461+0000","last_change":"2023-11-26T07:11:28.547277+0000","last_active":"2023-11-26T07:11:30.850461+0000","last_peered":"2023-11-26T07:11:30.850461+0000","last_clean":"2023-11-26T07:11:30.850461+0000","last_became_active":"2023-11-26T07:11:28.546992+0000","last_became_peered":"2023-11-26T07:11:28.546992+0000","last_unstale":"2023-11-26T07:11:30.850461+0000","last_undegraded":"2023-11-26T07:11:30.850461+0000","last_fullsized":"2023-11-26T07:11:30.850461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:42.974584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540403+0000","last_change":"2023-11-26T07:11:27.529785+0000","last_active":"2023-11-26T07:11:29.540403+0000","last_peered":"2023-11-26T07:11:29.540403+0000","last_clean":"2023-11-26T07:11:29.540403+0000","last_became_active":"2023-11-26T07:11:27.528686+0000","last_became_peered":"2023-11-26T07:11:27.528686+0000","last_unstale":"2023-11-26T07:11:29.540403+0000","last_undegraded":"2023-11-26T07:11:29.540403+0000","last_fullsized":"2023-11-26T07:11:29.540403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:21.053006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.921146+0000","last_change":"2023-11-26T07:11:28.546713+0000","last_active":"2023-11-26T07:11:30.921146+0000","last_peered":"2023-11-26T07:11:30.921146+0000","last_clean":"2023-11-26T07:11:30.921146+0000","last_became_active":"2023-11-26T07:11:28.546453+0000","last_became_peered":"2023-11-26T07:11:28.546453+0000","last_unstale":"2023-11-26T07:11:30.921146+0000","last_undegraded":"2023-11-26T07:11:30.921146+0000","last_fullsized":"2023-11-26T07:11:30.921146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.321984+0000","last_change":"2023-11-26T07:11:27.531109+0000","last_active":"2023-11-26T07:11:30.321984+0000","last_peered":"2023-11-26T07:11:30.321984+0000","last_clean":"2023-11-26T07:11:30.321984+0000","last_became_active":"2023-11-26T07:11:27.530786+0000","last_became_peered":"2023-11-26T07:11:27.530786+0000","last_unstale":"2023-11-26T07:11:30.321984+0000","last_undegraded":"2023-11-26T07:11:30.321984+0000","last_fullsized":"2023-11-26T07:11:30.321984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:51:52.471590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540761+0000","last_change":"2023-11-26T07:11:28.539795+0000","last_active":"2023-11-26T07:11:29.540761+0000","last_peered":"2023-11-26T07:11:29.540761+0000","last_clean":"2023-11-26T07:11:29.540761+0000","last_became_active":"2023-11-26T07:11:28.539562+0000","last_became_peered":"2023-11-26T07:11:28.539562+0000","last_unstale":"2023-11-26T07:11:29.540761+0000","last_undegraded":"2023-11-26T07:11:29.540761+0000","last_fullsized":"2023-11-26T07:11:29.540761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:02:05.837198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537674+0000","last_change":"2023-11-26T07:11:27.522266+0000","last_active":"2023-11-26T07:11:29.537674+0000","last_peered":"2023-11-26T07:11:29.537674+0000","last_clean":"2023-11-26T07:11:29.537674+0000","last_became_active":"2023-11-26T07:11:27.521972+0000","last_became_peered":"2023-11-26T07:11:27.521972+0000","last_unstale":"2023-11-26T07:11:29.537674+0000","last_undegraded":"2023-11-26T07:11:29.537674+0000","last_fullsized":"2023-11-26T07:11:29.537674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719435+0000","last_change":"2023-11-26T07:11:28.544499+0000","last_active":"2023-11-26T07:11:30.719435+0000","last_peered":"2023-11-26T07:11:30.719435+0000","last_clean":"2023-11-26T07:11:30.719435+0000","last_became_active":"2023-11-26T07:11:28.544154+0000","last_became_peered":"2023-11-26T07:11:28.544154+0000","last_unstale":"2023-11-26T07:11:30.719435+0000","last_undegraded":"2023-11-26T07:11:30.719435+0000","last_fullsized":"2023-11-26T07:11:30.719435+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:16:54.697667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.322016+0000","last_change":"2023-11-26T07:11:27.535061+0000","last_active":"2023-11-26T07:11:30.322016+0000","last_peered":"2023-11-26T07:11:30.322016+0000","last_clean":"2023-11-26T07:11:30.322016+0000","last_became_active":"2023-11-26T07:11:27.534792+0000","last_became_peered":"2023-11-26T07:11:27.534792+0000","last_unstale":"2023-11-26T07:11:30.322016+0000","last_undegraded":"2023-11-26T07:11:30.322016+0000","last_fullsized":"2023-11-26T07:11:30.322016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:26.498963+0000","last_clean_scrub_stamp":"2023-11-26T07:11:26.498963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:56.094442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.540720+0000","last_change":"2023-11-26T07:11:28.532153+0000","last_active":"2023-11-26T07:11:29.540720+0000","last_peered":"2023-11-26T07:11:29.540720+0000","last_clean":"2023-11-26T07:11:29.540720+0000","last_became_active":"2023-11-26T07:11:28.531880+0000","last_became_peered":"2023-11-26T07:11:28.531880+0000","last_unstale":"2023-11-26T07:11:29.540720+0000","last_undegraded":"2023-11-26T07:11:29.540720+0000","last_fullsized":"2023-11-26T07:11:29.540720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:02.252744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.920884+0000","last_change":"2023-11-26T07:11:28.544445+0000","last_active":"2023-11-26T07:11:30.920884+0000","last_peered":"2023-11-26T07:11:30.920884+0000","last_clean":"2023-11-26T07:11:30.920884+0000","last_became_active":"2023-11-26T07:11:28.544146+0000","last_became_peered":"2023-11-26T07:11:28.544146+0000","last_unstale":"2023-11-26T07:11:30.920884+0000","last_undegraded":"2023-11-26T07:11:30.920884+0000","last_fullsized":"2023-11-26T07:11:30.920884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:14.504204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:31.219688+0000","last_change":"2023-11-26T07:11:28.542940+0000","last_active":"2023-11-26T07:11:31.219688+0000","last_peered":"2023-11-26T07:11:31.219688+0000","last_clean":"2023-11-26T07:11:31.219688+0000","last_became_active":"2023-11-26T07:11:28.542607+0000","last_became_peered":"2023-11-26T07:11:28.542607+0000","last_unstale":"2023-11-26T07:11:31.219688+0000","last_undegraded":"2023-11-26T07:11:31.219688+0000","last_fullsized":"2023-11-26T07:11:31.219688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:48:32.075747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719379+0000","last_change":"2023-11-26T07:11:28.549728+0000","last_active":"2023-11-26T07:11:30.719379+0000","last_peered":"2023-11-26T07:11:30.719379+0000","last_clean":"2023-11-26T07:11:30.719379+0000","last_became_active":"2023-11-26T07:11:28.543537+0000","last_became_peered":"2023-11-26T07:11:28.543537+0000","last_unstale":"2023-11-26T07:11:30.719379+0000","last_undegraded":"2023-11-26T07:11:30.719379+0000","last_fullsized":"2023-11-26T07:11:30.719379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:24:47.640100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189278+0000","last_change":"2023-11-26T07:11:28.546921+0000","last_active":"2023-11-26T07:11:30.189278+0000","last_peered":"2023-11-26T07:11:30.189278+0000","last_clean":"2023-11-26T07:11:30.189278+0000","last_became_active":"2023-11-26T07:11:28.546543+0000","last_became_peered":"2023-11-26T07:11:28.546543+0000","last_unstale":"2023-11-26T07:11:30.189278+0000","last_undegraded":"2023-11-26T07:11:30.189278+0000","last_fullsized":"2023-11-26T07:11:30.189278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850701+0000","last_change":"2023-11-26T07:11:28.549516+0000","last_active":"2023-11-26T07:11:30.850701+0000","last_peered":"2023-11-26T07:11:30.850701+0000","last_clean":"2023-11-26T07:11:30.850701+0000","last_became_active":"2023-11-26T07:11:28.548917+0000","last_became_peered":"2023-11-26T07:11:28.548917+0000","last_unstale":"2023-11-26T07:11:30.850701+0000","last_undegraded":"2023-11-26T07:11:30.850701+0000","last_fullsized":"2023-11-26T07:11:30.850701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:14.369275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:29.537692+0000","last_change":"2023-11-26T07:11:28.541342+0000","last_active":"2023-11-26T07:11:29.537692+0000","last_peered":"2023-11-26T07:11:29.537692+0000","last_clean":"2023-11-26T07:11:29.537692+0000","last_became_active":"2023-11-26T07:11:28.540919+0000","last_became_peered":"2023-11-26T07:11:28.540919+0000","last_unstale":"2023-11-26T07:11:29.537692+0000","last_undegraded":"2023-11-26T07:11:29.537692+0000","last_fullsized":"2023-11-26T07:11:29.537692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.921212+0000","last_change":"2023-11-26T07:11:28.551636+0000","last_active":"2023-11-26T07:11:30.921212+0000","last_peered":"2023-11-26T07:11:30.921212+0000","last_clean":"2023-11-26T07:11:30.921212+0000","last_became_active":"2023-11-26T07:11:28.551335+0000","last_became_peered":"2023-11-26T07:11:28.551335+0000","last_unstale":"2023-11-26T07:11:30.921212+0000","last_undegraded":"2023-11-26T07:11:30.921212+0000","last_fullsized":"2023-11-26T07:11:30.921212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:22.057411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850489+0000","last_change":"2023-11-26T07:11:28.546402+0000","last_active":"2023-11-26T07:11:30.850489+0000","last_peered":"2023-11-26T07:11:30.850489+0000","last_clean":"2023-11-26T07:11:30.850489+0000","last_became_active":"2023-11-26T07:11:28.546132+0000","last_became_peered":"2023-11-26T07:11:28.546132+0000","last_unstale":"2023-11-26T07:11:30.850489+0000","last_undegraded":"2023-11-26T07:11:30.850489+0000","last_fullsized":"2023-11-26T07:11:30.850489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:32:19.211200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719685+0000","last_change":"2023-11-26T07:11:28.548148+0000","last_active":"2023-11-26T07:11:30.719685+0000","last_peered":"2023-11-26T07:11:30.719685+0000","last_clean":"2023-11-26T07:11:30.719685+0000","last_became_active":"2023-11-26T07:11:28.546933+0000","last_became_peered":"2023-11-26T07:11:28.546933+0000","last_unstale":"2023-11-26T07:11:30.719685+0000","last_undegraded":"2023-11-26T07:11:30.719685+0000","last_fullsized":"2023-11-26T07:11:30.719685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:16:48.717411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.850378+0000","last_change":"2023-11-26T07:11:28.549377+0000","last_active":"2023-11-26T07:11:30.850378+0000","last_peered":"2023-11-26T07:11:30.850378+0000","last_clean":"2023-11-26T07:11:30.850378+0000","last_became_active":"2023-11-26T07:11:28.549093+0000","last_became_peered":"2023-11-26T07:11:28.549093+0000","last_unstale":"2023-11-26T07:11:30.850378+0000","last_undegraded":"2023-11-26T07:11:30.850378+0000","last_fullsized":"2023-11-26T07:11:30.850378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.719517+0000","last_change":"2023-11-26T07:11:28.554891+0000","last_active":"2023-11-26T07:11:30.719517+0000","last_peered":"2023-11-26T07:11:30.719517+0000","last_clean":"2023-11-26T07:11:30.719517+0000","last_became_active":"2023-11-26T07:11:28.554617+0000","last_became_peered":"2023-11-26T07:11:28.554617+0000","last_unstale":"2023-11-26T07:11:30.719517+0000","last_undegraded":"2023-11-26T07:11:30.719517+0000","last_fullsized":"2023-11-26T07:11:30.719517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:14.369275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.189270+0000","last_change":"2023-11-26T07:11:28.552756+0000","last_active":"2023-11-26T07:11:30.189270+0000","last_peered":"2023-11-26T07:11:30.189270+0000","last_clean":"2023-11-26T07:11:30.189270+0000","last_became_active":"2023-11-26T07:11:28.552548+0000","last_became_peered":"2023-11-26T07:11:28.552548+0000","last_unstale":"2023-11-26T07:11:30.189270+0000","last_undegraded":"2023-11-26T07:11:30.189270+0000","last_fullsized":"2023-11-26T07:11:30.189270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:20.252887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-26T07:11:30.921199+0000","last_change":"2023-11-26T07:11:28.547396+0000","last_active":"2023-11-26T07:11:30.921199+0000","last_peered":"2023-11-26T07:11:30.921199+0000","last_clean":"2023-11-26T07:11:30.921199+0000","last_became_active":"2023-11-26T07:11:28.547176+0000","last_became_peered":"2023-11-26T07:11:28.547176+0000","last_unstale":"2023-11-26T07:11:30.921199+0000","last_undegraded":"2023-11-26T07:11:30.921199+0000","last_fullsized":"2023-11-26T07:11:30.921199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:27.503368+0000","last_clean_scrub_stamp":"2023-11-26T07:11:27.503368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:16.203745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:11:37.625 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-26T07:11:37.625 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-26T07:11:37.625 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-26T07:11:37.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-26T07:11:37.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.778+0000 7f7b0c7e2700 1 Processor -- start 2023-11-26T07:11:37.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.778+0000 7f7b0c7e2700 1 -- start start 2023-11-26T07:11:37.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.779+0000 7f7b0c7e2700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7b0415e080 0x7f7b040b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.779+0000 7f7b0c7e2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7b0406c710 0x7f7b040afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.779+0000 7f7b0c7e2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b040ae430 con 0x7f7b040af4d0 2023-11-26T07:11:37.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.779+0000 7f7b0c7e2700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b040af0b0 con 0x7f7b0406c710 2023-11-26T07:11:37.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.779+0000 7f7b0c7e2700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b040b05a0 con 0x7f7b0415e080 2023-11-26T07:11:37.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0ad7f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7b0406c710 0x7f7b040afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0ad7f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7b0406c710 0x7f7b040afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34594/0 (socket says 172.21.15.83:34594) 2023-11-26T07:11:37.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0ad7f700 1 -- 172.21.15.83:0/468041544 learned_addr learned my addr 172.21.15.83:0/468041544 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:37.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0a57e700 1 --2- 172.21.15.83:0/468041544 >> v2:172.21.15.203:3300/0 conn(0x7f7b0415e080 0x7f7b040b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0ad7f700 1 -- 172.21.15.83:0/468041544 >> v2:172.21.15.203:3300/0 conn(0x7f7b0415e080 msgr2=0x7f7b040b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:37.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0ad7f700 1 --2- 172.21.15.83:0/468041544 >> v2:172.21.15.203:3300/0 conn(0x7f7b0415e080 0x7f7b040b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0ad7f700 1 -- 172.21.15.83:0/468041544 >> v1:172.21.15.203:6789/0 conn(0x7f7b040af4d0 legacy=0x7f7b0406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0ad7f700 1 -- 172.21.15.83:0/468041544 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040ae710 con 0x7f7b0406c710 2023-11-26T07:11:37.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0ad7f700 1 --2- 172.21.15.83:0/468041544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b0406c710 0x7f7b040afb90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b04072a90 tx=0x7f7afc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f20d578ecbc5a36 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:37.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0957c700 1 -- 172.21.15.83:0/468041544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00e070 con 0x7f7b0406c710 2023-11-26T07:11:37.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0957c700 1 -- 172.21.15.83:0/468041544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc00b980 con 0x7f7b0406c710 2023-11-26T07:11:37.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.780+0000 7f7b0957c700 1 -- 172.21.15.83:0/468041544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00bd30 con 0x7f7b0406c710 2023-11-26T07:11:37.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.781+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/468041544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b0406c710 msgr2=0x7f7b040afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.781+0000 7f7b0c7e2700 1 --2- 172.21.15.83:0/468041544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b0406c710 0x7f7b040afb90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b04072a90 tx=0x7f7afc00b330 comp rx=0 tx=0).stop 2023-11-26T07:11:37.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.781+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/468041544 shutdown_connections 2023-11-26T07:11:37.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.781+0000 7f7b0c7e2700 1 --2- 172.21.15.83:0/468041544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b0406c710 0x7f7b040afb90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.781+0000 7f7b0c7e2700 1 --2- 172.21.15.83:0/468041544 >> v2:172.21.15.203:3300/0 conn(0x7f7b0415e080 0x7f7b040b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.781+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/468041544 >> 172.21.15.83:0/468041544 conn(0x7f7b04071b90 msgr2=0x7f7b04071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:37.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.781+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/468041544 shutdown_connections 2023-11-26T07:11:37.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.781+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/468041544 wait complete. 2023-11-26T07:11:37.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.782+0000 7f7b0c7e2700 1 Processor -- start 2023-11-26T07:11:37.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.782+0000 7f7b0c7e2700 1 -- start start 2023-11-26T07:11:37.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0c7e2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b0406c710 0x7f7b04151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0c7e2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b040af4d0 0x7f7b0415e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0c7e2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b0415e080 0x7f7b04153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0c7e2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04157c50 con 0x7f7b0406c710 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0c7e2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04157360 con 0x7f7b040af4d0 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0c7e2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04157600 con 0x7f7b0415e080 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0a57e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b0406c710 0x7f7b04151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0a57e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b0406c710 0x7f7b04151920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34596/0 (socket says 172.21.15.83:34596) 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0a57e700 1 -- 172.21.15.83:0/2884333132 learned_addr learned my addr 172.21.15.83:0/2884333132 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0ad7f700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b0415e080 0x7f7b04153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b09d7d700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b040af4d0 0x7f7b0415e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0a57e700 1 -- 172.21.15.83:0/2884333132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b0415e080 msgr2=0x7f7b04153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0a57e700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b0415e080 0x7f7b04153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0a57e700 1 -- 172.21.15.83:0/2884333132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b040af4d0 msgr2=0x7f7b0415e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0a57e700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b040af4d0 0x7f7b0415e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:37.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0a57e700 1 -- 172.21.15.83:0/2884333132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc009b30 con 0x7f7b0406c710 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0a57e700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b0406c710 0x7f7b04151920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7af400d800 tx=0x7f7af400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7afaffd700 1 -- 172.21.15.83:0/2884333132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40125f0 con 0x7f7b0406c710 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7afaffd700 1 -- 172.21.15.83:0/2884333132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4012750 con 0x7f7b0406c710 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7afaffd700 1 -- 172.21.15.83:0/2884333132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4012b00 con 0x7f7b0406c710 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/2884333132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b041578e0 con 0x7f7b0406c710 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.783+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/2884333132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0415e9a0 con 0x7f7b0406c710 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.784+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/2884333132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b040753f0 con 0x7f7b0406c710 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.785+0000 7f7afaffd700 1 -- 172.21.15.83:0/2884333132 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7af4012c60 con 0x7f7b0406c710 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.785+0000 7f7afaffd700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7ae806aed0 0x7f7ae806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.785+0000 7f7afaffd700 1 -- 172.21.15.83:0/2884333132 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7af40898f0 con 0x7f7b0406c710 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.785+0000 7f7b09d7d700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7ae806aed0 0x7f7ae806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.785+0000 7f7b09d7d700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7ae806aed0 0x7f7ae806d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7b00007fd0 tx=0x7f7b00009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:37.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:37.790+0000 7f7afaffd700 1 -- 172.21.15.83:0/2884333132 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7af4054a30 con 0x7f7b0406c710 2023-11-26T07:11:38.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.021+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/2884333132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7b04067f60 con 0x7f7b0406c710 2023-11-26T07:11:38.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.022+0000 7f7afaffd700 1 -- 172.21.15.83:0/2884333132 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7af4058050 con 0x7f7b0406c710 2023-11-26T07:11:38.023 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:38.023 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-26T07:11:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/2884333132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7ae806aed0 msgr2=0x7f7ae806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7ae806aed0 0x7f7ae806d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7b00007fd0 tx=0x7f7b00009040 comp rx=0 tx=0).stop 2023-11-26T07:11:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/2884333132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b0406c710 msgr2=0x7f7b04151920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b0406c710 0x7f7b04151920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7af400d800 tx=0x7f7af400dbc0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/2884333132 shutdown_connections 2023-11-26T07:11:38.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7ae806aed0 0x7f7ae806d390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b0415e080 0x7f7b04153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b0406c710 0x7f7b04151920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 --2- 172.21.15.83:0/2884333132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b040af4d0 0x7f7b0415e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/2884333132 >> 172.21.15.83:0/2884333132 conn(0x7f7b04071b90 msgr2=0x7f7b040b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/2884333132 shutdown_connections 2023-11-26T07:11:38.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.024+0000 7f7b0c7e2700 1 -- 172.21.15.83:0/2884333132 wait complete. 2023-11-26T07:11:38.037 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-26T07:11:38.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:38.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.156+0000 7fea86a9a700 1 Processor -- start 2023-11-26T07:11:38.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.156+0000 7fea86a9a700 1 -- start start 2023-11-26T07:11:38.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.157+0000 7fea86a9a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 0x7fea801604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.157+0000 7fea86a9a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fea800d1c70 0x7fea800d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.157+0000 7fea86a9a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea800b22d0 con 0x7fea800b3370 2023-11-26T07:11:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.157+0000 7fea86a9a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea800b2f50 con 0x7fea800d1c70 2023-11-26T07:11:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.157+0000 7fea86a9a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea800d2640 con 0x7fea8015bdd0 2023-11-26T07:11:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea84836700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 0x7fea801604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea85037700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fea800d1c70 0x7fea800d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea85037700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fea800d1c70 0x7fea800d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34058/0 (socket says 172.21.15.83:34058) 2023-11-26T07:11:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea7ffff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fea800b3370 0x7fea800b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:39262/0 (socket says 172.21.15.83:39262) 2023-11-26T07:11:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea84836700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 0x7fea801604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:45062/0 (socket says 172.21.15.83:45062) 2023-11-26T07:11:38.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea85037700 1 -- 172.21.15.83:0/2915817694 learned_addr learned my addr 172.21.15.83:0/2915817694 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea85037700 1 -- 172.21.15.83:0/2915817694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 msgr2=0x7fea801604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea85037700 1 --2- 172.21.15.83:0/2915817694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 0x7fea801604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea85037700 1 -- 172.21.15.83:0/2915817694 >> v1:172.21.15.203:6789/0 conn(0x7fea800b3370 legacy=0x7fea800b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea85037700 1 -- 172.21.15.83:0/2915817694 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea800b25b0 con 0x7fea800d1c70 2023-11-26T07:11:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea7f7fe700 1 -- 172.21.15.83:0/2915817694 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74955767 0 0) 0x7fea800b22d0 con 0x7fea800b3370 2023-11-26T07:11:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea84836700 1 --2- 172.21.15.83:0/2915817694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 0x7fea801604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea85037700 1 --2- 172.21.15.83:0/2915817694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea800d1c70 0x7fea800d2100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea78009e50 tx=0x7fea7800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f24b66b69f6713b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea7f7fe700 1 -- 172.21.15.83:0/2915817694 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800e070 con 0x7fea800d1c70 2023-11-26T07:11:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea7f7fe700 1 -- 172.21.15.83:0/2915817694 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7800b910 con 0x7fea800d1c70 2023-11-26T07:11:38.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.158+0000 7fea7f7fe700 1 -- 172.21.15.83:0/2915817694 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800bcc0 con 0x7fea800d1c70 2023-11-26T07:11:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.159+0000 7fea86a9a700 1 -- 172.21.15.83:0/2915817694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea800d1c70 msgr2=0x7fea800d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.159+0000 7fea86a9a700 1 --2- 172.21.15.83:0/2915817694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea800d1c70 0x7fea800d2100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea78009e50 tx=0x7fea7800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.159+0000 7fea86a9a700 1 -- 172.21.15.83:0/2915817694 shutdown_connections 2023-11-26T07:11:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.159+0000 7fea86a9a700 1 --2- 172.21.15.83:0/2915817694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea800d1c70 0x7fea800d2100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.159+0000 7fea86a9a700 1 --2- 172.21.15.83:0/2915817694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 0x7fea801604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.159+0000 7fea86a9a700 1 -- 172.21.15.83:0/2915817694 >> 172.21.15.83:0/2915817694 conn(0x7fea800731a0 msgr2=0x7fea8009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.159+0000 7fea86a9a700 1 -- 172.21.15.83:0/2915817694 shutdown_connections 2023-11-26T07:11:38.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.159+0000 7fea86a9a700 1 -- 172.21.15.83:0/2915817694 wait complete. 2023-11-26T07:11:38.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.160+0000 7fea86a9a700 1 Processor -- start 2023-11-26T07:11:38.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.160+0000 7fea86a9a700 1 -- start start 2023-11-26T07:11:38.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea86a9a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea800b3370 0x7fea801f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea86a9a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea800d1c70 0x7fea800c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea86a9a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 0x7fea801f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea86a9a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800ca030 con 0x7fea800d1c70 2023-11-26T07:11:38.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea86a9a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea801538f0 con 0x7fea800b3370 2023-11-26T07:11:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea86a9a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea800c9420 con 0x7fea8015bdd0 2023-11-26T07:11:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea7ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea800d1c70 0x7fea800c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea7ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea800d1c70 0x7fea800c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34610/0 (socket says 172.21.15.83:34610) 2023-11-26T07:11:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea7ffff700 1 -- 172.21.15.83:0/4081359509 learned_addr learned my addr 172.21.15.83:0/4081359509 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea85037700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 0x7fea801f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea84836700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea800b3370 0x7fea801f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea7ffff700 1 -- 172.21.15.83:0/4081359509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 msgr2=0x7fea801f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea7ffff700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 0x7fea801f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea7ffff700 1 -- 172.21.15.83:0/4081359509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea800b3370 msgr2=0x7fea801f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea7ffff700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea800b3370 0x7fea801f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea7ffff700 1 -- 172.21.15.83:0/4081359509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea740094a0 con 0x7fea800d1c70 2023-11-26T07:11:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea85037700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 0x7fea801f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea84836700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea800b3370 0x7fea801f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.161+0000 7fea7ffff700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea800d1c70 0x7fea800c8db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0011e0 tx=0x7fea6c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.162+0000 7fea7d7fa700 1 -- 172.21.15.83:0/4081359509 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c004c80 con 0x7fea800d1c70 2023-11-26T07:11:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.162+0000 7fea7d7fa700 1 -- 172.21.15.83:0/4081359509 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c004e00 con 0x7fea800d1c70 2023-11-26T07:11:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.162+0000 7fea86a9a700 1 -- 172.21.15.83:0/4081359509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea78009b30 con 0x7fea800d1c70 2023-11-26T07:11:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.162+0000 7fea7d7fa700 1 -- 172.21.15.83:0/4081359509 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c007310 con 0x7fea800d1c70 2023-11-26T07:11:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.162+0000 7fea86a9a700 1 -- 172.21.15.83:0/4081359509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea800c99e0 con 0x7fea800d1c70 2023-11-26T07:11:38.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.163+0000 7fea86a9a700 1 -- 172.21.15.83:0/4081359509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea800ca1a0 con 0x7fea800d1c70 2023-11-26T07:11:38.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.163+0000 7fea7d7fa700 1 -- 172.21.15.83:0/4081359509 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea6c007470 con 0x7fea800d1c70 2023-11-26T07:11:38.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.163+0000 7fea7d7fa700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea6406aed0 0x7fea6406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.163+0000 7fea7d7fa700 1 -- 172.21.15.83:0/4081359509 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea6c089320 con 0x7fea800d1c70 2023-11-26T07:11:38.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.163+0000 7fea84836700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea6406aed0 0x7fea6406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.164+0000 7fea84836700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea6406aed0 0x7fea6406d390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fea74009c50 tx=0x7fea74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:38.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.168+0000 7fea7d7fa700 1 -- 172.21.15.83:0/4081359509 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea6c054460 con 0x7fea800d1c70 2023-11-26T07:11:38.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.361+0000 7fea86a9a700 1 -- 172.21.15.83:0/4081359509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea80067f60 con 0x7fea800d1c70 2023-11-26T07:11:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.362+0000 7fea7d7fa700 1 -- 172.21.15.83:0/4081359509 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fea6c057a80 con 0x7fea800d1c70 2023-11-26T07:11:38.364 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:38.364 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:38.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 -- 172.21.15.83:0/4081359509 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea6406aed0 msgr2=0x7fea6406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea6406aed0 0x7fea6406d390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fea74009c50 tx=0x7fea74008040 comp rx=0 tx=0).stop 2023-11-26T07:11:38.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 -- 172.21.15.83:0/4081359509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea800d1c70 msgr2=0x7fea800c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea800d1c70 0x7fea800c8db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0011e0 tx=0x7fea6c00cae0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 -- 172.21.15.83:0/4081359509 shutdown_connections 2023-11-26T07:11:38.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea6406aed0 0x7fea6406d390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8015bdd0 0x7fea801f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea800b3370 0x7fea801f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 --2- 172.21.15.83:0/4081359509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea800d1c70 0x7fea800c8db0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 -- 172.21.15.83:0/4081359509 >> 172.21.15.83:0/4081359509 conn(0x7fea800731a0 msgr2=0x7fea800b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 -- 172.21.15.83:0/4081359509 shutdown_connections 2023-11-26T07:11:38.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.364+0000 7fea86a9a700 1 -- 172.21.15.83:0/4081359509 wait complete. 2023-11-26T07:11:38.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:11:38.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:11:38.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.500+0000 7f2ba0d12700 1 Processor -- start 2023-11-26T07:11:38.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.500+0000 7f2ba0d12700 1 -- start start 2023-11-26T07:11:38.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.501+0000 7f2ba0d12700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b9c15e160 0x7f2b9c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.501+0000 7f2ba0d12700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2b9c0afc90 0x7f2b9c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.501+0000 7f2ba0d12700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c0b3c60 con 0x7f2b9c0744d0 2023-11-26T07:11:38.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.501+0000 7f2ba0d12700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c0b4500 con 0x7f2b9c0afc90 2023-11-26T07:11:38.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.501+0000 7f2ba0d12700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c074960 con 0x7f2b9c15e160 2023-11-26T07:11:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.501+0000 7f2b99d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2b9c0afc90 0x7f2b9c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.501+0000 7f2b9ad9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2b9c0744d0 0x7f2b9c06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54660/0 (socket says 172.21.15.83:54660) 2023-11-26T07:11:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.501+0000 7f2b9ad9d700 1 -- 172.21.15.83:0/3046113692 learned_addr learned my addr 172.21.15.83:0/3046113692 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.502+0000 7f2b9a59c700 1 --2- 172.21.15.83:0/3046113692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b9c15e160 0x7f2b9c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.502+0000 7f2b99d9b700 1 -- 172.21.15.83:0/3046113692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b9c15e160 msgr2=0x7f2b9c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.502+0000 7f2b99d9b700 1 --2- 172.21.15.83:0/3046113692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b9c15e160 0x7f2b9c0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.502+0000 7f2b99d9b700 1 -- 172.21.15.83:0/3046113692 >> v1:172.21.15.83:6789/0 conn(0x7f2b9c0744d0 legacy=0x7f2b9c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.502+0000 7f2b99d9b700 1 -- 172.21.15.83:0/3046113692 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c0b58c0 con 0x7f2b9c0afc90 2023-11-26T07:11:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.502+0000 7f2b99d9b700 1 --2- 172.21.15.83:0/3046113692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b9c0afc90 0x7f2b9c073f90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009840 tx=0x7f2b8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3659dc4cb1b5d04f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.502+0000 7f2b9959a700 1 -- 172.21.15.83:0/3046113692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c011830 con 0x7f2b9c0afc90 2023-11-26T07:11:38.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.502+0000 7f2b9959a700 1 -- 172.21.15.83:0/3046113692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c011990 con 0x7f2b9c0afc90 2023-11-26T07:11:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.502+0000 7f2b9959a700 1 -- 172.21.15.83:0/3046113692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c011d20 con 0x7f2b9c0afc90 2023-11-26T07:11:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.503+0000 7f2ba0d12700 1 -- 172.21.15.83:0/3046113692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b9c0afc90 msgr2=0x7f2b9c073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.503+0000 7f2ba0d12700 1 --2- 172.21.15.83:0/3046113692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b9c0afc90 0x7f2b9c073f90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009840 tx=0x7f2b8c009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.503+0000 7f2ba0d12700 1 -- 172.21.15.83:0/3046113692 shutdown_connections 2023-11-26T07:11:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.503+0000 7f2ba0d12700 1 --2- 172.21.15.83:0/3046113692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b9c15e160 0x7f2b9c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.503+0000 7f2ba0d12700 1 --2- 172.21.15.83:0/3046113692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b9c0afc90 0x7f2b9c073f90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.503+0000 7f2ba0d12700 1 -- 172.21.15.83:0/3046113692 >> 172.21.15.83:0/3046113692 conn(0x7f2b9c0d0bf0 msgr2=0x7f2b9c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.503+0000 7f2ba0d12700 1 -- 172.21.15.83:0/3046113692 shutdown_connections 2023-11-26T07:11:38.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.503+0000 7f2ba0d12700 1 -- 172.21.15.83:0/3046113692 wait complete. 2023-11-26T07:11:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.504+0000 7f2ba0d12700 1 Processor -- start 2023-11-26T07:11:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.504+0000 7f2ba0d12700 1 -- start start 2023-11-26T07:11:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2ba0d12700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b9c0744d0 0x7f2b9c15f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2ba0d12700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b9c0afc90 0x7f2b9c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2ba0d12700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b9c15e160 0x7f2b9c15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2ba0d12700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c160270 con 0x7f2b9c0744d0 2023-11-26T07:11:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2ba0d12700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c145f00 con 0x7f2b9c15e160 2023-11-26T07:11:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2ba0d12700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c155040 con 0x7f2b9c0afc90 2023-11-26T07:11:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b9a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b9c0744d0 0x7f2b9c15f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b9a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b9c0744d0 0x7f2b9c15f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34622/0 (socket says 172.21.15.83:34622) 2023-11-26T07:11:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b9a59c700 1 -- 172.21.15.83:0/2952153640 learned_addr learned my addr 172.21.15.83:0/2952153640 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b99d9b700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b9c0afc90 0x7f2b9c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b9ad9d700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b9c15e160 0x7f2b9c15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b9a59c700 1 -- 172.21.15.83:0/2952153640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b9c0afc90 msgr2=0x7f2b9c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b9a59c700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b9c0afc90 0x7f2b9c159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b9a59c700 1 -- 172.21.15.83:0/2952153640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b9c15e160 msgr2=0x7f2b9c15a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b9a59c700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b9c15e160 0x7f2b9c15a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b9a59c700 1 -- 172.21.15.83:0/2952153640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c0094a0 con 0x7f2b9c0744d0 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b9ad9d700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b9c15e160 0x7f2b9c15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b9a59c700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b9c0744d0 0x7f2b9c15f1f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000c7f0 tx=0x7f2b9000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b8affd700 1 -- 172.21.15.83:0/2952153640 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90011800 con 0x7f2b9c0744d0 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b8affd700 1 -- 172.21.15.83:0/2952153640 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b90011960 con 0x7f2b9c0744d0 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2ba0d12700 1 -- 172.21.15.83:0/2952153640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c15f730 con 0x7f2b9c0744d0 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2b8affd700 1 -- 172.21.15.83:0/2952153640 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90011cf0 con 0x7f2b9c0744d0 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.505+0000 7f2ba0d12700 1 -- 172.21.15.83:0/2952153640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c15fcb0 con 0x7f2b9c0744d0 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.506+0000 7f2ba0d12700 1 -- 172.21.15.83:0/2952153640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c068fb0 con 0x7f2b9c0744d0 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.507+0000 7f2b8affd700 1 -- 172.21.15.83:0/2952153640 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b90018360 con 0x7f2b9c0744d0 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.507+0000 7f2b8affd700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2b7c06aed0 0x7f2b7c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.507+0000 7f2b99d9b700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2b7c06aed0 0x7f2b7c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.508+0000 7f2b8affd700 1 -- 172.21.15.83:0/2952153640 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b900894f0 con 0x7f2b9c0744d0 2023-11-26T07:11:38.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.508+0000 7f2b99d9b700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2b7c06aed0 0x7f2b7c06d390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c000c00 tx=0x7f2b8c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.512+0000 7f2b8affd700 1 -- 172.21.15.83:0/2952153640 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b900545a0 con 0x7f2b9c0744d0 2023-11-26T07:11:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.701+0000 7f2ba0d12700 1 -- 172.21.15.83:0/2952153640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b9c06c140 con 0x7f2b9c0744d0 2023-11-26T07:11:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.702+0000 7f2b8affd700 1 -- 172.21.15.83:0/2952153640 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b90057bc0 con 0x7f2b9c0744d0 2023-11-26T07:11:38.704 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:11:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 -- 172.21.15.83:0/2952153640 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2b7c06aed0 msgr2=0x7f2b7c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2b7c06aed0 0x7f2b7c06d390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c000c00 tx=0x7f2b8c008030 comp rx=0 tx=0).stop 2023-11-26T07:11:38.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 -- 172.21.15.83:0/2952153640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b9c0744d0 msgr2=0x7f2b9c15f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b9c0744d0 0x7f2b9c15f1f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000c7f0 tx=0x7f2b9000cbb0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 -- 172.21.15.83:0/2952153640 shutdown_connections 2023-11-26T07:11:38.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b9c15e160 0x7f2b9c15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b9c0744d0 0x7f2b9c15f1f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2b7c06aed0 0x7f2b7c06d390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 --2- 172.21.15.83:0/2952153640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b9c0afc90 0x7f2b9c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 -- 172.21.15.83:0/2952153640 >> 172.21.15.83:0/2952153640 conn(0x7f2b9c0d0bf0 msgr2=0x7f2b9c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 -- 172.21.15.83:0/2952153640 shutdown_connections 2023-11-26T07:11:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.704+0000 7f2ba0d12700 1 -- 172.21.15.83:0/2952153640 wait complete. 2023-11-26T07:11:38.717 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:11:38.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.838+0000 7f7f61b6c700 1 Processor -- start 2023-11-26T07:11:38.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.838+0000 7f7f61b6c700 1 -- start start 2023-11-26T07:11:38.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f61b6c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7f5c15e160 0x7f7f5c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f61b6c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7f5c0afc90 0x7f7f5c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f61b6c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0744d0 0x7f7f5c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f61b6c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5c0b4500 con 0x7f7f5c0afc90 2023-11-26T07:11:38.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f61b6c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5c074960 con 0x7f7f5c15e160 2023-11-26T07:11:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f61b6c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c0b3c60 con 0x7f7f5c0744d0 2023-11-26T07:11:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5affd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7f5c0afc90 0x7f7f5c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5affd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7f5c0afc90 0x7f7f5c073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34630/0 (socket says 172.21.15.83:34630) 2023-11-26T07:11:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5affd700 1 -- 172.21.15.83:0/3229131322 learned_addr learned my addr 172.21.15.83:0/3229131322 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5bfff700 1 --2- 172.21.15.83:0/3229131322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0744d0 0x7f7f5c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5b7fe700 1 --2- 172.21.15.83:0/3229131322 >> v2:172.21.15.203:3300/0 conn(0x7f7f5c15e160 0x7f7f5c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5affd700 1 -- 172.21.15.83:0/3229131322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0744d0 msgr2=0x7f7f5c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5affd700 1 --2- 172.21.15.83:0/3229131322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0744d0 0x7f7f5c06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5affd700 1 -- 172.21.15.83:0/3229131322 >> v2:172.21.15.203:3300/0 conn(0x7f7f5c15e160 msgr2=0x7f7f5c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:38.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5affd700 1 --2- 172.21.15.83:0/3229131322 >> v2:172.21.15.203:3300/0 conn(0x7f7f5c15e160 0x7f7f5c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5affd700 1 -- 172.21.15.83:0/3229131322 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c0b58c0 con 0x7f7f5c0afc90 2023-11-26T07:11:38.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5b7fe700 1 --2- 172.21.15.83:0/3229131322 >> v2:172.21.15.203:3300/0 conn(0x7f7f5c15e160 0x7f7f5c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:38.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.839+0000 7f7f5bfff700 1 --2- 172.21.15.83:0/3229131322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0744d0 0x7f7f5c06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:38.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f5affd700 1 --2- 172.21.15.83:0/3229131322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f5c0afc90 0x7f7f5c073f90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c09efa0 tx=0x7f7f4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b68095e48df8596 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:38.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f5a7fc700 1 -- 172.21.15.83:0/3229131322 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00b870 con 0x7f7f5c0afc90 2023-11-26T07:11:38.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f5a7fc700 1 -- 172.21.15.83:0/3229131322 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c00b9d0 con 0x7f7f5c0afc90 2023-11-26T07:11:38.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f5a7fc700 1 -- 172.21.15.83:0/3229131322 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00bd60 con 0x7f7f5c0afc90 2023-11-26T07:11:38.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f61b6c700 1 -- 172.21.15.83:0/3229131322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f5c0afc90 msgr2=0x7f7f5c073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f61b6c700 1 --2- 172.21.15.83:0/3229131322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f5c0afc90 0x7f7f5c073f90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c09efa0 tx=0x7f7f4c009bd0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f61b6c700 1 -- 172.21.15.83:0/3229131322 shutdown_connections 2023-11-26T07:11:38.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f61b6c700 1 --2- 172.21.15.83:0/3229131322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0744d0 0x7f7f5c06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f61b6c700 1 --2- 172.21.15.83:0/3229131322 >> v2:172.21.15.203:3300/0 conn(0x7f7f5c15e160 0x7f7f5c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f61b6c700 1 --2- 172.21.15.83:0/3229131322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f5c0afc90 0x7f7f5c073f90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f61b6c700 1 -- 172.21.15.83:0/3229131322 >> 172.21.15.83:0/3229131322 conn(0x7f7f5c0d0bf0 msgr2=0x7f7f5c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:38.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f61b6c700 1 -- 172.21.15.83:0/3229131322 shutdown_connections 2023-11-26T07:11:38.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.840+0000 7f7f61b6c700 1 -- 172.21.15.83:0/3229131322 wait complete. 2023-11-26T07:11:38.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.841+0000 7f7f61b6c700 1 Processor -- start 2023-11-26T07:11:38.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.842+0000 7f7f61b6c700 1 -- start start 2023-11-26T07:11:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.842+0000 7f7f61b6c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0744d0 0x7f7f5c14a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.842+0000 7f7f61b6c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f5c0afc90 0x7f7f5c156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.842+0000 7f7f61b6c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f5c15e160 0x7f7f5c1610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.842+0000 7f7f61b6c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c161610 con 0x7f7f5c0afc90 2023-11-26T07:11:38.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.842+0000 7f7f61b6c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c14b220 con 0x7f7f5c15e160 2023-11-26T07:11:38.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.842+0000 7f7f61b6c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c156c10 con 0x7f7f5c0744d0 2023-11-26T07:11:38.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f5affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f5c0afc90 0x7f7f5c156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f5affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f5c0afc90 0x7f7f5c156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34634/0 (socket says 172.21.15.83:34634) 2023-11-26T07:11:38.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f5affd700 1 -- 172.21.15.83:0/4182143454 learned_addr learned my addr 172.21.15.83:0/4182143454 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:38.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f5b7fe700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0744d0 0x7f7f5c14a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f5bfff700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f5c15e160 0x7f7f5c1610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f5affd700 1 -- 172.21.15.83:0/4182143454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0744d0 msgr2=0x7f7f5c14a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f5affd700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0744d0 0x7f7f5c14a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f5affd700 1 -- 172.21.15.83:0/4182143454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f5c15e160 msgr2=0x7f7f5c1610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f5affd700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f5c15e160 0x7f7f5c1610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f5affd700 1 -- 172.21.15.83:0/4182143454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c009550 con 0x7f7f5c0afc90 2023-11-26T07:11:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f5affd700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f5c0afc90 0x7f7f5c156510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c072ed0 tx=0x7f7f4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f53fff700 1 -- 172.21.15.83:0/4182143454 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c004310 con 0x7f7f5c0afc90 2023-11-26T07:11:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f53fff700 1 -- 172.21.15.83:0/4182143454 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c01f070 con 0x7f7f5c0afc90 2023-11-26T07:11:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f61b6c700 1 -- 172.21.15.83:0/4182143454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5c156ef0 con 0x7f7f5c0afc90 2023-11-26T07:11:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f53fff700 1 -- 172.21.15.83:0/4182143454 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0036a0 con 0x7f7f5c0afc90 2023-11-26T07:11:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.843+0000 7f7f61b6c700 1 -- 172.21.15.83:0/4182143454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5c147270 con 0x7f7f5c0afc90 2023-11-26T07:11:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.844+0000 7f7f61b6c700 1 -- 172.21.15.83:0/4182143454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c0b48e0 con 0x7f7f5c0afc90 2023-11-26T07:11:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.845+0000 7f7f53fff700 1 -- 172.21.15.83:0/4182143454 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7f4c021030 con 0x7f7f5c0afc90 2023-11-26T07:11:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.845+0000 7f7f53fff700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7f3c06aed0 0x7f7f3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.845+0000 7f7f53fff700 1 -- 172.21.15.83:0/4182143454 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f4c08abd0 con 0x7f7f5c0afc90 2023-11-26T07:11:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.845+0000 7f7f5b7fe700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7f3c06aed0 0x7f7f3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.846+0000 7f7f5b7fe700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7f3c06aed0 0x7f7f3c06d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c1f6ed0 tx=0x7f7f44008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:38.849+0000 7f7f53fff700 1 -- 172.21.15.83:0/4182143454 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f4c055df0 con 0x7f7f5c0afc90 2023-11-26T07:11:39.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.037+0000 7f7f61b6c700 1 -- 172.21.15.83:0/4182143454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f5c147490 con 0x7f7f5c0afc90 2023-11-26T07:11:39.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.038+0000 7f7f53fff700 1 -- 172.21.15.83:0/4182143454 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f4c059410 con 0x7f7f5c0afc90 2023-11-26T07:11:39.040 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:11:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 -- 172.21.15.83:0/4182143454 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7f3c06aed0 msgr2=0x7f7f3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7f3c06aed0 0x7f7f3c06d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c1f6ed0 tx=0x7f7f44008040 comp rx=0 tx=0).stop 2023-11-26T07:11:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 -- 172.21.15.83:0/4182143454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f5c0afc90 msgr2=0x7f7f5c156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f5c0afc90 0x7f7f5c156510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c072ed0 tx=0x7f7f4c004750 comp rx=0 tx=0).stop 2023-11-26T07:11:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 -- 172.21.15.83:0/4182143454 shutdown_connections 2023-11-26T07:11:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f5c15e160 0x7f7f5c1610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7f3c06aed0 0x7f7f3c06d390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0744d0 0x7f7f5c14a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 --2- 172.21.15.83:0/4182143454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f5c0afc90 0x7f7f5c156510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 -- 172.21.15.83:0/4182143454 >> 172.21.15.83:0/4182143454 conn(0x7f7f5c0d0bf0 msgr2=0x7f7f5c162530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 -- 172.21.15.83:0/4182143454 shutdown_connections 2023-11-26T07:11:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.040+0000 7f7f61b6c700 1 -- 172.21.15.83:0/4182143454 wait complete. 2023-11-26T07:11:39.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:11:39.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.175+0000 7faa45a27700 1 Processor -- start 2023-11-26T07:11:39.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.175+0000 7faa45a27700 1 -- start start 2023-11-26T07:11:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa45a27700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa4015e140 0x7faa400d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa45a27700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faa400b98c0 0x7faa400d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa45a27700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa400b80e0 con 0x7faa400d39d0 2023-11-26T07:11:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa45a27700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4015e520 con 0x7faa400b98c0 2023-11-26T07:11:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa45a27700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa400d2930 con 0x7faa4015e140 2023-11-26T07:11:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa3f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faa400b98c0 0x7faa400d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa3effd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa4015e140 0x7faa400d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa3e7fc700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7faa400d39d0 0x7faa400b59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:39300/0 (socket says 172.21.15.83:39300) 2023-11-26T07:11:39.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa3f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faa400b98c0 0x7faa400d4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34098/0 (socket says 172.21.15.83:34098) 2023-11-26T07:11:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa3e7fc700 1 -- 172.21.15.83:0/271255268 learned_addr learned my addr 172.21.15.83:0/271255268 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa3f7fe700 1 -- 172.21.15.83:0/271255268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa4015e140 msgr2=0x7faa400d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa3f7fe700 1 --2- 172.21.15.83:0/271255268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa4015e140 0x7faa400d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa3f7fe700 1 -- 172.21.15.83:0/271255268 >> v1:172.21.15.203:6789/0 conn(0x7faa400d39d0 legacy=0x7faa400b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa3dffb700 1 -- 172.21.15.83:0/271255268 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433746303 0 0) 0x7faa34001110 con 0x7faa400d39d0 2023-11-26T07:11:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa3f7fe700 1 -- 172.21.15.83:0/271255268 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa400d2c10 con 0x7faa400b98c0 2023-11-26T07:11:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.176+0000 7faa3f7fe700 1 --2- 172.21.15.83:0/271255268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa400b98c0 0x7faa400d4400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa38009d90 tx=0x7faa3800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33f72f6b82b8702d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.177+0000 7faa3dffb700 1 -- 172.21.15.83:0/271255268 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3800e070 con 0x7faa400b98c0 2023-11-26T07:11:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.177+0000 7faa3dffb700 1 -- 172.21.15.83:0/271255268 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3800bb20 con 0x7faa400b98c0 2023-11-26T07:11:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.177+0000 7faa3dffb700 1 -- 172.21.15.83:0/271255268 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38012430 con 0x7faa400b98c0 2023-11-26T07:11:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.177+0000 7faa45a27700 1 -- 172.21.15.83:0/271255268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa400b98c0 msgr2=0x7faa400d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.177+0000 7faa45a27700 1 --2- 172.21.15.83:0/271255268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa400b98c0 0x7faa400d4400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa38009d90 tx=0x7faa3800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.177+0000 7faa45a27700 1 -- 172.21.15.83:0/271255268 shutdown_connections 2023-11-26T07:11:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.177+0000 7faa45a27700 1 --2- 172.21.15.83:0/271255268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa400b98c0 0x7faa400d4400 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.177+0000 7faa45a27700 1 --2- 172.21.15.83:0/271255268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa4015e140 0x7faa400d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.177+0000 7faa45a27700 1 -- 172.21.15.83:0/271255268 >> 172.21.15.83:0/271255268 conn(0x7faa4006b410 msgr2=0x7faa4006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.177+0000 7faa45a27700 1 -- 172.21.15.83:0/271255268 shutdown_connections 2023-11-26T07:11:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.177+0000 7faa45a27700 1 -- 172.21.15.83:0/271255268 wait complete. 2023-11-26T07:11:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.178+0000 7faa45a27700 1 Processor -- start 2023-11-26T07:11:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.179+0000 7faa45a27700 1 -- start start 2023-11-26T07:11:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.179+0000 7faa45a27700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa400b98c0 0x7faa401f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.179+0000 7faa45a27700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa400d39d0 0x7faa4015eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.179+0000 7faa45a27700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4015e140 0x7faa4015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.179+0000 7faa45a27700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa400cdf80 con 0x7faa4015e140 2023-11-26T07:11:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.179+0000 7faa45a27700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa401f4830 con 0x7faa400d39d0 2023-11-26T07:11:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.179+0000 7faa45a27700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa401f3a70 con 0x7faa400b98c0 2023-11-26T07:11:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4015e140 0x7faa4015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4015e140 0x7faa4015f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34652/0 (socket says 172.21.15.83:34652) 2023-11-26T07:11:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa3f7fe700 1 -- 172.21.15.83:0/3523179183 learned_addr learned my addr 172.21.15.83:0/3523179183 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa3effd700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa400b98c0 0x7faa401f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa3e7fc700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa400d39d0 0x7faa4015eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa3f7fe700 1 -- 172.21.15.83:0/3523179183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa400b98c0 msgr2=0x7faa401f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa3f7fe700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa400b98c0 0x7faa401f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa3f7fe700 1 -- 172.21.15.83:0/3523179183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa400d39d0 msgr2=0x7faa4015eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa3f7fe700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa400d39d0 0x7faa4015eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa3f7fe700 1 -- 172.21.15.83:0/3523179183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa38009a70 con 0x7faa4015e140 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa3f7fe700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4015e140 0x7faa4015f0e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa38013000 tx=0x7faa38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa27fff700 1 -- 172.21.15.83:0/3523179183 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38012430 con 0x7faa4015e140 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa27fff700 1 -- 172.21.15.83:0/3523179183 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa38020070 con 0x7faa4015e140 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa45a27700 1 -- 172.21.15.83:0/3523179183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa401f3d50 con 0x7faa4015e140 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa27fff700 1 -- 172.21.15.83:0/3523179183 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38004280 con 0x7faa4015e140 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.180+0000 7faa45a27700 1 -- 172.21.15.83:0/3523179183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa401f4310 con 0x7faa4015e140 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.181+0000 7faa45a27700 1 -- 172.21.15.83:0/3523179183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa401f44d0 con 0x7faa4015e140 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.181+0000 7faa27fff700 1 -- 172.21.15.83:0/3523179183 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa380043e0 con 0x7faa4015e140 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.182+0000 7faa27fff700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faa2c06aed0 0x7faa2c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.182+0000 7faa3effd700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faa2c06aed0 0x7faa2c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.182+0000 7faa27fff700 1 -- 172.21.15.83:0/3523179183 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa38022070 con 0x7faa4015e140 2023-11-26T07:11:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.182+0000 7faa3effd700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faa2c06aed0 0x7faa2c06d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faa401527c0 tx=0x7faa30009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.186+0000 7faa27fff700 1 -- 172.21.15.83:0/3523179183 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa380566f0 con 0x7faa4015e140 2023-11-26T07:11:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.377+0000 7faa45a27700 1 -- 172.21.15.83:0/3523179183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa40068fb0 con 0x7faa4015e140 2023-11-26T07:11:39.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.377+0000 7faa27fff700 1 -- 172.21.15.83:0/3523179183 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa38059d10 con 0x7faa4015e140 2023-11-26T07:11:39.379 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:11:39.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.379+0000 7faa45a27700 1 -- 172.21.15.83:0/3523179183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faa2c06aed0 msgr2=0x7faa2c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.379+0000 7faa45a27700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faa2c06aed0 0x7faa2c06d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faa401527c0 tx=0x7faa30009210 comp rx=0 tx=0).stop 2023-11-26T07:11:39.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.379+0000 7faa45a27700 1 -- 172.21.15.83:0/3523179183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4015e140 msgr2=0x7faa4015f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.379+0000 7faa45a27700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4015e140 0x7faa4015f0e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa38013000 tx=0x7faa38004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.379+0000 7faa45a27700 1 -- 172.21.15.83:0/3523179183 shutdown_connections 2023-11-26T07:11:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.379+0000 7faa45a27700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4015e140 0x7faa4015f0e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.379+0000 7faa45a27700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faa2c06aed0 0x7faa2c06d390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.379+0000 7faa45a27700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa400b98c0 0x7faa401f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.379+0000 7faa45a27700 1 --2- 172.21.15.83:0/3523179183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa400d39d0 0x7faa4015eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.379+0000 7faa45a27700 1 -- 172.21.15.83:0/3523179183 >> 172.21.15.83:0/3523179183 conn(0x7faa4006b410 msgr2=0x7faa400b72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.380+0000 7faa45a27700 1 -- 172.21.15.83:0/3523179183 shutdown_connections 2023-11-26T07:11:39.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.380+0000 7faa45a27700 1 -- 172.21.15.83:0/3523179183 wait complete. 2023-11-26T07:11:39.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:11:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.515+0000 7f655830d700 1 Processor -- start 2023-11-26T07:11:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.515+0000 7f655830d700 1 -- start start 2023-11-26T07:11:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f655830d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f65500af4d0 0x7f65500b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f655830d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65500ae430 con 0x7f655015e080 2023-11-26T07:11:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f655830d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65500ae710 con 0x7f655006c710 2023-11-26T07:11:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f655830d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65500af0b0 con 0x7f65500af4d0 2023-11-26T07:11:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65560a9700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f655015e080 0x7f655006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54696/0 (socket says 172.21.15.83:54696) 2023-11-26T07:11:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65560a9700 1 -- 172.21.15.83:0/1005754218 learned_addr learned my addr 172.21.15.83:0/1005754218 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65568aa700 1 --2- 172.21.15.83:0/1005754218 >> v2:172.21.15.203:3300/0 conn(0x7f65500af4d0 0x7f65500b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799776391 0 0) 0x7f65500ae430 con 0x7f655015e080 2023-11-26T07:11:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f653c0035b0 con 0x7f655015e080 2023-11-26T07:11:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1879742609 0 0) 0x7f653c0035b0 con 0x7f655015e080 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 >> v2:172.21.15.203:3300/0 conn(0x7f65500af4d0 msgr2=0x7f65500b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 --2- 172.21.15.83:0/1005754218 >> v2:172.21.15.203:3300/0 conn(0x7f65500af4d0 0x7f65500b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 >> v1:172.21.15.203:6789/0 conn(0x7f655006c710 legacy=0x7f655006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65500ae9f0 con 0x7f655015e080 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f654c001670 con 0x7f655015e080 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 >> v1:172.21.15.83:6789/0 conn(0x7f655015e080 legacy=0x7f655006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 --2- 172.21.15.83:0/1005754218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c003a00 0x7f653c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 --2- 172.21.15.83:0/1005754218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c004850 0x7f653c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 --2- 172.21.15.83:0/1005754218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 0x7f653c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.516+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65500af0b0 con 0x7f653c005210 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c00c0b0 con 0x7f653c004850 2023-11-26T07:11:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c00c270 con 0x7f653c003a00 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65568aa700 1 --2- 172.21.15.83:0/1005754218 >> v2:172.21.15.203:3300/0 conn(0x7f65500af4d0 0x7f65500b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65558a8700 1 --2- 172.21.15.83:0/1005754218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c004850 0x7f653c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65568aa700 1 --2- 172.21.15.83:0/1005754218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 0x7f653c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65560a9700 1 --2- 172.21.15.83:0/1005754218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c003a00 0x7f653c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65568aa700 1 -- 172.21.15.83:0/1005754218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c003a00 msgr2=0x7f653c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65568aa700 1 --2- 172.21.15.83:0/1005754218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f653c003a00 0x7f653c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65568aa700 1 -- 172.21.15.83:0/1005754218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c004850 msgr2=0x7f653c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65568aa700 1 --2- 172.21.15.83:0/1005754218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f653c004850 0x7f653c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65568aa700 1 -- 172.21.15.83:0/1005754218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c0035b0 con 0x7f653c005210 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65568aa700 1 -- 172.21.15.83:0/1005754218 reap_dead start 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65568aa700 1 --2- 172.21.15.83:0/1005754218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 0x7f653c00ba10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f654800e350 tx=0x7f653c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3299f05ce67fc916 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654800ee10 con 0x7f653c005210 2023-11-26T07:11:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654800f040 con 0x7f653c005210 2023-11-26T07:11:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.517+0000 7f65550a7700 1 -- 172.21.15.83:0/1005754218 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548003c60 con 0x7f653c005210 2023-11-26T07:11:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.518+0000 7f655830d700 1 -- 172.21.15.83:0/1005754218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 msgr2=0x7f653c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.518+0000 7f655830d700 1 --2- 172.21.15.83:0/1005754218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 0x7f653c00ba10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f654800e350 tx=0x7f653c005880 comp rx=0 tx=0).stop 2023-11-26T07:11:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.518+0000 7f655830d700 1 -- 172.21.15.83:0/1005754218 shutdown_connections 2023-11-26T07:11:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.518+0000 7f655830d700 1 --2- 172.21.15.83:0/1005754218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 0x7f653c00ba10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.518+0000 7f655830d700 1 -- 172.21.15.83:0/1005754218 >> 172.21.15.83:0/1005754218 conn(0x7f6550071b90 msgr2=0x7f6550071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.518+0000 7f655830d700 1 -- 172.21.15.83:0/1005754218 shutdown_connections 2023-11-26T07:11:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.518+0000 7f655830d700 1 -- 172.21.15.83:0/1005754218 wait complete. 2023-11-26T07:11:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.519+0000 7f655830d700 1 Processor -- start 2023-11-26T07:11:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.519+0000 7f655830d700 1 -- start start 2023-11-26T07:11:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.520+0000 7f655830d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 0x7f65501420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.520+0000 7f655830d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6550153cb0 0x7f6550154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.520+0000 7f655830d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65500ce000 0x7f65500ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.520+0000 7f655830d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6550154670 con 0x7f653c005210 2023-11-26T07:11:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.520+0000 7f655830d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65500ced30 con 0x7f6550153cb0 2023-11-26T07:11:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.520+0000 7f655830d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65500cf010 con 0x7f65500ce000 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.520+0000 7f65560a9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 0x7f65501420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.520+0000 7f65560a9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 0x7f65501420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34666/0 (socket says 172.21.15.83:34666) 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.520+0000 7f65560a9700 1 -- 172.21.15.83:0/1351293665 learned_addr learned my addr 172.21.15.83:0/1351293665 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.521+0000 7f65560a9700 1 -- 172.21.15.83:0/1351293665 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65500ce000 msgr2=0x7f65500ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.521+0000 7f65560a9700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65500ce000 0x7f65500ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.521+0000 7f65560a9700 1 -- 172.21.15.83:0/1351293665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6550153cb0 msgr2=0x7f6550154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.521+0000 7f65560a9700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6550153cb0 0x7f6550154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.521+0000 7f65560a9700 1 -- 172.21.15.83:0/1351293665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65500cf2f0 con 0x7f653c005210 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.521+0000 7f65560a9700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 0x7f65501420e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f654c007290 tx=0x7f654c0075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.521+0000 7f6546ffd700 1 -- 172.21.15.83:0/1351293665 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c00f660 con 0x7f653c005210 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.521+0000 7f6546ffd700 1 -- 172.21.15.83:0/1351293665 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c00f7c0 con 0x7f653c005210 2023-11-26T07:11:39.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.521+0000 7f655830d700 1 -- 172.21.15.83:0/1351293665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65500cf5d0 con 0x7f653c005210 2023-11-26T07:11:39.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.521+0000 7f6546ffd700 1 -- 172.21.15.83:0/1351293665 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c00fb70 con 0x7f653c005210 2023-11-26T07:11:39.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.521+0000 7f655830d700 1 -- 172.21.15.83:0/1351293665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65500ce9c0 con 0x7f653c005210 2023-11-26T07:11:39.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.522+0000 7f655830d700 1 -- 172.21.15.83:0/1351293665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65501428e0 con 0x7f653c005210 2023-11-26T07:11:39.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.522+0000 7f6546ffd700 1 -- 172.21.15.83:0/1351293665 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f654c00fcd0 con 0x7f653c005210 2023-11-26T07:11:39.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.523+0000 7f6546ffd700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f653406aed0 0x7f653406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.523+0000 7f65558a8700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f653406aed0 0x7f653406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.523+0000 7f6546ffd700 1 -- 172.21.15.83:0/1351293665 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f654c0895b0 con 0x7f653c005210 2023-11-26T07:11:39.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.523+0000 7f65558a8700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f653406aed0 0x7f653406d390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6540000c00 tx=0x7f6540009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:39.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.527+0000 7f6546ffd700 1 -- 172.21.15.83:0/1351293665 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f654c054660 con 0x7f653c005210 2023-11-26T07:11:39.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.759+0000 7f655830d700 1 -- 172.21.15.83:0/1351293665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65500753f0 con 0x7f653c005210 2023-11-26T07:11:39.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.760+0000 7f6546ffd700 1 -- 172.21.15.83:0/1351293665 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f654c057c80 con 0x7f653c005210 2023-11-26T07:11:39.762 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:39.762 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2208741688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2208741688},{"type":"v1","addr":"172.21.15.83:6837","nonce":2208741688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2802126359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2802126359},{"type":"v1","addr":"172.21.15.83:6836","nonce":2802126359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1258871616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1258871616},{"type":"v1","addr":"172.21.15.203:6834","nonce":1258871616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:11:27.911475+0000","modified":"2023-11-26T07:11:30.937105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/2957324266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":2957324266},{"type":"v1","addr":"172.21.15.203:6835","nonce":2957324266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:11:39.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 -- 172.21.15.83:0/1351293665 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f653406aed0 msgr2=0x7f653406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f653406aed0 0x7f653406d390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6540000c00 tx=0x7f6540009040 comp rx=0 tx=0).stop 2023-11-26T07:11:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 -- 172.21.15.83:0/1351293665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 msgr2=0x7f65501420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 0x7f65501420e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f654c007290 tx=0x7f654c0075a0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 -- 172.21.15.83:0/1351293665 shutdown_connections 2023-11-26T07:11:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f653406aed0 0x7f653406d390 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65500ce000 0x7f65500ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f653c005210 0x7f65501420e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 --2- 172.21.15.83:0/1351293665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6550153cb0 0x7f6550154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 -- 172.21.15.83:0/1351293665 >> 172.21.15.83:0/1351293665 conn(0x7f6550071b90 msgr2=0x7f655006f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 -- 172.21.15.83:0/1351293665 shutdown_connections 2023-11-26T07:11:39.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.762+0000 7f655830d700 1 -- 172.21.15.83:0/1351293665 wait complete. 2023-11-26T07:11:39.765 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2023-11-26T07:11:39.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:39.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.893+0000 7f24b959d700 1 Processor -- start 2023-11-26T07:11:39.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.893+0000 7f24b959d700 1 -- start start 2023-11-26T07:11:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b959d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f24b40d0870 0x7f24b40d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b959d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b40b6ef0 0x7f24b40b7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b959d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b40d20b0 con 0x7f24b415e1e0 2023-11-26T07:11:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b959d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b40d3150 con 0x7f24b40d0870 2023-11-26T07:11:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b959d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40d1240 con 0x7f24b40b6ef0 2023-11-26T07:11:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b2ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f24b415e1e0 0x7f24b40d34c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54712/0 (socket says 172.21.15.83:54712) 2023-11-26T07:11:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b2ffd700 1 -- 172.21.15.83:0/1468813988 learned_addr learned my addr 172.21.15.83:0/1468813988 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719601513 0 0) 0x7f24b40d20b0 con 0x7f24b415e1e0 2023-11-26T07:11:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f249c0035b0 con 0x7f24b415e1e0 2023-11-26T07:11:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b27fc700 1 --2- 172.21.15.83:0/1468813988 >> v2:172.21.15.203:3300/0 conn(0x7f24b40d0870 0x7f24b40d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b37fe700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b40b6ef0 0x7f24b40b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 827719854 0 0) 0x7f249c0035b0 con 0x7f24b415e1e0 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b40b6ef0 msgr2=0x7f24b40b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b40b6ef0 0x7f24b40b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 >> v2:172.21.15.203:3300/0 conn(0x7f24b40d0870 msgr2=0x7f24b40d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 --2- 172.21.15.83:0/1468813988 >> v2:172.21.15.203:3300/0 conn(0x7f24b40d0870 0x7f24b40d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b40d2390 con 0x7f24b415e1e0 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f24a4001670 con 0x7f24b415e1e0 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 >> v1:172.21.15.83:6789/0 conn(0x7f24b415e1e0 legacy=0x7f24b40d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f249c003a00 0x7f249c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f249c004850 0x7f249c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b40b6ef0 0x7f249c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40d3150 con 0x7f249c003a00 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40d1240 con 0x7f249c004850 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249c00bb50 con 0x7f24b40b6ef0 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.894+0000 7f24b2ffd700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f249c003a00 0x7f249c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b37fe700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b40b6ef0 0x7f249c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b27fc700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f249c004850 0x7f249c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b2ffd700 1 -- 172.21.15.83:0/1468813988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b40b6ef0 msgr2=0x7f249c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b2ffd700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b40b6ef0 0x7f249c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b2ffd700 1 -- 172.21.15.83:0/1468813988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f249c004850 msgr2=0x7f249c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b2ffd700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f249c004850 0x7f249c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b2ffd700 1 -- 172.21.15.83:0/1468813988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c0035b0 con 0x7f249c003a00 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b2ffd700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f249c003a00 0x7f249c003e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f24a4001050 tx=0x7f249c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d84de7886405843 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4007c60 con 0x7f249c003a00 2023-11-26T07:11:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a400ce80 con 0x7f249c003a00 2023-11-26T07:11:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b1ffb700 1 -- 172.21.15.83:0/1468813988 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4005ba0 con 0x7f249c003a00 2023-11-26T07:11:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b959d700 1 -- 172.21.15.83:0/1468813988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f249c003a00 msgr2=0x7f249c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.895+0000 7f24b959d700 1 --2- 172.21.15.83:0/1468813988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f249c003a00 0x7f249c003e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f24a4001050 tx=0x7f249c005880 comp rx=0 tx=0).stop 2023-11-26T07:11:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.896+0000 7f24b37fe700 1 -- 172.21.15.83:0/1468813988 reap_dead start 2023-11-26T07:11:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.896+0000 7f24b959d700 1 -- 172.21.15.83:0/1468813988 shutdown_connections 2023-11-26T07:11:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.896+0000 7f24b959d700 1 -- 172.21.15.83:0/1468813988 >> 172.21.15.83:0/1468813988 conn(0x7f24b40b5b50 msgr2=0x7f24b40b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.896+0000 7f24b959d700 1 -- 172.21.15.83:0/1468813988 shutdown_connections 2023-11-26T07:11:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.896+0000 7f24b959d700 1 -- 172.21.15.83:0/1468813988 wait complete. 2023-11-26T07:11:39.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.897+0000 7f24b959d700 1 Processor -- start 2023-11-26T07:11:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.897+0000 7f24b959d700 1 -- start start 2023-11-26T07:11:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b959d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f249c003a00 0x7f24b40caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b959d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b4151040 0x7f24b40ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b959d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b41502e0 0x7f24b4150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b959d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40b4d80 con 0x7f24b41502e0 2023-11-26T07:11:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b959d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b4160fa0 con 0x7f249c003a00 2023-11-26T07:11:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b959d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40b42e0 con 0x7f24b4151040 2023-11-26T07:11:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b41502e0 0x7f24b4150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b41502e0 0x7f24b4150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34688/0 (socket says 172.21.15.83:34688) 2023-11-26T07:11:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b37fe700 1 -- 172.21.15.83:0/2343459674 learned_addr learned my addr 172.21.15.83:0/2343459674 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b2ffd700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f249c003a00 0x7f24b40caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b27fc700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b4151040 0x7f24b40ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b37fe700 1 -- 172.21.15.83:0/2343459674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b4151040 msgr2=0x7f24b40ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b37fe700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b4151040 0x7f24b40ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b37fe700 1 -- 172.21.15.83:0/2343459674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f249c003a00 msgr2=0x7f24b40caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b37fe700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f249c003a00 0x7f24b40caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b37fe700 1 -- 172.21.15.83:0/2343459674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b40cb470 con 0x7f24b41502e0 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b37fe700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b41502e0 0x7f24b4150770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24b4072830 tx=0x7f24ac0048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f249b7fe700 1 -- 172.21.15.83:0/2343459674 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac007900 con 0x7f24b41502e0 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f249b7fe700 1 -- 172.21.15.83:0/2343459674 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ac007a60 con 0x7f24b41502e0 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f249b7fe700 1 -- 172.21.15.83:0/2343459674 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac007df0 con 0x7f24b41502e0 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.898+0000 7f24b959d700 1 -- 172.21.15.83:0/2343459674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b4150cb0 con 0x7f24b41502e0 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.899+0000 7f24b959d700 1 -- 172.21.15.83:0/2343459674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b40b4580 con 0x7f24b41502e0 2023-11-26T07:11:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.899+0000 7f24b959d700 1 -- 172.21.15.83:0/2343459674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b40751e0 con 0x7f24b41502e0 2023-11-26T07:11:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.900+0000 7f249b7fe700 1 -- 172.21.15.83:0/2343459674 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24ac015420 con 0x7f24b41502e0 2023-11-26T07:11:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.900+0000 7f249b7fe700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f24a006aed0 0x7f24a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.900+0000 7f249b7fe700 1 -- 172.21.15.83:0/2343459674 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24ac0899c0 con 0x7f24b41502e0 2023-11-26T07:11:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.900+0000 7f24b2ffd700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f24a006aed0 0x7f24a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.901+0000 7f24b2ffd700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f24a006aed0 0x7f24a006d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f24a4001930 tx=0x7f24a400a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:39.905+0000 7f249b7fe700 1 -- 172.21.15.83:0/2343459674 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24ac054a60 con 0x7f24b41502e0 2023-11-26T07:11:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.094+0000 7f24b959d700 1 -- 172.21.15.83:0/2343459674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24b415ad20 con 0x7f24b41502e0 2023-11-26T07:11:40.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.095+0000 7f249b7fe700 1 -- 172.21.15.83:0/2343459674 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f24ac058080 con 0x7f24b41502e0 2023-11-26T07:11:40.097 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:40.097 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 -- 172.21.15.83:0/2343459674 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f24a006aed0 msgr2=0x7f24a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f24a006aed0 0x7f24a006d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f24a4001930 tx=0x7f24a400a000 comp rx=0 tx=0).stop 2023-11-26T07:11:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 -- 172.21.15.83:0/2343459674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b41502e0 msgr2=0x7f24b4150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b41502e0 0x7f24b4150770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24b4072830 tx=0x7f24ac0048c0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 -- 172.21.15.83:0/2343459674 shutdown_connections 2023-11-26T07:11:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24b41502e0 0x7f24b4150770 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f249c003a00 0x7f24b40caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f24a006aed0 0x7f24a006d390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 --2- 172.21.15.83:0/2343459674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f24b4151040 0x7f24b40ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 -- 172.21.15.83:0/2343459674 >> 172.21.15.83:0/2343459674 conn(0x7f24b40b5b50 msgr2=0x7f24b41533a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 -- 172.21.15.83:0/2343459674 shutdown_connections 2023-11-26T07:11:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.097+0000 7f24b959d700 1 -- 172.21.15.83:0/2343459674 wait complete. 2023-11-26T07:11:40.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:11:40.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:11:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.232+0000 7f05ad2d2700 1 Processor -- start 2023-11-26T07:11:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.232+0000 7f05ad2d2700 1 -- start start 2023-11-26T07:11:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05ad2d2700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05a80d45d0 0x7f05a80b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05ad2d2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a80d2930 con 0x7f05a815e140 2023-11-26T07:11:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05ad2d2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a80d2c10 con 0x7f05a80d39d0 2023-11-26T07:11:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05ad2d2700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a815e520 con 0x7f05a80d45d0 2023-11-26T07:11:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05a6ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f05a815e140 0x7f05a80d3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54732/0 (socket says 172.21.15.83:54732) 2023-11-26T07:11:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05a6ffd700 1 -- 172.21.15.83:0/3487722727 learned_addr learned my addr 172.21.15.83:0/3487722727 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05a67fc700 1 --2- 172.21.15.83:0/3487722727 >> v2:172.21.15.203:3300/0 conn(0x7f05a80d45d0 0x7f05a80b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05a5ffb700 1 -- 172.21.15.83:0/3487722727 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960584528 0 0) 0x7f05a80d2930 con 0x7f05a815e140 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05a5ffb700 1 -- 172.21.15.83:0/3487722727 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05900035b0 con 0x7f05a815e140 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05a5ffb700 1 -- 172.21.15.83:0/3487722727 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611094918 0 0) 0x7f05a80d2c10 con 0x7f05a80d39d0 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05a5ffb700 1 -- 172.21.15.83:0/3487722727 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a80d2930 con 0x7f05a80d39d0 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05a5ffb700 1 -- 172.21.15.83:0/3487722727 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1377110799 0 0) 0x7f05900035b0 con 0x7f05a815e140 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05a67fc700 1 -- 172.21.15.83:0/3487722727 >> v1:172.21.15.203:6789/0 conn(0x7f05a80d39d0 legacy=0x7f05a80b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05a67fc700 1 -- 172.21.15.83:0/3487722727 >> v1:172.21.15.83:6789/0 conn(0x7f05a815e140 legacy=0x7f05a80d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.233+0000 7f05a67fc700 1 -- 172.21.15.83:0/3487722727 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80d2ef0 con 0x7f05a80d45d0 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.234+0000 7f05a67fc700 1 --2- 172.21.15.83:0/3487722727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a80d45d0 0x7f05a80b70d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0598009840 tx=0x7f0598009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a01703d30d9ec565 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.234+0000 7f05a5ffb700 1 -- 172.21.15.83:0/3487722727 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c001670 con 0x7f05a80d45d0 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.234+0000 7f05a5ffb700 1 -- 172.21.15.83:0/3487722727 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0598004030 con 0x7f05a80d45d0 2023-11-26T07:11:40.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.234+0000 7f05a5ffb700 1 -- 172.21.15.83:0/3487722727 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980043c0 con 0x7f05a80d45d0 2023-11-26T07:11:40.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.234+0000 7f05ad2d2700 1 -- 172.21.15.83:0/3487722727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a80d45d0 msgr2=0x7f05a80b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.234+0000 7f05ad2d2700 1 --2- 172.21.15.83:0/3487722727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a80d45d0 0x7f05a80b70d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0598009840 tx=0x7f0598009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:40.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.234+0000 7f05ad2d2700 1 -- 172.21.15.83:0/3487722727 shutdown_connections 2023-11-26T07:11:40.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.234+0000 7f05ad2d2700 1 --2- 172.21.15.83:0/3487722727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a80d45d0 0x7f05a80b70d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.234+0000 7f05ad2d2700 1 -- 172.21.15.83:0/3487722727 >> 172.21.15.83:0/3487722727 conn(0x7f05a806b410 msgr2=0x7f05a806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:40.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.234+0000 7f05ad2d2700 1 -- 172.21.15.83:0/3487722727 shutdown_connections 2023-11-26T07:11:40.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.234+0000 7f05ad2d2700 1 -- 172.21.15.83:0/3487722727 wait complete. 2023-11-26T07:11:40.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.236+0000 7f05ad2d2700 1 Processor -- start 2023-11-26T07:11:40.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.236+0000 7f05ad2d2700 1 -- start start 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.236+0000 7f05ad2d2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a80d39d0 0x7f05a80b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.236+0000 7f05ad2d2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a80d45d0 0x7f05a80cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.236+0000 7f05ad2d2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a815e140 0x7f05a8160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.236+0000 7f05ad2d2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80cd5f0 con 0x7f05a815e140 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.236+0000 7f05ad2d2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80cce10 con 0x7f05a80d45d0 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.236+0000 7f05ad2d2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80cb500 con 0x7f05a80d39d0 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a815e140 0x7f05a8160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a815e140 0x7f05a8160950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34700/0 (socket says 172.21.15.83:34700) 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05a77fe700 1 -- 172.21.15.83:0/1658050135 learned_addr learned my addr 172.21.15.83:0/1658050135 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05a67fc700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a80d45d0 0x7f05a80cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05a6ffd700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a80d39d0 0x7f05a80b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05a77fe700 1 -- 172.21.15.83:0/1658050135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a80d39d0 msgr2=0x7f05a80b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05a77fe700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a80d39d0 0x7f05a80b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05a77fe700 1 -- 172.21.15.83:0/1658050135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a80d45d0 msgr2=0x7f05a80cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05a77fe700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a80d45d0 0x7f05a80cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05a77fe700 1 -- 172.21.15.83:0/1658050135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980094a0 con 0x7f05a815e140 2023-11-26T07:11:40.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05a77fe700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a815e140 0x7f05a8160950 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f05a000cc50 tx=0x7f05a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:40.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f058f7fe700 1 -- 172.21.15.83:0/1658050135 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a000ad80 con 0x7f05a815e140 2023-11-26T07:11:40.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f058f7fe700 1 -- 172.21.15.83:0/1658050135 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a000ce50 con 0x7f05a815e140 2023-11-26T07:11:40.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05ad2d2700 1 -- 172.21.15.83:0/1658050135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a80cb7e0 con 0x7f05a815e140 2023-11-26T07:11:40.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f058f7fe700 1 -- 172.21.15.83:0/1658050135 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0003c40 con 0x7f05a815e140 2023-11-26T07:11:40.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.237+0000 7f05ad2d2700 1 -- 172.21.15.83:0/1658050135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a80cbda0 con 0x7f05a815e140 2023-11-26T07:11:40.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.238+0000 7f05ad2d2700 1 -- 172.21.15.83:0/1658050135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a8068fb0 con 0x7f05a815e140 2023-11-26T07:11:40.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.238+0000 7f058f7fe700 1 -- 172.21.15.83:0/1658050135 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f05a00050d0 con 0x7f05a815e140 2023-11-26T07:11:40.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.239+0000 7f058f7fe700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f059406aed0 0x7f059406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.239+0000 7f058f7fe700 1 -- 172.21.15.83:0/1658050135 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05a0089720 con 0x7f05a815e140 2023-11-26T07:11:40.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.239+0000 7f05a6ffd700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f059406aed0 0x7f059406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.239+0000 7f05a6ffd700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f059406aed0 0x7f059406d390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f059c0017a0 tx=0x7f059c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:40.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.243+0000 7f058f7fe700 1 -- 172.21.15.83:0/1658050135 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05a00547e0 con 0x7f05a815e140 2023-11-26T07:11:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.434+0000 7f05ad2d2700 1 -- 172.21.15.83:0/1658050135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05a80d1bb0 con 0x7f05a815e140 2023-11-26T07:11:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.435+0000 7f058f7fe700 1 -- 172.21.15.83:0/1658050135 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05a0057e00 con 0x7f05a815e140 2023-11-26T07:11:40.436 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:11:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 -- 172.21.15.83:0/1658050135 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f059406aed0 msgr2=0x7f059406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f059406aed0 0x7f059406d390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f059c0017a0 tx=0x7f059c009040 comp rx=0 tx=0).stop 2023-11-26T07:11:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 -- 172.21.15.83:0/1658050135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a815e140 msgr2=0x7f05a8160950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a815e140 0x7f05a8160950 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f05a000cc50 tx=0x7f05a000a340 comp rx=0 tx=0).stop 2023-11-26T07:11:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 -- 172.21.15.83:0/1658050135 shutdown_connections 2023-11-26T07:11:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a815e140 0x7f05a8160950 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f059406aed0 0x7f059406d390 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a80d39d0 0x7f05a80b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 --2- 172.21.15.83:0/1658050135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a80d45d0 0x7f05a80cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 -- 172.21.15.83:0/1658050135 >> 172.21.15.83:0/1658050135 conn(0x7f05a806b410 msgr2=0x7f05a80d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:40.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 -- 172.21.15.83:0/1658050135 shutdown_connections 2023-11-26T07:11:40.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.437+0000 7f05ad2d2700 1 -- 172.21.15.83:0/1658050135 wait complete. 2023-11-26T07:11:40.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:11:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.571+0000 7f26f8428700 1 Processor -- start 2023-11-26T07:11:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.571+0000 7f26f8428700 1 -- start start 2023-11-26T07:11:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.572+0000 7f26f8428700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26f015f9e0 0x7f26f00add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.572+0000 7f26f8428700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f26f0073f90 0x7f26f0074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.572+0000 7f26f8428700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f006b080 con 0x7f26f00cf610 2023-11-26T07:11:40.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.572+0000 7f26f8428700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f0074960 con 0x7f26f0073f90 2023-11-26T07:11:40.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.572+0000 7f26f8428700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f015c290 con 0x7f26f015f9e0 2023-11-26T07:11:40.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.572+0000 7f26f69c5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f26f0073f90 0x7f26f0074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.572+0000 7f26f69c5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f26f0073f90 0x7f26f0074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34710/0 (socket says 172.21.15.83:34710) 2023-11-26T07:11:40.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.572+0000 7f26f69c5700 1 -- 172.21.15.83:0/1221965815 learned_addr learned my addr 172.21.15.83:0/1221965815 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:40.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.573+0000 7f26f59c3700 1 --2- 172.21.15.83:0/1221965815 >> v2:172.21.15.203:3300/0 conn(0x7f26f015f9e0 0x7f26f00add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.573+0000 7f26f69c5700 1 -- 172.21.15.83:0/1221965815 >> v2:172.21.15.203:3300/0 conn(0x7f26f015f9e0 msgr2=0x7f26f00add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:40.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.573+0000 7f26f69c5700 1 --2- 172.21.15.83:0/1221965815 >> v2:172.21.15.203:3300/0 conn(0x7f26f015f9e0 0x7f26f00add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.573+0000 7f26f69c5700 1 -- 172.21.15.83:0/1221965815 >> v1:172.21.15.203:6789/0 conn(0x7f26f00cf610 legacy=0x7f26f00b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.573+0000 7f26f69c5700 1 -- 172.21.15.83:0/1221965815 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f00cf9f0 con 0x7f26f0073f90 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.573+0000 7f26f69c5700 1 --2- 172.21.15.83:0/1221965815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0073f90 0x7f26f0074420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009e50 tx=0x7f26e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668ac509a6f282ac server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.573+0000 7f26f51c2700 1 -- 172.21.15.83:0/1221965815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800e070 con 0x7f26f0073f90 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.573+0000 7f26f51c2700 1 -- 172.21.15.83:0/1221965815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e800b980 con 0x7f26f0073f90 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.573+0000 7f26f51c2700 1 -- 172.21.15.83:0/1221965815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800bd30 con 0x7f26f0073f90 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.573+0000 7f26f8428700 1 -- 172.21.15.83:0/1221965815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0073f90 msgr2=0x7f26f0074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.573+0000 7f26f8428700 1 --2- 172.21.15.83:0/1221965815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0073f90 0x7f26f0074420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009e50 tx=0x7f26e800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.574+0000 7f26f8428700 1 -- 172.21.15.83:0/1221965815 shutdown_connections 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.574+0000 7f26f8428700 1 --2- 172.21.15.83:0/1221965815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0073f90 0x7f26f0074420 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.574+0000 7f26f8428700 1 --2- 172.21.15.83:0/1221965815 >> v2:172.21.15.203:3300/0 conn(0x7f26f015f9e0 0x7f26f00add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.574+0000 7f26f8428700 1 -- 172.21.15.83:0/1221965815 >> 172.21.15.83:0/1221965815 conn(0x7f26f00d0bf0 msgr2=0x7f26f00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.574+0000 7f26f8428700 1 -- 172.21.15.83:0/1221965815 shutdown_connections 2023-11-26T07:11:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.574+0000 7f26f8428700 1 -- 172.21.15.83:0/1221965815 wait complete. 2023-11-26T07:11:40.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.575+0000 7f26f8428700 1 Processor -- start 2023-11-26T07:11:40.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.575+0000 7f26f8428700 1 -- start start 2023-11-26T07:11:40.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f8428700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26f0073f90 0x7f26f014b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f8428700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26f00cf610 0x7f26f014c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f8428700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f015f9e0 0x7f26f009e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f8428700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f009de70 con 0x7f26f015f9e0 2023-11-26T07:11:40.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f8428700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f014bcf0 con 0x7f26f0073f90 2023-11-26T07:11:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f8428700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f009d260 con 0x7f26f00cf610 2023-11-26T07:11:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f69c5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f015f9e0 0x7f26f009e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f69c5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f015f9e0 0x7f26f009e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34716/0 (socket says 172.21.15.83:34716) 2023-11-26T07:11:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f69c5700 1 -- 172.21.15.83:0/3861901765 learned_addr learned my addr 172.21.15.83:0/3861901765 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f59c3700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26f00cf610 0x7f26f014c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f69c5700 1 -- 172.21.15.83:0/3861901765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26f00cf610 msgr2=0x7f26f014c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.576+0000 7f26f61c4700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26f0073f90 0x7f26f014b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.577+0000 7f26f69c5700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26f00cf610 0x7f26f014c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.577+0000 7f26f69c5700 1 -- 172.21.15.83:0/3861901765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26f0073f90 msgr2=0x7f26f014b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.577+0000 7f26f69c5700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26f0073f90 0x7f26f014b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.577+0000 7f26f69c5700 1 -- 172.21.15.83:0/3861901765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e8009b30 con 0x7f26f015f9e0 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.577+0000 7f26f69c5700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f015f9e0 0x7f26f009e300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f26f0073030 tx=0x7f26e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.577+0000 7f26e6ffd700 1 -- 172.21.15.83:0/3861901765 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e80092e0 con 0x7f26f015f9e0 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.577+0000 7f26e6ffd700 1 -- 172.21.15.83:0/3861901765 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e8004960 con 0x7f26f015f9e0 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.577+0000 7f26f8428700 1 -- 172.21.15.83:0/3861901765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f009d540 con 0x7f26f015f9e0 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.577+0000 7f26e6ffd700 1 -- 172.21.15.83:0/3861901765 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e801b670 con 0x7f26f015f9e0 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.577+0000 7f26f8428700 1 -- 172.21.15.83:0/3861901765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f009db00 con 0x7f26f015f9e0 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.578+0000 7f26f8428700 1 -- 172.21.15.83:0/3861901765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f009dfe0 con 0x7f26f015f9e0 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.578+0000 7f26e6ffd700 1 -- 172.21.15.83:0/3861901765 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f26e801b7d0 con 0x7f26f015f9e0 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.578+0000 7f26e6ffd700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f26d406aed0 0x7f26d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.579+0000 7f26f61c4700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f26d406aed0 0x7f26d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.579+0000 7f26e6ffd700 1 -- 172.21.15.83:0/3861901765 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26e808ab10 con 0x7f26f015f9e0 2023-11-26T07:11:40.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.579+0000 7f26f61c4700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f26d406aed0 0x7f26d406d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f26e0007fd0 tx=0x7f26e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:40.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.583+0000 7f26e6ffd700 1 -- 172.21.15.83:0/3861901765 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26e8055bc0 con 0x7f26f015f9e0 2023-11-26T07:11:40.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.775+0000 7f26f8428700 1 -- 172.21.15.83:0/3861901765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26f0068fb0 con 0x7f26f015f9e0 2023-11-26T07:11:40.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.775+0000 7f26e6ffd700 1 -- 172.21.15.83:0/3861901765 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26e80591e0 con 0x7f26f015f9e0 2023-11-26T07:11:40.777 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:11:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 -- 172.21.15.83:0/3861901765 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f26d406aed0 msgr2=0x7f26d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f26d406aed0 0x7f26d406d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f26e0007fd0 tx=0x7f26e0009040 comp rx=0 tx=0).stop 2023-11-26T07:11:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 -- 172.21.15.83:0/3861901765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f015f9e0 msgr2=0x7f26f009e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f015f9e0 0x7f26f009e300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f26f0073030 tx=0x7f26e8004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 -- 172.21.15.83:0/3861901765 shutdown_connections 2023-11-26T07:11:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f015f9e0 0x7f26f009e300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26f0073f90 0x7f26f014b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f26d406aed0 0x7f26d406d390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 --2- 172.21.15.83:0/3861901765 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26f00cf610 0x7f26f014c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 -- 172.21.15.83:0/3861901765 >> 172.21.15.83:0/3861901765 conn(0x7f26f00d0bf0 msgr2=0x7f26f00b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:40.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 -- 172.21.15.83:0/3861901765 shutdown_connections 2023-11-26T07:11:40.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.777+0000 7f26f8428700 1 -- 172.21.15.83:0/3861901765 wait complete. 2023-11-26T07:11:40.791 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:11:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.912+0000 7f7c95943700 1 Processor -- start 2023-11-26T07:11:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.912+0000 7f7c95943700 1 -- start start 2023-11-26T07:11:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c95943700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7c9015e140 0x7f7c900d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c95943700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7c900d39d0 0x7f7c900d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c95943700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c900b80e0 con 0x7f7c900b98c0 2023-11-26T07:11:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c95943700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c9015e520 con 0x7f7c9015e140 2023-11-26T07:11:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c95943700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c900d2930 con 0x7f7c900d39d0 2023-11-26T07:11:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c8effd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7c9015e140 0x7f7c900d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c8f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7c900b98c0 0x7f7c900b59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54760/0 (socket says 172.21.15.83:54760) 2023-11-26T07:11:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c8f7fe700 1 -- 172.21.15.83:0/2991174237 learned_addr learned my addr 172.21.15.83:0/2991174237 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c8dffb700 1 -- 172.21.15.83:0/2991174237 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1357319183 0 0) 0x7f7c88001490 con 0x7f7c900b98c0 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c8e7fc700 1 --2- 172.21.15.83:0/2991174237 >> v2:172.21.15.203:3300/0 conn(0x7f7c900d39d0 0x7f7c900d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c8dffb700 1 -- 172.21.15.83:0/2991174237 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c780035b0 con 0x7f7c900b98c0 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c8effd700 1 -- 172.21.15.83:0/2991174237 >> v2:172.21.15.203:3300/0 conn(0x7f7c900d39d0 msgr2=0x7f7c900d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c8effd700 1 --2- 172.21.15.83:0/2991174237 >> v2:172.21.15.203:3300/0 conn(0x7f7c900d39d0 0x7f7c900d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c8effd700 1 -- 172.21.15.83:0/2991174237 >> v1:172.21.15.83:6789/0 conn(0x7f7c900b98c0 legacy=0x7f7c900b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c8effd700 1 -- 172.21.15.83:0/2991174237 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c900d2c10 con 0x7f7c9015e140 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.913+0000 7f7c8effd700 1 --2- 172.21.15.83:0/2991174237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c9015e140 0x7f7c900d3ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7c900d4b20 tx=0x7f7c84009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40c3d364ee061b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.914+0000 7f7c8dffb700 1 -- 172.21.15.83:0/2991174237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84011830 con 0x7f7c9015e140 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.914+0000 7f7c8dffb700 1 -- 172.21.15.83:0/2991174237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c84011990 con 0x7f7c9015e140 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.914+0000 7f7c8dffb700 1 -- 172.21.15.83:0/2991174237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84011d20 con 0x7f7c9015e140 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.914+0000 7f7c95943700 1 -- 172.21.15.83:0/2991174237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c9015e140 msgr2=0x7f7c900d3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.914+0000 7f7c95943700 1 --2- 172.21.15.83:0/2991174237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c9015e140 0x7f7c900d3ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7c900d4b20 tx=0x7f7c84009b20 comp rx=0 tx=0).stop 2023-11-26T07:11:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.914+0000 7f7c95943700 1 -- 172.21.15.83:0/2991174237 shutdown_connections 2023-11-26T07:11:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.914+0000 7f7c95943700 1 --2- 172.21.15.83:0/2991174237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c9015e140 0x7f7c900d3ec0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.914+0000 7f7c95943700 1 --2- 172.21.15.83:0/2991174237 >> v2:172.21.15.203:3300/0 conn(0x7f7c900d39d0 0x7f7c900d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.914+0000 7f7c95943700 1 -- 172.21.15.83:0/2991174237 >> 172.21.15.83:0/2991174237 conn(0x7f7c9006b410 msgr2=0x7f7c9006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.914+0000 7f7c95943700 1 -- 172.21.15.83:0/2991174237 shutdown_connections 2023-11-26T07:11:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.914+0000 7f7c95943700 1 -- 172.21.15.83:0/2991174237 wait complete. 2023-11-26T07:11:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.915+0000 7f7c95943700 1 Processor -- start 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.915+0000 7f7c95943700 1 -- start start 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.916+0000 7f7c95943700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c900b98c0 0x7f7c900b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.916+0000 7f7c95943700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c900d39d0 0x7f7c90153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.916+0000 7f7c95943700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c9015e140 0x7f7c9015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.916+0000 7f7c95943700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900b6650 con 0x7f7c900b98c0 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.916+0000 7f7c95943700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900ca500 con 0x7f7c9015e140 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.916+0000 7f7c95943700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c901f3a40 con 0x7f7c900d39d0 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.916+0000 7f7c8effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c900b98c0 0x7f7c900b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.916+0000 7f7c8effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c900b98c0 0x7f7c900b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34724/0 (socket says 172.21.15.83:34724) 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.916+0000 7f7c8effd700 1 -- 172.21.15.83:0/1767490103 learned_addr learned my addr 172.21.15.83:0/1767490103 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.917+0000 7f7c8effd700 1 -- 172.21.15.83:0/1767490103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c900d39d0 msgr2=0x7f7c90153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.917+0000 7f7c8effd700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c900d39d0 0x7f7c90153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.917+0000 7f7c8effd700 1 -- 172.21.15.83:0/1767490103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c9015e140 msgr2=0x7f7c9015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.917+0000 7f7c8effd700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c9015e140 0x7f7c9015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.917+0000 7f7c8effd700 1 -- 172.21.15.83:0/1767490103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c840094a0 con 0x7f7c900b98c0 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.917+0000 7f7c8effd700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c900b98c0 0x7f7c900b3970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7c84000c00 tx=0x7f7c84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.917+0000 7f7c77fff700 1 -- 172.21.15.83:0/1767490103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c840036a0 con 0x7f7c900b98c0 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.917+0000 7f7c77fff700 1 -- 172.21.15.83:0/1767490103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8401e070 con 0x7f7c900b98c0 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.917+0000 7f7c95943700 1 -- 172.21.15.83:0/1767490103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c901f3d20 con 0x7f7c900b98c0 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.917+0000 7f7c77fff700 1 -- 172.21.15.83:0/1767490103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84003a20 con 0x7f7c900b98c0 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.917+0000 7f7c95943700 1 -- 172.21.15.83:0/1767490103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c901f42e0 con 0x7f7c900b98c0 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.918+0000 7f7c95943700 1 -- 172.21.15.83:0/1767490103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c90068fb0 con 0x7f7c900b98c0 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.918+0000 7f7c77fff700 1 -- 172.21.15.83:0/1767490103 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c84003b80 con 0x7f7c900b98c0 2023-11-26T07:11:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.919+0000 7f7c77fff700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c7c06aed0 0x7f7c7c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.919+0000 7f7c77fff700 1 -- 172.21.15.83:0/1767490103 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c8408a540 con 0x7f7c900b98c0 2023-11-26T07:11:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.919+0000 7f7c8e7fc700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c7c06aed0 0x7f7c7c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.919+0000 7f7c8e7fc700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c7c06aed0 0x7f7c7c06d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c900cdd60 tx=0x7f7c80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:40.923+0000 7f7c77fff700 1 -- 172.21.15.83:0/1767490103 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c84055680 con 0x7f7c900b98c0 2023-11-26T07:11:41.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.111+0000 7f7c95943700 1 -- 172.21.15.83:0/1767490103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7c900d1bb0 con 0x7f7c900b98c0 2023-11-26T07:11:41.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.111+0000 7f7c77fff700 1 -- 172.21.15.83:0/1767490103 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7c84058ca0 con 0x7f7c900b98c0 2023-11-26T07:11:41.113 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:11:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 -- 172.21.15.83:0/1767490103 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c7c06aed0 msgr2=0x7f7c7c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c7c06aed0 0x7f7c7c06d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7c900cdd60 tx=0x7f7c80008040 comp rx=0 tx=0).stop 2023-11-26T07:11:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 -- 172.21.15.83:0/1767490103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c900b98c0 msgr2=0x7f7c900b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c900b98c0 0x7f7c900b3970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7c84000c00 tx=0x7f7c84004750 comp rx=0 tx=0).stop 2023-11-26T07:11:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 -- 172.21.15.83:0/1767490103 shutdown_connections 2023-11-26T07:11:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7c9015e140 0x7f7c9015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c900b98c0 0x7f7c900b3970 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c7c06aed0 0x7f7c7c06d390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 --2- 172.21.15.83:0/1767490103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7c900d39d0 0x7f7c90153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 -- 172.21.15.83:0/1767490103 >> 172.21.15.83:0/1767490103 conn(0x7f7c9006b410 msgr2=0x7f7c9015e760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:41.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 -- 172.21.15.83:0/1767490103 shutdown_connections 2023-11-26T07:11:41.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.113+0000 7f7c95943700 1 -- 172.21.15.83:0/1767490103 wait complete. 2023-11-26T07:11:41.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:11:41.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.245+0000 7fcc36a82700 1 Processor -- start 2023-11-26T07:11:41.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.245+0000 7fcc36a82700 1 -- start start 2023-11-26T07:11:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc36a82700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc3015af10 0x7fcc3015e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc36a82700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcc300ce140 0x7fcc3015ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc36a82700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc301414c0 con 0x7fcc3015f460 2023-11-26T07:11:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc36a82700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3015f8c0 con 0x7fcc300ce140 2023-11-26T07:11:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc36a82700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc300ce9f0 con 0x7fcc3015af10 2023-11-26T07:11:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc2ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcc300ce140 0x7fcc3015ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc2ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcc300ce140 0x7fcc3015ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34734/0 (socket says 172.21.15.83:34734) 2023-11-26T07:11:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc2ffff700 1 -- 172.21.15.83:0/1485071508 learned_addr learned my addr 172.21.15.83:0/1485071508 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc3481e700 1 --2- 172.21.15.83:0/1485071508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc3015af10 0x7fcc3015e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc2ffff700 1 -- 172.21.15.83:0/1485071508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc3015af10 msgr2=0x7fcc3015e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc2ffff700 1 --2- 172.21.15.83:0/1485071508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc3015af10 0x7fcc3015e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc2ffff700 1 -- 172.21.15.83:0/1485071508 >> v1:172.21.15.203:6789/0 conn(0x7fcc3015f460 legacy=0x7fcc3015b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.246+0000 7fcc2ffff700 1 -- 172.21.15.83:0/1485071508 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc300cecd0 con 0x7fcc300ce140 2023-11-26T07:11:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc2ffff700 1 --2- 172.21.15.83:0/1485071508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc300ce140 0x7fcc3015ef20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098f0 tx=0x7fcc1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c7294d57b6fb01 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc2f7fe700 1 -- 172.21.15.83:0/1485071508 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c011830 con 0x7fcc300ce140 2023-11-26T07:11:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc2f7fe700 1 -- 172.21.15.83:0/1485071508 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c011990 con 0x7fcc300ce140 2023-11-26T07:11:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc2f7fe700 1 -- 172.21.15.83:0/1485071508 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c011d20 con 0x7fcc300ce140 2023-11-26T07:11:41.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc36a82700 1 -- 172.21.15.83:0/1485071508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc300ce140 msgr2=0x7fcc3015ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc36a82700 1 --2- 172.21.15.83:0/1485071508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc300ce140 0x7fcc3015ef20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0098f0 tx=0x7fcc1c009c00 comp rx=0 tx=0).stop 2023-11-26T07:11:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc36a82700 1 -- 172.21.15.83:0/1485071508 shutdown_connections 2023-11-26T07:11:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc36a82700 1 --2- 172.21.15.83:0/1485071508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc3015af10 0x7fcc3015e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc36a82700 1 --2- 172.21.15.83:0/1485071508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc300ce140 0x7fcc3015ef20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc36a82700 1 -- 172.21.15.83:0/1485071508 >> 172.21.15.83:0/1485071508 conn(0x7fcc300cd570 msgr2=0x7fcc300cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc36a82700 1 -- 172.21.15.83:0/1485071508 shutdown_connections 2023-11-26T07:11:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.247+0000 7fcc36a82700 1 -- 172.21.15.83:0/1485071508 wait complete. 2023-11-26T07:11:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.248+0000 7fcc36a82700 1 Processor -- start 2023-11-26T07:11:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.249+0000 7fcc36a82700 1 -- start start 2023-11-26T07:11:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.249+0000 7fcc36a82700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc300ce140 0x7fcc3013d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.249+0000 7fcc36a82700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc3015af10 0x7fcc300b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.249+0000 7fcc36a82700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc3015f460 0x7fcc300b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.249+0000 7fcc36a82700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc301595f0 con 0x7fcc3015af10 2023-11-26T07:11:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.249+0000 7fcc36a82700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc300b4b80 con 0x7fcc3015f460 2023-11-26T07:11:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.249+0000 7fcc36a82700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc300b5460 con 0x7fcc300ce140 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc3015af10 0x7fcc300b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc3015af10 0x7fcc300b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34740/0 (socket says 172.21.15.83:34740) 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2ffff700 1 -- 172.21.15.83:0/3772846221 learned_addr learned my addr 172.21.15.83:0/3772846221 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc3481e700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc300ce140 0x7fcc3013d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc3501f700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc3015f460 0x7fcc300b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2ffff700 1 -- 172.21.15.83:0/3772846221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc300ce140 msgr2=0x7fcc3013d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2ffff700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc300ce140 0x7fcc3013d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2ffff700 1 -- 172.21.15.83:0/3772846221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc3015f460 msgr2=0x7fcc300b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2ffff700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc3015f460 0x7fcc300b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2ffff700 1 -- 172.21.15.83:0/3772846221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c009550 con 0x7fcc3015af10 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc3481e700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc300ce140 0x7fcc3013d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:41.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc3501f700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc3015f460 0x7fcc300b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2ffff700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc3015af10 0x7fcc300b44f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009c40 tx=0x7fcc1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2d7fa700 1 -- 172.21.15.83:0/3772846221 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c004530 con 0x7fcc3015af10 2023-11-26T07:11:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2d7fa700 1 -- 172.21.15.83:0/3772846221 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c01f070 con 0x7fcc3015af10 2023-11-26T07:11:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc36a82700 1 -- 172.21.15.83:0/3772846221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc30158300 con 0x7fcc3015af10 2023-11-26T07:11:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc2d7fa700 1 -- 172.21.15.83:0/3772846221 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0038c0 con 0x7fcc3015af10 2023-11-26T07:11:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.250+0000 7fcc36a82700 1 -- 172.21.15.83:0/3772846221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc301579d0 con 0x7fcc3015af10 2023-11-26T07:11:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.251+0000 7fcc36a82700 1 -- 172.21.15.83:0/3772846221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc30158490 con 0x7fcc3015af10 2023-11-26T07:11:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.251+0000 7fcc2d7fa700 1 -- 172.21.15.83:0/3772846221 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc1c021030 con 0x7fcc3015af10 2023-11-26T07:11:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.252+0000 7fcc2d7fa700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc1406aed0 0x7fcc1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.252+0000 7fcc2d7fa700 1 -- 172.21.15.83:0/3772846221 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc1c08a9f0 con 0x7fcc3015af10 2023-11-26T07:11:41.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.252+0000 7fcc3481e700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc1406aed0 0x7fcc1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.252+0000 7fcc3481e700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc1406aed0 0x7fcc1406d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009c30 tx=0x7fcc24008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:41.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.256+0000 7fcc2d7fa700 1 -- 172.21.15.83:0/3772846221 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc1c055b30 con 0x7fcc3015af10 2023-11-26T07:11:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.489+0000 7fcc36a82700 1 -- 172.21.15.83:0/3772846221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc30067f60 con 0x7fcc3015af10 2023-11-26T07:11:41.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.490+0000 7fcc2d7fa700 1 -- 172.21.15.83:0/3772846221 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fcc1c015020 con 0x7fcc3015af10 2023-11-26T07:11:41.491 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:41.491 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2208741688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2208741688},{"type":"v1","addr":"172.21.15.83:6837","nonce":2208741688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2802126359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2802126359},{"type":"v1","addr":"172.21.15.83:6836","nonce":2802126359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1258871616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1258871616},{"type":"v1","addr":"172.21.15.203:6834","nonce":1258871616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:11:27.911475+0000","modified":"2023-11-26T07:11:30.937105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/2957324266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":2957324266},{"type":"v1","addr":"172.21.15.203:6835","nonce":2957324266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:11:41.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.491+0000 7fcc36a82700 1 -- 172.21.15.83:0/3772846221 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc1406aed0 msgr2=0x7fcc1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.491+0000 7fcc36a82700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc1406aed0 0x7fcc1406d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009c30 tx=0x7fcc24008040 comp rx=0 tx=0).stop 2023-11-26T07:11:41.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.491+0000 7fcc36a82700 1 -- 172.21.15.83:0/3772846221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc3015af10 msgr2=0x7fcc300b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.491+0000 7fcc36a82700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc3015af10 0x7fcc300b44f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009c40 tx=0x7fcc1c004750 comp rx=0 tx=0).stop 2023-11-26T07:11:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.492+0000 7fcc36a82700 1 -- 172.21.15.83:0/3772846221 shutdown_connections 2023-11-26T07:11:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.492+0000 7fcc36a82700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc3015f460 0x7fcc300b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.492+0000 7fcc36a82700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc1406aed0 0x7fcc1406d390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.492+0000 7fcc36a82700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc300ce140 0x7fcc3013d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.492+0000 7fcc36a82700 1 --2- 172.21.15.83:0/3772846221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc3015af10 0x7fcc300b44f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.492+0000 7fcc36a82700 1 -- 172.21.15.83:0/3772846221 >> 172.21.15.83:0/3772846221 conn(0x7fcc300cd570 msgr2=0x7fcc3015b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.492+0000 7fcc36a82700 1 -- 172.21.15.83:0/3772846221 shutdown_connections 2023-11-26T07:11:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.492+0000 7fcc36a82700 1 -- 172.21.15.83:0/3772846221 wait complete. 2023-11-26T07:11:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2023-11-26T07:11:41.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.623+0000 7fc283161700 1 Processor -- start 2023-11-26T07:11:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.623+0000 7fc283161700 1 -- start start 2023-11-26T07:11:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.624+0000 7fc283161700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc27c15bdd0 0x7fc27c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.624+0000 7fc283161700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 0x7fc27c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.624+0000 7fc283161700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27c0b22d0 con 0x7fc27c0b3370 2023-11-26T07:11:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.624+0000 7fc283161700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27c0b2f50 con 0x7fc27c15bdd0 2023-11-26T07:11:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.624+0000 7fc283161700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0d2640 con 0x7fc27c0d1c70 2023-11-26T07:11:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.624+0000 7fc280efd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc27c15bdd0 0x7fc27c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.624+0000 7fc27bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc27c0b3370 0x7fc27c0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54786/0 (socket says 172.21.15.83:54786) 2023-11-26T07:11:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.624+0000 7fc27bfff700 1 -- 172.21.15.83:0/178595387 learned_addr learned my addr 172.21.15.83:0/178595387 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc2816fe700 1 --2- 172.21.15.83:0/178595387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 0x7fc27c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc27b7fe700 1 -- 172.21.15.83:0/178595387 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 560800627 0 0) 0x7fc27c0b22d0 con 0x7fc27c0b3370 2023-11-26T07:11:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc27b7fe700 1 -- 172.21.15.83:0/178595387 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc26c0035b0 con 0x7fc27c0b3370 2023-11-26T07:11:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc280efd700 1 -- 172.21.15.83:0/178595387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 msgr2=0x7fc27c0d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc280efd700 1 --2- 172.21.15.83:0/178595387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 0x7fc27c0d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc280efd700 1 -- 172.21.15.83:0/178595387 >> v1:172.21.15.83:6789/0 conn(0x7fc27c0b3370 legacy=0x7fc27c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc280efd700 1 -- 172.21.15.83:0/178595387 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c0b25b0 con 0x7fc27c15bdd0 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc2816fe700 1 --2- 172.21.15.83:0/178595387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 0x7fc27c0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc280efd700 1 --2- 172.21.15.83:0/178595387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c15bdd0 0x7fc27c1604e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc270009840 tx=0x7fc270009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce332a734986c3e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc27b7fe700 1 -- 172.21.15.83:0/178595387 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270011830 con 0x7fc27c15bdd0 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc27b7fe700 1 -- 172.21.15.83:0/178595387 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc270011990 con 0x7fc27c15bdd0 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc27b7fe700 1 -- 172.21.15.83:0/178595387 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270011d20 con 0x7fc27c15bdd0 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc283161700 1 -- 172.21.15.83:0/178595387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c15bdd0 msgr2=0x7fc27c1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc283161700 1 --2- 172.21.15.83:0/178595387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c15bdd0 0x7fc27c1604e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc270009840 tx=0x7fc270009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc283161700 1 -- 172.21.15.83:0/178595387 shutdown_connections 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc283161700 1 --2- 172.21.15.83:0/178595387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 0x7fc27c0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc283161700 1 --2- 172.21.15.83:0/178595387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c15bdd0 0x7fc27c1604e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc283161700 1 -- 172.21.15.83:0/178595387 >> 172.21.15.83:0/178595387 conn(0x7fc27c0731a0 msgr2=0x7fc27c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.625+0000 7fc283161700 1 -- 172.21.15.83:0/178595387 shutdown_connections 2023-11-26T07:11:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.626+0000 7fc283161700 1 -- 172.21.15.83:0/178595387 wait complete. 2023-11-26T07:11:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.627+0000 7fc283161700 1 Processor -- start 2023-11-26T07:11:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.627+0000 7fc283161700 1 -- start start 2023-11-26T07:11:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.627+0000 7fc283161700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c0b3370 0x7fc27c0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.627+0000 7fc283161700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 0x7fc27c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc283161700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc27c15bdd0 0x7fc27c0b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc283161700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0b6d40 con 0x7fc27c0b3370 2023-11-26T07:11:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc283161700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0b57f0 con 0x7fc27c15bdd0 2023-11-26T07:11:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc283161700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c0b60c0 con 0x7fc27c0d1c70 2023-11-26T07:11:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc280efd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c0b3370 0x7fc27c0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc280efd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c0b3370 0x7fc27c0b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34750/0 (socket says 172.21.15.83:34750) 2023-11-26T07:11:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc280efd700 1 -- 172.21.15.83:0/2440043405 learned_addr learned my addr 172.21.15.83:0/2440043405 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc27bfff700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 0x7fc27c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc2816fe700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc27c15bdd0 0x7fc27c0b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc280efd700 1 -- 172.21.15.83:0/2440043405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 msgr2=0x7fc27c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc280efd700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 0x7fc27c0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc280efd700 1 -- 172.21.15.83:0/2440043405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc27c15bdd0 msgr2=0x7fc27c0b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc280efd700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc27c15bdd0 0x7fc27c0b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc280efd700 1 -- 172.21.15.83:0/2440043405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2700094a0 con 0x7fc27c0b3370 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc27bfff700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 0x7fc27c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc2816fe700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc27c15bdd0 0x7fc27c0b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc280efd700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c0b3370 0x7fc27c0b5ae0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc270009fd0 tx=0x7fc270004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc2797fa700 1 -- 172.21.15.83:0/2440043405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2700036a0 con 0x7fc27c0b3370 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc2797fa700 1 -- 172.21.15.83:0/2440043405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27001e070 con 0x7fc27c0b3370 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc283161700 1 -- 172.21.15.83:0/2440043405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27c0b63a0 con 0x7fc27c0b3370 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc2797fa700 1 -- 172.21.15.83:0/2440043405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270003a60 con 0x7fc27c0b3370 2023-11-26T07:11:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.628+0000 7fc283161700 1 -- 172.21.15.83:0/2440043405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27c0b6960 con 0x7fc27c0b3370 2023-11-26T07:11:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.629+0000 7fc283161700 1 -- 172.21.15.83:0/2440043405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27c067f60 con 0x7fc27c0b3370 2023-11-26T07:11:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.630+0000 7fc2797fa700 1 -- 172.21.15.83:0/2440043405 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc270003bc0 con 0x7fc27c0b3370 2023-11-26T07:11:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.630+0000 7fc2797fa700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 0x7fc26006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.630+0000 7fc2797fa700 1 -- 172.21.15.83:0/2440043405 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc27008a550 con 0x7fc27c0b3370 2023-11-26T07:11:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.630+0000 7fc27bfff700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 0x7fc26006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.630+0000 7fc27bfff700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 0x7fc26006d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0c5370 tx=0x7fc268009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.634+0000 7fc2797fa700 1 -- 172.21.15.83:0/2440043405 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc270055690 con 0x7fc27c0b3370 2023-11-26T07:11:41.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.824+0000 7fc283161700 1 -- 172.21.15.83:0/2440043405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc27c0b7c60 con 0x7fc27c0b3370 2023-11-26T07:11:41.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.825+0000 7fc2797fa700 1 -- 172.21.15.83:0/2440043405 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fc270058cb0 con 0x7fc27c0b3370 2023-11-26T07:11:41.827 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:41.827 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 -- 172.21.15.83:0/2440043405 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 msgr2=0x7fc26006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 0x7fc26006d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0c5370 tx=0x7fc268009040 comp rx=0 tx=0).stop 2023-11-26T07:11:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 -- 172.21.15.83:0/2440043405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c0b3370 msgr2=0x7fc27c0b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c0b3370 0x7fc27c0b5ae0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc270009fd0 tx=0x7fc270004750 comp rx=0 tx=0).stop 2023-11-26T07:11:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 -- 172.21.15.83:0/2440043405 shutdown_connections 2023-11-26T07:11:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc27c15bdd0 0x7fc27c0b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c0b3370 0x7fc27c0b5ae0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 0x7fc26006d390 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 --2- 172.21.15.83:0/2440043405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27c0d1c70 0x7fc27c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 -- 172.21.15.83:0/2440043405 >> 172.21.15.83:0/2440043405 conn(0x7fc27c0731a0 msgr2=0x7fc27c15f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 -- 172.21.15.83:0/2440043405 shutdown_connections 2023-11-26T07:11:41.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.827+0000 7fc283161700 1 -- 172.21.15.83:0/2440043405 wait complete. 2023-11-26T07:11:41.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:11:41.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.960+0000 7fe77b68f700 1 Processor -- start 2023-11-26T07:11:41.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.960+0000 7fe77b68f700 1 -- start start 2023-11-26T07:11:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe77b68f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7740adf00 0x7fe77406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe77b68f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7740af950 con 0x7fe77406c540 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe77b68f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7740afc30 con 0x7fe7741446a0 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe77b68f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77406c9f0 con 0x7fe7740adf00 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe778c2a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe7740adf00 0x7fe77406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe779c2c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe77406c540 0x7fe7740b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54802/0 (socket says 172.21.15.83:54802) 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe779c2c700 1 -- 172.21.15.83:0/4271671384 learned_addr learned my addr 172.21.15.83:0/4271671384 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe778c2a700 1 -- 172.21.15.83:0/4271671384 >> v1:172.21.15.203:6789/0 conn(0x7fe7741446a0 legacy=0x7fe7740b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe778c2a700 1 -- 172.21.15.83:0/4271671384 >> v1:172.21.15.83:6789/0 conn(0x7fe77406c540 legacy=0x7fe7740b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe76bfff700 1 -- 172.21.15.83:0/4271671384 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246922821 0 0) 0x7fe7740af950 con 0x7fe77406c540 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe778c2a700 1 -- 172.21.15.83:0/4271671384 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740aff10 con 0x7fe7740adf00 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe778c2a700 1 --2- 172.21.15.83:0/4271671384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7740adf00 0x7fe77406c000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe764009840 tx=0x7fe764009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbec0230b793818 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe76bfff700 1 -- 172.21.15.83:0/4271671384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764004230 con 0x7fe7740adf00 2023-11-26T07:11:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe76bfff700 1 -- 172.21.15.83:0/4271671384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe764004390 con 0x7fe7740adf00 2023-11-26T07:11:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.961+0000 7fe76bfff700 1 -- 172.21.15.83:0/4271671384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7640036a0 con 0x7fe7740adf00 2023-11-26T07:11:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.962+0000 7fe77b68f700 1 -- 172.21.15.83:0/4271671384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7740adf00 msgr2=0x7fe77406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.962+0000 7fe77b68f700 1 --2- 172.21.15.83:0/4271671384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7740adf00 0x7fe77406c000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe764009840 tx=0x7fe764009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.962+0000 7fe77b68f700 1 -- 172.21.15.83:0/4271671384 shutdown_connections 2023-11-26T07:11:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.962+0000 7fe77b68f700 1 --2- 172.21.15.83:0/4271671384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7740adf00 0x7fe77406c000 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.962+0000 7fe77b68f700 1 -- 172.21.15.83:0/4271671384 >> 172.21.15.83:0/4271671384 conn(0x7fe774071b90 msgr2=0x7fe774071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.962+0000 7fe77b68f700 1 -- 172.21.15.83:0/4271671384 shutdown_connections 2023-11-26T07:11:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.962+0000 7fe77b68f700 1 -- 172.21.15.83:0/4271671384 wait complete. 2023-11-26T07:11:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.963+0000 7fe77b68f700 1 Processor -- start 2023-11-26T07:11:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.963+0000 7fe77b68f700 1 -- start start 2023-11-26T07:11:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe77b68f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77406c540 0x7fe774154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe77b68f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740adf00 0x7fe7741596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe77b68f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7741446a0 0x7fe774140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe77b68f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe774154b40 con 0x7fe7741446a0 2023-11-26T07:11:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe77b68f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740cd970 con 0x7fe77406c540 2023-11-26T07:11:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe77b68f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740cdc50 con 0x7fe7740adf00 2023-11-26T07:11:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe779c2c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7741446a0 0x7fe774140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe779c2c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7741446a0 0x7fe774140c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34768/0 (socket says 172.21.15.83:34768) 2023-11-26T07:11:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe779c2c700 1 -- 172.21.15.83:0/277854866 learned_addr learned my addr 172.21.15.83:0/277854866 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe778c2a700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740adf00 0x7fe7741596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe77942b700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77406c540 0x7fe774154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe779c2c700 1 -- 172.21.15.83:0/277854866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740adf00 msgr2=0x7fe7741596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe779c2c700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740adf00 0x7fe7741596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe779c2c700 1 -- 172.21.15.83:0/277854866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77406c540 msgr2=0x7fe774154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe779c2c700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77406c540 0x7fe774154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.964+0000 7fe779c2c700 1 -- 172.21.15.83:0/277854866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7640094a0 con 0x7fe7741446a0 2023-11-26T07:11:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.965+0000 7fe779c2c700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7741446a0 0x7fe774140c30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00cca0 tx=0x7fe76c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.965+0000 7fe769ffb700 1 -- 172.21.15.83:0/277854866 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00cea0 con 0x7fe7741446a0 2023-11-26T07:11:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.965+0000 7fe769ffb700 1 -- 172.21.15.83:0/277854866 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76c00ade0 con 0x7fe7741446a0 2023-11-26T07:11:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.965+0000 7fe769ffb700 1 -- 172.21.15.83:0/277854866 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c007310 con 0x7fe7741446a0 2023-11-26T07:11:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.965+0000 7fe77b68f700 1 -- 172.21.15.83:0/277854866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7740cdf30 con 0x7fe7741446a0 2023-11-26T07:11:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.965+0000 7fe77b68f700 1 -- 172.21.15.83:0/277854866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7740cd600 con 0x7fe7741446a0 2023-11-26T07:11:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.966+0000 7fe77b68f700 1 -- 172.21.15.83:0/277854866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7741f4f60 con 0x7fe7741446a0 2023-11-26T07:11:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.966+0000 7fe769ffb700 1 -- 172.21.15.83:0/277854866 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe76c007470 con 0x7fe7741446a0 2023-11-26T07:11:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.966+0000 7fe769ffb700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe75806aed0 0x7fe75806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.967+0000 7fe77942b700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe75806aed0 0x7fe75806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.967+0000 7fe769ffb700 1 -- 172.21.15.83:0/277854866 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe76c089220 con 0x7fe7741446a0 2023-11-26T07:11:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.967+0000 7fe77942b700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe75806aed0 0x7fe75806d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe770009870 tx=0x7fe770008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:41.971+0000 7fe769ffb700 1 -- 172.21.15.83:0/277854866 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe76c054360 con 0x7fe7741446a0 2023-11-26T07:11:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.201+0000 7fe77b68f700 1 -- 172.21.15.83:0/277854866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7740753f0 con 0x7fe7741446a0 2023-11-26T07:11:42.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.201+0000 7fe769ffb700 1 -- 172.21.15.83:0/277854866 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fe76c057980 con 0x7fe7741446a0 2023-11-26T07:11:42.204 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:42.204 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2208741688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2208741688},{"type":"v1","addr":"172.21.15.83:6837","nonce":2208741688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2802126359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2802126359},{"type":"v1","addr":"172.21.15.83:6836","nonce":2802126359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1258871616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1258871616},{"type":"v1","addr":"172.21.15.203:6834","nonce":1258871616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:11:27.911475+0000","modified":"2023-11-26T07:11:30.937105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/2957324266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":2957324266},{"type":"v1","addr":"172.21.15.203:6835","nonce":2957324266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:11:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.203+0000 7fe77b68f700 1 -- 172.21.15.83:0/277854866 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe75806aed0 msgr2=0x7fe75806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.203+0000 7fe77b68f700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe75806aed0 0x7fe75806d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe770009870 tx=0x7fe770008040 comp rx=0 tx=0).stop 2023-11-26T07:11:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.203+0000 7fe77b68f700 1 -- 172.21.15.83:0/277854866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7741446a0 msgr2=0x7fe774140c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.203+0000 7fe77b68f700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7741446a0 0x7fe774140c30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00cca0 tx=0x7fe76c00a340 comp rx=0 tx=0).stop 2023-11-26T07:11:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.203+0000 7fe77b68f700 1 -- 172.21.15.83:0/277854866 shutdown_connections 2023-11-26T07:11:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.203+0000 7fe77b68f700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7741446a0 0x7fe774140c30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.203+0000 7fe77b68f700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77406c540 0x7fe774154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.203+0000 7fe77b68f700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe75806aed0 0x7fe75806d390 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.203+0000 7fe77b68f700 1 --2- 172.21.15.83:0/277854866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7740adf00 0x7fe7741596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.203+0000 7fe77b68f700 1 -- 172.21.15.83:0/277854866 >> 172.21.15.83:0/277854866 conn(0x7fe774071b90 msgr2=0x7fe7740b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.204+0000 7fe77b68f700 1 -- 172.21.15.83:0/277854866 shutdown_connections 2023-11-26T07:11:42.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.204+0000 7fe77b68f700 1 -- 172.21.15.83:0/277854866 wait complete. 2023-11-26T07:11:42.209 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2023-11-26T07:11:42.620 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:11:27.911475+0000', 'modified': '2023-11-26T07:11:30.937105+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6835/2957324266', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6833', 'nonce': 2957324266}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 2957324266}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:11:42.620 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:11:42.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-26T07:11:42.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.740+0000 7fd4c5163700 1 Processor -- start 2023-11-26T07:11:42.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.740+0000 7fd4c5163700 1 -- start start 2023-11-26T07:11:42.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4c5163700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4c015e080 0x7fd4c00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4c5163700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd4c006c710 0x7fd4c00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4c5163700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c00ae430 con 0x7fd4c00af4d0 2023-11-26T07:11:42.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4c5163700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c00af0b0 con 0x7fd4c006c710 2023-11-26T07:11:42.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4c5163700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00b05a0 con 0x7fd4c015e080 2023-11-26T07:11:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4bf59e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd4c006c710 0x7fd4c00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4be59c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd4c00af4d0 0x7fd4c006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54814/0 (socket says 172.21.15.83:54814) 2023-11-26T07:11:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4be59c700 1 -- 172.21.15.83:0/3227433484 learned_addr learned my addr 172.21.15.83:0/3227433484 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/3227433484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4c015e080 0x7fd4c00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4bf59e700 1 -- 172.21.15.83:0/3227433484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4c015e080 msgr2=0x7fd4c00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/3227433484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4c015e080 0x7fd4c00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4bf59e700 1 -- 172.21.15.83:0/3227433484 >> v1:172.21.15.83:6789/0 conn(0x7fd4c00af4d0 legacy=0x7fd4c006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4bf59e700 1 -- 172.21.15.83:0/3227433484 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c00ae710 con 0x7fd4c006c710 2023-11-26T07:11:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4bdd9b700 1 -- 172.21.15.83:0/3227433484 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554456194 0 0) 0x7fd4c00ae430 con 0x7fd4c00af4d0 2023-11-26T07:11:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/3227433484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c006c710 0x7fd4c00afb90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009dc0 tx=0x7fd4b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64926473fbba14f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4bdd9b700 1 -- 172.21.15.83:0/3227433484 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b800e070 con 0x7fd4c006c710 2023-11-26T07:11:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4bdd9b700 1 -- 172.21.15.83:0/3227433484 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b800bb90 con 0x7fd4c006c710 2023-11-26T07:11:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.741+0000 7fd4bdd9b700 1 -- 172.21.15.83:0/3227433484 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8012430 con 0x7fd4c006c710 2023-11-26T07:11:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.742+0000 7fd4c5163700 1 -- 172.21.15.83:0/3227433484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c006c710 msgr2=0x7fd4c00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.742+0000 7fd4c5163700 1 --2- 172.21.15.83:0/3227433484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c006c710 0x7fd4c00afb90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009dc0 tx=0x7fd4b800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.742+0000 7fd4c5163700 1 -- 172.21.15.83:0/3227433484 shutdown_connections 2023-11-26T07:11:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.742+0000 7fd4c5163700 1 --2- 172.21.15.83:0/3227433484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c006c710 0x7fd4c00afb90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.742+0000 7fd4c5163700 1 --2- 172.21.15.83:0/3227433484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4c015e080 0x7fd4c00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.742+0000 7fd4c5163700 1 -- 172.21.15.83:0/3227433484 >> 172.21.15.83:0/3227433484 conn(0x7fd4c0071b90 msgr2=0x7fd4c0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.742+0000 7fd4c5163700 1 -- 172.21.15.83:0/3227433484 shutdown_connections 2023-11-26T07:11:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.742+0000 7fd4c5163700 1 -- 172.21.15.83:0/3227433484 wait complete. 2023-11-26T07:11:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.743+0000 7fd4c5163700 1 Processor -- start 2023-11-26T07:11:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.743+0000 7fd4c5163700 1 -- start start 2023-11-26T07:11:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.744+0000 7fd4c5163700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4c006c710 0x7fd4c0150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.744+0000 7fd4c5163700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c00af4d0 0x7fd4c01545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.744+0000 7fd4c5163700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4c015e080 0x7fd4c00c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.744+0000 7fd4c5163700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c01f7a90 con 0x7fd4c00af4d0 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.744+0000 7fd4c5163700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c0154b10 con 0x7fd4c015e080 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.744+0000 7fd4c5163700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c01f6f10 con 0x7fd4c006c710 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.744+0000 7fd4be59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c00af4d0 0x7fd4c01545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.744+0000 7fd4be59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c00af4d0 0x7fd4c01545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34780/0 (socket says 172.21.15.83:34780) 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.744+0000 7fd4be59c700 1 -- 172.21.15.83:0/2735985770 learned_addr learned my addr 172.21.15.83:0/2735985770 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4c006c710 0x7fd4c0150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4c015e080 0x7fd4c00c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4be59c700 1 -- 172.21.15.83:0/2735985770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4c006c710 msgr2=0x7fd4c0150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4be59c700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4c006c710 0x7fd4c0150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4be59c700 1 -- 172.21.15.83:0/2735985770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4c015e080 msgr2=0x7fd4c00c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4be59c700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4c015e080 0x7fd4c00c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4be59c700 1 -- 172.21.15.83:0/2735985770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b8009a70 con 0x7fd4c00af4d0 2023-11-26T07:11:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4c015e080 0x7fd4c00c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4be59c700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c00af4d0 0x7fd4c01545d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400c540 tx=0x7fd4b400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4a77fe700 1 -- 172.21.15.83:0/2735985770 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4004c80 con 0x7fd4c00af4d0 2023-11-26T07:11:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4a77fe700 1 -- 172.21.15.83:0/2735985770 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b4004de0 con 0x7fd4c00af4d0 2023-11-26T07:11:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4c5163700 1 -- 172.21.15.83:0/2735985770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c01f71f0 con 0x7fd4c00af4d0 2023-11-26T07:11:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4a77fe700 1 -- 172.21.15.83:0/2735985770 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4007310 con 0x7fd4c00af4d0 2023-11-26T07:11:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.745+0000 7fd4c5163700 1 -- 172.21.15.83:0/2735985770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c01f77b0 con 0x7fd4c00af4d0 2023-11-26T07:11:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.746+0000 7fd4a77fe700 1 -- 172.21.15.83:0/2735985770 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd4b4007470 con 0x7fd4c00af4d0 2023-11-26T07:11:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.747+0000 7fd4a77fe700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4ac06aed0 0x7fd4ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.747+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4ac06aed0 0x7fd4ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.747+0000 7fd4a77fe700 1 -- 172.21.15.83:0/2735985770 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4b4089370 con 0x7fd4c00af4d0 2023-11-26T07:11:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.747+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4ac06aed0 0x7fd4ac06d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009770 tx=0x7fd4b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.749+0000 7fd4d135cb80 1 Processor -- start 2023-11-26T07:11:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.749+0000 7fd4d135cb80 1 -- start start 2023-11-26T07:11:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a9a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a69ed0 0x560961a9ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a7de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609619c7770 con 0x560961a51490 2023-11-26T07:11:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609619ae750 con 0x560961a69ed0 2023-11-26T07:11:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609619ac070 con 0x560961a54080 2023-11-26T07:11:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4bf59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a7de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4bf59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a7de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34790/0 (socket says 172.21.15.83:34790) 2023-11-26T07:11:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4bf59e700 1 -- 172.21.15.83:0/1785940192 learned_addr learned my addr 172.21.15.83:0/1785940192 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/1785940192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a9a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4be59c700 1 --2- 172.21.15.83:0/1785940192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a69ed0 0x560961a9ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4bf59e700 1 -- 172.21.15.83:0/1785940192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 msgr2=0x560961a9a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/1785940192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a9a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4bf59e700 1 -- 172.21.15.83:0/1785940192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a69ed0 msgr2=0x560961a9ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/1785940192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a69ed0 0x560961a9ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4bf59e700 1 -- 172.21.15.83:0/1785940192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609618245e0 con 0x560961a51490 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4be59c700 1 --2- 172.21.15.83:0/1785940192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a69ed0 0x560961a9ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/1785940192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a9a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/1785940192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a7de70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8014040 tx=0x7fd4b8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b7ce4c17b53a11 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4a4ff9700 1 -- 172.21.15.83:0/1785940192 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8003c20 con 0x560961a51490 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4a4ff9700 1 -- 172.21.15.83:0/1785940192 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b8021070 con 0x560961a51490 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.750+0000 7fd4a4ff9700 1 -- 172.21.15.83:0/1785940192 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80125e0 con 0x560961a51490 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.751+0000 7fd4d135cb80 1 -- 172.21.15.83:0/1785940192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 msgr2=0x560961a7de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.751+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/1785940192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a7de70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8014040 tx=0x7fd4b8010000 comp rx=0 tx=0).stop 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.751+0000 7fd4d135cb80 1 -- 172.21.15.83:0/1785940192 shutdown_connections 2023-11-26T07:11:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.751+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/1785940192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a7de70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.751+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/1785940192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a9a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.751+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/1785940192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a69ed0 0x560961a9ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.751+0000 7fd4d135cb80 1 -- 172.21.15.83:0/1785940192 >> 172.21.15.83:0/1785940192 conn(0x560961982310 msgr2=0x560961a98e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.751+0000 7fd4d135cb80 1 -- 172.21.15.83:0/1785940192 shutdown_connections 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.751+0000 7fd4d135cb80 1 -- 172.21.15.83:0/1785940192 wait complete. 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.752+0000 7fd4d135cb80 1 -- start start 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.752+0000 7fd4d135cb80 1 Processor -- start 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a51490 0x560961a8fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a48330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a48870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609619a7960 con 0x560961a69ed0 2023-11-26T07:11:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560961961510 con 0x560961a51490 2023-11-26T07:11:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609619611b0 con 0x560961a54080 2023-11-26T07:11:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4d135cb80 10 client.0 _ll_register_callbacks cb 0x56096188ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:11:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4bf59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a48870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4bf59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a48870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34796/0 (socket says 172.21.15.83:34796) 2023-11-26T07:11:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4bf59e700 1 -- 172.21.15.83:0/4097360683 learned_addr learned my addr 172.21.15.83:0/4097360683 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4be59c700 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a48330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a51490 0x560961a8fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4bf59e700 1 -- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 msgr2=0x560961a48330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a48330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4bf59e700 1 -- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a51490 msgr2=0x560961a8fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a51490 0x560961a8fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4bf59e700 1 -- 172.21.15.83:0/4097360683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c00ae710 con 0x560961a69ed0 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4be59c700 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a48330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a48870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800d010 tx=0x7fd4b8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eca3a59e2535a70 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd49dffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd49dffb700 1 -- 172.21.15.83:0/4097360683 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8004040 con 0x560961a69ed0 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd49dffb700 1 -- 172.21.15.83:0/4097360683 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4b80041a0 con 0x560961a69ed0 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4d135cb80 1 -- 172.21.15.83:0/4097360683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560961a46a40 con 0x560961a69ed0 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd49dffb700 1 -- 172.21.15.83:0/4097360683 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4b8004360 con 0x560961a69ed0 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4d135cb80 10 client.4659 fetch_fsmap learned FSMap version 8 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd4d135cb80 1 -- 172.21.15.83:0/4097360683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5609617d0d90 con 0x560961a69ed0 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.753+0000 7fd49dffb700 1 -- 172.21.15.83:0/4097360683 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560961a46a40 con 0x560961a69ed0 2023-11-26T07:11:42.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.754+0000 7fd49dffb700 1 -- 172.21.15.83:0/4097360683 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd4b80044c0 con 0x560961a69ed0 2023-11-26T07:11:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.754+0000 7fd4d135cb80 10 client.4659 fetch_fsmap finished waiting for FSMap version 8 2023-11-26T07:11:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.754+0000 7fd4d135cb80 10 client.4659 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:11:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.754+0000 7fd4d135cb80 20 client.4659 populate_metadata read hostname 'smithi083' 2023-11-26T07:11:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.754+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:6833/2957324266,v1:172.21.15.203:6835/2957324266] conn(0x560961a81a40 0x560961ab58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.754+0000 7fd4d135cb80 4 client.4659 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:11:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.754+0000 7fd4d135cb80 1 -- 172.21.15.83:0/4097360683 --> [v2:172.21.15.203:6833/2957324266,v1:172.21.15.203:6835/2957324266] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560961a22840 con 0x560961a81a40 2023-11-26T07:11:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.754+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:6833/2957324266,v1:172.21.15.203:6835/2957324266] conn(0x560961a81a40 0x560961ab58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.754+0000 7fd49dffb700 10 client.4659 ms_handle_connect on v2:172.21.15.203:6833/2957324266 2023-11-26T07:11:42.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.754+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:6833/2957324266,v1:172.21.15.203:6835/2957324266] conn(0x560961a81a40 0x560961ab58a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=731cada8f32bbb1 server_cookie=683f3b2cfd90d52f in_seq=0 out_seq=0 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd49dffb700 1 -- 172.21.15.83:0/4097360683 <== mds.0 v2:172.21.15.203:6833/2957324266 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560961a22840 con 0x560961a81a40 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd49dffb700 10 client.4659 handle_command_reply: tid=0 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 1 client.4659 shutdown 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 1 -- 172.21.15.83:0/4097360683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 msgr2=0x560961a48870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a48870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800d010 tx=0x7fd4b8004b40 comp rx=0 tx=0).stop 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 1 -- 172.21.15.83:0/4097360683 shutdown_connections 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:6833/2957324266,v1:172.21.15.203:6835/2957324266] conn(0x560961a81a40 0x560961ab58a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a48870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a51490 0x560961a8fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/4097360683 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a48330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd49dffb700 0 client.4659 ms_handle_reset on v2:172.21.15.203:6833/2957324266 2023-11-26T07:11:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 1 -- 172.21.15.83:0/4097360683 >> 172.21.15.83:0/4097360683 conn(0x560961982310 msgr2=0x560961a9cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 1 -- 172.21.15.83:0/4097360683 shutdown_connections 2023-11-26T07:11:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 1 -- 172.21.15.83:0/4097360683 wait complete. 2023-11-26T07:11:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.755+0000 7fd4d135cb80 20 client.4659 trim_cache size 0 max 16384 2023-11-26T07:11:42.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.773+0000 7fd4d135cb80 1 Processor -- start 2023-11-26T07:11:42.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.773+0000 7fd4d135cb80 1 -- start start 2023-11-26T07:11:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a92f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a8fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a69ed0 0x560961a96ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609618268d0 con 0x560961a51490 2023-11-26T07:11:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560961a81f50 con 0x560961a69ed0 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x560961a98f20 con 0x560961a54080 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4bed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a92f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4bed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a92f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34800/0 (socket says 172.21.15.83:34800) 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4bed9d700 1 -- 172.21.15.83:0/3732840158 learned_addr learned my addr 172.21.15.83:0/3732840158 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4bed9d700 1 -- 172.21.15.83:0/3732840158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 msgr2=0x560961a8fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/3732840158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a8fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4bed9d700 1 -- 172.21.15.83:0/3732840158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a69ed0 msgr2=0x560961a96ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/3732840158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a69ed0 0x560961a96ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4bed9d700 1 -- 172.21.15.83:0/3732840158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609617b9c10 con 0x560961a51490 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/3732840158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a92f10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009310 tx=0x7fd4b0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5a4017a0e39025 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4a57fa700 1 -- 172.21.15.83:0/3732840158 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00260b0 con 0x560961a51490 2023-11-26T07:11:42.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4a57fa700 1 -- 172.21.15.83:0/3732840158 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b0026210 con 0x560961a51490 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.774+0000 7fd4a57fa700 1 -- 172.21.15.83:0/3732840158 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00265a0 con 0x560961a51490 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.775+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3732840158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 msgr2=0x560961a92f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.775+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/3732840158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a92f10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009310 tx=0x7fd4b0014000 comp rx=0 tx=0).stop 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.775+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3732840158 shutdown_connections 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.775+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/3732840158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a69ed0 0x560961a96ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.775+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/3732840158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a51490 0x560961a92f10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.775+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/3732840158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961a8fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.775+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3732840158 >> 172.21.15.83:0/3732840158 conn(0x560961982310 msgr2=0x560961a93c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.775+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3732840158 shutdown_connections 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.775+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3732840158 wait complete. 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.775+0000 7fd4d135cb80 1 -- start start 2023-11-26T07:11:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.776+0000 7fd4d135cb80 1 Processor -- start 2023-11-26T07:11:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.776+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a51490 0x560961a8e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.776+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961afbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.776+0000 7fd4d135cb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a47cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.776+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560961afc430 con 0x560961a69ed0 2023-11-26T07:11:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.776+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560961a97fe0 con 0x560961a51490 2023-11-26T07:11:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.776+0000 7fd4d135cb80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x560961a8ee00 con 0x560961a54080 2023-11-26T07:11:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.776+0000 7fd4d135cb80 10 client.0 _ll_register_callbacks cb 0x56096188ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:11:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.776+0000 7fd4bf59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a47cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.776+0000 7fd4bf59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a47cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34810/0 (socket says 172.21.15.83:34810) 2023-11-26T07:11:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.776+0000 7fd4bf59e700 1 -- 172.21.15.83:0/3671497458 learned_addr learned my addr 172.21.15.83:0/3671497458 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4be59c700 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961afbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a51490 0x560961a8e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4bf59e700 1 -- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 msgr2=0x560961afbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961afbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4bf59e700 1 -- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a51490 msgr2=0x560961a8e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a51490 0x560961a8e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4bf59e700 1 -- 172.21.15.83:0/3671497458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b0015880 con 0x560961a69ed0 2023-11-26T07:11:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4bf59e700 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a47cb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8002230 tx=0x7fd4b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea6f5b20bfe20aa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd49dffb700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd49dffb700 1 -- 172.21.15.83:0/3671497458 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8012580 con 0x560961a69ed0 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd49dffb700 1 -- 172.21.15.83:0/3671497458 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4b80126e0 con 0x560961a69ed0 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3671497458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560961a69d60 con 0x560961a69ed0 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd49dffb700 1 -- 172.21.15.83:0/3671497458 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4b8004930 con 0x560961a69ed0 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4d135cb80 10 client.4665 fetch_fsmap learned FSMap version 8 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3671497458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5609617f7200 con 0x560961a69ed0 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd49dffb700 1 -- 172.21.15.83:0/3671497458 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b804b080 con 0x560961a69ed0 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd49dffb700 1 -- 172.21.15.83:0/3671497458 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd4b80128a0 con 0x560961a69ed0 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4d135cb80 10 client.4665 fetch_fsmap finished waiting for FSMap version 8 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4d135cb80 10 client.4665 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4d135cb80 20 client.4665 populate_metadata read hostname 'smithi083' 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:6833/2957324266,v1:172.21.15.203:6835/2957324266] conn(0x560961a91f00 0x560961a922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4d135cb80 4 client.4665 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:11:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.777+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3671497458 --> [v2:172.21.15.203:6833/2957324266,v1:172.21.15.203:6835/2957324266] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fd4b8021070 con 0x560961a91f00 2023-11-26T07:11:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:6833/2957324266,v1:172.21.15.203:6835/2957324266] conn(0x560961a91f00 0x560961a922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd49dffb700 10 client.4665 ms_handle_connect on v2:172.21.15.203:6833/2957324266 2023-11-26T07:11:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4bed9d700 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:6833/2957324266,v1:172.21.15.203:6835/2957324266] conn(0x560961a91f00 0x560961a922e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb0a6bc4462c35e2 server_cookie=8d0c97b53747d58b in_seq=0 out_seq=0 2023-11-26T07:11:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd49dffb700 1 -- 172.21.15.83:0/3671497458 <== mds.0 v2:172.21.15.203:6833/2957324266 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x7fd4b8021070 con 0x560961a91f00 2023-11-26T07:11:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd49dffb700 10 client.4665 handle_command_reply: tid=0 2023-11-26T07:11:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 1 client.4665 shutdown 2023-11-26T07:11:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3671497458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 msgr2=0x560961a47cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a47cb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8002230 tx=0x7fd4b8004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3671497458 shutdown_connections 2023-11-26T07:11:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:6833/2957324266,v1:172.21.15.203:6835/2957324266] conn(0x560961a91f00 0x560961a922e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560961a69ed0 0x560961a47cb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd49dffb700 0 client.4665 ms_handle_reset on v2:172.21.15.203:6833/2957324266 2023-11-26T07:11:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560961a51490 0x560961a8e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 1 --2- 172.21.15.83:0/3671497458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560961a54080 0x560961afbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3671497458 >> 172.21.15.83:0/3671497458 conn(0x560961982310 msgr2=0x560961aa88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:42.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3671497458 shutdown_connections 2023-11-26T07:11:42.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 1 -- 172.21.15.83:0/3671497458 wait complete. 2023-11-26T07:11:42.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.778+0000 7fd4d135cb80 20 client.4665 trim_cache size 0 max 16384 2023-11-26T07:11:42.791 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:11:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "0aae1049-cc8c-4011-a01f-378673e57a96", 2023-11-26T07:11:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-26T07:11:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 4285, 2023-11-26T07:11:42.791 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-26T07:11:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-26T07:11:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-26T07:11:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 14.244292413, 2023-11-26T07:11:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 8, 2023-11-26T07:11:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 21, 2023-11-26T07:11:42.792 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 21, 2023-11-26T07:11:42.793 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 21.863581915000001 2023-11-26T07:11:42.793 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:11:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 -- 172.21.15.83:0/2735985770 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4ac06aed0 msgr2=0x7fd4ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4ac06aed0 0x7fd4ac06d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0009770 tx=0x7fd4b0008040 comp rx=0 tx=0).stop 2023-11-26T07:11:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 -- 172.21.15.83:0/2735985770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c00af4d0 msgr2=0x7fd4c01545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c00af4d0 0x7fd4c01545d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400c540 tx=0x7fd4b400c900 comp rx=0 tx=0).stop 2023-11-26T07:11:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 -- 172.21.15.83:0/2735985770 shutdown_connections 2023-11-26T07:11:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4c015e080 0x7fd4c00c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4ac06aed0 0x7fd4ac06d390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4c006c710 0x7fd4c0150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 --2- 172.21.15.83:0/2735985770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c00af4d0 0x7fd4c01545d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 -- 172.21.15.83:0/2735985770 >> 172.21.15.83:0/2735985770 conn(0x7fd4c0071b90 msgr2=0x7fd4c006e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 -- 172.21.15.83:0/2735985770 shutdown_connections 2023-11-26T07:11:42.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:42.779+0000 7fd4c5163700 1 -- 172.21.15.83:0/2735985770 wait complete. 2023-11-26T07:11:42.797 INFO:teuthology.run_tasks:Running task kclient... 2023-11-26T07:11:42.820 INFO:tasks.kclient:Mounting kernel clients... 2023-11-26T07:11:42.820 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-26T07:11:42.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:42.820 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:11:42.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:42.837 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2023-11-26T07:11:42.905 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2023-11-26T07:11:42.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:11:42.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:42.907 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:11:42.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:42.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2023-11-26T07:11:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2023-11-26T07:11:43.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:11:43.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:43.032 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:11:43.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:43.053 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2023-11-26T07:11:43.127 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2023-11-26T07:11:43.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:11:43.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:43.129 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:11:43.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:43.188 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2023-11-26T07:11:43.257 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2023-11-26T07:11:43.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:11:43.259 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-11-26T07:11:43.259 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:11:43.259 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:11:43.259 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:11:43.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.379+0000 7fce85950700 1 Processor -- start 2023-11-26T07:11:43.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.379+0000 7fce85950700 1 -- start start 2023-11-26T07:11:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.379+0000 7fce85950700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fce800b1390 0x7fce8006b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.379+0000 7fce85950700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce800b1c60 con 0x7fce800d2390 2023-11-26T07:11:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.379+0000 7fce85950700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce800d0450 con 0x7fce800b0fb0 2023-11-26T07:11:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.379+0000 7fce85950700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce800d0720 con 0x7fce800b1390 2023-11-26T07:11:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.379+0000 7fce7e7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fce800b1390 0x7fce8006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.379+0000 7fce7f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fce800d2390 0x7fce8015d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54858/0 (socket says 172.21.15.83:54858) 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.379+0000 7fce7f7fe700 1 -- 172.21.15.83:0/2193242 learned_addr learned my addr 172.21.15.83:0/2193242 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.380+0000 7fce7e7fc700 1 -- 172.21.15.83:0/2193242 >> v1:172.21.15.203:6789/0 conn(0x7fce800b0fb0 legacy=0x7fce800cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.380+0000 7fce7e7fc700 1 -- 172.21.15.83:0/2193242 >> v1:172.21.15.83:6789/0 conn(0x7fce800d2390 legacy=0x7fce8015d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.380+0000 7fce7dffb700 1 -- 172.21.15.83:0/2193242 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038311424 0 0) 0x7fce800b1c60 con 0x7fce800d2390 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.380+0000 7fce7e7fc700 1 -- 172.21.15.83:0/2193242 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce800d0a00 con 0x7fce800b1390 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.380+0000 7fce7e7fc700 1 --2- 172.21.15.83:0/2193242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800b1390 0x7fce8006b750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fce70009840 tx=0x7fce70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d40ae6cadf4a294 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.380+0000 7fce7dffb700 1 -- 172.21.15.83:0/2193242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70004230 con 0x7fce800b1390 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.380+0000 7fce7dffb700 1 -- 172.21.15.83:0/2193242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce70004390 con 0x7fce800b1390 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.380+0000 7fce7dffb700 1 -- 172.21.15.83:0/2193242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700036a0 con 0x7fce800b1390 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.381+0000 7fce85950700 1 -- 172.21.15.83:0/2193242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800b1390 msgr2=0x7fce8006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:43.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.381+0000 7fce85950700 1 --2- 172.21.15.83:0/2193242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800b1390 0x7fce8006b750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fce70009840 tx=0x7fce70009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.381+0000 7fce85950700 1 -- 172.21.15.83:0/2193242 shutdown_connections 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.381+0000 7fce85950700 1 --2- 172.21.15.83:0/2193242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800b1390 0x7fce8006b750 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.381+0000 7fce85950700 1 -- 172.21.15.83:0/2193242 >> 172.21.15.83:0/2193242 conn(0x7fce8009aa00 msgr2=0x7fce8009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.381+0000 7fce85950700 1 -- 172.21.15.83:0/2193242 shutdown_connections 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.381+0000 7fce85950700 1 -- 172.21.15.83:0/2193242 wait complete. 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.382+0000 7fce85950700 1 Processor -- start 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.382+0000 7fce85950700 1 -- start start 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce85950700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce800b0fb0 0x7fce800c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce85950700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800b1390 0x7fce800c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce85950700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce800d2390 0x7fce800ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce85950700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce800c68d0 con 0x7fce800b1390 2023-11-26T07:11:43.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce85950700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce800c6bb0 con 0x7fce800d2390 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce85950700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce800c6e90 con 0x7fce800b0fb0 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce7e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800b1390 0x7fce800c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce7e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800b1390 0x7fce800c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34822/0 (socket says 172.21.15.83:34822) 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce7e7fc700 1 -- 172.21.15.83:0/3376824315 learned_addr learned my addr 172.21.15.83:0/3376824315 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce7f7fe700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce800d2390 0x7fce800ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce7effd700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce800b0fb0 0x7fce800c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce7e7fc700 1 -- 172.21.15.83:0/3376824315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce800b0fb0 msgr2=0x7fce800c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce7e7fc700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce800b0fb0 0x7fce800c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce7e7fc700 1 -- 172.21.15.83:0/3376824315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce800d2390 msgr2=0x7fce800ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce7e7fc700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce800d2390 0x7fce800ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce7e7fc700 1 -- 172.21.15.83:0/3376824315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce700094a0 con 0x7fce800b1390 2023-11-26T07:11:43.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce7e7fc700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800b1390 0x7fce800c61d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fce700090b0 tx=0x7fce70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce67fff700 1 -- 172.21.15.83:0/3376824315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70003de0 con 0x7fce800b1390 2023-11-26T07:11:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce67fff700 1 -- 172.21.15.83:0/3376824315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7001e070 con 0x7fce800b1390 2023-11-26T07:11:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce85950700 1 -- 172.21.15.83:0/3376824315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce800c7170 con 0x7fce800b1390 2023-11-26T07:11:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce67fff700 1 -- 172.21.15.83:0/3376824315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70011850 con 0x7fce800b1390 2023-11-26T07:11:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.383+0000 7fce85950700 1 -- 172.21.15.83:0/3376824315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce800caec0 con 0x7fce800b1390 2023-11-26T07:11:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.384+0000 7fce85950700 1 -- 172.21.15.83:0/3376824315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce800cb0b0 con 0x7fce800b1390 2023-11-26T07:11:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.385+0000 7fce67fff700 1 -- 172.21.15.83:0/3376824315 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce700119b0 con 0x7fce800b1390 2023-11-26T07:11:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.385+0000 7fce67fff700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fce6c06aed0 0x7fce6c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.385+0000 7fce7effd700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fce6c06aed0 0x7fce6c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.385+0000 7fce67fff700 1 -- 172.21.15.83:0/3376824315 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce7008a7e0 con 0x7fce800b1390 2023-11-26T07:11:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.386+0000 7fce7effd700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fce6c06aed0 0x7fce6c06d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fce74007fd0 tx=0x7fce74007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.390+0000 7fce67fff700 1 -- 172.21.15.83:0/3376824315 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce70055920 con 0x7fce800b1390 2023-11-26T07:11:43.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.627+0000 7fce85950700 1 -- 172.21.15.83:0/3376824315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fce80068480 con 0x7fce800b1390 2023-11-26T07:11:43.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.627+0000 7fce67fff700 1 -- 172.21.15.83:0/3376824315 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fce70020020 con 0x7fce800b1390 2023-11-26T07:11:43.630 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:11:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 -- 172.21.15.83:0/3376824315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fce6c06aed0 msgr2=0x7fce6c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:43.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fce6c06aed0 0x7fce6c06d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fce74007fd0 tx=0x7fce74007ec0 comp rx=0 tx=0).stop 2023-11-26T07:11:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 -- 172.21.15.83:0/3376824315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800b1390 msgr2=0x7fce800c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800b1390 0x7fce800c61d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fce700090b0 tx=0x7fce70004750 comp rx=0 tx=0).stop 2023-11-26T07:11:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 -- 172.21.15.83:0/3376824315 shutdown_connections 2023-11-26T07:11:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce800d2390 0x7fce800ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fce6c06aed0 0x7fce6c06d390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce800b0fb0 0x7fce800c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 --2- 172.21.15.83:0/3376824315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce800b1390 0x7fce800c61d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 -- 172.21.15.83:0/3376824315 >> 172.21.15.83:0/3376824315 conn(0x7fce8009aa00 msgr2=0x7fce800b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 -- 172.21.15.83:0/3376824315 shutdown_connections 2023-11-26T07:11:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:43.630+0000 7fce85950700 1 -- 172.21.15.83:0/3376824315 wait complete. 2023-11-26T07:11:43.644 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:11:43.644 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:11:43.644 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-26T07:11:43.645 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:11:43.645 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:11:43.645 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:11:43.645 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:11:43.645 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:11:43.645 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:11:43.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:43.645 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-26T07:11:43.662 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:11:43.662 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:11:43.662 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:11:43.662 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:43.662 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-26T07:11:43.662 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:43.662 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:43.662 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:43.663 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:43.663 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:43.663 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:43.663 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:43.663 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:11:43.663 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:43.663 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:11:43.663 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42383sec preferred_lft 42383sec 2023-11-26T07:11:43.663 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::3a41:fab:adc5:5d6c/64 scope link noprefixroute 2023-11-26T07:11:43.663 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:43.664 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-26T07:11:43.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:43.664 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:11:43.664 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:11:43.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2023-11-26T07:11:43.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2023-11-26T07:11:43.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2023-11-26T07:11:43.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-26T07:11:43.664 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:11:43.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:43.831 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:11:43.858 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-26T07:11:43.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:43.860 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-26T07:11:43.932 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-26T07:11:43.932 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:11:43.933 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:11:43.933 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:11:43.933 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:11:43.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:43.934 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:11:43.934 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:11:43.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:11:43.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:11:43.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:11:43.934 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:11:44.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:44.182 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:11:44.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:44.199 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2023-11-26T07:11:44.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:44.255 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:11:44.255 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:11:44.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:11:44.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-26T07:11:44.255 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:11:44.348 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-26T07:11:44.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:44.348 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:11:44.348 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:11:44.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-26T07:11:44.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-26T07:11:44.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-26T07:11:44.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-26T07:11:44.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-26T07:11:44.349 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:11:44.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:44.562 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:11:44.562 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:11:44.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-26T07:11:44.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2023-11-26T07:11:44.562 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:11:44.664 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:11:44.664 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:11:44.683 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:11:44.683 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:11:44.684 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:11:44.739 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:11:44.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:44.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:44.828 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:44.828 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:44.942 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:44.942 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:44.942 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:11:44.942 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:11:44.942 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:11:44.942 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:11:44.942 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:11:44.942 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:11:45.020 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:45.020 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:45.020 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:11:45.020 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:11:45.020 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:11:45.020 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:11:45.020 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:11:45.020 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:11:45.020 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:11:45.021 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:11:45.021 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:11:45.022 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:11:45.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:45.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:11:45.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:45.049 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:45.136 INFO:teuthology.orchestra.run.smithi083.stdout:client4427 2023-11-26T07:11:45.137 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:11:45.137 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:11:45.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:45.153 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:45.241 INFO:teuthology.orchestra.run.smithi083.stdout:client4427 2023-11-26T07:11:45.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:45.241 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:45.288 INFO:teuthology.orchestra.run.smithi083.stdout:client4427 2023-11-26T07:11:45.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:45.288 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/0aae1049-cc8c-4011-a01f-378673e57a96.client4427/status') 2023-11-26T07:11:45.342 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.4427 (3)172.21.15.83:0/2619209027 2023-11-26T07:11:45.342 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2023-11-26T07:11:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2023-11-26T07:11:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2023-11-26T07:11:45.342 INFO:teuthology.orchestra.run.smithi083.stderr:70 bytes copied, 2.6079e-05 s, 2.7 MB/s 2023-11-26T07:11:45.343 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-11-26T07:11:45.344 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:11:45.344 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-26T07:11:45.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:11:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.463+0000 7f39a3537700 1 Processor -- start 2023-11-26T07:11:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.463+0000 7f39a3537700 1 -- start start 2023-11-26T07:11:45.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a3537700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f399c15e080 0x7f399c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a3537700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f399c0af4d0 0x7f399c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a3537700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f399c06c000 0x7f399c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a3537700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399c0af0b0 con 0x7f399c06c000 2023-11-26T07:11:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a3537700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399c06c9f0 con 0x7f399c15e080 2023-11-26T07:11:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a3537700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0b05a0 con 0x7f399c0af4d0 2023-11-26T07:11:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a1ad4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f399c06c000 0x7f399c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a1ad4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f399c06c000 0x7f399c06c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34846/0 (socket says 172.21.15.83:34846) 2023-11-26T07:11:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a1ad4700 1 -- 172.21.15.83:0/140109969 learned_addr learned my addr 172.21.15.83:0/140109969 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a0ad2700 1 --2- 172.21.15.83:0/140109969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f399c0af4d0 0x7f399c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:45.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a12d3700 1 --2- 172.21.15.83:0/140109969 >> v2:172.21.15.203:3300/0 conn(0x7f399c15e080 0x7f399c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:45.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a1ad4700 1 -- 172.21.15.83:0/140109969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f399c0af4d0 msgr2=0x7f399c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:45.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a1ad4700 1 --2- 172.21.15.83:0/140109969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f399c0af4d0 0x7f399c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a1ad4700 1 -- 172.21.15.83:0/140109969 >> v2:172.21.15.203:3300/0 conn(0x7f399c15e080 msgr2=0x7f399c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:45.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a1ad4700 1 --2- 172.21.15.83:0/140109969 >> v2:172.21.15.203:3300/0 conn(0x7f399c15e080 0x7f399c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a1ad4700 1 -- 172.21.15.83:0/140109969 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c0ae430 con 0x7f399c06c000 2023-11-26T07:11:45.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a12d3700 1 --2- 172.21.15.83:0/140109969 >> v2:172.21.15.203:3300/0 conn(0x7f399c15e080 0x7f399c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:45.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f39a1ad4700 1 --2- 172.21.15.83:0/140109969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399c06c000 0x7f399c06c4b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f399c072a90 tx=0x7f399400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd9bb54abd02a09 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:45.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.464+0000 7f3993fff700 1 -- 172.21.15.83:0/140109969 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399400e070 con 0x7f399c06c000 2023-11-26T07:11:45.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.465+0000 7f3993fff700 1 -- 172.21.15.83:0/140109969 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399400b980 con 0x7f399c06c000 2023-11-26T07:11:45.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.465+0000 7f3993fff700 1 -- 172.21.15.83:0/140109969 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399400bd30 con 0x7f399c06c000 2023-11-26T07:11:45.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.465+0000 7f39a3537700 1 -- 172.21.15.83:0/140109969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399c06c000 msgr2=0x7f399c06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:45.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.465+0000 7f39a3537700 1 --2- 172.21.15.83:0/140109969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399c06c000 0x7f399c06c4b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f399c072a90 tx=0x7f399400b330 comp rx=0 tx=0).stop 2023-11-26T07:11:45.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.465+0000 7f39a3537700 1 -- 172.21.15.83:0/140109969 shutdown_connections 2023-11-26T07:11:45.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.465+0000 7f39a3537700 1 --2- 172.21.15.83:0/140109969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399c06c000 0x7f399c06c4b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.465+0000 7f39a3537700 1 --2- 172.21.15.83:0/140109969 >> v2:172.21.15.203:3300/0 conn(0x7f399c15e080 0x7f399c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.465+0000 7f39a3537700 1 --2- 172.21.15.83:0/140109969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f399c0af4d0 0x7f399c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.465+0000 7f39a3537700 1 -- 172.21.15.83:0/140109969 >> 172.21.15.83:0/140109969 conn(0x7f399c071b90 msgr2=0x7f399c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:45.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.465+0000 7f39a3537700 1 -- 172.21.15.83:0/140109969 shutdown_connections 2023-11-26T07:11:45.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.465+0000 7f39a3537700 1 -- 172.21.15.83:0/140109969 wait complete. 2023-11-26T07:11:45.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.466+0000 7f39a3537700 1 Processor -- start 2023-11-26T07:11:45.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.466+0000 7f39a3537700 1 -- start start 2023-11-26T07:11:45.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a3537700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f399c06c000 0x7f399c159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:45.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a3537700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f399c0af4d0 0x7f399c14a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:45.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a3537700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399c15e080 0x7f399c14a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:45.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a3537700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0cdb10 con 0x7f399c15e080 2023-11-26T07:11:45.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a3537700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c14aa90 con 0x7f399c06c000 2023-11-26T07:11:45.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a3537700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c154590 con 0x7f399c0af4d0 2023-11-26T07:11:45.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a1ad4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399c15e080 0x7f399c14a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:45.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a1ad4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399c15e080 0x7f399c14a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34852/0 (socket says 172.21.15.83:34852) 2023-11-26T07:11:45.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a1ad4700 1 -- 172.21.15.83:0/3630335301 learned_addr learned my addr 172.21.15.83:0/3630335301 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:45.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a0ad2700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f399c0af4d0 0x7f399c14a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:45.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a12d3700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f399c06c000 0x7f399c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:45.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a1ad4700 1 -- 172.21.15.83:0/3630335301 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f399c0af4d0 msgr2=0x7f399c14a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:45.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a1ad4700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f399c0af4d0 0x7f399c14a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a1ad4700 1 -- 172.21.15.83:0/3630335301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f399c06c000 msgr2=0x7f399c159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:45.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.467+0000 7f39a1ad4700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f399c06c000 0x7f399c159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.468+0000 7f39a1ad4700 1 -- 172.21.15.83:0/3630335301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3994009b30 con 0x7f399c15e080 2023-11-26T07:11:45.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.468+0000 7f39a1ad4700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399c15e080 0x7f399c14a550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f399c09f040 tx=0x7f3994004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:45.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.468+0000 7f3991ffb700 1 -- 172.21.15.83:0/3630335301 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994004250 con 0x7f399c15e080 2023-11-26T07:11:45.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.468+0000 7f3991ffb700 1 -- 172.21.15.83:0/3630335301 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39940043b0 con 0x7f399c15e080 2023-11-26T07:11:45.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.468+0000 7f39a3537700 1 -- 172.21.15.83:0/3630335301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c158ca0 con 0x7f399c15e080 2023-11-26T07:11:45.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.468+0000 7f3991ffb700 1 -- 172.21.15.83:0/3630335301 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994004760 con 0x7f399c15e080 2023-11-26T07:11:45.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.468+0000 7f39a3537700 1 -- 172.21.15.83:0/3630335301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c1589c0 con 0x7f399c15e080 2023-11-26T07:11:45.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.469+0000 7f39a3537700 1 -- 172.21.15.83:0/3630335301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c0753f0 con 0x7f399c15e080 2023-11-26T07:11:45.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.469+0000 7f3991ffb700 1 -- 172.21.15.83:0/3630335301 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f399401c440 con 0x7f399c15e080 2023-11-26T07:11:45.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.470+0000 7f3991ffb700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f398006aed0 0x7f398006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:45.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.470+0000 7f3991ffb700 1 -- 172.21.15.83:0/3630335301 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f399408ae40 con 0x7f399c15e080 2023-11-26T07:11:45.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.470+0000 7f39a12d3700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f398006aed0 0x7f398006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:45.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.470+0000 7f39a12d3700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f398006aed0 0x7f398006d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3998006fd0 tx=0x7f3998008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:45.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.474+0000 7f3991ffb700 1 -- 172.21.15.83:0/3630335301 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3994055f80 con 0x7f399c15e080 2023-11-26T07:11:45.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.710+0000 7f39a3537700 1 -- 172.21.15.83:0/3630335301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f399c067f60 con 0x7f399c15e080 2023-11-26T07:11:45.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.710+0000 7f3991ffb700 1 -- 172.21.15.83:0/3630335301 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3994020020 con 0x7f399c15e080 2023-11-26T07:11:45.713 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:11:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 -- 172.21.15.83:0/3630335301 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f398006aed0 msgr2=0x7f398006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f398006aed0 0x7f398006d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3998006fd0 tx=0x7f3998008040 comp rx=0 tx=0).stop 2023-11-26T07:11:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 -- 172.21.15.83:0/3630335301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399c15e080 msgr2=0x7f399c14a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399c15e080 0x7f399c14a550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f399c09f040 tx=0x7f3994004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 -- 172.21.15.83:0/3630335301 shutdown_connections 2023-11-26T07:11:45.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399c15e080 0x7f399c14a550 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f399c06c000 0x7f399c159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f398006aed0 0x7f398006d390 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 --2- 172.21.15.83:0/3630335301 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f399c0af4d0 0x7f399c14a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:45.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 -- 172.21.15.83:0/3630335301 >> 172.21.15.83:0/3630335301 conn(0x7f399c071b90 msgr2=0x7f399c15d270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:45.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 -- 172.21.15.83:0/3630335301 shutdown_connections 2023-11-26T07:11:45.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:45.712+0000 7f39a3537700 1 -- 172.21.15.83:0/3630335301 wait complete. 2023-11-26T07:11:45.727 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:11:45.727 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:11:45.727 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-26T07:11:45.727 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-26T07:11:45.727 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-26T07:11:45.727 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:11:45.727 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:11:45.728 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:11:45.728 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-26T07:11:45.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:45.728 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-26T07:11:45.745 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:11:45.745 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:11:45.745 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:11:45.745 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-26T07:11:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:45.746 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:45.746 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:45.746 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:45.747 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:11:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:11:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42381sec preferred_lft 42381sec 2023-11-26T07:11:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::3a41:fab:adc5:5d6c/64 scope link noprefixroute 2023-11-26T07:11:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:45.748 INFO:teuthology.orchestra.run.smithi083.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:11:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether fe:46:b0:72:7b:3f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:11:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::54b0:6eff:fe33:bb15/64 scope link 2023-11-26T07:11:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:45.748 INFO:teuthology.orchestra.run.smithi083.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:11:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether fe:46:b0:72:7b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:11:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::fc46:b0ff:fe72:7b3f/64 scope link tentative 2023-11-26T07:11:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:45.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:45.749 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:11:45.810 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-26T07:11:45.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:45.811 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-26T07:11:45.869 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-26T07:11:45.869 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:11:45.870 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:11:45.870 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:11:45.870 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:11:45.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:45.870 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:11:45.871 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:11:45.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:11:45.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:11:45.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:11:45.871 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:11:45.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:45.973 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:11:45.989 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:11:45.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:45.989 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2023-11-26T07:11:46.045 INFO:teuthology.orchestra.run.smithi083.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-26T07:11:46.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:46.046 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:11:46.046 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:11:46.046 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:11:46.046 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-26T07:11:46.047 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:11:46.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:46.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-26T07:11:46.176 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:11:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:11:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:11:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-26T07:11:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:46.177 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:11:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5a:36:68:bc:97:94 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:11:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:11:46.178 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:46.178 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5836:68ff:febc:9794/64 scope link 2023-11-26T07:11:46.178 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:46.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:46.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-26T07:11:46.267 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:11:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:11:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:11:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-26T07:11:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:46.268 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:11:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 5a:36:68:bc:97:94 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:11:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:11:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::5836:68ff:febc:9794/64 scope link 2023-11-26T07:11:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:46.281 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-26T07:11:46.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:46.282 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:11:46.282 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:11:46.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-26T07:11:46.282 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-26T07:11:46.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-26T07:11:46.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-26T07:11:46.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-26T07:11:46.283 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:11:46.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:46.539 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:11:46.539 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:11:46.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.1 up 2023-11-26T07:11:46.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.1 master ceph-brx 2023-11-26T07:11:46.540 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:11:46.646 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:11:46.646 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-26T07:11:46.671 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-26T07:11:46.671 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:11:46.671 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-26T07:11:46.728 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-11-26T07:11:46.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:46.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:46.813 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:46.813 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:46.836 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:46.836 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:46.836 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:11:46.837 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 2@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:11:46.837 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:11:46.837 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:11:46.837 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:11:46.837 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:11:46.918 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:46.918 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:46.918 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:11:46.918 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 2@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:11:46.918 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:11:46.918 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:11:46.918 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:11:46.918 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:11:46.918 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:11:46.918 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:11:46.919 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:11:46.920 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:11:46.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:46.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-26T07:11:46.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:46.947 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:47.018 INFO:teuthology.orchestra.run.smithi083.stdout:client4436 2023-11-26T07:11:47.019 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:11:47.019 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:11:47.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:47.076 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:47.148 INFO:teuthology.orchestra.run.smithi083.stdout:client4436 2023-11-26T07:11:47.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:47.148 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:47.219 INFO:teuthology.orchestra.run.smithi083.stdout:client4436 2023-11-26T07:11:47.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:47.220 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/0aae1049-cc8c-4011-a01f-378673e57a96.client4436/status') 2023-11-26T07:11:47.301 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.4436 (3)172.21.15.83:0/173830202 2023-11-26T07:11:47.301 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2023-11-26T07:11:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2023-11-26T07:11:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2023-11-26T07:11:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:69 bytes copied, 2.2316e-05 s, 3.1 MB/s 2023-11-26T07:11:47.302 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-11-26T07:11:47.303 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:11:47.303 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:11:47.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:11:47.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.462+0000 7f31df3c0700 1 Processor -- start 2023-11-26T07:11:47.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.462+0000 7f31df3c0700 1 -- start start 2023-11-26T07:11:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31df3c0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f31d806c710 0x7f31d80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31df3c0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d80af4d0 0x7f31d80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31df3c0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d80ae430 con 0x7f31d815e080 2023-11-26T07:11:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31df3c0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d80af0b0 con 0x7f31d806c710 2023-11-26T07:11:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31df3c0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80b05a0 con 0x7f31d80af4d0 2023-11-26T07:11:47.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31dc95b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f31d806c710 0x7f31d80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:47.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31dd15c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f31d815e080 0x7f31d806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54900/0 (socket says 172.21.15.83:54900) 2023-11-26T07:11:47.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31dd15c700 1 -- 172.21.15.83:0/1015454135 learned_addr learned my addr 172.21.15.83:0/1015454135 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:47.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31dd95d700 1 --2- 172.21.15.83:0/1015454135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d80af4d0 0x7f31d80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:47.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31cffff700 1 -- 172.21.15.83:0/1015454135 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832388640 0 0) 0x7f31d80ae430 con 0x7f31d815e080 2023-11-26T07:11:47.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31cffff700 1 -- 172.21.15.83:0/1015454135 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31c40035b0 con 0x7f31d815e080 2023-11-26T07:11:47.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31dc95b700 1 -- 172.21.15.83:0/1015454135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d80af4d0 msgr2=0x7f31d80afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:47.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31dc95b700 1 --2- 172.21.15.83:0/1015454135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d80af4d0 0x7f31d80afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:47.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31dc95b700 1 -- 172.21.15.83:0/1015454135 >> v1:172.21.15.83:6789/0 conn(0x7f31d815e080 legacy=0x7f31d806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:47.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31dc95b700 1 -- 172.21.15.83:0/1015454135 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d80ae710 con 0x7f31d806c710 2023-11-26T07:11:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.463+0000 7f31dc95b700 1 --2- 172.21.15.83:0/1015454135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31d806c710 0x7f31d80b5050 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009840 tx=0x7f31c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f85c5d970ead58a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.464+0000 7f31cffff700 1 -- 172.21.15.83:0/1015454135 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8011830 con 0x7f31d806c710 2023-11-26T07:11:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.464+0000 7f31cffff700 1 -- 172.21.15.83:0/1015454135 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c8011990 con 0x7f31d806c710 2023-11-26T07:11:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.464+0000 7f31cffff700 1 -- 172.21.15.83:0/1015454135 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8011d20 con 0x7f31d806c710 2023-11-26T07:11:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.464+0000 7f31df3c0700 1 -- 172.21.15.83:0/1015454135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31d806c710 msgr2=0x7f31d80b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.464+0000 7f31df3c0700 1 --2- 172.21.15.83:0/1015454135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31d806c710 0x7f31d80b5050 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009840 tx=0x7f31c8009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.464+0000 7f31df3c0700 1 -- 172.21.15.83:0/1015454135 shutdown_connections 2023-11-26T07:11:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.464+0000 7f31df3c0700 1 --2- 172.21.15.83:0/1015454135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d80af4d0 0x7f31d80afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.464+0000 7f31df3c0700 1 --2- 172.21.15.83:0/1015454135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31d806c710 0x7f31d80b5050 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.464+0000 7f31df3c0700 1 -- 172.21.15.83:0/1015454135 >> 172.21.15.83:0/1015454135 conn(0x7f31d8071b90 msgr2=0x7f31d8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.464+0000 7f31df3c0700 1 -- 172.21.15.83:0/1015454135 shutdown_connections 2023-11-26T07:11:47.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.464+0000 7f31df3c0700 1 -- 172.21.15.83:0/1015454135 wait complete. 2023-11-26T07:11:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.466+0000 7f31df3c0700 1 Processor -- start 2023-11-26T07:11:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.466+0000 7f31df3c0700 1 -- start start 2023-11-26T07:11:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.467+0000 7f31df3c0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d806c710 0x7f31d814a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.467+0000 7f31df3c0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31d80af4d0 0x7f31d815e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.467+0000 7f31df3c0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d815e080 0x7f31d80cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.467+0000 7f31df3c0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8153b40 con 0x7f31d80af4d0 2023-11-26T07:11:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.467+0000 7f31df3c0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d81546c0 con 0x7f31d806c710 2023-11-26T07:11:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.467+0000 7f31df3c0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8153d90 con 0x7f31d815e080 2023-11-26T07:11:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.467+0000 7f31dc95b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31d80af4d0 0x7f31d815e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.467+0000 7f31dc95b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31d80af4d0 0x7f31d815e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34868/0 (socket says 172.21.15.83:34868) 2023-11-26T07:11:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.467+0000 7f31dc95b700 1 -- 172.21.15.83:0/2780777756 learned_addr learned my addr 172.21.15.83:0/2780777756 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.467+0000 7f31dd15c700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d806c710 0x7f31d814a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31dd95d700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d815e080 0x7f31d80cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31dc95b700 1 -- 172.21.15.83:0/2780777756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d815e080 msgr2=0x7f31d80cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31dc95b700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d815e080 0x7f31d80cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31dc95b700 1 -- 172.21.15.83:0/2780777756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d806c710 msgr2=0x7f31d814a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31dc95b700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d806c710 0x7f31d814a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31dc95b700 1 -- 172.21.15.83:0/2780777756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c80094a0 con 0x7f31d80af4d0 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31dc95b700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31d80af4d0 0x7f31d815e460 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009fd0 tx=0x7f31c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31cdffb700 1 -- 172.21.15.83:0/2780777756 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80036a0 con 0x7f31d80af4d0 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31cdffb700 1 -- 172.21.15.83:0/2780777756 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c801f070 con 0x7f31d80af4d0 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31df3c0700 1 -- 172.21.15.83:0/2780777756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d8154070 con 0x7f31d80af4d0 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31cdffb700 1 -- 172.21.15.83:0/2780777756 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c8003a60 con 0x7f31d80af4d0 2023-11-26T07:11:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.468+0000 7f31df3c0700 1 -- 172.21.15.83:0/2780777756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d8156690 con 0x7f31d80af4d0 2023-11-26T07:11:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.469+0000 7f31df3c0700 1 -- 172.21.15.83:0/2780777756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d8156850 con 0x7f31d80af4d0 2023-11-26T07:11:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.470+0000 7f31cdffb700 1 -- 172.21.15.83:0/2780777756 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f31c8003bc0 con 0x7f31d80af4d0 2023-11-26T07:11:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.470+0000 7f31cdffb700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f31bc06aed0 0x7f31bc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.470+0000 7f31cdffb700 1 -- 172.21.15.83:0/2780777756 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31c8021070 con 0x7f31d80af4d0 2023-11-26T07:11:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.470+0000 7f31dd15c700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f31bc06aed0 0x7f31bc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.471+0000 7f31dd15c700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f31bc06aed0 0x7f31bc06d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f31d4007fd0 tx=0x7f31d4007dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:47.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.474+0000 7f31cdffb700 1 -- 172.21.15.83:0/2780777756 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f31c8056290 con 0x7f31d80af4d0 2023-11-26T07:11:47.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.711+0000 7f31df3c0700 1 -- 172.21.15.83:0/2780777756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f31d80753f0 con 0x7f31d80af4d0 2023-11-26T07:11:47.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.712+0000 7f31cdffb700 1 -- 172.21.15.83:0/2780777756 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f31c8015060 con 0x7f31d80af4d0 2023-11-26T07:11:47.714 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:11:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 -- 172.21.15.83:0/2780777756 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f31bc06aed0 msgr2=0x7f31bc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f31bc06aed0 0x7f31bc06d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f31d4007fd0 tx=0x7f31d4007dc0 comp rx=0 tx=0).stop 2023-11-26T07:11:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 -- 172.21.15.83:0/2780777756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31d80af4d0 msgr2=0x7f31d815e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31d80af4d0 0x7f31d815e460 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009fd0 tx=0x7f31c8004750 comp rx=0 tx=0).stop 2023-11-26T07:11:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 -- 172.21.15.83:0/2780777756 shutdown_connections 2023-11-26T07:11:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f31bc06aed0 0x7f31bc06d390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31d815e080 0x7f31d80cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:47.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31d806c710 0x7f31d814a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:47.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 --2- 172.21.15.83:0/2780777756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31d80af4d0 0x7f31d815e460 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:47.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 -- 172.21.15.83:0/2780777756 >> 172.21.15.83:0/2780777756 conn(0x7f31d8071b90 msgr2=0x7f31d80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:47.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 -- 172.21.15.83:0/2780777756 shutdown_connections 2023-11-26T07:11:47.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:47.714+0000 7f31df3c0700 1 -- 172.21.15.83:0/2780777756 wait complete. 2023-11-26T07:11:47.728 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:11:47.728 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:11:47.728 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2023-11-26T07:11:47.728 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:11:47.728 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:11:47.728 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:11:47.728 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:11:47.729 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:11:47.729 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:11:47.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:47.729 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2023-11-26T07:11:47.749 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:11:47.749 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:11:47.749 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:11:47.750 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42381sec preferred_lft 42381sec 2023-11-26T07:11:47.751 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::64b0:e04:3aa8:a78c/64 scope link noprefixroute 2023-11-26T07:11:47.751 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:47.751 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-26T07:11:47.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:47.751 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:11:47.751 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:11:47.751 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add name ceph-brx type bridge 2023-11-26T07:11:47.751 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr flush dev ceph-brx 2023-11-26T07:11:47.751 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set ceph-brx up 2023-11-26T07:11:47.751 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-26T07:11:47.752 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:11:47.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:47.935 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:11:47.963 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-11-26T07:11:47.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:47.965 DEBUG:teuthology.orchestra.run.smithi203:> route 2023-11-26T07:11:48.036 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2023-11-26T07:11:48.036 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:11:48.036 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:11:48.036 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:11:48.037 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:11:48.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:48.037 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:11:48.037 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:11:48.037 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:11:48.037 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:11:48.037 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:11:48.037 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:11:48.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:48.285 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:11:48.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:48.302 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2023-11-26T07:11:48.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:48.358 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:11:48.359 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:11:48.359 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:11:48.359 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-26T07:11:48.359 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:11:48.454 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-26T07:11:48.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:48.454 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:11:48.454 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:11:48.454 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-26T07:11:48.454 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-26T07:11:48.454 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-26T07:11:48.454 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-26T07:11:48.455 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-26T07:11:48.455 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:11:48.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:48.651 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:11:48.651 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:11:48.651 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2023-11-26T07:11:48.651 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.0 master ceph-brx 2023-11-26T07:11:48.651 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:11:48.754 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:11:48.754 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:11:48.772 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:11:48.772 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:11:48.772 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:11:48.827 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:11:48.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:48.828 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:48.934 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:48.934 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:49.038 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:49.038 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:49.038 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:11:49.038 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:11:49.038 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:11:49.038 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:11:49.038 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:11:49.038 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:11:49.071 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:49.071 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:49.071 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:11:49.071 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:11:49.071 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:11:49.071 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:11:49.071 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:11:49.071 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:11:49.072 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:11:49.072 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:11:49.072 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:11:49.089 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:11:49.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:49.090 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:11:49.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:49.119 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:49.191 INFO:teuthology.orchestra.run.smithi203.stdout:client4445 2023-11-26T07:11:49.191 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:49.191 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:11:49.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:49.248 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:49.319 INFO:teuthology.orchestra.run.smithi203.stdout:client4445 2023-11-26T07:11:49.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:49.320 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:49.391 INFO:teuthology.orchestra.run.smithi203.stdout:client4445 2023-11-26T07:11:49.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:49.391 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/0aae1049-cc8c-4011-a01f-378673e57a96.client4445/status') 2023-11-26T07:11:49.471 INFO:teuthology.orchestra.run.smithi203.stdout:instance: client.4445 (3)192.168.0.1:0/1793424692 2023-11-26T07:11:49.471 INFO:teuthology.orchestra.run.smithi203.stdout:blocklisted: false 2023-11-26T07:11:49.471 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records in 2023-11-26T07:11:49.471 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records out 2023-11-26T07:11:49.472 INFO:teuthology.orchestra.run.smithi203.stderr:69 bytes copied, 2.3851e-05 s, 2.9 MB/s 2023-11-26T07:11:49.472 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-11-26T07:11:49.473 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:11:49.473 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-26T07:11:49.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:11:49.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.604+0000 7fdb45b00700 1 Processor -- start 2023-11-26T07:11:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.604+0000 7fdb45b00700 1 -- start start 2023-11-26T07:11:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb45b00700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb4015e080 0x7fdb400b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb45b00700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdb4006c710 0x7fdb400afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb45b00700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb400ae430 con 0x7fdb400af4d0 2023-11-26T07:11:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb45b00700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb400af0b0 con 0x7fdb4006c710 2023-11-26T07:11:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb45b00700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400b05a0 con 0x7fdb4015e080 2023-11-26T07:11:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb3ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdb4006c710 0x7fdb400afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb3ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdb4006c710 0x7fdb400afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34876/0 (socket says 172.21.15.83:34876) 2023-11-26T07:11:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb3ffff700 1 -- 172.21.15.83:0/2689618416 learned_addr learned my addr 172.21.15.83:0/2689618416 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb3f7fe700 1 --2- 172.21.15.83:0/2689618416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb4015e080 0x7fdb400b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb3ffff700 1 -- 172.21.15.83:0/2689618416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb4015e080 msgr2=0x7fdb400b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb3ffff700 1 --2- 172.21.15.83:0/2689618416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb4015e080 0x7fdb400b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb3ffff700 1 -- 172.21.15.83:0/2689618416 >> v1:172.21.15.203:6789/0 conn(0x7fdb400af4d0 legacy=0x7fdb4006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb3ffff700 1 -- 172.21.15.83:0/2689618416 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb400ae710 con 0x7fdb4006c710 2023-11-26T07:11:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.605+0000 7fdb3ffff700 1 --2- 172.21.15.83:0/2689618416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb4006c710 0x7fdb400afb90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009e80 tx=0x7fdb3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445e7bcd1caafcfc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.606+0000 7fdb3e7fc700 1 -- 172.21.15.83:0/2689618416 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3800e070 con 0x7fdb4006c710 2023-11-26T07:11:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.606+0000 7fdb3e7fc700 1 -- 172.21.15.83:0/2689618416 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3800b980 con 0x7fdb4006c710 2023-11-26T07:11:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.606+0000 7fdb3e7fc700 1 -- 172.21.15.83:0/2689618416 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3800bd30 con 0x7fdb4006c710 2023-11-26T07:11:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.606+0000 7fdb45b00700 1 -- 172.21.15.83:0/2689618416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb4006c710 msgr2=0x7fdb400afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.606+0000 7fdb45b00700 1 --2- 172.21.15.83:0/2689618416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb4006c710 0x7fdb400afb90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009e80 tx=0x7fdb3800b330 comp rx=0 tx=0).stop 2023-11-26T07:11:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.606+0000 7fdb45b00700 1 -- 172.21.15.83:0/2689618416 shutdown_connections 2023-11-26T07:11:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.606+0000 7fdb45b00700 1 --2- 172.21.15.83:0/2689618416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb4006c710 0x7fdb400afb90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.606+0000 7fdb45b00700 1 --2- 172.21.15.83:0/2689618416 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb4015e080 0x7fdb400b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.606+0000 7fdb45b00700 1 -- 172.21.15.83:0/2689618416 >> 172.21.15.83:0/2689618416 conn(0x7fdb40071b90 msgr2=0x7fdb40071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.606+0000 7fdb45b00700 1 -- 172.21.15.83:0/2689618416 shutdown_connections 2023-11-26T07:11:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.606+0000 7fdb45b00700 1 -- 172.21.15.83:0/2689618416 wait complete. 2023-11-26T07:11:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.607+0000 7fdb45b00700 1 Processor -- start 2023-11-26T07:11:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.608+0000 7fdb45b00700 1 -- start start 2023-11-26T07:11:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.608+0000 7fdb45b00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb4006c710 0x7fdb40153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.608+0000 7fdb45b00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb400af4d0 0x7fdb40154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.608+0000 7fdb45b00700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb4015e080 0x7fdb40156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.608+0000 7fdb45b00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb40158290 con 0x7fdb4006c710 2023-11-26T07:11:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.608+0000 7fdb45b00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb40153b60 con 0x7fdb400af4d0 2023-11-26T07:11:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.608+0000 7fdb45b00700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb40157610 con 0x7fdb4015e080 2023-11-26T07:11:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.608+0000 7fdb3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb4006c710 0x7fdb40153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb4006c710 0x7fdb40153620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34878/0 (socket says 172.21.15.83:34878) 2023-11-26T07:11:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb3f7fe700 1 -- 172.21.15.83:0/2630453596 learned_addr learned my addr 172.21.15.83:0/2630453596 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb3ffff700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb4015e080 0x7fdb40156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb3effd700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb400af4d0 0x7fdb40154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb3f7fe700 1 -- 172.21.15.83:0/2630453596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb4015e080 msgr2=0x7fdb40156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb3f7fe700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb4015e080 0x7fdb40156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb3f7fe700 1 -- 172.21.15.83:0/2630453596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb400af4d0 msgr2=0x7fdb40154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb3f7fe700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb400af4d0 0x7fdb40154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb3f7fe700 1 -- 172.21.15.83:0/2630453596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb38009b30 con 0x7fdb4006c710 2023-11-26T07:11:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb3f7fe700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb4006c710 0x7fdb40153620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400d800 tx=0x7fdb3400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb27fff700 1 -- 172.21.15.83:0/2630453596 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb340125f0 con 0x7fdb4006c710 2023-11-26T07:11:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb27fff700 1 -- 172.21.15.83:0/2630453596 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34012750 con 0x7fdb4006c710 2023-11-26T07:11:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb45b00700 1 -- 172.21.15.83:0/2630453596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb401578f0 con 0x7fdb4006c710 2023-11-26T07:11:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb27fff700 1 -- 172.21.15.83:0/2630453596 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34012b00 con 0x7fdb4006c710 2023-11-26T07:11:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.609+0000 7fdb45b00700 1 -- 172.21.15.83:0/2630453596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb40157eb0 con 0x7fdb4006c710 2023-11-26T07:11:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.610+0000 7fdb45b00700 1 -- 172.21.15.83:0/2630453596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb400753f0 con 0x7fdb4006c710 2023-11-26T07:11:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.611+0000 7fdb27fff700 1 -- 172.21.15.83:0/2630453596 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb34012c60 con 0x7fdb4006c710 2023-11-26T07:11:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.611+0000 7fdb27fff700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb2c06aed0 0x7fdb2c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.611+0000 7fdb3effd700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb2c06aed0 0x7fdb2c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.611+0000 7fdb27fff700 1 -- 172.21.15.83:0/2630453596 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb34089640 con 0x7fdb4006c710 2023-11-26T07:11:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.612+0000 7fdb3effd700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb2c06aed0 0x7fdb2c06d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb400c6260 tx=0x7fdb30009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.616+0000 7fdb27fff700 1 -- 172.21.15.83:0/2630453596 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb340546f0 con 0x7fdb4006c710 2023-11-26T07:11:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.852+0000 7fdb45b00700 1 -- 172.21.15.83:0/2630453596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdb40067f60 con 0x7fdb4006c710 2023-11-26T07:11:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.852+0000 7fdb27fff700 1 -- 172.21.15.83:0/2630453596 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdb34057d10 con 0x7fdb4006c710 2023-11-26T07:11:49.855 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:11:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.854+0000 7fdb45b00700 1 -- 172.21.15.83:0/2630453596 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb2c06aed0 msgr2=0x7fdb2c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.854+0000 7fdb45b00700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb2c06aed0 0x7fdb2c06d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb400c6260 tx=0x7fdb30009040 comp rx=0 tx=0).stop 2023-11-26T07:11:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.854+0000 7fdb45b00700 1 -- 172.21.15.83:0/2630453596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb4006c710 msgr2=0x7fdb40153620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.854+0000 7fdb45b00700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb4006c710 0x7fdb40153620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400d800 tx=0x7fdb3400db10 comp rx=0 tx=0).stop 2023-11-26T07:11:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.854+0000 7fdb45b00700 1 -- 172.21.15.83:0/2630453596 shutdown_connections 2023-11-26T07:11:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.854+0000 7fdb45b00700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb2c06aed0 0x7fdb2c06d390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.854+0000 7fdb45b00700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb4015e080 0x7fdb40156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.854+0000 7fdb45b00700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb4006c710 0x7fdb40153620 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.854+0000 7fdb45b00700 1 --2- 172.21.15.83:0/2630453596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb400af4d0 0x7fdb40154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.854+0000 7fdb45b00700 1 -- 172.21.15.83:0/2630453596 >> 172.21.15.83:0/2630453596 conn(0x7fdb40071b90 msgr2=0x7fdb4006bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.855+0000 7fdb45b00700 1 -- 172.21.15.83:0/2630453596 shutdown_connections 2023-11-26T07:11:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:49.855+0000 7fdb45b00700 1 -- 172.21.15.83:0/2630453596 wait complete. 2023-11-26T07:11:49.869 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:11:49.869 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:11:49.869 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2023-11-26T07:11:49.869 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-26T07:11:49.869 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-26T07:11:49.869 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:11:49.870 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:11:49.870 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:11:49.870 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-26T07:11:49.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:49.870 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2023-11-26T07:11:49.888 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:11:49.888 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:11:49.889 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:11:49.889 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:49.889 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-11-26T07:11:49.889 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:49.889 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:49.889 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:49.889 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:49.890 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:49.890 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:11:49.890 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:49.890 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:11:49.890 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:49.890 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:11:49.890 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42379sec preferred_lft 42379sec 2023-11-26T07:11:49.891 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::64b0:e04:3aa8:a78c/64 scope link noprefixroute 2023-11-26T07:11:49.891 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:49.891 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:11:49.891 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:11:49.891 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:11:49.891 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:49.891 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1092:a6ff:fe63:3da7/64 scope link 2023-11-26T07:11:49.891 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:49.892 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:11:49.892 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:11:49.892 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::b88a:cff:fe60:2b6f/64 scope link 2023-11-26T07:11:49.892 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:49.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:49.892 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:11:49.955 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-11-26T07:11:49.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:49.957 DEBUG:teuthology.orchestra.run.smithi203:> route 2023-11-26T07:11:50.015 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2023-11-26T07:11:50.016 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:11:50.016 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:11:50.016 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:11:50.016 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:11:50.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:50.017 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:11:50.017 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:11:50.017 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:11:50.017 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:11:50.017 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:11:50.017 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:11:50.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:50.121 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:11:50.138 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:11:50.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:50.138 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2023-11-26T07:11:50.195 INFO:teuthology.orchestra.run.smithi203.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-26T07:11:50.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:50.195 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:11:50.195 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:11:50.195 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:11:50.196 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-26T07:11:50.196 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:11:50.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:50.287 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-26T07:11:50.327 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:11:50.327 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:11:50.327 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:11:50.327 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:50.327 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-11-26T07:11:50.327 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:50.328 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:11:50.328 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether fe:ba:c1:77:3b:96 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:11:50.328 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:11:50.328 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:50.328 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::fcba:c1ff:fe77:3b96/64 scope link tentative 2023-11-26T07:11:50.328 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:50.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:50.345 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-26T07:11:50.428 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:11:50.428 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:11:50.428 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:11:50.428 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:50.428 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-11-26T07:11:50.428 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:50.428 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:11:50.428 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether fe:ba:c1:77:3b:96 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:11:50.429 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:11:50.429 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:50.429 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::fcba:c1ff:fe77:3b96/64 scope link 2023-11-26T07:11:50.429 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:11:50.439 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-26T07:11:50.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:50.440 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:11:50.440 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:11:50.440 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-26T07:11:50.440 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-26T07:11:50.440 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-26T07:11:50.440 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-26T07:11:50.440 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-26T07:11:50.440 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:11:50.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:50.676 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:11:50.676 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:11:50.676 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.1 up 2023-11-26T07:11:50.676 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.1 master ceph-brx 2023-11-26T07:11:50.676 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:11:50.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:11:50.791 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-26T07:11:50.812 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-26T07:11:50.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:11:50.812 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-26T07:11:50.868 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-11-26T07:11:50.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:50.868 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:50.953 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:50.953 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:50.980 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:50.980 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:50.980 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:11:50.980 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 3@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:11:50.980 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:11:50.980 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:11:50.981 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:11:50.981 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:11:51.012 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:11:51.012 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:11:51.013 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:11:51.013 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 3@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:11:51.013 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:11:51.013 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:11:51.013 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:11:51.013 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:11:51.013 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:11:51.013 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:11:51.013 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:11:51.031 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:11:51.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:51.031 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-26T07:11:51.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:51.060 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:51.131 INFO:teuthology.orchestra.run.smithi203.stdout:client4579 2023-11-26T07:11:51.132 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:11:51.132 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:11:51.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:51.189 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:51.260 INFO:teuthology.orchestra.run.smithi203.stdout:client4579 2023-11-26T07:11:51.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:51.261 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:11:51.334 INFO:teuthology.orchestra.run.smithi203.stdout:client4579 2023-11-26T07:11:51.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:11:51.334 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/0aae1049-cc8c-4011-a01f-378673e57a96.client4579/status') 2023-11-26T07:11:51.414 INFO:teuthology.orchestra.run.smithi203.stdout:instance: client.4579 (3)192.168.0.2:0/2026916620 2023-11-26T07:11:51.415 INFO:teuthology.orchestra.run.smithi203.stdout:blocklisted: false 2023-11-26T07:11:51.415 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records in 2023-11-26T07:11:51.416 INFO:teuthology.orchestra.run.smithi203.stderr:0+1 records out 2023-11-26T07:11:51.416 INFO:teuthology.orchestra.run.smithi203.stderr:69 bytes copied, 2.0438e-05 s, 3.4 MB/s 2023-11-26T07:11:51.416 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-26T07:11:51.432 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:51.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.565+0000 7fadcff07700 1 Processor -- start 2023-11-26T07:11:51.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.565+0000 7fadcff07700 1 -- start start 2023-11-26T07:11:51.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadcff07700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fadc815e080 0x7fadc80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadcff07700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc80ae430 con 0x7fadc806c710 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadcff07700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc80ae710 con 0x7fadc80af4d0 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadcff07700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc80af0b0 con 0x7fadc815e080 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadce4a4700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fadc806c710 0x7fadc806e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54932/0 (socket says 172.21.15.83:54932) 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadce4a4700 1 -- 172.21.15.83:0/3607366858 learned_addr learned my addr 172.21.15.83:0/3607366858 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadcdca3700 1 --2- 172.21.15.83:0/3607366858 >> v2:172.21.15.203:3300/0 conn(0x7fadc815e080 0x7fadc80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244348311 0 0) 0x7fadc80ae430 con 0x7fadc806c710 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadb40035b0 con 0x7fadc806c710 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 465475662 0 0) 0x7fadb40035b0 con 0x7fadc806c710 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 >> v2:172.21.15.203:3300/0 conn(0x7fadc815e080 msgr2=0x7fadc80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadccca1700 1 --2- 172.21.15.83:0/3607366858 >> v2:172.21.15.203:3300/0 conn(0x7fadc815e080 0x7fadc80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 >> v1:172.21.15.203:6789/0 conn(0x7fadc80af4d0 legacy=0x7fadc806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc80ae9f0 con 0x7fadc806c710 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.566+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fadc00019f0 con 0x7fadc806c710 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 >> v1:172.21.15.83:6789/0 conn(0x7fadc806c710 legacy=0x7fadc806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadccca1700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadb4003a00 0x7fadb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadccca1700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadb4004850 0x7fadb4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadccca1700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb4005210 0x7fadb400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80af0b0 con 0x7fadb4005210 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb400c0b0 con 0x7fadb4003a00 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb400c270 con 0x7fadb4004850 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadce4a4700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb4005210 0x7fadb400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadcdca3700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadb4003a00 0x7fadb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadcd4a2700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadb4004850 0x7fadb4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadce4a4700 1 -- 172.21.15.83:0/3607366858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadb4004850 msgr2=0x7fadb4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadce4a4700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadb4004850 0x7fadb4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadce4a4700 1 -- 172.21.15.83:0/3607366858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadb4003a00 msgr2=0x7fadb4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadce4a4700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadb4003a00 0x7fadb4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadce4a4700 1 -- 172.21.15.83:0/3607366858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb40035b0 con 0x7fadb4005210 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadcdca3700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadb4003a00 0x7fadb4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadce4a4700 1 -- 172.21.15.83:0/3607366858 reap_dead start 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadce4a4700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb4005210 0x7fadb400ba10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fadc000a800 tx=0x7fadb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775f7ee5df8c232d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc000a2f0 con 0x7fadb4005210 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc0012600 con 0x7fadb4005210 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.567+0000 7fadccca1700 1 -- 172.21.15.83:0/3607366858 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc00129b0 con 0x7fadb4005210 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.568+0000 7fadcff07700 1 -- 172.21.15.83:0/3607366858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb4005210 msgr2=0x7fadb400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.568+0000 7fadcff07700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb4005210 0x7fadb400ba10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fadc000a800 tx=0x7fadb4005880 comp rx=0 tx=0).stop 2023-11-26T07:11:51.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.568+0000 7fadcff07700 1 -- 172.21.15.83:0/3607366858 shutdown_connections 2023-11-26T07:11:51.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.568+0000 7fadcff07700 1 --2- 172.21.15.83:0/3607366858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadb4005210 0x7fadb400ba10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.568+0000 7fadcff07700 1 -- 172.21.15.83:0/3607366858 >> 172.21.15.83:0/3607366858 conn(0x7fadc8071b90 msgr2=0x7fadc8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:51.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.568+0000 7fadcff07700 1 -- 172.21.15.83:0/3607366858 shutdown_connections 2023-11-26T07:11:51.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.568+0000 7fadcff07700 1 -- 172.21.15.83:0/3607366858 wait complete. 2023-11-26T07:11:51.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.569+0000 7fadcff07700 1 Processor -- start 2023-11-26T07:11:51.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.569+0000 7fadcff07700 1 -- start start 2023-11-26T07:11:51.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadcff07700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadb4005210 0x7fadc8144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadcff07700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadc8156340 0x7fadc8160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadcff07700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc8141bb0 0x7fadc8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadcff07700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc81608c0 con 0x7fadc8141bb0 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadcff07700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc815ff90 con 0x7fadb4005210 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadcff07700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8160270 con 0x7fadc8156340 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadce4a4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc8141bb0 0x7fadc8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadce4a4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc8141bb0 0x7fadc8073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34904/0 (socket says 172.21.15.83:34904) 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadce4a4700 1 -- 172.21.15.83:0/1461781059 learned_addr learned my addr 172.21.15.83:0/1461781059 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadcdca3700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadb4005210 0x7fadc8144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadcd4a2700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadc8156340 0x7fadc8160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadce4a4700 1 -- 172.21.15.83:0/1461781059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadc8156340 msgr2=0x7fadc8160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadce4a4700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadc8156340 0x7fadc8160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadce4a4700 1 -- 172.21.15.83:0/1461781059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadb4005210 msgr2=0x7fadc8144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadce4a4700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadb4005210 0x7fadc8144fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadce4a4700 1 -- 172.21.15.83:0/1461781059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc8160550 con 0x7fadc8141bb0 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.570+0000 7fadce4a4700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc8141bb0 0x7fadc8073f90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fadc8161a70 tx=0x7fadc0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.571+0000 7fadbe7fc700 1 -- 172.21.15.83:0/1461781059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0001250 con 0x7fadc8141bb0 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.571+0000 7fadbe7fc700 1 -- 172.21.15.83:0/1461781059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc0013dc0 con 0x7fadc8141bb0 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.571+0000 7fadcff07700 1 -- 172.21.15.83:0/1461781059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc815fc40 con 0x7fadc8141bb0 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.571+0000 7fadbe7fc700 1 -- 172.21.15.83:0/1461781059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc001d660 con 0x7fadc8141bb0 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.571+0000 7fadcff07700 1 -- 172.21.15.83:0/1461781059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc8161290 con 0x7fadc8141bb0 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.572+0000 7fadcff07700 1 -- 172.21.15.83:0/1461781059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc80753f0 con 0x7fadc8141bb0 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.572+0000 7fadbe7fc700 1 -- 172.21.15.83:0/1461781059 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadc001d7c0 con 0x7fadc8141bb0 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.573+0000 7fadbe7fc700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fadac06af00 0x7fadac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.573+0000 7fadbe7fc700 1 -- 172.21.15.83:0/1461781059 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadc008b300 con 0x7fadc8141bb0 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.573+0000 7fadcdca3700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fadac06af00 0x7fadac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.573+0000 7fadcdca3700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fadac06af00 0x7fadac06d3c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009be0 tx=0x7fadc4009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:51.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.577+0000 7fadbe7fc700 1 -- 172.21.15.83:0/1461781059 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadc0056440 con 0x7fadc8141bb0 2023-11-26T07:11:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.770+0000 7fadcff07700 1 -- 172.21.15.83:0/1461781059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadc8067f60 con 0x7fadc8141bb0 2023-11-26T07:11:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.771+0000 7fadbe7fc700 1 -- 172.21.15.83:0/1461781059 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fadc0059a60 con 0x7fadc8141bb0 2023-11-26T07:11:51.774 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:51.774 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 -- 172.21.15.83:0/1461781059 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fadac06af00 msgr2=0x7fadac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fadac06af00 0x7fadac06d3c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009be0 tx=0x7fadc4009210 comp rx=0 tx=0).stop 2023-11-26T07:11:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 -- 172.21.15.83:0/1461781059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc8141bb0 msgr2=0x7fadc8073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc8141bb0 0x7fadc8073f90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fadc8161a70 tx=0x7fadc0001d80 comp rx=0 tx=0).stop 2023-11-26T07:11:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 -- 172.21.15.83:0/1461781059 shutdown_connections 2023-11-26T07:11:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadc8141bb0 0x7fadc8073f90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadb4005210 0x7fadc8144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fadac06af00 0x7fadac06d3c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 --2- 172.21.15.83:0/1461781059 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadc8156340 0x7fadc8160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 -- 172.21.15.83:0/1461781059 >> 172.21.15.83:0/1461781059 conn(0x7fadc8071b90 msgr2=0x7fadc815d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 -- 172.21.15.83:0/1461781059 shutdown_connections 2023-11-26T07:11:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.773+0000 7fadcff07700 1 -- 172.21.15.83:0/1461781059 wait complete. 2023-11-26T07:11:51.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:11:51.789 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:11:51.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.909+0000 7fc3d0a69700 1 Processor -- start 2023-11-26T07:11:51.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.909+0000 7fc3d0a69700 1 -- start start 2023-11-26T07:11:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3d0a69700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc3cc15e160 0x7fc3cc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3d0a69700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3cc0afc90 0x7fc3cc073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3d0a69700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3cc0b3c60 con 0x7fc3cc0744d0 2023-11-26T07:11:51.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3d0a69700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3cc0b4500 con 0x7fc3cc0afc90 2023-11-26T07:11:51.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3d0a69700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3cc074940 con 0x7fc3cc15e160 2023-11-26T07:11:51.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3c9d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc3cc0afc90 0x7fc3cc073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3cad9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc3cc0744d0 0x7fc3cc06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54952/0 (socket says 172.21.15.83:54952) 2023-11-26T07:11:51.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3cad9d700 1 -- 172.21.15.83:0/803376105 learned_addr learned my addr 172.21.15.83:0/803376105 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:51.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3ca59c700 1 --2- 172.21.15.83:0/803376105 >> v2:172.21.15.203:3300/0 conn(0x7fc3cc15e160 0x7fc3cc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3c9d9b700 1 -- 172.21.15.83:0/803376105 >> v2:172.21.15.203:3300/0 conn(0x7fc3cc15e160 msgr2=0x7fc3cc0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:51.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3c9d9b700 1 --2- 172.21.15.83:0/803376105 >> v2:172.21.15.203:3300/0 conn(0x7fc3cc15e160 0x7fc3cc0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3c9d9b700 1 -- 172.21.15.83:0/803376105 >> v1:172.21.15.83:6789/0 conn(0x7fc3cc0744d0 legacy=0x7fc3cc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:51.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3c9d9b700 1 -- 172.21.15.83:0/803376105 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc0b58c0 con 0x7fc3cc0afc90 2023-11-26T07:11:51.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.910+0000 7fc3c959a700 1 -- 172.21.15.83:0/803376105 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863709330 0 0) 0x7fc3cc0b3c60 con 0x7fc3cc0744d0 2023-11-26T07:11:51.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3c9d9b700 1 --2- 172.21.15.83:0/803376105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3cc0afc90 0x7fc3cc073f90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc09efa0 tx=0x7fc3c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50ff2151f8013b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3c959a700 1 -- 172.21.15.83:0/803376105 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0011830 con 0x7fc3cc0afc90 2023-11-26T07:11:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3c959a700 1 -- 172.21.15.83:0/803376105 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c0011990 con 0x7fc3cc0afc90 2023-11-26T07:11:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3c959a700 1 -- 172.21.15.83:0/803376105 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0011d20 con 0x7fc3cc0afc90 2023-11-26T07:11:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3d0a69700 1 -- 172.21.15.83:0/803376105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3cc0afc90 msgr2=0x7fc3cc073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3d0a69700 1 --2- 172.21.15.83:0/803376105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3cc0afc90 0x7fc3cc073f90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc09efa0 tx=0x7fc3c0009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3d0a69700 1 -- 172.21.15.83:0/803376105 shutdown_connections 2023-11-26T07:11:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3d0a69700 1 --2- 172.21.15.83:0/803376105 >> v2:172.21.15.203:3300/0 conn(0x7fc3cc15e160 0x7fc3cc0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3d0a69700 1 --2- 172.21.15.83:0/803376105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3cc0afc90 0x7fc3cc073f90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3d0a69700 1 -- 172.21.15.83:0/803376105 >> 172.21.15.83:0/803376105 conn(0x7fc3cc0d0bf0 msgr2=0x7fc3cc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3d0a69700 1 -- 172.21.15.83:0/803376105 shutdown_connections 2023-11-26T07:11:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.911+0000 7fc3d0a69700 1 -- 172.21.15.83:0/803376105 wait complete. 2023-11-26T07:11:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.913+0000 7fc3d0a69700 1 Processor -- start 2023-11-26T07:11:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.913+0000 7fc3d0a69700 1 -- start start 2023-11-26T07:11:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.913+0000 7fc3d0a69700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3cc0744d0 0x7fc3cc1f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.913+0000 7fc3d0a69700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3cc0afc90 0x7fc3cc0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.913+0000 7fc3d0a69700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3cc15e160 0x7fc3cc0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.913+0000 7fc3d0a69700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc14d3f0 con 0x7fc3cc0744d0 2023-11-26T07:11:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.913+0000 7fc3d0a69700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc14c520 con 0x7fc3cc15e160 2023-11-26T07:11:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.913+0000 7fc3d0a69700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc14c770 con 0x7fc3cc0afc90 2023-11-26T07:11:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3ca59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3cc0744d0 0x7fc3cc1f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3ca59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3cc0744d0 0x7fc3cc1f42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34914/0 (socket says 172.21.15.83:34914) 2023-11-26T07:11:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3ca59c700 1 -- 172.21.15.83:0/704881473 learned_addr learned my addr 172.21.15.83:0/704881473 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:51.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3cad9d700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3cc15e160 0x7fc3cc0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3c9d9b700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3cc0afc90 0x7fc3cc0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3ca59c700 1 -- 172.21.15.83:0/704881473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3cc0afc90 msgr2=0x7fc3cc0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:51.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3ca59c700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3cc0afc90 0x7fc3cc0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3ca59c700 1 -- 172.21.15.83:0/704881473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3cc15e160 msgr2=0x7fc3cc0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:51.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3ca59c700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3cc15e160 0x7fc3cc0ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:51.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3ca59c700 1 -- 172.21.15.83:0/704881473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c00094a0 con 0x7fc3cc0744d0 2023-11-26T07:11:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3c9d9b700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3cc0afc90 0x7fc3cc0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3ca59c700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3cc0744d0 0x7fc3cc1f42a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00e810 tx=0x7fc3bc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3baffd700 1 -- 172.21.15.83:0/704881473 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc013610 con 0x7fc3cc0744d0 2023-11-26T07:11:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3baffd700 1 -- 172.21.15.83:0/704881473 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc013770 con 0x7fc3cc0744d0 2023-11-26T07:11:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3d0a69700 1 -- 172.21.15.83:0/704881473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3cc14ca50 con 0x7fc3cc0744d0 2023-11-26T07:11:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3baffd700 1 -- 172.21.15.83:0/704881473 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc013b20 con 0x7fc3cc0744d0 2023-11-26T07:11:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.914+0000 7fc3d0a69700 1 -- 172.21.15.83:0/704881473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3cc14d010 con 0x7fc3cc0744d0 2023-11-26T07:11:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.915+0000 7fc3d0a69700 1 -- 172.21.15.83:0/704881473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3cc068fb0 con 0x7fc3cc0744d0 2023-11-26T07:11:51.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.915+0000 7fc3baffd700 1 -- 172.21.15.83:0/704881473 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc3bc013c80 con 0x7fc3cc0744d0 2023-11-26T07:11:51.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.916+0000 7fc3baffd700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc3ac06aed0 0x7fc3ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:51.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.916+0000 7fc3baffd700 1 -- 172.21.15.83:0/704881473 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3bc089230 con 0x7fc3cc0744d0 2023-11-26T07:11:51.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.916+0000 7fc3c9d9b700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc3ac06aed0 0x7fc3ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:51.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.916+0000 7fc3c9d9b700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc3ac06aed0 0x7fc3ac06d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009b90 tx=0x7fc3c0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:51.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:51.920+0000 7fc3baffd700 1 -- 172.21.15.83:0/704881473 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc3bc0542f0 con 0x7fc3cc0744d0 2023-11-26T07:11:52.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.107+0000 7fc3d0a69700 1 -- 172.21.15.83:0/704881473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3cc06c140 con 0x7fc3cc0744d0 2023-11-26T07:11:52.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.107+0000 7fc3baffd700 1 -- 172.21.15.83:0/704881473 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3bc057910 con 0x7fc3cc0744d0 2023-11-26T07:11:52.110 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:11:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 -- 172.21.15.83:0/704881473 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc3ac06aed0 msgr2=0x7fc3ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc3ac06aed0 0x7fc3ac06d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009b90 tx=0x7fc3c0008030 comp rx=0 tx=0).stop 2023-11-26T07:11:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 -- 172.21.15.83:0/704881473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3cc0744d0 msgr2=0x7fc3cc1f42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3cc0744d0 0x7fc3cc1f42a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00e810 tx=0x7fc3bc00ebd0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 -- 172.21.15.83:0/704881473 shutdown_connections 2023-11-26T07:11:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3cc15e160 0x7fc3cc0ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3cc0744d0 0x7fc3cc1f42a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc3ac06aed0 0x7fc3ac06d390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 --2- 172.21.15.83:0/704881473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3cc0afc90 0x7fc3cc0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 -- 172.21.15.83:0/704881473 >> 172.21.15.83:0/704881473 conn(0x7fc3cc0d0bf0 msgr2=0x7fc3cc15d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:52.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 -- 172.21.15.83:0/704881473 shutdown_connections 2023-11-26T07:11:52.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.109+0000 7fc3d0a69700 1 -- 172.21.15.83:0/704881473 wait complete. 2023-11-26T07:11:52.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:11:52.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.246+0000 7f957c921700 1 Processor -- start 2023-11-26T07:11:52.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.246+0000 7f957c921700 1 -- start start 2023-11-26T07:11:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957c921700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f95780b5da0 0x7f95780afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957c921700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95780b3c60 con 0x7f95780746a0 2023-11-26T07:11:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957c921700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95780b58c0 con 0x7f957815e160 2023-11-26T07:11:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957c921700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95780b4500 con 0x7f95780b5da0 2023-11-26T07:11:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f9575d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f95780746a0 0x7f957806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54962/0 (socket says 172.21.15.83:54962) 2023-11-26T07:11:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f9575d9b700 1 -- 172.21.15.83:0/1165906774 learned_addr learned my addr 172.21.15.83:0/1165906774 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f9576d9d700 1 --2- 172.21.15.83:0/1165906774 >> v2:172.21.15.203:3300/0 conn(0x7f95780b5da0 0x7f95780afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919444373 0 0) 0x7f95780b3c60 con 0x7f95780746a0 2023-11-26T07:11:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95600035b0 con 0x7f95780746a0 2023-11-26T07:11:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2107504065 0 0) 0x7f95600035b0 con 0x7f95780746a0 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 >> v2:172.21.15.203:3300/0 conn(0x7f95780b5da0 msgr2=0x7f95780afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 --2- 172.21.15.83:0/1165906774 >> v2:172.21.15.203:3300/0 conn(0x7f95780b5da0 0x7f95780afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 >> v1:172.21.15.203:6789/0 conn(0x7f957815e160 legacy=0x7f9578073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957806bdc0 con 0x7f95780746a0 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f9568001670 con 0x7f95780746a0 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 >> v1:172.21.15.83:6789/0 conn(0x7f95780746a0 legacy=0x7f957806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 --2- 172.21.15.83:0/1165906774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9560003a00 0x7f9560003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 --2- 172.21.15.83:0/1165906774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9560004850 0x7f9560004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 --2- 172.21.15.83:0/1165906774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9560005210 0x7f956000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95780b4500 con 0x7f9560005210 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956000c0b0 con 0x7f9560003a00 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.247+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956000c270 con 0x7f9560004850 2023-11-26T07:11:52.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f9576d9d700 1 --2- 172.21.15.83:0/1165906774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9560005210 0x7f956000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f957659c700 1 --2- 172.21.15.83:0/1165906774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9560003a00 0x7f9560003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f9575d9b700 1 --2- 172.21.15.83:0/1165906774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9560004850 0x7f9560004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f9576d9d700 1 -- 172.21.15.83:0/1165906774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9560004850 msgr2=0x7f9560004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f9576d9d700 1 --2- 172.21.15.83:0/1165906774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9560004850 0x7f9560004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f9576d9d700 1 -- 172.21.15.83:0/1165906774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9560003a00 msgr2=0x7f9560003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f9576d9d700 1 --2- 172.21.15.83:0/1165906774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9560003a00 0x7f9560003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f9576d9d700 1 -- 172.21.15.83:0/1165906774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600035b0 con 0x7f9560005210 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f9576d9d700 1 -- 172.21.15.83:0/1165906774 reap_dead start 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f9576d9d700 1 --2- 172.21.15.83:0/1165906774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9560005210 0x7f956000ba10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9570009fc0 tx=0x7f9560005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d2dd2c8a247e1c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570003a20 con 0x7f9560005210 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957000f040 con 0x7f9560005210 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f957559a700 1 -- 172.21.15.83:0/1165906774 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570003de0 con 0x7f9560005210 2023-11-26T07:11:52.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f957c921700 1 -- 172.21.15.83:0/1165906774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9560005210 msgr2=0x7f956000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.248+0000 7f957c921700 1 --2- 172.21.15.83:0/1165906774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9560005210 0x7f956000ba10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9570009fc0 tx=0x7f9560005880 comp rx=0 tx=0).stop 2023-11-26T07:11:52.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.249+0000 7f957c921700 1 -- 172.21.15.83:0/1165906774 shutdown_connections 2023-11-26T07:11:52.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.249+0000 7f957c921700 1 --2- 172.21.15.83:0/1165906774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9560005210 0x7f956000ba10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.249+0000 7f957c921700 1 -- 172.21.15.83:0/1165906774 >> 172.21.15.83:0/1165906774 conn(0x7f95780d0bf0 msgr2=0x7f95780d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:52.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.249+0000 7f957c921700 1 -- 172.21.15.83:0/1165906774 shutdown_connections 2023-11-26T07:11:52.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.249+0000 7f957c921700 1 -- 172.21.15.83:0/1165906774 wait complete. 2023-11-26T07:11:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.250+0000 7f957c921700 1 Processor -- start 2023-11-26T07:11:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.250+0000 7f957c921700 1 -- start start 2023-11-26T07:11:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f957c921700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9560005210 0x7f9578147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f957c921700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9578154ba0 0x7f9578160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f957c921700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95781611b0 0x7f9578158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f957c921700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9578161660 con 0x7f9578154ba0 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f957c921700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9578157c30 con 0x7f95781611b0 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f957c921700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95781f7c80 con 0x7f9560005210 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9575d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9578154ba0 0x7f9578160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9575d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9578154ba0 0x7f9578160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34936/0 (socket says 172.21.15.83:34936) 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9575d9b700 1 -- 172.21.15.83:0/132260441 learned_addr learned my addr 172.21.15.83:0/132260441 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f957659c700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9560005210 0x7f9578147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9576d9d700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95781611b0 0x7f9578158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9575d9b700 1 -- 172.21.15.83:0/132260441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9560005210 msgr2=0x7f9578147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9575d9b700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9560005210 0x7f9578147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9575d9b700 1 -- 172.21.15.83:0/132260441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95781611b0 msgr2=0x7f9578158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9575d9b700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95781611b0 0x7f9578158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9575d9b700 1 -- 172.21.15.83:0/132260441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578147bb0 con 0x7f9578154ba0 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9576d9d700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95781611b0 0x7f9578158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f957659c700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9560005210 0x7f9578147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9575d9b700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9578154ba0 0x7f9578160c70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f95680072f0 tx=0x7f95680076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9566ffd700 1 -- 172.21.15.83:0/132260441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568001450 con 0x7f9578154ba0 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9566ffd700 1 -- 172.21.15.83:0/132260441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956800f7d0 con 0x7f9578154ba0 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f957c921700 1 -- 172.21.15.83:0/132260441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95781f7100 con 0x7f9578154ba0 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f9566ffd700 1 -- 172.21.15.83:0/132260441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956800fb80 con 0x7f9578154ba0 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.251+0000 7f957c921700 1 -- 172.21.15.83:0/132260441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95781f76c0 con 0x7f9578154ba0 2023-11-26T07:11:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.252+0000 7f957c921700 1 -- 172.21.15.83:0/132260441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9578068fb0 con 0x7f9578154ba0 2023-11-26T07:11:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.253+0000 7f9566ffd700 1 -- 172.21.15.83:0/132260441 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f956800fce0 con 0x7f9578154ba0 2023-11-26T07:11:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.253+0000 7f9566ffd700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f955806aed0 0x7f955806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.253+0000 7f9566ffd700 1 -- 172.21.15.83:0/132260441 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9568088ef0 con 0x7f9578154ba0 2023-11-26T07:11:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.253+0000 7f957659c700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f955806aed0 0x7f955806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.254+0000 7f957659c700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f955806aed0 0x7f955806d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f956c000c00 tx=0x7f956c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.258+0000 7f9566ffd700 1 -- 172.21.15.83:0/132260441 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9568054030 con 0x7f9578154ba0 2023-11-26T07:11:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.446+0000 7f957c921700 1 -- 172.21.15.83:0/132260441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9578159920 con 0x7f9578154ba0 2023-11-26T07:11:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.446+0000 7f9566ffd700 1 -- 172.21.15.83:0/132260441 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9578159920 con 0x7f9578154ba0 2023-11-26T07:11:52.449 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:11:52.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.448+0000 7f957c921700 1 -- 172.21.15.83:0/132260441 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f955806aed0 msgr2=0x7f955806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.448+0000 7f957c921700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f955806aed0 0x7f955806d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f956c000c00 tx=0x7f956c009040 comp rx=0 tx=0).stop 2023-11-26T07:11:52.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.448+0000 7f957c921700 1 -- 172.21.15.83:0/132260441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9578154ba0 msgr2=0x7f9578160c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.448+0000 7f957c921700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9578154ba0 0x7f9578160c70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f95680072f0 tx=0x7f95680076b0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.448+0000 7f957c921700 1 -- 172.21.15.83:0/132260441 shutdown_connections 2023-11-26T07:11:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.449+0000 7f957c921700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95781611b0 0x7f9578158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.449+0000 7f957c921700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f955806aed0 0x7f955806d390 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.449+0000 7f957c921700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9560005210 0x7f9578147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.449+0000 7f957c921700 1 --2- 172.21.15.83:0/132260441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9578154ba0 0x7f9578160c70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.449+0000 7f957c921700 1 -- 172.21.15.83:0/132260441 >> 172.21.15.83:0/132260441 conn(0x7f95780d0bf0 msgr2=0x7f95780d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.449+0000 7f957c921700 1 -- 172.21.15.83:0/132260441 shutdown_connections 2023-11-26T07:11:52.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.449+0000 7f957c921700 1 -- 172.21.15.83:0/132260441 wait complete. 2023-11-26T07:11:52.463 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:11:52.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.587+0000 7fc06f0b4700 1 Processor -- start 2023-11-26T07:11:52.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.587+0000 7fc06f0b4700 1 -- start start 2023-11-26T07:11:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.587+0000 7fc06f0b4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc06815b370 0x7fc06815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.587+0000 7fc06f0b4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0680ae5e0 con 0x7fc0680b3c60 2023-11-26T07:11:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc06f0b4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06815be30 con 0x7fc0681587f0 2023-11-26T07:11:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc06f0b4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06815c110 con 0x7fc06815b370 2023-11-26T07:11:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc06d651700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc0680b3c60 0x7fc0680af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54984/0 (socket says 172.21.15.83:54984) 2023-11-26T07:11:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc06d651700 1 -- 172.21.15.83:0/1018249027 learned_addr learned my addr 172.21.15.83:0/1018249027 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc067fff700 1 --2- 172.21.15.83:0/1018249027 >> v2:172.21.15.203:3300/0 conn(0x7fc06815b370 0x7fc06815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc0677fe700 1 -- 172.21.15.83:0/1018249027 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936737622 0 0) 0x7fc0680ae5e0 con 0x7fc0680b3c60 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc0677fe700 1 -- 172.21.15.83:0/1018249027 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0580057c0 con 0x7fc0680b3c60 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc0677fe700 1 -- 172.21.15.83:0/1018249027 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953983084 0 0) 0x7fc06815be30 con 0x7fc0681587f0 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc0677fe700 1 -- 172.21.15.83:0/1018249027 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0680ae5e0 con 0x7fc0681587f0 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc067fff700 1 -- 172.21.15.83:0/1018249027 >> v1:172.21.15.203:6789/0 conn(0x7fc0681587f0 legacy=0x7fc06806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc067fff700 1 -- 172.21.15.83:0/1018249027 >> v1:172.21.15.83:6789/0 conn(0x7fc0680b3c60 legacy=0x7fc0680af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc0677fe700 1 -- 172.21.15.83:0/1018249027 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3107705839 0 0) 0x7fc0580057c0 con 0x7fc0680b3c60 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc067fff700 1 -- 172.21.15.83:0/1018249027 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680ae860 con 0x7fc06815b370 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc067fff700 1 --2- 172.21.15.83:0/1018249027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc06815b370 0x7fc06815b800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc054009870 tx=0x7fc054009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4fd7bf207b570cb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.588+0000 7fc0677fe700 1 -- 172.21.15.83:0/1018249027 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054011830 con 0x7fc06815b370 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.589+0000 7fc0677fe700 1 -- 172.21.15.83:0/1018249027 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc054011990 con 0x7fc06815b370 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.589+0000 7fc0677fe700 1 -- 172.21.15.83:0/1018249027 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054011d00 con 0x7fc06815b370 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.589+0000 7fc06f0b4700 1 -- 172.21.15.83:0/1018249027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc06815b370 msgr2=0x7fc06815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.589+0000 7fc06f0b4700 1 --2- 172.21.15.83:0/1018249027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc06815b370 0x7fc06815b800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc054009870 tx=0x7fc054009b80 comp rx=0 tx=0).stop 2023-11-26T07:11:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.589+0000 7fc06f0b4700 1 -- 172.21.15.83:0/1018249027 shutdown_connections 2023-11-26T07:11:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.589+0000 7fc06f0b4700 1 --2- 172.21.15.83:0/1018249027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc06815b370 0x7fc06815b800 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.589+0000 7fc06f0b4700 1 -- 172.21.15.83:0/1018249027 >> 172.21.15.83:0/1018249027 conn(0x7fc0680d0bf0 msgr2=0x7fc0680d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.589+0000 7fc06f0b4700 1 -- 172.21.15.83:0/1018249027 shutdown_connections 2023-11-26T07:11:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.589+0000 7fc06f0b4700 1 -- 172.21.15.83:0/1018249027 wait complete. 2023-11-26T07:11:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.590+0000 7fc06f0b4700 1 Processor -- start 2023-11-26T07:11:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.590+0000 7fc06f0b4700 1 -- start start 2023-11-26T07:11:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.591+0000 7fc06f0b4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0680b3c60 0x7fc068155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.591+0000 7fc06f0b4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0681587f0 0x7fc068144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.591+0000 7fc06f0b4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06815b370 0x7fc068144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.591+0000 7fc06f0b4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06809b680 con 0x7fc06815b370 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.591+0000 7fc06f0b4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06809aa00 con 0x7fc0680b3c60 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.591+0000 7fc06f0b4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06809ace0 con 0x7fc0681587f0 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.591+0000 7fc06d651700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06815b370 0x7fc068144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.591+0000 7fc06d651700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06815b370 0x7fc068144770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34950/0 (socket says 172.21.15.83:34950) 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.591+0000 7fc06d651700 1 -- 172.21.15.83:0/3409091977 learned_addr learned my addr 172.21.15.83:0/3409091977 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.591+0000 7fc06ce50700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0680b3c60 0x7fc068155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.591+0000 7fc067fff700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0681587f0 0x7fc068144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc06d651700 1 -- 172.21.15.83:0/3409091977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0681587f0 msgr2=0x7fc068144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc06d651700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0681587f0 0x7fc068144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc06d651700 1 -- 172.21.15.83:0/3409091977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0680b3c60 msgr2=0x7fc068155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc06d651700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0680b3c60 0x7fc068155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc06d651700 1 -- 172.21.15.83:0/3409091977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0540094a0 con 0x7fc06815b370 2023-11-26T07:11:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc06ce50700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0680b3c60 0x7fc068155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc06d651700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06815b370 0x7fc068144770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc06000cac0 tx=0x7fc06000ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc0657fa700 1 -- 172.21.15.83:0/3409091977 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06000aaa0 con 0x7fc06815b370 2023-11-26T07:11:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc0657fa700 1 -- 172.21.15.83:0/3409091977 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06000ac00 con 0x7fc06815b370 2023-11-26T07:11:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc0657fa700 1 -- 172.21.15.83:0/3409091977 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060012400 con 0x7fc06815b370 2023-11-26T07:11:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc06f0b4700 1 -- 172.21.15.83:0/3409091977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06809afc0 con 0x7fc06815b370 2023-11-26T07:11:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.592+0000 7fc06f0b4700 1 -- 172.21.15.83:0/3409091977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc068145090 con 0x7fc06815b370 2023-11-26T07:11:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.593+0000 7fc06f0b4700 1 -- 172.21.15.83:0/3409091977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06809b7c0 con 0x7fc06815b370 2023-11-26T07:11:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.593+0000 7fc0657fa700 1 -- 172.21.15.83:0/3409091977 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc06001b3b0 con 0x7fc06815b370 2023-11-26T07:11:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.594+0000 7fc0657fa700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc04c06aed0 0x7fc04c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.594+0000 7fc0657fa700 1 -- 172.21.15.83:0/3409091977 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc060089360 con 0x7fc06815b370 2023-11-26T07:11:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.594+0000 7fc06ce50700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc04c06aed0 0x7fc04c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.594+0000 7fc06ce50700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc04c06aed0 0x7fc04c06d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0011e0 tx=0x7fc05c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.598+0000 7fc0657fa700 1 -- 172.21.15.83:0/3409091977 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc06809b7c0 con 0x7fc06815b370 2023-11-26T07:11:52.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.788+0000 7fc06f0b4700 1 -- 172.21.15.83:0/3409091977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc06809b7c0 con 0x7fc06815b370 2023-11-26T07:11:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.788+0000 7fc0657fa700 1 -- 172.21.15.83:0/3409091977 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc060017020 con 0x7fc06815b370 2023-11-26T07:11:52.791 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:11:52.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.790+0000 7fc06f0b4700 1 -- 172.21.15.83:0/3409091977 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc04c06aed0 msgr2=0x7fc04c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.790+0000 7fc06f0b4700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc04c06aed0 0x7fc04c06d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc05c0011e0 tx=0x7fc05c009040 comp rx=0 tx=0).stop 2023-11-26T07:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.790+0000 7fc06f0b4700 1 -- 172.21.15.83:0/3409091977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06815b370 msgr2=0x7fc068144770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.790+0000 7fc06f0b4700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06815b370 0x7fc068144770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc06000cac0 tx=0x7fc06000ce80 comp rx=0 tx=0).stop 2023-11-26T07:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.790+0000 7fc06f0b4700 1 -- 172.21.15.83:0/3409091977 shutdown_connections 2023-11-26T07:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.790+0000 7fc06f0b4700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06815b370 0x7fc068144770 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.790+0000 7fc06f0b4700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0680b3c60 0x7fc068155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.790+0000 7fc06f0b4700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc04c06aed0 0x7fc04c06d390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.790+0000 7fc06f0b4700 1 --2- 172.21.15.83:0/3409091977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0681587f0 0x7fc068144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.790+0000 7fc06f0b4700 1 -- 172.21.15.83:0/3409091977 >> 172.21.15.83:0/3409091977 conn(0x7fc0680d0bf0 msgr2=0x7fc06806b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.791+0000 7fc06f0b4700 1 -- 172.21.15.83:0/3409091977 shutdown_connections 2023-11-26T07:11:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.791+0000 7fc06f0b4700 1 -- 172.21.15.83:0/3409091977 wait complete. 2023-11-26T07:11:52.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:52.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.922+0000 7f63bda04700 1 Processor -- start 2023-11-26T07:11:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.922+0000 7f63bda04700 1 -- start start 2023-11-26T07:11:52.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63bda04700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f63b815e1e0 0x7f63b80d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63bda04700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f63b80d3a00 0x7f63b80d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63bda04700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b80d1200 con 0x7f63b80d0db0 2023-11-26T07:11:52.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63bda04700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b80d3150 con 0x7f63b815e1e0 2023-11-26T07:11:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63bda04700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b815e5c0 con 0x7f63b80d3a00 2023-11-26T07:11:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63b6ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f63b815e1e0 0x7f63b80d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63b77fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f63b80d0db0 0x7f63b806aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:54998/0 (socket says 172.21.15.83:54998) 2023-11-26T07:11:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63b77fe700 1 -- 172.21.15.83:0/3867965164 learned_addr learned my addr 172.21.15.83:0/3867965164 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63b67fc700 1 --2- 172.21.15.83:0/3867965164 >> v2:172.21.15.203:3300/0 conn(0x7f63b80d3a00 0x7f63b80d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63b6ffd700 1 -- 172.21.15.83:0/3867965164 >> v2:172.21.15.203:3300/0 conn(0x7f63b80d3a00 msgr2=0x7f63b80d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63b6ffd700 1 --2- 172.21.15.83:0/3867965164 >> v2:172.21.15.203:3300/0 conn(0x7f63b80d3a00 0x7f63b80d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63b6ffd700 1 -- 172.21.15.83:0/3867965164 >> v1:172.21.15.83:6789/0 conn(0x7f63b80d0db0 legacy=0x7f63b806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63b6ffd700 1 -- 172.21.15.83:0/3867965164 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b80d20b0 con 0x7f63b815e1e0 2023-11-26T07:11:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63b67fc700 1 --2- 172.21.15.83:0/3867965164 >> v2:172.21.15.203:3300/0 conn(0x7f63b80d3a00 0x7f63b80d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.923+0000 7f63b5ffb700 1 -- 172.21.15.83:0/3867965164 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096761220 0 0) 0x7f63b0001490 con 0x7f63b80d0db0 2023-11-26T07:11:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63b6ffd700 1 --2- 172.21.15.83:0/3867965164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b815e1e0 0x7f63b80d34c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f63b8161ba0 tx=0x7f63ac009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3855e0363a8f7057 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63b5ffb700 1 -- 172.21.15.83:0/3867965164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac011830 con 0x7f63b815e1e0 2023-11-26T07:11:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63b5ffb700 1 -- 172.21.15.83:0/3867965164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac011990 con 0x7f63b815e1e0 2023-11-26T07:11:52.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63b5ffb700 1 -- 172.21.15.83:0/3867965164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac011d20 con 0x7f63b815e1e0 2023-11-26T07:11:52.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63bda04700 1 -- 172.21.15.83:0/3867965164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b815e1e0 msgr2=0x7f63b80d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63bda04700 1 --2- 172.21.15.83:0/3867965164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b815e1e0 0x7f63b80d34c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f63b8161ba0 tx=0x7f63ac009b20 comp rx=0 tx=0).stop 2023-11-26T07:11:52.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63bda04700 1 -- 172.21.15.83:0/3867965164 shutdown_connections 2023-11-26T07:11:52.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63bda04700 1 --2- 172.21.15.83:0/3867965164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b815e1e0 0x7f63b80d34c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63bda04700 1 --2- 172.21.15.83:0/3867965164 >> v2:172.21.15.203:3300/0 conn(0x7f63b80d3a00 0x7f63b80d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63bda04700 1 -- 172.21.15.83:0/3867965164 >> 172.21.15.83:0/3867965164 conn(0x7f63b80b5b50 msgr2=0x7f63b80b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:52.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63bda04700 1 -- 172.21.15.83:0/3867965164 shutdown_connections 2023-11-26T07:11:52.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.924+0000 7f63bda04700 1 -- 172.21.15.83:0/3867965164 wait complete. 2023-11-26T07:11:52.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.926+0000 7f63bda04700 1 Processor -- start 2023-11-26T07:11:52.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.926+0000 7f63bda04700 1 -- start start 2023-11-26T07:11:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.926+0000 7f63bda04700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63b80d0db0 0x7f63b81594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.926+0000 7f63bda04700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b80d3a00 0x7f63b80cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.926+0000 7f63bda04700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63b815e1e0 0x7f63b81531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.926+0000 7f63bda04700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b81590c0 con 0x7f63b80d3a00 2023-11-26T07:11:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.926+0000 7f63bda04700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b8159a20 con 0x7f63b80d0db0 2023-11-26T07:11:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.926+0000 7f63bda04700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b81581a0 con 0x7f63b815e1e0 2023-11-26T07:11:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63b67fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b80d3a00 0x7f63b80cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63b67fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b80d3a00 0x7f63b80cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34962/0 (socket says 172.21.15.83:34962) 2023-11-26T07:11:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63b67fc700 1 -- 172.21.15.83:0/3541854896 learned_addr learned my addr 172.21.15.83:0/3541854896 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63b6ffd700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63b80d0db0 0x7f63b81594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63b77fe700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63b815e1e0 0x7f63b81531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63b67fc700 1 -- 172.21.15.83:0/3541854896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63b815e1e0 msgr2=0x7f63b81531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63b67fc700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63b815e1e0 0x7f63b81531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63b67fc700 1 -- 172.21.15.83:0/3541854896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63b80d0db0 msgr2=0x7f63b81594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63b67fc700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63b80d0db0 0x7f63b81594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63b67fc700 1 -- 172.21.15.83:0/3541854896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac0094a0 con 0x7f63b80d3a00 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63b67fc700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b80d3a00 0x7f63b80cd9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f63a800e7a0 tx=0x7f63a800eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f639ffff700 1 -- 172.21.15.83:0/3541854896 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8013610 con 0x7f63b80d3a00 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f639ffff700 1 -- 172.21.15.83:0/3541854896 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a8013770 con 0x7f63b80d3a00 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63bda04700 1 -- 172.21.15.83:0/3541854896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b80cdf10 con 0x7f63b80d3a00 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f639ffff700 1 -- 172.21.15.83:0/3541854896 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8013b00 con 0x7f63b80d3a00 2023-11-26T07:11:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.927+0000 7f63bda04700 1 -- 172.21.15.83:0/3541854896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b8158720 con 0x7f63b80d3a00 2023-11-26T07:11:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.928+0000 7f63bda04700 1 -- 172.21.15.83:0/3541854896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b80751e0 con 0x7f63b80d3a00 2023-11-26T07:11:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.929+0000 7f639ffff700 1 -- 172.21.15.83:0/3541854896 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f63a8013c60 con 0x7f63b80d3a00 2023-11-26T07:11:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.929+0000 7f639ffff700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f63a406aed0 0x7f63a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.929+0000 7f639ffff700 1 -- 172.21.15.83:0/3541854896 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63a8089120 con 0x7f63b80d3a00 2023-11-26T07:11:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.929+0000 7f63b6ffd700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f63a406aed0 0x7f63a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.929+0000 7f63b6ffd700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f63a406aed0 0x7f63a406d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0090b0 tx=0x7f63ac01c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:52.933+0000 7f639ffff700 1 -- 172.21.15.83:0/3541854896 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63a80541d0 con 0x7f63b80d3a00 2023-11-26T07:11:53.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.123+0000 7f63bda04700 1 -- 172.21.15.83:0/3541854896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63b8159200 con 0x7f63b80d3a00 2023-11-26T07:11:53.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.125+0000 7f639ffff700 1 -- 172.21.15.83:0/3541854896 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f63b8159200 con 0x7f63b80d3a00 2023-11-26T07:11:53.127 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:53.128 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 -- 172.21.15.83:0/3541854896 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f63a406aed0 msgr2=0x7f63a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f63a406aed0 0x7f63a406d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0090b0 tx=0x7f63ac01c040 comp rx=0 tx=0).stop 2023-11-26T07:11:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 -- 172.21.15.83:0/3541854896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b80d3a00 msgr2=0x7f63b80cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b80d3a00 0x7f63b80cd9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f63a800e7a0 tx=0x7f63a800eab0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 -- 172.21.15.83:0/3541854896 shutdown_connections 2023-11-26T07:11:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f63a406aed0 0x7f63a406d390 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63b815e1e0 0x7f63b81531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63b80d0db0 0x7f63b81594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 --2- 172.21.15.83:0/3541854896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b80d3a00 0x7f63b80cd9d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 -- 172.21.15.83:0/3541854896 >> 172.21.15.83:0/3541854896 conn(0x7f63b80b5b50 msgr2=0x7f63b80b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:53.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 -- 172.21.15.83:0/3541854896 shutdown_connections 2023-11-26T07:11:53.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.127+0000 7f63bda04700 1 -- 172.21.15.83:0/3541854896 wait complete. 2023-11-26T07:11:53.142 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:11:53.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:11:53.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.262+0000 7f5f877d8700 1 Processor -- start 2023-11-26T07:11:53.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.262+0000 7f5f877d8700 1 -- start start 2023-11-26T07:11:53.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.263+0000 7f5f877d8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5f800b5da0 0x7f5f800afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.263+0000 7f5f877d8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f800b3c60 con 0x7f5f800746a0 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.263+0000 7f5f877d8700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f800b58c0 con 0x7f5f8015e160 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.263+0000 7f5f877d8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f800b4500 con 0x7f5f800b5da0 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.263+0000 7f5f84d73700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5f800746a0 0x7f5f8006d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55010/0 (socket says 172.21.15.83:55010) 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.263+0000 7f5f84d73700 1 -- 172.21.15.83:0/680368831 learned_addr learned my addr 172.21.15.83:0/680368831 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.263+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143371054 0 0) 0x7f5f800b3c60 con 0x7f5f800746a0 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.263+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f6c0035b0 con 0x7f5f800746a0 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.263+0000 7f5f85d75700 1 --2- 172.21.15.83:0/680368831 >> v2:172.21.15.203:3300/0 conn(0x7f5f800b5da0 0x7f5f800afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3656498370 0 0) 0x7f5f6c0035b0 con 0x7f5f800746a0 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 >> v2:172.21.15.203:3300/0 conn(0x7f5f800b5da0 msgr2=0x7f5f800afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 --2- 172.21.15.83:0/680368831 >> v2:172.21.15.203:3300/0 conn(0x7f5f800b5da0 0x7f5f800afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 >> v1:172.21.15.203:6789/0 conn(0x7f5f8015e160 legacy=0x7f5f80073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8006bdc0 con 0x7f5f800746a0 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f5f7c001670 con 0x7f5f800746a0 2023-11-26T07:11:53.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 >> v1:172.21.15.83:6789/0 conn(0x7f5f800746a0 legacy=0x7f5f8006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f6c003a00 0x7f5f6c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f6c004850 0x7f5f6c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 0x7f5f6c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f800b4500 con 0x7f5f6c005210 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c00c0b0 con 0x7f5f6c003a00 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c00c270 con 0x7f5f6c004850 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f85d75700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 0x7f5f6c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f85574700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f6c003a00 0x7f5f6c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f84d73700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f6c004850 0x7f5f6c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f85d75700 1 -- 172.21.15.83:0/680368831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f6c004850 msgr2=0x7f5f6c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f85d75700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f6c004850 0x7f5f6c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f85d75700 1 -- 172.21.15.83:0/680368831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f6c003a00 msgr2=0x7f5f6c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f85d75700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f6c003a00 0x7f5f6c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f85d75700 1 -- 172.21.15.83:0/680368831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c0035b0 con 0x7f5f6c005210 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f84d73700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f6c004850 0x7f5f6c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f85d75700 1 -- 172.21.15.83:0/680368831 reap_dead start 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f85d75700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 0x7f5f6c00ba10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009fc0 tx=0x7f5f6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b983a2adffee25 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.264+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78003a20 con 0x7f5f6c005210 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.265+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7800f040 con 0x7f5f6c005210 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.265+0000 7f5f77fff700 1 -- 172.21.15.83:0/680368831 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78003de0 con 0x7f5f6c005210 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.265+0000 7f5f877d8700 1 -- 172.21.15.83:0/680368831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 msgr2=0x7f5f6c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.265+0000 7f5f877d8700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 0x7f5f6c00ba10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009fc0 tx=0x7f5f6c005880 comp rx=0 tx=0).stop 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.265+0000 7f5f877d8700 1 -- 172.21.15.83:0/680368831 shutdown_connections 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.265+0000 7f5f877d8700 1 --2- 172.21.15.83:0/680368831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 0x7f5f6c00ba10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.265+0000 7f5f877d8700 1 -- 172.21.15.83:0/680368831 >> 172.21.15.83:0/680368831 conn(0x7f5f800d0bf0 msgr2=0x7f5f800d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:53.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.265+0000 7f5f877d8700 1 -- 172.21.15.83:0/680368831 shutdown_connections 2023-11-26T07:11:53.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.265+0000 7f5f877d8700 1 -- 172.21.15.83:0/680368831 wait complete. 2023-11-26T07:11:53.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.266+0000 7f5f877d8700 1 Processor -- start 2023-11-26T07:11:53.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.266+0000 7f5f877d8700 1 -- start start 2023-11-26T07:11:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f877d8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 0x7f5f800cff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f877d8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f801e9c20 0x7f5f801f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f877d8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f801e6800 0x7f5f801f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f877d8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f801e9790 con 0x7f5f6c005210 2023-11-26T07:11:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f877d8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f801e8ea0 con 0x7f5f801e9c20 2023-11-26T07:11:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f877d8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f801e9140 con 0x7f5f801e6800 2023-11-26T07:11:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f85574700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 0x7f5f800cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f85574700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 0x7f5f800cff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34980/0 (socket says 172.21.15.83:34980) 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f85574700 1 -- 172.21.15.83:0/2631915547 learned_addr learned my addr 172.21.15.83:0/2631915547 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f85574700 1 -- 172.21.15.83:0/2631915547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f801e6800 msgr2=0x7f5f801f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f85574700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f801e6800 0x7f5f801f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f85574700 1 -- 172.21.15.83:0/2631915547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f801e9c20 msgr2=0x7f5f801f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f85574700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f801e9c20 0x7f5f801f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.267+0000 7f5f85574700 1 -- 172.21.15.83:0/2631915547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f801e9420 con 0x7f5f6c005210 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.268+0000 7f5f85574700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 0x7f5f800cff30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000f820 tx=0x7f5f7000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.268+0000 7f5f75ffb700 1 -- 172.21.15.83:0/2631915547 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f700052e0 con 0x7f5f6c005210 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.268+0000 7f5f75ffb700 1 -- 172.21.15.83:0/2631915547 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f70005460 con 0x7f5f6c005210 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.268+0000 7f5f75ffb700 1 -- 172.21.15.83:0/2631915547 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f700038c0 con 0x7f5f6c005210 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.268+0000 7f5f877d8700 1 -- 172.21.15.83:0/2631915547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f800d0470 con 0x7f5f6c005210 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.268+0000 7f5f877d8700 1 -- 172.21.15.83:0/2631915547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f801f8000 con 0x7f5f6c005210 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.269+0000 7f5f877d8700 1 -- 172.21.15.83:0/2631915547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f80068fb0 con 0x7f5f6c005210 2023-11-26T07:11:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.269+0000 7f5f75ffb700 1 -- 172.21.15.83:0/2631915547 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f70003a20 con 0x7f5f6c005210 2023-11-26T07:11:53.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.269+0000 7f5f75ffb700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f6406aed0 0x7f5f6406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.270+0000 7f5f75ffb700 1 -- 172.21.15.83:0/2631915547 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f70089390 con 0x7f5f6c005210 2023-11-26T07:11:53.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.270+0000 7f5f84d73700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f6406aed0 0x7f5f6406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.270+0000 7f5f84d73700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f6406aed0 0x7f5f6406d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c001b40 tx=0x7f5f7c001110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:53.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.274+0000 7f5f75ffb700 1 -- 172.21.15.83:0/2631915547 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f70054440 con 0x7f5f6c005210 2023-11-26T07:11:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.462+0000 7f5f877d8700 1 -- 172.21.15.83:0/2631915547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f8006c140 con 0x7f5f6c005210 2023-11-26T07:11:53.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.463+0000 7f5f75ffb700 1 -- 172.21.15.83:0/2631915547 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f70057a60 con 0x7f5f6c005210 2023-11-26T07:11:53.465 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:11:53.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 -- 172.21.15.83:0/2631915547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f6406aed0 msgr2=0x7f5f6406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f6406aed0 0x7f5f6406d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c001b40 tx=0x7f5f7c001110 comp rx=0 tx=0).stop 2023-11-26T07:11:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 -- 172.21.15.83:0/2631915547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 msgr2=0x7f5f800cff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 0x7f5f800cff30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000f820 tx=0x7f5f7000fbe0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 -- 172.21.15.83:0/2631915547 shutdown_connections 2023-11-26T07:11:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5f6406aed0 0x7f5f6406d390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f801e6800 0x7f5f801f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f6c005210 0x7f5f800cff30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 --2- 172.21.15.83:0/2631915547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f801e9c20 0x7f5f801f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 -- 172.21.15.83:0/2631915547 >> 172.21.15.83:0/2631915547 conn(0x7f5f800d0bf0 msgr2=0x7f5f800d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 -- 172.21.15.83:0/2631915547 shutdown_connections 2023-11-26T07:11:53.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.465+0000 7f5f877d8700 1 -- 172.21.15.83:0/2631915547 wait complete. 2023-11-26T07:11:53.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:11:53.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.599+0000 7f4b3c150700 1 Processor -- start 2023-11-26T07:11:53.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.599+0000 7f4b3c150700 1 -- start start 2023-11-26T07:11:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b3c150700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4b340aedc0 0x7f4b340ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b3c150700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b3415e290 0x7f4b340af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b3c150700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3406e400 con 0x7f4b340ad870 2023-11-26T07:11:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b3c150700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3406fde0 con 0x7f4b340aedc0 2023-11-26T07:11:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b3c150700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340af6e0 con 0x7f4b3415e290 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b39eec700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4b340aedc0 0x7f4b340ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b396eb700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b3415e290 0x7f4b340af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b3a6ed700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4b340ad870 0x7f4b3406f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:39650/0 (socket says 172.21.15.83:39650) 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b3a6ed700 1 -- 172.21.15.83:0/799585298 learned_addr learned my addr 172.21.15.83:0/799585298 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b39eec700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4b340aedc0 0x7f4b340ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34440/0 (socket says 172.21.15.83:34440) 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b396eb700 1 -- 172.21.15.83:0/799585298 >> v2:172.21.15.203:3300/0 conn(0x7f4b340aedc0 msgr2=0x7f4b340ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b38eea700 1 -- 172.21.15.83:0/799585298 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289739212 0 0) 0x7f4b3406e400 con 0x7f4b340ad870 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b396eb700 1 --2- 172.21.15.83:0/799585298 >> v2:172.21.15.203:3300/0 conn(0x7f4b340aedc0 0x7f4b340ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.600+0000 7f4b396eb700 1 -- 172.21.15.83:0/799585298 >> v1:172.21.15.203:6789/0 conn(0x7f4b340ad870 legacy=0x7f4b3406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.601+0000 7f4b396eb700 1 -- 172.21.15.83:0/799585298 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3406e630 con 0x7f4b3415e290 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.601+0000 7f4b396eb700 1 --2- 172.21.15.83:0/799585298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b3415e290 0x7f4b340af1a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009840 tx=0x7f4b24009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb7217c6112c5ac server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:53.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.601+0000 7f4b38eea700 1 -- 172.21.15.83:0/799585298 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24011830 con 0x7f4b3415e290 2023-11-26T07:11:53.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.601+0000 7f4b38eea700 1 -- 172.21.15.83:0/799585298 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b24011990 con 0x7f4b3415e290 2023-11-26T07:11:53.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.601+0000 7f4b38eea700 1 -- 172.21.15.83:0/799585298 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24011d20 con 0x7f4b3415e290 2023-11-26T07:11:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.601+0000 7f4b3c150700 1 -- 172.21.15.83:0/799585298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b3415e290 msgr2=0x7f4b340af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.601+0000 7f4b3c150700 1 --2- 172.21.15.83:0/799585298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b3415e290 0x7f4b340af1a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009840 tx=0x7f4b24009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.601+0000 7f4b3c150700 1 -- 172.21.15.83:0/799585298 shutdown_connections 2023-11-26T07:11:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.601+0000 7f4b3c150700 1 --2- 172.21.15.83:0/799585298 >> v2:172.21.15.203:3300/0 conn(0x7f4b340aedc0 0x7f4b340ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.601+0000 7f4b3c150700 1 --2- 172.21.15.83:0/799585298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b3415e290 0x7f4b340af1a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.601+0000 7f4b3c150700 1 -- 172.21.15.83:0/799585298 >> 172.21.15.83:0/799585298 conn(0x7f4b340b2050 msgr2=0x7f4b340b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.602+0000 7f4b3c150700 1 -- 172.21.15.83:0/799585298 shutdown_connections 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.602+0000 7f4b3c150700 1 -- 172.21.15.83:0/799585298 wait complete. 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.603+0000 7f4b3c150700 1 Processor -- start 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.603+0000 7f4b3c150700 1 -- start start 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.603+0000 7f4b3c150700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b340ad870 0x7f4b3414f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.603+0000 7f4b3c150700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b340aedc0 0x7f4b3415f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.603+0000 7f4b3c150700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b3415e290 0x7f4b3415f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.603+0000 7f4b3c150700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b341f5320 con 0x7f4b340aedc0 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.603+0000 7f4b3c150700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3414f540 con 0x7f4b3415e290 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.603+0000 7f4b3c150700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b341f4910 con 0x7f4b340ad870 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b396eb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b340aedc0 0x7f4b3415f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b396eb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b340aedc0 0x7f4b3415f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34996/0 (socket says 172.21.15.83:34996) 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b396eb700 1 -- 172.21.15.83:0/2494621033 learned_addr learned my addr 172.21.15.83:0/2494621033 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b39eec700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b340ad870 0x7f4b3414f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b3a6ed700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b3415e290 0x7f4b3415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b396eb700 1 -- 172.21.15.83:0/2494621033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b340ad870 msgr2=0x7f4b3414f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b396eb700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b340ad870 0x7f4b3414f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b396eb700 1 -- 172.21.15.83:0/2494621033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b3415e290 msgr2=0x7f4b3415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b396eb700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b3415e290 0x7f4b3415f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b396eb700 1 -- 172.21.15.83:0/2494621033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b240094a0 con 0x7f4b340aedc0 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b396eb700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b340aedc0 0x7f4b3415f020 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4b240090b0 tx=0x7f4b24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b2a7fc700 1 -- 172.21.15.83:0/2494621033 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24004550 con 0x7f4b340aedc0 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b2a7fc700 1 -- 172.21.15.83:0/2494621033 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2401f070 con 0x7f4b340aedc0 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b3c150700 1 -- 172.21.15.83:0/2494621033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b341f4630 con 0x7f4b340aedc0 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b2a7fc700 1 -- 172.21.15.83:0/2494621033 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b240038c0 con 0x7f4b340aedc0 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.604+0000 7f4b3c150700 1 -- 172.21.15.83:0/2494621033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b341f4f40 con 0x7f4b340aedc0 2023-11-26T07:11:53.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.605+0000 7f4b3c150700 1 -- 172.21.15.83:0/2494621033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b34068d90 con 0x7f4b340aedc0 2023-11-26T07:11:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.606+0000 7f4b2a7fc700 1 -- 172.21.15.83:0/2494621033 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b24003a20 con 0x7f4b340aedc0 2023-11-26T07:11:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.606+0000 7f4b2a7fc700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b1806aed0 0x7f4b1806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.606+0000 7f4b2a7fc700 1 -- 172.21.15.83:0/2494621033 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b24021070 con 0x7f4b340aedc0 2023-11-26T07:11:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.606+0000 7f4b39eec700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b1806aed0 0x7f4b1806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.606+0000 7f4b39eec700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b1806aed0 0x7f4b1806d390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b34159410 tx=0x7f4b30009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.610+0000 7f4b2a7fc700 1 -- 172.21.15.83:0/2494621033 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b24055ec0 con 0x7f4b340aedc0 2023-11-26T07:11:53.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.799+0000 7f4b3c150700 1 -- 172.21.15.83:0/2494621033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b341f4ac0 con 0x7f4b340aedc0 2023-11-26T07:11:53.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.800+0000 7f4b2a7fc700 1 -- 172.21.15.83:0/2494621033 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b2402b020 con 0x7f4b340aedc0 2023-11-26T07:11:53.802 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:11:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.801+0000 7f4b3c150700 1 -- 172.21.15.83:0/2494621033 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b1806aed0 msgr2=0x7f4b1806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.802+0000 7f4b3c150700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b1806aed0 0x7f4b1806d390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b34159410 tx=0x7f4b30009210 comp rx=0 tx=0).stop 2023-11-26T07:11:53.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.802+0000 7f4b3c150700 1 -- 172.21.15.83:0/2494621033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b340aedc0 msgr2=0x7f4b3415f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.802+0000 7f4b3c150700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b340aedc0 0x7f4b3415f020 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4b240090b0 tx=0x7f4b24004750 comp rx=0 tx=0).stop 2023-11-26T07:11:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.802+0000 7f4b3c150700 1 -- 172.21.15.83:0/2494621033 shutdown_connections 2023-11-26T07:11:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.802+0000 7f4b3c150700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b3415e290 0x7f4b3415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.802+0000 7f4b3c150700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b1806aed0 0x7f4b1806d390 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.802+0000 7f4b3c150700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b340ad870 0x7f4b3414f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.802+0000 7f4b3c150700 1 --2- 172.21.15.83:0/2494621033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b340aedc0 0x7f4b3415f020 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.802+0000 7f4b3c150700 1 -- 172.21.15.83:0/2494621033 >> 172.21.15.83:0/2494621033 conn(0x7f4b340b2050 msgr2=0x7f4b34159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.802+0000 7f4b3c150700 1 -- 172.21.15.83:0/2494621033 shutdown_connections 2023-11-26T07:11:53.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.802+0000 7f4b3c150700 1 -- 172.21.15.83:0/2494621033 wait complete. 2023-11-26T07:11:53.816 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:11:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.939+0000 7f534de22700 1 Processor -- start 2023-11-26T07:11:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.939+0000 7f534de22700 1 -- start start 2023-11-26T07:11:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.940+0000 7f534de22700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f534815e160 0x7f53480b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.940+0000 7f534de22700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f53480afc90 0x7f5348073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.940+0000 7f534de22700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53480b3c60 con 0x7f53480744d0 2023-11-26T07:11:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.940+0000 7f534de22700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53480b4500 con 0x7f53480afc90 2023-11-26T07:11:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.940+0000 7f534de22700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5348074940 con 0x7f534815e160 2023-11-26T07:11:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.940+0000 7f5346ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f53480afc90 0x7f5348073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.940+0000 7f5347fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f53480744d0 0x7f534806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55046/0 (socket says 172.21.15.83:55046) 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.940+0000 7f5347fff700 1 -- 172.21.15.83:0/1632087535 learned_addr learned my addr 172.21.15.83:0/1632087535 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f5346ffd700 1 -- 172.21.15.83:0/1632087535 >> v2:172.21.15.203:3300/0 conn(0x7f534815e160 msgr2=0x7f53480b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f53477fe700 1 --2- 172.21.15.83:0/1632087535 >> v2:172.21.15.203:3300/0 conn(0x7f534815e160 0x7f53480b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f5346ffd700 1 --2- 172.21.15.83:0/1632087535 >> v2:172.21.15.203:3300/0 conn(0x7f534815e160 0x7f53480b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f53467fc700 1 -- 172.21.15.83:0/1632087535 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997456889 0 0) 0x7f53480b3c60 con 0x7f53480744d0 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f5346ffd700 1 -- 172.21.15.83:0/1632087535 >> v1:172.21.15.83:6789/0 conn(0x7f53480744d0 legacy=0x7f534806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f5346ffd700 1 -- 172.21.15.83:0/1632087535 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53480b58c0 con 0x7f53480afc90 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f5346ffd700 1 --2- 172.21.15.83:0/1632087535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53480afc90 0x7f5348073f90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f534809efa0 tx=0x7f5338009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9b41ca4b8c0515 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f53467fc700 1 -- 172.21.15.83:0/1632087535 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5338011830 con 0x7f53480afc90 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f53467fc700 1 -- 172.21.15.83:0/1632087535 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5338011990 con 0x7f53480afc90 2023-11-26T07:11:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f53467fc700 1 -- 172.21.15.83:0/1632087535 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5338011d20 con 0x7f53480afc90 2023-11-26T07:11:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f534de22700 1 -- 172.21.15.83:0/1632087535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53480afc90 msgr2=0x7f5348073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f534de22700 1 --2- 172.21.15.83:0/1632087535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53480afc90 0x7f5348073f90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f534809efa0 tx=0x7f5338009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f534de22700 1 -- 172.21.15.83:0/1632087535 shutdown_connections 2023-11-26T07:11:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f534de22700 1 --2- 172.21.15.83:0/1632087535 >> v2:172.21.15.203:3300/0 conn(0x7f534815e160 0x7f53480b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f534de22700 1 --2- 172.21.15.83:0/1632087535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53480afc90 0x7f5348073f90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.941+0000 7f534de22700 1 -- 172.21.15.83:0/1632087535 >> 172.21.15.83:0/1632087535 conn(0x7f53480d0bf0 msgr2=0x7f53480d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.942+0000 7f534de22700 1 -- 172.21.15.83:0/1632087535 shutdown_connections 2023-11-26T07:11:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.942+0000 7f534de22700 1 -- 172.21.15.83:0/1632087535 wait complete. 2023-11-26T07:11:53.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.943+0000 7f534de22700 1 Processor -- start 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.943+0000 7f534de22700 1 -- start start 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f534de22700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53480744d0 0x7f53481f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f534de22700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53480afc90 0x7f53480add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f534de22700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534815e160 0x7f53480ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f534de22700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534814d3f0 con 0x7f53480afc90 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f534de22700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534814c520 con 0x7f534815e160 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f534de22700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534814c770 con 0x7f53480744d0 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f5346ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53480afc90 0x7f53480add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f5346ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53480afc90 0x7f53480add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35008/0 (socket says 172.21.15.83:35008) 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f5346ffd700 1 -- 172.21.15.83:0/334359044 learned_addr learned my addr 172.21.15.83:0/334359044 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f5347fff700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534815e160 0x7f53480ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f53477fe700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53480744d0 0x7f53481f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f5346ffd700 1 -- 172.21.15.83:0/334359044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53480744d0 msgr2=0x7f53481f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.944+0000 7f5346ffd700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53480744d0 0x7f53481f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.945+0000 7f5346ffd700 1 -- 172.21.15.83:0/334359044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534815e160 msgr2=0x7f53480ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.945+0000 7f5346ffd700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534815e160 0x7f53480ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.945+0000 7f5346ffd700 1 -- 172.21.15.83:0/334359044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53380094a0 con 0x7f53480afc90 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.945+0000 7f53477fe700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53480744d0 0x7f53481f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.945+0000 7f5346ffd700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53480afc90 0x7f53480add40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f53380097c0 tx=0x7f5338004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.945+0000 7f532ffff700 1 -- 172.21.15.83:0/334359044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53380045f0 con 0x7f53480afc90 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.945+0000 7f532ffff700 1 -- 172.21.15.83:0/334359044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533801f070 con 0x7f53480afc90 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.945+0000 7f534de22700 1 -- 172.21.15.83:0/334359044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534814ca50 con 0x7f53480afc90 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.945+0000 7f532ffff700 1 -- 172.21.15.83:0/334359044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5338003b00 con 0x7f53480afc90 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.945+0000 7f534de22700 1 -- 172.21.15.83:0/334359044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534814d010 con 0x7f53480afc90 2023-11-26T07:11:53.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.946+0000 7f534de22700 1 -- 172.21.15.83:0/334359044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5348068fb0 con 0x7f53480afc90 2023-11-26T07:11:53.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.946+0000 7f532ffff700 1 -- 172.21.15.83:0/334359044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5338003c60 con 0x7f53480afc90 2023-11-26T07:11:53.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.947+0000 7f532ffff700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f533406aed0 0x7f533406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:53.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.947+0000 7f532ffff700 1 -- 172.21.15.83:0/334359044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5338021070 con 0x7f53480afc90 2023-11-26T07:11:53.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.947+0000 7f53477fe700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f533406aed0 0x7f533406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:53.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.947+0000 7f53477fe700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f533406aed0 0x7f533406d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f53480aea50 tx=0x7f533c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:53.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:53.951+0000 7f532ffff700 1 -- 172.21.15.83:0/334359044 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5338056140 con 0x7f53480afc90 2023-11-26T07:11:54.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.138+0000 7f534de22700 1 -- 172.21.15.83:0/334359044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f534806c140 con 0x7f53480afc90 2023-11-26T07:11:54.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.138+0000 7f532ffff700 1 -- 172.21.15.83:0/334359044 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f533802b0a0 con 0x7f53480afc90 2023-11-26T07:11:54.141 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:11:54.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 -- 172.21.15.83:0/334359044 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f533406aed0 msgr2=0x7f533406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f533406aed0 0x7f533406d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f53480aea50 tx=0x7f533c008040 comp rx=0 tx=0).stop 2023-11-26T07:11:54.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 -- 172.21.15.83:0/334359044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53480afc90 msgr2=0x7f53480add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53480afc90 0x7f53480add40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f53380097c0 tx=0x7f5338004750 comp rx=0 tx=0).stop 2023-11-26T07:11:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 -- 172.21.15.83:0/334359044 shutdown_connections 2023-11-26T07:11:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f534815e160 0x7f53480ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f533406aed0 0x7f533406d390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53480744d0 0x7f53481f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 --2- 172.21.15.83:0/334359044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53480afc90 0x7f53480add40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 -- 172.21.15.83:0/334359044 >> 172.21.15.83:0/334359044 conn(0x7f53480d0bf0 msgr2=0x7f534815d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 -- 172.21.15.83:0/334359044 shutdown_connections 2023-11-26T07:11:54.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.141+0000 7f534de22700 1 -- 172.21.15.83:0/334359044 wait complete. 2023-11-26T07:11:54.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:54.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.273+0000 7fe31f40d700 1 Processor -- start 2023-11-26T07:11:54.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.273+0000 7fe31f40d700 1 -- start start 2023-11-26T07:11:54.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31f40d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3180adf00 0x7fe31806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31f40d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31806c540 0x7fe3180b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31f40d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3180af950 con 0x7fe3181446a0 2023-11-26T07:11:54.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31f40d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe318144a80 con 0x7fe3180adf00 2023-11-26T07:11:54.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31f40d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31806c9c0 con 0x7fe31806c540 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31d9aa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31806c540 0x7fe3180b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31c9a8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3180adf00 0x7fe31806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31d9aa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31806c540 0x7fe3180b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:45478/0 (socket says 172.21.15.83:45478) 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31c9a8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe3180adf00 0x7fe31806c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34470/0 (socket says 172.21.15.83:34470) 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31d9aa700 1 -- 172.21.15.83:0/4240921717 learned_addr learned my addr 172.21.15.83:0/4240921717 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31d1a9700 1 --1- 172.21.15.83:0/4240921717 >> v1:172.21.15.203:6789/0 conn(0x7fe3181446a0 0x7fe3180b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:39674/0 (socket says 172.21.15.83:39674) 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31c9a8700 1 -- 172.21.15.83:0/4240921717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31806c540 msgr2=0x7fe3180b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31c9a8700 1 --2- 172.21.15.83:0/4240921717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31806c540 0x7fe3180b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31c9a8700 1 -- 172.21.15.83:0/4240921717 >> v1:172.21.15.203:6789/0 conn(0x7fe3181446a0 legacy=0x7fe3180b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe30ffff700 1 -- 172.21.15.83:0/4240921717 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176042570 0 0) 0x7fe3180af950 con 0x7fe3181446a0 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31c9a8700 1 -- 172.21.15.83:0/4240921717 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3180afc30 con 0x7fe3180adf00 2023-11-26T07:11:54.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.274+0000 7fe31d9aa700 1 --2- 172.21.15.83:0/4240921717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31806c540 0x7fe3180b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:54.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe31c9a8700 1 --2- 172.21.15.83:0/4240921717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3180adf00 0x7fe31806c000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe308009840 tx=0x7fe308009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2948c069a97e48a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:54.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe30ffff700 1 -- 172.21.15.83:0/4240921717 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308011830 con 0x7fe3180adf00 2023-11-26T07:11:54.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe30ffff700 1 -- 172.21.15.83:0/4240921717 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe308011990 con 0x7fe3180adf00 2023-11-26T07:11:54.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe30ffff700 1 -- 172.21.15.83:0/4240921717 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308011d20 con 0x7fe3180adf00 2023-11-26T07:11:54.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe31f40d700 1 -- 172.21.15.83:0/4240921717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3180adf00 msgr2=0x7fe31806c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe31f40d700 1 --2- 172.21.15.83:0/4240921717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3180adf00 0x7fe31806c000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe308009840 tx=0x7fe308009b50 comp rx=0 tx=0).stop 2023-11-26T07:11:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe31f40d700 1 -- 172.21.15.83:0/4240921717 shutdown_connections 2023-11-26T07:11:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe31f40d700 1 --2- 172.21.15.83:0/4240921717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe31806c540 0x7fe3180b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe31f40d700 1 --2- 172.21.15.83:0/4240921717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3180adf00 0x7fe31806c000 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe31f40d700 1 -- 172.21.15.83:0/4240921717 >> 172.21.15.83:0/4240921717 conn(0x7fe318071b90 msgr2=0x7fe318071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe31f40d700 1 -- 172.21.15.83:0/4240921717 shutdown_connections 2023-11-26T07:11:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.275+0000 7fe31f40d700 1 -- 172.21.15.83:0/4240921717 wait complete. 2023-11-26T07:11:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.276+0000 7fe31f40d700 1 Processor -- start 2023-11-26T07:11:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.277+0000 7fe31f40d700 1 -- start start 2023-11-26T07:11:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.277+0000 7fe31f40d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31806c540 0x7fe31813a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.277+0000 7fe31f40d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3180adf00 0x7fe3181569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.277+0000 7fe31f40d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3181446a0 0x7fe3180b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.277+0000 7fe31f40d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180d0480 con 0x7fe3181446a0 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.277+0000 7fe31f40d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31813ab90 con 0x7fe31806c540 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.277+0000 7fe31f40d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180cd970 con 0x7fe3180adf00 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.277+0000 7fe31d9aa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3181446a0 0x7fe3180b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.277+0000 7fe31d9aa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3181446a0 0x7fe3180b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35026/0 (socket says 172.21.15.83:35026) 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.277+0000 7fe31d9aa700 1 -- 172.21.15.83:0/714802701 learned_addr learned my addr 172.21.15.83:0/714802701 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe31d1a9700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31806c540 0x7fe31813a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe31c9a8700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3180adf00 0x7fe3181569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe31d9aa700 1 -- 172.21.15.83:0/714802701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3180adf00 msgr2=0x7fe3181569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe31d9aa700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3180adf00 0x7fe3181569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe31d9aa700 1 -- 172.21.15.83:0/714802701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31806c540 msgr2=0x7fe31813a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe31d9aa700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31806c540 0x7fe31813a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe31d9aa700 1 -- 172.21.15.83:0/714802701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe310009b30 con 0x7fe3181446a0 2023-11-26T07:11:54.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe31c9a8700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3180adf00 0x7fe3181569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:54.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe31d9aa700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3181446a0 0x7fe3180b3d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe31000efc0 tx=0x7fe310009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:54.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe30dffb700 1 -- 172.21.15.83:0/714802701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310018070 con 0x7fe3181446a0 2023-11-26T07:11:54.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe30dffb700 1 -- 172.21.15.83:0/714802701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31000f040 con 0x7fe3181446a0 2023-11-26T07:11:54.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe30dffb700 1 -- 172.21.15.83:0/714802701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310003c40 con 0x7fe3181446a0 2023-11-26T07:11:54.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe31f40d700 1 -- 172.21.15.83:0/714802701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3080094a0 con 0x7fe3181446a0 2023-11-26T07:11:54.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.278+0000 7fe31f40d700 1 -- 172.21.15.83:0/714802701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3180cfdc0 con 0x7fe3181446a0 2023-11-26T07:11:54.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.279+0000 7fe31f40d700 1 -- 172.21.15.83:0/714802701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3180753f0 con 0x7fe3181446a0 2023-11-26T07:11:54.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.279+0000 7fe30dffb700 1 -- 172.21.15.83:0/714802701 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe31000c9e0 con 0x7fe3181446a0 2023-11-26T07:11:54.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.280+0000 7fe30dffb700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe2fc06aed0 0x7fe2fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.280+0000 7fe30dffb700 1 -- 172.21.15.83:0/714802701 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe310089880 con 0x7fe3181446a0 2023-11-26T07:11:54.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.280+0000 7fe31d1a9700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe2fc06aed0 0x7fe2fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.280+0000 7fe31d1a9700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe2fc06aed0 0x7fe2fc06d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe314000f60 tx=0x7fe314007c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.284+0000 7fe30dffb700 1 -- 172.21.15.83:0/714802701 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3100549c0 con 0x7fe3181446a0 2023-11-26T07:11:54.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.475+0000 7fe31f40d700 1 -- 172.21.15.83:0/714802701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe318067f60 con 0x7fe3181446a0 2023-11-26T07:11:54.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.476+0000 7fe30dffb700 1 -- 172.21.15.83:0/714802701 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7fe310057fe0 con 0x7fe3181446a0 2023-11-26T07:11:54.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:54.479 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 -- 172.21.15.83:0/714802701 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe2fc06aed0 msgr2=0x7fe2fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe2fc06aed0 0x7fe2fc06d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe314000f60 tx=0x7fe314007c90 comp rx=0 tx=0).stop 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 -- 172.21.15.83:0/714802701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3181446a0 msgr2=0x7fe3180b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3181446a0 0x7fe3180b3d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe31000efc0 tx=0x7fe310009f90 comp rx=0 tx=0).stop 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 -- 172.21.15.83:0/714802701 shutdown_connections 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3181446a0 0x7fe3180b3d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31806c540 0x7fe31813a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe2fc06aed0 0x7fe2fc06d390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 --2- 172.21.15.83:0/714802701 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3180adf00 0x7fe3181569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 -- 172.21.15.83:0/714802701 >> 172.21.15.83:0/714802701 conn(0x7fe318071b90 msgr2=0x7fe31815f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 -- 172.21.15.83:0/714802701 shutdown_connections 2023-11-26T07:11:54.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.478+0000 7fe31f40d700 1 -- 172.21.15.83:0/714802701 wait complete. 2023-11-26T07:11:54.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:11:54.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:11:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.615+0000 7f7beccb1700 1 Processor -- start 2023-11-26T07:11:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.615+0000 7f7beccb1700 1 -- start start 2023-11-26T07:11:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.616+0000 7f7beccb1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7be81587f0 0x7f7be806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.616+0000 7f7beccb1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7be815b370 0x7f7be815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.616+0000 7f7beccb1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7be80b3c60 0x7f7be80b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.616+0000 7f7beccb1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be80b4610 con 0x7f7be80b3c60 2023-11-26T07:11:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.616+0000 7f7beccb1700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be806b530 con 0x7f7be815b370 2023-11-26T07:11:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.616+0000 7f7beccb1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be80ae5e0 con 0x7f7be81587f0 2023-11-26T07:11:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.616+0000 7f7be6d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7be80b3c60 0x7f7be80b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.616+0000 7f7be6d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7be80b3c60 0x7f7be80b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35034/0 (socket says 172.21.15.83:35034) 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.616+0000 7f7be6d9d700 1 -- 172.21.15.83:0/193078486 learned_addr learned my addr 172.21.15.83:0/193078486 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be5d9b700 1 --2- 172.21.15.83:0/193078486 >> v2:172.21.15.203:3300/0 conn(0x7f7be815b370 0x7f7be815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be659c700 1 --2- 172.21.15.83:0/193078486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7be81587f0 0x7f7be806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be6d9d700 1 -- 172.21.15.83:0/193078486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7be81587f0 msgr2=0x7f7be806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be6d9d700 1 --2- 172.21.15.83:0/193078486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7be81587f0 0x7f7be806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be6d9d700 1 -- 172.21.15.83:0/193078486 >> v2:172.21.15.203:3300/0 conn(0x7f7be815b370 msgr2=0x7f7be815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be6d9d700 1 --2- 172.21.15.83:0/193078486 >> v2:172.21.15.203:3300/0 conn(0x7f7be815b370 0x7f7be815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be6d9d700 1 -- 172.21.15.83:0/193078486 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be815be30 con 0x7f7be80b3c60 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be5d9b700 1 --2- 172.21.15.83:0/193078486 >> v2:172.21.15.203:3300/0 conn(0x7f7be815b370 0x7f7be815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be659c700 1 --2- 172.21.15.83:0/193078486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7be81587f0 0x7f7be806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be6d9d700 1 --2- 172.21.15.83:0/193078486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7be80b3c60 0x7f7be80b40d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7be8072e10 tx=0x7f7be000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d83a6a5c6563272 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be559a700 1 -- 172.21.15.83:0/193078486 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be000e070 con 0x7f7be80b3c60 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be559a700 1 -- 172.21.15.83:0/193078486 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be000b9c0 con 0x7f7be80b3c60 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7be559a700 1 -- 172.21.15.83:0/193078486 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be000bd70 con 0x7f7be80b3c60 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7beccb1700 1 -- 172.21.15.83:0/193078486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7be80b3c60 msgr2=0x7f7be80b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.617+0000 7f7beccb1700 1 --2- 172.21.15.83:0/193078486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7be80b3c60 0x7f7be80b40d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7be8072e10 tx=0x7f7be000b330 comp rx=0 tx=0).stop 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.618+0000 7f7beccb1700 1 -- 172.21.15.83:0/193078486 shutdown_connections 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.618+0000 7f7beccb1700 1 --2- 172.21.15.83:0/193078486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7be80b3c60 0x7f7be80b40d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.618+0000 7f7beccb1700 1 --2- 172.21.15.83:0/193078486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7be81587f0 0x7f7be806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.618+0000 7f7beccb1700 1 --2- 172.21.15.83:0/193078486 >> v2:172.21.15.203:3300/0 conn(0x7f7be815b370 0x7f7be815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.618+0000 7f7beccb1700 1 -- 172.21.15.83:0/193078486 >> 172.21.15.83:0/193078486 conn(0x7f7be80d0bf0 msgr2=0x7f7be80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.618+0000 7f7beccb1700 1 -- 172.21.15.83:0/193078486 shutdown_connections 2023-11-26T07:11:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.618+0000 7f7beccb1700 1 -- 172.21.15.83:0/193078486 wait complete. 2023-11-26T07:11:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.619+0000 7f7beccb1700 1 Processor -- start 2023-11-26T07:11:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.619+0000 7f7beccb1700 1 -- start start 2023-11-26T07:11:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7beccb1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7be80b3c60 0x7f7be81f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7beccb1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7be81587f0 0x7f7be81548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7beccb1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7be815b370 0x7f7be8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7beccb1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be809e500 con 0x7f7be80b3c60 2023-11-26T07:11:54.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7beccb1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be81f1c40 con 0x7f7be815b370 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7beccb1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be81f1ee0 con 0x7f7be81587f0 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7be659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7be80b3c60 0x7f7be81f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7be659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7be80b3c60 0x7f7be81f1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35036/0 (socket says 172.21.15.83:35036) 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7be659c700 1 -- 172.21.15.83:0/4010111954 learned_addr learned my addr 172.21.15.83:0/4010111954 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7be6d9d700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7be815b370 0x7f7be8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7be5d9b700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7be81587f0 0x7f7be81548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7be659c700 1 -- 172.21.15.83:0/4010111954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7be81587f0 msgr2=0x7f7be81548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7be659c700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7be81587f0 0x7f7be81548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7be659c700 1 -- 172.21.15.83:0/4010111954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7be815b370 msgr2=0x7f7be8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7be659c700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7be815b370 0x7f7be8073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7be659c700 1 -- 172.21.15.83:0/4010111954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be0009b30 con 0x7f7be80b3c60 2023-11-26T07:11:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7be659c700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7be80b3c60 0x7f7be81f1700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c750 tx=0x7f7bdc00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7bd6ffd700 1 -- 172.21.15.83:0/4010111954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc011800 con 0x7f7be80b3c60 2023-11-26T07:11:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7bd6ffd700 1 -- 172.21.15.83:0/4010111954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bdc011960 con 0x7f7be80b3c60 2023-11-26T07:11:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7beccb1700 1 -- 172.21.15.83:0/4010111954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be81f21c0 con 0x7f7be80b3c60 2023-11-26T07:11:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7bd6ffd700 1 -- 172.21.15.83:0/4010111954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc011cd0 con 0x7f7be80b3c60 2023-11-26T07:11:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.620+0000 7f7beccb1700 1 -- 172.21.15.83:0/4010111954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be81f2a80 con 0x7f7be80b3c60 2023-11-26T07:11:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.621+0000 7f7beccb1700 1 -- 172.21.15.83:0/4010111954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be81f26e0 con 0x7f7be80b3c60 2023-11-26T07:11:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.622+0000 7f7bd6ffd700 1 -- 172.21.15.83:0/4010111954 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bdc021350 con 0x7f7be80b3c60 2023-11-26T07:11:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.622+0000 7f7bd6ffd700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7bc806aed0 0x7f7bc806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.622+0000 7f7bd6ffd700 1 -- 172.21.15.83:0/4010111954 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bdc089410 con 0x7f7be80b3c60 2023-11-26T07:11:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.622+0000 7f7be5d9b700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7bc806aed0 0x7f7bc806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.622+0000 7f7be5d9b700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7bc806aed0 0x7f7bc806d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7be815ddb0 tx=0x7f7bd8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:54.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.626+0000 7f7bd6ffd700 1 -- 172.21.15.83:0/4010111954 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bdc0544c0 con 0x7f7be80b3c60 2023-11-26T07:11:54.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.818+0000 7f7beccb1700 1 -- 172.21.15.83:0/4010111954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7be81f28c0 con 0x7f7be80b3c60 2023-11-26T07:11:54.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.818+0000 7f7bd6ffd700 1 -- 172.21.15.83:0/4010111954 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bdc057ae0 con 0x7f7be80b3c60 2023-11-26T07:11:54.821 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:11:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 -- 172.21.15.83:0/4010111954 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7bc806aed0 msgr2=0x7f7bc806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7bc806aed0 0x7f7bc806d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7be815ddb0 tx=0x7f7bd8008040 comp rx=0 tx=0).stop 2023-11-26T07:11:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 -- 172.21.15.83:0/4010111954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7be80b3c60 msgr2=0x7f7be81f1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7be80b3c60 0x7f7be81f1700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c750 tx=0x7f7bdc00ca60 comp rx=0 tx=0).stop 2023-11-26T07:11:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 -- 172.21.15.83:0/4010111954 shutdown_connections 2023-11-26T07:11:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7be815b370 0x7f7be8073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7be80b3c60 0x7f7be81f1700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7bc806aed0 0x7f7bc806d390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 --2- 172.21.15.83:0/4010111954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7be81587f0 0x7f7be81548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 -- 172.21.15.83:0/4010111954 >> 172.21.15.83:0/4010111954 conn(0x7f7be80d0bf0 msgr2=0x7f7be80aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:54.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 -- 172.21.15.83:0/4010111954 shutdown_connections 2023-11-26T07:11:54.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.820+0000 7f7beccb1700 1 -- 172.21.15.83:0/4010111954 wait complete. 2023-11-26T07:11:54.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:11:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.955+0000 7fcb24d3b700 1 Processor -- start 2023-11-26T07:11:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.955+0000 7fcb24d3b700 1 -- start start 2023-11-26T07:11:54.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.956+0000 7fcb24d3b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb200cc540 0x7fcb2006e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.956+0000 7fcb24d3b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 0x7fcb200b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.956+0000 7fcb24d3b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb200b2f20 con 0x7fcb2015a620 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.956+0000 7fcb24d3b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2006ef90 con 0x7fcb200cc540 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.956+0000 7fcb24d3b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2015aa00 con 0x7fcb2006eb20 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1dd9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb200cc540 0x7fcb2006e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1dd9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb200cc540 0x7fcb2006e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35046/0 (socket says 172.21.15.83:35046) 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1dd9b700 1 -- 172.21.15.83:0/3985610539 learned_addr learned my addr 172.21.15.83:0/3985610539 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1ed9d700 1 --2- 172.21.15.83:0/3985610539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 0x7fcb200b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1dd9b700 1 -- 172.21.15.83:0/3985610539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 msgr2=0x7fcb200b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1dd9b700 1 --2- 172.21.15.83:0/3985610539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 0x7fcb200b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1dd9b700 1 -- 172.21.15.83:0/3985610539 >> v1:172.21.15.203:6789/0 conn(0x7fcb2015a620 legacy=0x7fcb200ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1dd9b700 1 -- 172.21.15.83:0/3985610539 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200ce960 con 0x7fcb200cc540 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1ed9d700 1 --2- 172.21.15.83:0/3985610539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 0x7fcb200b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:54.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1dd9b700 1 --2- 172.21.15.83:0/3985610539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb200cc540 0x7fcb2006e5e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcb100098f0 tx=0x7fcb10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e53112ca5ce134d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1d59a700 1 -- 172.21.15.83:0/3985610539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000b8a0 con 0x7fcb200cc540 2023-11-26T07:11:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1d59a700 1 -- 172.21.15.83:0/3985610539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1000ba00 con 0x7fcb200cc540 2023-11-26T07:11:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.957+0000 7fcb1d59a700 1 -- 172.21.15.83:0/3985610539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000bd90 con 0x7fcb200cc540 2023-11-26T07:11:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.958+0000 7fcb24d3b700 1 -- 172.21.15.83:0/3985610539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb200cc540 msgr2=0x7fcb2006e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.958+0000 7fcb24d3b700 1 --2- 172.21.15.83:0/3985610539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb200cc540 0x7fcb2006e5e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcb100098f0 tx=0x7fcb10009c00 comp rx=0 tx=0).stop 2023-11-26T07:11:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.958+0000 7fcb24d3b700 1 -- 172.21.15.83:0/3985610539 shutdown_connections 2023-11-26T07:11:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.958+0000 7fcb24d3b700 1 --2- 172.21.15.83:0/3985610539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 0x7fcb200b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.958+0000 7fcb24d3b700 1 --2- 172.21.15.83:0/3985610539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb200cc540 0x7fcb2006e5e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.958+0000 7fcb24d3b700 1 -- 172.21.15.83:0/3985610539 >> 172.21.15.83:0/3985610539 conn(0x7fcb200b2050 msgr2=0x7fcb200b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.958+0000 7fcb24d3b700 1 -- 172.21.15.83:0/3985610539 shutdown_connections 2023-11-26T07:11:54.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.958+0000 7fcb24d3b700 1 -- 172.21.15.83:0/3985610539 wait complete. 2023-11-26T07:11:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.959+0000 7fcb24d3b700 1 Processor -- start 2023-11-26T07:11:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.959+0000 7fcb24d3b700 1 -- start start 2023-11-26T07:11:54.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb24d3b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 0x7fcb20148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb24d3b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb200cc540 0x7fcb20154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb24d3b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb2015a620 0x7fcb2015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb24d3b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20153cd0 con 0x7fcb200cc540 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb24d3b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20153fb0 con 0x7fcb2015a620 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb24d3b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20154290 con 0x7fcb2006eb20 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb1dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb200cc540 0x7fcb20154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb1dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb200cc540 0x7fcb20154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35052/0 (socket says 172.21.15.83:35052) 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb1dd9b700 1 -- 172.21.15.83:0/1951980663 learned_addr learned my addr 172.21.15.83:0/1951980663 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb1ed9d700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb2015a620 0x7fcb2015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb1e59c700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 0x7fcb20148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb1dd9b700 1 -- 172.21.15.83:0/1951980663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 msgr2=0x7fcb20148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb1dd9b700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 0x7fcb20148ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb1dd9b700 1 -- 172.21.15.83:0/1951980663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb2015a620 msgr2=0x7fcb2015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb1dd9b700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb2015a620 0x7fcb2015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb1dd9b700 1 -- 172.21.15.83:0/1951980663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb10009550 con 0x7fcb200cc540 2023-11-26T07:11:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.960+0000 7fcb1e59c700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 0x7fcb20148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.961+0000 7fcb1dd9b700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb200cc540 0x7fcb20154d70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009130 tx=0x7fcb10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.961+0000 7fcb0effd700 1 -- 172.21.15.83:0/1951980663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb100036a0 con 0x7fcb200cc540 2023-11-26T07:11:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.961+0000 7fcb0effd700 1 -- 172.21.15.83:0/1951980663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1001e070 con 0x7fcb200cc540 2023-11-26T07:11:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.961+0000 7fcb0effd700 1 -- 172.21.15.83:0/1951980663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10003a60 con 0x7fcb200cc540 2023-11-26T07:11:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.961+0000 7fcb24d3b700 1 -- 172.21.15.83:0/1951980663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb20154570 con 0x7fcb200cc540 2023-11-26T07:11:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.961+0000 7fcb24d3b700 1 -- 172.21.15.83:0/1951980663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb201538f0 con 0x7fcb200cc540 2023-11-26T07:11:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.962+0000 7fcb24d3b700 1 -- 172.21.15.83:0/1951980663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb20068d90 con 0x7fcb200cc540 2023-11-26T07:11:54.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.962+0000 7fcb0effd700 1 -- 172.21.15.83:0/1951980663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb10003bc0 con 0x7fcb200cc540 2023-11-26T07:11:54.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.963+0000 7fcb0effd700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcb0006aed0 0x7fcb0006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:54.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.963+0000 7fcb0effd700 1 -- 172.21.15.83:0/1951980663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb10020070 con 0x7fcb200cc540 2023-11-26T07:11:54.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.963+0000 7fcb1e59c700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcb0006aed0 0x7fcb0006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:54.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.963+0000 7fcb1e59c700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcb0006aed0 0x7fcb0006d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb14000f60 tx=0x7fcb14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:54.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:54.967+0000 7fcb0effd700 1 -- 172.21.15.83:0/1951980663 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb10055c00 con 0x7fcb200cc540 2023-11-26T07:11:55.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.156+0000 7fcb24d3b700 1 -- 172.21.15.83:0/1951980663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb20153a80 con 0x7fcb200cc540 2023-11-26T07:11:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.156+0000 7fcb0effd700 1 -- 172.21.15.83:0/1951980663 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb1002a020 con 0x7fcb200cc540 2023-11-26T07:11:55.159 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:11:55.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.158+0000 7fcb24d3b700 1 -- 172.21.15.83:0/1951980663 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcb0006aed0 msgr2=0x7fcb0006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.158+0000 7fcb24d3b700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcb0006aed0 0x7fcb0006d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb14000f60 tx=0x7fcb14009040 comp rx=0 tx=0).stop 2023-11-26T07:11:55.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.158+0000 7fcb24d3b700 1 -- 172.21.15.83:0/1951980663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb200cc540 msgr2=0x7fcb20154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.158+0000 7fcb24d3b700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb200cc540 0x7fcb20154d70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009130 tx=0x7fcb10004750 comp rx=0 tx=0).stop 2023-11-26T07:11:55.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.158+0000 7fcb24d3b700 1 -- 172.21.15.83:0/1951980663 shutdown_connections 2023-11-26T07:11:55.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.158+0000 7fcb24d3b700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb2015a620 0x7fcb2015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.158+0000 7fcb24d3b700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcb0006aed0 0x7fcb0006d390 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.158+0000 7fcb24d3b700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2006eb20 0x7fcb20148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.158+0000 7fcb24d3b700 1 --2- 172.21.15.83:0/1951980663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb200cc540 0x7fcb20154d70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.158+0000 7fcb24d3b700 1 -- 172.21.15.83:0/1951980663 >> 172.21.15.83:0/1951980663 conn(0x7fcb200b2050 msgr2=0x7fcb200af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:55.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.158+0000 7fcb24d3b700 1 -- 172.21.15.83:0/1951980663 shutdown_connections 2023-11-26T07:11:55.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.159+0000 7fcb24d3b700 1 -- 172.21.15.83:0/1951980663 wait complete. 2023-11-26T07:11:55.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:11:55.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.297+0000 7f1ffbd41700 1 Processor -- start 2023-11-26T07:11:55.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.297+0000 7f1ffbd41700 1 -- start start 2023-11-26T07:11:55.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ffbd41700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1ff40cf110 0x7f1ff40cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ffbd41700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff40d0140 con 0x7f1ff40c6c30 2023-11-26T07:11:55.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ffbd41700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff40d03e0 con 0x7f1ff415bf50 2023-11-26T07:11:55.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ffbd41700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff415c330 con 0x7f1ff40cf110 2023-11-26T07:11:55.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff92dc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1ff40c6c30 0x7f1ff40cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55098/0 (socket says 172.21.15.83:55098) 2023-11-26T07:11:55.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff92dc700 1 -- 172.21.15.83:0/1490309855 learned_addr learned my addr 172.21.15.83:0/1490309855 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:55.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315437802 0 0) 0x7f1ff40d0140 con 0x7f1ff40c6c30 2023-11-26T07:11:55.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ffa2de700 1 --2- 172.21.15.83:0/1490309855 >> v2:172.21.15.203:3300/0 conn(0x7f1ff40cf110 0x7f1ff40cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe00035b0 con 0x7f1ff40c6c30 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4154816308 0 0) 0x7f1fe00035b0 con 0x7f1ff40c6c30 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 >> v2:172.21.15.203:3300/0 conn(0x7f1ff40cf110 msgr2=0x7f1ff40cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff8adb700 1 --2- 172.21.15.83:0/1490309855 >> v2:172.21.15.203:3300/0 conn(0x7f1ff40cf110 0x7f1ff40cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 >> v1:172.21.15.203:6789/0 conn(0x7f1ff415bf50 legacy=0x7f1ff40ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff40d06c0 con 0x7f1ff40c6c30 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f1fe4001670 con 0x7f1ff40c6c30 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 >> v1:172.21.15.83:6789/0 conn(0x7f1ff40c6c30 legacy=0x7f1ff40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.298+0000 7f1ff8adb700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe0003a00 0x7f1fe0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff8adb700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fe0004850 0x7f1fe0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff8adb700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fe0005210 0x7f1fe000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff415c330 con 0x7f1fe0003a00 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe000c0b0 con 0x7f1fe0005210 2023-11-26T07:11:55.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe000c270 con 0x7f1fe0004850 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff9add700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe0003a00 0x7f1fe0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ffa2de700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fe0005210 0x7f1fe000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff92dc700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fe0004850 0x7f1fe0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff9add700 1 -- 172.21.15.83:0/1490309855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fe0004850 msgr2=0x7f1fe0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff9add700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fe0004850 0x7f1fe0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff9add700 1 -- 172.21.15.83:0/1490309855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fe0005210 msgr2=0x7f1fe000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff9add700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fe0005210 0x7f1fe000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff9add700 1 -- 172.21.15.83:0/1490309855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe00035b0 con 0x7f1fe0003a00 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ffa2de700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fe0005210 0x7f1fe000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ffa2de700 1 -- 172.21.15.83:0/1490309855 reap_dead start 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff9add700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe0003a00 0x7f1fe0003e20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00017c0 tx=0x7f1fe0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9be3e60aaff813 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00052d0 con 0x7f1fe0003a00 2023-11-26T07:11:55.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff0005430 con 0x7f1fe0003a00 2023-11-26T07:11:55.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.299+0000 7f1ff8adb700 1 -- 172.21.15.83:0/1490309855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00038c0 con 0x7f1fe0003a00 2023-11-26T07:11:55.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.300+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1490309855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe0003a00 msgr2=0x7f1fe0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.300+0000 7f1ffbd41700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe0003a00 0x7f1fe0003e20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00017c0 tx=0x7f1fe0005880 comp rx=0 tx=0).stop 2023-11-26T07:11:55.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.300+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1490309855 shutdown_connections 2023-11-26T07:11:55.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.300+0000 7f1ffbd41700 1 --2- 172.21.15.83:0/1490309855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fe0003a00 0x7f1fe0003e20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.300+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1490309855 >> 172.21.15.83:0/1490309855 conn(0x7f1ff40b3780 msgr2=0x7f1ff40b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:55.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.300+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1490309855 shutdown_connections 2023-11-26T07:11:55.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.300+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1490309855 wait complete. 2023-11-26T07:11:55.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.301+0000 7f1ffbd41700 1 Processor -- start 2023-11-26T07:11:55.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.301+0000 7f1ffbd41700 1 -- start start 2023-11-26T07:11:55.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ffbd41700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fe0003a00 0x7f1ff4151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ffbd41700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff406b9b0 0x7f1ff415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ffbd41700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ff40cae50 0x7f1ff40cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ffbd41700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40cb840 con 0x7f1ff406b9b0 2023-11-26T07:11:55.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ffbd41700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4151680 con 0x7f1fe0003a00 2023-11-26T07:11:55.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ffbd41700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff415f6f0 con 0x7f1ff40cae50 2023-11-26T07:11:55.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ff92dc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff406b9b0 0x7f1ff415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ff92dc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff406b9b0 0x7f1ff415f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35072/0 (socket says 172.21.15.83:35072) 2023-11-26T07:11:55.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ff92dc700 1 -- 172.21.15.83:0/1865027267 learned_addr learned my addr 172.21.15.83:0/1865027267 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:55.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ff9add700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fe0003a00 0x7f1ff4151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ffa2de700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ff40cae50 0x7f1ff40cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ff92dc700 1 -- 172.21.15.83:0/1865027267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ff40cae50 msgr2=0x7f1ff40cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ff92dc700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ff40cae50 0x7f1ff40cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ff92dc700 1 -- 172.21.15.83:0/1865027267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fe0003a00 msgr2=0x7f1ff4151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ff92dc700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fe0003a00 0x7f1ff4151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ff92dc700 1 -- 172.21.15.83:0/1865027267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff406bf30 con 0x7f1ff406b9b0 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ff92dc700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff406b9b0 0x7f1ff415f1b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e830 tx=0x7f1fe400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1fea7fc700 1 -- 172.21.15.83:0/1865027267 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400c5c0 con 0x7f1ff406b9b0 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1fea7fc700 1 -- 172.21.15.83:0/1865027267 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe400c740 con 0x7f1ff406b9b0 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1fea7fc700 1 -- 172.21.15.83:0/1865027267 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400caf0 con 0x7f1ff406b9b0 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1865027267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff406c210 con 0x7f1ff406b9b0 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.302+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1865027267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff40b4bf0 con 0x7f1ff406b9b0 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.303+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1865027267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff40b4db0 con 0x7f1ff406b9b0 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.304+0000 7f1fea7fc700 1 -- 172.21.15.83:0/1865027267 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1fe400cc50 con 0x7f1ff406b9b0 2023-11-26T07:11:55.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.304+0000 7f1fea7fc700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1fd806aed0 0x7f1fd806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.304+0000 7f1fea7fc700 1 -- 172.21.15.83:0/1865027267 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fe4089530 con 0x7f1ff406b9b0 2023-11-26T07:11:55.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.304+0000 7f1ff9add700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1fd806aed0 0x7f1fd806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.305+0000 7f1ff9add700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1fd806aed0 0x7f1fd806d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0000c00 tx=0x7f1ff0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:55.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.309+0000 7f1fea7fc700 1 -- 172.21.15.83:0/1865027267 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1fe40544b0 con 0x7f1ff406b9b0 2023-11-26T07:11:55.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.498+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1865027267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ff40b4f40 con 0x7f1ff406b9b0 2023-11-26T07:11:55.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.498+0000 7f1fea7fc700 1 -- 172.21.15.83:0/1865027267 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1fe4057ad0 con 0x7f1ff406b9b0 2023-11-26T07:11:55.501 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:11:55.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1865027267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1fd806aed0 msgr2=0x7f1fd806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1fd806aed0 0x7f1fd806d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0000c00 tx=0x7f1ff0009030 comp rx=0 tx=0).stop 2023-11-26T07:11:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1865027267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff406b9b0 msgr2=0x7f1ff415f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff406b9b0 0x7f1ff415f1b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e830 tx=0x7f1fe400ebf0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1865027267 shutdown_connections 2023-11-26T07:11:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1fd806aed0 0x7f1fd806d390 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ff40cae50 0x7f1ff40cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fe0003a00 0x7f1ff4151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 --2- 172.21.15.83:0/1865027267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff406b9b0 0x7f1ff415f1b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1865027267 >> 172.21.15.83:0/1865027267 conn(0x7f1ff40b3780 msgr2=0x7f1ff40c70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1865027267 shutdown_connections 2023-11-26T07:11:55.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.500+0000 7f1ffbd41700 1 -- 172.21.15.83:0/1865027267 wait complete. 2023-11-26T07:11:55.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:55.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.633+0000 7f72c36e7700 1 Processor -- start 2023-11-26T07:11:55.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.633+0000 7f72c36e7700 1 -- start start 2023-11-26T07:11:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c36e7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f72bc06c710 0x7f72bc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c36e7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f72bc0af4d0 0x7f72bc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c36e7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72bc0ae430 con 0x7f72bc15e080 2023-11-26T07:11:55.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c36e7700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc0af0b0 con 0x7f72bc06c710 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c36e7700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc0b05a0 con 0x7f72bc0af4d0 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c0c82700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f72bc06c710 0x7f72bc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c0c82700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f72bc06c710 0x7f72bc0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35080/0 (socket says 172.21.15.83:35080) 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c0c82700 1 -- 172.21.15.83:0/2606176904 learned_addr learned my addr 172.21.15.83:0/2606176904 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c1c84700 1 --2- 172.21.15.83:0/2606176904 >> v2:172.21.15.203:3300/0 conn(0x7f72bc0af4d0 0x7f72bc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c0c82700 1 -- 172.21.15.83:0/2606176904 >> v2:172.21.15.203:3300/0 conn(0x7f72bc0af4d0 msgr2=0x7f72bc0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c0c82700 1 --2- 172.21.15.83:0/2606176904 >> v2:172.21.15.203:3300/0 conn(0x7f72bc0af4d0 0x7f72bc0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.634+0000 7f72c0c82700 1 -- 172.21.15.83:0/2606176904 >> v1:172.21.15.203:6789/0 conn(0x7f72bc15e080 legacy=0x7f72bc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72c0c82700 1 -- 172.21.15.83:0/2606176904 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc0ae710 con 0x7f72bc06c710 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72c0c82700 1 --2- 172.21.15.83:0/2606176904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72bc06c710 0x7f72bc0b5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f72bc09eea0 tx=0x7f72b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc663239814163af server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72b3fff700 1 -- 172.21.15.83:0/2606176904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800b860 con 0x7f72bc06c710 2023-11-26T07:11:55.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72b3fff700 1 -- 172.21.15.83:0/2606176904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800b9c0 con 0x7f72bc06c710 2023-11-26T07:11:55.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72b3fff700 1 -- 172.21.15.83:0/2606176904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800bd50 con 0x7f72bc06c710 2023-11-26T07:11:55.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72c36e7700 1 -- 172.21.15.83:0/2606176904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72bc06c710 msgr2=0x7f72bc0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72c36e7700 1 --2- 172.21.15.83:0/2606176904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72bc06c710 0x7f72bc0b5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f72bc09eea0 tx=0x7f72b8009c00 comp rx=0 tx=0).stop 2023-11-26T07:11:55.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72c36e7700 1 -- 172.21.15.83:0/2606176904 shutdown_connections 2023-11-26T07:11:55.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72c36e7700 1 --2- 172.21.15.83:0/2606176904 >> v2:172.21.15.203:3300/0 conn(0x7f72bc0af4d0 0x7f72bc0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72c36e7700 1 --2- 172.21.15.83:0/2606176904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72bc06c710 0x7f72bc0b5050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72c36e7700 1 -- 172.21.15.83:0/2606176904 >> 172.21.15.83:0/2606176904 conn(0x7f72bc071b90 msgr2=0x7f72bc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:55.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72c36e7700 1 -- 172.21.15.83:0/2606176904 shutdown_connections 2023-11-26T07:11:55.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.635+0000 7f72c36e7700 1 -- 172.21.15.83:0/2606176904 wait complete. 2023-11-26T07:11:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.636+0000 7f72c36e7700 1 Processor -- start 2023-11-26T07:11:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c36e7700 1 -- start start 2023-11-26T07:11:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c36e7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72bc06c710 0x7f72bc141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c36e7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72bc0af4d0 0x7f72bc0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c36e7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72bc15e080 0x7f72bc142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c36e7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc0cf390 con 0x7f72bc06c710 2023-11-26T07:11:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c36e7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc0ce710 con 0x7f72bc0af4d0 2023-11-26T07:11:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c36e7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc0ce9f0 con 0x7f72bc15e080 2023-11-26T07:11:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c1483700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72bc06c710 0x7f72bc141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c1483700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72bc06c710 0x7f72bc141de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35084/0 (socket says 172.21.15.83:35084) 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c1483700 1 -- 172.21.15.83:0/2002562086 learned_addr learned my addr 172.21.15.83:0/2002562086 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c0c82700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72bc0af4d0 0x7f72bc0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.637+0000 7f72c1c84700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72bc15e080 0x7f72bc142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72c1483700 1 -- 172.21.15.83:0/2002562086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72bc15e080 msgr2=0x7f72bc142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72c1483700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72bc15e080 0x7f72bc142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72c1483700 1 -- 172.21.15.83:0/2002562086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72bc0af4d0 msgr2=0x7f72bc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72c1483700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72bc0af4d0 0x7f72bc0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72c1483700 1 -- 172.21.15.83:0/2002562086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8009550 con 0x7f72bc06c710 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72c1c84700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72bc15e080 0x7f72bc142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72c1483700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72bc06c710 0x7f72bc141de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00c450 tx=0x7f72ac00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72b1ffb700 1 -- 172.21.15.83:0/2002562086 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0052d0 con 0x7f72bc06c710 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72b1ffb700 1 -- 172.21.15.83:0/2002562086 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac005430 con 0x7f72bc06c710 2023-11-26T07:11:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72c36e7700 1 -- 172.21.15.83:0/2002562086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc0cecd0 con 0x7f72bc06c710 2023-11-26T07:11:55.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72b1ffb700 1 -- 172.21.15.83:0/2002562086 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0038c0 con 0x7f72bc06c710 2023-11-26T07:11:55.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.638+0000 7f72c36e7700 1 -- 172.21.15.83:0/2002562086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72bc142320 con 0x7f72bc06c710 2023-11-26T07:11:55.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.639+0000 7f72c36e7700 1 -- 172.21.15.83:0/2002562086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72bc0753f0 con 0x7f72bc06c710 2023-11-26T07:11:55.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.639+0000 7f72b1ffb700 1 -- 172.21.15.83:0/2002562086 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f72ac003a20 con 0x7f72bc06c710 2023-11-26T07:11:55.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.640+0000 7f72b1ffb700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72a006aed0 0x7f72a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.640+0000 7f72b1ffb700 1 -- 172.21.15.83:0/2002562086 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72ac088960 con 0x7f72bc06c710 2023-11-26T07:11:55.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.640+0000 7f72c0c82700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72a006aed0 0x7f72a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.640+0000 7f72c0c82700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72a006aed0 0x7f72a006d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f72b8000c00 tx=0x7f72b8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.644+0000 7f72b1ffb700 1 -- 172.21.15.83:0/2002562086 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f72ac053a10 con 0x7f72bc06c710 2023-11-26T07:11:55.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.836+0000 7f72c36e7700 1 -- 172.21.15.83:0/2002562086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72bc0cf600 con 0x7f72bc06c710 2023-11-26T07:11:55.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.837+0000 7f72b1ffb700 1 -- 172.21.15.83:0/2002562086 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f72ac017090 con 0x7f72bc06c710 2023-11-26T07:11:55.840 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:55.840 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.839+0000 7f72c36e7700 1 -- 172.21.15.83:0/2002562086 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72a006aed0 msgr2=0x7f72a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.839+0000 7f72c36e7700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72a006aed0 0x7f72a006d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f72b8000c00 tx=0x7f72b8008030 comp rx=0 tx=0).stop 2023-11-26T07:11:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.839+0000 7f72c36e7700 1 -- 172.21.15.83:0/2002562086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72bc06c710 msgr2=0x7f72bc141de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.839+0000 7f72c36e7700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72bc06c710 0x7f72bc141de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00c450 tx=0x7f72ac00c760 comp rx=0 tx=0).stop 2023-11-26T07:11:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.839+0000 7f72c36e7700 1 -- 172.21.15.83:0/2002562086 shutdown_connections 2023-11-26T07:11:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.839+0000 7f72c36e7700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72a006aed0 0x7f72a006d390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.839+0000 7f72c36e7700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72bc15e080 0x7f72bc142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.839+0000 7f72c36e7700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72bc06c710 0x7f72bc141de0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.839+0000 7f72c36e7700 1 --2- 172.21.15.83:0/2002562086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72bc0af4d0 0x7f72bc0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.839+0000 7f72c36e7700 1 -- 172.21.15.83:0/2002562086 >> 172.21.15.83:0/2002562086 conn(0x7f72bc071b90 msgr2=0x7f72bc06bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:55.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.840+0000 7f72c36e7700 1 -- 172.21.15.83:0/2002562086 shutdown_connections 2023-11-26T07:11:55.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.840+0000 7f72c36e7700 1 -- 172.21.15.83:0/2002562086 wait complete. 2023-11-26T07:11:55.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:11:55.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:11:55.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.975+0000 7fa529fc6700 1 Processor -- start 2023-11-26T07:11:55.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.975+0000 7fa529fc6700 1 -- start start 2023-11-26T07:11:55.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.976+0000 7fa529fc6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5240c6c30 0x7fa5240cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.976+0000 7fa529fc6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 0x7fa5240cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.976+0000 7fa529fc6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5240d0140 con 0x7fa52415bf50 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.976+0000 7fa529fc6700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52415c330 con 0x7fa5240c6c30 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.976+0000 7fa529fc6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5240cf3d0 con 0x7fa5240cef40 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.976+0000 7fa523fff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 0x7fa5240cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.976+0000 7fa523fff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 0x7fa5240cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:45554/0 (socket says 172.21.15.83:45554) 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.976+0000 7fa523fff700 1 -- 172.21.15.83:0/2258831509 learned_addr learned my addr 172.21.15.83:0/2258831509 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.976+0000 7fa522ffd700 1 --2- 172.21.15.83:0/2258831509 >> v2:172.21.15.203:3300/0 conn(0x7fa5240c6c30 0x7fa5240cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.977+0000 7fa523fff700 1 -- 172.21.15.83:0/2258831509 >> v2:172.21.15.203:3300/0 conn(0x7fa5240c6c30 msgr2=0x7fa5240cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.977+0000 7fa523fff700 1 --2- 172.21.15.83:0/2258831509 >> v2:172.21.15.203:3300/0 conn(0x7fa5240c6c30 0x7fa5240cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.977+0000 7fa523fff700 1 -- 172.21.15.83:0/2258831509 >> v1:172.21.15.203:6789/0 conn(0x7fa52415bf50 legacy=0x7fa5240ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.977+0000 7fa523fff700 1 -- 172.21.15.83:0/2258831509 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5240d03e0 con 0x7fa5240cef40 2023-11-26T07:11:55.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.977+0000 7fa5227fc700 1 -- 172.21.15.83:0/2258831509 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387550259 0 0) 0x7fa5240d0140 con 0x7fa52415bf50 2023-11-26T07:11:55.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.977+0000 7fa523fff700 1 --2- 172.21.15.83:0/2258831509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 0x7fa5240cc6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009e50 tx=0x7fa51c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc281fca2be6a50 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:55.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.977+0000 7fa5227fc700 1 -- 172.21.15.83:0/2258831509 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00e070 con 0x7fa5240cef40 2023-11-26T07:11:55.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.977+0000 7fa5227fc700 1 -- 172.21.15.83:0/2258831509 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c00b910 con 0x7fa5240cef40 2023-11-26T07:11:55.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.977+0000 7fa5227fc700 1 -- 172.21.15.83:0/2258831509 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00bcc0 con 0x7fa5240cef40 2023-11-26T07:11:55.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.978+0000 7fa529fc6700 1 -- 172.21.15.83:0/2258831509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 msgr2=0x7fa5240cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.978+0000 7fa529fc6700 1 --2- 172.21.15.83:0/2258831509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 0x7fa5240cc6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009e50 tx=0x7fa51c00b330 comp rx=0 tx=0).stop 2023-11-26T07:11:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.978+0000 7fa529fc6700 1 -- 172.21.15.83:0/2258831509 shutdown_connections 2023-11-26T07:11:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.978+0000 7fa529fc6700 1 --2- 172.21.15.83:0/2258831509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 0x7fa5240cc6c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.978+0000 7fa529fc6700 1 --2- 172.21.15.83:0/2258831509 >> v2:172.21.15.203:3300/0 conn(0x7fa5240c6c30 0x7fa5240cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.978+0000 7fa529fc6700 1 -- 172.21.15.83:0/2258831509 >> 172.21.15.83:0/2258831509 conn(0x7fa5240b3780 msgr2=0x7fa5240b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.978+0000 7fa529fc6700 1 -- 172.21.15.83:0/2258831509 shutdown_connections 2023-11-26T07:11:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.978+0000 7fa529fc6700 1 -- 172.21.15.83:0/2258831509 wait complete. 2023-11-26T07:11:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.979+0000 7fa529fc6700 1 Processor -- start 2023-11-26T07:11:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.979+0000 7fa529fc6700 1 -- start start 2023-11-26T07:11:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa529fc6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5240c6c30 0x7fa524154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa529fc6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 0x7fa524159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa529fc6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa52415bf50 0x7fa5241599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa529fc6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa524155630 con 0x7fa52415bf50 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa529fc6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52406b090 con 0x7fa5240c6c30 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa529fc6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52406b370 con 0x7fa5240cef40 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa523fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa52415bf50 0x7fa5241599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa523fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa52415bf50 0x7fa5241599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35102/0 (socket says 172.21.15.83:35102) 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa523fff700 1 -- 172.21.15.83:0/3225131925 learned_addr learned my addr 172.21.15.83:0/3225131925 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa5237fe700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5240c6c30 0x7fa524154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa522ffd700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 0x7fa524159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa523fff700 1 -- 172.21.15.83:0/3225131925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 msgr2=0x7fa524159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa523fff700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 0x7fa524159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa523fff700 1 -- 172.21.15.83:0/3225131925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5240c6c30 msgr2=0x7fa524154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:55.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa523fff700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5240c6c30 0x7fa524154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa523fff700 1 -- 172.21.15.83:0/3225131925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c009b30 con 0x7fa52415bf50 2023-11-26T07:11:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.980+0000 7fa523fff700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa52415bf50 0x7fa5241599b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa5240cdc10 tx=0x7fa51c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.981+0000 7fa50bfff700 1 -- 172.21.15.83:0/3225131925 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c003ea0 con 0x7fa52415bf50 2023-11-26T07:11:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.981+0000 7fa50bfff700 1 -- 172.21.15.83:0/3225131925 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c0048e0 con 0x7fa52415bf50 2023-11-26T07:11:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.981+0000 7fa50bfff700 1 -- 172.21.15.83:0/3225131925 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c01c680 con 0x7fa52415bf50 2023-11-26T07:11:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.981+0000 7fa529fc6700 1 -- 172.21.15.83:0/3225131925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52406b650 con 0x7fa52415bf50 2023-11-26T07:11:55.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.981+0000 7fa529fc6700 1 -- 172.21.15.83:0/3225131925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52406cf40 con 0x7fa52415bf50 2023-11-26T07:11:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.982+0000 7fa529fc6700 1 -- 172.21.15.83:0/3225131925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52406bca0 con 0x7fa52415bf50 2023-11-26T07:11:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.982+0000 7fa50bfff700 1 -- 172.21.15.83:0/3225131925 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa51c01c7e0 con 0x7fa52415bf50 2023-11-26T07:11:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.982+0000 7fa50bfff700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa51006aed0 0x7fa51006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.982+0000 7fa50bfff700 1 -- 172.21.15.83:0/3225131925 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa51c08b210 con 0x7fa52415bf50 2023-11-26T07:11:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.982+0000 7fa5237fe700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa51006aed0 0x7fa51006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.983+0000 7fa5237fe700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa51006aed0 0x7fa51006d390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa518000f60 tx=0x7fa518009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:55.987+0000 7fa50bfff700 1 -- 172.21.15.83:0/3225131925 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa51c0562d0 con 0x7fa52415bf50 2023-11-26T07:11:56.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.175+0000 7fa529fc6700 1 -- 172.21.15.83:0/3225131925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa52406bb10 con 0x7fa52415bf50 2023-11-26T07:11:56.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.176+0000 7fa50bfff700 1 -- 172.21.15.83:0/3225131925 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa51c0598f0 con 0x7fa52415bf50 2023-11-26T07:11:56.179 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:11:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 -- 172.21.15.83:0/3225131925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa51006aed0 msgr2=0x7fa51006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa51006aed0 0x7fa51006d390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa518000f60 tx=0x7fa518009040 comp rx=0 tx=0).stop 2023-11-26T07:11:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 -- 172.21.15.83:0/3225131925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa52415bf50 msgr2=0x7fa5241599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa52415bf50 0x7fa5241599b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa5240cdc10 tx=0x7fa51c004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 -- 172.21.15.83:0/3225131925 shutdown_connections 2023-11-26T07:11:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa52415bf50 0x7fa5241599b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5240c6c30 0x7fa524154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa51006aed0 0x7fa51006d390 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 --2- 172.21.15.83:0/3225131925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5240cef40 0x7fa524159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 -- 172.21.15.83:0/3225131925 >> 172.21.15.83:0/3225131925 conn(0x7fa5240b3780 msgr2=0x7fa5240ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 -- 172.21.15.83:0/3225131925 shutdown_connections 2023-11-26T07:11:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.178+0000 7fa529fc6700 1 -- 172.21.15.83:0/3225131925 wait complete. 2023-11-26T07:11:56.193 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:11:56.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.312+0000 7f9471827700 1 Processor -- start 2023-11-26T07:11:56.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.312+0000 7f9471827700 1 -- start start 2023-11-26T07:11:56.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.313+0000 7f9471827700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f946c0cc540 0x7f946c06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.313+0000 7f9471827700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c06ef50 con 0x7f946c15a620 2023-11-26T07:11:56.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.313+0000 7f9471827700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c0b2f20 con 0x7f946c06eb20 2023-11-26T07:11:56.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.313+0000 7f9471827700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c15aa00 con 0x7f946c0cc540 2023-11-26T07:11:56.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.313+0000 7f946affd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f946c15a620 0x7f946c0ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55146/0 (socket says 172.21.15.83:55146) 2023-11-26T07:11:56.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.313+0000 7f946affd700 1 -- 172.21.15.83:0/2985312810 learned_addr learned my addr 172.21.15.83:0/2985312810 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:56.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f946a7fc700 1 --2- 172.21.15.83:0/2985312810 >> v2:172.21.15.203:3300/0 conn(0x7f946c0cc540 0x7f946c06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997328089 0 0) 0x7f946c06ef50 con 0x7f946c15a620 2023-11-26T07:11:56.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94580035b0 con 0x7f946c15a620 2023-11-26T07:11:56.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3233732099 0 0) 0x7f94580035b0 con 0x7f946c15a620 2023-11-26T07:11:56.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 >> v2:172.21.15.203:3300/0 conn(0x7f946c0cc540 msgr2=0x7f946c06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:56.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 --2- 172.21.15.83:0/2985312810 >> v2:172.21.15.203:3300/0 conn(0x7f946c0cc540 0x7f946c06e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 >> v1:172.21.15.203:6789/0 conn(0x7f946c06eb20 legacy=0x7f946c0b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c0ce960 con 0x7f946c15a620 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f94640019f0 con 0x7f946c15a620 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 >> v1:172.21.15.83:6789/0 conn(0x7f946c15a620 legacy=0x7f946c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 --2- 172.21.15.83:0/2985312810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9458003a00 0x7f9458003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 --2- 172.21.15.83:0/2985312810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9458004850 0x7f9458004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 --2- 172.21.15.83:0/2985312810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9458005210 0x7f945800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c15aa00 con 0x7f9458005210 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945800c0b0 con 0x7f9458004850 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945800c270 con 0x7f9458003a00 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f946b7fe700 1 --2- 172.21.15.83:0/2985312810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9458005210 0x7f945800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f946a7fc700 1 --2- 172.21.15.83:0/2985312810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9458004850 0x7f9458004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f946affd700 1 --2- 172.21.15.83:0/2985312810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9458003a00 0x7f9458003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f946b7fe700 1 -- 172.21.15.83:0/2985312810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9458003a00 msgr2=0x7f9458003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f946b7fe700 1 --2- 172.21.15.83:0/2985312810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9458003a00 0x7f9458003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f946b7fe700 1 -- 172.21.15.83:0/2985312810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9458004850 msgr2=0x7f9458004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f946b7fe700 1 --2- 172.21.15.83:0/2985312810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9458004850 0x7f9458004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f946b7fe700 1 -- 172.21.15.83:0/2985312810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94580035b0 con 0x7f9458005210 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.314+0000 7f946b7fe700 1 -- 172.21.15.83:0/2985312810 reap_dead start 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.315+0000 7f946b7fe700 1 --2- 172.21.15.83:0/2985312810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9458005210 0x7f945800ba10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f945c0017c0 tx=0x7f9458005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9498479460a81264 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.315+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0052d0 con 0x7f9458005210 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.315+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c005430 con 0x7f9458005210 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.315+0000 7f9469ffb700 1 -- 172.21.15.83:0/2985312810 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0038c0 con 0x7f9458005210 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.315+0000 7f9471827700 1 -- 172.21.15.83:0/2985312810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9458005210 msgr2=0x7f945800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.315+0000 7f9471827700 1 --2- 172.21.15.83:0/2985312810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9458005210 0x7f945800ba10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f945c0017c0 tx=0x7f9458005880 comp rx=0 tx=0).stop 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.315+0000 7f9471827700 1 -- 172.21.15.83:0/2985312810 shutdown_connections 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.315+0000 7f9471827700 1 --2- 172.21.15.83:0/2985312810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9458005210 0x7f945800ba10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.315+0000 7f9471827700 1 -- 172.21.15.83:0/2985312810 >> 172.21.15.83:0/2985312810 conn(0x7f946c0b2050 msgr2=0x7f946c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.315+0000 7f9471827700 1 -- 172.21.15.83:0/2985312810 shutdown_connections 2023-11-26T07:11:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.315+0000 7f9471827700 1 -- 172.21.15.83:0/2985312810 wait complete. 2023-11-26T07:11:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.316+0000 7f9471827700 1 Processor -- start 2023-11-26T07:11:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.317+0000 7f9471827700 1 -- start start 2023-11-26T07:11:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.317+0000 7f9471827700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9458005210 0x7f946c14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.317+0000 7f9471827700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f946c14be40 0x7f946c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.317+0000 7f9471827700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946c15b900 0x7f946c1f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.317+0000 7f9471827700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1f7740 con 0x7f946c15b900 2023-11-26T07:11:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.317+0000 7f9471827700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1502f0 con 0x7f9458005210 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.317+0000 7f9471827700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0b14e0 con 0x7f946c14be40 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.317+0000 7f946b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946c15b900 0x7f946c1f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.317+0000 7f946b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946c15b900 0x7f946c1f7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35122/0 (socket says 172.21.15.83:35122) 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.317+0000 7f946b7fe700 1 -- 172.21.15.83:0/733046708 learned_addr learned my addr 172.21.15.83:0/733046708 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f946affd700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9458005210 0x7f946c14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f946a7fc700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f946c14be40 0x7f946c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f946b7fe700 1 -- 172.21.15.83:0/733046708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f946c14be40 msgr2=0x7f946c15b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f946b7fe700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f946c14be40 0x7f946c15b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f946b7fe700 1 -- 172.21.15.83:0/733046708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9458005210 msgr2=0x7f946c14f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f946b7fe700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9458005210 0x7f946c14f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f946b7fe700 1 -- 172.21.15.83:0/733046708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c14fd30 con 0x7f946c15b900 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f946a7fc700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f946c14be40 0x7f946c15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f946b7fe700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946c15b900 0x7f946c1f7b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f946c0ce890 tx=0x7f945c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:56.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f9463fff700 1 -- 172.21.15.83:0/733046708 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c020070 con 0x7f946c15b900 2023-11-26T07:11:56.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f9463fff700 1 -- 172.21.15.83:0/733046708 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c001290 con 0x7f946c15b900 2023-11-26T07:11:56.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f9471827700 1 -- 172.21.15.83:0/733046708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c150010 con 0x7f946c15b900 2023-11-26T07:11:56.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f9463fff700 1 -- 172.21.15.83:0/733046708 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c01c420 con 0x7f946c15b900 2023-11-26T07:11:56.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.318+0000 7f9471827700 1 -- 172.21.15.83:0/733046708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c14c2c0 con 0x7f946c15b900 2023-11-26T07:11:56.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.319+0000 7f9471827700 1 -- 172.21.15.83:0/733046708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c068d90 con 0x7f946c15b900 2023-11-26T07:11:56.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.319+0000 7f9463fff700 1 -- 172.21.15.83:0/733046708 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f945c01c580 con 0x7f946c15b900 2023-11-26T07:11:56.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.320+0000 7f9463fff700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f944c06aed0 0x7f944c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.320+0000 7f9463fff700 1 -- 172.21.15.83:0/733046708 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f945c0890b0 con 0x7f946c15b900 2023-11-26T07:11:56.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.320+0000 7f946affd700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f944c06aed0 0x7f944c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.320+0000 7f946affd700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f944c06aed0 0x7f944c06d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9464000f80 tx=0x7f9464006040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:56.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.324+0000 7f9463fff700 1 -- 172.21.15.83:0/733046708 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f945c0541f0 con 0x7f946c15b900 2023-11-26T07:11:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.514+0000 7f9471827700 1 -- 172.21.15.83:0/733046708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f946c1f7880 con 0x7f946c15b900 2023-11-26T07:11:56.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.514+0000 7f9463fff700 1 -- 172.21.15.83:0/733046708 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f946c1f7880 con 0x7f946c15b900 2023-11-26T07:11:56.517 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:11:56.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 -- 172.21.15.83:0/733046708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f944c06aed0 msgr2=0x7f944c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f944c06aed0 0x7f944c06d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9464000f80 tx=0x7f9464006040 comp rx=0 tx=0).stop 2023-11-26T07:11:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 -- 172.21.15.83:0/733046708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946c15b900 msgr2=0x7f946c1f7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946c15b900 0x7f946c1f7b60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f946c0ce890 tx=0x7f945c001120 comp rx=0 tx=0).stop 2023-11-26T07:11:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 -- 172.21.15.83:0/733046708 shutdown_connections 2023-11-26T07:11:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f946c15b900 0x7f946c1f7b60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9458005210 0x7f946c14f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f944c06aed0 0x7f944c06d390 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 --2- 172.21.15.83:0/733046708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f946c14be40 0x7f946c15b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 -- 172.21.15.83:0/733046708 >> 172.21.15.83:0/733046708 conn(0x7f946c0b2050 msgr2=0x7f946c071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 -- 172.21.15.83:0/733046708 shutdown_connections 2023-11-26T07:11:56.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.517+0000 7f9471827700 1 -- 172.21.15.83:0/733046708 wait complete. 2023-11-26T07:11:56.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:11:56.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.652+0000 7f1b48d6c700 1 Processor -- start 2023-11-26T07:11:56.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.652+0000 7f1b48d6c700 1 -- start start 2023-11-26T07:11:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.653+0000 7f1b48d6c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1b440b4310 0x7f1b4406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.653+0000 7f1b48d6c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4406ba50 con 0x7f1b4415e080 2023-11-26T07:11:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.653+0000 7f1b48d6c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4406b120 con 0x7f1b440ce430 2023-11-26T07:11:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.653+0000 7f1b48d6c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b440d1200 con 0x7f1b440b4310 2023-11-26T07:11:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.653+0000 7f1b4259c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1b4415e080 0x7f1b440d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55166/0 (socket says 172.21.15.83:55166) 2023-11-26T07:11:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.653+0000 7f1b4259c700 1 -- 172.21.15.83:0/2687777182 learned_addr learned my addr 172.21.15.83:0/2687777182 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b41d9b700 1 --2- 172.21.15.83:0/2687777182 >> v2:172.21.15.203:3300/0 conn(0x7f1b440b4310 0x7f1b4406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141849416 0 0) 0x7f1b4406ba50 con 0x7f1b4415e080 2023-11-26T07:11:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b300035b0 con 0x7f1b4415e080 2023-11-26T07:11:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4161521609 0 0) 0x7f1b300035b0 con 0x7f1b4415e080 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 >> v2:172.21.15.203:3300/0 conn(0x7f1b440b4310 msgr2=0x7f1b4406bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 --2- 172.21.15.83:0/2687777182 >> v2:172.21.15.203:3300/0 conn(0x7f1b440b4310 0x7f1b4406bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 >> v1:172.21.15.203:6789/0 conn(0x7f1b440ce430 legacy=0x7f1b440cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4406b400 con 0x7f1b4415e080 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f1b2c001670 con 0x7f1b4415e080 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 >> v1:172.21.15.83:6789/0 conn(0x7f1b4415e080 legacy=0x7f1b440d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b30003a00 0x7f1b30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b30004400 0x7f1b300048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30005170 0x7f1b3000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440d1200 con 0x7f1b30003a00 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3000c0b0 con 0x7f1b30005170 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3000c270 con 0x7f1b30004400 2023-11-26T07:11:56.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b41d9b700 1 --2- 172.21.15.83:0/2687777182 >> v2:172.21.15.203:3300/0 conn(0x7f1b440b4310 0x7f1b4406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4259c700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b30003a00 0x7f1b30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b42d9d700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30005170 0x7f1b3000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b41d9b700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b30004400 0x7f1b300048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4259c700 1 -- 172.21.15.83:0/2687777182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b30004400 msgr2=0x7f1b300048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4259c700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b30004400 0x7f1b300048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4259c700 1 -- 172.21.15.83:0/2687777182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30005170 msgr2=0x7f1b3000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4259c700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30005170 0x7f1b3000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b4259c700 1 -- 172.21.15.83:0/2687777182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b300035b0 con 0x7f1b30003a00 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b41d9b700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b30004400 0x7f1b300048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b42d9d700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30005170 0x7f1b3000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.654+0000 7f1b42d9d700 1 -- 172.21.15.83:0/2687777182 reap_dead start 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.655+0000 7f1b4259c700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b30003a00 0x7f1b30003de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00c2f0 tx=0x7f1b30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dda4df67704fcd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:56.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.655+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00ce30 con 0x7f1b30003a00 2023-11-26T07:11:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.655+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2c005960 con 0x7f1b30003a00 2023-11-26T07:11:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.655+0000 7f1b4159a700 1 -- 172.21.15.83:0/2687777182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c005d10 con 0x7f1b30003a00 2023-11-26T07:11:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.655+0000 7f1b48d6c700 1 -- 172.21.15.83:0/2687777182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b30003a00 msgr2=0x7f1b30003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.655+0000 7f1b48d6c700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b30003a00 0x7f1b30003de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00c2f0 tx=0x7f1b30005880 comp rx=0 tx=0).stop 2023-11-26T07:11:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.655+0000 7f1b48d6c700 1 -- 172.21.15.83:0/2687777182 shutdown_connections 2023-11-26T07:11:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.655+0000 7f1b48d6c700 1 --2- 172.21.15.83:0/2687777182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b30003a00 0x7f1b30003de0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.655+0000 7f1b48d6c700 1 -- 172.21.15.83:0/2687777182 >> 172.21.15.83:0/2687777182 conn(0x7f1b440b3780 msgr2=0x7f1b440b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.655+0000 7f1b48d6c700 1 -- 172.21.15.83:0/2687777182 shutdown_connections 2023-11-26T07:11:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.655+0000 7f1b48d6c700 1 -- 172.21.15.83:0/2687777182 wait complete. 2023-11-26T07:11:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.657+0000 7f1b48d6c700 1 Processor -- start 2023-11-26T07:11:56.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.657+0000 7f1b48d6c700 1 -- start start 2023-11-26T07:11:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.657+0000 7f1b48d6c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b30003a00 0x7f1b4415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.657+0000 7f1b48d6c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b441f7e70 0x7f1b44070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.657+0000 7f1b48d6c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b440cb2e0 0x7f1b44155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.657+0000 7f1b48d6c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b441f7180 con 0x7f1b440cb2e0 2023-11-26T07:11:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.657+0000 7f1b48d6c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4415caf0 con 0x7f1b441f7e70 2023-11-26T07:11:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.657+0000 7f1b48d6c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b441f7390 con 0x7f1b30003a00 2023-11-26T07:11:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b42d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b440cb2e0 0x7f1b44155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b42d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b440cb2e0 0x7f1b44155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35140/0 (socket says 172.21.15.83:35140) 2023-11-26T07:11:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b42d9d700 1 -- 172.21.15.83:0/557247589 learned_addr learned my addr 172.21.15.83:0/557247589 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b4259c700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b30003a00 0x7f1b4415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b41d9b700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b441f7e70 0x7f1b44070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b42d9d700 1 -- 172.21.15.83:0/557247589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b30003a00 msgr2=0x7f1b4415c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b42d9d700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b30003a00 0x7f1b4415c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b42d9d700 1 -- 172.21.15.83:0/557247589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b441f7e70 msgr2=0x7f1b44070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b42d9d700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b441f7e70 0x7f1b44070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b42d9d700 1 -- 172.21.15.83:0/557247589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b441f7670 con 0x7f1b440cb2e0 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b42d9d700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b440cb2e0 0x7f1b44155db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c001990 tx=0x7f1b3c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b3affd700 1 -- 172.21.15.83:0/557247589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c003c10 con 0x7f1b440cb2e0 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b3affd700 1 -- 172.21.15.83:0/557247589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c003d70 con 0x7f1b440cb2e0 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b48d6c700 1 -- 172.21.15.83:0/557247589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b441f7950 con 0x7f1b440cb2e0 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b3affd700 1 -- 172.21.15.83:0/557247589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c005310 con 0x7f1b440cb2e0 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.658+0000 7f1b48d6c700 1 -- 172.21.15.83:0/557247589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b44070590 con 0x7f1b440cb2e0 2023-11-26T07:11:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.659+0000 7f1b48d6c700 1 -- 172.21.15.83:0/557247589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b44070780 con 0x7f1b440cb2e0 2023-11-26T07:11:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.660+0000 7f1b3affd700 1 -- 172.21.15.83:0/557247589 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b3c005470 con 0x7f1b440cb2e0 2023-11-26T07:11:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.660+0000 7f1b3affd700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b2406aed0 0x7f1b2406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.660+0000 7f1b3affd700 1 -- 172.21.15.83:0/557247589 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b3c0895a0 con 0x7f1b440cb2e0 2023-11-26T07:11:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.660+0000 7f1b4259c700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b2406aed0 0x7f1b2406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.660+0000 7f1b4259c700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b2406aed0 0x7f1b2406d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c000c00 tx=0x7f1b2c007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.664+0000 7f1b3affd700 1 -- 172.21.15.83:0/557247589 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b3c054650 con 0x7f1b440cb2e0 2023-11-26T07:11:56.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:11:56.676 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:11:56.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.863+0000 7f1b48d6c700 1 -- 172.21.15.83:0/557247589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b44068fb0 con 0x7f1b440cb2e0 2023-11-26T07:11:56.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.863+0000 7f1b3affd700 1 -- 172.21.15.83:0/557247589 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b3c057c70 con 0x7f1b440cb2e0 2023-11-26T07:11:56.866 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:11:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 -- 172.21.15.83:0/557247589 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b2406aed0 msgr2=0x7f1b2406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b2406aed0 0x7f1b2406d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c000c00 tx=0x7f1b2c007000 comp rx=0 tx=0).stop 2023-11-26T07:11:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 -- 172.21.15.83:0/557247589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b440cb2e0 msgr2=0x7f1b44155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b440cb2e0 0x7f1b44155db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c001990 tx=0x7f1b3c00ee90 comp rx=0 tx=0).stop 2023-11-26T07:11:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 -- 172.21.15.83:0/557247589 shutdown_connections 2023-11-26T07:11:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b440cb2e0 0x7f1b44155db0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1b2406aed0 0x7f1b2406d390 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b30003a00 0x7f1b4415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 --2- 172.21.15.83:0/557247589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b441f7e70 0x7f1b44070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 -- 172.21.15.83:0/557247589 >> 172.21.15.83:0/557247589 conn(0x7f1b440b3780 msgr2=0x7f1b440b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 -- 172.21.15.83:0/557247589 shutdown_connections 2023-11-26T07:11:56.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:56.865+0000 7f1b48d6c700 1 -- 172.21.15.83:0/557247589 wait complete. 2023-11-26T07:11:56.941 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2023-11-26T07:11:56.942 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:11:56.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:11:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.059+0000 7ff4a1960700 1 Processor -- start 2023-11-26T07:11:57.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.060+0000 7ff4a1960700 1 -- start start 2023-11-26T07:11:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.060+0000 7ff4a1960700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff49c15a3b0 0x7ff49c0b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.060+0000 7ff4a1960700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff49c0cf360 0x7ff49c0b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.060+0000 7ff4a1960700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff49c0b3da0 0x7ff49c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.060+0000 7ff4a1960700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49c0cf8b0 con 0x7ff49c0b3da0 2023-11-26T07:11:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.060+0000 7ff4a1960700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49c15a790 con 0x7ff49c0cf360 2023-11-26T07:11:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.060+0000 7ff4a1960700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0b4c40 con 0x7ff49c15a3b0 2023-11-26T07:11:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.060+0000 7ff49b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff49c0b3da0 0x7ff49c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff49b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff49c0b3da0 0x7ff49c06dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35148/0 (socket says 172.21.15.83:35148) 2023-11-26T07:11:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff49b7fe700 1 -- 172.21.15.83:0/2872733635 learned_addr learned my addr 172.21.15.83:0/2872733635 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff49affd700 1 --2- 172.21.15.83:0/2872733635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff49c15a3b0 0x7ff49c0b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff49a7fc700 1 --2- 172.21.15.83:0/2872733635 >> v2:172.21.15.203:3300/0 conn(0x7ff49c0cf360 0x7ff49c0b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff49b7fe700 1 -- 172.21.15.83:0/2872733635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff49c15a3b0 msgr2=0x7ff49c0b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff49b7fe700 1 --2- 172.21.15.83:0/2872733635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff49c15a3b0 0x7ff49c0b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff49b7fe700 1 -- 172.21.15.83:0/2872733635 >> v2:172.21.15.203:3300/0 conn(0x7ff49c0cf360 msgr2=0x7ff49c0b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff49b7fe700 1 --2- 172.21.15.83:0/2872733635 >> v2:172.21.15.203:3300/0 conn(0x7ff49c0cf360 0x7ff49c0b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff49b7fe700 1 -- 172.21.15.83:0/2872733635 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c070c60 con 0x7ff49c0b3da0 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff49b7fe700 1 --2- 172.21.15.83:0/2872733635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff49c0b3da0 0x7ff49c06dd60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0d0a90 tx=0x7ff49400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581d803a11261fbf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff499ffb700 1 -- 172.21.15.83:0/2872733635 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400e070 con 0x7ff49c0b3da0 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff499ffb700 1 -- 172.21.15.83:0/2872733635 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49400b9c0 con 0x7ff49c0b3da0 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.061+0000 7ff499ffb700 1 -- 172.21.15.83:0/2872733635 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400bd70 con 0x7ff49c0b3da0 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.062+0000 7ff4a1960700 1 -- 172.21.15.83:0/2872733635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff49c0b3da0 msgr2=0x7ff49c06dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.062+0000 7ff4a1960700 1 --2- 172.21.15.83:0/2872733635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff49c0b3da0 0x7ff49c06dd60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0d0a90 tx=0x7ff49400b330 comp rx=0 tx=0).stop 2023-11-26T07:11:57.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.062+0000 7ff4a1960700 1 -- 172.21.15.83:0/2872733635 shutdown_connections 2023-11-26T07:11:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.062+0000 7ff4a1960700 1 --2- 172.21.15.83:0/2872733635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff49c0b3da0 0x7ff49c06dd60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.062+0000 7ff4a1960700 1 --2- 172.21.15.83:0/2872733635 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff49c15a3b0 0x7ff49c0b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.062+0000 7ff4a1960700 1 --2- 172.21.15.83:0/2872733635 >> v2:172.21.15.203:3300/0 conn(0x7ff49c0cf360 0x7ff49c0b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.062+0000 7ff4a1960700 1 -- 172.21.15.83:0/2872733635 >> 172.21.15.83:0/2872733635 conn(0x7ff49c0b1740 msgr2=0x7ff49c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.062+0000 7ff4a1960700 1 -- 172.21.15.83:0/2872733635 shutdown_connections 2023-11-26T07:11:57.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.062+0000 7ff4a1960700 1 -- 172.21.15.83:0/2872733635 wait complete. 2023-11-26T07:11:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.063+0000 7ff4a1960700 1 Processor -- start 2023-11-26T07:11:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.063+0000 7ff4a1960700 1 -- start start 2023-11-26T07:11:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff4a1960700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff49c0b3da0 0x7ff49c15d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff4a1960700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff49c0cf360 0x7ff49c1f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff4a1960700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff49c15a3b0 0x7ff49c06dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff4a1960700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c15ba50 con 0x7ff49c0cf360 2023-11-26T07:11:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff4a1960700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c1f71f0 con 0x7ff49c15a3b0 2023-11-26T07:11:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff4a1960700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c1f7450 con 0x7ff49c0b3da0 2023-11-26T07:11:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff49c0cf360 0x7ff49c1f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff49c0cf360 0x7ff49c1f6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35152/0 (socket says 172.21.15.83:35152) 2023-11-26T07:11:57.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49a7fc700 1 -- 172.21.15.83:0/1739121181 learned_addr learned my addr 172.21.15.83:0/1739121181 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49affd700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff49c0b3da0 0x7ff49c15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49a7fc700 1 -- 172.21.15.83:0/1739121181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff49c0b3da0 msgr2=0x7ff49c15d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49a7fc700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff49c0b3da0 0x7ff49c15d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49b7fe700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff49c15a3b0 0x7ff49c06dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49a7fc700 1 -- 172.21.15.83:0/1739121181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff49c15a3b0 msgr2=0x7ff49c06dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49a7fc700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff49c15a3b0 0x7ff49c06dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49a7fc700 1 -- 172.21.15.83:0/1739121181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff494009b30 con 0x7ff49c0cf360 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49b7fe700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff49c15a3b0 0x7ff49c06dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.064+0000 7ff49a7fc700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff49c0cf360 0x7ff49c1f6cb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00d680 tx=0x7ff48c00da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.065+0000 7ff483fff700 1 -- 172.21.15.83:0/1739121181 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c004440 con 0x7ff49c0cf360 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.065+0000 7ff483fff700 1 -- 172.21.15.83:0/1739121181 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c0045a0 con 0x7ff49c0cf360 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.065+0000 7ff483fff700 1 -- 172.21.15.83:0/1739121181 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0038c0 con 0x7ff49c0cf360 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.065+0000 7ff4a1960700 1 -- 172.21.15.83:0/1739121181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c1f7730 con 0x7ff49c0cf360 2023-11-26T07:11:57.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.065+0000 7ff4a1960700 1 -- 172.21.15.83:0/1739121181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c15bca0 con 0x7ff49c0cf360 2023-11-26T07:11:57.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.066+0000 7ff4a1960700 1 -- 172.21.15.83:0/1739121181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c0751e0 con 0x7ff49c0cf360 2023-11-26T07:11:57.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.066+0000 7ff483fff700 1 -- 172.21.15.83:0/1739121181 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff48c003a20 con 0x7ff49c0cf360 2023-11-26T07:11:57.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.066+0000 7ff483fff700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff48806aed0 0x7ff48806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.066+0000 7ff483fff700 1 -- 172.21.15.83:0/1739121181 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff48c0895f0 con 0x7ff49c0cf360 2023-11-26T07:11:57.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.066+0000 7ff49affd700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff48806aed0 0x7ff48806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.067+0000 7ff49affd700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff48806aed0 0x7ff48806d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff490009950 tx=0x7ff490008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:57.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.071+0000 7ff483fff700 1 -- 172.21.15.83:0/1739121181 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff48c0546b0 con 0x7ff49c0cf360 2023-11-26T07:11:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.265+0000 7ff4a1960700 1 -- 172.21.15.83:0/1739121181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff49c0b3100 con 0x7ff49c0cf360 2023-11-26T07:11:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.266+0000 7ff483fff700 1 -- 172.21.15.83:0/1739121181 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7ff48c057cd0 con 0x7ff49c0cf360 2023-11-26T07:11:57.268 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:11:57.269 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 -- 172.21.15.83:0/1739121181 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff48806aed0 msgr2=0x7ff48806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff48806aed0 0x7ff48806d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff490009950 tx=0x7ff490008040 comp rx=0 tx=0).stop 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 -- 172.21.15.83:0/1739121181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff49c0cf360 msgr2=0x7ff49c1f6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff49c0cf360 0x7ff49c1f6cb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00d680 tx=0x7ff48c00da40 comp rx=0 tx=0).stop 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 -- 172.21.15.83:0/1739121181 shutdown_connections 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff49c15a3b0 0x7ff49c06dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff48806aed0 0x7ff48806d390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff49c0b3da0 0x7ff49c15d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 --2- 172.21.15.83:0/1739121181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff49c0cf360 0x7ff49c1f6cb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 -- 172.21.15.83:0/1739121181 >> 172.21.15.83:0/1739121181 conn(0x7ff49c0b1740 msgr2=0x7ff49c1e2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 -- 172.21.15.83:0/1739121181 shutdown_connections 2023-11-26T07:11:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.268+0000 7ff4a1960700 1 -- 172.21.15.83:0/1739121181 wait complete. 2023-11-26T07:11:57.283 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:11:57.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:11:57.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.404+0000 7f87fa507700 1 Processor -- start 2023-11-26T07:11:57.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.404+0000 7f87fa507700 1 -- start start 2023-11-26T07:11:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87fa507700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f87f415e160 0x7f87f40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87fa507700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f40b3c60 con 0x7f87f40afc90 2023-11-26T07:11:57.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87fa507700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f40b58c0 con 0x7f87f40746a0 2023-11-26T07:11:57.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87fa507700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f40b4500 con 0x7f87f415e160 2023-11-26T07:11:57.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87f37fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f87f40afc90 0x7f87f4073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55198/0 (socket says 172.21.15.83:55198) 2023-11-26T07:11:57.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87f37fe700 1 -- 172.21.15.83:0/2158741984 learned_addr learned my addr 172.21.15.83:0/2158741984 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:57.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87f3fff700 1 --2- 172.21.15.83:0/2158741984 >> v2:172.21.15.203:3300/0 conn(0x7f87f415e160 0x7f87f40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243538004 0 0) 0x7f87f40b3c60 con 0x7f87f40afc90 2023-11-26T07:11:57.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87dc0035b0 con 0x7f87f40afc90 2023-11-26T07:11:57.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1428232432 0 0) 0x7f87dc0035b0 con 0x7f87f40afc90 2023-11-26T07:11:57.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 >> v2:172.21.15.203:3300/0 conn(0x7f87f415e160 msgr2=0x7f87f40b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:57.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87f2ffd700 1 --2- 172.21.15.83:0/2158741984 >> v2:172.21.15.203:3300/0 conn(0x7f87f415e160 0x7f87f40b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 >> v1:172.21.15.203:6789/0 conn(0x7f87f40746a0 legacy=0x7f87f406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f40b0140 con 0x7f87f40afc90 2023-11-26T07:11:57.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.405+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f87e4001670 con 0x7f87f40afc90 2023-11-26T07:11:57.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 >> v1:172.21.15.83:6789/0 conn(0x7f87f40afc90 legacy=0x7f87f4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f2ffd700 1 --2- 172.21.15.83:0/2158741984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87dc003a00 0x7f87dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f2ffd700 1 --2- 172.21.15.83:0/2158741984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87dc004850 0x7f87dc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f2ffd700 1 --2- 172.21.15.83:0/2158741984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc005210 0x7f87dc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f40b4500 con 0x7f87dc005210 2023-11-26T07:11:57.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc00c0b0 con 0x7f87dc004850 2023-11-26T07:11:57.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc00c270 con 0x7f87dc003a00 2023-11-26T07:11:57.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f8aa4700 1 --2- 172.21.15.83:0/2158741984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc005210 0x7f87dc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f3fff700 1 --2- 172.21.15.83:0/2158741984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87dc003a00 0x7f87dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f37fe700 1 --2- 172.21.15.83:0/2158741984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87dc004850 0x7f87dc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f8aa4700 1 -- 172.21.15.83:0/2158741984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87dc003a00 msgr2=0x7f87dc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:57.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f8aa4700 1 --2- 172.21.15.83:0/2158741984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87dc003a00 0x7f87dc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f8aa4700 1 -- 172.21.15.83:0/2158741984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87dc004850 msgr2=0x7f87dc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f8aa4700 1 --2- 172.21.15.83:0/2158741984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87dc004850 0x7f87dc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f8aa4700 1 -- 172.21.15.83:0/2158741984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0035b0 con 0x7f87dc005210 2023-11-26T07:11:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f8aa4700 1 -- 172.21.15.83:0/2158741984 reap_dead start 2023-11-26T07:11:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f8aa4700 1 --2- 172.21.15.83:0/2158741984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc005210 0x7f87dc00ba10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f87f4162dd0 tx=0x7f87dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c7f6132cba33f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec001db0 con 0x7f87dc005210 2023-11-26T07:11:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ec00cc70 con 0x7f87dc005210 2023-11-26T07:11:57.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.406+0000 7f87f2ffd700 1 -- 172.21.15.83:0/2158741984 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec003a00 con 0x7f87dc005210 2023-11-26T07:11:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.407+0000 7f87fa507700 1 -- 172.21.15.83:0/2158741984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc005210 msgr2=0x7f87dc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.407+0000 7f87fa507700 1 --2- 172.21.15.83:0/2158741984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc005210 0x7f87dc00ba10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f87f4162dd0 tx=0x7f87dc005880 comp rx=0 tx=0).stop 2023-11-26T07:11:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.407+0000 7f87fa507700 1 -- 172.21.15.83:0/2158741984 shutdown_connections 2023-11-26T07:11:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.407+0000 7f87fa507700 1 --2- 172.21.15.83:0/2158741984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87dc005210 0x7f87dc00ba10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.407+0000 7f87fa507700 1 -- 172.21.15.83:0/2158741984 >> 172.21.15.83:0/2158741984 conn(0x7f87f40d0bf0 msgr2=0x7f87f40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.407+0000 7f87fa507700 1 -- 172.21.15.83:0/2158741984 shutdown_connections 2023-11-26T07:11:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.407+0000 7f87fa507700 1 -- 172.21.15.83:0/2158741984 wait complete. 2023-11-26T07:11:57.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.408+0000 7f87fa507700 1 Processor -- start 2023-11-26T07:11:57.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.408+0000 7f87fa507700 1 -- start start 2023-11-26T07:11:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87fa507700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87dc005210 0x7f87f4159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87fa507700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87f415b010 0x7f87f41f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87fa507700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87f40cf7b0 0x7f87f40cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87fa507700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f415aae0 con 0x7f87f415b010 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87fa507700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f415a240 con 0x7f87dc005210 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87fa507700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f415a520 con 0x7f87f40cf7b0 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87f37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87f415b010 0x7f87f41f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87f37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87f415b010 0x7f87f41f76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35172/0 (socket says 172.21.15.83:35172) 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87f37fe700 1 -- 172.21.15.83:0/143941558 learned_addr learned my addr 172.21.15.83:0/143941558 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87f3fff700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87dc005210 0x7f87f4159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87f37fe700 1 -- 172.21.15.83:0/143941558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87f40cf7b0 msgr2=0x7f87f40cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87f37fe700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87f40cf7b0 0x7f87f40cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87f37fe700 1 -- 172.21.15.83:0/143941558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87dc005210 msgr2=0x7f87f4159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87f37fe700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87dc005210 0x7f87f4159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87f37fe700 1 -- 172.21.15.83:0/143941558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f415a800 con 0x7f87f415b010 2023-11-26T07:11:57.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87f3fff700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87dc005210 0x7f87f4159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:11:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.409+0000 7f87f37fe700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87f415b010 0x7f87f41f76f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f87e4007340 tx=0x7f87e4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.410+0000 7f87f0ff9700 1 -- 172.21.15.83:0/143941558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4001640 con 0x7f87f415b010 2023-11-26T07:11:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.410+0000 7f87f0ff9700 1 -- 172.21.15.83:0/143941558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e400f8d0 con 0x7f87f415b010 2023-11-26T07:11:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.410+0000 7f87fa507700 1 -- 172.21.15.83:0/143941558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f41f5160 con 0x7f87f415b010 2023-11-26T07:11:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.410+0000 7f87f0ff9700 1 -- 172.21.15.83:0/143941558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e400fca0 con 0x7f87f415b010 2023-11-26T07:11:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.410+0000 7f87fa507700 1 -- 172.21.15.83:0/143941558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f40d0180 con 0x7f87f415b010 2023-11-26T07:11:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.411+0000 7f87fa507700 1 -- 172.21.15.83:0/143941558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f41f4fa0 con 0x7f87f415b010 2023-11-26T07:11:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.411+0000 7f87f0ff9700 1 -- 172.21.15.83:0/143941558 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f87e401d460 con 0x7f87f415b010 2023-11-26T07:11:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.411+0000 7f87f0ff9700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f87e006aed0 0x7f87e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.412+0000 7f87f0ff9700 1 -- 172.21.15.83:0/143941558 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87e40895d0 con 0x7f87f415b010 2023-11-26T07:11:57.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.412+0000 7f87f3fff700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f87e006aed0 0x7f87e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.412+0000 7f87f3fff700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f87e006aed0 0x7f87e006d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009470 tx=0x7f87e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:57.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.416+0000 7f87f0ff9700 1 -- 172.21.15.83:0/143941558 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f87e4054680 con 0x7f87f415b010 2023-11-26T07:11:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.607+0000 7f87fa507700 1 -- 172.21.15.83:0/143941558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87f41f53a0 con 0x7f87f415b010 2023-11-26T07:11:57.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.608+0000 7f87f0ff9700 1 -- 172.21.15.83:0/143941558 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87f41f53a0 con 0x7f87f415b010 2023-11-26T07:11:57.610 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:11:57.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 -- 172.21.15.83:0/143941558 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f87e006aed0 msgr2=0x7f87e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f87e006aed0 0x7f87e006d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009470 tx=0x7f87e8008040 comp rx=0 tx=0).stop 2023-11-26T07:11:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 -- 172.21.15.83:0/143941558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87f415b010 msgr2=0x7f87f41f76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87f415b010 0x7f87f41f76f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f87e4007340 tx=0x7f87e4007700 comp rx=0 tx=0).stop 2023-11-26T07:11:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 -- 172.21.15.83:0/143941558 shutdown_connections 2023-11-26T07:11:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f87e006aed0 0x7f87e006d390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87f40cf7b0 0x7f87f40cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87dc005210 0x7f87f4159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 --2- 172.21.15.83:0/143941558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87f415b010 0x7f87f41f76f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 -- 172.21.15.83:0/143941558 >> 172.21.15.83:0/143941558 conn(0x7f87f40d0bf0 msgr2=0x7f87f40b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 -- 172.21.15.83:0/143941558 shutdown_connections 2023-11-26T07:11:57.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.610+0000 7f87fa507700 1 -- 172.21.15.83:0/143941558 wait complete. 2023-11-26T07:11:57.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:11:57.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.745+0000 7f5462330700 1 Processor -- start 2023-11-26T07:11:57.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.745+0000 7f5462330700 1 -- start start 2023-11-26T07:11:57.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.746+0000 7f5462330700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f545c151c30 0x7f545c0ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.746+0000 7f5462330700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0cf0b0 0x7f545c15e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.746+0000 7f5462330700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0b55a0 con 0x7f545c06fab0 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.746+0000 7f5462330700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c062900 con 0x7f545c151c30 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.746+0000 7f5462330700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0b52c0 con 0x7f545c0cf0b0 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.746+0000 7f545bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f545c151c30 0x7f545c0ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.746+0000 7f545bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f545c151c30 0x7f545c0ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35178/0 (socket says 172.21.15.83:35178) 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.746+0000 7f545bfff700 1 -- 172.21.15.83:0/2599792990 learned_addr learned my addr 172.21.15.83:0/2599792990 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f54608cd700 1 --2- 172.21.15.83:0/2599792990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0cf0b0 0x7f545c15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f545bfff700 1 -- 172.21.15.83:0/2599792990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0cf0b0 msgr2=0x7f545c15e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f545bfff700 1 --2- 172.21.15.83:0/2599792990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0cf0b0 0x7f545c15e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f545bfff700 1 -- 172.21.15.83:0/2599792990 >> v1:172.21.15.203:6789/0 conn(0x7f545c06fab0 legacy=0x7f545c0ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f545bfff700 1 -- 172.21.15.83:0/2599792990 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c062340 con 0x7f545c151c30 2023-11-26T07:11:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f545bfff700 1 --2- 172.21.15.83:0/2599792990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c151c30 0x7f545c0ccfb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f545000bb90 tx=0x7f545000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45876ff5e7724b61 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f545affd700 1 -- 172.21.15.83:0/2599792990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545000dd50 con 0x7f545c151c30 2023-11-26T07:11:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f545affd700 1 -- 172.21.15.83:0/2599792990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5450010470 con 0x7f545c151c30 2023-11-26T07:11:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f545affd700 1 -- 172.21.15.83:0/2599792990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450010800 con 0x7f545c151c30 2023-11-26T07:11:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f5462330700 1 -- 172.21.15.83:0/2599792990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c151c30 msgr2=0x7f545c0ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f5462330700 1 --2- 172.21.15.83:0/2599792990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c151c30 0x7f545c0ccfb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f545000bb90 tx=0x7f545000bea0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f5462330700 1 -- 172.21.15.83:0/2599792990 shutdown_connections 2023-11-26T07:11:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f5462330700 1 --2- 172.21.15.83:0/2599792990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0cf0b0 0x7f545c15e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f5462330700 1 --2- 172.21.15.83:0/2599792990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c151c30 0x7f545c0ccfb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f5462330700 1 -- 172.21.15.83:0/2599792990 >> 172.21.15.83:0/2599792990 conn(0x7f545c0b2050 msgr2=0x7f545c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f5462330700 1 -- 172.21.15.83:0/2599792990 shutdown_connections 2023-11-26T07:11:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.747+0000 7f5462330700 1 -- 172.21.15.83:0/2599792990 wait complete. 2023-11-26T07:11:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.749+0000 7f5462330700 1 Processor -- start 2023-11-26T07:11:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.749+0000 7f5462330700 1 -- start start 2023-11-26T07:11:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.749+0000 7f5462330700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c06fab0 0x7f545c0aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.749+0000 7f5462330700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f545c0cf0b0 0x7f545c0ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.749+0000 7f5462330700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c151c30 0x7f545c1e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.749+0000 7f5462330700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0a9b60 con 0x7f545c06fab0 2023-11-26T07:11:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.749+0000 7f5462330700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0a9e40 con 0x7f545c0cf0b0 2023-11-26T07:11:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.749+0000 7f5462330700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0aa120 con 0x7f545c151c30 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f545bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c06fab0 0x7f545c0aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f545bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c06fab0 0x7f545c0aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35184/0 (socket says 172.21.15.83:35184) 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f545bfff700 1 -- 172.21.15.83:0/3350292564 learned_addr learned my addr 172.21.15.83:0/3350292564 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f54608cd700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c151c30 0x7f545c1e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f545b7fe700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f545c0cf0b0 0x7f545c0ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f545bfff700 1 -- 172.21.15.83:0/3350292564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c151c30 msgr2=0x7f545c1e54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f545bfff700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c151c30 0x7f545c1e54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f545bfff700 1 -- 172.21.15.83:0/3350292564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f545c0cf0b0 msgr2=0x7f545c0ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f545bfff700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f545c0cf0b0 0x7f545c0ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f545bfff700 1 -- 172.21.15.83:0/3350292564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545000b7f0 con 0x7f545c06fab0 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f54608cd700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c151c30 0x7f545c1e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:57.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f545b7fe700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f545c0cf0b0 0x7f545c0ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f545bfff700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c06fab0 0x7f545c0aed10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f545000bee0 tx=0x7f5450004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f5458ff9700 1 -- 172.21.15.83:0/3350292564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450003b30 con 0x7f545c06fab0 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f5458ff9700 1 -- 172.21.15.83:0/3350292564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5450003c90 con 0x7f545c06fab0 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f5462330700 1 -- 172.21.15.83:0/3350292564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c0aa400 con 0x7f545c06fab0 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f5458ff9700 1 -- 172.21.15.83:0/3350292564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54500105e0 con 0x7f545c06fab0 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.750+0000 7f5462330700 1 -- 172.21.15.83:0/3350292564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c0af250 con 0x7f545c06fab0 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.751+0000 7f5462330700 1 -- 172.21.15.83:0/3350292564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c068d90 con 0x7f545c06fab0 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.752+0000 7f5458ff9700 1 -- 172.21.15.83:0/3350292564 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5450019400 con 0x7f545c06fab0 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.752+0000 7f5458ff9700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544806aed0 0x7f544806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.752+0000 7f5458ff9700 1 -- 172.21.15.83:0/3350292564 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f545008aa40 con 0x7f545c06fab0 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.752+0000 7f545b7fe700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544806aed0 0x7f544806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.752+0000 7f545b7fe700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544806aed0 0x7f544806d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f545c0732f0 tx=0x7f544c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:57.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.757+0000 7f5458ff9700 1 -- 172.21.15.83:0/3350292564 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f545008ce60 con 0x7f545c06fab0 2023-11-26T07:11:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.944+0000 7f5462330700 1 -- 172.21.15.83:0/3350292564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f545c068fb0 con 0x7f545c06fab0 2023-11-26T07:11:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.945+0000 7f5458ff9700 1 -- 172.21.15.83:0/3350292564 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5450055b80 con 0x7f545c06fab0 2023-11-26T07:11:57.947 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:11:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 -- 172.21.15.83:0/3350292564 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544806aed0 msgr2=0x7f544806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544806aed0 0x7f544806d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f545c0732f0 tx=0x7f544c009040 comp rx=0 tx=0).stop 2023-11-26T07:11:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 -- 172.21.15.83:0/3350292564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c06fab0 msgr2=0x7f545c0aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c06fab0 0x7f545c0aed10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f545000bee0 tx=0x7f5450004750 comp rx=0 tx=0).stop 2023-11-26T07:11:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 -- 172.21.15.83:0/3350292564 shutdown_connections 2023-11-26T07:11:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544806aed0 0x7f544806d390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c151c30 0x7f545c1e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c06fab0 0x7f545c0aed10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 --2- 172.21.15.83:0/3350292564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f545c0cf0b0 0x7f545c0ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 -- 172.21.15.83:0/3350292564 >> 172.21.15.83:0/3350292564 conn(0x7f545c0b2050 msgr2=0x7f545c0b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 -- 172.21.15.83:0/3350292564 shutdown_connections 2023-11-26T07:11:57.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:57.947+0000 7f5462330700 1 -- 172.21.15.83:0/3350292564 wait complete. 2023-11-26T07:11:57.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:11:58.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.081+0000 7fb841e5d700 1 Processor -- start 2023-11-26T07:11:58.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.082+0000 7fb841e5d700 1 -- start start 2023-11-26T07:11:58.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.082+0000 7fb841e5d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb83c0b93c0 0x7fb83c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.082+0000 7fb841e5d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb83c0b3250 0x7fb83c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.082+0000 7fb841e5d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83c0b6fb0 con 0x7fb83c15e320 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.082+0000 7fb841e5d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb83c0d4da0 con 0x7fb83c0b3250 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.082+0000 7fb841e5d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb83c0d2c10 con 0x7fb83c0b93c0 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.082+0000 7fb83bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb83c0b3250 0x7fb83c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.082+0000 7fb83b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb83c15e320 0x7fb83c0d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55232/0 (socket says 172.21.15.83:55232) 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.083+0000 7fb83b7fe700 1 -- 172.21.15.83:0/60442799 learned_addr learned my addr 172.21.15.83:0/60442799 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.083+0000 7fb83affd700 1 --2- 172.21.15.83:0/60442799 >> v2:172.21.15.203:3300/0 conn(0x7fb83c0b93c0 0x7fb83c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.083+0000 7fb83bfff700 1 -- 172.21.15.83:0/60442799 >> v2:172.21.15.203:3300/0 conn(0x7fb83c0b93c0 msgr2=0x7fb83c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.083+0000 7fb83bfff700 1 --2- 172.21.15.83:0/60442799 >> v2:172.21.15.203:3300/0 conn(0x7fb83c0b93c0 0x7fb83c06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.083+0000 7fb83bfff700 1 -- 172.21.15.83:0/60442799 >> v1:172.21.15.83:6789/0 conn(0x7fb83c15e320 legacy=0x7fb83c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:58.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.083+0000 7fb83bfff700 1 -- 172.21.15.83:0/60442799 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c0d2ef0 con 0x7fb83c0b3250 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.083+0000 7fb83a7fc700 1 -- 172.21.15.83:0/60442799 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 63848160 0 0) 0x7fb83c0b6fb0 con 0x7fb83c15e320 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.083+0000 7fb83bfff700 1 --2- 172.21.15.83:0/60442799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83c0b3250 0x7fb83c06b530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0b69c0 tx=0x7fb83400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fded77006a8d7e3f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.083+0000 7fb83a7fc700 1 -- 172.21.15.83:0/60442799 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83400e070 con 0x7fb83c0b3250 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.083+0000 7fb83a7fc700 1 -- 172.21.15.83:0/60442799 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83400bbd0 con 0x7fb83c0b3250 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.083+0000 7fb83a7fc700 1 -- 172.21.15.83:0/60442799 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834012430 con 0x7fb83c0b3250 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.084+0000 7fb841e5d700 1 -- 172.21.15.83:0/60442799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83c0b3250 msgr2=0x7fb83c06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.084+0000 7fb841e5d700 1 --2- 172.21.15.83:0/60442799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83c0b3250 0x7fb83c06b530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0b69c0 tx=0x7fb83400b330 comp rx=0 tx=0).stop 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.084+0000 7fb841e5d700 1 -- 172.21.15.83:0/60442799 shutdown_connections 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.084+0000 7fb841e5d700 1 --2- 172.21.15.83:0/60442799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83c0b3250 0x7fb83c06b530 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.084+0000 7fb841e5d700 1 --2- 172.21.15.83:0/60442799 >> v2:172.21.15.203:3300/0 conn(0x7fb83c0b93c0 0x7fb83c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.084+0000 7fb841e5d700 1 -- 172.21.15.83:0/60442799 >> 172.21.15.83:0/60442799 conn(0x7fb83c0d3e50 msgr2=0x7fb83c0d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.084+0000 7fb841e5d700 1 -- 172.21.15.83:0/60442799 shutdown_connections 2023-11-26T07:11:58.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.084+0000 7fb841e5d700 1 -- 172.21.15.83:0/60442799 wait complete. 2023-11-26T07:11:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.085+0000 7fb841e5d700 1 Processor -- start 2023-11-26T07:11:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.085+0000 7fb841e5d700 1 -- start start 2023-11-26T07:11:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb841e5d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb83c0b3250 0x7fb83c161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb841e5d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb83c0b93c0 0x7fb83c154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb841e5d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83c15e320 0x7fb83c15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb841e5d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c155240 con 0x7fb83c15e320 2023-11-26T07:11:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb841e5d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c15f410 con 0x7fb83c0b3250 2023-11-26T07:11:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb841e5d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c15f6b0 con 0x7fb83c0b93c0 2023-11-26T07:11:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb83bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83c15e320 0x7fb83c15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb83bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83c15e320 0x7fb83c15a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35202/0 (socket says 172.21.15.83:35202) 2023-11-26T07:11:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb83bfff700 1 -- 172.21.15.83:0/1792586263 learned_addr learned my addr 172.21.15.83:0/1792586263 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb83b7fe700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb83c0b3250 0x7fb83c161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb83affd700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb83c0b93c0 0x7fb83c154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb83bfff700 1 -- 172.21.15.83:0/1792586263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb83c0b93c0 msgr2=0x7fb83c154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb83bfff700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb83c0b93c0 0x7fb83c154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb83bfff700 1 -- 172.21.15.83:0/1792586263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb83c0b3250 msgr2=0x7fb83c161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb83bfff700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb83c0b3250 0x7fb83c161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb83bfff700 1 -- 172.21.15.83:0/1792586263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb834009a70 con 0x7fb83c15e320 2023-11-26T07:11:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.086+0000 7fb83bfff700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83c15e320 0x7fb83c15a4d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb83c074c90 tx=0x7fb834004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.087+0000 7fb833fff700 1 -- 172.21.15.83:0/1792586263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834012430 con 0x7fb83c15e320 2023-11-26T07:11:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.087+0000 7fb833fff700 1 -- 172.21.15.83:0/1792586263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb834020070 con 0x7fb83c15e320 2023-11-26T07:11:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.087+0000 7fb841e5d700 1 -- 172.21.15.83:0/1792586263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83c15f990 con 0x7fb83c15e320 2023-11-26T07:11:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.087+0000 7fb833fff700 1 -- 172.21.15.83:0/1792586263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834004280 con 0x7fb83c15e320 2023-11-26T07:11:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.087+0000 7fb841e5d700 1 -- 172.21.15.83:0/1792586263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83c155680 con 0x7fb83c15e320 2023-11-26T07:11:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.088+0000 7fb841e5d700 1 -- 172.21.15.83:0/1792586263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83c15ffe0 con 0x7fb83c15e320 2023-11-26T07:11:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.088+0000 7fb833fff700 1 -- 172.21.15.83:0/1792586263 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb8340043e0 con 0x7fb83c15e320 2023-11-26T07:11:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.088+0000 7fb833fff700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb81c06aed0 0x7fb81c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.088+0000 7fb833fff700 1 -- 172.21.15.83:0/1792586263 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb834022070 con 0x7fb83c15e320 2023-11-26T07:11:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.088+0000 7fb83b7fe700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb81c06aed0 0x7fb81c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.089+0000 7fb83b7fe700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb81c06aed0 0x7fb81c06d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb824000f60 tx=0x7fb824007e70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:58.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.093+0000 7fb833fff700 1 -- 172.21.15.83:0/1792586263 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb834056580 con 0x7fb83c15e320 2023-11-26T07:11:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.282+0000 7fb841e5d700 1 -- 172.21.15.83:0/1792586263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb83c0cfc10 con 0x7fb83c15e320 2023-11-26T07:11:58.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.283+0000 7fb833fff700 1 -- 172.21.15.83:0/1792586263 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb834059ba0 con 0x7fb83c15e320 2023-11-26T07:11:58.287 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:11:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 -- 172.21.15.83:0/1792586263 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb81c06aed0 msgr2=0x7fb81c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb81c06aed0 0x7fb81c06d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb824000f60 tx=0x7fb824007e70 comp rx=0 tx=0).stop 2023-11-26T07:11:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 -- 172.21.15.83:0/1792586263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83c15e320 msgr2=0x7fb83c15a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83c15e320 0x7fb83c15a4d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb83c074c90 tx=0x7fb834004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 -- 172.21.15.83:0/1792586263 shutdown_connections 2023-11-26T07:11:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83c15e320 0x7fb83c15a4d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb83c0b3250 0x7fb83c161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb81c06aed0 0x7fb81c06d390 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 --2- 172.21.15.83:0/1792586263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb83c0b93c0 0x7fb83c154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 -- 172.21.15.83:0/1792586263 >> 172.21.15.83:0/1792586263 conn(0x7fb83c0d3e50 msgr2=0x7fb83c0c7190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 -- 172.21.15.83:0/1792586263 shutdown_connections 2023-11-26T07:11:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.285+0000 7fb841e5d700 1 -- 172.21.15.83:0/1792586263 wait complete. 2023-11-26T07:11:58.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:11:58.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.417+0000 7fd1e34ab700 1 Processor -- start 2023-11-26T07:11:58.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.417+0000 7fd1e34ab700 1 -- start start 2023-11-26T07:11:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.418+0000 7fd1e34ab700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd1dc13e000 0x7fd1dc15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.418+0000 7fd1e34ab700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd1dc15efb0 0x7fd1dc0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.418+0000 7fd1e34ab700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1dc0b4290 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.418+0000 7fd1e34ab700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1dc15f430 con 0x7fd1dc13e000 2023-11-26T07:11:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.418+0000 7fd1e34ab700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1dc13e570 con 0x7fd1dc15efb0 2023-11-26T07:11:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.418+0000 7fd1e1247700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd1dc13e000 0x7fd1dc15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.418+0000 7fd1e1247700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd1dc13e000 0x7fd1dc15ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35206/0 (socket says 172.21.15.83:35206) 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.418+0000 7fd1e1247700 1 -- 172.21.15.83:0/3591401017 learned_addr learned my addr 172.21.15.83:0/3591401017 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.419+0000 7fd1e0a46700 1 --2- 172.21.15.83:0/3591401017 >> v2:172.21.15.203:3300/0 conn(0x7fd1dc15efb0 0x7fd1dc0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.419+0000 7fd1e1247700 1 -- 172.21.15.83:0/3591401017 >> v2:172.21.15.203:3300/0 conn(0x7fd1dc15efb0 msgr2=0x7fd1dc0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.419+0000 7fd1e1247700 1 --2- 172.21.15.83:0/3591401017 >> v2:172.21.15.203:3300/0 conn(0x7fd1dc15efb0 0x7fd1dc0cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.419+0000 7fd1e1247700 1 -- 172.21.15.83:0/3591401017 >> v1:172.21.15.203:6789/0 conn(0x7fd1dc0cfd40 legacy=0x7fd1dc15b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.419+0000 7fd1e1247700 1 -- 172.21.15.83:0/3591401017 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc0d05d0 con 0x7fd1dc13e000 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.419+0000 7fd1e1247700 1 --2- 172.21.15.83:0/3591401017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1dc13e000 0x7fd1dc15ea70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc15e9d0 tx=0x7fd1cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbcb5295f40f9a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.419+0000 7fd1dbfff700 1 -- 172.21.15.83:0/3591401017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc00e070 con 0x7fd1dc13e000 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.419+0000 7fd1dbfff700 1 -- 172.21.15.83:0/3591401017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc00b910 con 0x7fd1dc13e000 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.419+0000 7fd1dbfff700 1 -- 172.21.15.83:0/3591401017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc00bcc0 con 0x7fd1dc13e000 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.420+0000 7fd1e34ab700 1 -- 172.21.15.83:0/3591401017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1dc13e000 msgr2=0x7fd1dc15ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:58.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.420+0000 7fd1e34ab700 1 --2- 172.21.15.83:0/3591401017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1dc13e000 0x7fd1dc15ea70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc15e9d0 tx=0x7fd1cc00b330 comp rx=0 tx=0).stop 2023-11-26T07:11:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.420+0000 7fd1e34ab700 1 -- 172.21.15.83:0/3591401017 shutdown_connections 2023-11-26T07:11:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.420+0000 7fd1e34ab700 1 --2- 172.21.15.83:0/3591401017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1dc13e000 0x7fd1dc15ea70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.420+0000 7fd1e34ab700 1 --2- 172.21.15.83:0/3591401017 >> v2:172.21.15.203:3300/0 conn(0x7fd1dc15efb0 0x7fd1dc0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.420+0000 7fd1e34ab700 1 -- 172.21.15.83:0/3591401017 >> 172.21.15.83:0/3591401017 conn(0x7fd1dc072990 msgr2=0x7fd1dc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.420+0000 7fd1e34ab700 1 -- 172.21.15.83:0/3591401017 shutdown_connections 2023-11-26T07:11:58.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.420+0000 7fd1e34ab700 1 -- 172.21.15.83:0/3591401017 wait complete. 2023-11-26T07:11:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.421+0000 7fd1e34ab700 1 Processor -- start 2023-11-26T07:11:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.421+0000 7fd1e34ab700 1 -- start start 2023-11-26T07:11:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e34ab700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1dc0cfd40 0x7fd1dc1415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e34ab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1dc13e000 0x7fd1dc13a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e34ab700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1dc15efb0 0x7fd1dc0b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e34ab700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc143f10 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e34ab700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc141b10 con 0x7fd1dc13e000 2023-11-26T07:11:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e34ab700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc143290 con 0x7fd1dc15efb0 2023-11-26T07:11:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e1247700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1dc0cfd40 0x7fd1dc1415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e1247700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1dc0cfd40 0x7fd1dc1415d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35212/0 (socket says 172.21.15.83:35212) 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e1247700 1 -- 172.21.15.83:0/2727726138 learned_addr learned my addr 172.21.15.83:0/2727726138 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e0a46700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1dc13e000 0x7fd1dc13a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e1a48700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1dc15efb0 0x7fd1dc0b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e1247700 1 -- 172.21.15.83:0/2727726138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1dc15efb0 msgr2=0x7fd1dc0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e1247700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1dc15efb0 0x7fd1dc0b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e1247700 1 -- 172.21.15.83:0/2727726138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1dc13e000 msgr2=0x7fd1dc13a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e1247700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1dc13e000 0x7fd1dc13a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e1247700 1 -- 172.21.15.83:0/2727726138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc009b30 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e1247700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1dc0cfd40 0x7fd1dc1415d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc06c770 tx=0x7fd1cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1d9ffb700 1 -- 172.21.15.83:0/2727726138 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc004970 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1d9ffb700 1 -- 172.21.15.83:0/2727726138 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc0099b0 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1e34ab700 1 -- 172.21.15.83:0/2727726138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc143570 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.422+0000 7fd1d9ffb700 1 -- 172.21.15.83:0/2727726138 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc01c660 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.423+0000 7fd1e34ab700 1 -- 172.21.15.83:0/2727726138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc143b30 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.424+0000 7fd1e34ab700 1 -- 172.21.15.83:0/2727726138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1dc067f60 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.424+0000 7fd1d9ffb700 1 -- 172.21.15.83:0/2727726138 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1cc01c7c0 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.424+0000 7fd1d9ffb700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd1c806aed0 0x7fd1c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.424+0000 7fd1d9ffb700 1 -- 172.21.15.83:0/2727726138 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1cc08af50 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.424+0000 7fd1e0a46700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd1c806aed0 0x7fd1c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.425+0000 7fd1e0a46700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd1c806aed0 0x7fd1c806d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc13dc70 tx=0x7fd1d0009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.429+0000 7fd1d9ffb700 1 -- 172.21.15.83:0/2727726138 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1cc056090 con 0x7fd1dc0cfd40 2023-11-26T07:11:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:58.701+0000 7fd1e34ab700 1 -- 172.21.15.83:0/2727726138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7fd1dc0b26b0 con 0x7fd1dc0cfd40 2023-11-26T07:11:59.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.686+0000 7fd1d9ffb700 1 -- 172.21.15.83:0/2727726138 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v53) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd1cc0596b0 con 0x7fd1dc0cfd40 2023-11-26T07:11:59.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.689+0000 7fd1e34ab700 1 -- 172.21.15.83:0/2727726138 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd1c806aed0 msgr2=0x7fd1c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:59.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.689+0000 7fd1e34ab700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd1c806aed0 0x7fd1c806d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc13dc70 tx=0x7fd1d0009210 comp rx=0 tx=0).stop 2023-11-26T07:11:59.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.689+0000 7fd1e34ab700 1 -- 172.21.15.83:0/2727726138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1dc0cfd40 msgr2=0x7fd1dc1415d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.689+0000 7fd1e34ab700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1dc0cfd40 0x7fd1dc1415d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc06c770 tx=0x7fd1cc004ad0 comp rx=0 tx=0).stop 2023-11-26T07:11:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.689+0000 7fd1e34ab700 1 -- 172.21.15.83:0/2727726138 shutdown_connections 2023-11-26T07:11:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.689+0000 7fd1e34ab700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd1c806aed0 0x7fd1c806d390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.689+0000 7fd1e34ab700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1dc15efb0 0x7fd1dc0b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.689+0000 7fd1e34ab700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1dc0cfd40 0x7fd1dc1415d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.689+0000 7fd1e34ab700 1 --2- 172.21.15.83:0/2727726138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1dc13e000 0x7fd1dc13a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.689+0000 7fd1e34ab700 1 -- 172.21.15.83:0/2727726138 >> 172.21.15.83:0/2727726138 conn(0x7fd1dc072990 msgr2=0x7fd1dc06b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:59.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.689+0000 7fd1e34ab700 1 -- 172.21.15.83:0/2727726138 shutdown_connections 2023-11-26T07:11:59.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.690+0000 7fd1e34ab700 1 -- 172.21.15.83:0/2727726138 wait complete. 2023-11-26T07:11:59.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:11:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.833+0000 7f5ae9c83700 1 Processor -- start 2023-11-26T07:11:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.833+0000 7f5ae9c83700 1 -- start start 2023-11-26T07:11:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.834+0000 7f5ae9c83700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae40b5da0 0x7f5ae40afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.834+0000 7f5ae9c83700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae40b3c60 con 0x7f5ae415e160 2023-11-26T07:11:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.834+0000 7f5ae9c83700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae40b58c0 con 0x7f5ae40746a0 2023-11-26T07:11:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.834+0000 7f5ae9c83700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae40b4500 con 0x7f5ae40b5da0 2023-11-26T07:11:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.834+0000 7f5ae37fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5ae415e160 0x7f5ae4073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55260/0 (socket says 172.21.15.83:55260) 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.834+0000 7f5ae37fe700 1 -- 172.21.15.83:0/1988955854 learned_addr learned my addr 172.21.15.83:0/1988955854 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346571343 0 0) 0x7f5ae40b3c60 con 0x7f5ae415e160 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae3fff700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae40b5da0 0x7f5ae40afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad00035b0 con 0x7f5ae415e160 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2960559145 0 0) 0x7f5ad00035b0 con 0x7f5ae415e160 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae40b5da0 msgr2=0x7f5ae40afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae40b5da0 0x7f5ae40afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 >> v1:172.21.15.203:6789/0 conn(0x7f5ae40746a0 legacy=0x7f5ae406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae406bdc0 con 0x7f5ae415e160 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f5acc001670 con 0x7f5ae415e160 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 >> v1:172.21.15.83:6789/0 conn(0x7f5ae415e160 legacy=0x7f5ae4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae40b5da0 0x7f5ad0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad0003f70 0x7f5ad0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad0004d90 0x7f5ad0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae40b4500 con 0x7f5ad0004d90 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad000ba70 con 0x7f5ad0003f70 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad000bc30 con 0x7f5ae40b5da0 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae3fff700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad0004d90 0x7f5ad0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae37fe700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae40b5da0 0x7f5ad0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae2ffd700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad0003f70 0x7f5ad0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae3fff700 1 -- 172.21.15.83:0/1988955854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae40b5da0 msgr2=0x7f5ad0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae3fff700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae40b5da0 0x7f5ad0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae3fff700 1 -- 172.21.15.83:0/1988955854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad0003f70 msgr2=0x7f5ad0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae3fff700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ad0003f70 0x7f5ad0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae3fff700 1 -- 172.21.15.83:0/1988955854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad00035b0 con 0x7f5ad0004d90 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.835+0000 7f5ae3fff700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad0004d90 0x7f5ad0005210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009fc0 tx=0x7f5ad0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7133b8a6b0724ffd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.836+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc00ec30 con 0x7f5ad0004d90 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.836+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc00f040 con 0x7f5ad0004d90 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.836+0000 7f5ae27fc700 1 -- 172.21.15.83:0/1988955854 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc014480 con 0x7f5ad0004d90 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.836+0000 7f5ae9c83700 1 -- 172.21.15.83:0/1988955854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad0004d90 msgr2=0x7f5ad0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.836+0000 7f5ae9c83700 1 --2- 172.21.15.83:0/1988955854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad0004d90 0x7f5ad0005210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009fc0 tx=0x7f5ad0005880 comp rx=0 tx=0).stop 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.836+0000 7f5ae3fff700 1 -- 172.21.15.83:0/1988955854 reap_dead start 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.836+0000 7f5ae9c83700 1 -- 172.21.15.83:0/1988955854 shutdown_connections 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.836+0000 7f5ae9c83700 1 -- 172.21.15.83:0/1988955854 >> 172.21.15.83:0/1988955854 conn(0x7f5ae40d0bf0 msgr2=0x7f5ae40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.836+0000 7f5ae9c83700 1 -- 172.21.15.83:0/1988955854 shutdown_connections 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.836+0000 7f5ae9c83700 1 -- 172.21.15.83:0/1988955854 wait complete. 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.837+0000 7f5ae9c83700 1 Processor -- start 2023-11-26T07:11:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.838+0000 7f5ae9c83700 1 -- start start 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.838+0000 7f5ae9c83700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae415c470 0x7f5ae41610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.838+0000 7f5ae9c83700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae41f4710 0x7f5ae415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.838+0000 7f5ae9c83700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae415c850 0x7f5ae41553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.838+0000 7f5ae9c83700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae4160010 con 0x7f5ae415c470 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.838+0000 7f5ae9c83700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae41602f0 con 0x7f5ae415c850 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.838+0000 7f5ae9c83700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae41605d0 con 0x7f5ae41f4710 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.838+0000 7f5ae37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae415c470 0x7f5ae41610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.838+0000 7f5ae37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae415c470 0x7f5ae41610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35232/0 (socket says 172.21.15.83:35232) 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.838+0000 7f5ae37fe700 1 -- 172.21.15.83:0/3446297451 learned_addr learned my addr 172.21.15.83:0/3446297451 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae2ffd700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae41f4710 0x7f5ae415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae3fff700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae415c850 0x7f5ae41553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae37fe700 1 -- 172.21.15.83:0/3446297451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae41f4710 msgr2=0x7f5ae415f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae37fe700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae41f4710 0x7f5ae415f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae37fe700 1 -- 172.21.15.83:0/3446297451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae415c850 msgr2=0x7f5ae41553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae37fe700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae415c850 0x7f5ae41553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae37fe700 1 -- 172.21.15.83:0/3446297451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae41608b0 con 0x7f5ae415c470 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae2ffd700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae41f4710 0x7f5ae415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae3fff700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae415c850 0x7f5ae41553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae37fe700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae415c470 0x7f5ae41610b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5acc007290 tx=0x7f5acc007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5adbfff700 1 -- 172.21.15.83:0/3446297451 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc00f8d0 con 0x7f5ae415c470 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5adbfff700 1 -- 172.21.15.83:0/3446297451 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc00fa50 con 0x7f5ae415c470 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae9c83700 1 -- 172.21.15.83:0/3446297451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae4160b90 con 0x7f5ae415c470 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5adbfff700 1 -- 172.21.15.83:0/3446297451 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc00fe00 con 0x7f5ae415c470 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.839+0000 7f5ae9c83700 1 -- 172.21.15.83:0/3446297451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae41558f0 con 0x7f5ae415c470 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.840+0000 7f5ae9c83700 1 -- 172.21.15.83:0/3446297451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae4160f00 con 0x7f5ae415c470 2023-11-26T07:11:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.840+0000 7f5adbfff700 1 -- 172.21.15.83:0/3446297451 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5acc01d460 con 0x7f5ae415c470 2023-11-26T07:11:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.841+0000 7f5adbfff700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ac406aed0 0x7f5ac406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:11:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.841+0000 7f5adbfff700 1 -- 172.21.15.83:0/3446297451 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5acc0896c0 con 0x7f5ae415c470 2023-11-26T07:11:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.841+0000 7f5ae2ffd700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ac406aed0 0x7f5ac406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:11:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.841+0000 7f5ae2ffd700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ac406aed0 0x7f5ac406d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5ae41f4b10 tx=0x7f5ad4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:11:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:11:59.845+0000 7f5adbfff700 1 -- 172.21.15.83:0/3446297451 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5acc054800 con 0x7f5ae415c470 2023-11-26T07:12:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.035+0000 7f5ae9c83700 1 -- 172.21.15.83:0/3446297451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5ae40682a0 con 0x7f5ae415c470 2023-11-26T07:12:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.038+0000 7f5adbfff700 1 -- 172.21.15.83:0/3446297451 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5acc057e20 con 0x7f5ae415c470 2023-11-26T07:12:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 -- 172.21.15.83:0/3446297451 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ac406aed0 msgr2=0x7f5ac406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ac406aed0 0x7f5ac406d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5ae41f4b10 tx=0x7f5ad4009040 comp rx=0 tx=0).stop 2023-11-26T07:12:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 -- 172.21.15.83:0/3446297451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae415c470 msgr2=0x7f5ae41610b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae415c470 0x7f5ae41610b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5acc007290 tx=0x7f5acc007650 comp rx=0 tx=0).stop 2023-11-26T07:12:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 -- 172.21.15.83:0/3446297451 shutdown_connections 2023-11-26T07:12:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae415c850 0x7f5ae41553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ae415c470 0x7f5ae41610b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ac406aed0 0x7f5ac406d390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 --2- 172.21.15.83:0/3446297451 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae41f4710 0x7f5ae415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 -- 172.21.15.83:0/3446297451 >> 172.21.15.83:0/3446297451 conn(0x7f5ae40d0bf0 msgr2=0x7f5ae4073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 -- 172.21.15.83:0/3446297451 shutdown_connections 2023-11-26T07:12:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.040+0000 7f5ae9c83700 1 -- 172.21.15.83:0/3446297451 wait complete. 2023-11-26T07:12:00.055 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:12:00.055 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:12:00.095 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:12:00.141 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:12:00.142 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:00.142 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:00.159 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:00.159 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:00.216 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:00.216 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:00.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:00.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.431+0000 7f14bc157700 1 Processor -- start 2023-11-26T07:12:00.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.431+0000 7f14bc157700 1 -- start start 2023-11-26T07:12:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14bc157700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14b41446a0 0x7f14b40b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14bc157700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14b406c710 0x7f14b40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14bc157700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b40af950 con 0x7f14b40adf00 2023-11-26T07:12:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14bc157700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b4144a80 con 0x7f14b41446a0 2023-11-26T07:12:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14bc157700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40afc30 con 0x7f14b406c710 2023-11-26T07:12:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14b9ef3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14b41446a0 0x7f14b40b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14b9ef3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14b41446a0 0x7f14b40b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35240/0 (socket says 172.21.15.83:35240) 2023-11-26T07:12:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14b9ef3700 1 -- 172.21.15.83:0/387504996 learned_addr learned my addr 172.21.15.83:0/387504996 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14b9ef3700 1 -- 172.21.15.83:0/387504996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14b406c710 msgr2=0x7f14b40b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14b9ef3700 1 --2- 172.21.15.83:0/387504996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14b406c710 0x7f14b40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14b9ef3700 1 -- 172.21.15.83:0/387504996 >> v1:172.21.15.203:6789/0 conn(0x7f14b40adf00 legacy=0x7f14b406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.432+0000 7f14b9ef3700 1 -- 172.21.15.83:0/387504996 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b40aff10 con 0x7f14b41446a0 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.433+0000 7f14b9ef3700 1 --2- 172.21.15.83:0/387504996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b41446a0 0x7f14b40b5ae0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f14a40098f0 tx=0x7f14a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6be05b9cfdf4287 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.433+0000 7f14b8ef1700 1 -- 172.21.15.83:0/387504996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4004230 con 0x7f14b41446a0 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.433+0000 7f14b8ef1700 1 -- 172.21.15.83:0/387504996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a4004390 con 0x7f14b41446a0 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.433+0000 7f14b8ef1700 1 -- 172.21.15.83:0/387504996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a40036a0 con 0x7f14b41446a0 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.433+0000 7f14bc157700 1 -- 172.21.15.83:0/387504996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b41446a0 msgr2=0x7f14b40b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.433+0000 7f14bc157700 1 --2- 172.21.15.83:0/387504996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b41446a0 0x7f14b40b5ae0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f14a40098f0 tx=0x7f14a4009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.433+0000 7f14bc157700 1 -- 172.21.15.83:0/387504996 shutdown_connections 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.433+0000 7f14bc157700 1 --2- 172.21.15.83:0/387504996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14b406c710 0x7f14b40b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.433+0000 7f14bc157700 1 --2- 172.21.15.83:0/387504996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b41446a0 0x7f14b40b5ae0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.433+0000 7f14bc157700 1 -- 172.21.15.83:0/387504996 >> 172.21.15.83:0/387504996 conn(0x7f14b4071b90 msgr2=0x7f14b4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.433+0000 7f14bc157700 1 -- 172.21.15.83:0/387504996 shutdown_connections 2023-11-26T07:12:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.434+0000 7f14bc157700 1 -- 172.21.15.83:0/387504996 wait complete. 2023-11-26T07:12:00.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.435+0000 7f14bc157700 1 Processor -- start 2023-11-26T07:12:00.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.435+0000 7f14bc157700 1 -- start start 2023-11-26T07:12:00.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14bc157700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14b406c710 0x7f14b4143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14bc157700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b40adf00 0x7f14b4143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14bc157700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14b41446a0 0x7f14b41380e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14bc157700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40cf3d0 con 0x7f14b40adf00 2023-11-26T07:12:00.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14bc157700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40cf6b0 con 0x7f14b406c710 2023-11-26T07:12:00.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14bc157700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40cf990 con 0x7f14b41446a0 2023-11-26T07:12:00.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14b96f2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b40adf00 0x7f14b4143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:00.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14b96f2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b40adf00 0x7f14b4143b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35246/0 (socket says 172.21.15.83:35246) 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14b96f2700 1 -- 172.21.15.83:0/3910853616 learned_addr learned my addr 172.21.15.83:0/3910853616 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14b9ef3700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14b406c710 0x7f14b4143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14b96f2700 1 -- 172.21.15.83:0/3910853616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14b41446a0 msgr2=0x7f14b41380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14b96f2700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14b41446a0 0x7f14b41380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14b96f2700 1 -- 172.21.15.83:0/3910853616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14b406c710 msgr2=0x7f14b4143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14b96f2700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14b406c710 0x7f14b4143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14b96f2700 1 -- 172.21.15.83:0/3910853616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a4009550 con 0x7f14b40adf00 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14b9ef3700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14b406c710 0x7f14b4143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.436+0000 7f14b96f2700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b40adf00 0x7f14b4143b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c450 tx=0x7f14b000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.437+0000 7f14aa7fc700 1 -- 172.21.15.83:0/3910853616 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0016070 con 0x7f14b40adf00 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.437+0000 7f14aa7fc700 1 -- 172.21.15.83:0/3910853616 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b0006860 con 0x7f14b40adf00 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.437+0000 7f14bc157700 1 -- 172.21.15.83:0/3910853616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b40cfc70 con 0x7f14b40adf00 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.437+0000 7f14aa7fc700 1 -- 172.21.15.83:0/3910853616 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0006bf0 con 0x7f14b40adf00 2023-11-26T07:12:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.437+0000 7f14bc157700 1 -- 172.21.15.83:0/3910853616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b4138620 con 0x7f14b40adf00 2023-11-26T07:12:00.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.437+0000 7f14bc157700 1 -- 172.21.15.83:0/3910853616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b40753f0 con 0x7f14b40adf00 2023-11-26T07:12:00.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.438+0000 7f14aa7fc700 1 -- 172.21.15.83:0/3910853616 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14b0006d50 con 0x7f14b40adf00 2023-11-26T07:12:00.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.438+0000 7f14aa7fc700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f149806aed0 0x7f149806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.438+0000 7f14aa7fc700 1 -- 172.21.15.83:0/3910853616 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14b0088cd0 con 0x7f14b40adf00 2023-11-26T07:12:00.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.438+0000 7f14b9ef3700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f149806aed0 0x7f149806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:00.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.439+0000 7f14b9ef3700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f149806aed0 0x7f149806d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f14b4154120 tx=0x7f14a4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.443+0000 7f14aa7fc700 1 -- 172.21.15.83:0/3910853616 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14b0053d80 con 0x7f14b40adf00 2023-11-26T07:12:00.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.641+0000 7f14bc157700 1 -- 172.21.15.83:0/3910853616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14b40ae6a0 con 0x7f14b40adf00 2023-11-26T07:12:00.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.642+0000 7f14aa7fc700 1 -- 172.21.15.83:0/3910853616 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f14b00573a0 con 0x7f14b40adf00 2023-11-26T07:12:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:00.645 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:00.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.644+0000 7f14bc157700 1 -- 172.21.15.83:0/3910853616 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f149806aed0 msgr2=0x7f149806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.644+0000 7f14bc157700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f149806aed0 0x7f149806d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f14b4154120 tx=0x7f14a4008030 comp rx=0 tx=0).stop 2023-11-26T07:12:00.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.644+0000 7f14bc157700 1 -- 172.21.15.83:0/3910853616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b40adf00 msgr2=0x7f14b4143b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.644+0000 7f14bc157700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b40adf00 0x7f14b4143b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c450 tx=0x7f14b000c810 comp rx=0 tx=0).stop 2023-11-26T07:12:00.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.644+0000 7f14bc157700 1 -- 172.21.15.83:0/3910853616 shutdown_connections 2023-11-26T07:12:00.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.644+0000 7f14bc157700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f149806aed0 0x7f149806d390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.644+0000 7f14bc157700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14b41446a0 0x7f14b41380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.644+0000 7f14bc157700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14b406c710 0x7f14b4143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.644+0000 7f14bc157700 1 --2- 172.21.15.83:0/3910853616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14b40adf00 0x7f14b4143b40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.644+0000 7f14bc157700 1 -- 172.21.15.83:0/3910853616 >> 172.21.15.83:0/3910853616 conn(0x7f14b4071b90 msgr2=0x7f14b40b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:00.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.645+0000 7f14bc157700 1 -- 172.21.15.83:0/3910853616 shutdown_connections 2023-11-26T07:12:00.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.645+0000 7f14bc157700 1 -- 172.21.15.83:0/3910853616 wait complete. 2023-11-26T07:12:00.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:12:00.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:12:00.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.781+0000 7fa019e68700 1 Processor -- start 2023-11-26T07:12:00.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.781+0000 7fa019e68700 1 -- start start 2023-11-26T07:12:00.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.781+0000 7fa019e68700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa014158570 0x7fa014161cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.781+0000 7fa019e68700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa014158950 0x7fa0140b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.781+0000 7fa019e68700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01406d3d0 con 0x7fa01415f400 2023-11-26T07:12:00.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.781+0000 7fa019e68700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0140ae870 con 0x7fa014158570 2023-11-26T07:12:00.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.781+0000 7fa019e68700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141625d0 con 0x7fa014158950 2023-11-26T07:12:00.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa0137fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa014158570 0x7fa014161cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:00.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa013fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa01415f400 0x7fa014071370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55296/0 (socket says 172.21.15.83:55296) 2023-11-26T07:12:00.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa013fff700 1 -- 172.21.15.83:0/1310991871 learned_addr learned my addr 172.21.15.83:0/1310991871 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:00.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa012ffd700 1 --2- 172.21.15.83:0/1310991871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa014158950 0x7fa0140b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa0137fe700 1 -- 172.21.15.83:0/1310991871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa014158950 msgr2=0x7fa0140b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa0137fe700 1 --2- 172.21.15.83:0/1310991871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa014158950 0x7fa0140b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa0137fe700 1 -- 172.21.15.83:0/1310991871 >> v1:172.21.15.83:6789/0 conn(0x7fa01415f400 legacy=0x7fa014071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa0137fe700 1 -- 172.21.15.83:0/1310991871 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0141628b0 con 0x7fa014158570 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa012ffd700 1 --2- 172.21.15.83:0/1310991871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa014158950 0x7fa0140b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa0137fe700 1 --2- 172.21.15.83:0/1310991871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014158570 0x7fa014161cd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa004009840 tx=0x7fa004009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d762f6b066db0df1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa0127fc700 1 -- 172.21.15.83:0/1310991871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004011830 con 0x7fa014158570 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa0127fc700 1 -- 172.21.15.83:0/1310991871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa004011990 con 0x7fa014158570 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.782+0000 7fa0127fc700 1 -- 172.21.15.83:0/1310991871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004011d20 con 0x7fa014158570 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.783+0000 7fa019e68700 1 -- 172.21.15.83:0/1310991871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014158570 msgr2=0x7fa014161cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.783+0000 7fa019e68700 1 --2- 172.21.15.83:0/1310991871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014158570 0x7fa014161cd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa004009840 tx=0x7fa004009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.783+0000 7fa019e68700 1 -- 172.21.15.83:0/1310991871 shutdown_connections 2023-11-26T07:12:00.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.783+0000 7fa019e68700 1 --2- 172.21.15.83:0/1310991871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014158570 0x7fa014161cd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.783+0000 7fa019e68700 1 --2- 172.21.15.83:0/1310991871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa014158950 0x7fa0140b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.783+0000 7fa019e68700 1 -- 172.21.15.83:0/1310991871 >> 172.21.15.83:0/1310991871 conn(0x7fa01406aff0 msgr2=0x7fa01406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.783+0000 7fa019e68700 1 -- 172.21.15.83:0/1310991871 shutdown_connections 2023-11-26T07:12:00.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.783+0000 7fa019e68700 1 -- 172.21.15.83:0/1310991871 wait complete. 2023-11-26T07:12:00.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.784+0000 7fa019e68700 1 Processor -- start 2023-11-26T07:12:00.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.784+0000 7fa019e68700 1 -- start start 2023-11-26T07:12:00.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa019e68700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa014158570 0x7fa014155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa019e68700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014158950 0x7fa0141448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa019e68700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01415f400 0x7fa01415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa019e68700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01415cf60 con 0x7fa014158950 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa019e68700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa014144ff0 con 0x7fa014158570 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa019e68700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141452d0 con 0x7fa01415f400 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa012ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014158950 0x7fa0141448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa012ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014158950 0x7fa0141448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35260/0 (socket says 172.21.15.83:35260) 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa012ffd700 1 -- 172.21.15.83:0/1606198814 learned_addr learned my addr 172.21.15.83:0/1606198814 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa0137fe700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa014158570 0x7fa014155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa013fff700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01415f400 0x7fa01415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa012ffd700 1 -- 172.21.15.83:0/1606198814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01415f400 msgr2=0x7fa01415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa012ffd700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01415f400 0x7fa01415adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa012ffd700 1 -- 172.21.15.83:0/1606198814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa014158570 msgr2=0x7fa014155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa012ffd700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa014158570 0x7fa014155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa012ffd700 1 -- 172.21.15.83:0/1606198814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0040094a0 con 0x7fa014158950 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa0137fe700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa014158570 0x7fa014155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.785+0000 7fa012ffd700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014158950 0x7fa0141448d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa00800c7f0 tx=0x7fa00800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.786+0000 7f9ffbfff700 1 -- 172.21.15.83:0/1606198814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008011800 con 0x7fa014158950 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.786+0000 7f9ffbfff700 1 -- 172.21.15.83:0/1606198814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa008011960 con 0x7fa014158950 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.786+0000 7fa019e68700 1 -- 172.21.15.83:0/1606198814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0141455b0 con 0x7fa014158950 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.786+0000 7f9ffbfff700 1 -- 172.21.15.83:0/1606198814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008011cf0 con 0x7fa014158950 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.786+0000 7fa019e68700 1 -- 172.21.15.83:0/1606198814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0141552c0 con 0x7fa014158950 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.786+0000 7fa019e68700 1 -- 172.21.15.83:0/1606198814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01415e490 con 0x7fa014158950 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.787+0000 7f9ffbfff700 1 -- 172.21.15.83:0/1606198814 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa008018360 con 0x7fa014158950 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.787+0000 7f9ffbfff700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa00006aed0 0x7fa00006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:00.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.787+0000 7f9ffbfff700 1 -- 172.21.15.83:0/1606198814 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa008089650 con 0x7fa014158950 2023-11-26T07:12:00.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.787+0000 7fa0137fe700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa00006aed0 0x7fa00006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:00.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.788+0000 7fa0137fe700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa00006aed0 0x7fa00006d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa0040090b0 tx=0x7fa004008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:00.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.792+0000 7f9ffbfff700 1 -- 172.21.15.83:0/1606198814 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa008054700 con 0x7fa014158950 2023-11-26T07:12:00.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.978+0000 7fa019e68700 1 -- 172.21.15.83:0/1606198814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa014145ad0 con 0x7fa014158950 2023-11-26T07:12:00.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.979+0000 7f9ffbfff700 1 -- 172.21.15.83:0/1606198814 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa014145ad0 con 0x7fa014158950 2023-11-26T07:12:00.981 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:12:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 -- 172.21.15.83:0/1606198814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa00006aed0 msgr2=0x7fa00006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa00006aed0 0x7fa00006d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa0040090b0 tx=0x7fa004008030 comp rx=0 tx=0).stop 2023-11-26T07:12:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 -- 172.21.15.83:0/1606198814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014158950 msgr2=0x7fa0141448d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014158950 0x7fa0141448d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa00800c7f0 tx=0x7fa00800cbb0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 -- 172.21.15.83:0/1606198814 shutdown_connections 2023-11-26T07:12:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa00006aed0 0x7fa00006d390 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01415f400 0x7fa01415adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa014158570 0x7fa014155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 --2- 172.21.15.83:0/1606198814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa014158950 0x7fa0141448d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:00.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 -- 172.21.15.83:0/1606198814 >> 172.21.15.83:0/1606198814 conn(0x7fa01406aff0 msgr2=0x7fa014160530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 -- 172.21.15.83:0/1606198814 shutdown_connections 2023-11-26T07:12:00.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:00.981+0000 7fa019e68700 1 -- 172.21.15.83:0/1606198814 wait complete. 2023-11-26T07:12:00.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:12:01.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.116+0000 7fb6805d2700 1 Processor -- start 2023-11-26T07:12:01.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.116+0000 7fb6805d2700 1 -- start start 2023-11-26T07:12:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb6805d2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb67815ba30 0x7fb67806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb6805d2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb678162800 0x7fb67806c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb6805d2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67815f280 con 0x7fb67815e540 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb6805d2700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67815f090 con 0x7fb678162800 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb6805d2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67815f560 con 0x7fb67815ba30 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67db6d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb678162800 0x7fb67806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67eb6f700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb67815e540 0x7fb67815e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55310/0 (socket says 172.21.15.83:55310) 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67eb6f700 1 -- 172.21.15.83:0/813567172 learned_addr learned my addr 172.21.15.83:0/813567172 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67e36e700 1 --2- 172.21.15.83:0/813567172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb67815ba30 0x7fb67806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67db6d700 1 -- 172.21.15.83:0/813567172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb67815ba30 msgr2=0x7fb67806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67db6d700 1 --2- 172.21.15.83:0/813567172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb67815ba30 0x7fb67806c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67db6d700 1 -- 172.21.15.83:0/813567172 >> v1:172.21.15.83:6789/0 conn(0x7fb67815e540 legacy=0x7fb67815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67db6d700 1 -- 172.21.15.83:0/813567172 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb678161d20 con 0x7fb678162800 2023-11-26T07:12:01.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67d36c700 1 -- 172.21.15.83:0/813567172 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400985693 0 0) 0x7fb67815f280 con 0x7fb67815e540 2023-11-26T07:12:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67db6d700 1 --2- 172.21.15.83:0/813567172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb678162800 0x7fb67806c540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb674009840 tx=0x7fb674009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826ca625cc6f88aa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67d36c700 1 -- 172.21.15.83:0/813567172 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674011830 con 0x7fb678162800 2023-11-26T07:12:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67d36c700 1 -- 172.21.15.83:0/813567172 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb674011990 con 0x7fb678162800 2023-11-26T07:12:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.117+0000 7fb67d36c700 1 -- 172.21.15.83:0/813567172 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674011d20 con 0x7fb678162800 2023-11-26T07:12:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.118+0000 7fb6805d2700 1 -- 172.21.15.83:0/813567172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb678162800 msgr2=0x7fb67806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.118+0000 7fb6805d2700 1 --2- 172.21.15.83:0/813567172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb678162800 0x7fb67806c540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb674009840 tx=0x7fb674009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.118+0000 7fb6805d2700 1 -- 172.21.15.83:0/813567172 shutdown_connections 2023-11-26T07:12:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.118+0000 7fb6805d2700 1 --2- 172.21.15.83:0/813567172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb67815ba30 0x7fb67806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.118+0000 7fb6805d2700 1 --2- 172.21.15.83:0/813567172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb678162800 0x7fb67806c540 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.118+0000 7fb6805d2700 1 -- 172.21.15.83:0/813567172 >> 172.21.15.83:0/813567172 conn(0x7fb678071a70 msgr2=0x7fb678071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:01.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.118+0000 7fb6805d2700 1 -- 172.21.15.83:0/813567172 shutdown_connections 2023-11-26T07:12:01.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.118+0000 7fb6805d2700 1 -- 172.21.15.83:0/813567172 wait complete. 2023-11-26T07:12:01.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.119+0000 7fb6805d2700 1 Processor -- start 2023-11-26T07:12:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.119+0000 7fb6805d2700 1 -- start start 2023-11-26T07:12:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb6805d2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb67815ba30 0x7fb6780cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb6805d2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb67815e540 0x7fb67815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb6805d2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb678162800 0x7fb678144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb6805d2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6780b5780 con 0x7fb67815e540 2023-11-26T07:12:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb6805d2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6780cd310 con 0x7fb67815ba30 2023-11-26T07:12:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb6805d2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6780cdfc0 con 0x7fb678162800 2023-11-26T07:12:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67db6d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb67815e540 0x7fb67815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67db6d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb67815e540 0x7fb67815aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35274/0 (socket says 172.21.15.83:35274) 2023-11-26T07:12:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67db6d700 1 -- 172.21.15.83:0/699895239 learned_addr learned my addr 172.21.15.83:0/699895239 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:01.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67e36e700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb67815ba30 0x7fb6780cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67eb6f700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb678162800 0x7fb678144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67db6d700 1 -- 172.21.15.83:0/699895239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb678162800 msgr2=0x7fb678144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67db6d700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb678162800 0x7fb678144f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67db6d700 1 -- 172.21.15.83:0/699895239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb67815ba30 msgr2=0x7fb6780cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67db6d700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb67815ba30 0x7fb6780cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67db6d700 1 -- 172.21.15.83:0/699895239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6740094a0 con 0x7fb67815e540 2023-11-26T07:12:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67eb6f700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb678162800 0x7fb678144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb67db6d700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb67815e540 0x7fb67815aea0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb674014040 tx=0x7fb674004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.120+0000 7fb66effd700 1 -- 172.21.15.83:0/699895239 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740038a0 con 0x7fb67815e540 2023-11-26T07:12:01.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.121+0000 7fb66effd700 1 -- 172.21.15.83:0/699895239 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67401f070 con 0x7fb67815e540 2023-11-26T07:12:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.121+0000 7fb6805d2700 1 -- 172.21.15.83:0/699895239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67815bf90 con 0x7fb67815e540 2023-11-26T07:12:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.121+0000 7fb66effd700 1 -- 172.21.15.83:0/699895239 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674003c00 con 0x7fb67815e540 2023-11-26T07:12:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.121+0000 7fb6805d2700 1 -- 172.21.15.83:0/699895239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67815b3e0 con 0x7fb67815e540 2023-11-26T07:12:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.121+0000 7fb6805d2700 1 -- 172.21.15.83:0/699895239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67815e130 con 0x7fb67815e540 2023-11-26T07:12:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.122+0000 7fb66effd700 1 -- 172.21.15.83:0/699895239 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb67401a430 con 0x7fb67815e540 2023-11-26T07:12:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.122+0000 7fb66effd700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb65c06aed0 0x7fb65c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.122+0000 7fb66effd700 1 -- 172.21.15.83:0/699895239 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb674021070 con 0x7fb67815e540 2023-11-26T07:12:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.122+0000 7fb67e36e700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb65c06aed0 0x7fb65c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.123+0000 7fb67e36e700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb65c06aed0 0x7fb65c06d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb668009930 tx=0x7fb668008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:01.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.127+0000 7fb66effd700 1 -- 172.21.15.83:0/699895239 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6740560a0 con 0x7fb67815e540 2023-11-26T07:12:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.316+0000 7fb6805d2700 1 -- 172.21.15.83:0/699895239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb6780b58c0 con 0x7fb67815e540 2023-11-26T07:12:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.316+0000 7fb66effd700 1 -- 172.21.15.83:0/699895239 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb6740596c0 con 0x7fb67815e540 2023-11-26T07:12:01.319 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:12:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.318+0000 7fb6805d2700 1 -- 172.21.15.83:0/699895239 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb65c06aed0 msgr2=0x7fb65c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.318+0000 7fb6805d2700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb65c06aed0 0x7fb65c06d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb668009930 tx=0x7fb668008040 comp rx=0 tx=0).stop 2023-11-26T07:12:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.318+0000 7fb6805d2700 1 -- 172.21.15.83:0/699895239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb67815e540 msgr2=0x7fb67815aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.318+0000 7fb6805d2700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb67815e540 0x7fb67815aea0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb674014040 tx=0x7fb674004750 comp rx=0 tx=0).stop 2023-11-26T07:12:01.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.318+0000 7fb6805d2700 1 -- 172.21.15.83:0/699895239 shutdown_connections 2023-11-26T07:12:01.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.318+0000 7fb6805d2700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb65c06aed0 0x7fb65c06d390 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.318+0000 7fb6805d2700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb678162800 0x7fb678144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.318+0000 7fb6805d2700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb67815ba30 0x7fb6780cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.318+0000 7fb6805d2700 1 --2- 172.21.15.83:0/699895239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb67815e540 0x7fb67815aea0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.318+0000 7fb6805d2700 1 -- 172.21.15.83:0/699895239 >> 172.21.15.83:0/699895239 conn(0x7fb678071a70 msgr2=0x7fb6780746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:01.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.318+0000 7fb6805d2700 1 -- 172.21.15.83:0/699895239 shutdown_connections 2023-11-26T07:12:01.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.319+0000 7fb6805d2700 1 -- 172.21.15.83:0/699895239 wait complete. 2023-11-26T07:12:01.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:12:01.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.453+0000 7fbb705d7700 1 Processor -- start 2023-11-26T07:12:01.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.453+0000 7fbb705d7700 1 -- start start 2023-11-26T07:12:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.454+0000 7fbb705d7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680b5da0 0x7fbb680afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.454+0000 7fbb705d7700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb680b3c60 con 0x7fbb680746a0 2023-11-26T07:12:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.454+0000 7fbb705d7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb680b58c0 con 0x7fbb6815e160 2023-11-26T07:12:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.454+0000 7fbb705d7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb680b4500 con 0x7fbb680b5da0 2023-11-26T07:12:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.454+0000 7fbb6db72700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbb680746a0 0x7fbb6806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55320/0 (socket says 172.21.15.83:55320) 2023-11-26T07:12:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.454+0000 7fbb6db72700 1 -- 172.21.15.83:0/120873179 learned_addr learned my addr 172.21.15.83:0/120873179 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.454+0000 7fbb6eb74700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680b5da0 0x7fbb680afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.454+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118718624 0 0) 0x7fbb680b3c60 con 0x7fbb680746a0 2023-11-26T07:12:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.454+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb540035b0 con 0x7fbb680746a0 2023-11-26T07:12:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396270321 0 0) 0x7fbb680b58c0 con 0x7fbb6815e160 2023-11-26T07:12:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb680b3c60 con 0x7fbb6815e160 2023-11-26T07:12:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4261911623 0 0) 0x7fbb540035b0 con 0x7fbb680746a0 2023-11-26T07:12:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680b5da0 msgr2=0x7fbb680afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680b5da0 0x7fbb680afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 >> v1:172.21.15.203:6789/0 conn(0x7fbb6815e160 legacy=0x7fbb68073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6806bdc0 con 0x7fbb680746a0 2023-11-26T07:12:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fbb64001670 con 0x7fbb680746a0 2023-11-26T07:12:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 >> v1:172.21.15.83:6789/0 conn(0x7fbb680746a0 legacy=0x7fbb6806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680b5da0 0x7fbb54003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 0x7fbb54004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb54004d90 0x7fbb54005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb680b4500 con 0x7fbb54004170 2023-11-26T07:12:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5400bb50 con 0x7fbb54004d90 2023-11-26T07:12:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5400bd10 con 0x7fbb680b5da0 2023-11-26T07:12:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6db72700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 0x7fbb54004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6eb74700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb54004d90 0x7fbb54005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6e373700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680b5da0 0x7fbb54003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6db72700 1 -- 172.21.15.83:0/120873179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680b5da0 msgr2=0x7fbb54003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6db72700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680b5da0 0x7fbb54003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6db72700 1 -- 172.21.15.83:0/120873179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb54004d90 msgr2=0x7fbb54005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6db72700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb54004d90 0x7fbb54005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6db72700 1 -- 172.21.15.83:0/120873179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540035b0 con 0x7fbb54004170 2023-11-26T07:12:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6db72700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 0x7fbb54004850 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c160 tx=0x7fbb54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dece7a809062e153 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6400cb80 con 0x7fbb54004170 2023-11-26T07:12:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6400cce0 con 0x7fbb54004170 2023-11-26T07:12:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.455+0000 7fbb6d371700 1 -- 172.21.15.83:0/120873179 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64005ba0 con 0x7fbb54004170 2023-11-26T07:12:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.456+0000 7fbb705d7700 1 -- 172.21.15.83:0/120873179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 msgr2=0x7fbb54004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.456+0000 7fbb705d7700 1 --2- 172.21.15.83:0/120873179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 0x7fbb54004850 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400c160 tx=0x7fbb54005880 comp rx=0 tx=0).stop 2023-11-26T07:12:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.456+0000 7fbb6eb74700 1 -- 172.21.15.83:0/120873179 reap_dead start 2023-11-26T07:12:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.456+0000 7fbb705d7700 1 -- 172.21.15.83:0/120873179 shutdown_connections 2023-11-26T07:12:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.456+0000 7fbb705d7700 1 -- 172.21.15.83:0/120873179 >> 172.21.15.83:0/120873179 conn(0x7fbb680d0bf0 msgr2=0x7fbb680d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.456+0000 7fbb705d7700 1 -- 172.21.15.83:0/120873179 shutdown_connections 2023-11-26T07:12:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.456+0000 7fbb705d7700 1 -- 172.21.15.83:0/120873179 wait complete. 2023-11-26T07:12:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.457+0000 7fbb705d7700 1 Processor -- start 2023-11-26T07:12:01.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.457+0000 7fbb705d7700 1 -- start start 2023-11-26T07:12:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb705d7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 0x7fbb68155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb705d7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680cff10 0x7fbb68157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb705d7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb68158260 0x7fbb681f4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb705d7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb681586c0 con 0x7fbb54004170 2023-11-26T07:12:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb705d7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb68160ed0 con 0x7fbb68158260 2023-11-26T07:12:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb705d7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb681602c0 con 0x7fbb680cff10 2023-11-26T07:12:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb6e373700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 0x7fbb68155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb6e373700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 0x7fbb68155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35290/0 (socket says 172.21.15.83:35290) 2023-11-26T07:12:01.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb6e373700 1 -- 172.21.15.83:0/3451924497 learned_addr learned my addr 172.21.15.83:0/3451924497 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:01.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb6eb74700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb68158260 0x7fbb681f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb6db72700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680cff10 0x7fbb68157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb6e373700 1 -- 172.21.15.83:0/3451924497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680cff10 msgr2=0x7fbb68157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb6e373700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680cff10 0x7fbb68157d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb6e373700 1 -- 172.21.15.83:0/3451924497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb68158260 msgr2=0x7fbb681f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb6e373700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb68158260 0x7fbb681f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.458+0000 7fbb6e373700 1 -- 172.21.15.83:0/3451924497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb681605a0 con 0x7fbb54004170 2023-11-26T07:12:01.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.459+0000 7fbb6eb74700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb68158260 0x7fbb681f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:01.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.459+0000 7fbb6e373700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 0x7fbb68155b80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800f830 tx=0x7fbb5800fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:01.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.459+0000 7fbb5effd700 1 -- 172.21.15.83:0/3451924497 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58005320 con 0x7fbb54004170 2023-11-26T07:12:01.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.459+0000 7fbb5effd700 1 -- 172.21.15.83:0/3451924497 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb58005480 con 0x7fbb54004170 2023-11-26T07:12:01.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.459+0000 7fbb705d7700 1 -- 172.21.15.83:0/3451924497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb68160880 con 0x7fbb54004170 2023-11-26T07:12:01.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.459+0000 7fbb5effd700 1 -- 172.21.15.83:0/3451924497 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58006aa0 con 0x7fbb54004170 2023-11-26T07:12:01.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.459+0000 7fbb705d7700 1 -- 172.21.15.83:0/3451924497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb680d0390 con 0x7fbb54004170 2023-11-26T07:12:01.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.460+0000 7fbb705d7700 1 -- 172.21.15.83:0/3451924497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb68068fb0 con 0x7fbb54004170 2023-11-26T07:12:01.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.460+0000 7fbb5effd700 1 -- 172.21.15.83:0/3451924497 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb58006c00 con 0x7fbb54004170 2023-11-26T07:12:01.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.461+0000 7fbb5effd700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbb4c06aed0 0x7fbb4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.461+0000 7fbb5effd700 1 -- 172.21.15.83:0/3451924497 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb58088fd0 con 0x7fbb54004170 2023-11-26T07:12:01.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.461+0000 7fbb6db72700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbb4c06aed0 0x7fbb4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.461+0000 7fbb6db72700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbb4c06aed0 0x7fbb4c06d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbb6806fa70 tx=0x7fbb64007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:01.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.465+0000 7fbb5effd700 1 -- 172.21.15.83:0/3451924497 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb58054080 con 0x7fbb54004170 2023-11-26T07:12:01.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.655+0000 7fbb705d7700 1 -- 172.21.15.83:0/3451924497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb680affc0 con 0x7fbb54004170 2023-11-26T07:12:01.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.656+0000 7fbb5effd700 1 -- 172.21.15.83:0/3451924497 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb580576a0 con 0x7fbb54004170 2023-11-26T07:12:01.658 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:12:01.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 -- 172.21.15.83:0/3451924497 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbb4c06aed0 msgr2=0x7fbb4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbb4c06aed0 0x7fbb4c06d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbb6806fa70 tx=0x7fbb64007000 comp rx=0 tx=0).stop 2023-11-26T07:12:01.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 -- 172.21.15.83:0/3451924497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 msgr2=0x7fbb68155b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 0x7fbb68155b80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800f830 tx=0x7fbb5800fbf0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 -- 172.21.15.83:0/3451924497 shutdown_connections 2023-11-26T07:12:01.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb68158260 0x7fbb681f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb54004170 0x7fbb68155b80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbb4c06aed0 0x7fbb4c06d390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 --2- 172.21.15.83:0/3451924497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb680cff10 0x7fbb68157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 -- 172.21.15.83:0/3451924497 >> 172.21.15.83:0/3451924497 conn(0x7fbb680d0bf0 msgr2=0x7fbb6806dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:01.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 -- 172.21.15.83:0/3451924497 shutdown_connections 2023-11-26T07:12:01.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.658+0000 7fbb705d7700 1 -- 172.21.15.83:0/3451924497 wait complete. 2023-11-26T07:12:01.672 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.791+0000 7ff114e5e700 1 Processor -- start 2023-11-26T07:12:01.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.791+0000 7ff114e5e700 1 -- start start 2023-11-26T07:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.792+0000 7ff114e5e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff1100b19a0 0x7ff1100d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.792+0000 7ff114e5e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff1100622a0 0x7ff11015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.792+0000 7ff114e5e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1100c6520 0x7ff1100c6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.792+0000 7ff114e5e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1100b2d50 con 0x7ff1100622a0 2023-11-26T07:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.792+0000 7ff114e5e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11015c8f0 con 0x7ff1100b19a0 2023-11-26T07:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.792+0000 7ff114e5e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff110062730 con 0x7ff1100c6520 2023-11-26T07:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10dd9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff1100622a0 0x7ff11015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10dd9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff1100622a0 0x7ff11015cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35300/0 (socket says 172.21.15.83:35300) 2023-11-26T07:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10dd9b700 1 -- 172.21.15.83:0/1478287921 learned_addr learned my addr 172.21.15.83:0/1478287921 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10e59c700 1 --2- 172.21.15.83:0/1478287921 >> v2:172.21.15.203:3300/0 conn(0x7ff1100b19a0 0x7ff1100d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10ed9d700 1 --2- 172.21.15.83:0/1478287921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1100c6520 0x7ff1100c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10dd9b700 1 -- 172.21.15.83:0/1478287921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1100c6520 msgr2=0x7ff1100c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10dd9b700 1 --2- 172.21.15.83:0/1478287921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1100c6520 0x7ff1100c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10dd9b700 1 -- 172.21.15.83:0/1478287921 >> v2:172.21.15.203:3300/0 conn(0x7ff1100b19a0 msgr2=0x7ff1100d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10dd9b700 1 --2- 172.21.15.83:0/1478287921 >> v2:172.21.15.203:3300/0 conn(0x7ff1100b19a0 0x7ff1100d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10dd9b700 1 -- 172.21.15.83:0/1478287921 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1100d0790 con 0x7ff1100622a0 2023-11-26T07:12:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10dd9b700 1 --2- 172.21.15.83:0/1478287921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1100622a0 0x7ff11015cd80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff1100723a0 tx=0x7ff104009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860a2c6173a90db8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10d59a700 1 -- 172.21.15.83:0/1478287921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10400b870 con 0x7ff1100622a0 2023-11-26T07:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10d59a700 1 -- 172.21.15.83:0/1478287921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10400b9d0 con 0x7ff1100622a0 2023-11-26T07:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.793+0000 7ff10d59a700 1 -- 172.21.15.83:0/1478287921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10400bd60 con 0x7ff1100622a0 2023-11-26T07:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.794+0000 7ff114e5e700 1 -- 172.21.15.83:0/1478287921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1100622a0 msgr2=0x7ff11015cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.794+0000 7ff114e5e700 1 --2- 172.21.15.83:0/1478287921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1100622a0 0x7ff11015cd80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff1100723a0 tx=0x7ff104009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.794+0000 7ff114e5e700 1 -- 172.21.15.83:0/1478287921 shutdown_connections 2023-11-26T07:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.794+0000 7ff114e5e700 1 --2- 172.21.15.83:0/1478287921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1100c6520 0x7ff1100c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.794+0000 7ff114e5e700 1 --2- 172.21.15.83:0/1478287921 >> v2:172.21.15.203:3300/0 conn(0x7ff1100b19a0 0x7ff1100d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.794+0000 7ff114e5e700 1 --2- 172.21.15.83:0/1478287921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1100622a0 0x7ff11015cd80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.794+0000 7ff114e5e700 1 -- 172.21.15.83:0/1478287921 >> 172.21.15.83:0/1478287921 conn(0x7ff1100b1e90 msgr2=0x7ff1100b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.794+0000 7ff114e5e700 1 -- 172.21.15.83:0/1478287921 shutdown_connections 2023-11-26T07:12:01.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.794+0000 7ff114e5e700 1 -- 172.21.15.83:0/1478287921 wait complete. 2023-11-26T07:12:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.795+0000 7ff114e5e700 1 Processor -- start 2023-11-26T07:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.795+0000 7ff114e5e700 1 -- start start 2023-11-26T07:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff114e5e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1100622a0 0x7ff1100b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff114e5e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1100b19a0 0x7ff1100af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff114e5e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1100c6520 0x7ff11014c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff114e5e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015d2d0 con 0x7ff1100b19a0 2023-11-26T07:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff114e5e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015d5b0 con 0x7ff1100622a0 2023-11-26T07:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff114e5e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015d890 con 0x7ff1100c6520 2023-11-26T07:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1100b19a0 0x7ff1100af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1100b19a0 0x7ff1100af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35306/0 (socket says 172.21.15.83:35306) 2023-11-26T07:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10dd9b700 1 -- 172.21.15.83:0/667006056 learned_addr learned my addr 172.21.15.83:0/667006056 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10e59c700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1100622a0 0x7ff1100b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10ed9d700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1100c6520 0x7ff11014c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10dd9b700 1 -- 172.21.15.83:0/667006056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1100c6520 msgr2=0x7ff11014c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10dd9b700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1100c6520 0x7ff11014c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10dd9b700 1 -- 172.21.15.83:0/667006056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1100622a0 msgr2=0x7ff1100b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10dd9b700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1100622a0 0x7ff1100b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10dd9b700 1 -- 172.21.15.83:0/667006056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff104009550 con 0x7ff1100b19a0 2023-11-26T07:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10e59c700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1100622a0 0x7ff1100b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.796+0000 7ff10dd9b700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1100b19a0 0x7ff1100af5b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff104000c00 tx=0x7ff104004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.797+0000 7ff0feffd700 1 -- 172.21.15.83:0/667006056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104004540 con 0x7ff1100b19a0 2023-11-26T07:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.797+0000 7ff0feffd700 1 -- 172.21.15.83:0/667006056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1040036a0 con 0x7ff1100b19a0 2023-11-26T07:12:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.797+0000 7ff114e5e700 1 -- 172.21.15.83:0/667006056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11015db70 con 0x7ff1100b19a0 2023-11-26T07:12:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.797+0000 7ff0feffd700 1 -- 172.21.15.83:0/667006056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104003a50 con 0x7ff1100b19a0 2023-11-26T07:12:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.797+0000 7ff114e5e700 1 -- 172.21.15.83:0/667006056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11015ad60 con 0x7ff1100b19a0 2023-11-26T07:12:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.797+0000 7ff114e5e700 1 -- 172.21.15.83:0/667006056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11015e1c0 con 0x7ff1100b19a0 2023-11-26T07:12:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.798+0000 7ff0feffd700 1 -- 172.21.15.83:0/667006056 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff104021070 con 0x7ff1100b19a0 2023-11-26T07:12:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.798+0000 7ff0feffd700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff0f006aed0 0x7ff0f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.799+0000 7ff0feffd700 1 -- 172.21.15.83:0/667006056 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff10408ac90 con 0x7ff1100b19a0 2023-11-26T07:12:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.799+0000 7ff10e59c700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff0f006aed0 0x7ff0f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.799+0000 7ff10e59c700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff0f006aed0 0x7ff0f006d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff100009970 tx=0x7ff100008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:01.803+0000 7ff0feffd700 1 -- 172.21.15.83:0/667006056 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff104055dd0 con 0x7ff1100b19a0 2023-11-26T07:12:02.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.010+0000 7ff114e5e700 1 -- 172.21.15.83:0/667006056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff110067f60 con 0x7ff1100b19a0 2023-11-26T07:12:02.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.010+0000 7ff0feffd700 1 -- 172.21.15.83:0/667006056 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff104029080 con 0x7ff1100b19a0 2023-11-26T07:12:02.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 -- 172.21.15.83:0/667006056 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff0f006aed0 msgr2=0x7ff0f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff0f006aed0 0x7ff0f006d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff100009970 tx=0x7ff100008040 comp rx=0 tx=0).stop 2023-11-26T07:12:02.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 -- 172.21.15.83:0/667006056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1100b19a0 msgr2=0x7ff1100af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1100b19a0 0x7ff1100af5b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff104000c00 tx=0x7ff104004750 comp rx=0 tx=0).stop 2023-11-26T07:12:02.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 -- 172.21.15.83:0/667006056 shutdown_connections 2023-11-26T07:12:02.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff0f006aed0 0x7ff0f006d390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1100c6520 0x7ff11014c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1100622a0 0x7ff1100b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 --2- 172.21.15.83:0/667006056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1100b19a0 0x7ff1100af5b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 -- 172.21.15.83:0/667006056 >> 172.21.15.83:0/667006056 conn(0x7ff1100b1e90 msgr2=0x7ff11006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:02.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 -- 172.21.15.83:0/667006056 shutdown_connections 2023-11-26T07:12:02.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.012+0000 7ff114e5e700 1 -- 172.21.15.83:0/667006056 wait complete. 2023-11-26T07:12:02.016 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:12:02.026 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:12:02.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:02.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-26T07:12:02.067 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:12:02.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:02.068 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:12:02.129 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:02.129 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:02.150 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:02.151 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:02.211 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:02.212 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:02.272 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:02.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.390+0000 7f5de0a19700 1 Processor -- start 2023-11-26T07:12:02.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.390+0000 7f5de0a19700 1 -- start start 2023-11-26T07:12:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.391+0000 7f5de0a19700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5ddc06c710 0x7f5ddc15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.391+0000 7f5de0a19700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc15f520 con 0x7f5ddc144660 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.391+0000 7f5de0a19700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc15f800 con 0x7f5ddc15ea60 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.391+0000 7f5de0a19700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ddc0b4230 con 0x7f5ddc06c710 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.391+0000 7f5dda59c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5ddc144660 0x7f5ddc0b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55352/0 (socket says 172.21.15.83:55352) 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dda59c700 1 -- 172.21.15.83:0/3540933709 learned_addr learned my addr 172.21.15.83:0/3540933709 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5ddad9d700 1 --2- 172.21.15.83:0/3540933709 >> v2:172.21.15.203:3300/0 conn(0x7f5ddc06c710 0x7f5ddc15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836181729 0 0) 0x7f5ddc15f520 con 0x7f5ddc144660 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dc40035b0 con 0x7f5ddc144660 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1609626035 0 0) 0x7f5dc40035b0 con 0x7f5ddc144660 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 >> v2:172.21.15.203:3300/0 conn(0x7f5ddc06c710 msgr2=0x7f5ddc15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 --2- 172.21.15.83:0/3540933709 >> v2:172.21.15.203:3300/0 conn(0x7f5ddc06c710 0x7f5ddc15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 >> v1:172.21.15.203:6789/0 conn(0x7f5ddc15ea60 legacy=0x7f5ddc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc0b3e50 con 0x7f5ddc144660 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f5dd0001670 con 0x7f5ddc144660 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 >> v1:172.21.15.83:6789/0 conn(0x7f5ddc144660 legacy=0x7f5ddc0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 --2- 172.21.15.83:0/3540933709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 0x7f5dc4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 --2- 172.21.15.83:0/3540933709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc4004850 0x7f5dc4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 --2- 172.21.15.83:0/3540933709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5dc4005210 0x7f5dc400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0b4230 con 0x7f5dc4003a00 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc400c1b0 con 0x7f5dc4004850 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc400c370 con 0x7f5dc4005210 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dda59c700 1 --2- 172.21.15.83:0/3540933709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 0x7f5dc4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dd9d9b700 1 --2- 172.21.15.83:0/3540933709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc4004850 0x7f5dc4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dda59c700 1 -- 172.21.15.83:0/3540933709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5dc4005210 msgr2=0x7f5dc400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5ddad9d700 1 --2- 172.21.15.83:0/3540933709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5dc4005210 0x7f5dc400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dda59c700 1 --2- 172.21.15.83:0/3540933709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5dc4005210 0x7f5dc400ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dda59c700 1 -- 172.21.15.83:0/3540933709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc4004850 msgr2=0x7f5dc4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dda59c700 1 --2- 172.21.15.83:0/3540933709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5dc4004850 0x7f5dc4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.392+0000 7f5dda59c700 1 -- 172.21.15.83:0/3540933709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40035b0 con 0x7f5dc4003a00 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5ddad9d700 1 -- 172.21.15.83:0/3540933709 reap_dead start 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5dda59c700 1 --2- 172.21.15.83:0/3540933709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 0x7f5dc4003e20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000c240 tx=0x7f5dc4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b205cb70ecca6576 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0007c10 con 0x7f5dc4003a00 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd000cda0 con 0x7f5dc4003a00 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5dd959a700 1 -- 172.21.15.83:0/3540933709 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0005ba0 con 0x7f5dc4003a00 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5de0a19700 1 -- 172.21.15.83:0/3540933709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 msgr2=0x7f5dc4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5de0a19700 1 --2- 172.21.15.83:0/3540933709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 0x7f5dc4003e20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000c240 tx=0x7f5dc4005880 comp rx=0 tx=0).stop 2023-11-26T07:12:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5de0a19700 1 -- 172.21.15.83:0/3540933709 shutdown_connections 2023-11-26T07:12:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5de0a19700 1 --2- 172.21.15.83:0/3540933709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 0x7f5dc4003e20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5de0a19700 1 -- 172.21.15.83:0/3540933709 >> 172.21.15.83:0/3540933709 conn(0x7f5ddc071ac0 msgr2=0x7f5ddc071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5de0a19700 1 -- 172.21.15.83:0/3540933709 shutdown_connections 2023-11-26T07:12:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.393+0000 7f5de0a19700 1 -- 172.21.15.83:0/3540933709 wait complete. 2023-11-26T07:12:02.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.395+0000 7f5de0a19700 1 Processor -- start 2023-11-26T07:12:02.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.395+0000 7f5de0a19700 1 -- start start 2023-11-26T07:12:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.395+0000 7f5de0a19700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 0x7f5ddc06b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5de0a19700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ddc0afa90 0x7f5ddc0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5de0a19700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ddc1efad0 0x7f5ddc1eff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5de0a19700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0aa870 con 0x7f5dc4003a00 2023-11-26T07:12:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5de0a19700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0aab50 con 0x7f5ddc0afa90 2023-11-26T07:12:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5de0a19700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0aae30 con 0x7f5ddc1efad0 2023-11-26T07:12:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dda59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 0x7f5ddc06b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dda59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 0x7f5ddc06b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35324/0 (socket says 172.21.15.83:35324) 2023-11-26T07:12:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dda59c700 1 -- 172.21.15.83:0/2228610736 learned_addr learned my addr 172.21.15.83:0/2228610736 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dd9d9b700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ddc0afa90 0x7f5ddc0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5ddad9d700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ddc1efad0 0x7f5ddc1eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dda59c700 1 -- 172.21.15.83:0/2228610736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ddc1efad0 msgr2=0x7f5ddc1eff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dda59c700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ddc1efad0 0x7f5ddc1eff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dda59c700 1 -- 172.21.15.83:0/2228610736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ddc0afa90 msgr2=0x7f5ddc0aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dda59c700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ddc0afa90 0x7f5ddc0aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dda59c700 1 -- 172.21.15.83:0/2228610736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc0ab110 con 0x7f5dc4003a00 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5ddad9d700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ddc1efad0 0x7f5ddc1eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dda59c700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 0x7f5ddc06b260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400c180 tx=0x7f5dd0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dcaffd700 1 -- 172.21.15.83:0/2228610736 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0004c80 con 0x7f5dc4003a00 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dcaffd700 1 -- 172.21.15.83:0/2228610736 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd0004de0 con 0x7f5dc4003a00 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5de0a19700 1 -- 172.21.15.83:0/2228610736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ddc0ab3f0 con 0x7f5dc4003a00 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5dcaffd700 1 -- 172.21.15.83:0/2228610736 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0007310 con 0x7f5dc4003a00 2023-11-26T07:12:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.396+0000 7f5de0a19700 1 -- 172.21.15.83:0/2228610736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ddc1f04a0 con 0x7f5dc4003a00 2023-11-26T07:12:02.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.397+0000 7f5de0a19700 1 -- 172.21.15.83:0/2228610736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ddc0ae670 con 0x7f5dc4003a00 2023-11-26T07:12:02.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.398+0000 7f5dcaffd700 1 -- 172.21.15.83:0/2228610736 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5dd0007470 con 0x7f5dc4003a00 2023-11-26T07:12:02.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.398+0000 7f5dcaffd700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5dbc06aed0 0x7f5dbc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.398+0000 7f5dd9d9b700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5dbc06aed0 0x7f5dbc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.398+0000 7f5dcaffd700 1 -- 172.21.15.83:0/2228610736 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5dd0089eb0 con 0x7f5dc4003a00 2023-11-26T07:12:02.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.399+0000 7f5dd9d9b700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5dbc06aed0 0x7f5dbc06d390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc15a850 tx=0x7f5dcc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:02.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.403+0000 7f5dcaffd700 1 -- 172.21.15.83:0/2228610736 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ddc0ae670 con 0x7f5dc4003a00 2023-11-26T07:12:02.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.596+0000 7f5de0a19700 1 -- 172.21.15.83:0/2228610736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ddc0ae670 con 0x7f5dc4003a00 2023-11-26T07:12:02.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.597+0000 7f5dcaffd700 1 -- 172.21.15.83:0/2228610736 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f5ddc0ae670 con 0x7f5dc4003a00 2023-11-26T07:12:02.600 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:02.600 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 -- 172.21.15.83:0/2228610736 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5dbc06aed0 msgr2=0x7f5dbc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5dbc06aed0 0x7f5dbc06d390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc15a850 tx=0x7f5dcc009040 comp rx=0 tx=0).stop 2023-11-26T07:12:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 -- 172.21.15.83:0/2228610736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 msgr2=0x7f5ddc06b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 0x7f5ddc06b260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400c180 tx=0x7f5dd0001a00 comp rx=0 tx=0).stop 2023-11-26T07:12:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 -- 172.21.15.83:0/2228610736 shutdown_connections 2023-11-26T07:12:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5dbc06aed0 0x7f5dbc06d390 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ddc1efad0 0x7f5ddc1eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dc4003a00 0x7f5ddc06b260 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 --2- 172.21.15.83:0/2228610736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ddc0afa90 0x7f5ddc0aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 -- 172.21.15.83:0/2228610736 >> 172.21.15.83:0/2228610736 conn(0x7f5ddc071ac0 msgr2=0x7f5ddc06f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 -- 172.21.15.83:0/2228610736 shutdown_connections 2023-11-26T07:12:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.599+0000 7f5de0a19700 1 -- 172.21.15.83:0/2228610736 wait complete. 2023-11-26T07:12:02.614 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:12:02.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:12:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.735+0000 7fb094fd9700 1 Processor -- start 2023-11-26T07:12:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.735+0000 7fb094fd9700 1 -- start start 2023-11-26T07:12:02.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.736+0000 7fb094fd9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb0900b3bf0 0x7fb09006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.736+0000 7fb094fd9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb090073f90 0x7fb090074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.736+0000 7fb094fd9700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0900b0590 con 0x7fb09015e150 2023-11-26T07:12:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.736+0000 7fb094fd9700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0900b4360 con 0x7fb0900b3bf0 2023-11-26T07:12:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.736+0000 7fb094fd9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb090074960 con 0x7fb090073f90 2023-11-26T07:12:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.736+0000 7fb08dd9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb0900b3bf0 0x7fb09006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.736+0000 7fb08ed9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb090073f90 0x7fb090074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.736+0000 7fb08e59c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fb09015e150 0x7fb09006f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:39990/0 (socket says 172.21.15.83:39990) 2023-11-26T07:12:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.736+0000 7fb08e59c700 1 -- 172.21.15.83:0/1808839344 learned_addr learned my addr 172.21.15.83:0/1808839344 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:02.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.737+0000 7fb08dd9b700 1 -- 172.21.15.83:0/1808839344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb090073f90 msgr2=0x7fb090074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.737+0000 7fb08dd9b700 1 --2- 172.21.15.83:0/1808839344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb090073f90 0x7fb090074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.737+0000 7fb08dd9b700 1 -- 172.21.15.83:0/1808839344 >> v1:172.21.15.203:6789/0 conn(0x7fb09015e150 legacy=0x7fb09006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.737+0000 7fb08dd9b700 1 -- 172.21.15.83:0/1808839344 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0900b4080 con 0x7fb0900b3bf0 2023-11-26T07:12:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.737+0000 7fb08d59a700 1 -- 172.21.15.83:0/1808839344 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858685266 0 0) 0x7fb0900b0590 con 0x7fb09015e150 2023-11-26T07:12:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.737+0000 7fb08ed9d700 1 --2- 172.21.15.83:0/1808839344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb090073f90 0x7fb090074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:12:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.737+0000 7fb08dd9b700 1 --2- 172.21.15.83:0/1808839344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0900b3bf0 0x7fb09006e5c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb080009840 tx=0x7fb080009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9433ead4682f027d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.737+0000 7fb08d59a700 1 -- 172.21.15.83:0/1808839344 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080011830 con 0x7fb0900b3bf0 2023-11-26T07:12:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.737+0000 7fb08d59a700 1 -- 172.21.15.83:0/1808839344 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080011990 con 0x7fb0900b3bf0 2023-11-26T07:12:02.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.737+0000 7fb08d59a700 1 -- 172.21.15.83:0/1808839344 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080011d20 con 0x7fb0900b3bf0 2023-11-26T07:12:02.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.738+0000 7fb094fd9700 1 -- 172.21.15.83:0/1808839344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0900b3bf0 msgr2=0x7fb09006e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.738+0000 7fb094fd9700 1 --2- 172.21.15.83:0/1808839344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0900b3bf0 0x7fb09006e5c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb080009840 tx=0x7fb080009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:02.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.738+0000 7fb094fd9700 1 -- 172.21.15.83:0/1808839344 shutdown_connections 2023-11-26T07:12:02.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.738+0000 7fb094fd9700 1 --2- 172.21.15.83:0/1808839344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb090073f90 0x7fb090074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.738+0000 7fb094fd9700 1 --2- 172.21.15.83:0/1808839344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0900b3bf0 0x7fb09006e5c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.738+0000 7fb094fd9700 1 -- 172.21.15.83:0/1808839344 >> 172.21.15.83:0/1808839344 conn(0x7fb0900d0be0 msgr2=0x7fb0900d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.738+0000 7fb094fd9700 1 -- 172.21.15.83:0/1808839344 shutdown_connections 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.738+0000 7fb094fd9700 1 -- 172.21.15.83:0/1808839344 wait complete. 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.739+0000 7fb094fd9700 1 Processor -- start 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.739+0000 7fb094fd9700 1 -- start start 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb094fd9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb090073f90 0x7fb090153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb094fd9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb09015e150 0x7fb090155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb094fd9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900b1700 0x7fb0901465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb094fd9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb090153670 con 0x7fb09015e150 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb094fd9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb090156380 con 0x7fb090073f90 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb094fd9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09015e6d0 con 0x7fb0900b1700 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb08dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb09015e150 0x7fb090155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb08dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb09015e150 0x7fb090155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35340/0 (socket says 172.21.15.83:35340) 2023-11-26T07:12:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb08dd9b700 1 -- 172.21.15.83:0/1216693851 learned_addr learned my addr 172.21.15.83:0/1216693851 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb08e59c700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb090073f90 0x7fb090153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb08ed9d700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900b1700 0x7fb0901465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb08dd9b700 1 -- 172.21.15.83:0/1216693851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900b1700 msgr2=0x7fb0901465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb08dd9b700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900b1700 0x7fb0901465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb08dd9b700 1 -- 172.21.15.83:0/1216693851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb090073f90 msgr2=0x7fb090153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb08dd9b700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb090073f90 0x7fb090153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb08dd9b700 1 -- 172.21.15.83:0/1216693851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb088009a70 con 0x7fb09015e150 2023-11-26T07:12:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.740+0000 7fb08ed9d700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900b1700 0x7fb0901465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.741+0000 7fb08dd9b700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb09015e150 0x7fb090155e40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb080009fd0 tx=0x7fb080004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.741+0000 7fb07effd700 1 -- 172.21.15.83:0/1216693851 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0800036a0 con 0x7fb09015e150 2023-11-26T07:12:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.741+0000 7fb07effd700 1 -- 172.21.15.83:0/1216693851 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08001f070 con 0x7fb09015e150 2023-11-26T07:12:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.741+0000 7fb094fd9700 1 -- 172.21.15.83:0/1216693851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0800094a0 con 0x7fb09015e150 2023-11-26T07:12:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.741+0000 7fb07effd700 1 -- 172.21.15.83:0/1216693851 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080003a20 con 0x7fb09015e150 2023-11-26T07:12:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.741+0000 7fb094fd9700 1 -- 172.21.15.83:0/1216693851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09015ec90 con 0x7fb09015e150 2023-11-26T07:12:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.742+0000 7fb094fd9700 1 -- 172.21.15.83:0/1216693851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb090068fb0 con 0x7fb09015e150 2023-11-26T07:12:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.742+0000 7fb07effd700 1 -- 172.21.15.83:0/1216693851 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb080003b80 con 0x7fb09015e150 2023-11-26T07:12:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.743+0000 7fb07effd700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07006aed0 0x7fb07006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.743+0000 7fb07effd700 1 -- 172.21.15.83:0/1216693851 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb080021070 con 0x7fb09015e150 2023-11-26T07:12:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.743+0000 7fb08e59c700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07006aed0 0x7fb07006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.743+0000 7fb08e59c700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07006aed0 0x7fb07006d390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb084000f60 tx=0x7fb084007e70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:02.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.747+0000 7fb07effd700 1 -- 172.21.15.83:0/1216693851 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb080056020 con 0x7fb09015e150 2023-11-26T07:12:02.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.934+0000 7fb094fd9700 1 -- 172.21.15.83:0/1216693851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb090161930 con 0x7fb09015e150 2023-11-26T07:12:02.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.934+0000 7fb07effd700 1 -- 172.21.15.83:0/1216693851 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0800150a0 con 0x7fb09015e150 2023-11-26T07:12:02.937 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:12:02.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 -- 172.21.15.83:0/1216693851 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07006aed0 msgr2=0x7fb07006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07006aed0 0x7fb07006d390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb084000f60 tx=0x7fb084007e70 comp rx=0 tx=0).stop 2023-11-26T07:12:02.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 -- 172.21.15.83:0/1216693851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb09015e150 msgr2=0x7fb090155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:02.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb09015e150 0x7fb090155e40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb080009fd0 tx=0x7fb080004750 comp rx=0 tx=0).stop 2023-11-26T07:12:02.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 -- 172.21.15.83:0/1216693851 shutdown_connections 2023-11-26T07:12:02.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07006aed0 0x7fb07006d390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900b1700 0x7fb0901465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb090073f90 0x7fb090153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 --2- 172.21.15.83:0/1216693851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb09015e150 0x7fb090155e40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:02.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 -- 172.21.15.83:0/1216693851 >> 172.21.15.83:0/1216693851 conn(0x7fb0900d0be0 msgr2=0x7fb09006be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:02.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 -- 172.21.15.83:0/1216693851 shutdown_connections 2023-11-26T07:12:02.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:02.936+0000 7fb094fd9700 1 -- 172.21.15.83:0/1216693851 wait complete. 2023-11-26T07:12:02.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:12:03.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.071+0000 7f8bfef4b700 1 Processor -- start 2023-11-26T07:12:03.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.071+0000 7f8bfef4b700 1 -- start start 2023-11-26T07:12:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.072+0000 7f8bfef4b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8bf81587f0 0x7f8bf806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.072+0000 7f8bfef4b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bf80b4370 0x7f8bf815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.072+0000 7f8bfef4b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf806b530 con 0x7f8bf815b370 2023-11-26T07:12:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.072+0000 7f8bfef4b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf80ae5e0 con 0x7f8bf80b4370 2023-11-26T07:12:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.072+0000 7f8bfef4b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf815be30 con 0x7f8bf81587f0 2023-11-26T07:12:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.072+0000 7f8bfd4e8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bf80b4370 0x7f8bf815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.072+0000 7f8bfd4e8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bf80b4370 0x7f8bf815b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35350/0 (socket says 172.21.15.83:35350) 2023-11-26T07:12:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.072+0000 7f8bfd4e8700 1 -- 172.21.15.83:0/2781963732 learned_addr learned my addr 172.21.15.83:0/2781963732 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bfcce7700 1 --2- 172.21.15.83:0/2781963732 >> v2:172.21.15.203:3300/0 conn(0x7f8bf81587f0 0x7f8bf806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bfd4e8700 1 -- 172.21.15.83:0/2781963732 >> v2:172.21.15.203:3300/0 conn(0x7f8bf81587f0 msgr2=0x7f8bf806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bfd4e8700 1 --2- 172.21.15.83:0/2781963732 >> v2:172.21.15.203:3300/0 conn(0x7f8bf81587f0 0x7f8bf806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bfd4e8700 1 -- 172.21.15.83:0/2781963732 >> v1:172.21.15.203:6789/0 conn(0x7f8bf815b370 legacy=0x7f8bf80b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bfd4e8700 1 -- 172.21.15.83:0/2781963732 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf815c110 con 0x7f8bf80b4370 2023-11-26T07:12:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bfcce7700 1 --2- 172.21.15.83:0/2781963732 >> v2:172.21.15.203:3300/0 conn(0x7f8bf81587f0 0x7f8bf806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bfd4e8700 1 --2- 172.21.15.83:0/2781963732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bf80b4370 0x7f8bf815b800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009e50 tx=0x7f8bf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abdcb835a017b98 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bf77fe700 1 -- 172.21.15.83:0/2781963732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000e070 con 0x7f8bf80b4370 2023-11-26T07:12:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bf77fe700 1 -- 172.21.15.83:0/2781963732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf000b980 con 0x7f8bf80b4370 2023-11-26T07:12:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bf77fe700 1 -- 172.21.15.83:0/2781963732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000bd30 con 0x7f8bf80b4370 2023-11-26T07:12:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bfef4b700 1 -- 172.21.15.83:0/2781963732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bf80b4370 msgr2=0x7f8bf815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.073+0000 7f8bfef4b700 1 --2- 172.21.15.83:0/2781963732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bf80b4370 0x7f8bf815b800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009e50 tx=0x7f8bf000b330 comp rx=0 tx=0).stop 2023-11-26T07:12:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.074+0000 7f8bfef4b700 1 -- 172.21.15.83:0/2781963732 shutdown_connections 2023-11-26T07:12:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.074+0000 7f8bfef4b700 1 --2- 172.21.15.83:0/2781963732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bf80b4370 0x7f8bf815b800 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.074+0000 7f8bfef4b700 1 --2- 172.21.15.83:0/2781963732 >> v2:172.21.15.203:3300/0 conn(0x7f8bf81587f0 0x7f8bf806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.074+0000 7f8bfef4b700 1 -- 172.21.15.83:0/2781963732 >> 172.21.15.83:0/2781963732 conn(0x7f8bf80d0bf0 msgr2=0x7f8bf80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.074+0000 7f8bfef4b700 1 -- 172.21.15.83:0/2781963732 shutdown_connections 2023-11-26T07:12:03.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.074+0000 7f8bfef4b700 1 -- 172.21.15.83:0/2781963732 wait complete. 2023-11-26T07:12:03.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.075+0000 7f8bfef4b700 1 Processor -- start 2023-11-26T07:12:03.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.075+0000 7f8bfef4b700 1 -- start start 2023-11-26T07:12:03.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bfef4b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf80b4370 0x7f8bf81542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bfef4b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bf81587f0 0x7f8bf809e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bfef4b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bf815b370 0x7f8bf809cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bfef4b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf809d5f0 con 0x7f8bf81587f0 2023-11-26T07:12:03.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bfef4b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf809d8d0 con 0x7f8bf815b370 2023-11-26T07:12:03.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bfef4b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf809dbb0 con 0x7f8bf80b4370 2023-11-26T07:12:03.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bf81587f0 0x7f8bf809e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bf81587f0 0x7f8bf809e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35354/0 (socket says 172.21.15.83:35354) 2023-11-26T07:12:03.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf7fff700 1 -- 172.21.15.83:0/1639646315 learned_addr learned my addr 172.21.15.83:0/1639646315 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:03.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bfd4e8700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bf815b370 0x7f8bf809cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bfcce7700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf80b4370 0x7f8bf81542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf7fff700 1 -- 172.21.15.83:0/1639646315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf80b4370 msgr2=0x7f8bf81542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf7fff700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf80b4370 0x7f8bf81542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf7fff700 1 -- 172.21.15.83:0/1639646315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bf815b370 msgr2=0x7f8bf809cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf7fff700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bf815b370 0x7f8bf809cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf7fff700 1 -- 172.21.15.83:0/1639646315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf0009b30 con 0x7f8bf81587f0 2023-11-26T07:12:03.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf7fff700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bf81587f0 0x7f8bf809e690 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8be400c430 tx=0x7f8be400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:03.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf57fa700 1 -- 172.21.15.83:0/1639646315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be40050d0 con 0x7f8bf81587f0 2023-11-26T07:12:03.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf57fa700 1 -- 172.21.15.83:0/1639646315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be4005250 con 0x7f8bf81587f0 2023-11-26T07:12:03.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bfef4b700 1 -- 172.21.15.83:0/1639646315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf809de90 con 0x7f8bf81587f0 2023-11-26T07:12:03.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.076+0000 7f8bf57fa700 1 -- 172.21.15.83:0/1639646315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4006860 con 0x7f8bf81587f0 2023-11-26T07:12:03.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.077+0000 7f8bfef4b700 1 -- 172.21.15.83:0/1639646315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf81547f0 con 0x7f8bf81587f0 2023-11-26T07:12:03.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.078+0000 7f8bfef4b700 1 -- 172.21.15.83:0/1639646315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf8068fb0 con 0x7f8bf81587f0 2023-11-26T07:12:03.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.078+0000 7f8bf57fa700 1 -- 172.21.15.83:0/1639646315 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8be4003680 con 0x7f8bf81587f0 2023-11-26T07:12:03.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.078+0000 7f8bf57fa700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8bdc06aed0 0x7f8bdc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.078+0000 7f8bf57fa700 1 -- 172.21.15.83:0/1639646315 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8be40886a0 con 0x7f8bf81587f0 2023-11-26T07:12:03.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.078+0000 7f8bfcce7700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8bdc06aed0 0x7f8bdc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.079+0000 7f8bfcce7700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8bdc06aed0 0x7f8bdc06d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8074f50 tx=0x7f8bec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:03.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.083+0000 7f8bf57fa700 1 -- 172.21.15.83:0/1639646315 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8be4053750 con 0x7f8bf81587f0 2023-11-26T07:12:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.273+0000 7f8bfef4b700 1 -- 172.21.15.83:0/1639646315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8bf815b800 con 0x7f8bf81587f0 2023-11-26T07:12:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.273+0000 7f8bf57fa700 1 -- 172.21.15.83:0/1639646315 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8be4056d70 con 0x7f8bf81587f0 2023-11-26T07:12:03.276 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:12:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.275+0000 7f8bfef4b700 1 -- 172.21.15.83:0/1639646315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8bdc06aed0 msgr2=0x7f8bdc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.275+0000 7f8bfef4b700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8bdc06aed0 0x7f8bdc06d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8074f50 tx=0x7f8bec008040 comp rx=0 tx=0).stop 2023-11-26T07:12:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.275+0000 7f8bfef4b700 1 -- 172.21.15.83:0/1639646315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bf81587f0 msgr2=0x7f8bf809e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.275+0000 7f8bfef4b700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bf81587f0 0x7f8bf809e690 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8be400c430 tx=0x7f8be400c740 comp rx=0 tx=0).stop 2023-11-26T07:12:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.276+0000 7f8bfef4b700 1 -- 172.21.15.83:0/1639646315 shutdown_connections 2023-11-26T07:12:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.276+0000 7f8bfef4b700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bf815b370 0x7f8bf809cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.276+0000 7f8bfef4b700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8bdc06aed0 0x7f8bdc06d390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.276+0000 7f8bfef4b700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf80b4370 0x7f8bf81542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.276+0000 7f8bfef4b700 1 --2- 172.21.15.83:0/1639646315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bf81587f0 0x7f8bf809e690 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.276+0000 7f8bfef4b700 1 -- 172.21.15.83:0/1639646315 >> 172.21.15.83:0/1639646315 conn(0x7f8bf80d0bf0 msgr2=0x7f8bf80d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.276+0000 7f8bfef4b700 1 -- 172.21.15.83:0/1639646315 shutdown_connections 2023-11-26T07:12:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.276+0000 7f8bfef4b700 1 -- 172.21.15.83:0/1639646315 wait complete. 2023-11-26T07:12:03.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:12:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.411+0000 7fe367a78700 1 Processor -- start 2023-11-26T07:12:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.412+0000 7fe367a78700 1 -- start start 2023-11-26T07:12:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.412+0000 7fe367a78700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe36006f380 0x7fe36006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.412+0000 7fe367a78700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36006fa50 con 0x7fe360073f90 2023-11-26T07:12:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe367a78700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36006f7d0 con 0x7fe36015e1b0 2023-11-26T07:12:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe367a78700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36015e770 con 0x7fe36006f380 2023-11-26T07:12:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe366015700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe36006f380 0x7fe36006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe365814700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe360073f90 0x7fe3600b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55400/0 (socket says 172.21.15.83:55400) 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe365814700 1 -- 172.21.15.83:0/864566633 learned_addr learned my addr 172.21.15.83:0/864566633 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe366015700 1 -- 172.21.15.83:0/864566633 >> v1:172.21.15.203:6789/0 conn(0x7fe36015e1b0 legacy=0x7fe360074370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe366015700 1 -- 172.21.15.83:0/864566633 >> v1:172.21.15.83:6789/0 conn(0x7fe360073f90 legacy=0x7fe3600b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe366015700 1 -- 172.21.15.83:0/864566633 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3600b5630 con 0x7fe36006f380 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe364812700 1 -- 172.21.15.83:0/864566633 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360007417 0 0) 0x7fe350001110 con 0x7fe360073f90 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe366015700 1 --2- 172.21.15.83:0/864566633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe36006f380 0x7fe36006aff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe358009d90 tx=0x7fe35800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa45d49073f81f1e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe364812700 1 -- 172.21.15.83:0/864566633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35800e070 con 0x7fe36006f380 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe364812700 1 -- 172.21.15.83:0/864566633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35800bbd0 con 0x7fe36006f380 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.413+0000 7fe364812700 1 -- 172.21.15.83:0/864566633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358003a20 con 0x7fe36006f380 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.414+0000 7fe367a78700 1 -- 172.21.15.83:0/864566633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe36006f380 msgr2=0x7fe36006aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.414+0000 7fe367a78700 1 --2- 172.21.15.83:0/864566633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe36006f380 0x7fe36006aff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe358009d90 tx=0x7fe35800b330 comp rx=0 tx=0).stop 2023-11-26T07:12:03.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.414+0000 7fe367a78700 1 -- 172.21.15.83:0/864566633 shutdown_connections 2023-11-26T07:12:03.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.414+0000 7fe367a78700 1 --2- 172.21.15.83:0/864566633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe36006f380 0x7fe36006aff0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.414+0000 7fe367a78700 1 -- 172.21.15.83:0/864566633 >> 172.21.15.83:0/864566633 conn(0x7fe3600d0c40 msgr2=0x7fe3600d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:03.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.414+0000 7fe367a78700 1 -- 172.21.15.83:0/864566633 shutdown_connections 2023-11-26T07:12:03.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.414+0000 7fe367a78700 1 -- 172.21.15.83:0/864566633 wait complete. 2023-11-26T07:12:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.415+0000 7fe367a78700 1 Processor -- start 2023-11-26T07:12:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.415+0000 7fe367a78700 1 -- start start 2023-11-26T07:12:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe367a78700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36006f380 0x7fe3600cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe367a78700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360073f90 0x7fe3601e93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe367a78700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36015e1b0 0x7fe360145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe367a78700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3601e9900 con 0x7fe360073f90 2023-11-26T07:12:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe367a78700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe360145f70 con 0x7fe36015e1b0 2023-11-26T07:12:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe367a78700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3601e9b80 con 0x7fe36006f380 2023-11-26T07:12:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe365013700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360073f90 0x7fe3601e93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe365013700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360073f90 0x7fe3601e93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35368/0 (socket says 172.21.15.83:35368) 2023-11-26T07:12:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe365013700 1 -- 172.21.15.83:0/1141247395 learned_addr learned my addr 172.21.15.83:0/1141247395 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe366015700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36015e1b0 0x7fe360145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe365814700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36006f380 0x7fe3600cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe365013700 1 -- 172.21.15.83:0/1141247395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36006f380 msgr2=0x7fe3600cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe365013700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36006f380 0x7fe3600cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe365013700 1 -- 172.21.15.83:0/1141247395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36015e1b0 msgr2=0x7fe360145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe365013700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36015e1b0 0x7fe360145a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe365013700 1 -- 172.21.15.83:0/1141247395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe358009a70 con 0x7fe360073f90 2023-11-26T07:12:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe366015700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36015e1b0 0x7fe360145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe365013700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360073f90 0x7fe3601e93c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00b450 tx=0x7fe35c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:03.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe3567fc700 1 -- 172.21.15.83:0/1141247395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c011830 con 0x7fe360073f90 2023-11-26T07:12:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.416+0000 7fe3567fc700 1 -- 172.21.15.83:0/1141247395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c011990 con 0x7fe360073f90 2023-11-26T07:12:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.417+0000 7fe367a78700 1 -- 172.21.15.83:0/1141247395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3601e9e60 con 0x7fe360073f90 2023-11-26T07:12:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.417+0000 7fe3567fc700 1 -- 172.21.15.83:0/1141247395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c011d40 con 0x7fe360073f90 2023-11-26T07:12:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.417+0000 7fe367a78700 1 -- 172.21.15.83:0/1141247395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe360146420 con 0x7fe360073f90 2023-11-26T07:12:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.417+0000 7fe367a78700 1 -- 172.21.15.83:0/1141247395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3601ea380 con 0x7fe360073f90 2023-11-26T07:12:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.418+0000 7fe3567fc700 1 -- 172.21.15.83:0/1141247395 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe35c01a400 con 0x7fe360073f90 2023-11-26T07:12:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.418+0000 7fe3567fc700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe34406aed0 0x7fe34406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.418+0000 7fe3567fc700 1 -- 172.21.15.83:0/1141247395 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe35c089260 con 0x7fe360073f90 2023-11-26T07:12:03.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.418+0000 7fe365814700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe34406aed0 0x7fe34406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.419+0000 7fe365814700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe34406aed0 0x7fe34406d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe350000f60 tx=0x7fe350009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:03.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.423+0000 7fe3567fc700 1 -- 172.21.15.83:0/1141247395 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe35c0543a0 con 0x7fe360073f90 2023-11-26T07:12:03.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.612+0000 7fe367a78700 1 -- 172.21.15.83:0/1141247395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe360068fb0 con 0x7fe360073f90 2023-11-26T07:12:03.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.613+0000 7fe3567fc700 1 -- 172.21.15.83:0/1141247395 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe35c0579c0 con 0x7fe360073f90 2023-11-26T07:12:03.615 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:12:03.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 -- 172.21.15.83:0/1141247395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe34406aed0 msgr2=0x7fe34406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe34406aed0 0x7fe34406d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe350000f60 tx=0x7fe350009040 comp rx=0 tx=0).stop 2023-11-26T07:12:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 -- 172.21.15.83:0/1141247395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360073f90 msgr2=0x7fe3601e93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360073f90 0x7fe3601e93c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00b450 tx=0x7fe35c00b810 comp rx=0 tx=0).stop 2023-11-26T07:12:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 -- 172.21.15.83:0/1141247395 shutdown_connections 2023-11-26T07:12:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe36015e1b0 0x7fe360145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe34406aed0 0x7fe34406d390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe36006f380 0x7fe3600cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 --2- 172.21.15.83:0/1141247395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe360073f90 0x7fe3601e93c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 -- 172.21.15.83:0/1141247395 >> 172.21.15.83:0/1141247395 conn(0x7fe3600d0c40 msgr2=0x7fe36006e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 -- 172.21.15.83:0/1141247395 shutdown_connections 2023-11-26T07:12:03.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.615+0000 7fe367a78700 1 -- 172.21.15.83:0/1141247395 wait complete. 2023-11-26T07:12:03.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:12:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.748+0000 7f4c0e2c4700 1 Processor -- start 2023-11-26T07:12:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.748+0000 7f4c0e2c4700 1 -- start start 2023-11-26T07:12:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.749+0000 7f4c0e2c4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c0806c710 0x7f4c080cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.749+0000 7f4c0e2c4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4c080b49a0 0x7f4c080d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.749+0000 7f4c0e2c4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080d09a0 con 0x7f4c08152cc0 2023-11-26T07:12:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.749+0000 7f4c0e2c4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0815c4b0 con 0x7f4c0806c710 2023-11-26T07:12:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.749+0000 7f4c0e2c4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0815c790 con 0x7f4c080b49a0 2023-11-26T07:12:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.749+0000 7f4c077fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c0806c710 0x7f4c080cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.749+0000 7f4c07fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4c08152cc0 0x7f4c0806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55414/0 (socket says 172.21.15.83:55414) 2023-11-26T07:12:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.749+0000 7f4c07fff700 1 -- 172.21.15.83:0/2631104433 learned_addr learned my addr 172.21.15.83:0/2631104433 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.750+0000 7f4c077fe700 1 -- 172.21.15.83:0/2631104433 >> v2:172.21.15.203:3300/0 conn(0x7f4c080b49a0 msgr2=0x7f4c080d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.750+0000 7f4c0c861700 1 --2- 172.21.15.83:0/2631104433 >> v2:172.21.15.203:3300/0 conn(0x7f4c080b49a0 0x7f4c080d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.750+0000 7f4c077fe700 1 --2- 172.21.15.83:0/2631104433 >> v2:172.21.15.203:3300/0 conn(0x7f4c080b49a0 0x7f4c080d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.750+0000 7f4c06ffd700 1 -- 172.21.15.83:0/2631104433 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083859469 0 0) 0x7f4c080d09a0 con 0x7f4c08152cc0 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.750+0000 7f4c077fe700 1 -- 172.21.15.83:0/2631104433 >> v1:172.21.15.83:6789/0 conn(0x7f4c08152cc0 legacy=0x7f4c0806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.750+0000 7f4c077fe700 1 -- 172.21.15.83:0/2631104433 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0815ca70 con 0x7f4c0806c710 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.750+0000 7f4c0c861700 1 --2- 172.21.15.83:0/2631104433 >> v2:172.21.15.203:3300/0 conn(0x7f4c080b49a0 0x7f4c080d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.750+0000 7f4c077fe700 1 --2- 172.21.15.83:0/2631104433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c0806c710 0x7f4c080cff20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4c080b6300 tx=0x7f4bf8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6d4c7dc5bf12d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.750+0000 7f4c06ffd700 1 -- 172.21.15.83:0/2631104433 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8011830 con 0x7f4c0806c710 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.750+0000 7f4c06ffd700 1 -- 172.21.15.83:0/2631104433 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf8011990 con 0x7f4c0806c710 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.750+0000 7f4c06ffd700 1 -- 172.21.15.83:0/2631104433 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8011d20 con 0x7f4c0806c710 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.751+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/2631104433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c0806c710 msgr2=0x7f4c080cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.751+0000 7f4c0e2c4700 1 --2- 172.21.15.83:0/2631104433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c0806c710 0x7f4c080cff20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4c080b6300 tx=0x7f4bf8009b20 comp rx=0 tx=0).stop 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.751+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/2631104433 shutdown_connections 2023-11-26T07:12:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.751+0000 7f4c0e2c4700 1 --2- 172.21.15.83:0/2631104433 >> v2:172.21.15.203:3300/0 conn(0x7f4c080b49a0 0x7f4c080d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.751+0000 7f4c0e2c4700 1 --2- 172.21.15.83:0/2631104433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c0806c710 0x7f4c080cff20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.751+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/2631104433 >> 172.21.15.83:0/2631104433 conn(0x7f4c08071b90 msgr2=0x7f4c08071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.751+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/2631104433 shutdown_connections 2023-11-26T07:12:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.751+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/2631104433 wait complete. 2023-11-26T07:12:03.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.752+0000 7f4c0e2c4700 1 Processor -- start 2023-11-26T07:12:03.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.752+0000 7f4c0e2c4700 1 -- start start 2023-11-26T07:12:03.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0e2c4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c0806c710 0x7f4c0806da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0e2c4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c080b49a0 0x7f4c0814e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0e2c4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08152cc0 0x7f4c0814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0e2c4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c081534d0 con 0x7f4c08152cc0 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0e2c4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0814eec0 con 0x7f4c0806c710 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0e2c4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c08150250 con 0x7f4c080b49a0 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0c861700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08152cc0 0x7f4c0814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0c861700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08152cc0 0x7f4c0814f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35384/0 (socket says 172.21.15.83:35384) 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0c861700 1 -- 172.21.15.83:0/259444638 learned_addr learned my addr 172.21.15.83:0/259444638 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c07fff700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c0806c710 0x7f4c0806da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c077fe700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c080b49a0 0x7f4c0814e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0c861700 1 -- 172.21.15.83:0/259444638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c080b49a0 msgr2=0x7f4c0814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0c861700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c080b49a0 0x7f4c0814e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0c861700 1 -- 172.21.15.83:0/259444638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c0806c710 msgr2=0x7f4c0806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0c861700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c0806c710 0x7f4c0806da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0c861700 1 -- 172.21.15.83:0/259444638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf80094a0 con 0x7f4c08152cc0 2023-11-26T07:12:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c0c861700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08152cc0 0x7f4c0814f0d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000cef0 tx=0x7f4c0000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.753+0000 7f4c04ff9700 1 -- 172.21.15.83:0/259444638 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000ad00 con 0x7f4c08152cc0 2023-11-26T07:12:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.754+0000 7f4c04ff9700 1 -- 172.21.15.83:0/259444638 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0000ae60 con 0x7f4c08152cc0 2023-11-26T07:12:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.754+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/259444638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0814f9b0 con 0x7f4c08152cc0 2023-11-26T07:12:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.754+0000 7f4c04ff9700 1 -- 172.21.15.83:0/259444638 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00015680 con 0x7f4c08152cc0 2023-11-26T07:12:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.754+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/259444638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0814ff70 con 0x7f4c08152cc0 2023-11-26T07:12:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.754+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/259444638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c080753f0 con 0x7f4c08152cc0 2023-11-26T07:12:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.755+0000 7f4c04ff9700 1 -- 172.21.15.83:0/259444638 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c0000d430 con 0x7f4c08152cc0 2023-11-26T07:12:03.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.755+0000 7f4c04ff9700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4bf406aed0 0x7f4bf406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:03.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.756+0000 7f4c07fff700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4bf406aed0 0x7f4bf406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:03.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.756+0000 7f4c04ff9700 1 -- 172.21.15.83:0/259444638 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c00089aa0 con 0x7f4c08152cc0 2023-11-26T07:12:03.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.756+0000 7f4c07fff700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4bf406aed0 0x7f4bf406d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc000f60 tx=0x7f4bfc007e70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:03.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.760+0000 7f4c04ff9700 1 -- 172.21.15.83:0/259444638 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c00054b50 con 0x7f4c08152cc0 2023-11-26T07:12:03.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.965+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/259444638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4c0815f9e0 con 0x7f4c08152cc0 2023-11-26T07:12:03.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.965+0000 7f4c04ff9700 1 -- 172.21.15.83:0/259444638 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4c0815f9e0 con 0x7f4c08152cc0 2023-11-26T07:12:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/259444638 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4bf406aed0 msgr2=0x7f4bf406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4bf406aed0 0x7f4bf406d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc000f60 tx=0x7f4bfc007e70 comp rx=0 tx=0).stop 2023-11-26T07:12:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/259444638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08152cc0 msgr2=0x7f4c0814f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08152cc0 0x7f4c0814f0d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000cef0 tx=0x7f4c0000cf20 comp rx=0 tx=0).stop 2023-11-26T07:12:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/259444638 shutdown_connections 2023-11-26T07:12:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08152cc0 0x7f4c0814f0d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c0806c710 0x7f4c0806da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4bf406aed0 0x7f4bf406d390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 --2- 172.21.15.83:0/259444638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c080b49a0 0x7f4c0814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/259444638 >> 172.21.15.83:0/259444638 conn(0x7f4c08071b90 msgr2=0x7f4c0815b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/259444638 shutdown_connections 2023-11-26T07:12:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:03.967+0000 7f4c0e2c4700 1 -- 172.21.15.83:0/259444638 wait complete. 2023-11-26T07:12:03.971 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:12:03.981 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:12:03.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:03.982 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-26T07:12:04.026 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:12:04.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:04.026 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:12:04.098 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:04.098 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:04.116 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:04.116 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:04.172 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:04.172 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:04.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:04.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.387+0000 7f85cbfca700 1 Processor -- start 2023-11-26T07:12:04.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.387+0000 7f85cbfca700 1 -- start start 2023-11-26T07:12:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85cbfca700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40b1a70 0x7f85c40b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85cbfca700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c40cfd40 con 0x7f85c415b200 2023-11-26T07:12:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85cbfca700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c40affa0 con 0x7f85c415ae20 2023-11-26T07:12:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85cbfca700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40d0020 con 0x7f85c40b1a70 2023-11-26T07:12:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c9565700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f85c415b200 0x7f85c4162b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55430/0 (socket says 172.21.15.83:55430) 2023-11-26T07:12:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c9565700 1 -- 172.21.15.83:0/3238299994 learned_addr learned my addr 172.21.15.83:0/3238299994 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186708939 0 0) 0x7f85c40cfd40 con 0x7f85c415b200 2023-11-26T07:12:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b00035b0 con 0x7f85c415b200 2023-11-26T07:12:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85ca567700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40b1a70 0x7f85c40b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3854151339 0 0) 0x7f85b00035b0 con 0x7f85c415b200 2023-11-26T07:12:04.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40b1a70 msgr2=0x7f85c40b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40b1a70 0x7f85c40b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 >> v1:172.21.15.203:6789/0 conn(0x7f85c415ae20 legacy=0x7f85c40b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40b0280 con 0x7f85c415b200 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f85b4001670 con 0x7f85c415b200 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 >> v1:172.21.15.83:6789/0 conn(0x7f85c415b200 legacy=0x7f85c4162b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40b1a70 0x7f85b0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85b0004060 0x7f85b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b0004d90 0x7f85b0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40d0020 con 0x7f85b0004d90 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b000bbd0 con 0x7f85b0004060 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b000bd90 con 0x7f85c40b1a70 2023-11-26T07:12:04.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.388+0000 7f85ca567700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b0004d90 0x7f85b0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85c9565700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85b0004060 0x7f85b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85c9d66700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40b1a70 0x7f85b0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85ca567700 1 -- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40b1a70 msgr2=0x7f85b0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85ca567700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40b1a70 0x7f85b0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85ca567700 1 -- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85b0004060 msgr2=0x7f85b0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85ca567700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85b0004060 0x7f85b0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85ca567700 1 -- 172.21.15.83:0/3238299994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b00035b0 con 0x7f85b0004d90 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85c9565700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85b0004060 0x7f85b0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85ca567700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b0004d90 0x7f85b0005210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f85c40b2ee0 tx=0x7f85b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e7f09f0d2e3d5e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc007e20 con 0x7f85b0004d90 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc003a20 con 0x7f85b0004d90 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85c8d64700 1 -- 172.21.15.83:0/3238299994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc003dd0 con 0x7f85b0004d90 2023-11-26T07:12:04.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85cbfca700 1 -- 172.21.15.83:0/3238299994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b0004d90 msgr2=0x7f85b0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.389+0000 7f85cbfca700 1 --2- 172.21.15.83:0/3238299994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85b0004d90 0x7f85b0005210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f85c40b2ee0 tx=0x7f85b0005880 comp rx=0 tx=0).stop 2023-11-26T07:12:04.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.390+0000 7f85ca567700 1 -- 172.21.15.83:0/3238299994 reap_dead start 2023-11-26T07:12:04.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.390+0000 7f85cbfca700 1 -- 172.21.15.83:0/3238299994 shutdown_connections 2023-11-26T07:12:04.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.390+0000 7f85cbfca700 1 -- 172.21.15.83:0/3238299994 >> 172.21.15.83:0/3238299994 conn(0x7f85c40b57b0 msgr2=0x7f85c40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:04.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.390+0000 7f85cbfca700 1 -- 172.21.15.83:0/3238299994 shutdown_connections 2023-11-26T07:12:04.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.390+0000 7f85cbfca700 1 -- 172.21.15.83:0/3238299994 wait complete. 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.391+0000 7f85cbfca700 1 Processor -- start 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.391+0000 7f85cbfca700 1 -- start start 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85cbfca700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85b0004d90 0x7f85c41f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85cbfca700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c40af7f0 0x7f85c41f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85cbfca700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c415d650 0x7f85c40c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85cbfca700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c41f1510 con 0x7f85c415d650 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85cbfca700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c415e560 con 0x7f85c40af7f0 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85cbfca700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c415dfa0 con 0x7f85b0004d90 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ca567700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c415d650 0x7f85c40c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ca567700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c415d650 0x7f85c40c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35404/0 (socket says 172.21.15.83:35404) 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ca567700 1 -- 172.21.15.83:0/2139890871 learned_addr learned my addr 172.21.15.83:0/2139890871 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:04.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85c9d66700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85b0004d90 0x7f85c41f0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85c9565700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c40af7f0 0x7f85c41f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ca567700 1 -- 172.21.15.83:0/2139890871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85b0004d90 msgr2=0x7f85c41f0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ca567700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85b0004d90 0x7f85c41f0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ca567700 1 -- 172.21.15.83:0/2139890871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c40af7f0 msgr2=0x7f85c41f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ca567700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c40af7f0 0x7f85c41f7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ca567700 1 -- 172.21.15.83:0/2139890871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c415e280 con 0x7f85c415d650 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ca567700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c415d650 0x7f85c40c9610 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f85bc013000 tx=0x7f85bc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2139890871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc009410 con 0x7f85c415d650 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2139890871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc004240 con 0x7f85c415d650 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85cbfca700 1 -- 172.21.15.83:0/2139890871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c41f7140 con 0x7f85c415d650 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2139890871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc0045f0 con 0x7f85c415d650 2023-11-26T07:12:04.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.392+0000 7f85cbfca700 1 -- 172.21.15.83:0/2139890871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c41f7e60 con 0x7f85c415d650 2023-11-26T07:12:04.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.393+0000 7f85cbfca700 1 -- 172.21.15.83:0/2139890871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c4067f60 con 0x7f85c415d650 2023-11-26T07:12:04.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.394+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2139890871 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85bc004750 con 0x7f85c415d650 2023-11-26T07:12:04.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.394+0000 7f85ba7fc700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85a806aed0 0x7f85a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.394+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2139890871 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85bc08a980 con 0x7f85c415d650 2023-11-26T07:12:04.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.394+0000 7f85c9d66700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85a806aed0 0x7f85a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.394+0000 7f85c9d66700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85a806aed0 0x7f85a806d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f85c0000c00 tx=0x7f85c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:04.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.398+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2139890871 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85bc055c90 con 0x7f85c415d650 2023-11-26T07:12:04.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.590+0000 7f85cbfca700 1 -- 172.21.15.83:0/2139890871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85c40cde80 con 0x7f85c415d650 2023-11-26T07:12:04.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.592+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2139890871 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f85bc082e60 con 0x7f85c415d650 2023-11-26T07:12:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:04.595 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:04.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 -- 172.21.15.83:0/2139890871 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85a806aed0 msgr2=0x7f85a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85a806aed0 0x7f85a806d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f85c0000c00 tx=0x7f85c0009040 comp rx=0 tx=0).stop 2023-11-26T07:12:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 -- 172.21.15.83:0/2139890871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c415d650 msgr2=0x7f85c40c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c415d650 0x7f85c40c9610 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f85bc013000 tx=0x7f85bc004b40 comp rx=0 tx=0).stop 2023-11-26T07:12:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 -- 172.21.15.83:0/2139890871 shutdown_connections 2023-11-26T07:12:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c415d650 0x7f85c40c9610 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85a806aed0 0x7f85a806d390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85b0004d90 0x7f85c41f0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 --2- 172.21.15.83:0/2139890871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c40af7f0 0x7f85c41f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 -- 172.21.15.83:0/2139890871 >> 172.21.15.83:0/2139890871 conn(0x7f85c40b57b0 msgr2=0x7f85c40b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 -- 172.21.15.83:0/2139890871 shutdown_connections 2023-11-26T07:12:04.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.594+0000 7f85cbfca700 1 -- 172.21.15.83:0/2139890871 wait complete. 2023-11-26T07:12:04.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:12:04.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:12:04.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.730+0000 7fdbaaab3700 1 Processor -- start 2023-11-26T07:12:04.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.730+0000 7fdbaaab3700 1 -- start start 2023-11-26T07:12:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdbaaab3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdba40cf610 0x7fdba40add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdbaaab3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba406b080 con 0x7fdba4073f90 2023-11-26T07:12:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdbaaab3700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba415c290 con 0x7fdba415f9e0 2023-11-26T07:12:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdbaaab3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba40cf9f0 con 0x7fdba40cf610 2023-11-26T07:12:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdba884f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdba40cf610 0x7fdba40add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdba9050700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdba4073f90 0x7fdba406b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55452/0 (socket says 172.21.15.83:55452) 2023-11-26T07:12:04.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdba9050700 1 -- 172.21.15.83:0/768158472 learned_addr learned my addr 172.21.15.83:0/768158472 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:04.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdba884f700 1 -- 172.21.15.83:0/768158472 >> v1:172.21.15.203:6789/0 conn(0x7fdba415f9e0 legacy=0x7fdba40b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:04.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdba884f700 1 -- 172.21.15.83:0/768158472 >> v1:172.21.15.83:6789/0 conn(0x7fdba4073f90 legacy=0x7fdba406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdba884f700 1 -- 172.21.15.83:0/768158472 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba415ff80 con 0x7fdba40cf610 2023-11-26T07:12:04.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdba37fe700 1 -- 172.21.15.83:0/768158472 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387987579 0 0) 0x7fdba406b080 con 0x7fdba4073f90 2023-11-26T07:12:04.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.731+0000 7fdba884f700 1 --2- 172.21.15.83:0/768158472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdba40cf610 0x7fdba40add40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009870 tx=0x7fdb98009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce17930f23149339 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:04.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.732+0000 7fdba37fe700 1 -- 172.21.15.83:0/768158472 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98004230 con 0x7fdba40cf610 2023-11-26T07:12:04.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.732+0000 7fdba37fe700 1 -- 172.21.15.83:0/768158472 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98004390 con 0x7fdba40cf610 2023-11-26T07:12:04.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.732+0000 7fdba37fe700 1 -- 172.21.15.83:0/768158472 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb980036a0 con 0x7fdba40cf610 2023-11-26T07:12:04.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.732+0000 7fdbaaab3700 1 -- 172.21.15.83:0/768158472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdba40cf610 msgr2=0x7fdba40add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.732+0000 7fdbaaab3700 1 --2- 172.21.15.83:0/768158472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdba40cf610 0x7fdba40add40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009870 tx=0x7fdb98009b80 comp rx=0 tx=0).stop 2023-11-26T07:12:04.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.732+0000 7fdbaaab3700 1 -- 172.21.15.83:0/768158472 shutdown_connections 2023-11-26T07:12:04.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.732+0000 7fdbaaab3700 1 --2- 172.21.15.83:0/768158472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdba40cf610 0x7fdba40add40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.732+0000 7fdbaaab3700 1 -- 172.21.15.83:0/768158472 >> 172.21.15.83:0/768158472 conn(0x7fdba40d0bf0 msgr2=0x7fdba40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:04.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.732+0000 7fdbaaab3700 1 -- 172.21.15.83:0/768158472 shutdown_connections 2023-11-26T07:12:04.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.732+0000 7fdbaaab3700 1 -- 172.21.15.83:0/768158472 wait complete. 2023-11-26T07:12:04.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.733+0000 7fdbaaab3700 1 Processor -- start 2023-11-26T07:12:04.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.733+0000 7fdbaaab3700 1 -- start start 2023-11-26T07:12:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.734+0000 7fdbaaab3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdba4073f90 0x7fdba414a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.734+0000 7fdbaaab3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdba40cf610 0x7fdba4151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.734+0000 7fdbaaab3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba415f9e0 0x7fdba415a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.734+0000 7fdbaaab3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba415dfa0 con 0x7fdba4073f90 2023-11-26T07:12:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.734+0000 7fdbaaab3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba414ac70 con 0x7fdba415f9e0 2023-11-26T07:12:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.734+0000 7fdbaaab3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba415c7e0 con 0x7fdba40cf610 2023-11-26T07:12:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.734+0000 7fdba884f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdba4073f90 0x7fdba414a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.734+0000 7fdba884f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdba4073f90 0x7fdba414a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35414/0 (socket says 172.21.15.83:35414) 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.734+0000 7fdba884f700 1 -- 172.21.15.83:0/3920424081 learned_addr learned my addr 172.21.15.83:0/3920424081 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdba884f700 1 -- 172.21.15.83:0/3920424081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdba40cf610 msgr2=0x7fdba4151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdba884f700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdba40cf610 0x7fdba4151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdba884f700 1 -- 172.21.15.83:0/3920424081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba415f9e0 msgr2=0x7fdba415a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdba9050700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba415f9e0 0x7fdba415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdba884f700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba415f9e0 0x7fdba415a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdba884f700 1 -- 172.21.15.83:0/3920424081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb980094a0 con 0x7fdba4073f90 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdba9050700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba415f9e0 0x7fdba415a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdba884f700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdba4073f90 0x7fdba414a730 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdb980090b0 tx=0x7fdb98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdba17fa700 1 -- 172.21.15.83:0/3920424081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98011830 con 0x7fdba4073f90 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdba17fa700 1 -- 172.21.15.83:0/3920424081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9801e070 con 0x7fdba4073f90 2023-11-26T07:12:04.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdbaaab3700 1 -- 172.21.15.83:0/3920424081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba4153a80 con 0x7fdba4073f90 2023-11-26T07:12:04.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdba17fa700 1 -- 172.21.15.83:0/3920424081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98011bd0 con 0x7fdba4073f90 2023-11-26T07:12:04.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.735+0000 7fdbaaab3700 1 -- 172.21.15.83:0/3920424081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba415dc30 con 0x7fdba4073f90 2023-11-26T07:12:04.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.736+0000 7fdbaaab3700 1 -- 172.21.15.83:0/3920424081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba4153fa0 con 0x7fdba4073f90 2023-11-26T07:12:04.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.736+0000 7fdba17fa700 1 -- 172.21.15.83:0/3920424081 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb98011d30 con 0x7fdba4073f90 2023-11-26T07:12:04.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.737+0000 7fdba17fa700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb8806aed0 0x7fdb8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:04.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.737+0000 7fdba17fa700 1 -- 172.21.15.83:0/3920424081 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb9808a970 con 0x7fdba4073f90 2023-11-26T07:12:04.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.737+0000 7fdba3fff700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb8806aed0 0x7fdb8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:04.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.737+0000 7fdba3fff700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb8806aed0 0x7fdb8806d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdba4157b80 tx=0x7fdb90006c50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:04.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.741+0000 7fdba17fa700 1 -- 172.21.15.83:0/3920424081 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdba4153fa0 con 0x7fdba4073f90 2023-11-26T07:12:04.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.930+0000 7fdbaaab3700 1 -- 172.21.15.83:0/3920424081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdba4153fa0 con 0x7fdba4073f90 2023-11-26T07:12:04.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.931+0000 7fdba17fa700 1 -- 172.21.15.83:0/3920424081 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdba4153fa0 con 0x7fdba4073f90 2023-11-26T07:12:04.933 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:12:04.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 -- 172.21.15.83:0/3920424081 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb8806aed0 msgr2=0x7fdb8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb8806aed0 0x7fdb8806d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdba4157b80 tx=0x7fdb90006c50 comp rx=0 tx=0).stop 2023-11-26T07:12:04.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 -- 172.21.15.83:0/3920424081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdba4073f90 msgr2=0x7fdba414a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:04.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdba4073f90 0x7fdba414a730 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdb980090b0 tx=0x7fdb98004750 comp rx=0 tx=0).stop 2023-11-26T07:12:04.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 -- 172.21.15.83:0/3920424081 shutdown_connections 2023-11-26T07:12:04.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdba415f9e0 0x7fdba415a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdba4073f90 0x7fdba414a730 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdb8806aed0 0x7fdb8806d390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 --2- 172.21.15.83:0/3920424081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdba40cf610 0x7fdba4151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:04.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 -- 172.21.15.83:0/3920424081 >> 172.21.15.83:0/3920424081 conn(0x7fdba40d0bf0 msgr2=0x7fdba40b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:04.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 -- 172.21.15.83:0/3920424081 shutdown_connections 2023-11-26T07:12:04.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:04.933+0000 7fdbaaab3700 1 -- 172.21.15.83:0/3920424081 wait complete. 2023-11-26T07:12:04.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:12:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.068+0000 7ff9464f6700 1 Processor -- start 2023-11-26T07:12:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.068+0000 7ff9464f6700 1 -- start start 2023-11-26T07:12:05.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.068+0000 7ff9464f6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff9400b5da0 0x7ff9400afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.068+0000 7ff9464f6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9400b3c60 con 0x7ff94015e160 2023-11-26T07:12:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.068+0000 7ff9464f6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9400b58c0 con 0x7ff9400746a0 2023-11-26T07:12:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.068+0000 7ff9464f6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9400b4500 con 0x7ff9400b5da0 2023-11-26T07:12:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.069+0000 7ff944a93700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff9400b5da0 0x7ff9400afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.069+0000 7ff93ffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff94015e160 0x7ff940073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55462/0 (socket says 172.21.15.83:55462) 2023-11-26T07:12:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.069+0000 7ff93ffff700 1 -- 172.21.15.83:0/2603102350 learned_addr learned my addr 172.21.15.83:0/2603102350 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.069+0000 7ff944a93700 1 -- 172.21.15.83:0/2603102350 >> v1:172.21.15.203:6789/0 conn(0x7ff9400746a0 legacy=0x7ff94006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.069+0000 7ff944a93700 1 -- 172.21.15.83:0/2603102350 >> v1:172.21.15.83:6789/0 conn(0x7ff94015e160 legacy=0x7ff940073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.069+0000 7ff944a93700 1 -- 172.21.15.83:0/2603102350 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94006bdc0 con 0x7ff9400b5da0 2023-11-26T07:12:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.069+0000 7ff944a93700 1 --2- 172.21.15.83:0/2603102350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9400b5da0 0x7ff9400afc90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff938009d90 tx=0x7ff93800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91dcb55a1afd90 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.069+0000 7ff93effd700 1 -- 172.21.15.83:0/2603102350 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93800e070 con 0x7ff9400b5da0 2023-11-26T07:12:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.069+0000 7ff93effd700 1 -- 172.21.15.83:0/2603102350 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93800bbd0 con 0x7ff9400b5da0 2023-11-26T07:12:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.069+0000 7ff93effd700 1 -- 172.21.15.83:0/2603102350 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938012430 con 0x7ff9400b5da0 2023-11-26T07:12:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.070+0000 7ff9464f6700 1 -- 172.21.15.83:0/2603102350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9400b5da0 msgr2=0x7ff9400afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.070+0000 7ff9464f6700 1 --2- 172.21.15.83:0/2603102350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9400b5da0 0x7ff9400afc90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff938009d90 tx=0x7ff93800b330 comp rx=0 tx=0).stop 2023-11-26T07:12:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.070+0000 7ff9464f6700 1 -- 172.21.15.83:0/2603102350 shutdown_connections 2023-11-26T07:12:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.070+0000 7ff9464f6700 1 --2- 172.21.15.83:0/2603102350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9400b5da0 0x7ff9400afc90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.070+0000 7ff9464f6700 1 -- 172.21.15.83:0/2603102350 >> 172.21.15.83:0/2603102350 conn(0x7ff9400d0bf0 msgr2=0x7ff9400d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.070+0000 7ff9464f6700 1 -- 172.21.15.83:0/2603102350 shutdown_connections 2023-11-26T07:12:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.070+0000 7ff9464f6700 1 -- 172.21.15.83:0/2603102350 wait complete. 2023-11-26T07:12:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.071+0000 7ff9464f6700 1 Processor -- start 2023-11-26T07:12:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.071+0000 7ff9464f6700 1 -- start start 2023-11-26T07:12:05.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff9464f6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9400746a0 0x7ff940155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff9464f6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9400b5da0 0x7ff940156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff9464f6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94015e160 0x7ff940157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff9464f6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401583a0 con 0x7ff94015e160 2023-11-26T07:12:05.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff9464f6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff940156040 con 0x7ff9400746a0 2023-11-26T07:12:05.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff9464f6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94015cd90 con 0x7ff9400b5da0 2023-11-26T07:12:05.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff944a93700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94015e160 0x7ff940157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff944a93700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94015e160 0x7ff940157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35430/0 (socket says 172.21.15.83:35430) 2023-11-26T07:12:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff944a93700 1 -- 172.21.15.83:0/740379819 learned_addr learned my addr 172.21.15.83:0/740379819 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff93ffff700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9400746a0 0x7ff940155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff93f7fe700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9400b5da0 0x7ff940156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff944a93700 1 -- 172.21.15.83:0/740379819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9400b5da0 msgr2=0x7ff940156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff944a93700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9400b5da0 0x7ff940156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff944a93700 1 -- 172.21.15.83:0/740379819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9400746a0 msgr2=0x7ff940155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff944a93700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9400746a0 0x7ff940155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff944a93700 1 -- 172.21.15.83:0/740379819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938009a70 con 0x7ff94015e160 2023-11-26T07:12:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.072+0000 7ff944a93700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94015e160 0x7ff940157180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff938014040 tx=0x7ff938004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.073+0000 7ff93cff9700 1 -- 172.21.15.83:0/740379819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938012430 con 0x7ff94015e160 2023-11-26T07:12:05.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.073+0000 7ff93cff9700 1 -- 172.21.15.83:0/740379819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff938020070 con 0x7ff94015e160 2023-11-26T07:12:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.073+0000 7ff93cff9700 1 -- 172.21.15.83:0/740379819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938004280 con 0x7ff94015e160 2023-11-26T07:12:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.073+0000 7ff9464f6700 1 -- 172.21.15.83:0/740379819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9401f3e20 con 0x7ff94015e160 2023-11-26T07:12:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.073+0000 7ff9464f6700 1 -- 172.21.15.83:0/740379819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff940157b00 con 0x7ff94015e160 2023-11-26T07:12:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.074+0000 7ff9464f6700 1 -- 172.21.15.83:0/740379819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff940068fb0 con 0x7ff94015e160 2023-11-26T07:12:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.074+0000 7ff93cff9700 1 -- 172.21.15.83:0/740379819 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9380043e0 con 0x7ff94015e160 2023-11-26T07:12:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.074+0000 7ff93cff9700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff92c06aed0 0x7ff92c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.074+0000 7ff93cff9700 1 -- 172.21.15.83:0/740379819 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff938022070 con 0x7ff94015e160 2023-11-26T07:12:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.074+0000 7ff93ffff700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff92c06aed0 0x7ff92c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.075+0000 7ff93ffff700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff92c06aed0 0x7ff92c06d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff934005910 tx=0x7ff934009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.079+0000 7ff93cff9700 1 -- 172.21.15.83:0/740379819 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff938056600 con 0x7ff94015e160 2023-11-26T07:12:05.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.268+0000 7ff9464f6700 1 -- 172.21.15.83:0/740379819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff94006b6d0 con 0x7ff94015e160 2023-11-26T07:12:05.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.269+0000 7ff93cff9700 1 -- 172.21.15.83:0/740379819 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff938059c20 con 0x7ff94015e160 2023-11-26T07:12:05.271 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:12:05.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.270+0000 7ff9464f6700 1 -- 172.21.15.83:0/740379819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff92c06aed0 msgr2=0x7ff92c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.270+0000 7ff9464f6700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff92c06aed0 0x7ff92c06d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff934005910 tx=0x7ff934009040 comp rx=0 tx=0).stop 2023-11-26T07:12:05.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.270+0000 7ff9464f6700 1 -- 172.21.15.83:0/740379819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94015e160 msgr2=0x7ff940157180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.270+0000 7ff9464f6700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94015e160 0x7ff940157180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff938014040 tx=0x7ff938004ad0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.270+0000 7ff9464f6700 1 -- 172.21.15.83:0/740379819 shutdown_connections 2023-11-26T07:12:05.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.270+0000 7ff9464f6700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff94015e160 0x7ff940157180 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.271+0000 7ff9464f6700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9400746a0 0x7ff940155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.271+0000 7ff9464f6700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff92c06aed0 0x7ff92c06d390 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.271+0000 7ff9464f6700 1 --2- 172.21.15.83:0/740379819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9400b5da0 0x7ff940156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.271+0000 7ff9464f6700 1 -- 172.21.15.83:0/740379819 >> 172.21.15.83:0/740379819 conn(0x7ff9400d0bf0 msgr2=0x7ff940161ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:05.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.271+0000 7ff9464f6700 1 -- 172.21.15.83:0/740379819 shutdown_connections 2023-11-26T07:12:05.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.271+0000 7ff9464f6700 1 -- 172.21.15.83:0/740379819 wait complete. 2023-11-26T07:12:05.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:12:05.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.406+0000 7f1f738dd700 1 Processor -- start 2023-11-26T07:12:05.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.407+0000 7f1f738dd700 1 -- start start 2023-11-26T07:12:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f738dd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1f6c1587f0 0x7f1f6c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f738dd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f6c15b370 0x7f1f6c15b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f738dd700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6c06b530 con 0x7f1f6c0b3c60 2023-11-26T07:12:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f738dd700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6c0ae5e0 con 0x7f1f6c15b370 2023-11-26T07:12:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f738dd700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6c15be30 con 0x7f1f6c1587f0 2023-11-26T07:12:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f70e78700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f6c15b370 0x7f1f6c15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f70e78700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f6c15b370 0x7f1f6c15b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35438/0 (socket says 172.21.15.83:35438) 2023-11-26T07:12:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f70e78700 1 -- 172.21.15.83:0/1862334599 learned_addr learned my addr 172.21.15.83:0/1862334599 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f71679700 1 --2- 172.21.15.83:0/1862334599 >> v2:172.21.15.203:3300/0 conn(0x7f1f6c1587f0 0x7f1f6c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f70e78700 1 -- 172.21.15.83:0/1862334599 >> v2:172.21.15.203:3300/0 conn(0x7f1f6c1587f0 msgr2=0x7f1f6c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f70e78700 1 --2- 172.21.15.83:0/1862334599 >> v2:172.21.15.203:3300/0 conn(0x7f1f6c1587f0 0x7f1f6c06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f70e78700 1 -- 172.21.15.83:0/1862334599 >> v1:172.21.15.203:6789/0 conn(0x7f1f6c0b3c60 legacy=0x7f1f6c0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f70e78700 1 -- 172.21.15.83:0/1862334599 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c15c110 con 0x7f1f6c15b370 2023-11-26T07:12:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f71679700 1 --2- 172.21.15.83:0/1862334599 >> v2:172.21.15.203:3300/0 conn(0x7f1f6c1587f0 0x7f1f6c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.408+0000 7f1f70e78700 1 --2- 172.21.15.83:0/1862334599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f6c15b370 0x7f1f6c15b800 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0098f0 tx=0x7f1f5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57eecd8a9e6ab90 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.409+0000 7f1f63fff700 1 -- 172.21.15.83:0/1862334599 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00b860 con 0x7f1f6c15b370 2023-11-26T07:12:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.409+0000 7f1f63fff700 1 -- 172.21.15.83:0/1862334599 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c00b9c0 con 0x7f1f6c15b370 2023-11-26T07:12:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.409+0000 7f1f63fff700 1 -- 172.21.15.83:0/1862334599 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00bd50 con 0x7f1f6c15b370 2023-11-26T07:12:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.409+0000 7f1f738dd700 1 -- 172.21.15.83:0/1862334599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f6c15b370 msgr2=0x7f1f6c15b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.409+0000 7f1f738dd700 1 --2- 172.21.15.83:0/1862334599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f6c15b370 0x7f1f6c15b800 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0098f0 tx=0x7f1f5c009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.409+0000 7f1f738dd700 1 -- 172.21.15.83:0/1862334599 shutdown_connections 2023-11-26T07:12:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.409+0000 7f1f738dd700 1 --2- 172.21.15.83:0/1862334599 >> v2:172.21.15.203:3300/0 conn(0x7f1f6c1587f0 0x7f1f6c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.409+0000 7f1f738dd700 1 --2- 172.21.15.83:0/1862334599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f6c15b370 0x7f1f6c15b800 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.409+0000 7f1f738dd700 1 -- 172.21.15.83:0/1862334599 >> 172.21.15.83:0/1862334599 conn(0x7f1f6c0d0bf0 msgr2=0x7f1f6c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:05.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.409+0000 7f1f738dd700 1 -- 172.21.15.83:0/1862334599 shutdown_connections 2023-11-26T07:12:05.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.409+0000 7f1f738dd700 1 -- 172.21.15.83:0/1862334599 wait complete. 2023-11-26T07:12:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.410+0000 7f1f738dd700 1 Processor -- start 2023-11-26T07:12:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.410+0000 7f1f738dd700 1 -- start start 2023-11-26T07:12:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.411+0000 7f1f738dd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f6c0b3c60 0x7f1f6c09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.411+0000 7f1f738dd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f6c1587f0 0x7f1f6c14b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.411+0000 7f1f738dd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f6c15b370 0x7f1f6c09d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.411+0000 7f1f738dd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c145570 con 0x7f1f6c1587f0 2023-11-26T07:12:05.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.411+0000 7f1f738dd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c145850 con 0x7f1f6c0b3c60 2023-11-26T07:12:05.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.411+0000 7f1f738dd700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c145b30 con 0x7f1f6c15b370 2023-11-26T07:12:05.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.411+0000 7f1f70e78700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f6c1587f0 0x7f1f6c14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.411+0000 7f1f70e78700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f6c1587f0 0x7f1f6c14b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35444/0 (socket says 172.21.15.83:35444) 2023-11-26T07:12:05.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.411+0000 7f1f70e78700 1 -- 172.21.15.83:0/3007968460 learned_addr learned my addr 172.21.15.83:0/3007968460 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:05.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.411+0000 7f1f71679700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f6c0b3c60 0x7f1f6c09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.411+0000 7f1f71e7a700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f6c15b370 0x7f1f6c09d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f70e78700 1 -- 172.21.15.83:0/3007968460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f6c15b370 msgr2=0x7f1f6c09d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f70e78700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f6c15b370 0x7f1f6c09d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f70e78700 1 -- 172.21.15.83:0/3007968460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f6c0b3c60 msgr2=0x7f1f6c09aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f70e78700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f6c0b3c60 0x7f1f6c09aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f70e78700 1 -- 172.21.15.83:0/3007968460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c009550 con 0x7f1f6c1587f0 2023-11-26T07:12:05.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f71e7a700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f6c15b370 0x7f1f6c09d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:05.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f70e78700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f6c1587f0 0x7f1f6c14b200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009c40 tx=0x7f1f5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:05.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f61ffb700 1 -- 172.21.15.83:0/3007968460 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0045d0 con 0x7f1f6c1587f0 2023-11-26T07:12:05.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f61ffb700 1 -- 172.21.15.83:0/3007968460 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c01e070 con 0x7f1f6c1587f0 2023-11-26T07:12:05.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f738dd700 1 -- 172.21.15.83:0/3007968460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c145e10 con 0x7f1f6c1587f0 2023-11-26T07:12:05.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f61ffb700 1 -- 172.21.15.83:0/3007968460 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0038c0 con 0x7f1f6c1587f0 2023-11-26T07:12:05.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.412+0000 7f1f738dd700 1 -- 172.21.15.83:0/3007968460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c14b740 con 0x7f1f6c1587f0 2023-11-26T07:12:05.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.413+0000 7f1f738dd700 1 -- 172.21.15.83:0/3007968460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c068fb0 con 0x7f1f6c1587f0 2023-11-26T07:12:05.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.413+0000 7f1f61ffb700 1 -- 172.21.15.83:0/3007968460 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f5c003a20 con 0x7f1f6c1587f0 2023-11-26T07:12:05.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.414+0000 7f1f61ffb700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f5006aed0 0x7f1f5006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.414+0000 7f1f61ffb700 1 -- 172.21.15.83:0/3007968460 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f5c020070 con 0x7f1f6c1587f0 2023-11-26T07:12:05.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.414+0000 7f1f71679700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f5006aed0 0x7f1f5006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.414+0000 7f1f71679700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f5006aed0 0x7f1f5006d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009990 tx=0x7f1f68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:05.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.418+0000 7f1f61ffb700 1 -- 172.21.15.83:0/3007968460 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f5c055c10 con 0x7f1f6c1587f0 2023-11-26T07:12:05.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.608+0000 7f1f738dd700 1 -- 172.21.15.83:0/3007968460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f6c06c320 con 0x7f1f6c1587f0 2023-11-26T07:12:05.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.608+0000 7f1f61ffb700 1 -- 172.21.15.83:0/3007968460 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f5c0150a0 con 0x7f1f6c1587f0 2023-11-26T07:12:05.611 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:12:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 -- 172.21.15.83:0/3007968460 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f5006aed0 msgr2=0x7f1f5006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f5006aed0 0x7f1f5006d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009990 tx=0x7f1f68008040 comp rx=0 tx=0).stop 2023-11-26T07:12:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 -- 172.21.15.83:0/3007968460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f6c1587f0 msgr2=0x7f1f6c14b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f6c1587f0 0x7f1f6c14b200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009c40 tx=0x7f1f5c004750 comp rx=0 tx=0).stop 2023-11-26T07:12:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 -- 172.21.15.83:0/3007968460 shutdown_connections 2023-11-26T07:12:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f5006aed0 0x7f1f5006d390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f6c15b370 0x7f1f6c09d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f6c0b3c60 0x7f1f6c09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 --2- 172.21.15.83:0/3007968460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f6c1587f0 0x7f1f6c14b200 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 -- 172.21.15.83:0/3007968460 >> 172.21.15.83:0/3007968460 conn(0x7f1f6c0d0bf0 msgr2=0x7f1f6c0aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 -- 172.21.15.83:0/3007968460 shutdown_connections 2023-11-26T07:12:05.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.610+0000 7f1f738dd700 1 -- 172.21.15.83:0/3007968460 wait complete. 2023-11-26T07:12:05.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:12:05.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.744+0000 7fc432e27700 1 Processor -- start 2023-11-26T07:12:05.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.744+0000 7fc432e27700 1 -- start start 2023-11-26T07:12:05.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc432e27700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc42c0af4d0 0x7fc42c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc432e27700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc42c0ae430 con 0x7fc42c15e080 2023-11-26T07:12:05.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc432e27700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc42c0ae710 con 0x7fc42c06c710 2023-11-26T07:12:05.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc432e27700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc42c0af0b0 con 0x7fc42c0af4d0 2023-11-26T07:12:05.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc430bc3700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc42c15e080 0x7fc42c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55490/0 (socket says 172.21.15.83:55490) 2023-11-26T07:12:05.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc430bc3700 1 -- 172.21.15.83:0/485710132 learned_addr learned my addr 172.21.15.83:0/485710132 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:05.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984855333 0 0) 0x7fc42c0ae430 con 0x7fc42c15e080 2023-11-26T07:12:05.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc41c0035b0 con 0x7fc42c15e080 2023-11-26T07:12:05.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc4313c4700 1 --2- 172.21.15.83:0/485710132 >> v2:172.21.15.203:3300/0 conn(0x7fc42c0af4d0 0x7fc42c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3005404557 0 0) 0x7fc41c0035b0 con 0x7fc42c15e080 2023-11-26T07:12:05.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 >> v2:172.21.15.203:3300/0 conn(0x7fc42c0af4d0 msgr2=0x7fc42c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:05.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc42b7fe700 1 --2- 172.21.15.83:0/485710132 >> v2:172.21.15.203:3300/0 conn(0x7fc42c0af4d0 0x7fc42c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 >> v1:172.21.15.203:6789/0 conn(0x7fc42c06c710 legacy=0x7fc42c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c0ae9f0 con 0x7fc42c15e080 2023-11-26T07:12:05.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fc420001670 con 0x7fc42c15e080 2023-11-26T07:12:05.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.745+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 >> v1:172.21.15.83:6789/0 conn(0x7fc42c15e080 legacy=0x7fc42c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc42b7fe700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c003a00 0x7fc41c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc42b7fe700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c004850 0x7fc41c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc42b7fe700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c005210 0x7fc41c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c0af0b0 con 0x7fc41c003a00 2023-11-26T07:12:05.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c00c0b0 con 0x7fc41c004850 2023-11-26T07:12:05.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c00c270 con 0x7fc41c005210 2023-11-26T07:12:05.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc430bc3700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c003a00 0x7fc41c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc42bfff700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c004850 0x7fc41c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc4313c4700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c005210 0x7fc41c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc430bc3700 1 -- 172.21.15.83:0/485710132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c005210 msgr2=0x7fc41c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:05.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc430bc3700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc41c005210 0x7fc41c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc430bc3700 1 -- 172.21.15.83:0/485710132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c004850 msgr2=0x7fc41c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:05.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc430bc3700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c004850 0x7fc41c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc4313c4700 1 -- 172.21.15.83:0/485710132 reap_dead start 2023-11-26T07:12:05.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc430bc3700 1 -- 172.21.15.83:0/485710132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c0035b0 con 0x7fc41c003a00 2023-11-26T07:12:05.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc42bfff700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c004850 0x7fc41c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:05.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc430bc3700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c003a00 0x7fc41c003e20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc42000c240 tx=0x7fc41c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca1e1cf9d5b2256 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:05.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420007c10 con 0x7fc41c003a00 2023-11-26T07:12:05.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42000cd60 con 0x7fc41c003a00 2023-11-26T07:12:05.754 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.746+0000 7fc42b7fe700 1 -- 172.21.15.83:0/485710132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420005ba0 con 0x7fc41c003a00 2023-11-26T07:12:05.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.747+0000 7fc432e27700 1 -- 172.21.15.83:0/485710132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c003a00 msgr2=0x7fc41c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.747+0000 7fc432e27700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c003a00 0x7fc41c003e20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc42000c240 tx=0x7fc41c005880 comp rx=0 tx=0).stop 2023-11-26T07:12:05.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.747+0000 7fc432e27700 1 -- 172.21.15.83:0/485710132 shutdown_connections 2023-11-26T07:12:05.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.747+0000 7fc432e27700 1 --2- 172.21.15.83:0/485710132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc41c003a00 0x7fc41c003e20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.747+0000 7fc432e27700 1 -- 172.21.15.83:0/485710132 >> 172.21.15.83:0/485710132 conn(0x7fc42c071b90 msgr2=0x7fc42c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:05.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.747+0000 7fc432e27700 1 -- 172.21.15.83:0/485710132 shutdown_connections 2023-11-26T07:12:05.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.747+0000 7fc432e27700 1 -- 172.21.15.83:0/485710132 wait complete. 2023-11-26T07:12:05.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.748+0000 7fc432e27700 1 Processor -- start 2023-11-26T07:12:05.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.748+0000 7fc432e27700 1 -- start start 2023-11-26T07:12:05.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc432e27700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c003a00 0x7fc42c141a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc432e27700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c1558f0 0x7fc42c155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc432e27700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc42c158c10 0x7fc42c1590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.758 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc432e27700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c1562b0 con 0x7fc42c1558f0 2023-11-26T07:12:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc432e27700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c159960 con 0x7fc41c003a00 2023-11-26T07:12:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc432e27700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c159c40 con 0x7fc42c158c10 2023-11-26T07:12:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc42bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c1558f0 0x7fc42c155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc42bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c1558f0 0x7fc42c155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35462/0 (socket says 172.21.15.83:35462) 2023-11-26T07:12:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc42bfff700 1 -- 172.21.15.83:0/2455015289 learned_addr learned my addr 172.21.15.83:0/2455015289 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc430bc3700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c003a00 0x7fc42c141a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc4313c4700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc42c158c10 0x7fc42c1590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc42bfff700 1 -- 172.21.15.83:0/2455015289 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc42c158c10 msgr2=0x7fc42c1590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc42bfff700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc42c158c10 0x7fc42c1590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc42bfff700 1 -- 172.21.15.83:0/2455015289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c003a00 msgr2=0x7fc42c141a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.759 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc42bfff700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c003a00 0x7fc42c141a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc42bfff700 1 -- 172.21.15.83:0/2455015289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c159f20 con 0x7fc42c1558f0 2023-11-26T07:12:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc42bfff700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c1558f0 0x7fc42c155d70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc41800e460 tx=0x7fc41800e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.749+0000 7fc4297fa700 1 -- 172.21.15.83:0/2455015289 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4180052e0 con 0x7fc42c1558f0 2023-11-26T07:12:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.750+0000 7fc4297fa700 1 -- 172.21.15.83:0/2455015289 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc418005440 con 0x7fc42c1558f0 2023-11-26T07:12:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.750+0000 7fc432e27700 1 -- 172.21.15.83:0/2455015289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42c15a200 con 0x7fc42c1558f0 2023-11-26T07:12:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.750+0000 7fc4297fa700 1 -- 172.21.15.83:0/2455015289 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4180038c0 con 0x7fc42c1558f0 2023-11-26T07:12:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.750+0000 7fc432e27700 1 -- 172.21.15.83:0/2455015289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42c159600 con 0x7fc42c1558f0 2023-11-26T07:12:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.750+0000 7fc432e27700 1 -- 172.21.15.83:0/2455015289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42c142250 con 0x7fc42c1558f0 2023-11-26T07:12:05.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.751+0000 7fc4297fa700 1 -- 172.21.15.83:0/2455015289 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc418003a20 con 0x7fc42c1558f0 2023-11-26T07:12:05.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.751+0000 7fc4297fa700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc41006aed0 0x7fc41006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:05.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.751+0000 7fc4297fa700 1 -- 172.21.15.83:0/2455015289 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc418088b90 con 0x7fc42c1558f0 2023-11-26T07:12:05.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.751+0000 7fc430bc3700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc41006aed0 0x7fc41006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:05.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.752+0000 7fc430bc3700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc41006aed0 0x7fc41006d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc420000c00 tx=0x7fc420007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:05.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.756+0000 7fc4297fa700 1 -- 172.21.15.83:0/2455015289 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc418053d80 con 0x7fc42c1558f0 2023-11-26T07:12:05.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.962+0000 7fc432e27700 1 -- 172.21.15.83:0/2455015289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc42c0753f0 con 0x7fc42c1558f0 2023-11-26T07:12:05.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.962+0000 7fc4297fa700 1 -- 172.21.15.83:0/2455015289 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc418019080 con 0x7fc42c1558f0 2023-11-26T07:12:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 -- 172.21.15.83:0/2455015289 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc41006aed0 msgr2=0x7fc41006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc41006aed0 0x7fc41006d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc420000c00 tx=0x7fc420007000 comp rx=0 tx=0).stop 2023-11-26T07:12:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 -- 172.21.15.83:0/2455015289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c1558f0 msgr2=0x7fc42c155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c1558f0 0x7fc42c155d70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc41800e460 tx=0x7fc41800e820 comp rx=0 tx=0).stop 2023-11-26T07:12:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 -- 172.21.15.83:0/2455015289 shutdown_connections 2023-11-26T07:12:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc41006aed0 0x7fc41006d390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc42c158c10 0x7fc42c1590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc41c003a00 0x7fc42c141a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 --2- 172.21.15.83:0/2455015289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc42c1558f0 0x7fc42c155d70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 -- 172.21.15.83:0/2455015289 >> 172.21.15.83:0/2455015289 conn(0x7fc42c071b90 msgr2=0x7fc42c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 -- 172.21.15.83:0/2455015289 shutdown_connections 2023-11-26T07:12:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:05.964+0000 7fc432e27700 1 -- 172.21.15.83:0/2455015289 wait complete. 2023-11-26T07:12:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:12:05.979 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-26T07:12:05.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:05.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-26T07:12:06.017 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:12:06.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:06.017 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-26T07:12:06.074 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:06.074 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:06.095 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:06.095 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:06.156 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:06.156 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:06.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:06.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.334+0000 7f57c780e700 1 Processor -- start 2023-11-26T07:12:06.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.334+0000 7f57c780e700 1 -- start start 2023-11-26T07:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57c780e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c00af4d0 0x7f57c00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57c780e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c00ae430 con 0x7f57c015e080 2023-11-26T07:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57c780e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c00ae710 con 0x7f57c006c710 2023-11-26T07:12:06.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57c780e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00af0b0 con 0x7f57c00af4d0 2023-11-26T07:12:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57c55aa700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f57c015e080 0x7f57c006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55510/0 (socket says 172.21.15.83:55510) 2023-11-26T07:12:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57c55aa700 1 -- 172.21.15.83:0/77881832 learned_addr learned my addr 172.21.15.83:0/77881832 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57c5dab700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c00af4d0 0x7f57c00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979011132 0 0) 0x7f57c00ae430 con 0x7f57c015e080 2023-11-26T07:12:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57ac0035b0 con 0x7f57c015e080 2023-11-26T07:12:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2733794628 0 0) 0x7f57ac0035b0 con 0x7f57c015e080 2023-11-26T07:12:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c00af4d0 msgr2=0x7f57c00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57b7fff700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c00af4d0 0x7f57c00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 >> v1:172.21.15.203:6789/0 conn(0x7f57c006c710 legacy=0x7f57c006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c00ae9f0 con 0x7f57c015e080 2023-11-26T07:12:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f57bc001670 con 0x7f57c015e080 2023-11-26T07:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 >> v1:172.21.15.83:6789/0 conn(0x7f57c015e080 legacy=0x7f57c006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57b7fff700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57ac003a30 0x7f57ac003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.335+0000 7f57b7fff700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c00af4d0 0x7f57ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57b7fff700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 0x7f57ac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00af0b0 con 0x7f57ac004d90 2023-11-26T07:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac00bac0 con 0x7f57ac003a30 2023-11-26T07:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac00bc80 con 0x7f57c00af4d0 2023-11-26T07:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57c5dab700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 0x7f57ac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57c55aa700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57ac003a30 0x7f57ac003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57c4da9700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c00af4d0 0x7f57ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57c5dab700 1 -- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c00af4d0 msgr2=0x7f57ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57c5dab700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c00af4d0 0x7f57ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57c5dab700 1 -- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57ac003a30 msgr2=0x7f57ac003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57c5dab700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57ac003a30 0x7f57ac003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57c5dab700 1 -- 172.21.15.83:0/77881832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac0035b0 con 0x7f57ac004d90 2023-11-26T07:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57c4da9700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c00af4d0 0x7f57ac004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57c55aa700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57ac003a30 0x7f57ac003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57c5dab700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 0x7f57ac005210 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009f90 tx=0x7f57ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5141b2441dc0fc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b800ec30 con 0x7f57ac004d90 2023-11-26T07:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b800f040 con 0x7f57ac004d90 2023-11-26T07:12:06.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.336+0000 7f57b7fff700 1 -- 172.21.15.83:0/77881832 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8014480 con 0x7f57ac004d90 2023-11-26T07:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.337+0000 7f57c780e700 1 -- 172.21.15.83:0/77881832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 msgr2=0x7f57ac005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.337+0000 7f57c780e700 1 --2- 172.21.15.83:0/77881832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 0x7f57ac005210 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009f90 tx=0x7f57ac005880 comp rx=0 tx=0).stop 2023-11-26T07:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.337+0000 7f57c5dab700 1 -- 172.21.15.83:0/77881832 reap_dead start 2023-11-26T07:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.337+0000 7f57c780e700 1 -- 172.21.15.83:0/77881832 shutdown_connections 2023-11-26T07:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.337+0000 7f57c780e700 1 -- 172.21.15.83:0/77881832 >> 172.21.15.83:0/77881832 conn(0x7f57c0071b90 msgr2=0x7f57c0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.337+0000 7f57c780e700 1 -- 172.21.15.83:0/77881832 shutdown_connections 2023-11-26T07:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.337+0000 7f57c780e700 1 -- 172.21.15.83:0/77881832 wait complete. 2023-11-26T07:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.338+0000 7f57c780e700 1 Processor -- start 2023-11-26T07:12:06.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.338+0000 7f57c780e700 1 -- start start 2023-11-26T07:12:06.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c780e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 0x7f57c00cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c780e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c0141de0 0x7f57c0144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c780e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57c0144bb0 0x7f57c01425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c780e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c0145030 con 0x7f57ac004d90 2023-11-26T07:12:06.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c780e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00cf6b0 con 0x7f57c0144bb0 2023-11-26T07:12:06.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c780e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00cf080 con 0x7f57c0141de0 2023-11-26T07:12:06.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c55aa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 0x7f57c00cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c55aa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 0x7f57c00cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35482/0 (socket says 172.21.15.83:35482) 2023-11-26T07:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c55aa700 1 -- 172.21.15.83:0/1014175347 learned_addr learned my addr 172.21.15.83:0/1014175347 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c5dab700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57c0144bb0 0x7f57c01425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c4da9700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c0141de0 0x7f57c0144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c55aa700 1 -- 172.21.15.83:0/1014175347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c0141de0 msgr2=0x7f57c0144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c55aa700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c0141de0 0x7f57c0144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c55aa700 1 -- 172.21.15.83:0/1014175347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57c0144bb0 msgr2=0x7f57c01425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c55aa700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57c0144bb0 0x7f57c01425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c55aa700 1 -- 172.21.15.83:0/1014175347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c0155550 con 0x7f57ac004d90 2023-11-26T07:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c5dab700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57c0144bb0 0x7f57c01425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c55aa700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 0x7f57c00cfb40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00eaf0 tx=0x7f57bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57b5ffb700 1 -- 172.21.15.83:0/1014175347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc005e00 con 0x7f57ac004d90 2023-11-26T07:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57b5ffb700 1 -- 172.21.15.83:0/1014175347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc00c7b0 con 0x7f57ac004d90 2023-11-26T07:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57c780e700 1 -- 172.21.15.83:0/1014175347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c0155830 con 0x7f57ac004d90 2023-11-26T07:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.339+0000 7f57b5ffb700 1 -- 172.21.15.83:0/1014175347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00cb20 con 0x7f57ac004d90 2023-11-26T07:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.340+0000 7f57c780e700 1 -- 172.21.15.83:0/1014175347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c00cf340 con 0x7f57ac004d90 2023-11-26T07:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.340+0000 7f57c780e700 1 -- 172.21.15.83:0/1014175347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c00cf500 con 0x7f57ac004d90 2023-11-26T07:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.341+0000 7f57b5ffb700 1 -- 172.21.15.83:0/1014175347 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57bc00cc80 con 0x7f57ac004d90 2023-11-26T07:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.341+0000 7f57b5ffb700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f57a406aed0 0x7f57a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.341+0000 7f57b5ffb700 1 -- 172.21.15.83:0/1014175347 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57bc089440 con 0x7f57ac004d90 2023-11-26T07:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.341+0000 7f57c4da9700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f57a406aed0 0x7f57a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.342+0000 7f57c4da9700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f57a406aed0 0x7f57a406d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57c0072bb0 tx=0x7f57b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:06.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.346+0000 7f57b5ffb700 1 -- 172.21.15.83:0/1014175347 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57bc0544f0 con 0x7f57ac004d90 2023-11-26T07:12:06.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.536+0000 7f57c780e700 1 -- 172.21.15.83:0/1014175347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57c00753f0 con 0x7f57ac004d90 2023-11-26T07:12:06.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.537+0000 7f57b5ffb700 1 -- 172.21.15.83:0/1014175347 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f57bc057b10 con 0x7f57ac004d90 2023-11-26T07:12:06.540 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:06.540 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:11:29.530714+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 -- 172.21.15.83:0/1014175347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f57a406aed0 msgr2=0x7f57a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f57a406aed0 0x7f57a406d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57c0072bb0 tx=0x7f57b0009040 comp rx=0 tx=0).stop 2023-11-26T07:12:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 -- 172.21.15.83:0/1014175347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 msgr2=0x7f57c00cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 0x7f57c00cfb40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00eaf0 tx=0x7f57bc00eeb0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 -- 172.21.15.83:0/1014175347 shutdown_connections 2023-11-26T07:12:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57c0144bb0 0x7f57c01425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac004d90 0x7f57c00cfb40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f57a406aed0 0x7f57a406d390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 --2- 172.21.15.83:0/1014175347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57c0141de0 0x7f57c0144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 -- 172.21.15.83:0/1014175347 >> 172.21.15.83:0/1014175347 conn(0x7f57c0071b90 msgr2=0x7f57c006f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:06.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 -- 172.21.15.83:0/1014175347 shutdown_connections 2023-11-26T07:12:06.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.540+0000 7f57c780e700 1 -- 172.21.15.83:0/1014175347 wait complete. 2023-11-26T07:12:06.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:12:06.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:12:06.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.676+0000 7f92a64a8700 1 Processor -- start 2023-11-26T07:12:06.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.676+0000 7f92a64a8700 1 -- start start 2023-11-26T07:12:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.676+0000 7f92a64a8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f92a015bf50 0x7f92a00ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.676+0000 7f92a64a8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a00d0140 con 0x7f92a00ce830 2023-11-26T07:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.676+0000 7f92a64a8700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a00cf3c0 con 0x7f92a00c6c30 2023-11-26T07:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.676+0000 7f92a64a8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a015c330 con 0x7f92a015bf50 2023-11-26T07:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f92a00ce830 0x7f92a00cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55530/0 (socket says 172.21.15.83:55530) 2023-11-26T07:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929f7fe700 1 -- 172.21.15.83:0/4266092873 learned_addr learned my addr 172.21.15.83:0/4266092873 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879940329 0 0) 0x7f92a00d0140 con 0x7f92a00ce830 2023-11-26T07:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f928c0035b0 con 0x7f92a00ce830 2023-11-26T07:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929ffff700 1 --2- 172.21.15.83:0/4266092873 >> v2:172.21.15.203:3300/0 conn(0x7f92a015bf50 0x7f92a00ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 962956739 0 0) 0x7f928c0035b0 con 0x7f92a00ce830 2023-11-26T07:12:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 >> v2:172.21.15.203:3300/0 conn(0x7f92a015bf50 msgr2=0x7f92a00ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:06.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 --2- 172.21.15.83:0/4266092873 >> v2:172.21.15.203:3300/0 conn(0x7f92a015bf50 0x7f92a00ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 >> v1:172.21.15.203:6789/0 conn(0x7f92a00c6c30 legacy=0x7f92a00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a00d03e0 con 0x7f92a00ce830 2023-11-26T07:12:06.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f9288001670 con 0x7f92a00ce830 2023-11-26T07:12:06.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 >> v1:172.21.15.83:6789/0 conn(0x7f92a00ce830 legacy=0x7f92a00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 --2- 172.21.15.83:0/4266092873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c003a00 0x7f928c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 --2- 172.21.15.83:0/4266092873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f928c004850 0x7f928c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 --2- 172.21.15.83:0/4266092873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f928c005210 0x7f928c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a015c330 con 0x7f928c003a00 2023-11-26T07:12:06.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c00c0b0 con 0x7f928c005210 2023-11-26T07:12:06.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928c00c270 con 0x7f928c004850 2023-11-26T07:12:06.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929ffff700 1 --2- 172.21.15.83:0/4266092873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c003a00 0x7f928c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f92a4a45700 1 --2- 172.21.15.83:0/4266092873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f928c005210 0x7f928c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.677+0000 7f929f7fe700 1 --2- 172.21.15.83:0/4266092873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f928c004850 0x7f928c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f929ffff700 1 -- 172.21.15.83:0/4266092873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f928c004850 msgr2=0x7f928c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:06.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f929ffff700 1 --2- 172.21.15.83:0/4266092873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f928c004850 0x7f928c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f929ffff700 1 -- 172.21.15.83:0/4266092873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f928c005210 msgr2=0x7f928c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f929ffff700 1 --2- 172.21.15.83:0/4266092873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f928c005210 0x7f928c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f929ffff700 1 -- 172.21.15.83:0/4266092873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c0035b0 con 0x7f928c003a00 2023-11-26T07:12:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f92a4a45700 1 -- 172.21.15.83:0/4266092873 reap_dead start 2023-11-26T07:12:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f929ffff700 1 --2- 172.21.15.83:0/4266092873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c003a00 0x7f928c003e20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9290009b60 tx=0x7f928c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa98e4b68d3e3c6f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929000e6c0 con 0x7f928c003a00 2023-11-26T07:12:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929000e820 con 0x7f928c003a00 2023-11-26T07:12:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f929effd700 1 -- 172.21.15.83:0/4266092873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929000eb90 con 0x7f928c003a00 2023-11-26T07:12:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f92a64a8700 1 -- 172.21.15.83:0/4266092873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c003a00 msgr2=0x7f928c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f92a64a8700 1 --2- 172.21.15.83:0/4266092873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c003a00 0x7f928c003e20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9290009b60 tx=0x7f928c005880 comp rx=0 tx=0).stop 2023-11-26T07:12:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f92a64a8700 1 -- 172.21.15.83:0/4266092873 shutdown_connections 2023-11-26T07:12:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f92a64a8700 1 --2- 172.21.15.83:0/4266092873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c003a00 0x7f928c003e20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f92a64a8700 1 -- 172.21.15.83:0/4266092873 >> 172.21.15.83:0/4266092873 conn(0x7f92a00b3780 msgr2=0x7f92a00b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f92a64a8700 1 -- 172.21.15.83:0/4266092873 shutdown_connections 2023-11-26T07:12:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.678+0000 7f92a64a8700 1 -- 172.21.15.83:0/4266092873 wait complete. 2023-11-26T07:12:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.680+0000 7f92a64a8700 1 Processor -- start 2023-11-26T07:12:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.680+0000 7f92a64a8700 1 -- start start 2023-11-26T07:12:06.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.680+0000 7f92a64a8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f928c003a00 0x7f92a01f2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a64a8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92a0156690 0x7f92a0143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a64a8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0156b10 0x7f92a0143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a64a8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a014f830 con 0x7f92a0156b10 2023-11-26T07:12:06.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a64a8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a0157050 con 0x7f928c003a00 2023-11-26T07:12:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a64a8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a0157330 con 0x7f92a0156690 2023-11-26T07:12:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a4a45700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0156b10 0x7f92a0143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a4a45700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0156b10 0x7f92a0143bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35506/0 (socket says 172.21.15.83:35506) 2023-11-26T07:12:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a4a45700 1 -- 172.21.15.83:0/1085427203 learned_addr learned my addr 172.21.15.83:0/1085427203 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f929f7fe700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92a0156690 0x7f92a0143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f929ffff700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f928c003a00 0x7f92a01f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a4a45700 1 -- 172.21.15.83:0/1085427203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92a0156690 msgr2=0x7f92a0143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a4a45700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92a0156690 0x7f92a0143680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a4a45700 1 -- 172.21.15.83:0/1085427203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f928c003a00 msgr2=0x7f92a01f2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a4a45700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f928c003a00 0x7f92a01f2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a4a45700 1 -- 172.21.15.83:0/1085427203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a0157610 con 0x7f92a0156b10 2023-11-26T07:12:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f929ffff700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f928c003a00 0x7f92a01f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a4a45700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0156b10 0x7f92a0143bc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f92a00cc190 tx=0x7f929800ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f929cff9700 1 -- 172.21.15.83:0/1085427203 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9298003cd0 con 0x7f92a0156b10 2023-11-26T07:12:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f929cff9700 1 -- 172.21.15.83:0/1085427203 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9298003e30 con 0x7f92a0156b10 2023-11-26T07:12:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f929cff9700 1 -- 172.21.15.83:0/1085427203 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929800d940 con 0x7f92a0156b10 2023-11-26T07:12:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a64a8700 1 -- 172.21.15.83:0/1085427203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a01578f0 con 0x7f92a0156b10 2023-11-26T07:12:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.681+0000 7f92a64a8700 1 -- 172.21.15.83:0/1085427203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a01f2ec0 con 0x7f92a0156b10 2023-11-26T07:12:06.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.682+0000 7f92a64a8700 1 -- 172.21.15.83:0/1085427203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a0068fb0 con 0x7f92a0156b10 2023-11-26T07:12:06.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.683+0000 7f929cff9700 1 -- 172.21.15.83:0/1085427203 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f929800daa0 con 0x7f92a0156b10 2023-11-26T07:12:06.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.683+0000 7f929cff9700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f928006aed0 0x7f928006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:06.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.683+0000 7f929cff9700 1 -- 172.21.15.83:0/1085427203 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92980896d0 con 0x7f92a0156b10 2023-11-26T07:12:06.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.683+0000 7f929ffff700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f928006aed0 0x7f928006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:06.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.684+0000 7f929ffff700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f928006aed0 0x7f928006d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92900090b0 tx=0x7f9290008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:06.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.688+0000 7f929cff9700 1 -- 172.21.15.83:0/1085427203 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9298054770 con 0x7f92a0156b10 2023-11-26T07:12:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.876+0000 7f92a64a8700 1 -- 172.21.15.83:0/1085427203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f92a0157e10 con 0x7f92a0156b10 2023-11-26T07:12:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.876+0000 7f929cff9700 1 -- 172.21.15.83:0/1085427203 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f92a0157e10 con 0x7f92a0156b10 2023-11-26T07:12:06.879 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:12:06.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.878+0000 7f92a64a8700 1 -- 172.21.15.83:0/1085427203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f928006aed0 msgr2=0x7f928006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.878+0000 7f92a64a8700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f928006aed0 0x7f928006d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92900090b0 tx=0x7f9290008030 comp rx=0 tx=0).stop 2023-11-26T07:12:06.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.878+0000 7f92a64a8700 1 -- 172.21.15.83:0/1085427203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0156b10 msgr2=0x7f92a0143bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:06.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.878+0000 7f92a64a8700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0156b10 0x7f92a0143bc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f92a00cc190 tx=0x7f929800ff20 comp rx=0 tx=0).stop 2023-11-26T07:12:06.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.878+0000 7f92a64a8700 1 -- 172.21.15.83:0/1085427203 shutdown_connections 2023-11-26T07:12:06.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.878+0000 7f92a64a8700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a0156b10 0x7f92a0143bc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.878+0000 7f92a64a8700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f928c003a00 0x7f92a01f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.878+0000 7f92a64a8700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f928006aed0 0x7f928006d390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.878+0000 7f92a64a8700 1 --2- 172.21.15.83:0/1085427203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92a0156690 0x7f92a0143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:06.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.878+0000 7f92a64a8700 1 -- 172.21.15.83:0/1085427203 >> 172.21.15.83:0/1085427203 conn(0x7f92a00b3780 msgr2=0x7f92a0159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:06.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.879+0000 7f92a64a8700 1 -- 172.21.15.83:0/1085427203 shutdown_connections 2023-11-26T07:12:06.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:06.879+0000 7f92a64a8700 1 -- 172.21.15.83:0/1085427203 wait complete. 2023-11-26T07:12:06.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:12:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.014+0000 7f281622c700 1 Processor -- start 2023-11-26T07:12:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.014+0000 7f281622c700 1 -- start start 2023-11-26T07:12:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f281622c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f28100cc540 0x7f281006e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f281622c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281006ef50 con 0x7f281006eb20 2023-11-26T07:12:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f281622c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28100b2f20 con 0x7f281015a620 2023-11-26T07:12:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f281622c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281015aa00 con 0x7f28100cc540 2023-11-26T07:12:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280ffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f281006eb20 0x7f28100b3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55554/0 (socket says 172.21.15.83:55554) 2023-11-26T07:12:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280ffff700 1 -- 172.21.15.83:0/2291471240 learned_addr learned my addr 172.21.15.83:0/2291471240 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413923238 0 0) 0x7f281006ef50 con 0x7f281006eb20 2023-11-26T07:12:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27f80035b0 con 0x7f281006eb20 2023-11-26T07:12:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280effd700 1 --2- 172.21.15.83:0/2291471240 >> v2:172.21.15.203:3300/0 conn(0x7f28100cc540 0x7f281006e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2914440632 0 0) 0x7f27f80035b0 con 0x7f281006eb20 2023-11-26T07:12:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 >> v2:172.21.15.203:3300/0 conn(0x7f28100cc540 msgr2=0x7f281006e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280e7fc700 1 --2- 172.21.15.83:0/2291471240 >> v2:172.21.15.203:3300/0 conn(0x7f28100cc540 0x7f281006e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 >> v1:172.21.15.203:6789/0 conn(0x7f281015a620 legacy=0x7f28100ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28100ce960 con 0x7f281006eb20 2023-11-26T07:12:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f28080019f0 con 0x7f281006eb20 2023-11-26T07:12:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 >> v1:172.21.15.83:6789/0 conn(0x7f281006eb20 legacy=0x7f28100b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280e7fc700 1 --2- 172.21.15.83:0/2291471240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f8003a00 0x7f27f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.015+0000 7f280e7fc700 1 --2- 172.21.15.83:0/2291471240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f8004850 0x7f27f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280e7fc700 1 --2- 172.21.15.83:0/2291471240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f8005210 0x7f27f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281015aa00 con 0x7f27f8004850 2023-11-26T07:12:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f800c160 con 0x7f27f8005210 2023-11-26T07:12:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f800c320 con 0x7f27f8003a00 2023-11-26T07:12:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280effd700 1 --2- 172.21.15.83:0/2291471240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f8004850 0x7f27f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280ffff700 1 --2- 172.21.15.83:0/2291471240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f8005210 0x7f27f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280f7fe700 1 --2- 172.21.15.83:0/2291471240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f8003a00 0x7f27f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280effd700 1 -- 172.21.15.83:0/2291471240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f8003a00 msgr2=0x7f27f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280effd700 1 --2- 172.21.15.83:0/2291471240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f8003a00 0x7f27f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280effd700 1 -- 172.21.15.83:0/2291471240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f8005210 msgr2=0x7f27f800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280effd700 1 --2- 172.21.15.83:0/2291471240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27f8005210 0x7f27f800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280ffff700 1 -- 172.21.15.83:0/2291471240 reap_dead start 2023-11-26T07:12:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280effd700 1 -- 172.21.15.83:0/2291471240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80035b0 con 0x7f27f8004850 2023-11-26T07:12:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280effd700 1 --2- 172.21.15.83:0/2291471240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f8004850 0x7f27f8004cd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2800009890 tx=0x7f27f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df06a463de0e262a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280000c070 con 0x7f27f8004850 2023-11-26T07:12:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2800013610 con 0x7f27f8004850 2023-11-26T07:12:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.016+0000 7f280e7fc700 1 -- 172.21.15.83:0/2291471240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28000139a0 con 0x7f27f8004850 2023-11-26T07:12:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.017+0000 7f281622c700 1 -- 172.21.15.83:0/2291471240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f8004850 msgr2=0x7f27f8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.017+0000 7f281622c700 1 --2- 172.21.15.83:0/2291471240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f8004850 0x7f27f8004cd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2800009890 tx=0x7f27f8005880 comp rx=0 tx=0).stop 2023-11-26T07:12:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.017+0000 7f281622c700 1 -- 172.21.15.83:0/2291471240 shutdown_connections 2023-11-26T07:12:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.017+0000 7f281622c700 1 --2- 172.21.15.83:0/2291471240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f8004850 0x7f27f8004cd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.017+0000 7f281622c700 1 -- 172.21.15.83:0/2291471240 >> 172.21.15.83:0/2291471240 conn(0x7f28100b2050 msgr2=0x7f28100b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.017+0000 7f281622c700 1 -- 172.21.15.83:0/2291471240 shutdown_connections 2023-11-26T07:12:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.017+0000 7f281622c700 1 -- 172.21.15.83:0/2291471240 wait complete. 2023-11-26T07:12:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.018+0000 7f281622c700 1 Processor -- start 2023-11-26T07:12:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.018+0000 7f281622c700 1 -- start start 2023-11-26T07:12:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f281622c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f8004850 0x7f281014d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f281622c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281014c360 0x7f2810158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f281622c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281014c7e0 0x7f28101593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f281622c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28101e3bc0 con 0x7f281014c7e0 2023-11-26T07:12:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f281622c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28101e2f80 con 0x7f281014c360 2023-11-26T07:12:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f281622c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28101e3220 con 0x7f27f8004850 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281014c7e0 0x7f28101593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281014c7e0 0x7f28101593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35524/0 (socket says 172.21.15.83:35524) 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280ffff700 1 -- 172.21.15.83:0/48055969 learned_addr learned my addr 172.21.15.83:0/48055969 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280effd700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281014c360 0x7f2810158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280f7fe700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f8004850 0x7f281014d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280ffff700 1 -- 172.21.15.83:0/48055969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f8004850 msgr2=0x7f281014d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280ffff700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f8004850 0x7f281014d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280ffff700 1 -- 172.21.15.83:0/48055969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281014c360 msgr2=0x7f2810158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280ffff700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281014c360 0x7f2810158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280ffff700 1 -- 172.21.15.83:0/48055969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28101e3500 con 0x7f281014c7e0 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280effd700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281014c360 0x7f2810158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:07.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280f7fe700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f8004850 0x7f281014d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.019+0000 7f280ffff700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281014c7e0 0x7f28101593a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f28100b1fe0 tx=0x7f28080119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.020+0000 7f27f7fff700 1 -- 172.21.15.83:0/48055969 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808012860 con 0x7f281014c7e0 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.020+0000 7f27f7fff700 1 -- 172.21.15.83:0/48055969 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28080129c0 con 0x7f281014c7e0 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.020+0000 7f281622c700 1 -- 172.21.15.83:0/48055969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28101e37e0 con 0x7f281014c7e0 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.020+0000 7f27f7fff700 1 -- 172.21.15.83:0/48055969 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808012d30 con 0x7f281014c7e0 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.020+0000 7f281622c700 1 -- 172.21.15.83:0/48055969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281015d050 con 0x7f281014c7e0 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.021+0000 7f281622c700 1 -- 172.21.15.83:0/48055969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2810068d90 con 0x7f281014c7e0 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.021+0000 7f27f7fff700 1 -- 172.21.15.83:0/48055969 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2808018020 con 0x7f281014c7e0 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.021+0000 7f27f7fff700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f27fc06aed0 0x7f27fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.021+0000 7f27f7fff700 1 -- 172.21.15.83:0/48055969 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f280808c870 con 0x7f281014c7e0 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.021+0000 7f280f7fe700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f27fc06aed0 0x7f27fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.022+0000 7f280f7fe700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f27fc06aed0 0x7f27fc06d390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2810064e10 tx=0x7f2804009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:07.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.026+0000 7f27f7fff700 1 -- 172.21.15.83:0/48055969 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28080579b0 con 0x7f281014c7e0 2023-11-26T07:12:07.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.213+0000 7f281622c700 1 -- 172.21.15.83:0/48055969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28101e3d00 con 0x7f281014c7e0 2023-11-26T07:12:07.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.214+0000 7f27f7fff700 1 -- 172.21.15.83:0/48055969 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28101e3d00 con 0x7f281014c7e0 2023-11-26T07:12:07.216 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:12:07.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 -- 172.21.15.83:0/48055969 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f27fc06aed0 msgr2=0x7f27fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f27fc06aed0 0x7f27fc06d390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2810064e10 tx=0x7f2804009040 comp rx=0 tx=0).stop 2023-11-26T07:12:07.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 -- 172.21.15.83:0/48055969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281014c7e0 msgr2=0x7f28101593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281014c7e0 0x7f28101593a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f28100b1fe0 tx=0x7f28080119a0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 -- 172.21.15.83:0/48055969 shutdown_connections 2023-11-26T07:12:07.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281014c7e0 0x7f28101593a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f27fc06aed0 0x7f27fc06d390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27f8004850 0x7f281014d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 --2- 172.21.15.83:0/48055969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f281014c360 0x7f2810158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 -- 172.21.15.83:0/48055969 >> 172.21.15.83:0/48055969 conn(0x7f28100b2050 msgr2=0x7f28100b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:07.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 -- 172.21.15.83:0/48055969 shutdown_connections 2023-11-26T07:12:07.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.216+0000 7f281622c700 1 -- 172.21.15.83:0/48055969 wait complete. 2023-11-26T07:12:07.231 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:12:07.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.351+0000 7fa0c1a1c700 1 Processor -- start 2023-11-26T07:12:07.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.351+0000 7fa0c1a1c700 1 -- start start 2023-11-26T07:12:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0c1a1c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0bc15e160 0x7fa0bc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0c1a1c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0bc0afc90 0x7fa0bc073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0c1a1c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0bc0b3c60 con 0x7fa0bc0744d0 2023-11-26T07:12:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0c1a1c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0bc0b4500 con 0x7fa0bc15e160 2023-11-26T07:12:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0c1a1c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc074960 con 0x7fa0bc0afc90 2023-11-26T07:12:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0ba7fc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0bc0afc90 0x7fa0bc073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0baffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0bc15e160 0x7fa0bc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0bb7fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa0bc0744d0 0x7fa0bc06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:40192/0 (socket says 172.21.15.83:40192) 2023-11-26T07:12:07.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0bb7fe700 1 -- 172.21.15.83:0/1283390057 learned_addr learned my addr 172.21.15.83:0/1283390057 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0b9ffb700 1 -- 172.21.15.83:0/1283390057 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727511996 0 0) 0x7fa0bc0b3c60 con 0x7fa0bc0744d0 2023-11-26T07:12:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0b9ffb700 1 -- 172.21.15.83:0/1283390057 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a40035b0 con 0x7fa0bc0744d0 2023-11-26T07:12:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0baffd700 1 -- 172.21.15.83:0/1283390057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0bc0afc90 msgr2=0x7fa0bc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0baffd700 1 --2- 172.21.15.83:0/1283390057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0bc0afc90 0x7fa0bc073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0baffd700 1 -- 172.21.15.83:0/1283390057 >> v1:172.21.15.203:6789/0 conn(0x7fa0bc0744d0 legacy=0x7fa0bc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0baffd700 1 -- 172.21.15.83:0/1283390057 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc0b58c0 con 0x7fa0bc15e160 2023-11-26T07:12:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.352+0000 7fa0ba7fc700 1 --2- 172.21.15.83:0/1283390057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0bc0afc90 0x7fa0bc073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:12:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0baffd700 1 --2- 172.21.15.83:0/1283390057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0bc15e160 0x7fa0bc0b5da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009840 tx=0x7fa0b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cd30b1013ac6c8e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0b9ffb700 1 -- 172.21.15.83:0/1283390057 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0011830 con 0x7fa0bc15e160 2023-11-26T07:12:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0b9ffb700 1 -- 172.21.15.83:0/1283390057 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b0011990 con 0x7fa0bc15e160 2023-11-26T07:12:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0b9ffb700 1 -- 172.21.15.83:0/1283390057 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0011d20 con 0x7fa0bc15e160 2023-11-26T07:12:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/1283390057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0bc15e160 msgr2=0x7fa0bc0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0c1a1c700 1 --2- 172.21.15.83:0/1283390057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0bc15e160 0x7fa0bc0b5da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009840 tx=0x7fa0b0009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/1283390057 shutdown_connections 2023-11-26T07:12:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0c1a1c700 1 --2- 172.21.15.83:0/1283390057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0bc15e160 0x7fa0bc0b5da0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0c1a1c700 1 --2- 172.21.15.83:0/1283390057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0bc0afc90 0x7fa0bc073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/1283390057 >> 172.21.15.83:0/1283390057 conn(0x7fa0bc0d0bf0 msgr2=0x7fa0bc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/1283390057 shutdown_connections 2023-11-26T07:12:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.353+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/1283390057 wait complete. 2023-11-26T07:12:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.355+0000 7fa0c1a1c700 1 Processor -- start 2023-11-26T07:12:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.355+0000 7fa0c1a1c700 1 -- start start 2023-11-26T07:12:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.356+0000 7fa0c1a1c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0bc0744d0 0x7fa0bc0c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.356+0000 7fa0c1a1c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0bc0afc90 0x7fa0bc1f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.356+0000 7fa0c1a1c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0bc15e160 0x7fa0bc1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.356+0000 7fa0c1a1c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc15e8e0 con 0x7fa0bc0afc90 2023-11-26T07:12:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.356+0000 7fa0c1a1c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc15ebc0 con 0x7fa0bc0744d0 2023-11-26T07:12:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.356+0000 7fa0c1a1c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc15eea0 con 0x7fa0bc15e160 2023-11-26T07:12:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.356+0000 7fa0ba7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0bc0afc90 0x7fa0bc1f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.356+0000 7fa0ba7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0bc0afc90 0x7fa0bc1f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35540/0 (socket says 172.21.15.83:35540) 2023-11-26T07:12:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.356+0000 7fa0ba7fc700 1 -- 172.21.15.83:0/2549722861 learned_addr learned my addr 172.21.15.83:0/2549722861 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.356+0000 7fa0baffd700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0bc0744d0 0x7fa0bc0c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.356+0000 7fa0bb7fe700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0bc15e160 0x7fa0bc1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0ba7fc700 1 -- 172.21.15.83:0/2549722861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0bc15e160 msgr2=0x7fa0bc1515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0ba7fc700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0bc15e160 0x7fa0bc1515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0ba7fc700 1 -- 172.21.15.83:0/2549722861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0bc0744d0 msgr2=0x7fa0bc0c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0ba7fc700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0bc0744d0 0x7fa0bc0c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0ba7fc700 1 -- 172.21.15.83:0/2549722861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac0094a0 con 0x7fa0bc0afc90 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0baffd700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0bc0744d0 0x7fa0bc0c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0ba7fc700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0bc0afc90 0x7fa0bc1f7170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00ebe0 tx=0x7fa0ac009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0a3fff700 1 -- 172.21.15.83:0/2549722861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00c910 con 0x7fa0bc0afc90 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/2549722861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b00094a0 con 0x7fa0bc0afc90 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0a3fff700 1 -- 172.21.15.83:0/2549722861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac00ca70 con 0x7fa0bc0afc90 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0a3fff700 1 -- 172.21.15.83:0/2549722861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00cde0 con 0x7fa0bc0afc90 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.357+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/2549722861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc15f460 con 0x7fa0bc0afc90 2023-11-26T07:12:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.358+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/2549722861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc068fb0 con 0x7fa0bc0afc90 2023-11-26T07:12:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.358+0000 7fa0a3fff700 1 -- 172.21.15.83:0/2549722861 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0ac013470 con 0x7fa0bc0afc90 2023-11-26T07:12:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.359+0000 7fa0a3fff700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa0a806aed0 0x7fa0a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.359+0000 7fa0a3fff700 1 -- 172.21.15.83:0/2549722861 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0ac089860 con 0x7fa0bc0afc90 2023-11-26T07:12:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.359+0000 7fa0baffd700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa0a806aed0 0x7fa0a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.359+0000 7fa0baffd700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa0a806aed0 0x7fa0a806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009fd0 tx=0x7fa0b001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.363+0000 7fa0a3fff700 1 -- 172.21.15.83:0/2549722861 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0ac054920 con 0x7fa0bc0afc90 2023-11-26T07:12:07.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.557+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/2549722861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0bc06c140 con 0x7fa0bc0afc90 2023-11-26T07:12:07.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.558+0000 7fa0a3fff700 1 -- 172.21.15.83:0/2549722861 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0ac057f40 con 0x7fa0bc0afc90 2023-11-26T07:12:07.560 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:12:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/2549722861 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa0a806aed0 msgr2=0x7fa0a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa0a806aed0 0x7fa0a806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009fd0 tx=0x7fa0b001c040 comp rx=0 tx=0).stop 2023-11-26T07:12:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/2549722861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0bc0afc90 msgr2=0x7fa0bc1f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0bc0afc90 0x7fa0bc1f7170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00ebe0 tx=0x7fa0ac009ec0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/2549722861 shutdown_connections 2023-11-26T07:12:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa0a806aed0 0x7fa0a806d390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0bc15e160 0x7fa0bc1515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0bc0744d0 0x7fa0bc0c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 --2- 172.21.15.83:0/2549722861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0bc0afc90 0x7fa0bc1f7170 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/2549722861 >> 172.21.15.83:0/2549722861 conn(0x7fa0bc0d0bf0 msgr2=0x7fa0bc15ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/2549722861 shutdown_connections 2023-11-26T07:12:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.560+0000 7fa0c1a1c700 1 -- 172.21.15.83:0/2549722861 wait complete. 2023-11-26T07:12:07.574 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:12:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.693+0000 7f06090d8700 1 Processor -- start 2023-11-26T07:12:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.693+0000 7f06090d8700 1 -- start start 2023-11-26T07:12:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f06090d8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06041446a0 0x7f06040b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f06090d8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06040af950 con 0x7f06040adf00 2023-11-26T07:12:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f06090d8700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06040afc30 con 0x7f060406c710 2023-11-26T07:12:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f06090d8700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0604144a80 con 0x7f06041446a0 2023-11-26T07:12:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f0602d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06041446a0 0x7f06040b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f060259c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f06040adf00 0x7f060406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55586/0 (socket says 172.21.15.83:55586) 2023-11-26T07:12:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f060259c700 1 -- 172.21.15.83:0/3281458459 learned_addr learned my addr 172.21.15.83:0/3281458459 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f0602d9d700 1 -- 172.21.15.83:0/3281458459 >> v1:172.21.15.203:6789/0 conn(0x7f060406c710 legacy=0x7f06040b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f0602d9d700 1 -- 172.21.15.83:0/3281458459 >> v1:172.21.15.83:6789/0 conn(0x7f06040adf00 legacy=0x7f060406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f0602d9d700 1 -- 172.21.15.83:0/3281458459 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040aff10 con 0x7f06041446a0 2023-11-26T07:12:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f0601d9b700 1 -- 172.21.15.83:0/3281458459 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947055218 0 0) 0x7f06040af950 con 0x7f06040adf00 2023-11-26T07:12:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f0602d9d700 1 --2- 172.21.15.83:0/3281458459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06041446a0 0x7f06040b5ae0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009840 tx=0x7f05f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62be5d5174c919f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f0601d9b700 1 -- 172.21.15.83:0/3281458459 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4011830 con 0x7f06041446a0 2023-11-26T07:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f0601d9b700 1 -- 172.21.15.83:0/3281458459 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f4011990 con 0x7f06041446a0 2023-11-26T07:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.694+0000 7f0601d9b700 1 -- 172.21.15.83:0/3281458459 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4011d20 con 0x7f06041446a0 2023-11-26T07:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.695+0000 7f06090d8700 1 -- 172.21.15.83:0/3281458459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06041446a0 msgr2=0x7f06040b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.695+0000 7f06090d8700 1 --2- 172.21.15.83:0/3281458459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06041446a0 0x7f06040b5ae0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009840 tx=0x7f05f4009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.695+0000 7f06090d8700 1 -- 172.21.15.83:0/3281458459 shutdown_connections 2023-11-26T07:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.695+0000 7f06090d8700 1 --2- 172.21.15.83:0/3281458459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06041446a0 0x7f06040b5ae0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.695+0000 7f06090d8700 1 -- 172.21.15.83:0/3281458459 >> 172.21.15.83:0/3281458459 conn(0x7f0604071b90 msgr2=0x7f0604071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.695+0000 7f06090d8700 1 -- 172.21.15.83:0/3281458459 shutdown_connections 2023-11-26T07:12:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.695+0000 7f06090d8700 1 -- 172.21.15.83:0/3281458459 wait complete. 2023-11-26T07:12:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.696+0000 7f06090d8700 1 Processor -- start 2023-11-26T07:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.697+0000 7f06090d8700 1 -- start start 2023-11-26T07:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.697+0000 7f06090d8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f060406c710 0x7f060415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.697+0000 7f06090d8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06040adf00 0x7f060409dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.697+0000 7f06090d8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06040cfe30 0x7f060409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.697+0000 7f06090d8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060415e120 con 0x7f06040adf00 2023-11-26T07:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.697+0000 7f06090d8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060415d880 con 0x7f060406c710 2023-11-26T07:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.697+0000 7f06090d8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060415db60 con 0x7f06040cfe30 2023-11-26T07:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.697+0000 7f060259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06040adf00 0x7f060409dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.697+0000 7f060259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06040adf00 0x7f060409dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35552/0 (socket says 172.21.15.83:35552) 2023-11-26T07:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.697+0000 7f060259c700 1 -- 172.21.15.83:0/1966013267 learned_addr learned my addr 172.21.15.83:0/1966013267 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f0602d9d700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f060406c710 0x7f060415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f060359e700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06040cfe30 0x7f060409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f060259c700 1 -- 172.21.15.83:0/1966013267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06040cfe30 msgr2=0x7f060409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f060259c700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06040cfe30 0x7f060409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f060259c700 1 -- 172.21.15.83:0/1966013267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f060406c710 msgr2=0x7f060415a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f060259c700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f060406c710 0x7f060415a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f060259c700 1 -- 172.21.15.83:0/1966013267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f40094a0 con 0x7f06040adf00 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f060259c700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06040adf00 0x7f060409dd80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f05f800c3f0 tx=0x7f05f800c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f05eb7fe700 1 -- 172.21.15.83:0/1966013267 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8004c80 con 0x7f06040adf00 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f05eb7fe700 1 -- 172.21.15.83:0/1966013267 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f8004de0 con 0x7f06040adf00 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f06090d8700 1 -- 172.21.15.83:0/1966013267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060415de40 con 0x7f06040adf00 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f05eb7fe700 1 -- 172.21.15.83:0/1966013267 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8007310 con 0x7f06040adf00 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.698+0000 7f06090d8700 1 -- 172.21.15.83:0/1966013267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060415d510 con 0x7f06040adf00 2023-11-26T07:12:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.699+0000 7f06090d8700 1 -- 172.21.15.83:0/1966013267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06040753f0 con 0x7f06040adf00 2023-11-26T07:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.699+0000 7f05eb7fe700 1 -- 172.21.15.83:0/1966013267 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f05f8007470 con 0x7f06040adf00 2023-11-26T07:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.700+0000 7f05eb7fe700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05f006aed0 0x7f05f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.700+0000 7f05eb7fe700 1 -- 172.21.15.83:0/1966013267 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05f8089120 con 0x7f06040adf00 2023-11-26T07:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.700+0000 7f0602d9d700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05f006aed0 0x7f05f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.700+0000 7f0602d9d700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05f006aed0 0x7f05f006d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f060406d3f0 tx=0x7f05f401c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:07.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.704+0000 7f05eb7fe700 1 -- 172.21.15.83:0/1966013267 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05f80541d0 con 0x7f06040adf00 2023-11-26T07:12:07.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.909+0000 7f06090d8700 1 -- 172.21.15.83:0/1966013267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0604067f60 con 0x7f06040adf00 2023-11-26T07:12:07.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.910+0000 7f05eb7fe700 1 -- 172.21.15.83:0/1966013267 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f05f80577f0 con 0x7f06040adf00 2023-11-26T07:12:07.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 -- 172.21.15.83:0/1966013267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05f006aed0 msgr2=0x7f05f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05f006aed0 0x7f05f006d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f060406d3f0 tx=0x7f05f401c040 comp rx=0 tx=0).stop 2023-11-26T07:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 -- 172.21.15.83:0/1966013267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06040adf00 msgr2=0x7f060409dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06040adf00 0x7f060409dd80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f05f800c3f0 tx=0x7f05f800c700 comp rx=0 tx=0).stop 2023-11-26T07:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 -- 172.21.15.83:0/1966013267 shutdown_connections 2023-11-26T07:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05f006aed0 0x7f05f006d390 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06040cfe30 0x7f060409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f060406c710 0x7f060415a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 --2- 172.21.15.83:0/1966013267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06040adf00 0x7f060409dd80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 -- 172.21.15.83:0/1966013267 >> 172.21.15.83:0/1966013267 conn(0x7f0604071b90 msgr2=0x7f060406e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 -- 172.21.15.83:0/1966013267 shutdown_connections 2023-11-26T07:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:07.912+0000 7f06090d8700 1 -- 172.21.15.83:0/1966013267 wait complete. 2023-11-26T07:12:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:12:07.926 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-26T07:12:07.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:07.927 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-26T07:12:07.968 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:12:07.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:07.968 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-26T07:12:08.028 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:12:08.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:08.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.146+0000 7f853a6ed700 1 Processor -- start 2023-11-26T07:12:08.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.146+0000 7f853a6ed700 1 -- start start 2023-11-26T07:12:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.146+0000 7f853a6ed700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8534152cc0 0x7f853406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.146+0000 7f853a6ed700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85340b49a0 0x7f853406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.146+0000 7f853a6ed700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853415c4b0 con 0x7f85340cff20 2023-11-26T07:12:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.146+0000 7f853a6ed700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853415c790 con 0x7f85340b49a0 2023-11-26T07:12:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.146+0000 7f853a6ed700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853415ca70 con 0x7f8534152cc0 2023-11-26T07:12:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f85337fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85340b49a0 0x7f853406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f85337fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85340b49a0 0x7f853406c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35558/0 (socket says 172.21.15.83:35558) 2023-11-26T07:12:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f85337fe700 1 -- 172.21.15.83:0/402850912 learned_addr learned my addr 172.21.15.83:0/402850912 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f8533fff700 1 --2- 172.21.15.83:0/402850912 >> v2:172.21.15.203:3300/0 conn(0x7f8534152cc0 0x7f853406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f85337fe700 1 -- 172.21.15.83:0/402850912 >> v2:172.21.15.203:3300/0 conn(0x7f8534152cc0 msgr2=0x7f853406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f85337fe700 1 --2- 172.21.15.83:0/402850912 >> v2:172.21.15.203:3300/0 conn(0x7f8534152cc0 0x7f853406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f85337fe700 1 -- 172.21.15.83:0/402850912 >> v1:172.21.15.203:6789/0 conn(0x7f85340cff20 legacy=0x7f85340d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f85337fe700 1 -- 172.21.15.83:0/402850912 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853415cd50 con 0x7f85340b49a0 2023-11-26T07:12:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f85337fe700 1 --2- 172.21.15.83:0/402850912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85340b49a0 0x7f853406c540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8534075030 tx=0x7f8528009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cdc21c39cf06ae server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f8532ffd700 1 -- 172.21.15.83:0/402850912 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852800b870 con 0x7f85340b49a0 2023-11-26T07:12:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f8532ffd700 1 -- 172.21.15.83:0/402850912 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852800b9d0 con 0x7f85340b49a0 2023-11-26T07:12:08.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.147+0000 7f8532ffd700 1 -- 172.21.15.83:0/402850912 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852800bd60 con 0x7f85340b49a0 2023-11-26T07:12:08.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.148+0000 7f853a6ed700 1 -- 172.21.15.83:0/402850912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85340b49a0 msgr2=0x7f853406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.148+0000 7f853a6ed700 1 --2- 172.21.15.83:0/402850912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85340b49a0 0x7f853406c540 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8534075030 tx=0x7f8528009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.148+0000 7f853a6ed700 1 -- 172.21.15.83:0/402850912 shutdown_connections 2023-11-26T07:12:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.148+0000 7f853a6ed700 1 --2- 172.21.15.83:0/402850912 >> v2:172.21.15.203:3300/0 conn(0x7f8534152cc0 0x7f853406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.148+0000 7f853a6ed700 1 --2- 172.21.15.83:0/402850912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85340b49a0 0x7f853406c540 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.148+0000 7f853a6ed700 1 -- 172.21.15.83:0/402850912 >> 172.21.15.83:0/402850912 conn(0x7f8534071b90 msgr2=0x7f8534071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.148+0000 7f853a6ed700 1 -- 172.21.15.83:0/402850912 shutdown_connections 2023-11-26T07:12:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.148+0000 7f853a6ed700 1 -- 172.21.15.83:0/402850912 wait complete. 2023-11-26T07:12:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.149+0000 7f853a6ed700 1 Processor -- start 2023-11-26T07:12:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.149+0000 7f853a6ed700 1 -- start start 2023-11-26T07:12:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f853a6ed700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340b49a0 0x7f8534147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f853a6ed700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85340cff20 0x7f85340cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f853a6ed700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534152cc0 0x7f85340cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f853a6ed700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85340cfdb0 con 0x7f8534152cc0 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f853a6ed700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85341590e0 con 0x7f85340cff20 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f853a6ed700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85341593c0 con 0x7f85340b49a0 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f8538c8a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534152cc0 0x7f85340cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f8538c8a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534152cc0 0x7f85340cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35568/0 (socket says 172.21.15.83:35568) 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f8538c8a700 1 -- 172.21.15.83:0/2182856073 learned_addr learned my addr 172.21.15.83:0/2182856073 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f85337fe700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85340cff20 0x7f85340cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f8533fff700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340b49a0 0x7f8534147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f8538c8a700 1 -- 172.21.15.83:0/2182856073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340b49a0 msgr2=0x7f8534147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f8538c8a700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340b49a0 0x7f8534147560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f8538c8a700 1 -- 172.21.15.83:0/2182856073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85340cff20 msgr2=0x7f85340cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f8538c8a700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85340cff20 0x7f85340cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f8538c8a700 1 -- 172.21.15.83:0/2182856073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8528009550 con 0x7f8534152cc0 2023-11-26T07:12:08.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.150+0000 7f85337fe700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85340cff20 0x7f85340cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:08.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.151+0000 7f8538c8a700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534152cc0 0x7f85340cf780 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f852c00cc50 tx=0x7f852c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:08.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.151+0000 7f8530ff9700 1 -- 172.21.15.83:0/2182856073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c00ad80 con 0x7f8534152cc0 2023-11-26T07:12:08.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.151+0000 7f8530ff9700 1 -- 172.21.15.83:0/2182856073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c00ce50 con 0x7f8534152cc0 2023-11-26T07:12:08.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.151+0000 7f853a6ed700 1 -- 172.21.15.83:0/2182856073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85341596a0 con 0x7f8534152cc0 2023-11-26T07:12:08.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.151+0000 7f8530ff9700 1 -- 172.21.15.83:0/2182856073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c003c40 con 0x7f8534152cc0 2023-11-26T07:12:08.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.151+0000 7f853a6ed700 1 -- 172.21.15.83:0/2182856073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853406f900 con 0x7f8534152cc0 2023-11-26T07:12:08.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.152+0000 7f853a6ed700 1 -- 172.21.15.83:0/2182856073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85340753f0 con 0x7f8534152cc0 2023-11-26T07:12:08.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.152+0000 7f8530ff9700 1 -- 172.21.15.83:0/2182856073 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f852c0050d0 con 0x7f8534152cc0 2023-11-26T07:12:08.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.153+0000 7f8530ff9700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f852006aed0 0x7f852006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.153+0000 7f8530ff9700 1 -- 172.21.15.83:0/2182856073 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f852c089660 con 0x7f8534152cc0 2023-11-26T07:12:08.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.153+0000 7f8533fff700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f852006aed0 0x7f852006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.153+0000 7f8533fff700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f852006aed0 0x7f852006d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85340b2330 tx=0x7f8524008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:08.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.157+0000 7f8530ff9700 1 -- 172.21.15.83:0/2182856073 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f852c0547a0 con 0x7f8534152cc0 2023-11-26T07:12:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.390+0000 7f853a6ed700 1 -- 172.21.15.83:0/2182856073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8534067f60 con 0x7f8534152cc0 2023-11-26T07:12:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.390+0000 7f8530ff9700 1 -- 172.21.15.83:0/2182856073 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f852c057dc0 con 0x7f8534152cc0 2023-11-26T07:12:08.393 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:08.393 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2208741688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2208741688},{"type":"v1","addr":"172.21.15.83:6837","nonce":2208741688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2802126359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2802126359},{"type":"v1","addr":"172.21.15.83:6836","nonce":2802126359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1258871616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1258871616},{"type":"v1","addr":"172.21.15.203:6834","nonce":1258871616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:11:27.911475+0000","modified":"2023-11-26T07:11:30.937105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/2957324266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":2957324266},{"type":"v1","addr":"172.21.15.203:6835","nonce":2957324266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:12:08.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 -- 172.21.15.83:0/2182856073 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f852006aed0 msgr2=0x7f852006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f852006aed0 0x7f852006d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85340b2330 tx=0x7f8524008040 comp rx=0 tx=0).stop 2023-11-26T07:12:08.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 -- 172.21.15.83:0/2182856073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534152cc0 msgr2=0x7f85340cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534152cc0 0x7f85340cf780 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f852c00cc50 tx=0x7f852c00a340 comp rx=0 tx=0).stop 2023-11-26T07:12:08.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 -- 172.21.15.83:0/2182856073 shutdown_connections 2023-11-26T07:12:08.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534152cc0 0x7f85340cf780 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f852006aed0 0x7f852006d390 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85340b49a0 0x7f8534147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 --2- 172.21.15.83:0/2182856073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85340cff20 0x7f85340cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 -- 172.21.15.83:0/2182856073 >> 172.21.15.83:0/2182856073 conn(0x7f8534071b90 msgr2=0x7f853415a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 -- 172.21.15.83:0/2182856073 shutdown_connections 2023-11-26T07:12:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.393+0000 7f853a6ed700 1 -- 172.21.15.83:0/2182856073 wait complete. 2023-11-26T07:12:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2023-11-26T07:12:08.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:08.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.524+0000 7fba58cd2700 1 Processor -- start 2023-11-26T07:12:08.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.524+0000 7fba58cd2700 1 -- start start 2023-11-26T07:12:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.525+0000 7fba58cd2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba5406c710 0x7fba540b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.525+0000 7fba58cd2700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fba540af4d0 0x7fba540afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.525+0000 7fba58cd2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba540ae430 con 0x7fba5415e080 2023-11-26T07:12:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.525+0000 7fba58cd2700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba540af0b0 con 0x7fba540af4d0 2023-11-26T07:12:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.525+0000 7fba58cd2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba540b05a0 con 0x7fba5406c710 2023-11-26T07:12:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba52d9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fba540af4d0 0x7fba540afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba51d9b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba5406c710 0x7fba540b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba5259c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fba5415e080 0x7fba5406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:40230/0 (socket says 172.21.15.83:40230) 2023-11-26T07:12:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba5259c700 1 -- 172.21.15.83:0/3391775525 learned_addr learned my addr 172.21.15.83:0/3391775525 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba52d9d700 1 -- 172.21.15.83:0/3391775525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba5406c710 msgr2=0x7fba540b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba52d9d700 1 --2- 172.21.15.83:0/3391775525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba5406c710 0x7fba540b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba52d9d700 1 -- 172.21.15.83:0/3391775525 >> v1:172.21.15.203:6789/0 conn(0x7fba5415e080 legacy=0x7fba5406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba52d9d700 1 -- 172.21.15.83:0/3391775525 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540ae710 con 0x7fba540af4d0 2023-11-26T07:12:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba52d9d700 1 --2- 172.21.15.83:0/3391775525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba540af4d0 0x7fba540afb90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009dc0 tx=0x7fba4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68d7e669c9524ac8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba5159a700 1 -- 172.21.15.83:0/3391775525 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c00e070 con 0x7fba540af4d0 2023-11-26T07:12:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba5159a700 1 -- 172.21.15.83:0/3391775525 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c00bae0 con 0x7fba540af4d0 2023-11-26T07:12:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.526+0000 7fba5159a700 1 -- 172.21.15.83:0/3391775525 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c00be90 con 0x7fba540af4d0 2023-11-26T07:12:08.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.527+0000 7fba58cd2700 1 -- 172.21.15.83:0/3391775525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba540af4d0 msgr2=0x7fba540afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.527+0000 7fba58cd2700 1 --2- 172.21.15.83:0/3391775525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba540af4d0 0x7fba540afb90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009dc0 tx=0x7fba4c00b330 comp rx=0 tx=0).stop 2023-11-26T07:12:08.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.527+0000 7fba58cd2700 1 -- 172.21.15.83:0/3391775525 shutdown_connections 2023-11-26T07:12:08.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.527+0000 7fba58cd2700 1 --2- 172.21.15.83:0/3391775525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba540af4d0 0x7fba540afb90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.527+0000 7fba58cd2700 1 --2- 172.21.15.83:0/3391775525 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba5406c710 0x7fba540b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.527+0000 7fba58cd2700 1 -- 172.21.15.83:0/3391775525 >> 172.21.15.83:0/3391775525 conn(0x7fba54071b90 msgr2=0x7fba54071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:08.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.527+0000 7fba58cd2700 1 -- 172.21.15.83:0/3391775525 shutdown_connections 2023-11-26T07:12:08.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.527+0000 7fba58cd2700 1 -- 172.21.15.83:0/3391775525 wait complete. 2023-11-26T07:12:08.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.528+0000 7fba58cd2700 1 Processor -- start 2023-11-26T07:12:08.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.528+0000 7fba58cd2700 1 -- start start 2023-11-26T07:12:08.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.529+0000 7fba58cd2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5406c710 0x7fba54160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.529+0000 7fba58cd2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba540af4d0 0x7fba54142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.529+0000 7fba58cd2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba5415e080 0x7fba541451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.529+0000 7fba58cd2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba54160ab0 con 0x7fba5406c710 2023-11-26T07:12:08.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.529+0000 7fba58cd2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba54161410 con 0x7fba540af4d0 2023-11-26T07:12:08.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.529+0000 7fba58cd2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5415fee0 con 0x7fba5415e080 2023-11-26T07:12:08.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.529+0000 7fba5259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5406c710 0x7fba54160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.529+0000 7fba5259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5406c710 0x7fba54160ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35578/0 (socket says 172.21.15.83:35578) 2023-11-26T07:12:08.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.529+0000 7fba5259c700 1 -- 172.21.15.83:0/3966876990 learned_addr learned my addr 172.21.15.83:0/3966876990 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:08.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.529+0000 7fba51d9b700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba540af4d0 0x7fba54142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.529+0000 7fba52d9d700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba5415e080 0x7fba541451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.530+0000 7fba5259c700 1 -- 172.21.15.83:0/3966876990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba5415e080 msgr2=0x7fba541451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.530+0000 7fba5259c700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba5415e080 0x7fba541451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.530+0000 7fba5259c700 1 -- 172.21.15.83:0/3966876990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba540af4d0 msgr2=0x7fba54142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.530+0000 7fba5259c700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba540af4d0 0x7fba54142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.530+0000 7fba5259c700 1 -- 172.21.15.83:0/3966876990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c009a70 con 0x7fba5406c710 2023-11-26T07:12:08.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.530+0000 7fba5259c700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5406c710 0x7fba54160ed0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fba4800c490 tx=0x7fba4800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:08.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.530+0000 7fba42ffd700 1 -- 172.21.15.83:0/3966876990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48005d70 con 0x7fba5406c710 2023-11-26T07:12:08.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.530+0000 7fba42ffd700 1 -- 172.21.15.83:0/3966876990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4800ce20 con 0x7fba5406c710 2023-11-26T07:12:08.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.530+0000 7fba58cd2700 1 -- 172.21.15.83:0/3966876990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba54142590 con 0x7fba5406c710 2023-11-26T07:12:08.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.530+0000 7fba42ffd700 1 -- 172.21.15.83:0/3966876990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba480072f0 con 0x7fba5406c710 2023-11-26T07:12:08.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.530+0000 7fba58cd2700 1 -- 172.21.15.83:0/3966876990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba541603f0 con 0x7fba5406c710 2023-11-26T07:12:08.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.531+0000 7fba58cd2700 1 -- 172.21.15.83:0/3966876990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba540753f0 con 0x7fba5406c710 2023-11-26T07:12:08.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.532+0000 7fba42ffd700 1 -- 172.21.15.83:0/3966876990 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba48007450 con 0x7fba5406c710 2023-11-26T07:12:08.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.533+0000 7fba42ffd700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba3406aed0 0x7fba3406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.533+0000 7fba42ffd700 1 -- 172.21.15.83:0/3966876990 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba48088d30 con 0x7fba5406c710 2023-11-26T07:12:08.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.533+0000 7fba51d9b700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba3406aed0 0x7fba3406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.533+0000 7fba51d9b700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba3406aed0 0x7fba3406d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fba540cff00 tx=0x7fba44009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:08.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.536+0000 7fba42ffd700 1 -- 172.21.15.83:0/3966876990 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba48053dc0 con 0x7fba5406c710 2023-11-26T07:12:08.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.736+0000 7fba58cd2700 1 -- 172.21.15.83:0/3966876990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba54067f60 con 0x7fba5406c710 2023-11-26T07:12:08.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.738+0000 7fba42ffd700 1 -- 172.21.15.83:0/3966876990 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16742 (secure 0 0 0) 0x7fba48017020 con 0x7fba5406c710 2023-11-26T07:12:08.740 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:08.741 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:08.707968+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:08.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 -- 172.21.15.83:0/3966876990 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba3406aed0 msgr2=0x7fba3406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba3406aed0 0x7fba3406d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fba540cff00 tx=0x7fba44009210 comp rx=0 tx=0).stop 2023-11-26T07:12:08.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 -- 172.21.15.83:0/3966876990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5406c710 msgr2=0x7fba54160ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5406c710 0x7fba54160ed0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fba4800c490 tx=0x7fba4800c7a0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 -- 172.21.15.83:0/3966876990 shutdown_connections 2023-11-26T07:12:08.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba3406aed0 0x7fba3406d390 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba5415e080 0x7fba541451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5406c710 0x7fba54160ed0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 --2- 172.21.15.83:0/3966876990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba540af4d0 0x7fba54142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 -- 172.21.15.83:0/3966876990 >> 172.21.15.83:0/3966876990 conn(0x7fba54071b90 msgr2=0x7fba5406ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:08.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 -- 172.21.15.83:0/3966876990 shutdown_connections 2023-11-26T07:12:08.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.740+0000 7fba58cd2700 1 -- 172.21.15.83:0/3966876990 wait complete. 2023-11-26T07:12:08.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:11:25.635076+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:11:26.877047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:12:08.756 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:12:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.876+0000 7f0aa0f67700 1 Processor -- start 2023-11-26T07:12:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.876+0000 7f0aa0f67700 1 -- start start 2023-11-26T07:12:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0aa0f67700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a9c15bf50 0x7f0a9c0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0aa0f67700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0a9c0ce830 0x7f0a9c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0aa0f67700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c0d0140 con 0x7f0a9c0cf1f0 2023-11-26T07:12:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0aa0f67700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c15c330 con 0x7f0a9c15bf50 2023-11-26T07:12:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0aa0f67700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0d03e0 con 0x7f0a9c0ce830 2023-11-26T07:12:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0a9a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0a9c15bf50 0x7f0a9c0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0a9ad9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0a9c0cf1f0 0x7f0a9c0c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55626/0 (socket says 172.21.15.83:55626) 2023-11-26T07:12:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0a9ad9d700 1 -- 172.21.15.83:0/2520438061 learned_addr learned my addr 172.21.15.83:0/2520438061 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0a99d9b700 1 --2- 172.21.15.83:0/2520438061 >> v2:172.21.15.203:3300/0 conn(0x7f0a9c0ce830 0x7f0a9c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0a9a59c700 1 -- 172.21.15.83:0/2520438061 >> v2:172.21.15.203:3300/0 conn(0x7f0a9c0ce830 msgr2=0x7f0a9c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0a9a59c700 1 --2- 172.21.15.83:0/2520438061 >> v2:172.21.15.203:3300/0 conn(0x7f0a9c0ce830 0x7f0a9c0cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0a9a59c700 1 -- 172.21.15.83:0/2520438061 >> v1:172.21.15.83:6789/0 conn(0x7f0a9c0cf1f0 legacy=0x7f0a9c0c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0a9a59c700 1 -- 172.21.15.83:0/2520438061 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0d06c0 con 0x7f0a9c15bf50 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.877+0000 7f0a9959a700 1 -- 172.21.15.83:0/2520438061 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793387359 0 0) 0x7f0a9c0d0140 con 0x7f0a9c0cf1f0 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0a9a59c700 1 --2- 172.21.15.83:0/2520438061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a9c15bf50 0x7f0a9c0ce2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c072420 tx=0x7f0a90009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f302a1549754ab54 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0a9959a700 1 -- 172.21.15.83:0/2520438061 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90011830 con 0x7f0a9c15bf50 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0a9959a700 1 -- 172.21.15.83:0/2520438061 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a90011990 con 0x7f0a9c15bf50 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0a9959a700 1 -- 172.21.15.83:0/2520438061 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90011d20 con 0x7f0a9c15bf50 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0aa0f67700 1 -- 172.21.15.83:0/2520438061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a9c15bf50 msgr2=0x7f0a9c0ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0aa0f67700 1 --2- 172.21.15.83:0/2520438061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a9c15bf50 0x7f0a9c0ce2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c072420 tx=0x7f0a90009b20 comp rx=0 tx=0).stop 2023-11-26T07:12:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0aa0f67700 1 -- 172.21.15.83:0/2520438061 shutdown_connections 2023-11-26T07:12:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0aa0f67700 1 --2- 172.21.15.83:0/2520438061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a9c15bf50 0x7f0a9c0ce2f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0aa0f67700 1 --2- 172.21.15.83:0/2520438061 >> v2:172.21.15.203:3300/0 conn(0x7f0a9c0ce830 0x7f0a9c0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0aa0f67700 1 -- 172.21.15.83:0/2520438061 >> 172.21.15.83:0/2520438061 conn(0x7f0a9c0b3780 msgr2=0x7f0a9c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0aa0f67700 1 -- 172.21.15.83:0/2520438061 shutdown_connections 2023-11-26T07:12:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.878+0000 7f0aa0f67700 1 -- 172.21.15.83:0/2520438061 wait complete. 2023-11-26T07:12:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.880+0000 7f0aa0f67700 1 Processor -- start 2023-11-26T07:12:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.880+0000 7f0aa0f67700 1 -- start start 2023-11-26T07:12:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.880+0000 7f0aa0f67700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a9c0ce830 0x7f0a9c146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.880+0000 7f0aa0f67700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a9c0cf1f0 0x7f0a9c0b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.880+0000 7f0aa0f67700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a9c15bf50 0x7f0a9c0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.880+0000 7f0aa0f67700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c147b40 con 0x7f0a9c15bf50 2023-11-26T07:12:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.880+0000 7f0aa0f67700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c15a870 con 0x7f0a9c0cf1f0 2023-11-26T07:12:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.880+0000 7f0aa0f67700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0b6eb0 con 0x7f0a9c0ce830 2023-11-26T07:12:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a9ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a9c15bf50 0x7f0a9c0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a9ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a9c15bf50 0x7f0a9c0b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35594/0 (socket says 172.21.15.83:35594) 2023-11-26T07:12:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a9ad9d700 1 -- 172.21.15.83:0/4005685267 learned_addr learned my addr 172.21.15.83:0/4005685267 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a9a59c700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a9c0ce830 0x7f0a9c146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a99d9b700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a9c0cf1f0 0x7f0a9c0b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a9ad9d700 1 -- 172.21.15.83:0/4005685267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a9c0ce830 msgr2=0x7f0a9c146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a9ad9d700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a9c0ce830 0x7f0a9c146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a9ad9d700 1 -- 172.21.15.83:0/4005685267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a9c0cf1f0 msgr2=0x7f0a9c0b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a9ad9d700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a9c0cf1f0 0x7f0a9c0b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a9ad9d700 1 -- 172.21.15.83:0/4005685267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a900094a0 con 0x7f0a9c15bf50 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a9ad9d700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a9c15bf50 0x7f0a9c0b1b90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400cbe0 tx=0x7f0a9400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a8affd700 1 -- 172.21.15.83:0/4005685267 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400aaa0 con 0x7f0a9c15bf50 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a8affd700 1 -- 172.21.15.83:0/4005685267 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9400ac20 con 0x7f0a9c15bf50 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0aa0f67700 1 -- 172.21.15.83:0/4005685267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c0b7190 con 0x7f0a9c15bf50 2023-11-26T07:12:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0a8affd700 1 -- 172.21.15.83:0/4005685267 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a940070d0 con 0x7f0a9c15bf50 2023-11-26T07:12:08.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.881+0000 7f0aa0f67700 1 -- 172.21.15.83:0/4005685267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c1477d0 con 0x7f0a9c15bf50 2023-11-26T07:12:08.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.882+0000 7f0aa0f67700 1 -- 172.21.15.83:0/4005685267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c147cf0 con 0x7f0a9c15bf50 2023-11-26T07:12:08.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.883+0000 7f0a8affd700 1 -- 172.21.15.83:0/4005685267 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a94007230 con 0x7f0a9c15bf50 2023-11-26T07:12:08.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.883+0000 7f0a8affd700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0a7c06aed0 0x7f0a7c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:08.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.883+0000 7f0a8affd700 1 -- 172.21.15.83:0/4005685267 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0a94089380 con 0x7f0a9c15bf50 2023-11-26T07:12:08.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.883+0000 7f0a9a59c700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0a7c06aed0 0x7f0a7c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:08.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.883+0000 7f0a9a59c700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0a7c06aed0 0x7f0a7c06d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0a900090b0 tx=0x7f0a9001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:08.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:08.887+0000 7f0a8affd700 1 -- 172.21.15.83:0/4005685267 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a9c147cf0 con 0x7f0a9c15bf50 2023-11-26T07:12:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.076+0000 7f0aa0f67700 1 -- 172.21.15.83:0/4005685267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a9c068fb0 con 0x7f0a9c15bf50 2023-11-26T07:12:09.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.077+0000 7f0a8affd700 1 -- 172.21.15.83:0/4005685267 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a940543b0 con 0x7f0a9c15bf50 2023-11-26T07:12:09.080 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:12:09.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 -- 172.21.15.83:0/4005685267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0a7c06aed0 msgr2=0x7f0a7c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0a7c06aed0 0x7f0a7c06d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0a900090b0 tx=0x7f0a9001c040 comp rx=0 tx=0).stop 2023-11-26T07:12:09.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 -- 172.21.15.83:0/4005685267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a9c15bf50 msgr2=0x7f0a9c0b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a9c15bf50 0x7f0a9c0b1b90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400cbe0 tx=0x7f0a9400cef0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 -- 172.21.15.83:0/4005685267 shutdown_connections 2023-11-26T07:12:09.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a9c15bf50 0x7f0a9c0b1b90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0a7c06aed0 0x7f0a7c06d390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a9c0ce830 0x7f0a9c146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 --2- 172.21.15.83:0/4005685267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a9c0cf1f0 0x7f0a9c0b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 -- 172.21.15.83:0/4005685267 >> 172.21.15.83:0/4005685267 conn(0x7f0a9c0b3780 msgr2=0x7f0a9c070370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 -- 172.21.15.83:0/4005685267 shutdown_connections 2023-11-26T07:12:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.079+0000 7f0aa0f67700 1 -- 172.21.15.83:0/4005685267 wait complete. 2023-11-26T07:12:09.094 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:12:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.214+0000 7f2e324fb700 1 Processor -- start 2023-11-26T07:12:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.214+0000 7f2e324fb700 1 -- start start 2023-11-26T07:12:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.215+0000 7f2e324fb700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e2c15e160 0x7f2e2c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.215+0000 7f2e324fb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e2c0afc90 0x7f2e2c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.215+0000 7f2e324fb700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2c0b3c60 con 0x7f2e2c0744d0 2023-11-26T07:12:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.215+0000 7f2e324fb700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c0b4500 con 0x7f2e2c0afc90 2023-11-26T07:12:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.215+0000 7f2e324fb700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c074960 con 0x7f2e2c15e160 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.215+0000 7f2e2b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e2c0afc90 0x7f2e2c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.215+0000 7f2e30a98700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2e2c0744d0 0x7f2e2c06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55642/0 (socket says 172.21.15.83:55642) 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.215+0000 7f2e30a98700 1 -- 172.21.15.83:0/3441238549 learned_addr learned my addr 172.21.15.83:0/3441238549 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.216+0000 7f2e2bfff700 1 --2- 172.21.15.83:0/3441238549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e2c15e160 0x7f2e2c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.216+0000 7f2e2b7fe700 1 -- 172.21.15.83:0/3441238549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e2c15e160 msgr2=0x7f2e2c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.216+0000 7f2e2b7fe700 1 --2- 172.21.15.83:0/3441238549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e2c15e160 0x7f2e2c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.216+0000 7f2e2b7fe700 1 -- 172.21.15.83:0/3441238549 >> v1:172.21.15.83:6789/0 conn(0x7f2e2c0744d0 legacy=0x7f2e2c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.216+0000 7f2e2b7fe700 1 -- 172.21.15.83:0/3441238549 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0b58c0 con 0x7f2e2c0afc90 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.216+0000 7f2e2affd700 1 -- 172.21.15.83:0/3441238549 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788681648 0 0) 0x7f2e2c0b3c60 con 0x7f2e2c0744d0 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.216+0000 7f2e2b7fe700 1 --2- 172.21.15.83:0/3441238549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c0afc90 0x7f2e2c073f90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009840 tx=0x7f2e1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1736d16843015b2c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.216+0000 7f2e2affd700 1 -- 172.21.15.83:0/3441238549 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c011830 con 0x7f2e2c0afc90 2023-11-26T07:12:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.216+0000 7f2e2affd700 1 -- 172.21.15.83:0/3441238549 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c011990 con 0x7f2e2c0afc90 2023-11-26T07:12:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.216+0000 7f2e2affd700 1 -- 172.21.15.83:0/3441238549 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c011d20 con 0x7f2e2c0afc90 2023-11-26T07:12:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.217+0000 7f2e324fb700 1 -- 172.21.15.83:0/3441238549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c0afc90 msgr2=0x7f2e2c073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.217+0000 7f2e324fb700 1 --2- 172.21.15.83:0/3441238549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c0afc90 0x7f2e2c073f90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009840 tx=0x7f2e1c009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.217+0000 7f2e324fb700 1 -- 172.21.15.83:0/3441238549 shutdown_connections 2023-11-26T07:12:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.217+0000 7f2e324fb700 1 --2- 172.21.15.83:0/3441238549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e2c15e160 0x7f2e2c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.217+0000 7f2e324fb700 1 --2- 172.21.15.83:0/3441238549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c0afc90 0x7f2e2c073f90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.217+0000 7f2e324fb700 1 -- 172.21.15.83:0/3441238549 >> 172.21.15.83:0/3441238549 conn(0x7f2e2c0d0bf0 msgr2=0x7f2e2c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.217+0000 7f2e324fb700 1 -- 172.21.15.83:0/3441238549 shutdown_connections 2023-11-26T07:12:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.217+0000 7f2e324fb700 1 -- 172.21.15.83:0/3441238549 wait complete. 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.218+0000 7f2e324fb700 1 Processor -- start 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.218+0000 7f2e324fb700 1 -- start start 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.219+0000 7f2e324fb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c0744d0 0x7f2e2c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.219+0000 7f2e324fb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e2c0afc90 0x7f2e2c15ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.219+0000 7f2e324fb700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e2c15e160 0x7f2e2c0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.219+0000 7f2e324fb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c159070 con 0x7f2e2c0744d0 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.219+0000 7f2e324fb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c1f38b0 con 0x7f2e2c0afc90 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.219+0000 7f2e324fb700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c15f860 con 0x7f2e2c15e160 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.219+0000 7f2e2bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c0744d0 0x7f2e2c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.219+0000 7f2e2bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c0744d0 0x7f2e2c158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35604/0 (socket says 172.21.15.83:35604) 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.219+0000 7f2e2bfff700 1 -- 172.21.15.83:0/3789994734 learned_addr learned my addr 172.21.15.83:0/3789994734 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:09.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e30a98700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e2c15e160 0x7f2e2c0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e2b7fe700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e2c0afc90 0x7f2e2c15ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e2bfff700 1 -- 172.21.15.83:0/3789994734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e2c15e160 msgr2=0x7f2e2c0c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e2bfff700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e2c15e160 0x7f2e2c0c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e2bfff700 1 -- 172.21.15.83:0/3789994734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e2c0afc90 msgr2=0x7f2e2c15ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e2bfff700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e2c0afc90 0x7f2e2c15ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e2bfff700 1 -- 172.21.15.83:0/3789994734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c0094a0 con 0x7f2e2c0744d0 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e2b7fe700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e2c0afc90 0x7f2e2c15ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e2bfff700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c0744d0 0x7f2e2c158b30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400d800 tx=0x7f2e1400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e28ff9700 1 -- 172.21.15.83:0/3789994734 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e140125f0 con 0x7f2e2c0744d0 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e28ff9700 1 -- 172.21.15.83:0/3789994734 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e14012750 con 0x7f2e2c0744d0 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e324fb700 1 -- 172.21.15.83:0/3789994734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2c15fb40 con 0x7f2e2c0744d0 2023-11-26T07:12:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e28ff9700 1 -- 172.21.15.83:0/3789994734 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14012b00 con 0x7f2e2c0744d0 2023-11-26T07:12:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.220+0000 7f2e324fb700 1 -- 172.21.15.83:0/3789994734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c160100 con 0x7f2e2c0744d0 2023-11-26T07:12:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.221+0000 7f2e324fb700 1 -- 172.21.15.83:0/3789994734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2c1f3c00 con 0x7f2e2c0744d0 2023-11-26T07:12:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.221+0000 7f2e28ff9700 1 -- 172.21.15.83:0/3789994734 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e14012c60 con 0x7f2e2c0744d0 2023-11-26T07:12:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.222+0000 7f2e28ff9700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e0c06aed0 0x7f2e0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.222+0000 7f2e28ff9700 1 -- 172.21.15.83:0/3789994734 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2e14089ab0 con 0x7f2e2c0744d0 2023-11-26T07:12:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.222+0000 7f2e2b7fe700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e0c06aed0 0x7f2e0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.222+0000 7f2e2b7fe700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e0c06aed0 0x7f2e0c06d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c06bc80 tx=0x7f2e1c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:09.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.227+0000 7f2e28ff9700 1 -- 172.21.15.83:0/3789994734 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e14054a60 con 0x7f2e2c0744d0 2023-11-26T07:12:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.417+0000 7f2e324fb700 1 -- 172.21.15.83:0/3789994734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e2c068fb0 con 0x7f2e2c0744d0 2023-11-26T07:12:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.418+0000 7f2e28ff9700 1 -- 172.21.15.83:0/3789994734 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2e14058080 con 0x7f2e2c0744d0 2023-11-26T07:12:09.420 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:12:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 -- 172.21.15.83:0/3789994734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e0c06aed0 msgr2=0x7f2e0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e0c06aed0 0x7f2e0c06d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c06bc80 tx=0x7f2e1c008030 comp rx=0 tx=0).stop 2023-11-26T07:12:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 -- 172.21.15.83:0/3789994734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c0744d0 msgr2=0x7f2e2c158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c0744d0 0x7f2e2c158b30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400d800 tx=0x7f2e1400dbc0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 -- 172.21.15.83:0/3789994734 shutdown_connections 2023-11-26T07:12:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e0c06aed0 0x7f2e0c06d390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e2c15e160 0x7f2e2c0c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e2c0744d0 0x7f2e2c158b30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 --2- 172.21.15.83:0/3789994734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e2c0afc90 0x7f2e2c15ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 -- 172.21.15.83:0/3789994734 >> 172.21.15.83:0/3789994734 conn(0x7f2e2c0d0bf0 msgr2=0x7f2e2c06ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 -- 172.21.15.83:0/3789994734 shutdown_connections 2023-11-26T07:12:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.420+0000 7f2e324fb700 1 -- 172.21.15.83:0/3789994734 wait complete. 2023-11-26T07:12:09.434 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:12:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4b656e700 1 Processor -- start 2023-11-26T07:12:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4b656e700 1 -- start start 2023-11-26T07:12:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4b656e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4b00afb40 0x7ff4b00cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4b656e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b00cccd0 con 0x7ff4b015ada0 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4b656e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b0062440 con 0x7ff4b00cfa80 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4b656e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00cd2e0 con 0x7ff4b00afb40 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4affff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff4b015ada0 0x7ff4b006f810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55652/0 (socket says 172.21.15.83:55652) 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4affff700 1 -- 172.21.15.83:0/4015355293 learned_addr learned my addr 172.21.15.83:0/4015355293 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4af7fe700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4b00afb40 0x7ff4b00cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247528362 0 0) 0x7ff4b00cccd0 con 0x7ff4b015ada0 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4980035b0 con 0x7ff4b015ada0 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338098634 0 0) 0x7ff4b0062440 con 0x7ff4b00cfa80 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4b00cccd0 con 0x7ff4b00cfa80 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3386804281 0 0) 0x7ff4980035b0 con 0x7ff4b015ada0 2023-11-26T07:12:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4b00afb40 msgr2=0x7ff4b00cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4b00afb40 0x7ff4b00cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 >> v1:172.21.15.203:6789/0 conn(0x7ff4b00cfa80 legacy=0x7ff4b006d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b0062720 con 0x7ff4b015ada0 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7ff4a0001670 con 0x7ff4b015ada0 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 >> v1:172.21.15.83:6789/0 conn(0x7ff4b015ada0 legacy=0x7ff4b006f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4b00afb40 0x7ff498003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff498004120 0x7ff498004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff498004d90 0x7ff498005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00cd2e0 con 0x7ff498004120 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49800bba0 con 0x7ff498004d90 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.556+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49800bd30 con 0x7ff4b00afb40 2023-11-26T07:12:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4af7fe700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff498004120 0x7ff498004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4affff700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4b00afb40 0x7ff498003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4b4b0b700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff498004d90 0x7ff498005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4af7fe700 1 -- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4b00afb40 msgr2=0x7ff498003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4af7fe700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4b00afb40 0x7ff498003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4af7fe700 1 -- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff498004d90 msgr2=0x7ff498005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4af7fe700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff498004d90 0x7ff498005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4af7fe700 1 -- 172.21.15.83:0/4015355293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980035b0 con 0x7ff498004120 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4affff700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4b00afb40 0x7ff498003bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4af7fe700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff498004120 0x7ff498004850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009b60 tx=0x7ff498005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbd1ae21d6c6f9a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400e6c0 con 0x7ff498004120 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a400e820 con 0x7ff498004120 2023-11-26T07:12:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.557+0000 7ff4aeffd700 1 -- 172.21.15.83:0/4015355293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400eb90 con 0x7ff498004120 2023-11-26T07:12:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.558+0000 7ff4b656e700 1 -- 172.21.15.83:0/4015355293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff498004120 msgr2=0x7ff498004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.558+0000 7ff4b656e700 1 --2- 172.21.15.83:0/4015355293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff498004120 0x7ff498004850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009b60 tx=0x7ff498005880 comp rx=0 tx=0).stop 2023-11-26T07:12:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.558+0000 7ff4b4b0b700 1 -- 172.21.15.83:0/4015355293 reap_dead start 2023-11-26T07:12:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.558+0000 7ff4b656e700 1 -- 172.21.15.83:0/4015355293 shutdown_connections 2023-11-26T07:12:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.558+0000 7ff4b656e700 1 -- 172.21.15.83:0/4015355293 >> 172.21.15.83:0/4015355293 conn(0x7ff4b00b2050 msgr2=0x7ff4b00b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.558+0000 7ff4b656e700 1 -- 172.21.15.83:0/4015355293 shutdown_connections 2023-11-26T07:12:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.558+0000 7ff4b656e700 1 -- 172.21.15.83:0/4015355293 wait complete. 2023-11-26T07:12:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.559+0000 7ff4b656e700 1 Processor -- start 2023-11-26T07:12:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.559+0000 7ff4b656e700 1 -- start start 2023-11-26T07:12:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4b656e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff498004120 0x7ff4b00aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4b656e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4b00cbfb0 0x7ff4b00b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4b656e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b00b3990 0x7ff4b00ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4b656e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00cbb90 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4b656e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00b3e40 con 0x7ff4b00b3990 2023-11-26T07:12:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4b656e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00aefa0 con 0x7ff498004120 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4af7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4b00cbfb0 0x7ff4b00b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4af7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4b00cbfb0 0x7ff4b00b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35626/0 (socket says 172.21.15.83:35626) 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4af7fe700 1 -- 172.21.15.83:0/1728964636 learned_addr learned my addr 172.21.15.83:0/1728964636 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4b4b0b700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b00b3990 0x7ff4b00ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4affff700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff498004120 0x7ff4b00aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4af7fe700 1 -- 172.21.15.83:0/1728964636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff498004120 msgr2=0x7ff4b00aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4af7fe700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff498004120 0x7ff4b00aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4af7fe700 1 -- 172.21.15.83:0/1728964636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b00b3990 msgr2=0x7ff4b00ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4af7fe700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b00b3990 0x7ff4b00ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4af7fe700 1 -- 172.21.15.83:0/1728964636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b00caf10 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4b4b0b700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b00b3990 0x7ff4b00ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:09.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4af7fe700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4b00cbfb0 0x7ff4b00b3450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400e4b0 tx=0x7ff4a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4acff9700 1 -- 172.21.15.83:0/1728964636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400e6c0 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4acff9700 1 -- 172.21.15.83:0/1728964636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a4003ad0 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4acff9700 1 -- 172.21.15.83:0/1728964636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4003e80 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4b656e700 1 -- 172.21.15.83:0/1728964636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b00cb1f0 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.560+0000 7ff4b656e700 1 -- 172.21.15.83:0/1728964636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b00cb7b0 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.561+0000 7ff4b656e700 1 -- 172.21.15.83:0/1728964636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b0068d90 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.562+0000 7ff4acff9700 1 -- 172.21.15.83:0/1728964636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4a4013470 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.562+0000 7ff4acff9700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff49c06aed0 0x7ff49c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.562+0000 7ff4acff9700 1 -- 172.21.15.83:0/1728964636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff4a408ae40 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.562+0000 7ff4affff700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff49c06aed0 0x7ff49c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.563+0000 7ff4affff700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff49c06aed0 0x7ff49c06d390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00016b0 tx=0x7ff4a0001e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:09.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.568+0000 7ff4acff9700 1 -- 172.21.15.83:0/1728964636 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4a4055ea0 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.763+0000 7ff4b656e700 1 -- 172.21.15.83:0/1728964636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4b0068fb0 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.763+0000 7ff4acff9700 1 -- 172.21.15.83:0/1728964636 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4a40170a0 con 0x7ff4b00cbfb0 2023-11-26T07:12:09.766 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:12:09.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 -- 172.21.15.83:0/1728964636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff49c06aed0 msgr2=0x7ff49c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff49c06aed0 0x7ff49c06d390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00016b0 tx=0x7ff4a0001e30 comp rx=0 tx=0).stop 2023-11-26T07:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 -- 172.21.15.83:0/1728964636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4b00cbfb0 msgr2=0x7ff4b00b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4b00cbfb0 0x7ff4b00b3450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400e4b0 tx=0x7ff4a40047c0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 -- 172.21.15.83:0/1728964636 shutdown_connections 2023-11-26T07:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4b00b3990 0x7ff4b00ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff49c06aed0 0x7ff49c06d390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff498004120 0x7ff4b00aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 --2- 172.21.15.83:0/1728964636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4b00cbfb0 0x7ff4b00b3450 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 -- 172.21.15.83:0/1728964636 >> 172.21.15.83:0/1728964636 conn(0x7ff4b00b2050 msgr2=0x7ff4b015dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 -- 172.21.15.83:0/1728964636 shutdown_connections 2023-11-26T07:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.766+0000 7ff4b656e700 1 -- 172.21.15.83:0/1728964636 wait complete. 2023-11-26T07:12:09.780 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:09.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.898+0000 7ff424d6e700 1 Processor -- start 2023-11-26T07:12:09.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.898+0000 7ff424d6e700 1 -- start start 2023-11-26T07:12:09.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff420158670 0x7ff4200622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff4200c9fc0 0x7ff4200ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4200cfff0 con 0x7ff420158a50 2023-11-26T07:12:09.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4200cfd10 con 0x7ff420158670 2023-11-26T07:12:09.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff420161f50 con 0x7ff4200c9fc0 2023-11-26T07:12:09.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41e59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff420158670 0x7ff4200622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41dd9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff420158a50 0x7ff4200c98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55672/0 (socket says 172.21.15.83:55672) 2023-11-26T07:12:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41dd9b700 1 -- 172.21.15.83:0/1836389070 learned_addr learned my addr 172.21.15.83:0/1836389070 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41e59c700 1 -- 172.21.15.83:0/1836389070 >> v2:172.21.15.203:3300/0 conn(0x7ff4200c9fc0 msgr2=0x7ff4200ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41e59c700 1 --2- 172.21.15.83:0/1836389070 >> v2:172.21.15.203:3300/0 conn(0x7ff4200c9fc0 0x7ff4200ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41e59c700 1 -- 172.21.15.83:0/1836389070 >> v1:172.21.15.83:6789/0 conn(0x7ff420158a50 legacy=0x7ff4200c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41e59c700 1 -- 172.21.15.83:0/1836389070 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff420162230 con 0x7ff420158670 2023-11-26T07:12:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41d59a700 1 -- 172.21.15.83:0/1836389070 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102028987 0 0) 0x7ff4200cfff0 con 0x7ff420158a50 2023-11-26T07:12:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41e59c700 1 --2- 172.21.15.83:0/1836389070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff420158670 0x7ff4200622a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff4200b0070 tx=0x7ff410009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735f541b219b27b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41d59a700 1 -- 172.21.15.83:0/1836389070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410011830 con 0x7ff420158670 2023-11-26T07:12:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41d59a700 1 -- 172.21.15.83:0/1836389070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff410011990 con 0x7ff420158670 2023-11-26T07:12:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff41d59a700 1 -- 172.21.15.83:0/1836389070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410011d20 con 0x7ff420158670 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 -- 172.21.15.83:0/1836389070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff420158670 msgr2=0x7ff4200622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 --2- 172.21.15.83:0/1836389070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff420158670 0x7ff4200622a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff4200b0070 tx=0x7ff410009b20 comp rx=0 tx=0).stop 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 -- 172.21.15.83:0/1836389070 shutdown_connections 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 --2- 172.21.15.83:0/1836389070 >> v2:172.21.15.203:3300/0 conn(0x7ff4200c9fc0 0x7ff4200ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 --2- 172.21.15.83:0/1836389070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff420158670 0x7ff4200622a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 -- 172.21.15.83:0/1836389070 >> 172.21.15.83:0/1836389070 conn(0x7ff4200b57b0 msgr2=0x7ff4200b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 -- 172.21.15.83:0/1836389070 shutdown_connections 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.899+0000 7ff424d6e700 1 -- 172.21.15.83:0/1836389070 wait complete. 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.901+0000 7ff424d6e700 1 Processor -- start 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.901+0000 7ff424d6e700 1 -- start start 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.901+0000 7ff424d6e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4200c9fc0 0x7ff4200cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.901+0000 7ff424d6e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff420158670 0x7ff4200ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.901+0000 7ff424d6e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff420158a50 0x7ff4200cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.901+0000 7ff424d6e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200cd820 con 0x7ff420158670 2023-11-26T07:12:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.901+0000 7ff424d6e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff420148570 con 0x7ff4200c9fc0 2023-11-26T07:12:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.901+0000 7ff424d6e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200b3120 con 0x7ff420158a50 2023-11-26T07:12:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.901+0000 7ff41dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff420158670 0x7ff4200ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.901+0000 7ff41dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff420158670 0x7ff4200ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35640/0 (socket says 172.21.15.83:35640) 2023-11-26T07:12:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.901+0000 7ff41dd9b700 1 -- 172.21.15.83:0/2388730946 learned_addr learned my addr 172.21.15.83:0/2388730946 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff41e59c700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4200c9fc0 0x7ff4200cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff41ed9d700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff420158a50 0x7ff4200cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff41dd9b700 1 -- 172.21.15.83:0/2388730946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff420158a50 msgr2=0x7ff4200cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff41dd9b700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff420158a50 0x7ff4200cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff41dd9b700 1 -- 172.21.15.83:0/2388730946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4200c9fc0 msgr2=0x7ff4200cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff41dd9b700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4200c9fc0 0x7ff4200cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff41dd9b700 1 -- 172.21.15.83:0/2388730946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4100094a0 con 0x7ff420158670 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff41e59c700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4200c9fc0 0x7ff4200cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff41dd9b700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff420158670 0x7ff4200ccda0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c460 tx=0x7ff41400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff40effd700 1 -- 172.21.15.83:0/2388730946 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414004c80 con 0x7ff420158670 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff40effd700 1 -- 172.21.15.83:0/2388730946 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff414004e00 con 0x7ff420158670 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff424d6e700 1 -- 172.21.15.83:0/2388730946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4200b3400 con 0x7ff420158670 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff40effd700 1 -- 172.21.15.83:0/2388730946 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414007310 con 0x7ff420158670 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.902+0000 7ff424d6e700 1 -- 172.21.15.83:0/2388730946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4200b39c0 con 0x7ff420158670 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.903+0000 7ff424d6e700 1 -- 172.21.15.83:0/2388730946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4200b3d30 con 0x7ff420158670 2023-11-26T07:12:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.903+0000 7ff40effd700 1 -- 172.21.15.83:0/2388730946 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff414007470 con 0x7ff420158670 2023-11-26T07:12:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.905+0000 7ff40effd700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff40006aed0 0x7ff40006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.905+0000 7ff40effd700 1 -- 172.21.15.83:0/2388730946 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff414089310 con 0x7ff420158670 2023-11-26T07:12:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.905+0000 7ff41e59c700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff40006aed0 0x7ff40006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.905+0000 7ff41e59c700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff40006aed0 0x7ff40006d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff4100090b0 tx=0x7ff410008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:09.909+0000 7ff40effd700 1 -- 172.21.15.83:0/2388730946 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff414054340 con 0x7ff420158670 2023-11-26T07:12:10.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.139+0000 7ff424d6e700 1 -- 172.21.15.83:0/2388730946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff420067f60 con 0x7ff420158670 2023-11-26T07:12:10.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.140+0000 7ff40effd700 1 -- 172.21.15.83:0/2388730946 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7ff414057960 con 0x7ff420158670 2023-11-26T07:12:10.141 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:10.141 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2208741688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2208741688},{"type":"v1","addr":"172.21.15.83:6837","nonce":2208741688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2802126359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2802126359},{"type":"v1","addr":"172.21.15.83:6836","nonce":2802126359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1258871616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1258871616},{"type":"v1","addr":"172.21.15.203:6834","nonce":1258871616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:11:27.911475+0000","modified":"2023-11-26T07:11:30.937105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/2957324266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":2957324266},{"type":"v1","addr":"172.21.15.203:6835","nonce":2957324266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:12:10.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 -- 172.21.15.83:0/2388730946 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff40006aed0 msgr2=0x7ff40006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff40006aed0 0x7ff40006d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff4100090b0 tx=0x7ff410008030 comp rx=0 tx=0).stop 2023-11-26T07:12:10.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 -- 172.21.15.83:0/2388730946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff420158670 msgr2=0x7ff4200ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff420158670 0x7ff4200ccda0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c460 tx=0x7ff41400c820 comp rx=0 tx=0).stop 2023-11-26T07:12:10.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 -- 172.21.15.83:0/2388730946 shutdown_connections 2023-11-26T07:12:10.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff40006aed0 0x7ff40006d390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff420158a50 0x7ff4200cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4200c9fc0 0x7ff4200cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 --2- 172.21.15.83:0/2388730946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff420158670 0x7ff4200ccda0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 -- 172.21.15.83:0/2388730946 >> 172.21.15.83:0/2388730946 conn(0x7ff4200b57b0 msgr2=0x7ff42006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:10.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 -- 172.21.15.83:0/2388730946 shutdown_connections 2023-11-26T07:12:10.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.142+0000 7ff424d6e700 1 -- 172.21.15.83:0/2388730946 wait complete. 2023-11-26T07:12:10.145 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2023-11-26T07:12:10.156 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:10.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.274+0000 7f2231ece700 1 Processor -- start 2023-11-26T07:12:10.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.274+0000 7f2231ece700 1 -- start start 2023-11-26T07:12:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f2231ece700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f222c158a50 0x7f222c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f2231ece700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f222c162a30 0x7f222c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f2231ece700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222c0cfff0 con 0x7f222c158670 2023-11-26T07:12:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f2231ece700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222c0629b0 con 0x7f222c158a50 2023-11-26T07:12:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f2231ece700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c0cfd10 con 0x7f222c162a30 2023-11-26T07:12:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f222affd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f222c158a50 0x7f222c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f222affd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f222c158a50 0x7f222c0c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35648/0 (socket says 172.21.15.83:35648) 2023-11-26T07:12:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f222affd700 1 -- 172.21.15.83:0/3839139553 learned_addr learned my addr 172.21.15.83:0/3839139553 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:10.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f222bfff700 1 --2- 172.21.15.83:0/3839139553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f222c162a30 0x7f222c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f222affd700 1 -- 172.21.15.83:0/3839139553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f222c162a30 msgr2=0x7f222c0c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f222affd700 1 --2- 172.21.15.83:0/3839139553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f222c162a30 0x7f222c0c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f222affd700 1 -- 172.21.15.83:0/3839139553 >> v1:172.21.15.203:6789/0 conn(0x7f222c158670 legacy=0x7f222c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.275+0000 7f222affd700 1 -- 172.21.15.83:0/3839139553 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c161f50 con 0x7f222c158a50 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f222affd700 1 --2- 172.21.15.83:0/3839139553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222c158a50 0x7f222c0c98b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f221c0098f0 tx=0x7f221c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4391492f6073807d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f222a7fc700 1 -- 172.21.15.83:0/3839139553 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c00b8a0 con 0x7f222c158a50 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f222a7fc700 1 -- 172.21.15.83:0/3839139553 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221c00ba00 con 0x7f222c158a50 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f222a7fc700 1 -- 172.21.15.83:0/3839139553 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c00bd90 con 0x7f222c158a50 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f2231ece700 1 -- 172.21.15.83:0/3839139553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222c158a50 msgr2=0x7f222c0c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f2231ece700 1 --2- 172.21.15.83:0/3839139553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222c158a50 0x7f222c0c98b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f221c0098f0 tx=0x7f221c009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f2231ece700 1 -- 172.21.15.83:0/3839139553 shutdown_connections 2023-11-26T07:12:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f2231ece700 1 --2- 172.21.15.83:0/3839139553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f222c162a30 0x7f222c0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f2231ece700 1 --2- 172.21.15.83:0/3839139553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222c158a50 0x7f222c0c98b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f2231ece700 1 -- 172.21.15.83:0/3839139553 >> 172.21.15.83:0/3839139553 conn(0x7f222c0b57b0 msgr2=0x7f222c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f2231ece700 1 -- 172.21.15.83:0/3839139553 shutdown_connections 2023-11-26T07:12:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.276+0000 7f2231ece700 1 -- 172.21.15.83:0/3839139553 wait complete. 2023-11-26T07:12:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.278+0000 7f2231ece700 1 Processor -- start 2023-11-26T07:12:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.278+0000 7f2231ece700 1 -- start start 2023-11-26T07:12:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.278+0000 7f2231ece700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f222c158670 0x7f222c158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.278+0000 7f2231ece700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222c158a50 0x7f222c14d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.278+0000 7f2231ece700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222c162a30 0x7f222c15d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.278+0000 7f2231ece700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c1579d0 con 0x7f222c162a30 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.278+0000 7f2231ece700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c159370 con 0x7f222c158a50 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f2231ece700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c0b3460 con 0x7f222c158670 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222c162a30 0x7f222c15d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222c162a30 0x7f222c15d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35656/0 (socket says 172.21.15.83:35656) 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222bfff700 1 -- 172.21.15.83:0/745709495 learned_addr learned my addr 172.21.15.83:0/745709495 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222b7fe700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f222c158670 0x7f222c158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222affd700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222c158a50 0x7f222c14d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222bfff700 1 -- 172.21.15.83:0/745709495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f222c158670 msgr2=0x7f222c158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222bfff700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f222c158670 0x7f222c158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222bfff700 1 -- 172.21.15.83:0/745709495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222c158a50 msgr2=0x7f222c14d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222bfff700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222c158a50 0x7f222c14d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222bfff700 1 -- 172.21.15.83:0/745709495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c009550 con 0x7f222c162a30 2023-11-26T07:12:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222affd700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222c158a50 0x7f222c14d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f222bfff700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222c162a30 0x7f222c15d330 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2224007e90 tx=0x7f222400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f2223fff700 1 -- 172.21.15.83:0/745709495 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222400ede0 con 0x7f222c162a30 2023-11-26T07:12:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f2223fff700 1 -- 172.21.15.83:0/745709495 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2224014420 con 0x7f222c162a30 2023-11-26T07:12:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f2231ece700 1 -- 172.21.15.83:0/745709495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c0b3160 con 0x7f222c162a30 2023-11-26T07:12:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f2223fff700 1 -- 172.21.15.83:0/745709495 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22240147d0 con 0x7f222c162a30 2023-11-26T07:12:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.279+0000 7f2231ece700 1 -- 172.21.15.83:0/745709495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c159880 con 0x7f222c162a30 2023-11-26T07:12:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.280+0000 7f2231ece700 1 -- 172.21.15.83:0/745709495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c0b35d0 con 0x7f222c162a30 2023-11-26T07:12:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.281+0000 7f2223fff700 1 -- 172.21.15.83:0/745709495 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2224014930 con 0x7f222c162a30 2023-11-26T07:12:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.281+0000 7f2223fff700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f220c06aed0 0x7f220c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.281+0000 7f222b7fe700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f220c06aed0 0x7f220c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.281+0000 7f2223fff700 1 -- 172.21.15.83:0/745709495 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2224089d30 con 0x7f222c162a30 2023-11-26T07:12:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.282+0000 7f222b7fe700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f220c06aed0 0x7f220c06d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2214007fd0 tx=0x7f2214009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:10.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.286+0000 7f2223fff700 1 -- 172.21.15.83:0/745709495 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2224054d60 con 0x7f222c162a30 2023-11-26T07:12:10.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.482+0000 7f2231ece700 1 -- 172.21.15.83:0/745709495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f222c067f60 con 0x7f222c162a30 2023-11-26T07:12:10.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.483+0000 7f2223fff700 1 -- 172.21.15.83:0/745709495 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f2224058380 con 0x7f222c162a30 2023-11-26T07:12:10.485 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:10.485 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:09.712121+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 -- 172.21.15.83:0/745709495 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f220c06aed0 msgr2=0x7f220c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f220c06aed0 0x7f220c06d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2214007fd0 tx=0x7f2214009040 comp rx=0 tx=0).stop 2023-11-26T07:12:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 -- 172.21.15.83:0/745709495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222c162a30 msgr2=0x7f222c15d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222c162a30 0x7f222c15d330 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2224007e90 tx=0x7f222400e3c0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 -- 172.21.15.83:0/745709495 shutdown_connections 2023-11-26T07:12:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222c162a30 0x7f222c15d330 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f220c06aed0 0x7f220c06d390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f222c158670 0x7f222c158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 --2- 172.21.15.83:0/745709495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222c158a50 0x7f222c14d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 -- 172.21.15.83:0/745709495 >> 172.21.15.83:0/745709495 conn(0x7f222c0b57b0 msgr2=0x7f222c0b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 -- 172.21.15.83:0/745709495 shutdown_connections 2023-11-26T07:12:10.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.485+0000 7f2231ece700 1 -- 172.21.15.83:0/745709495 wait complete. 2023-11-26T07:12:10.499 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:12:10.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.617+0000 7f6b97ebb700 1 Processor -- start 2023-11-26T07:12:10.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.618+0000 7f6b97ebb700 1 -- start start 2023-11-26T07:12:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b97ebb700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6b9015c160 0x7f6b900622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b97ebb700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b900b04b0 0x7f6b90165c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b97ebb700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9015c540 con 0x7f6b900b2870 2023-11-26T07:12:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b97ebb700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b900b2ee0 con 0x7f6b9015c160 2023-11-26T07:12:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b97ebb700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900add40 con 0x7f6b900b04b0 2023-11-26T07:12:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b95456700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6b900b2870 0x7f6b900afda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55702/0 (socket says 172.21.15.83:55702) 2023-11-26T07:12:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b95456700 1 -- 172.21.15.83:0/1519226798 learned_addr learned my addr 172.21.15.83:0/1519226798 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b96458700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b900b04b0 0x7f6b90165c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574057331 0 0) 0x7f6b9015c540 con 0x7f6b900b2870 2023-11-26T07:12:10.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b95c57700 1 --2- 172.21.15.83:0/1519226798 >> v2:172.21.15.203:3300/0 conn(0x7f6b9015c160 0x7f6b900622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b7c0035b0 con 0x7f6b900b2870 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1766373666 0 0) 0x7f6b7c0035b0 con 0x7f6b900b2870 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b900b04b0 msgr2=0x7f6b90165c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b94c55700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b900b04b0 0x7f6b90165c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 >> v2:172.21.15.203:3300/0 conn(0x7f6b9015c160 msgr2=0x7f6b900622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b94c55700 1 --2- 172.21.15.83:0/1519226798 >> v2:172.21.15.203:3300/0 conn(0x7f6b9015c160 0x7f6b900622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b900ad410 con 0x7f6b900b2870 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f6b80001670 con 0x7f6b900b2870 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b95c57700 1 --2- 172.21.15.83:0/1519226798 >> v2:172.21.15.203:3300/0 conn(0x7f6b9015c160 0x7f6b900622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 >> v1:172.21.15.83:6789/0 conn(0x7f6b900b2870 legacy=0x7f6b900afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.619+0000 7f6b94c55700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b7c003a00 0x7f6b7c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b94c55700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 0x7f6b7c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b94c55700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b900b04b0 0x7f6b7c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900b2ee0 con 0x7f6b7c004850 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900add40 con 0x7f6b7c003a00 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c00ba40 con 0x7f6b900b04b0 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b95456700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 0x7f6b7c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b95c57700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b7c003a00 0x7f6b7c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b96458700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b900b04b0 0x7f6b7c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b95456700 1 -- 172.21.15.83:0/1519226798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b900b04b0 msgr2=0x7f6b7c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b95456700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b900b04b0 0x7f6b7c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b95456700 1 -- 172.21.15.83:0/1519226798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b7c003a00 msgr2=0x7f6b7c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b95456700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b7c003a00 0x7f6b7c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b95456700 1 -- 172.21.15.83:0/1519226798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c0035b0 con 0x7f6b7c004850 2023-11-26T07:12:10.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b95456700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 0x7f6b7c004cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000c370 tx=0x7f6b7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c2502b2d327da2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80007c60 con 0x7f6b7c004850 2023-11-26T07:12:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b80005960 con 0x7f6b7c004850 2023-11-26T07:12:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.620+0000 7f6b94c55700 1 -- 172.21.15.83:0/1519226798 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80005d10 con 0x7f6b7c004850 2023-11-26T07:12:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.621+0000 7f6b97ebb700 1 -- 172.21.15.83:0/1519226798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 msgr2=0x7f6b7c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.621+0000 7f6b97ebb700 1 --2- 172.21.15.83:0/1519226798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 0x7f6b7c004cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000c370 tx=0x7f6b7c005880 comp rx=0 tx=0).stop 2023-11-26T07:12:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.621+0000 7f6b96458700 1 -- 172.21.15.83:0/1519226798 reap_dead start 2023-11-26T07:12:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.621+0000 7f6b97ebb700 1 -- 172.21.15.83:0/1519226798 shutdown_connections 2023-11-26T07:12:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.621+0000 7f6b97ebb700 1 -- 172.21.15.83:0/1519226798 >> 172.21.15.83:0/1519226798 conn(0x7f6b900b57b0 msgr2=0x7f6b900b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.621+0000 7f6b97ebb700 1 -- 172.21.15.83:0/1519226798 shutdown_connections 2023-11-26T07:12:10.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.621+0000 7f6b97ebb700 1 -- 172.21.15.83:0/1519226798 wait complete. 2023-11-26T07:12:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.622+0000 7f6b97ebb700 1 Processor -- start 2023-11-26T07:12:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.622+0000 7f6b97ebb700 1 -- start start 2023-11-26T07:12:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b97ebb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 0x7f6b901519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b97ebb700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b9015def0 0x7f6b900c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b97ebb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b900c9b50 0x7f6b900cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b97ebb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900c9fd0 con 0x7f6b7c004850 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b97ebb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b901f70e0 con 0x7f6b900c9b50 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b97ebb700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b901f7f40 con 0x7f6b9015def0 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b95c57700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 0x7f6b901519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b95c57700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 0x7f6b901519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35672/0 (socket says 172.21.15.83:35672) 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b95c57700 1 -- 172.21.15.83:0/2096786355 learned_addr learned my addr 172.21.15.83:0/2096786355 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b96458700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b900c9b50 0x7f6b900cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b95c57700 1 -- 172.21.15.83:0/2096786355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b9015def0 msgr2=0x7f6b900c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b95456700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b9015def0 0x7f6b900c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b95c57700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b9015def0 0x7f6b900c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b95c57700 1 -- 172.21.15.83:0/2096786355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b900c9b50 msgr2=0x7f6b900cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b95c57700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b900c9b50 0x7f6b900cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b95c57700 1 -- 172.21.15.83:0/2096786355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b90151f00 con 0x7f6b7c004850 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b96458700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b900c9b50 0x7f6b900cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b95456700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b9015def0 0x7f6b900c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.623+0000 7f6b95c57700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 0x7f6b901519c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c011710 tx=0x7f6b8c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.624+0000 7f6b867fc700 1 -- 172.21.15.83:0/2096786355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c004440 con 0x7f6b7c004850 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.624+0000 7f6b867fc700 1 -- 172.21.15.83:0/2096786355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c0045a0 con 0x7f6b7c004850 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.624+0000 7f6b97ebb700 1 -- 172.21.15.83:0/2096786355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b901f7330 con 0x7f6b7c004850 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.624+0000 7f6b867fc700 1 -- 172.21.15.83:0/2096786355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c0038a0 con 0x7f6b7c004850 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.624+0000 7f6b97ebb700 1 -- 172.21.15.83:0/2096786355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b901f78f0 con 0x7f6b7c004850 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.625+0000 7f6b97ebb700 1 -- 172.21.15.83:0/2096786355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b900b4100 con 0x7f6b7c004850 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.625+0000 7f6b867fc700 1 -- 172.21.15.83:0/2096786355 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6b8c003a00 con 0x7f6b7c004850 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.625+0000 7f6b867fc700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b7406aed0 0x7f6b7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.625+0000 7f6b867fc700 1 -- 172.21.15.83:0/2096786355 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6b8c089b20 con 0x7f6b7c004850 2023-11-26T07:12:10.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.625+0000 7f6b95456700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b7406aed0 0x7f6b7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:10.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.626+0000 7f6b95456700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b7406aed0 0x7f6b7406d390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6b900b3140 tx=0x7f6b8000a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:10.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.630+0000 7f6b867fc700 1 -- 172.21.15.83:0/2096786355 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b8c054b50 con 0x7f6b7c004850 2023-11-26T07:12:10.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.861+0000 7f6b97ebb700 1 -- 172.21.15.83:0/2096786355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b90067f60 con 0x7f6b7c004850 2023-11-26T07:12:10.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.862+0000 7f6b867fc700 1 -- 172.21.15.83:0/2096786355 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7f6b8c00c090 con 0x7f6b7c004850 2023-11-26T07:12:10.863 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:10.863 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2208741688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2208741688},{"type":"v1","addr":"172.21.15.83:6837","nonce":2208741688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2802126359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2802126359},{"type":"v1","addr":"172.21.15.83:6836","nonce":2802126359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1258871616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1258871616},{"type":"v1","addr":"172.21.15.203:6834","nonce":1258871616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:11:27.911475+0000","modified":"2023-11-26T07:11:30.937105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/2957324266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":2957324266},{"type":"v1","addr":"172.21.15.203:6835","nonce":2957324266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:12:10.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 -- 172.21.15.83:0/2096786355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b7406aed0 msgr2=0x7f6b7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b7406aed0 0x7f6b7406d390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6b900b3140 tx=0x7f6b8000a000 comp rx=0 tx=0).stop 2023-11-26T07:12:10.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 -- 172.21.15.83:0/2096786355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 msgr2=0x7f6b901519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:10.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 0x7f6b901519c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c011710 tx=0x7f6b8c011ad0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 -- 172.21.15.83:0/2096786355 shutdown_connections 2023-11-26T07:12:10.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b900c9b50 0x7f6b900cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b7c004850 0x7f6b901519c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b7406aed0 0x7f6b7406d390 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 --2- 172.21.15.83:0/2096786355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b9015def0 0x7f6b900c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:10.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 -- 172.21.15.83:0/2096786355 >> 172.21.15.83:0/2096786355 conn(0x7f6b900b57b0 msgr2=0x7f6b900cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:10.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 -- 172.21.15.83:0/2096786355 shutdown_connections 2023-11-26T07:12:10.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.864+0000 7f6b97ebb700 1 -- 172.21.15.83:0/2096786355 wait complete. 2023-11-26T07:12:10.867 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2023-11-26T07:12:10.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:10.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.995+0000 7fba2b9d8700 1 Processor -- start 2023-11-26T07:12:10.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.996+0000 7fba2b9d8700 1 -- start start 2023-11-26T07:12:10.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.996+0000 7fba2b9d8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fba24158670 0x7fba240622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.996+0000 7fba2b9d8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fba240c9fc0 0x7fba240ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:10.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.996+0000 7fba2b9d8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba240cfff0 con 0x7fba24158a50 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.996+0000 7fba2b9d8700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba240cfd10 con 0x7fba240c9fc0 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.996+0000 7fba2b9d8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba24161f50 con 0x7fba24158670 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba29f75700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fba240c9fc0 0x7fba240ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba28f73700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fba24158a50 0x7fba240c98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55720/0 (socket says 172.21.15.83:55720) 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba28f73700 1 -- 172.21.15.83:0/555380144 learned_addr learned my addr 172.21.15.83:0/555380144 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba29f75700 1 -- 172.21.15.83:0/555380144 >> v2:172.21.15.203:3300/0 conn(0x7fba24158670 msgr2=0x7fba240622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba29774700 1 --2- 172.21.15.83:0/555380144 >> v2:172.21.15.203:3300/0 conn(0x7fba24158670 0x7fba240622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba29f75700 1 --2- 172.21.15.83:0/555380144 >> v2:172.21.15.203:3300/0 conn(0x7fba24158670 0x7fba240622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba29f75700 1 -- 172.21.15.83:0/555380144 >> v1:172.21.15.83:6789/0 conn(0x7fba24158a50 legacy=0x7fba240c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba29f75700 1 -- 172.21.15.83:0/555380144 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24162230 con 0x7fba240c9fc0 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba1bfff700 1 -- 172.21.15.83:0/555380144 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944257210 0 0) 0x7fba240cfff0 con 0x7fba24158a50 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba29774700 1 --2- 172.21.15.83:0/555380144 >> v2:172.21.15.203:3300/0 conn(0x7fba24158670 0x7fba240622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba29f75700 1 --2- 172.21.15.83:0/555380144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba240c9fc0 0x7fba240ce1a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fba2409e5c0 tx=0x7fba1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33bda2b8ff4abd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba1bfff700 1 -- 172.21.15.83:0/555380144 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00e070 con 0x7fba240c9fc0 2023-11-26T07:12:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba1bfff700 1 -- 172.21.15.83:0/555380144 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c00bbd0 con 0x7fba240c9fc0 2023-11-26T07:12:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.997+0000 7fba1bfff700 1 -- 172.21.15.83:0/555380144 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c012430 con 0x7fba240c9fc0 2023-11-26T07:12:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.998+0000 7fba2b9d8700 1 -- 172.21.15.83:0/555380144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba240c9fc0 msgr2=0x7fba240ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.998+0000 7fba2b9d8700 1 --2- 172.21.15.83:0/555380144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba240c9fc0 0x7fba240ce1a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fba2409e5c0 tx=0x7fba1c00b330 comp rx=0 tx=0).stop 2023-11-26T07:12:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.998+0000 7fba2b9d8700 1 -- 172.21.15.83:0/555380144 shutdown_connections 2023-11-26T07:12:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.998+0000 7fba2b9d8700 1 --2- 172.21.15.83:0/555380144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba240c9fc0 0x7fba240ce1a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.998+0000 7fba2b9d8700 1 --2- 172.21.15.83:0/555380144 >> v2:172.21.15.203:3300/0 conn(0x7fba24158670 0x7fba240622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.998+0000 7fba2b9d8700 1 -- 172.21.15.83:0/555380144 >> 172.21.15.83:0/555380144 conn(0x7fba240b57b0 msgr2=0x7fba240b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.998+0000 7fba2b9d8700 1 -- 172.21.15.83:0/555380144 shutdown_connections 2023-11-26T07:12:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.998+0000 7fba2b9d8700 1 -- 172.21.15.83:0/555380144 wait complete. 2023-11-26T07:12:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.999+0000 7fba2b9d8700 1 Processor -- start 2023-11-26T07:12:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:10.999+0000 7fba2b9d8700 1 -- start start 2023-11-26T07:12:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.000+0000 7fba2b9d8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba240c9fc0 0x7fba24149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.000+0000 7fba2b9d8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba24158670 0x7fba241531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.000+0000 7fba2b9d8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba24158a50 0x7fba241609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.000+0000 7fba2b9d8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba240b54d0 con 0x7fba240c9fc0 2023-11-26T07:12:11.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.000+0000 7fba2b9d8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba240b5040 con 0x7fba24158670 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.000+0000 7fba2b9d8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba240b4430 con 0x7fba24158a50 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.000+0000 7fba29774700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba240c9fc0 0x7fba24149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.000+0000 7fba29774700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba240c9fc0 0x7fba24149860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35686/0 (socket says 172.21.15.83:35686) 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.000+0000 7fba29774700 1 -- 172.21.15.83:0/197119096 learned_addr learned my addr 172.21.15.83:0/197119096 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba28f73700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba24158670 0x7fba241531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba29f75700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba24158a50 0x7fba241609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba29774700 1 -- 172.21.15.83:0/197119096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba24158a50 msgr2=0x7fba241609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba29774700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba24158a50 0x7fba241609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba29774700 1 -- 172.21.15.83:0/197119096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba24158670 msgr2=0x7fba241531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba29774700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba24158670 0x7fba241531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba29774700 1 -- 172.21.15.83:0/197119096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c009a70 con 0x7fba240c9fc0 2023-11-26T07:12:11.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba29f75700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba24158a50 0x7fba241609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba29774700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba240c9fc0 0x7fba24149860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c750 tx=0x7fba1400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba19ffb700 1 -- 172.21.15.83:0/197119096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14011800 con 0x7fba240c9fc0 2023-11-26T07:12:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba19ffb700 1 -- 172.21.15.83:0/197119096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba14011960 con 0x7fba240c9fc0 2023-11-26T07:12:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba19ffb700 1 -- 172.21.15.83:0/197119096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14011cd0 con 0x7fba240c9fc0 2023-11-26T07:12:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba2b9d8700 1 -- 172.21.15.83:0/197119096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba240b4710 con 0x7fba240c9fc0 2023-11-26T07:12:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.001+0000 7fba2b9d8700 1 -- 172.21.15.83:0/197119096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba240b4cd0 con 0x7fba240c9fc0 2023-11-26T07:12:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.002+0000 7fba2b9d8700 1 -- 172.21.15.83:0/197119096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba24162a30 con 0x7fba240c9fc0 2023-11-26T07:12:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.002+0000 7fba19ffb700 1 -- 172.21.15.83:0/197119096 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba14021350 con 0x7fba240c9fc0 2023-11-26T07:12:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.003+0000 7fba19ffb700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba0806aed0 0x7fba0806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.003+0000 7fba19ffb700 1 -- 172.21.15.83:0/197119096 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fba14089510 con 0x7fba240c9fc0 2023-11-26T07:12:11.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.003+0000 7fba28f73700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba0806aed0 0x7fba0806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.003+0000 7fba28f73700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba0806aed0 0x7fba0806d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba24153ed0 tx=0x7fba20007c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:11.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.007+0000 7fba19ffb700 1 -- 172.21.15.83:0/197119096 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba140545d0 con 0x7fba240c9fc0 2023-11-26T07:12:11.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.238+0000 7fba2b9d8700 1 -- 172.21.15.83:0/197119096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba240b4e60 con 0x7fba240c9fc0 2023-11-26T07:12:11.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.239+0000 7fba19ffb700 1 -- 172.21.15.83:0/197119096 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4913 (secure 0 0 0) 0x7fba14057bf0 con 0x7fba240c9fc0 2023-11-26T07:12:11.240 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:11.241 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2208741688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2208741688},{"type":"v1","addr":"172.21.15.83:6837","nonce":2208741688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/2802126359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2802126359},{"type":"v1","addr":"172.21.15.83:6836","nonce":2802126359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1258871616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1258871616},{"type":"v1","addr":"172.21.15.203:6834","nonce":1258871616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:11:27.911475+0000","modified":"2023-11-26T07:11:30.937105+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/2957324266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":2957324266},{"type":"v1","addr":"172.21.15.203:6835","nonce":2957324266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:12:11.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 -- 172.21.15.83:0/197119096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba0806aed0 msgr2=0x7fba0806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba0806aed0 0x7fba0806d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fba24153ed0 tx=0x7fba20007c90 comp rx=0 tx=0).stop 2023-11-26T07:12:11.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 -- 172.21.15.83:0/197119096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba240c9fc0 msgr2=0x7fba24149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba240c9fc0 0x7fba24149860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c750 tx=0x7fba1400cb10 comp rx=0 tx=0).stop 2023-11-26T07:12:11.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 -- 172.21.15.83:0/197119096 shutdown_connections 2023-11-26T07:12:11.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fba0806aed0 0x7fba0806d390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba24158a50 0x7fba241609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba240c9fc0 0x7fba24149860 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 --2- 172.21.15.83:0/197119096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba24158670 0x7fba241531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 -- 172.21.15.83:0/197119096 >> 172.21.15.83:0/197119096 conn(0x7fba240b57b0 msgr2=0x7fba24061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 -- 172.21.15.83:0/197119096 shutdown_connections 2023-11-26T07:12:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.241+0000 7fba2b9d8700 1 -- 172.21.15.83:0/197119096 wait complete. 2023-11-26T07:12:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2023-11-26T07:12:11.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.373+0000 7fa414a35700 1 Processor -- start 2023-11-26T07:12:11.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.373+0000 7fa414a35700 1 -- start start 2023-11-26T07:12:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.374+0000 7fa414a35700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa41015bdd0 0x7fa4101604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.374+0000 7fa414a35700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4100b22d0 con 0x7fa4100d1c70 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.374+0000 7fa414a35700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4100b25b0 con 0x7fa4100b3370 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.374+0000 7fa414a35700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4100b2f50 con 0x7fa41015bdd0 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.374+0000 7fa40ed9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa4100d1c70 0x7fa410165ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55738/0 (socket says 172.21.15.83:55738) 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.374+0000 7fa40ed9d700 1 -- 172.21.15.83:0/2444810677 learned_addr learned my addr 172.21.15.83:0/2444810677 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964595389 0 0) 0x7fa4100b22d0 con 0x7fa4100d1c70 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f80035b0 con 0x7fa4100d1c70 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40e59c700 1 --2- 172.21.15.83:0/2444810677 >> v2:172.21.15.203:3300/0 conn(0x7fa41015bdd0 0x7fa4101604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1704814691 0 0) 0x7fa3f80035b0 con 0x7fa4100d1c70 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 >> v2:172.21.15.203:3300/0 conn(0x7fa41015bdd0 msgr2=0x7fa4101604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 --2- 172.21.15.83:0/2444810677 >> v2:172.21.15.203:3300/0 conn(0x7fa41015bdd0 0x7fa4101604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 >> v1:172.21.15.203:6789/0 conn(0x7fa4100b3370 legacy=0x7fa4100b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4100b2890 con 0x7fa4100d1c70 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fa4080019f0 con 0x7fa4100d1c70 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40e59c700 1 --2- 172.21.15.83:0/2444810677 >> v2:172.21.15.203:3300/0 conn(0x7fa41015bdd0 0x7fa4101604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 >> v1:172.21.15.83:6789/0 conn(0x7fa4100d1c70 legacy=0x7fa410165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 --2- 172.21.15.83:0/2444810677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f8003a00 0x7fa3f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 --2- 172.21.15.83:0/2444810677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f8004850 0x7fa3f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 --2- 172.21.15.83:0/2444810677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 0x7fa3f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4100b2f50 con 0x7fa3f8005210 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f800c160 con 0x7fa3f8003a00 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f800c320 con 0x7fa3f8004850 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40ed9d700 1 --2- 172.21.15.83:0/2444810677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 0x7fa3f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40e59c700 1 --2- 172.21.15.83:0/2444810677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f8003a00 0x7fa3f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40dd9b700 1 --2- 172.21.15.83:0/2444810677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f8004850 0x7fa3f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40ed9d700 1 -- 172.21.15.83:0/2444810677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f8004850 msgr2=0x7fa3f8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40ed9d700 1 --2- 172.21.15.83:0/2444810677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3f8004850 0x7fa3f8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40ed9d700 1 -- 172.21.15.83:0/2444810677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f8003a00 msgr2=0x7fa3f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40ed9d700 1 --2- 172.21.15.83:0/2444810677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3f8003a00 0x7fa3f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.375+0000 7fa40ed9d700 1 -- 172.21.15.83:0/2444810677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f80035b0 con 0x7fa3f8005210 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa40ed9d700 1 -- 172.21.15.83:0/2444810677 reap_dead start 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa40ed9d700 1 --2- 172.21.15.83:0/2444810677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 0x7fa3f800ba10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa40800a830 tx=0x7fa3f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35887defd1e6f196 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800a2f0 con 0x7fa3f8005210 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa408012620 con 0x7fa3f8005210 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa40d59a700 1 -- 172.21.15.83:0/2444810677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080129d0 con 0x7fa3f8005210 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa414a35700 1 -- 172.21.15.83:0/2444810677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 msgr2=0x7fa3f800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa414a35700 1 --2- 172.21.15.83:0/2444810677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 0x7fa3f800ba10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa40800a830 tx=0x7fa3f8005880 comp rx=0 tx=0).stop 2023-11-26T07:12:11.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa414a35700 1 -- 172.21.15.83:0/2444810677 shutdown_connections 2023-11-26T07:12:11.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa414a35700 1 --2- 172.21.15.83:0/2444810677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 0x7fa3f800ba10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa414a35700 1 -- 172.21.15.83:0/2444810677 >> 172.21.15.83:0/2444810677 conn(0x7fa4100731a0 msgr2=0x7fa41009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:11.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa414a35700 1 -- 172.21.15.83:0/2444810677 shutdown_connections 2023-11-26T07:12:11.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.376+0000 7fa414a35700 1 -- 172.21.15.83:0/2444810677 wait complete. 2023-11-26T07:12:11.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.378+0000 7fa414a35700 1 Processor -- start 2023-11-26T07:12:11.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.378+0000 7fa414a35700 1 -- start start 2023-11-26T07:12:11.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.378+0000 7fa414a35700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 0x7fa4100c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.378+0000 7fa414a35700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4100c6390 0x7fa4100cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.378+0000 7fa414a35700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4101567c0 0x7fa410156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.378+0000 7fa414a35700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4100c9fb0 con 0x7fa3f8005210 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.378+0000 7fa414a35700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4100ca570 con 0x7fa4100c6390 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.378+0000 7fa414a35700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa410157180 con 0x7fa4101567c0 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 0x7fa4100c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 0x7fa4100c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35706/0 (socket says 172.21.15.83:35706) 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40e59c700 1 -- 172.21.15.83:0/1646712759 learned_addr learned my addr 172.21.15.83:0/1646712759 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40dd9b700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4100c6390 0x7fa4100cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40ed9d700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4101567c0 0x7fa410156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40e59c700 1 -- 172.21.15.83:0/1646712759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4101567c0 msgr2=0x7fa410156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40e59c700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4101567c0 0x7fa410156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40e59c700 1 -- 172.21.15.83:0/1646712759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4100c6390 msgr2=0x7fa4100cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40e59c700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4100c6390 0x7fa4100cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40e59c700 1 -- 172.21.15.83:0/1646712759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4100c6810 con 0x7fa3f8005210 2023-11-26T07:12:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40dd9b700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4100c6390 0x7fa4100cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa40e59c700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 0x7fa4100c57f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa4000096e0 tx=0x7fa400009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa3feffd700 1 -- 172.21.15.83:0/1646712759 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400015880 con 0x7fa3f8005210 2023-11-26T07:12:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa3feffd700 1 -- 172.21.15.83:0/1646712759 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa400015a00 con 0x7fa3f8005210 2023-11-26T07:12:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa414a35700 1 -- 172.21.15.83:0/1646712759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4100c6aa0 con 0x7fa3f8005210 2023-11-26T07:12:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa3feffd700 1 -- 172.21.15.83:0/1646712759 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400015db0 con 0x7fa3f8005210 2023-11-26T07:12:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.379+0000 7fa414a35700 1 -- 172.21.15.83:0/1646712759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4100ca200 con 0x7fa3f8005210 2023-11-26T07:12:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.380+0000 7fa414a35700 1 -- 172.21.15.83:0/1646712759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4100ca850 con 0x7fa3f8005210 2023-11-26T07:12:11.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.380+0000 7fa3feffd700 1 -- 172.21.15.83:0/1646712759 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa4000143d0 con 0x7fa3f8005210 2023-11-26T07:12:11.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.381+0000 7fa3feffd700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3f006af00 0x7fa3f006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.381+0000 7fa3feffd700 1 -- 172.21.15.83:0/1646712759 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa400089af0 con 0x7fa3f8005210 2023-11-26T07:12:11.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.381+0000 7fa40dd9b700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3f006af00 0x7fa3f006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.381+0000 7fa40dd9b700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3f006af00 0x7fa3f006d3c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa410070b00 tx=0x7fa404009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:11.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.385+0000 7fa3feffd700 1 -- 172.21.15.83:0/1646712759 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa400054a90 con 0x7fa3f8005210 2023-11-26T07:12:11.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.576+0000 7fa414a35700 1 -- 172.21.15.83:0/1646712759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa410067f60 con 0x7fa3f8005210 2023-11-26T07:12:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.577+0000 7fa3feffd700 1 -- 172.21.15.83:0/1646712759 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7fa4000580b0 con 0x7fa3f8005210 2023-11-26T07:12:11.579 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:11.579 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:09.712121+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:11:25.635076+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:11:26.877047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":1}]},{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.579+0000 7fa414a35700 1 -- 172.21.15.83:0/1646712759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3f006af00 msgr2=0x7fa3f006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.579+0000 7fa414a35700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3f006af00 0x7fa3f006d3c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa410070b00 tx=0x7fa404009040 comp rx=0 tx=0).stop 2023-11-26T07:12:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.579+0000 7fa414a35700 1 -- 172.21.15.83:0/1646712759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 msgr2=0x7fa4100c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.579+0000 7fa414a35700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 0x7fa4100c57f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa4000096e0 tx=0x7fa400009aa0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.579+0000 7fa414a35700 1 -- 172.21.15.83:0/1646712759 shutdown_connections 2023-11-26T07:12:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.579+0000 7fa414a35700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3f006af00 0x7fa3f006d3c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.579+0000 7fa414a35700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4101567c0 0x7fa410156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.579+0000 7fa414a35700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3f8005210 0x7fa4100c57f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.579+0000 7fa414a35700 1 --2- 172.21.15.83:0/1646712759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4100c6390 0x7fa4100cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.579+0000 7fa414a35700 1 -- 172.21.15.83:0/1646712759 >> 172.21.15.83:0/1646712759 conn(0x7fa4100731a0 msgr2=0x7fa41009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.580+0000 7fa414a35700 1 -- 172.21.15.83:0/1646712759 shutdown_connections 2023-11-26T07:12:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.580+0000 7fa414a35700 1 -- 172.21.15.83:0/1646712759 wait complete. 2023-11-26T07:12:11.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:12:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.713+0000 7fecd9727700 1 Processor -- start 2023-11-26T07:12:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.713+0000 7fecd9727700 1 -- start start 2023-11-26T07:12:11.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd9727700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecd415e1e0 0x7fecd40d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd9727700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecd40d3a00 0x7fecd40d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd9727700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd40d20b0 con 0x7fecd40d0db0 2023-11-26T07:12:11.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd9727700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd40d3150 con 0x7fecd40d3a00 2023-11-26T07:12:11.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd9727700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40d1240 con 0x7fecd415e1e0 2023-11-26T07:12:11.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd27fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecd40d3a00 0x7fecd40d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd27fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecd40d3a00 0x7fecd40d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35716/0 (socket says 172.21.15.83:35716) 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd27fc700 1 -- 172.21.15.83:0/4061838832 learned_addr learned my addr 172.21.15.83:0/4061838832 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd27fc700 1 -- 172.21.15.83:0/4061838832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecd415e1e0 msgr2=0x7fecd40d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd2ffd700 1 --2- 172.21.15.83:0/4061838832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecd415e1e0 0x7fecd40d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd27fc700 1 --2- 172.21.15.83:0/4061838832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecd415e1e0 0x7fecd40d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd27fc700 1 -- 172.21.15.83:0/4061838832 >> v1:172.21.15.203:6789/0 conn(0x7fecd40d0db0 legacy=0x7fecd406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd27fc700 1 -- 172.21.15.83:0/4061838832 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40d2390 con 0x7fecd40d3a00 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.714+0000 7fecd27fc700 1 --2- 172.21.15.83:0/4061838832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecd40d3a00 0x7fecd40d0870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fecc40098f0 tx=0x7fecc4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31700f50223b1322 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.715+0000 7fecd1ffb700 1 -- 172.21.15.83:0/4061838832 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4004230 con 0x7fecd40d3a00 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.715+0000 7fecd1ffb700 1 -- 172.21.15.83:0/4061838832 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc4004390 con 0x7fecd40d3a00 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.715+0000 7fecd1ffb700 1 -- 172.21.15.83:0/4061838832 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc40036a0 con 0x7fecd40d3a00 2023-11-26T07:12:11.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.715+0000 7fecd9727700 1 -- 172.21.15.83:0/4061838832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecd40d3a00 msgr2=0x7fecd40d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.715+0000 7fecd9727700 1 --2- 172.21.15.83:0/4061838832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecd40d3a00 0x7fecd40d0870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fecc40098f0 tx=0x7fecc4009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:11.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.715+0000 7fecd9727700 1 -- 172.21.15.83:0/4061838832 shutdown_connections 2023-11-26T07:12:11.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.715+0000 7fecd9727700 1 --2- 172.21.15.83:0/4061838832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecd415e1e0 0x7fecd40d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.715+0000 7fecd9727700 1 --2- 172.21.15.83:0/4061838832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecd40d3a00 0x7fecd40d0870 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.715+0000 7fecd9727700 1 -- 172.21.15.83:0/4061838832 >> 172.21.15.83:0/4061838832 conn(0x7fecd40b5b50 msgr2=0x7fecd40b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:11.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.715+0000 7fecd9727700 1 -- 172.21.15.83:0/4061838832 shutdown_connections 2023-11-26T07:12:11.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.715+0000 7fecd9727700 1 -- 172.21.15.83:0/4061838832 wait complete. 2023-11-26T07:12:11.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.716+0000 7fecd9727700 1 Processor -- start 2023-11-26T07:12:11.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.717+0000 7fecd9727700 1 -- start start 2023-11-26T07:12:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.717+0000 7fecd9727700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecd40d0db0 0x7fecd4153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.717+0000 7fecd9727700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecd40d3a00 0x7fecd4156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.717+0000 7fecd9727700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd415e1e0 0x7fecd4154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.717+0000 7fecd9727700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40caf30 con 0x7fecd40d0db0 2023-11-26T07:12:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.717+0000 7fecd9727700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4155320 con 0x7fecd415e1e0 2023-11-26T07:12:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.717+0000 7fecd9727700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd4156e30 con 0x7fecd40d3a00 2023-11-26T07:12:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.717+0000 7fecd2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecd40d0db0 0x7fecd4153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.717+0000 7fecd2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecd40d0db0 0x7fecd4153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35720/0 (socket says 172.21.15.83:35720) 2023-11-26T07:12:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.717+0000 7fecd2ffd700 1 -- 172.21.15.83:0/2198116183 learned_addr learned my addr 172.21.15.83:0/2198116183 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecd27fc700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecd40d3a00 0x7fecd4156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecd37fe700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd415e1e0 0x7fecd4154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecd2ffd700 1 -- 172.21.15.83:0/2198116183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecd40d3a00 msgr2=0x7fecd4156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecd2ffd700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecd40d3a00 0x7fecd4156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecd2ffd700 1 -- 172.21.15.83:0/2198116183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd415e1e0 msgr2=0x7fecd4154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecd2ffd700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd415e1e0 0x7fecd4154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecd2ffd700 1 -- 172.21.15.83:0/2198116183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc4009550 con 0x7fecd40d0db0 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecd37fe700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd415e1e0 0x7fecd4154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecd2ffd700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecd40d0db0 0x7fecd4153ca0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fecc800c7f0 tx=0x7fecc800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecbb7fe700 1 -- 172.21.15.83:0/2198116183 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8011800 con 0x7fecd40d0db0 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecbb7fe700 1 -- 172.21.15.83:0/2198116183 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc8011960 con 0x7fecd40d0db0 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecbb7fe700 1 -- 172.21.15.83:0/2198116183 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8011cf0 con 0x7fecd40d0db0 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecd9727700 1 -- 172.21.15.83:0/2198116183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd40cb180 con 0x7fecd40d0db0 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.718+0000 7fecd9727700 1 -- 172.21.15.83:0/2198116183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd40cb740 con 0x7fecd40d0db0 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.719+0000 7fecd9727700 1 -- 172.21.15.83:0/2198116183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd40751e0 con 0x7fecd40d0db0 2023-11-26T07:12:11.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.719+0000 7fecbb7fe700 1 -- 172.21.15.83:0/2198116183 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fecc8018360 con 0x7fecd40d0db0 2023-11-26T07:12:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.720+0000 7fecbb7fe700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fecc006aed0 0x7fecc006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.720+0000 7fecbb7fe700 1 -- 172.21.15.83:0/2198116183 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fecc8089570 con 0x7fecd40d0db0 2023-11-26T07:12:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.720+0000 7fecd27fc700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fecc006aed0 0x7fecc006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:11.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.720+0000 7fecd27fc700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fecc006aed0 0x7fecc006d390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009130 tx=0x7fecc4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:11.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.724+0000 7fecbb7fe700 1 -- 172.21.15.83:0/2198116183 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fecc80545a0 con 0x7fecd40d0db0 2023-11-26T07:12:11.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.960+0000 7fecd9727700 1 -- 172.21.15.83:0/2198116183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fecd40cb930 con 0x7fecd40d0db0 2023-11-26T07:12:11.962 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-26T07:12:11.961+0000 7fcd25823700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:12:11.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.962+0000 7fecbb7fe700 1 -- 172.21.15.83:0/2198116183 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fecc8057bc0 con 0x7fecd40d0db0 2023-11-26T07:12:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.965+0000 7fecd9727700 1 -- 172.21.15.83:0/2198116183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fecc006aed0 msgr2=0x7fecc006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.965+0000 7fecd9727700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fecc006aed0 0x7fecc006d390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009130 tx=0x7fecc4008030 comp rx=0 tx=0).stop 2023-11-26T07:12:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.965+0000 7fecd9727700 1 -- 172.21.15.83:0/2198116183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecd40d0db0 msgr2=0x7fecd4153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.965+0000 7fecd9727700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecd40d0db0 0x7fecd4153ca0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fecc800c7f0 tx=0x7fecc800cbb0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.965+0000 7fecd9727700 1 -- 172.21.15.83:0/2198116183 shutdown_connections 2023-11-26T07:12:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.965+0000 7fecd9727700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd415e1e0 0x7fecd4154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.965+0000 7fecd9727700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecd40d0db0 0x7fecd4153ca0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.965+0000 7fecd9727700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fecc006aed0 0x7fecc006d390 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.966+0000 7fecd9727700 1 --2- 172.21.15.83:0/2198116183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecd40d3a00 0x7fecd4156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.966+0000 7fecd9727700 1 -- 172.21.15.83:0/2198116183 >> 172.21.15.83:0/2198116183 conn(0x7fecd40b5b50 msgr2=0x7fecd40b6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.966+0000 7fecd9727700 1 -- 172.21.15.83:0/2198116183 shutdown_connections 2023-11-26T07:12:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:11.966+0000 7fecd9727700 1 -- 172.21.15.83:0/2198116183 wait complete. 2023-11-26T07:12:11.968 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:12:11.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:12:12.011 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-11-26T07:12:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.100+0000 7f29825ae700 1 Processor -- start 2023-11-26T07:12:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.100+0000 7f29825ae700 1 -- start start 2023-11-26T07:12:12.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f29825ae700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f297c0746a0 0x7f297c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:12.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f29825ae700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c0b3c60 con 0x7f297c15e160 2023-11-26T07:12:12.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f29825ae700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c0b58c0 con 0x7f297c0afc90 2023-11-26T07:12:12.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f29825ae700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c0b4500 con 0x7f297c0746a0 2023-11-26T07:12:12.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f297b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f297c0746a0 0x7f297c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:12.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f297bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f297c15e160 0x7f297c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55768/0 (socket says 172.21.15.83:55768) 2023-11-26T07:12:12.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f297b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f297c0746a0 0x7f297c0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35730/0 (socket says 172.21.15.83:35730) 2023-11-26T07:12:12.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f297bfff700 1 -- 172.21.15.83:0/450161687 learned_addr learned my addr 172.21.15.83:0/450161687 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f297b7fe700 1 -- 172.21.15.83:0/450161687 >> v1:172.21.15.203:6789/0 conn(0x7f297c0afc90 legacy=0x7f297c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f297b7fe700 1 -- 172.21.15.83:0/450161687 >> v1:172.21.15.83:6789/0 conn(0x7f297c15e160 legacy=0x7f297c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f297affd700 1 -- 172.21.15.83:0/450161687 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908164047 0 0) 0x7f297c0b3c60 con 0x7f297c15e160 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.101+0000 7f297b7fe700 1 -- 172.21.15.83:0/450161687 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c06bdc0 con 0x7f297c0746a0 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.102+0000 7f297b7fe700 1 --2- 172.21.15.83:0/450161687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c0746a0 0x7f297c0b5da0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f296c009840 tx=0x7f296c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9565fa3f5e8b39a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.102+0000 7f297affd700 1 -- 172.21.15.83:0/450161687 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c011830 con 0x7f297c0746a0 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.102+0000 7f297affd700 1 -- 172.21.15.83:0/450161687 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c011990 con 0x7f297c0746a0 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.102+0000 7f297affd700 1 -- 172.21.15.83:0/450161687 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c011d20 con 0x7f297c0746a0 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.102+0000 7f29825ae700 1 -- 172.21.15.83:0/450161687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c0746a0 msgr2=0x7f297c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.102+0000 7f29825ae700 1 --2- 172.21.15.83:0/450161687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c0746a0 0x7f297c0b5da0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f296c009840 tx=0x7f296c009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.102+0000 7f29825ae700 1 -- 172.21.15.83:0/450161687 shutdown_connections 2023-11-26T07:12:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.102+0000 7f29825ae700 1 --2- 172.21.15.83:0/450161687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c0746a0 0x7f297c0b5da0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:12.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.102+0000 7f29825ae700 1 -- 172.21.15.83:0/450161687 >> 172.21.15.83:0/450161687 conn(0x7f297c0d0bf0 msgr2=0x7f297c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:12.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.102+0000 7f29825ae700 1 -- 172.21.15.83:0/450161687 shutdown_connections 2023-11-26T07:12:12.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.102+0000 7f29825ae700 1 -- 172.21.15.83:0/450161687 wait complete. 2023-11-26T07:12:12.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.104+0000 7f29825ae700 1 Processor -- start 2023-11-26T07:12:12.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.104+0000 7f29825ae700 1 -- start start 2023-11-26T07:12:12.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f29825ae700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0746a0 0x7f297c152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:12.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f29825ae700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c0afc90 0x7f297c155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:12.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f29825ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c15e160 0x7f297c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:12.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f29825ae700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c1600e0 con 0x7f297c0afc90 2023-11-26T07:12:12.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f29825ae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c06ef90 con 0x7f297c15e160 2023-11-26T07:12:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f29825ae700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c15f600 con 0x7f297c0746a0 2023-11-26T07:12:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f297b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c0afc90 0x7f297c155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f297b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c0afc90 0x7f297c155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35734/0 (socket says 172.21.15.83:35734) 2023-11-26T07:12:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f297b7fe700 1 -- 172.21.15.83:0/4272293050 learned_addr learned my addr 172.21.15.83:0/4272293050 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f2980b4b700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c15e160 0x7f297c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f297bfff700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0746a0 0x7f297c152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f297b7fe700 1 -- 172.21.15.83:0/4272293050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0746a0 msgr2=0x7f297c152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f297b7fe700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0746a0 0x7f297c152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:12.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f297b7fe700 1 -- 172.21.15.83:0/4272293050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c15e160 msgr2=0x7f297c15c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f297b7fe700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c15e160 0x7f297c15c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f297b7fe700 1 -- 172.21.15.83:0/4272293050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0094a0 con 0x7f297c0afc90 2023-11-26T07:12:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.105+0000 7f2980b4b700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c15e160 0x7f297c15c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.106+0000 7f297b7fe700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c0afc90 0x7f297c155ae0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f296c0090b0 tx=0x7f296c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.106+0000 7f2978ff9700 1 -- 172.21.15.83:0/4272293050 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0036a0 con 0x7f297c0afc90 2023-11-26T07:12:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.106+0000 7f2978ff9700 1 -- 172.21.15.83:0/4272293050 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c01f070 con 0x7f297c0afc90 2023-11-26T07:12:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.106+0000 7f2978ff9700 1 -- 172.21.15.83:0/4272293050 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c003a00 con 0x7f297c0afc90 2023-11-26T07:12:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.106+0000 7f29825ae700 1 -- 172.21.15.83:0/4272293050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c15f8e0 con 0x7f297c0afc90 2023-11-26T07:12:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.106+0000 7f29825ae700 1 -- 172.21.15.83:0/4272293050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c156020 con 0x7f297c0afc90 2023-11-26T07:12:12.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.107+0000 7f29825ae700 1 -- 172.21.15.83:0/4272293050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c068fb0 con 0x7f297c0afc90 2023-11-26T07:12:12.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.107+0000 7f2978ff9700 1 -- 172.21.15.83:0/4272293050 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f296c003b60 con 0x7f297c0afc90 2023-11-26T07:12:12.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.107+0000 7f2978ff9700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806aed0 0x7f296806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:12.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.108+0000 7f2978ff9700 1 -- 172.21.15.83:0/4272293050 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f296c021070 con 0x7f297c0afc90 2023-11-26T07:12:12.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.108+0000 7f297bfff700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806aed0 0x7f296806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:12.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.108+0000 7f297bfff700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806aed0 0x7f296806d390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2970000f60 tx=0x7f2970007d90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:12.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.112+0000 7f2978ff9700 1 -- 172.21.15.83:0/4272293050 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f296c056130 con 0x7f297c0afc90 2023-11-26T07:12:12.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.348+0000 7f29825ae700 1 -- 172.21.15.83:0/4272293050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f297c06c140 con 0x7f297c0afc90 2023-11-26T07:12:12.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.967+0000 7f2978ff9700 1 -- 172.21.15.83:0/4272293050 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f296c059750 con 0x7f297c0afc90 2023-11-26T07:12:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 -- 172.21.15.83:0/4272293050 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806aed0 msgr2=0x7f296806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806aed0 0x7f296806d390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2970000f60 tx=0x7f2970007d90 comp rx=0 tx=0).stop 2023-11-26T07:12:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 -- 172.21.15.83:0/4272293050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c0afc90 msgr2=0x7f297c155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c0afc90 0x7f297c155ae0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f296c0090b0 tx=0x7f296c004750 comp rx=0 tx=0).stop 2023-11-26T07:12:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 -- 172.21.15.83:0/4272293050 shutdown_connections 2023-11-26T07:12:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c15e160 0x7f297c15c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806aed0 0x7f296806d390 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0746a0 0x7f297c152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 --2- 172.21.15.83:0/4272293050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c0afc90 0x7f297c155ae0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 -- 172.21.15.83:0/4272293050 >> 172.21.15.83:0/4272293050 conn(0x7f297c0d0bf0 msgr2=0x7f297c0aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:12.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 -- 172.21.15.83:0/4272293050 shutdown_connections 2023-11-26T07:12:12.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:12.970+0000 7f29825ae700 1 -- 172.21.15.83:0/4272293050 wait complete. 2023-11-26T07:12:12.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:12:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.115+0000 7f3e82ba4700 1 Processor -- start 2023-11-26T07:12:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.115+0000 7f3e82ba4700 1 -- start start 2023-11-26T07:12:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e82ba4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3e7c0b01e0 0x7f3e7c06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e82ba4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e7c0b59c0 0x7f3e7c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e82ba4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e7c073f90 0x7f3e7c0743e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e82ba4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c1470a0 con 0x7f3e7c073f90 2023-11-26T07:12:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e82ba4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c074920 con 0x7f3e7c0b01e0 2023-11-26T07:12:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e82ba4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0b5e40 con 0x7f3e7c0b59c0 2023-11-26T07:12:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e81141700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e7c073f90 0x7f3e7c0743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e81141700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e7c073f90 0x7f3e7c0743e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35742/0 (socket says 172.21.15.83:35742) 2023-11-26T07:12:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e81141700 1 -- 172.21.15.83:0/3223634578 learned_addr learned my addr 172.21.15.83:0/3223634578 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e81141700 1 -- 172.21.15.83:0/3223634578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e7c0b59c0 msgr2=0x7f3e7c0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e81141700 1 --2- 172.21.15.83:0/3223634578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e7c0b59c0 0x7f3e7c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e80940700 1 --2- 172.21.15.83:0/3223634578 >> v2:172.21.15.203:3300/0 conn(0x7f3e7c0b01e0 0x7f3e7c06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e81141700 1 -- 172.21.15.83:0/3223634578 >> v2:172.21.15.203:3300/0 conn(0x7f3e7c0b01e0 msgr2=0x7f3e7c06e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e81141700 1 --2- 172.21.15.83:0/3223634578 >> v2:172.21.15.203:3300/0 conn(0x7f3e7c0b01e0 0x7f3e7c06e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e81141700 1 -- 172.21.15.83:0/3223634578 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c0b05c0 con 0x7f3e7c073f90 2023-11-26T07:12:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e81141700 1 --2- 172.21.15.83:0/3223634578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7c073f90 0x7f3e7c0743e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c072ce0 tx=0x7f3e7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afa281c4ed791a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e7b7fe700 1 -- 172.21.15.83:0/3223634578 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7400e070 con 0x7f3e7c073f90 2023-11-26T07:12:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.116+0000 7f3e7b7fe700 1 -- 172.21.15.83:0/3223634578 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7400b9c0 con 0x7f3e7c073f90 2023-11-26T07:12:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.117+0000 7f3e7b7fe700 1 -- 172.21.15.83:0/3223634578 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7400bd70 con 0x7f3e7c073f90 2023-11-26T07:12:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.117+0000 7f3e82ba4700 1 -- 172.21.15.83:0/3223634578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7c073f90 msgr2=0x7f3e7c0743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.117+0000 7f3e82ba4700 1 --2- 172.21.15.83:0/3223634578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7c073f90 0x7f3e7c0743e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c072ce0 tx=0x7f3e7400b330 comp rx=0 tx=0).stop 2023-11-26T07:12:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.117+0000 7f3e82ba4700 1 -- 172.21.15.83:0/3223634578 shutdown_connections 2023-11-26T07:12:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.117+0000 7f3e82ba4700 1 --2- 172.21.15.83:0/3223634578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7c073f90 0x7f3e7c0743e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.117+0000 7f3e82ba4700 1 --2- 172.21.15.83:0/3223634578 >> v2:172.21.15.203:3300/0 conn(0x7f3e7c0b01e0 0x7f3e7c06e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.117+0000 7f3e82ba4700 1 --2- 172.21.15.83:0/3223634578 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e7c0b59c0 0x7f3e7c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.117+0000 7f3e82ba4700 1 -- 172.21.15.83:0/3223634578 >> 172.21.15.83:0/3223634578 conn(0x7f3e7c0d0b40 msgr2=0x7f3e7c0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.117+0000 7f3e82ba4700 1 -- 172.21.15.83:0/3223634578 shutdown_connections 2023-11-26T07:12:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.117+0000 7f3e82ba4700 1 -- 172.21.15.83:0/3223634578 wait complete. 2023-11-26T07:12:13.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.118+0000 7f3e82ba4700 1 Processor -- start 2023-11-26T07:12:13.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.118+0000 7f3e82ba4700 1 -- start start 2023-11-26T07:12:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.119+0000 7f3e82ba4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7c073f90 0x7f3e7c1f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.119+0000 7f3e82ba4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e7c0b01e0 0x7f3e7c0c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.119+0000 7f3e82ba4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e7c0b59c0 0x7f3e7c13e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.119+0000 7f3e82ba4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c1f5770 con 0x7f3e7c073f90 2023-11-26T07:12:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.119+0000 7f3e82ba4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c1f4bf0 con 0x7f3e7c0b59c0 2023-11-26T07:12:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.119+0000 7f3e82ba4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0c66f0 con 0x7f3e7c0b01e0 2023-11-26T07:12:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.119+0000 7f3e80940700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7c073f90 0x7f3e7c1f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.119+0000 7f3e80940700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7c073f90 0x7f3e7c1f46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35746/0 (socket says 172.21.15.83:35746) 2023-11-26T07:12:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.119+0000 7f3e80940700 1 -- 172.21.15.83:0/447631432 learned_addr learned my addr 172.21.15.83:0/447631432 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.119+0000 7f3e80940700 1 -- 172.21.15.83:0/447631432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e7c0b01e0 msgr2=0x7f3e7c0c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.119+0000 7f3e80940700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e7c0b01e0 0x7f3e7c0c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.120+0000 7f3e80940700 1 -- 172.21.15.83:0/447631432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e7c0b59c0 msgr2=0x7f3e7c13e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.120+0000 7f3e80940700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e7c0b59c0 0x7f3e7c13e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.120+0000 7f3e80940700 1 -- 172.21.15.83:0/447631432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74009b30 con 0x7f3e7c073f90 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.120+0000 7f3e80940700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7c073f90 0x7f3e7c1f46b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000c7f0 tx=0x7f3e7000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.120+0000 7f3e797fa700 1 -- 172.21.15.83:0/447631432 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70011800 con 0x7f3e7c073f90 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.120+0000 7f3e797fa700 1 -- 172.21.15.83:0/447631432 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e70011960 con 0x7f3e7c073f90 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.120+0000 7f3e82ba4700 1 -- 172.21.15.83:0/447631432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c1f4dd0 con 0x7f3e7c073f90 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.120+0000 7f3e797fa700 1 -- 172.21.15.83:0/447631432 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70011cf0 con 0x7f3e7c073f90 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.120+0000 7f3e82ba4700 1 -- 172.21.15.83:0/447631432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c1f5390 con 0x7f3e7c073f90 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.121+0000 7f3e82ba4700 1 -- 172.21.15.83:0/447631432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c067f60 con 0x7f3e7c073f90 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.121+0000 7f3e797fa700 1 -- 172.21.15.83:0/447631432 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e70018360 con 0x7f3e7c073f90 2023-11-26T07:12:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.122+0000 7f3e797fa700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e6006aed0 0x7f3e6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:13.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.122+0000 7f3e797fa700 1 -- 172.21.15.83:0/447631432 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f3e70089690 con 0x7f3e7c073f90 2023-11-26T07:12:13.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.122+0000 7f3e7bfff700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e6006aed0 0x7f3e6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:13.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.122+0000 7f3e7bfff700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e6006aed0 0x7f3e6006d390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0c7c50 tx=0x7f3e68006c80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.126+0000 7f3e797fa700 1 -- 172.21.15.83:0/447631432 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e70054680 con 0x7f3e7c073f90 2023-11-26T07:12:13.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.315+0000 7f3e82ba4700 1 -- 172.21.15.83:0/447631432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3e7c06c980 con 0x7f3e7c073f90 2023-11-26T07:12:13.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.969+0000 7f3e797fa700 1 -- 172.21.15.83:0/447631432 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3e70057ca0 con 0x7f3e7c073f90 2023-11-26T07:12:13.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.972+0000 7f3e82ba4700 1 -- 172.21.15.83:0/447631432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e6006aed0 msgr2=0x7f3e6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:13.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.972+0000 7f3e82ba4700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e6006aed0 0x7f3e6006d390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0c7c50 tx=0x7f3e68006c80 comp rx=0 tx=0).stop 2023-11-26T07:12:13.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.972+0000 7f3e82ba4700 1 -- 172.21.15.83:0/447631432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7c073f90 msgr2=0x7f3e7c1f46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:13.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.972+0000 7f3e82ba4700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7c073f90 0x7f3e7c1f46b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000c7f0 tx=0x7f3e7000cbb0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.973+0000 7f3e82ba4700 1 -- 172.21.15.83:0/447631432 shutdown_connections 2023-11-26T07:12:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.973+0000 7f3e82ba4700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e7c0b59c0 0x7f3e7c13e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.973+0000 7f3e82ba4700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7c073f90 0x7f3e7c1f46b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.973+0000 7f3e82ba4700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e6006aed0 0x7f3e6006d390 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.973+0000 7f3e82ba4700 1 --2- 172.21.15.83:0/447631432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e7c0b01e0 0x7f3e7c0c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.973+0000 7f3e82ba4700 1 -- 172.21.15.83:0/447631432 >> 172.21.15.83:0/447631432 conn(0x7f3e7c0d0b40 msgr2=0x7f3e7c0b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.973+0000 7f3e82ba4700 1 -- 172.21.15.83:0/447631432 shutdown_connections 2023-11-26T07:12:13.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:13.973+0000 7f3e82ba4700 1 -- 172.21.15.83:0/447631432 wait complete. 2023-11-26T07:12:13.976 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:12:13.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:12:14.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.116+0000 7faf3d5c7700 1 Processor -- start 2023-11-26T07:12:14.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.116+0000 7faf3d5c7700 1 -- start start 2023-11-26T07:12:14.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.117+0000 7faf3d5c7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faf380b01e0 0x7faf3806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:14.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.117+0000 7faf3d5c7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf380b59c0 0x7faf380b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.117+0000 7faf3d5c7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf38073f90 0x7faf380743e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.117+0000 7faf3d5c7700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf381470a0 con 0x7faf38073f90 2023-11-26T07:12:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.117+0000 7faf3d5c7700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf38074920 con 0x7faf380b01e0 2023-11-26T07:12:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.117+0000 7faf3d5c7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf380b5e40 con 0x7faf380b59c0 2023-11-26T07:12:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.117+0000 7faf377fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf38073f90 0x7faf380743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.117+0000 7faf377fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf38073f90 0x7faf380743e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35758/0 (socket says 172.21.15.83:35758) 2023-11-26T07:12:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.117+0000 7faf377fe700 1 -- 172.21.15.83:0/1965917474 learned_addr learned my addr 172.21.15.83:0/1965917474 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.118+0000 7faf377fe700 1 -- 172.21.15.83:0/1965917474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf380b59c0 msgr2=0x7faf380b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.118+0000 7faf36ffd700 1 --2- 172.21.15.83:0/1965917474 >> v2:172.21.15.203:3300/0 conn(0x7faf380b01e0 0x7faf3806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.118+0000 7faf367fc700 1 --2- 172.21.15.83:0/1965917474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf380b59c0 0x7faf380b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.118+0000 7faf377fe700 1 --2- 172.21.15.83:0/1965917474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf380b59c0 0x7faf380b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.118+0000 7faf377fe700 1 -- 172.21.15.83:0/1965917474 >> v2:172.21.15.203:3300/0 conn(0x7faf380b01e0 msgr2=0x7faf3806e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.118+0000 7faf377fe700 1 --2- 172.21.15.83:0/1965917474 >> v2:172.21.15.203:3300/0 conn(0x7faf380b01e0 0x7faf3806e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.118+0000 7faf377fe700 1 -- 172.21.15.83:0/1965917474 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380b05c0 con 0x7faf38073f90 2023-11-26T07:12:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.118+0000 7faf377fe700 1 --2- 172.21.15.83:0/1965917474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf38073f90 0x7faf380743e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faf38072ce0 tx=0x7faf3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbccbd614e4d162 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.118+0000 7faf35ffb700 1 -- 172.21.15.83:0/1965917474 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3000e070 con 0x7faf38073f90 2023-11-26T07:12:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.118+0000 7faf35ffb700 1 -- 172.21.15.83:0/1965917474 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3000b9c0 con 0x7faf38073f90 2023-11-26T07:12:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.118+0000 7faf35ffb700 1 -- 172.21.15.83:0/1965917474 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3000bd70 con 0x7faf38073f90 2023-11-26T07:12:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.119+0000 7faf3d5c7700 1 -- 172.21.15.83:0/1965917474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf38073f90 msgr2=0x7faf380743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.119+0000 7faf3d5c7700 1 --2- 172.21.15.83:0/1965917474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf38073f90 0x7faf380743e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faf38072ce0 tx=0x7faf3000b330 comp rx=0 tx=0).stop 2023-11-26T07:12:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.119+0000 7faf3d5c7700 1 -- 172.21.15.83:0/1965917474 shutdown_connections 2023-11-26T07:12:14.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.119+0000 7faf3d5c7700 1 --2- 172.21.15.83:0/1965917474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf38073f90 0x7faf380743e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:14.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.119+0000 7faf3d5c7700 1 --2- 172.21.15.83:0/1965917474 >> v2:172.21.15.203:3300/0 conn(0x7faf380b01e0 0x7faf3806e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:14.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.119+0000 7faf3d5c7700 1 --2- 172.21.15.83:0/1965917474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf380b59c0 0x7faf380b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:14.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.119+0000 7faf3d5c7700 1 -- 172.21.15.83:0/1965917474 >> 172.21.15.83:0/1965917474 conn(0x7faf380d0b40 msgr2=0x7faf380d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:14.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.119+0000 7faf3d5c7700 1 -- 172.21.15.83:0/1965917474 shutdown_connections 2023-11-26T07:12:14.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.119+0000 7faf3d5c7700 1 -- 172.21.15.83:0/1965917474 wait complete. 2023-11-26T07:12:14.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.120+0000 7faf3d5c7700 1 Processor -- start 2023-11-26T07:12:14.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.120+0000 7faf3d5c7700 1 -- start start 2023-11-26T07:12:14.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf3d5c7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf38073f90 0x7faf3813ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:14.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf3d5c7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf380b01e0 0x7faf380c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:14.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf3d5c7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf380b59c0 0x7faf3813e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:14.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf3d5c7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf381409c0 con 0x7faf380b01e0 2023-11-26T07:12:14.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf3d5c7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf380c6070 con 0x7faf380b59c0 2023-11-26T07:12:14.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf3d5c7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3813f2d0 con 0x7faf38073f90 2023-11-26T07:12:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf367fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf380b01e0 0x7faf380c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf367fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf380b01e0 0x7faf380c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35762/0 (socket says 172.21.15.83:35762) 2023-11-26T07:12:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf367fc700 1 -- 172.21.15.83:0/3950092627 learned_addr learned my addr 172.21.15.83:0/3950092627 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf36ffd700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf38073f90 0x7faf3813ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf377fe700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf380b59c0 0x7faf3813e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf367fc700 1 -- 172.21.15.83:0/3950092627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf38073f90 msgr2=0x7faf3813ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf367fc700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf38073f90 0x7faf3813ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf367fc700 1 -- 172.21.15.83:0/3950092627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf380b59c0 msgr2=0x7faf3813e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf367fc700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf380b59c0 0x7faf3813e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf367fc700 1 -- 172.21.15.83:0/3950092627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf30009b30 con 0x7faf380b01e0 2023-11-26T07:12:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf377fe700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf380b59c0 0x7faf3813e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf367fc700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf380b01e0 0x7faf380c5b30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faf2800c7f0 tx=0x7faf2800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.121+0000 7faf1f7fe700 1 -- 172.21.15.83:0/3950092627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28011800 con 0x7faf380b01e0 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.122+0000 7faf1f7fe700 1 -- 172.21.15.83:0/3950092627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf28011960 con 0x7faf380b01e0 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.122+0000 7faf1f7fe700 1 -- 172.21.15.83:0/3950092627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28011cd0 con 0x7faf380b01e0 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.122+0000 7faf3d5c7700 1 -- 172.21.15.83:0/3950092627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3813f5b0 con 0x7faf380b01e0 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.122+0000 7faf3d5c7700 1 -- 172.21.15.83:0/3950092627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3813fb70 con 0x7faf380b01e0 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.122+0000 7faf3d5c7700 1 -- 172.21.15.83:0/3950092627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf38140b00 con 0x7faf380b01e0 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.123+0000 7faf1f7fe700 1 -- 172.21.15.83:0/3950092627 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf28010350 con 0x7faf380b01e0 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.123+0000 7faf1f7fe700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faf2406aed0 0x7faf2406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.123+0000 7faf1f7fe700 1 -- 172.21.15.83:0/3950092627 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5473+0+0 (secure 0 0 0) 0x7faf28088f90 con 0x7faf380b01e0 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.123+0000 7faf36ffd700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faf2406aed0 0x7faf2406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:14.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.124+0000 7faf36ffd700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faf2406aed0 0x7faf2406d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00ab60 tx=0x7faf2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:14.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.128+0000 7faf1f7fe700 1 -- 172.21.15.83:0/3950092627 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf280bbe90 con 0x7faf380b01e0 2023-11-26T07:12:14.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.322+0000 7faf3d5c7700 1 -- 172.21.15.83:0/3950092627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faf38068670 con 0x7faf380b01e0 2023-11-26T07:12:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.973+0000 7faf1f7fe700 1 -- 172.21.15.83:0/3950092627 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7faf38140b00 con 0x7faf380b01e0 2023-11-26T07:12:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.975+0000 7faf3d5c7700 1 -- 172.21.15.83:0/3950092627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faf2406aed0 msgr2=0x7faf2406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.975+0000 7faf3d5c7700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faf2406aed0 0x7faf2406d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00ab60 tx=0x7faf2c008040 comp rx=0 tx=0).stop 2023-11-26T07:12:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.976+0000 7faf3d5c7700 1 -- 172.21.15.83:0/3950092627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf380b01e0 msgr2=0x7faf380c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.976+0000 7faf3d5c7700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf380b01e0 0x7faf380c5b30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faf2800c7f0 tx=0x7faf2800cb00 comp rx=0 tx=0).stop 2023-11-26T07:12:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.976+0000 7faf3d5c7700 1 -- 172.21.15.83:0/3950092627 shutdown_connections 2023-11-26T07:12:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.976+0000 7faf3d5c7700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf380b59c0 0x7faf3813e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.976+0000 7faf3d5c7700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7faf2406aed0 0x7faf2406d390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.976+0000 7faf3d5c7700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf38073f90 0x7faf3813ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.976+0000 7faf3d5c7700 1 --2- 172.21.15.83:0/3950092627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf380b01e0 0x7faf380c5b30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.976+0000 7faf3d5c7700 1 -- 172.21.15.83:0/3950092627 >> 172.21.15.83:0/3950092627 conn(0x7faf380d0b40 msgr2=0x7faf380b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.976+0000 7faf3d5c7700 1 -- 172.21.15.83:0/3950092627 shutdown_connections 2023-11-26T07:12:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:14.976+0000 7faf3d5c7700 1 -- 172.21.15.83:0/3950092627 wait complete. 2023-11-26T07:12:14.979 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2023-11-26T07:12:14.994 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:12:14.994 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:12:14.995 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:12:14.995 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:12:14.995 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:12:14.995 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:12:14.995 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:12:14.995 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:12:14.996 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:12:14.996 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:12:14.996 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:12:14.996 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:12:14.996 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:12:14.996 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:12:14.997 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:12:14.997 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:12:15.006 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:12:15.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:12:15.007 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:12:15.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:12:15.009 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:12:15.009 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:12:15.010 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:12:15.010 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:12:15.049 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:12:15.049 DEBUG:teuthology.parallel:result is None 2023-11-26T07:12:15.049 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:12:15.050 DEBUG:teuthology.parallel:result is None 2023-11-26T07:12:15.051 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:12:15.052 DEBUG:teuthology.parallel:result is None 2023-11-26T07:12:15.052 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:12:15.052 DEBUG:teuthology.parallel:result is None 2023-11-26T07:12:15.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:12:15.131 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-26T07:12:15.132 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-26T07:12:15.132 INFO:tasks.ceph.mds.c.smithi083.stderr:2023-11-26T07:12:15.130+0000 7fa01b9a1700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-26T07:12:15.134 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-11-26T07:12:15.134 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-11-26T07:12:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.180+0000 7f6c6083c700 1 Processor -- start 2023-11-26T07:12:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.181+0000 7f6c6083c700 1 -- start start 2023-11-26T07:12:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.181+0000 7f6c6083c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c5c13e000 0x7f6c5c15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.181+0000 7f6c6083c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c15efb0 0x7f6c5c0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.181+0000 7f6c6083c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5c0b4290 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.181+0000 7f6c6083c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5c15f430 con 0x7f6c5c13e000 2023-11-26T07:12:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c6083c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c13e570 con 0x7f6c5c15efb0 2023-11-26T07:12:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c5a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c5c13e000 0x7f6c5c15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c5a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c5c13e000 0x7f6c5c15ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35790/0 (socket says 172.21.15.83:35790) 2023-11-26T07:12:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c5a59c700 1 -- 172.21.15.83:0/3403940031 learned_addr learned my addr 172.21.15.83:0/3403940031 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c59d9b700 1 --2- 172.21.15.83:0/3403940031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c15efb0 0x7f6c5c0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c5a59c700 1 -- 172.21.15.83:0/3403940031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c15efb0 msgr2=0x7f6c5c0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c5a59c700 1 --2- 172.21.15.83:0/3403940031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c15efb0 0x7f6c5c0cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c5a59c700 1 -- 172.21.15.83:0/3403940031 >> v1:172.21.15.203:6789/0 conn(0x7f6c5c0cfd40 legacy=0x7f6c5c15b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c5a59c700 1 -- 172.21.15.83:0/3403940031 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c0d05d0 con 0x7f6c5c13e000 2023-11-26T07:12:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c5a59c700 1 --2- 172.21.15.83:0/3403940031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c5c13e000 0x7f6c5c15ea70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009e50 tx=0x7f6c4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89cbff0f38c12c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c5959a700 1 -- 172.21.15.83:0/3403940031 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00e070 con 0x7f6c5c13e000 2023-11-26T07:12:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c5959a700 1 -- 172.21.15.83:0/3403940031 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c00b910 con 0x7f6c5c13e000 2023-11-26T07:12:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.182+0000 7f6c5959a700 1 -- 172.21.15.83:0/3403940031 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00bcc0 con 0x7f6c5c13e000 2023-11-26T07:12:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.183+0000 7f6c6083c700 1 -- 172.21.15.83:0/3403940031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c5c13e000 msgr2=0x7f6c5c15ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.183+0000 7f6c6083c700 1 --2- 172.21.15.83:0/3403940031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c5c13e000 0x7f6c5c15ea70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009e50 tx=0x7f6c4c00b330 comp rx=0 tx=0).stop 2023-11-26T07:12:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.183+0000 7f6c6083c700 1 -- 172.21.15.83:0/3403940031 shutdown_connections 2023-11-26T07:12:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.183+0000 7f6c6083c700 1 --2- 172.21.15.83:0/3403940031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c5c13e000 0x7f6c5c15ea70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.183+0000 7f6c6083c700 1 --2- 172.21.15.83:0/3403940031 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c15efb0 0x7f6c5c0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.183+0000 7f6c6083c700 1 -- 172.21.15.83:0/3403940031 >> 172.21.15.83:0/3403940031 conn(0x7f6c5c072990 msgr2=0x7f6c5c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.183+0000 7f6c6083c700 1 -- 172.21.15.83:0/3403940031 shutdown_connections 2023-11-26T07:12:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.183+0000 7f6c6083c700 1 -- 172.21.15.83:0/3403940031 wait complete. 2023-11-26T07:12:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.184+0000 7f6c6083c700 1 Processor -- start 2023-11-26T07:12:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.184+0000 7f6c6083c700 1 -- start start 2023-11-26T07:12:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c6083c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c5c0cfd40 0x7f6c5c13d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c6083c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c13e000 0x7f6c5c158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c6083c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c15efb0 0x7f6c5c15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c6083c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c06dee0 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c6083c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c06e1c0 con 0x7f6c5c15efb0 2023-11-26T07:12:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c6083c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c06e4a0 con 0x7f6c5c13e000 2023-11-26T07:12:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c5a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c5c0cfd40 0x7f6c5c13d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c5a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c5c0cfd40 0x7f6c5c13d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35796/0 (socket says 172.21.15.83:35796) 2023-11-26T07:12:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c5a59c700 1 -- 172.21.15.83:0/99232418 learned_addr learned my addr 172.21.15.83:0/99232418 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c5ad9d700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c15efb0 0x7f6c5c15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c59d9b700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c13e000 0x7f6c5c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c5a59c700 1 -- 172.21.15.83:0/99232418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c13e000 msgr2=0x7f6c5c158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c5a59c700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c13e000 0x7f6c5c158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c5a59c700 1 -- 172.21.15.83:0/99232418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c15efb0 msgr2=0x7f6c5c15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c5a59c700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c15efb0 0x7f6c5c15a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.185+0000 7f6c5a59c700 1 -- 172.21.15.83:0/99232418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c009b30 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.186+0000 7f6c5a59c700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c5c0cfd40 0x7f6c5c13d540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00b370 tx=0x7f6c4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.186+0000 7f6c52ffd700 1 -- 172.21.15.83:0/99232418 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c004800 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.186+0000 7f6c52ffd700 1 -- 172.21.15.83:0/99232418 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c0092e0 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.186+0000 7f6c52ffd700 1 -- 172.21.15.83:0/99232418 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c01c680 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.186+0000 7f6c6083c700 1 -- 172.21.15.83:0/99232418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5c06e780 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.186+0000 7f6c6083c700 1 -- 172.21.15.83:0/99232418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5c06dc50 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.187+0000 7f6c6083c700 1 -- 172.21.15.83:0/99232418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5c067f60 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.187+0000 7f6c52ffd700 1 -- 172.21.15.83:0/99232418 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c4c01c7e0 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.187+0000 7f6c52ffd700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6c3c06aed0 0x7f6c3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.187+0000 7f6c52ffd700 1 -- 172.21.15.83:0/99232418 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6c4c08ab70 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.188+0000 7f6c59d9b700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6c3c06aed0 0x7f6c3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.188+0000 7f6c59d9b700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6c3c06aed0 0x7f6c3c06d390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c1f7180 tx=0x7f6c44009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.192+0000 7f6c52ffd700 1 -- 172.21.15.83:0/99232418 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c4c055fb0 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.400+0000 7f6c6083c700 1 -- 172.21.15.83:0/99232418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6c5c15bf30 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.986+0000 7f6c52ffd700 1 -- 172.21.15.83:0/99232418 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6c4c0595d0 con 0x7f6c5c0cfd40 2023-11-26T07:12:15.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 -- 172.21.15.83:0/99232418 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6c3c06aed0 msgr2=0x7f6c3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:15.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6c3c06aed0 0x7f6c3c06d390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c1f7180 tx=0x7f6c44009210 comp rx=0 tx=0).stop 2023-11-26T07:12:15.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 -- 172.21.15.83:0/99232418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c5c0cfd40 msgr2=0x7f6c5c13d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:15.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c5c0cfd40 0x7f6c5c13d540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00b370 tx=0x7f6c4c004ad0 comp rx=0 tx=0).stop 2023-11-26T07:12:15.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 -- 172.21.15.83:0/99232418 shutdown_connections 2023-11-26T07:12:15.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c5c15efb0 0x7f6c5c15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:15.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c5c0cfd40 0x7f6c5c13d540 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:15.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6c3c06aed0 0x7f6c3c06d390 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:15.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 --2- 172.21.15.83:0/99232418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5c13e000 0x7f6c5c158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:15.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 -- 172.21.15.83:0/99232418 >> 172.21.15.83:0/99232418 conn(0x7f6c5c072990 msgr2=0x7f6c5c15c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:15.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 -- 172.21.15.83:0/99232418 shutdown_connections 2023-11-26T07:12:15.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:15.990+0000 7f6c6083c700 1 -- 172.21.15.83:0/99232418 wait complete. 2023-11-26T07:12:15.993 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2023-11-26T07:12:16.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:12:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.126+0000 7f5297d40700 1 Processor -- start 2023-11-26T07:12:16.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.127+0000 7f5297d40700 1 -- start start 2023-11-26T07:12:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.127+0000 7f5297d40700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f52901446a0 0x7f52900b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.127+0000 7f5297d40700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52900adf00 0x7f529006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.127+0000 7f5297d40700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52900af950 con 0x7f529006c540 2023-11-26T07:12:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.127+0000 7f5297d40700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529006c9f0 con 0x7f52900adf00 2023-11-26T07:12:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.127+0000 7f5297d40700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5290144a80 con 0x7f52901446a0 2023-11-26T07:12:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f52952db700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52900adf00 0x7f529006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f52952db700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f52900adf00 0x7f529006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35808/0 (socket says 172.21.15.83:35808) 2023-11-26T07:12:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f52952db700 1 -- 172.21.15.83:0/1650431335 learned_addr learned my addr 172.21.15.83:0/1650431335 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f52962dd700 1 --1- 172.21.15.83:0/1650431335 >> v1:172.21.15.83:6789/0 conn(0x7f529006c540 0x7f529006da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55850/0 (socket says 172.21.15.83:55850) 2023-11-26T07:12:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f5295adc700 1 --2- 172.21.15.83:0/1650431335 >> v2:172.21.15.203:3300/0 conn(0x7f52901446a0 0x7f52900b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f52952db700 1 -- 172.21.15.83:0/1650431335 >> v2:172.21.15.203:3300/0 conn(0x7f52901446a0 msgr2=0x7f52900b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f52952db700 1 --2- 172.21.15.83:0/1650431335 >> v2:172.21.15.203:3300/0 conn(0x7f52901446a0 0x7f52900b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f5294ada700 1 -- 172.21.15.83:0/1650431335 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674206370 0 0) 0x7f52900af950 con 0x7f529006c540 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f52952db700 1 -- 172.21.15.83:0/1650431335 >> v1:172.21.15.83:6789/0 conn(0x7f529006c540 legacy=0x7f529006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f52952db700 1 -- 172.21.15.83:0/1650431335 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52900afc30 con 0x7f52900adf00 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f52952db700 1 --2- 172.21.15.83:0/1650431335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52900adf00 0x7f529006c000 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f52900729a0 tx=0x7f528c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1c6865c7a42a1d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f5294ada700 1 -- 172.21.15.83:0/1650431335 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c00dd10 con 0x7f52900adf00 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f5294ada700 1 -- 172.21.15.83:0/1650431335 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c00de70 con 0x7f52900adf00 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.128+0000 7f5294ada700 1 -- 172.21.15.83:0/1650431335 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c010670 con 0x7f52900adf00 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.129+0000 7f5297d40700 1 -- 172.21.15.83:0/1650431335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52900adf00 msgr2=0x7f529006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.129+0000 7f5297d40700 1 --2- 172.21.15.83:0/1650431335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52900adf00 0x7f529006c000 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f52900729a0 tx=0x7f528c00bea0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.129+0000 7f5297d40700 1 -- 172.21.15.83:0/1650431335 shutdown_connections 2023-11-26T07:12:16.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.129+0000 7f5297d40700 1 --2- 172.21.15.83:0/1650431335 >> v2:172.21.15.203:3300/0 conn(0x7f52901446a0 0x7f52900b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.129+0000 7f5297d40700 1 --2- 172.21.15.83:0/1650431335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52900adf00 0x7f529006c000 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.129+0000 7f5297d40700 1 -- 172.21.15.83:0/1650431335 >> 172.21.15.83:0/1650431335 conn(0x7f5290071b90 msgr2=0x7f5290071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.129+0000 7f5297d40700 1 -- 172.21.15.83:0/1650431335 shutdown_connections 2023-11-26T07:12:16.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.129+0000 7f5297d40700 1 -- 172.21.15.83:0/1650431335 wait complete. 2023-11-26T07:12:16.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.130+0000 7f5297d40700 1 Processor -- start 2023-11-26T07:12:16.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.130+0000 7f5297d40700 1 -- start start 2023-11-26T07:12:16.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5297d40700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529006c540 0x7f529015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5297d40700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52900adf00 0x7f529006b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5297d40700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52901446a0 0x7f529013c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5297d40700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529006be80 con 0x7f529006c540 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5297d40700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529015dd80 con 0x7f52900adf00 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5297d40700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529015e060 con 0x7f52901446a0 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5295adc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529006c540 0x7f529015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5295adc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529006c540 0x7f529015d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35812/0 (socket says 172.21.15.83:35812) 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5295adc700 1 -- 172.21.15.83:0/808823641 learned_addr learned my addr 172.21.15.83:0/808823641 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f52952db700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52900adf00 0x7f529006b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f52962dd700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52901446a0 0x7f529013c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5295adc700 1 -- 172.21.15.83:0/808823641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52901446a0 msgr2=0x7f529013c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5295adc700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52901446a0 0x7f529013c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5295adc700 1 -- 172.21.15.83:0/808823641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52900adf00 msgr2=0x7f529006b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5295adc700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52900adf00 0x7f529006b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5295adc700 1 -- 172.21.15.83:0/808823641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c00b7f0 con 0x7f529006c540 2023-11-26T07:12:16.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5295adc700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529006c540 0x7f529015d6a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f528400d800 tx=0x7f528400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:16.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f52827fc700 1 -- 172.21.15.83:0/808823641 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52840125f0 con 0x7f529006c540 2023-11-26T07:12:16.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f52827fc700 1 -- 172.21.15.83:0/808823641 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284012750 con 0x7f529006c540 2023-11-26T07:12:16.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5297d40700 1 -- 172.21.15.83:0/808823641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529013c1a0 con 0x7f529006c540 2023-11-26T07:12:16.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f52827fc700 1 -- 172.21.15.83:0/808823641 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284012b00 con 0x7f529006c540 2023-11-26T07:12:16.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.131+0000 7f5297d40700 1 -- 172.21.15.83:0/808823641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529013cee0 con 0x7f529006c540 2023-11-26T07:12:16.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.132+0000 7f5297d40700 1 -- 172.21.15.83:0/808823641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52900753f0 con 0x7f529006c540 2023-11-26T07:12:16.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.133+0000 7f52827fc700 1 -- 172.21.15.83:0/808823641 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5284012c60 con 0x7f529006c540 2023-11-26T07:12:16.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.133+0000 7f52827fc700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527406aed0 0x7f527406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.133+0000 7f52827fc700 1 -- 172.21.15.83:0/808823641 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f52840895c0 con 0x7f529006c540 2023-11-26T07:12:16.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.133+0000 7f52952db700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527406aed0 0x7f527406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.134+0000 7f52952db700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527406aed0 0x7f527406d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f52900729a0 tx=0x7f528c013000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:16.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.138+0000 7f52827fc700 1 -- 172.21.15.83:0/808823641 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5284054a50 con 0x7f529006c540 2023-11-26T07:12:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.401+0000 7f5297d40700 1 -- 172.21.15.83:0/808823641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5290068670 con 0x7f529006c540 2023-11-26T07:12:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.402+0000 7f52827fc700 1 -- 172.21.15.83:0/808823641 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f5284058070 con 0x7f529006c540 2023-11-26T07:12:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:16.404 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:12:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2023-11-26T07:12:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2023-11-26T07:12:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlA8pxNxAAt5NuewAjm7yPThCguJ6sEA==", 2023-11-26T07:12:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:12:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:12:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlHBgDCBAAk6egig5krY7kiMiHkMKrWg==", 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlbKEQAxAAM2tBBQwM+tt2d8JU37YURg==", 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2023-11-26T07:12:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlI3ZHDxAArtLmBVGr/vAB/YGwEFfhGw==", 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBo72JlhuSTKRAA6LRDOHBQVl5TL5tIHzyfzA==", 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBr72JlteU6KRAAuuHTYosYpl+KptywVmaeHA==", 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2023-11-26T07:12:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBu72JlEb6dKBAAebp4pnu9XZsSRJpojTZcvQ==", 2023-11-26T07:12:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2023-11-26T07:12:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBx72JlbTJ3KBAA1JUxxAoyCzTtfgubUxSJEA==", 2023-11-26T07:12:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2023-11-26T07:12:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB372JlufqqLhAAm6ZUt9juWj5yAIDlnCXqWg==", 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB672Jl1L3vLRAAX06GcsntcmB+7L1CsfnJzw==", 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB972Jlv41HLRAALtuyuREvGmTuDZt/RT0UcA==", 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCA72JlKCHsLBAAAXDzA7AN36wcLjhK97nlkg==", 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ==", 2023-11-26T07:12:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JleNpIGhAAKwoQj4zB9YOxpFMD28juCA==", 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JldAHlFhAAb3u9KljU9vssDl54zgno2Q==", 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:12:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JluiQUIBAA06mqpeCzgpbSLo2/fulfmw==", 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2023-11-26T07:12:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlD4jbGhAA05xtSS5EzQTuhBjVqS2JpA==", 2023-11-26T07:12:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:12:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:12:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:12:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-mds", 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCG72Jl8k3oKxAAbZQnoGnm5kyzsdLVFEHDow==", 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-mds" 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-mgr", 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCG72JlmVjoKxAAF48vZqjnqROQ0rg51aGrCg==", 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-26T07:12:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-osd", 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCG72JlOmDoKxAAHvAfALTWAce1v+mLcloFww==", 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-osd" 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rbd", 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCG72JlpGfoKxAALIMQ6Mv8h+aNVaD4q0b+YQ==", 2023-11-26T07:12:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCG72JleW/oKxAAhnGwjBg3WMqjngf4H8PZMg==", 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.bootstrap-rgw", 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCG72JlJnfoKxAAbefrwfI2Ddj9aaywz6P7Hg==", 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlsA0VNBAAfSlnVquhI+vpLFwmgLGGVA==", 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72Jl3qkjMBAA+jcxYuDG/UHhB7W7HDwmaw==", 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:12:16.422 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:12:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 -- 172.21.15.83:0/808823641 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527406aed0 msgr2=0x7f527406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527406aed0 0x7f527406d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f52900729a0 tx=0x7f528c013000 comp rx=0 tx=0).stop 2023-11-26T07:12:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 -- 172.21.15.83:0/808823641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529006c540 msgr2=0x7f529015d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529006c540 0x7f529015d6a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f528400d800 tx=0x7f528400dbc0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 -- 172.21.15.83:0/808823641 shutdown_connections 2023-11-26T07:12:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527406aed0 0x7f527406d390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52901446a0 0x7f529013c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529006c540 0x7f529015d6a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 --2- 172.21.15.83:0/808823641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52900adf00 0x7f529006b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 -- 172.21.15.83:0/808823641 >> 172.21.15.83:0/808823641 conn(0x7f5290071b90 msgr2=0x7f52900b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 -- 172.21.15.83:0/808823641 shutdown_connections 2023-11-26T07:12:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.404+0000 7f5297d40700 1 -- 172.21.15.83:0/808823641 wait complete. 2023-11-26T07:12:16.425 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-26T07:12:16.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.546+0000 7f954d9bf700 1 Processor -- start 2023-11-26T07:12:16.547 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.546+0000 7f954d9bf700 1 -- start start 2023-11-26T07:12:16.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.546+0000 7f954d9bf700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f95480b0fb0 0x7f95480cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.546+0000 7f954d9bf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480d0280 0x7f954815d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.546+0000 7f954d9bf700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95480b1c60 con 0x7f95480b1390 2023-11-26T07:12:16.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.546+0000 7f954d9bf700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95480d0720 con 0x7f95480b0fb0 2023-11-26T07:12:16.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.546+0000 7f954d9bf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95480d0a00 con 0x7f95480d0280 2023-11-26T07:12:16.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f95467fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f95480b1390 0x7f954806b750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55862/0 (socket says 172.21.15.83:55862) 2023-11-26T07:12:16.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f95467fc700 1 -- 172.21.15.83:0/363311594 learned_addr learned my addr 172.21.15.83:0/363311594 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:16.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f9546ffd700 1 --2- 172.21.15.83:0/363311594 >> v2:172.21.15.203:3300/0 conn(0x7f95480b0fb0 0x7f95480cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f95477fe700 1 --2- 172.21.15.83:0/363311594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480d0280 0x7f954815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f9545ffb700 1 -- 172.21.15.83:0/363311594 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021344537 0 0) 0x7f95480b1c60 con 0x7f95480b1390 2023-11-26T07:12:16.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f9545ffb700 1 -- 172.21.15.83:0/363311594 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95300035b0 con 0x7f95480b1390 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f9546ffd700 1 -- 172.21.15.83:0/363311594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480d0280 msgr2=0x7f954815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f9546ffd700 1 --2- 172.21.15.83:0/363311594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480d0280 0x7f954815d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f9546ffd700 1 -- 172.21.15.83:0/363311594 >> v1:172.21.15.83:6789/0 conn(0x7f95480b1390 legacy=0x7f954806b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f9545ffb700 1 -- 172.21.15.83:0/363311594 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3281766176 0 0) 0x7f95300035b0 con 0x7f95480b1390 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f9546ffd700 1 -- 172.21.15.83:0/363311594 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95480d0ce0 con 0x7f95480b0fb0 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f95477fe700 1 --2- 172.21.15.83:0/363311594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480d0280 0x7f954815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f9546ffd700 1 --2- 172.21.15.83:0/363311594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95480b0fb0 0x7f95480cfd40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9538009840 tx=0x7f9538009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dcacb24029460a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.547+0000 7f9545ffb700 1 -- 172.21.15.83:0/363311594 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c001670 con 0x7f95480b0fb0 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.548+0000 7f9545ffb700 1 -- 172.21.15.83:0/363311594 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9538004030 con 0x7f95480b0fb0 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.548+0000 7f9545ffb700 1 -- 172.21.15.83:0/363311594 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95380043a0 con 0x7f95480b0fb0 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.548+0000 7f954d9bf700 1 -- 172.21.15.83:0/363311594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95480b0fb0 msgr2=0x7f95480cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.548+0000 7f954d9bf700 1 --2- 172.21.15.83:0/363311594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95480b0fb0 0x7f95480cfd40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9538009840 tx=0x7f9538009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:16.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.548+0000 7f954d9bf700 1 -- 172.21.15.83:0/363311594 shutdown_connections 2023-11-26T07:12:16.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.548+0000 7f954d9bf700 1 --2- 172.21.15.83:0/363311594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480d0280 0x7f954815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.548+0000 7f954d9bf700 1 --2- 172.21.15.83:0/363311594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95480b0fb0 0x7f95480cfd40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.548+0000 7f954d9bf700 1 -- 172.21.15.83:0/363311594 >> 172.21.15.83:0/363311594 conn(0x7f954809aa00 msgr2=0x7f954809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:16.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.548+0000 7f954d9bf700 1 -- 172.21.15.83:0/363311594 shutdown_connections 2023-11-26T07:12:16.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.548+0000 7f954d9bf700 1 -- 172.21.15.83:0/363311594 wait complete. 2023-11-26T07:12:16.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.549+0000 7f954d9bf700 1 Processor -- start 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.550+0000 7f954d9bf700 1 -- start start 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.550+0000 7f954d9bf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480b0fb0 0x7f954815f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.550+0000 7f954d9bf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95480b1390 0x7f95480b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.550+0000 7f954d9bf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95480d0280 0x7f95480b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.550+0000 7f954d9bf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95480b5780 con 0x7f95480d0280 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.550+0000 7f954d9bf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95480c9930 con 0x7f95480b1390 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.550+0000 7f954d9bf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95480b7690 con 0x7f95480b0fb0 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.550+0000 7f95477fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95480d0280 0x7f95480b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.550+0000 7f95477fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95480d0280 0x7f95480b7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35830/0 (socket says 172.21.15.83:35830) 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.550+0000 7f95477fe700 1 -- 172.21.15.83:0/3141954542 learned_addr learned my addr 172.21.15.83:0/3141954542 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f9546ffd700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480b0fb0 0x7f954815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f95467fc700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95480b1390 0x7f95480b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f95477fe700 1 -- 172.21.15.83:0/3141954542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480b0fb0 msgr2=0x7f954815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f95477fe700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480b0fb0 0x7f954815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f95477fe700 1 -- 172.21.15.83:0/3141954542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95480b1390 msgr2=0x7f95480b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f95477fe700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95480b1390 0x7f95480b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f95477fe700 1 -- 172.21.15.83:0/3141954542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540009a70 con 0x7f95480d0280 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f9546ffd700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480b0fb0 0x7f954815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f95477fe700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95480d0280 0x7f95480b7150 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f954000efc0 tx=0x7f954000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f952ffff700 1 -- 172.21.15.83:0/3141954542 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540009d70 con 0x7f95480d0280 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f952ffff700 1 -- 172.21.15.83:0/3141954542 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954000f040 con 0x7f95480d0280 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f954d9bf700 1 -- 172.21.15.83:0/3141954542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95380094a0 con 0x7f95480d0280 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f952ffff700 1 -- 172.21.15.83:0/3141954542 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954000ce00 con 0x7f95480d0280 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.551+0000 7f954d9bf700 1 -- 172.21.15.83:0/3141954542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95480b54a0 con 0x7f95480d0280 2023-11-26T07:12:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.552+0000 7f954d9bf700 1 -- 172.21.15.83:0/3141954542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9548068480 con 0x7f95480d0280 2023-11-26T07:12:16.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.553+0000 7f952ffff700 1 -- 172.21.15.83:0/3141954542 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9540014480 con 0x7f95480d0280 2023-11-26T07:12:16.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.553+0000 7f952ffff700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953406aed0 0x7f953406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.553+0000 7f952ffff700 1 -- 172.21.15.83:0/3141954542 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f95400899e0 con 0x7f95480d0280 2023-11-26T07:12:16.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.553+0000 7f9546ffd700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953406aed0 0x7f953406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.553+0000 7f9546ffd700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953406aed0 0x7f953406d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9538009b90 tx=0x7f9538013040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:16.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.557+0000 7f952ffff700 1 -- 172.21.15.83:0/3141954542 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9540054de0 con 0x7f95480d0280 2023-11-26T07:12:16.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.827+0000 7f954d9bf700 1 -- 172.21.15.83:0/3141954542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f95480b7b90 con 0x7f95480d0280 2023-11-26T07:12:16.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.829+0000 7f952ffff700 1 -- 172.21.15.83:0/3141954542 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9540058400 con 0x7f95480d0280 2023-11-26T07:12:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 -- 172.21.15.83:0/3141954542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953406aed0 msgr2=0x7f953406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953406aed0 0x7f953406d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9538009b90 tx=0x7f9538013040 comp rx=0 tx=0).stop 2023-11-26T07:12:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 -- 172.21.15.83:0/3141954542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95480d0280 msgr2=0x7f95480b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95480d0280 0x7f95480b7150 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f954000efc0 tx=0x7f954000c3a0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 -- 172.21.15.83:0/3141954542 shutdown_connections 2023-11-26T07:12:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95480d0280 0x7f95480b7150 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953406aed0 0x7f953406d390 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95480b0fb0 0x7f954815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 --2- 172.21.15.83:0/3141954542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95480b1390 0x7f95480b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 -- 172.21.15.83:0/3141954542 >> 172.21.15.83:0/3141954542 conn(0x7f954809aa00 msgr2=0x7f9548160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 -- 172.21.15.83:0/3141954542 shutdown_connections 2023-11-26T07:12:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.831+0000 7f954d9bf700 1 -- 172.21.15.83:0/3141954542 wait complete. 2023-11-26T07:12:16.845 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-26T07:12:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.964+0000 7fb03b70b700 1 Processor -- start 2023-11-26T07:12:16.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.964+0000 7fb03b70b700 1 -- start start 2023-11-26T07:12:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.965+0000 7fb03b70b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03406c710 0x7fb0340b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.965+0000 7fb03b70b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0340ae430 con 0x7fb0340af4d0 2023-11-26T07:12:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.965+0000 7fb03b70b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0340ae710 con 0x7fb03415e080 2023-11-26T07:12:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.965+0000 7fb03b70b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340af0b0 con 0x7fb03406c710 2023-11-26T07:12:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.965+0000 7fb039ca8700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb0340af4d0 0x7fb03406e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55878/0 (socket says 172.21.15.83:55878) 2023-11-26T07:12:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.965+0000 7fb039ca8700 1 -- 172.21.15.83:0/890131462 learned_addr learned my addr 172.21.15.83:0/890131462 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.965+0000 7fb038ca6700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03406c710 0x7fb0340b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.965+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987935502 0 0) 0x7fb0340ae430 con 0x7fb0340af4d0 2023-11-26T07:12:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0200035b0 con 0x7fb0340af4d0 2023-11-26T07:12:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859725909 0 0) 0x7fb0340ae710 con 0x7fb03415e080 2023-11-26T07:12:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0340ae430 con 0x7fb03415e080 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1588258045 0 0) 0x7fb0200035b0 con 0x7fb0340af4d0 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03406c710 msgr2=0x7fb0340b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03406c710 0x7fb0340b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 >> v1:172.21.15.203:6789/0 conn(0x7fb03415e080 legacy=0x7fb03406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0340ae9f0 con 0x7fb0340af4d0 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fb02c0019f0 con 0x7fb0340af4d0 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 >> v1:172.21.15.83:6789/0 conn(0x7fb0340af4d0 legacy=0x7fb03406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03406c710 0x7fb020003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb020004170 0x7fb020004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020004d90 0x7fb020005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340af0b0 con 0x7fb020004170 2023-11-26T07:12:16.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02000bb50 con 0x7fb020004d90 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02000bd10 con 0x7fb03406c710 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb038ca6700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb020004170 0x7fb020004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb039ca8700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020004d90 0x7fb020005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb0394a7700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03406c710 0x7fb020003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb038ca6700 1 -- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03406c710 msgr2=0x7fb020003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb038ca6700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb03406c710 0x7fb020003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb038ca6700 1 -- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020004d90 msgr2=0x7fb020005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb038ca6700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020004d90 0x7fb020005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb038ca6700 1 -- 172.21.15.83:0/890131462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0200035b0 con 0x7fb020004170 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb039ca8700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020004d90 0x7fb020005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.966+0000 7fb038ca6700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb020004170 0x7fb020004850 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb030009b60 tx=0x7fb020005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480f3ffc2227c35b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.967+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000e6c0 con 0x7fb020004170 2023-11-26T07:12:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.967+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03000e820 con 0x7fb020004170 2023-11-26T07:12:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.967+0000 7fb02bfff700 1 -- 172.21.15.83:0/890131462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000eb90 con 0x7fb020004170 2023-11-26T07:12:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.967+0000 7fb03b70b700 1 -- 172.21.15.83:0/890131462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb020004170 msgr2=0x7fb020004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.967+0000 7fb03b70b700 1 --2- 172.21.15.83:0/890131462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb020004170 0x7fb020004850 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb030009b60 tx=0x7fb020005880 comp rx=0 tx=0).stop 2023-11-26T07:12:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.967+0000 7fb039ca8700 1 -- 172.21.15.83:0/890131462 reap_dead start 2023-11-26T07:12:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.967+0000 7fb03b70b700 1 -- 172.21.15.83:0/890131462 shutdown_connections 2023-11-26T07:12:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.967+0000 7fb03b70b700 1 -- 172.21.15.83:0/890131462 >> 172.21.15.83:0/890131462 conn(0x7fb034071b90 msgr2=0x7fb034071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.967+0000 7fb03b70b700 1 -- 172.21.15.83:0/890131462 shutdown_connections 2023-11-26T07:12:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.967+0000 7fb03b70b700 1 -- 172.21.15.83:0/890131462 wait complete. 2023-11-26T07:12:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.968+0000 7fb03b70b700 1 Processor -- start 2023-11-26T07:12:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.968+0000 7fb03b70b700 1 -- start start 2023-11-26T07:12:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.969+0000 7fb03b70b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020004170 0x7fb034149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.969+0000 7fb03b70b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb034154770 0x7fb034160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.969+0000 7fb03b70b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb034154bf0 0x7fb034160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.969+0000 7fb03b70b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03414a130 con 0x7fb034154bf0 2023-11-26T07:12:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.969+0000 7fb03b70b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340cda40 con 0x7fb020004170 2023-11-26T07:12:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.969+0000 7fb03b70b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340cdd20 con 0x7fb034154770 2023-11-26T07:12:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.969+0000 7fb039ca8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb034154bf0 0x7fb034160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.969+0000 7fb039ca8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb034154bf0 0x7fb034160b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35850/0 (socket says 172.21.15.83:35850) 2023-11-26T07:12:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.969+0000 7fb039ca8700 1 -- 172.21.15.83:0/3879141759 learned_addr learned my addr 172.21.15.83:0/3879141759 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.970+0000 7fb039ca8700 1 -- 172.21.15.83:0/3879141759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb034154770 msgr2=0x7fb034160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.970+0000 7fb039ca8700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb034154770 0x7fb034160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.970+0000 7fb039ca8700 1 -- 172.21.15.83:0/3879141759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020004170 msgr2=0x7fb034149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.970+0000 7fb039ca8700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020004170 0x7fb034149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.970+0000 7fb039ca8700 1 -- 172.21.15.83:0/3879141759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0340ce000 con 0x7fb034154bf0 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.970+0000 7fb039ca8700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb034154bf0 0x7fb034160b70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb03406d350 tx=0x7fb02c00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.970+0000 7fb029ffb700 1 -- 172.21.15.83:0/3879141759 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c00ec50 con 0x7fb034154bf0 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.970+0000 7fb029ffb700 1 -- 172.21.15.83:0/3879141759 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c00edb0 con 0x7fb034154bf0 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.970+0000 7fb029ffb700 1 -- 172.21.15.83:0/3879141759 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c013660 con 0x7fb034154bf0 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.970+0000 7fb03b70b700 1 -- 172.21.15.83:0/3879141759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0340ce2e0 con 0x7fb034154bf0 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.970+0000 7fb03b70b700 1 -- 172.21.15.83:0/3879141759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0341471d0 con 0x7fb034154bf0 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.971+0000 7fb03b70b700 1 -- 172.21.15.83:0/3879141759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0340753f0 con 0x7fb034154bf0 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.971+0000 7fb029ffb700 1 -- 172.21.15.83:0/3879141759 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb02c0137c0 con 0x7fb034154bf0 2023-11-26T07:12:16.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.972+0000 7fb029ffb700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb01806aed0 0x7fb01806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:16.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.972+0000 7fb029ffb700 1 -- 172.21.15.83:0/3879141759 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb02c089820 con 0x7fb034154bf0 2023-11-26T07:12:16.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.972+0000 7fb0394a7700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb01806aed0 0x7fb01806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:16.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.972+0000 7fb0394a7700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb01806aed0 0x7fb01806d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb024001790 tx=0x7fb024009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:16.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:16.976+0000 7fb029ffb700 1 -- 172.21.15.83:0/3879141759 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb02c054cb0 con 0x7fb034154bf0 2023-11-26T07:12:17.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.247+0000 7fb03b70b700 1 -- 172.21.15.83:0/3879141759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fb034147360 con 0x7fb034154bf0 2023-11-26T07:12:17.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.249+0000 7fb029ffb700 1 -- 172.21.15.83:0/3879141759 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb034147360 con 0x7fb034154bf0 2023-11-26T07:12:17.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 -- 172.21.15.83:0/3879141759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb01806aed0 msgr2=0x7fb01806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:17.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb01806aed0 0x7fb01806d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb024001790 tx=0x7fb024009040 comp rx=0 tx=0).stop 2023-11-26T07:12:17.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 -- 172.21.15.83:0/3879141759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb034154bf0 msgr2=0x7fb034160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:17.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb034154bf0 0x7fb034160b70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb03406d350 tx=0x7fb02c00e440 comp rx=0 tx=0).stop 2023-11-26T07:12:17.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 -- 172.21.15.83:0/3879141759 shutdown_connections 2023-11-26T07:12:17.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb034154bf0 0x7fb034160b70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb020004170 0x7fb034149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb01806aed0 0x7fb01806d390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 --2- 172.21.15.83:0/3879141759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb034154770 0x7fb034160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 -- 172.21.15.83:0/3879141759 >> 172.21.15.83:0/3879141759 conn(0x7fb034071b90 msgr2=0x7fb0340b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:17.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 -- 172.21.15.83:0/3879141759 shutdown_connections 2023-11-26T07:12:17.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.251+0000 7fb03b70b700 1 -- 172.21.15.83:0/3879141759 wait complete. 2023-11-26T07:12:17.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-26T07:12:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.384+0000 7fc27a225700 1 Processor -- start 2023-11-26T07:12:17.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.384+0000 7fc27a225700 1 -- start start 2023-11-26T07:12:17.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.385+0000 7fc27a225700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27415bdd0 0x7fc2741604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.385+0000 7fc27a225700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2740b3370 0x7fc2740b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.385+0000 7fc27a225700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc2740d1c70 0x7fc2740d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.385+0000 7fc27a225700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2740b2f50 con 0x7fc2740b3370 2023-11-26T07:12:17.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.385+0000 7fc27a225700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2740d2660 con 0x7fc2740d1c70 2023-11-26T07:12:17.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.385+0000 7fc27a225700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740b2030 con 0x7fc27415bdd0 2023-11-26T07:12:17.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.385+0000 7fc272ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2740b3370 0x7fc2740b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.385+0000 7fc272ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc2740b3370 0x7fc2740b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35856/0 (socket says 172.21.15.83:35856) 2023-11-26T07:12:17.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.385+0000 7fc272ffd700 1 -- 172.21.15.83:0/2734570298 learned_addr learned my addr 172.21.15.83:0/2734570298 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:17.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.386+0000 7fc272ffd700 1 -- 172.21.15.83:0/2734570298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27415bdd0 msgr2=0x7fc2741604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:17.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.386+0000 7fc273fff700 1 --2- 172.21.15.83:0/2734570298 >> v2:172.21.15.203:3300/0 conn(0x7fc2740d1c70 0x7fc2740d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.386+0000 7fc2737fe700 1 --2- 172.21.15.83:0/2734570298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27415bdd0 0x7fc2741604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.386+0000 7fc272ffd700 1 --2- 172.21.15.83:0/2734570298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27415bdd0 0x7fc2741604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.386+0000 7fc272ffd700 1 -- 172.21.15.83:0/2734570298 >> v2:172.21.15.203:3300/0 conn(0x7fc2740d1c70 msgr2=0x7fc2740d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:17.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.386+0000 7fc272ffd700 1 --2- 172.21.15.83:0/2734570298 >> v2:172.21.15.203:3300/0 conn(0x7fc2740d1c70 0x7fc2740d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.386+0000 7fc272ffd700 1 -- 172.21.15.83:0/2734570298 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2740b22d0 con 0x7fc2740b3370 2023-11-26T07:12:17.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.386+0000 7fc272ffd700 1 --2- 172.21.15.83:0/2734570298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2740b3370 0x7fc2740b1af0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc274160a20 tx=0x7fc264009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcce22200f3813e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:17.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.386+0000 7fc2727fc700 1 -- 172.21.15.83:0/2734570298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26400b870 con 0x7fc2740b3370 2023-11-26T07:12:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.386+0000 7fc2727fc700 1 -- 172.21.15.83:0/2734570298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26400b9d0 con 0x7fc2740b3370 2023-11-26T07:12:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.386+0000 7fc2727fc700 1 -- 172.21.15.83:0/2734570298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26400bd60 con 0x7fc2740b3370 2023-11-26T07:12:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.387+0000 7fc27a225700 1 -- 172.21.15.83:0/2734570298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2740b3370 msgr2=0x7fc2740b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.387+0000 7fc27a225700 1 --2- 172.21.15.83:0/2734570298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2740b3370 0x7fc2740b1af0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc274160a20 tx=0x7fc264009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.387+0000 7fc27a225700 1 -- 172.21.15.83:0/2734570298 shutdown_connections 2023-11-26T07:12:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.387+0000 7fc27a225700 1 --2- 172.21.15.83:0/2734570298 >> v2:172.21.15.203:3300/0 conn(0x7fc2740d1c70 0x7fc2740d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.387+0000 7fc27a225700 1 --2- 172.21.15.83:0/2734570298 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc27415bdd0 0x7fc2741604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.387+0000 7fc27a225700 1 --2- 172.21.15.83:0/2734570298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2740b3370 0x7fc2740b1af0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.387+0000 7fc27a225700 1 -- 172.21.15.83:0/2734570298 >> 172.21.15.83:0/2734570298 conn(0x7fc2740731a0 msgr2=0x7fc27409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.387+0000 7fc27a225700 1 -- 172.21.15.83:0/2734570298 shutdown_connections 2023-11-26T07:12:17.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.387+0000 7fc27a225700 1 -- 172.21.15.83:0/2734570298 wait complete. 2023-11-26T07:12:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.388+0000 7fc27a225700 1 Processor -- start 2023-11-26T07:12:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.388+0000 7fc27a225700 1 -- start start 2023-11-26T07:12:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc27a225700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2740b3370 0x7fc2740c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc27a225700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2740d1c70 0x7fc27415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc27a225700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27415bdd0 0x7fc27415d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc27a225700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27415ec50 con 0x7fc27415bdd0 2023-11-26T07:12:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc27a225700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc274157b10 con 0x7fc2740b3370 2023-11-26T07:12:17.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc27a225700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27415dfd0 con 0x7fc2740d1c70 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc273fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27415bdd0 0x7fc27415d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc273fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27415bdd0 0x7fc27415d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35866/0 (socket says 172.21.15.83:35866) 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc273fff700 1 -- 172.21.15.83:0/1184608024 learned_addr learned my addr 172.21.15.83:0/1184608024 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc2737fe700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2740b3370 0x7fc2740c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc272ffd700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2740d1c70 0x7fc27415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc273fff700 1 -- 172.21.15.83:0/1184608024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2740d1c70 msgr2=0x7fc27415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc273fff700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2740d1c70 0x7fc27415cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc273fff700 1 -- 172.21.15.83:0/1184608024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2740b3370 msgr2=0x7fc2740c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc273fff700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2740b3370 0x7fc2740c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc273fff700 1 -- 172.21.15.83:0/1184608024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc264009550 con 0x7fc27415bdd0 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc273fff700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27415bdd0 0x7fc27415d510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00cd20 tx=0x7fc26c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:17.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc25bfff700 1 -- 172.21.15.83:0/1184608024 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c003a20 con 0x7fc27415bdd0 2023-11-26T07:12:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc25bfff700 1 -- 172.21.15.83:0/1184608024 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c003b80 con 0x7fc27415bdd0 2023-11-26T07:12:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc27a225700 1 -- 172.21.15.83:0/1184608024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27415e2b0 con 0x7fc27415bdd0 2023-11-26T07:12:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.389+0000 7fc25bfff700 1 -- 172.21.15.83:0/1184608024 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c004020 con 0x7fc27415bdd0 2023-11-26T07:12:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.390+0000 7fc27a225700 1 -- 172.21.15.83:0/1184608024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27415e870 con 0x7fc27415bdd0 2023-11-26T07:12:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.390+0000 7fc27a225700 1 -- 172.21.15.83:0/1184608024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27415eec0 con 0x7fc27415bdd0 2023-11-26T07:12:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.391+0000 7fc25bfff700 1 -- 172.21.15.83:0/1184608024 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc26c004180 con 0x7fc27415bdd0 2023-11-26T07:12:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.391+0000 7fc25bfff700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 0x7fc26006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.391+0000 7fc25bfff700 1 -- 172.21.15.83:0/1184608024 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc26c012070 con 0x7fc27415bdd0 2023-11-26T07:12:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.391+0000 7fc2737fe700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 0x7fc26006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.392+0000 7fc2737fe700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 0x7fc26006d390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc2740cf390 tx=0x7fc268008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:17.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.396+0000 7fc25bfff700 1 -- 172.21.15.83:0/1184608024 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc26c054e00 con 0x7fc27415bdd0 2023-11-26T07:12:17.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.664+0000 7fc27a225700 1 -- 172.21.15.83:0/1184608024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fc274067f60 con 0x7fc27415bdd0 2023-11-26T07:12:17.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.668+0000 7fc25bfff700 1 -- 172.21.15.83:0/1184608024 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc26c058420 con 0x7fc27415bdd0 2023-11-26T07:12:17.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.670+0000 7fc27a225700 1 -- 172.21.15.83:0/1184608024 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 msgr2=0x7fc26006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:17.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.670+0000 7fc27a225700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 0x7fc26006d390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc2740cf390 tx=0x7fc268008040 comp rx=0 tx=0).stop 2023-11-26T07:12:17.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.670+0000 7fc27a225700 1 -- 172.21.15.83:0/1184608024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27415bdd0 msgr2=0x7fc27415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:17.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.670+0000 7fc27a225700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27415bdd0 0x7fc27415d510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00cd20 tx=0x7fc26c00a570 comp rx=0 tx=0).stop 2023-11-26T07:12:17.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.670+0000 7fc27a225700 1 -- 172.21.15.83:0/1184608024 shutdown_connections 2023-11-26T07:12:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.670+0000 7fc27a225700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27415bdd0 0x7fc27415d510 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.670+0000 7fc27a225700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2740b3370 0x7fc2740c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.670+0000 7fc27a225700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26006aed0 0x7fc26006d390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.670+0000 7fc27a225700 1 --2- 172.21.15.83:0/1184608024 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2740d1c70 0x7fc27415cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.670+0000 7fc27a225700 1 -- 172.21.15.83:0/1184608024 >> 172.21.15.83:0/1184608024 conn(0x7fc2740731a0 msgr2=0x7fc27409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.671+0000 7fc27a225700 1 -- 172.21.15.83:0/1184608024 shutdown_connections 2023-11-26T07:12:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.671+0000 7fc27a225700 1 -- 172.21.15.83:0/1184608024 wait complete. 2023-11-26T07:12:17.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-26T07:12:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.803+0000 7f5028f77700 1 Processor -- start 2023-11-26T07:12:17.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.803+0000 7f5028f77700 1 -- start start 2023-11-26T07:12:17.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5028f77700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f50240d1ae0 0x7f50240d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5028f77700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50240b1b20 0x7f50240b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5028f77700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50240cd060 con 0x7f502415f560 2023-11-26T07:12:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5028f77700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502415f940 con 0x7f50240b1b20 2023-11-26T07:12:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5028f77700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50240cdec0 con 0x7f50240d1ae0 2023-11-26T07:12:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5022d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50240b1b20 0x7f50240b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5022d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50240b1b20 0x7f50240b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35874/0 (socket says 172.21.15.83:35874) 2023-11-26T07:12:17.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5022d9d700 1 -- 172.21.15.83:0/1935087561 learned_addr learned my addr 172.21.15.83:0/1935087561 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5021d9b700 1 --2- 172.21.15.83:0/1935087561 >> v2:172.21.15.203:3300/0 conn(0x7f50240d1ae0 0x7f50240d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5022d9d700 1 -- 172.21.15.83:0/1935087561 >> v2:172.21.15.203:3300/0 conn(0x7f50240d1ae0 msgr2=0x7f50240d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5022d9d700 1 --2- 172.21.15.83:0/1935087561 >> v2:172.21.15.203:3300/0 conn(0x7f50240d1ae0 0x7f50240d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5022d9d700 1 -- 172.21.15.83:0/1935087561 >> v1:172.21.15.203:6789/0 conn(0x7f502415f560 legacy=0x7f50240d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5022d9d700 1 -- 172.21.15.83:0/1935087561 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50240b1420 con 0x7f50240b1b20 2023-11-26T07:12:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f5022d9d700 1 --2- 172.21.15.83:0/1935087561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50240b1b20 0x7f50240b1f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f502409b840 tx=0x7f501c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d2f0df383c6374 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.804+0000 7f502159a700 1 -- 172.21.15.83:0/1935087561 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00e070 con 0x7f50240b1b20 2023-11-26T07:12:17.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.805+0000 7f502159a700 1 -- 172.21.15.83:0/1935087561 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c00b910 con 0x7f50240b1b20 2023-11-26T07:12:17.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.805+0000 7f502159a700 1 -- 172.21.15.83:0/1935087561 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00bcc0 con 0x7f50240b1b20 2023-11-26T07:12:17.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.805+0000 7f5028f77700 1 -- 172.21.15.83:0/1935087561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50240b1b20 msgr2=0x7f50240b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:17.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.805+0000 7f5028f77700 1 --2- 172.21.15.83:0/1935087561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50240b1b20 0x7f50240b1f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f502409b840 tx=0x7f501c00b330 comp rx=0 tx=0).stop 2023-11-26T07:12:17.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.805+0000 7f5028f77700 1 -- 172.21.15.83:0/1935087561 shutdown_connections 2023-11-26T07:12:17.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.805+0000 7f5028f77700 1 --2- 172.21.15.83:0/1935087561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50240b1b20 0x7f50240b1f00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.805+0000 7f5028f77700 1 --2- 172.21.15.83:0/1935087561 >> v2:172.21.15.203:3300/0 conn(0x7f50240d1ae0 0x7f50240d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.805+0000 7f5028f77700 1 -- 172.21.15.83:0/1935087561 >> 172.21.15.83:0/1935087561 conn(0x7f502409aa00 msgr2=0x7f502409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:17.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.805+0000 7f5028f77700 1 -- 172.21.15.83:0/1935087561 shutdown_connections 2023-11-26T07:12:17.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.805+0000 7f5028f77700 1 -- 172.21.15.83:0/1935087561 wait complete. 2023-11-26T07:12:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.806+0000 7f5028f77700 1 Processor -- start 2023-11-26T07:12:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.807+0000 7f5028f77700 1 -- start start 2023-11-26T07:12:17.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.807+0000 7f5028f77700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50240b1b20 0x7f50240c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.807+0000 7f5028f77700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50240d1ae0 0x7f50240c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.807+0000 7f5028f77700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502415f560 0x7f50240c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.807+0000 7f5028f77700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50240c9c30 con 0x7f50240b1b20 2023-11-26T07:12:17.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.807+0000 7f5028f77700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50240c4fc0 con 0x7f50240d1ae0 2023-11-26T07:12:17.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.807+0000 7f5028f77700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50240c9e20 con 0x7f502415f560 2023-11-26T07:12:17.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.807+0000 7f502259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50240b1b20 0x7f50240c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.807+0000 7f502259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50240b1b20 0x7f50240c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35876/0 (socket says 172.21.15.83:35876) 2023-11-26T07:12:17.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.807+0000 7f502259c700 1 -- 172.21.15.83:0/915969429 learned_addr learned my addr 172.21.15.83:0/915969429 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:17.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f5021d9b700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50240d1ae0 0x7f50240c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f5022d9d700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502415f560 0x7f50240c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f502259c700 1 -- 172.21.15.83:0/915969429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502415f560 msgr2=0x7f50240c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:17.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f502259c700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502415f560 0x7f50240c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f502259c700 1 -- 172.21.15.83:0/915969429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50240d1ae0 msgr2=0x7f50240c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:17.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f502259c700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50240d1ae0 0x7f50240c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:17.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f502259c700 1 -- 172.21.15.83:0/915969429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c009b30 con 0x7f50240b1b20 2023-11-26T07:12:17.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f5022d9d700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502415f560 0x7f50240c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:17.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f5021d9b700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50240d1ae0 0x7f50240c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:17.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f502259c700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50240b1b20 0x7f50240c67c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f501400c4e0 tx=0x7f501400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f5012ffd700 1 -- 172.21.15.83:0/915969429 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50140052d0 con 0x7f50240b1b20 2023-11-26T07:12:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f5012ffd700 1 -- 172.21.15.83:0/915969429 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5014005430 con 0x7f50240b1b20 2023-11-26T07:12:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f5028f77700 1 -- 172.21.15.83:0/915969429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50240ca100 con 0x7f50240b1b20 2023-11-26T07:12:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f5012ffd700 1 -- 172.21.15.83:0/915969429 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50140038c0 con 0x7f50240b1b20 2023-11-26T07:12:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.808+0000 7f5028f77700 1 -- 172.21.15.83:0/915969429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502406b570 con 0x7f50240b1b20 2023-11-26T07:12:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.809+0000 7f5028f77700 1 -- 172.21.15.83:0/915969429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50240714a0 con 0x7f50240b1b20 2023-11-26T07:12:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.809+0000 7f5012ffd700 1 -- 172.21.15.83:0/915969429 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5014003a20 con 0x7f50240b1b20 2023-11-26T07:12:17.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.810+0000 7f5012ffd700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f500406aed0 0x7f500406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:17.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.810+0000 7f5012ffd700 1 -- 172.21.15.83:0/915969429 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5014088760 con 0x7f50240b1b20 2023-11-26T07:12:17.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.810+0000 7f5021d9b700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f500406aed0 0x7f500406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:17.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.810+0000 7f5021d9b700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f500406aed0 0x7f500406d390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5018009c30 tx=0x7f5018008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:17.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:17.814+0000 7f5012ffd700 1 -- 172.21.15.83:0/915969429 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5014053b60 con 0x7f50240b1b20 2023-11-26T07:12:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.084+0000 7f5028f77700 1 -- 172.21.15.83:0/915969429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f5024075310 con 0x7f50240b1b20 2023-11-26T07:12:18.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.088+0000 7f5012ffd700 1 -- 172.21.15.83:0/915969429 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5014057180 con 0x7f50240b1b20 2023-11-26T07:12:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 -- 172.21.15.83:0/915969429 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f500406aed0 msgr2=0x7f500406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f500406aed0 0x7f500406d390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5018009c30 tx=0x7f5018008040 comp rx=0 tx=0).stop 2023-11-26T07:12:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 -- 172.21.15.83:0/915969429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50240b1b20 msgr2=0x7f50240c67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50240b1b20 0x7f50240c67c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f501400c4e0 tx=0x7f501400c8a0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 -- 172.21.15.83:0/915969429 shutdown_connections 2023-11-26T07:12:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f500406aed0 0x7f500406d390 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502415f560 0x7f50240c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50240b1b20 0x7f50240c67c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 --2- 172.21.15.83:0/915969429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50240d1ae0 0x7f50240c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 -- 172.21.15.83:0/915969429 >> 172.21.15.83:0/915969429 conn(0x7f502409aa00 msgr2=0x7f50240b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 -- 172.21.15.83:0/915969429 shutdown_connections 2023-11-26T07:12:18.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.090+0000 7f5028f77700 1 -- 172.21.15.83:0/915969429 wait complete. 2023-11-26T07:12:18.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-26T07:12:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.223+0000 7f8d77322700 1 Processor -- start 2023-11-26T07:12:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.223+0000 7f8d77322700 1 -- start start 2023-11-26T07:12:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d77322700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d701446a0 0x7f8d700b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d77322700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7006c710 0x7f8d700b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d77322700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700af950 con 0x7f8d700adf00 2023-11-26T07:12:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d77322700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d70144a80 con 0x7f8d701446a0 2023-11-26T07:12:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d77322700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700afc30 con 0x7f8d7006c710 2023-11-26T07:12:18.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d750be700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d701446a0 0x7f8d700b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d748bd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8d700adf00 0x7f8d7006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55924/0 (socket says 172.21.15.83:55924) 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d750be700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8d701446a0 0x7f8d700b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35884/0 (socket says 172.21.15.83:35884) 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d748bd700 1 -- 172.21.15.83:0/4111958130 learned_addr learned my addr 172.21.15.83:0/4111958130 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d750be700 1 -- 172.21.15.83:0/4111958130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7006c710 msgr2=0x7f8d700b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d750be700 1 --2- 172.21.15.83:0/4111958130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7006c710 0x7f8d700b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d750be700 1 -- 172.21.15.83:0/4111958130 >> v1:172.21.15.83:6789/0 conn(0x7f8d700adf00 legacy=0x7f8d7006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d750be700 1 -- 172.21.15.83:0/4111958130 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700aff10 con 0x7f8d701446a0 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.224+0000 7f8d67fff700 1 -- 172.21.15.83:0/4111958130 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1291826551 0 0) 0x7f8d700af950 con 0x7f8d700adf00 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d750be700 1 --2- 172.21.15.83:0/4111958130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d701446a0 0x7f8d700b5ae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009840 tx=0x7f8d60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f22fa410641130 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d67fff700 1 -- 172.21.15.83:0/4111958130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60011830 con 0x7f8d701446a0 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d67fff700 1 -- 172.21.15.83:0/4111958130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d60011990 con 0x7f8d701446a0 2023-11-26T07:12:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d67fff700 1 -- 172.21.15.83:0/4111958130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60011d20 con 0x7f8d701446a0 2023-11-26T07:12:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d77322700 1 -- 172.21.15.83:0/4111958130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d701446a0 msgr2=0x7f8d700b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d77322700 1 --2- 172.21.15.83:0/4111958130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d701446a0 0x7f8d700b5ae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009840 tx=0x7f8d60009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d77322700 1 -- 172.21.15.83:0/4111958130 shutdown_connections 2023-11-26T07:12:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d77322700 1 --2- 172.21.15.83:0/4111958130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7006c710 0x7f8d700b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d77322700 1 --2- 172.21.15.83:0/4111958130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d701446a0 0x7f8d700b5ae0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d77322700 1 -- 172.21.15.83:0/4111958130 >> 172.21.15.83:0/4111958130 conn(0x7f8d70071b90 msgr2=0x7f8d70071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d77322700 1 -- 172.21.15.83:0/4111958130 shutdown_connections 2023-11-26T07:12:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.225+0000 7f8d77322700 1 -- 172.21.15.83:0/4111958130 wait complete. 2023-11-26T07:12:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.227+0000 7f8d77322700 1 Processor -- start 2023-11-26T07:12:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.227+0000 7f8d77322700 1 -- start start 2023-11-26T07:12:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.227+0000 7f8d77322700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7006c710 0x7f8d701431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.227+0000 7f8d77322700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d700adf00 0x7f8d701f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.227+0000 7f8d77322700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d701446a0 0x7f8d701f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.227+0000 7f8d77322700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7009c2d0 con 0x7f8d700adf00 2023-11-26T07:12:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.227+0000 7f8d77322700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7009b6c0 con 0x7f8d701446a0 2023-11-26T07:12:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.227+0000 7f8d77322700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7009b9a0 con 0x7f8d7006c710 2023-11-26T07:12:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.227+0000 7f8d748bd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d700adf00 0x7f8d701f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.227+0000 7f8d748bd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d700adf00 0x7f8d701f1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35890/0 (socket says 172.21.15.83:35890) 2023-11-26T07:12:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d748bd700 1 -- 172.21.15.83:0/3444289890 learned_addr learned my addr 172.21.15.83:0/3444289890 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d758bf700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d701446a0 0x7f8d701f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d750be700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7006c710 0x7f8d701431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d748bd700 1 -- 172.21.15.83:0/3444289890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7006c710 msgr2=0x7f8d701431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d748bd700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7006c710 0x7f8d701431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d748bd700 1 -- 172.21.15.83:0/3444289890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d701446a0 msgr2=0x7f8d701f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d748bd700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d701446a0 0x7f8d701f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d748bd700 1 -- 172.21.15.83:0/3444289890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d600094a0 con 0x7f8d700adf00 2023-11-26T07:12:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d748bd700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d700adf00 0x7f8d701f1e90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c4d0 tx=0x7f8d6c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d65ffb700 1 -- 172.21.15.83:0/3444289890 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c004c80 con 0x7f8d700adf00 2023-11-26T07:12:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d65ffb700 1 -- 172.21.15.83:0/3444289890 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c004e00 con 0x7f8d700adf00 2023-11-26T07:12:18.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d65ffb700 1 -- 172.21.15.83:0/3444289890 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c007310 con 0x7f8d700adf00 2023-11-26T07:12:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d77322700 1 -- 172.21.15.83:0/3444289890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7009bc80 con 0x7f8d700adf00 2023-11-26T07:12:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.228+0000 7f8d77322700 1 -- 172.21.15.83:0/3444289890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d701436e0 con 0x7f8d700adf00 2023-11-26T07:12:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.229+0000 7f8d77322700 1 -- 172.21.15.83:0/3444289890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d700753f0 con 0x7f8d700adf00 2023-11-26T07:12:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.229+0000 7f8d65ffb700 1 -- 172.21.15.83:0/3444289890 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d6c007470 con 0x7f8d700adf00 2023-11-26T07:12:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.230+0000 7f8d65ffb700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8d5406aed0 0x7f8d5406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.230+0000 7f8d65ffb700 1 -- 172.21.15.83:0/3444289890 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8d6c088fd0 con 0x7f8d700adf00 2023-11-26T07:12:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.230+0000 7f8d750be700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8d5406aed0 0x7f8d5406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.230+0000 7f8d750be700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8d5406aed0 0x7f8d5406d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8d600090b0 tx=0x7f8d6001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.234+0000 7f8d65ffb700 1 -- 172.21.15.83:0/3444289890 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d6c0543d0 con 0x7f8d700adf00 2023-11-26T07:12:18.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.506+0000 7f8d77322700 1 -- 172.21.15.83:0/3444289890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f8d70067f60 con 0x7f8d700adf00 2023-11-26T07:12:18.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.508+0000 7f8d65ffb700 1 -- 172.21.15.83:0/3444289890 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8d6c0579f0 con 0x7f8d700adf00 2023-11-26T07:12:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 -- 172.21.15.83:0/3444289890 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8d5406aed0 msgr2=0x7f8d5406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8d5406aed0 0x7f8d5406d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8d600090b0 tx=0x7f8d6001c040 comp rx=0 tx=0).stop 2023-11-26T07:12:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 -- 172.21.15.83:0/3444289890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d700adf00 msgr2=0x7f8d701f1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d700adf00 0x7f8d701f1e90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c4d0 tx=0x7f8d6c00c890 comp rx=0 tx=0).stop 2023-11-26T07:12:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 -- 172.21.15.83:0/3444289890 shutdown_connections 2023-11-26T07:12:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d701446a0 0x7f8d701f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8d5406aed0 0x7f8d5406d390 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7006c710 0x7f8d701431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 --2- 172.21.15.83:0/3444289890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d700adf00 0x7f8d701f1e90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 -- 172.21.15.83:0/3444289890 >> 172.21.15.83:0/3444289890 conn(0x7f8d70071b90 msgr2=0x7f8d70071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 -- 172.21.15.83:0/3444289890 shutdown_connections 2023-11-26T07:12:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.510+0000 7f8d77322700 1 -- 172.21.15.83:0/3444289890 wait complete. 2023-11-26T07:12:18.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-26T07:12:18.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.644+0000 7f53cc890700 1 Processor -- start 2023-11-26T07:12:18.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.644+0000 7f53cc890700 1 -- start start 2023-11-26T07:12:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53cc890700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815bdd0 0x7f53c81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53cc890700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53c80d1c70 0x7f53c80d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53cc890700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c80b22d0 con 0x7f53c80b3370 2023-11-26T07:12:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53cc890700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c80b2f50 con 0x7f53c80d1c70 2023-11-26T07:12:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53cc890700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80d2640 con 0x7f53c815bdd0 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c5d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f53c80b3370 0x7f53c80b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55936/0 (socket says 172.21.15.83:55936) 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c5d9b700 1 -- 172.21.15.83:0/708428899 learned_addr learned my addr 172.21.15.83:0/708428899 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c659c700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815bdd0 0x7f53c81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c6d9d700 1 --2- 172.21.15.83:0/708428899 >> v2:172.21.15.203:3300/0 conn(0x7f53c80d1c70 0x7f53c80d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815164165 0 0) 0x7f53c80b22d0 con 0x7f53c80b3370 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b00035b0 con 0x7f53c80b3370 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3867067642 0 0) 0x7f53b00035b0 con 0x7f53c80b3370 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815bdd0 msgr2=0x7f53c81604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815bdd0 0x7f53c81604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 >> v2:172.21.15.203:3300/0 conn(0x7f53c80d1c70 msgr2=0x7f53c80d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 --2- 172.21.15.83:0/708428899 >> v2:172.21.15.203:3300/0 conn(0x7f53c80d1c70 0x7f53c80d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c80b25b0 con 0x7f53c80b3370 2023-11-26T07:12:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f53bc001670 con 0x7f53c80b3370 2023-11-26T07:12:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 >> v1:172.21.15.83:6789/0 conn(0x7f53c80b3370 legacy=0x7f53c80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b0003a00 0x7f53b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 0x7f53b0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815bdd0 0x7f53b0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80b2f50 con 0x7f53b0004850 2023-11-26T07:12:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80d2640 con 0x7f53b0003a00 2023-11-26T07:12:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.645+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b000ba10 con 0x7f53c815bdd0 2023-11-26T07:12:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c5d9b700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 0x7f53b0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c659c700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b0003a00 0x7f53b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c6d9d700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815bdd0 0x7f53b0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c5d9b700 1 -- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815bdd0 msgr2=0x7f53b0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c5d9b700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815bdd0 0x7f53b0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c5d9b700 1 -- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b0003a00 msgr2=0x7f53b0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c5d9b700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b0003a00 0x7f53b0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c5d9b700 1 -- 172.21.15.83:0/708428899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b00035b0 con 0x7f53b0004850 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c6d9d700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c815bdd0 0x7f53b0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c5d9b700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 0x7f53b0004cd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c320 tx=0x7f53b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2160cc102e9dac6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc00ce60 con 0x7f53b0004850 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc005960 con 0x7f53b0004850 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.646+0000 7f53c559a700 1 -- 172.21.15.83:0/708428899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc005d10 con 0x7f53b0004850 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.647+0000 7f53cc890700 1 -- 172.21.15.83:0/708428899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 msgr2=0x7f53b0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.647+0000 7f53cc890700 1 --2- 172.21.15.83:0/708428899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 0x7f53b0004cd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00c320 tx=0x7f53b0005880 comp rx=0 tx=0).stop 2023-11-26T07:12:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.647+0000 7f53c6d9d700 1 -- 172.21.15.83:0/708428899 reap_dead start 2023-11-26T07:12:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.647+0000 7f53cc890700 1 -- 172.21.15.83:0/708428899 shutdown_connections 2023-11-26T07:12:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.647+0000 7f53cc890700 1 -- 172.21.15.83:0/708428899 >> 172.21.15.83:0/708428899 conn(0x7f53c80731a0 msgr2=0x7f53c809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.647+0000 7f53cc890700 1 -- 172.21.15.83:0/708428899 shutdown_connections 2023-11-26T07:12:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.647+0000 7f53cc890700 1 -- 172.21.15.83:0/708428899 wait complete. 2023-11-26T07:12:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.648+0000 7f53cc890700 1 Processor -- start 2023-11-26T07:12:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.648+0000 7f53cc890700 1 -- start start 2023-11-26T07:12:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.649+0000 7f53cc890700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 0x7f53c81f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.649+0000 7f53cc890700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c8148c30 0x7f53c81490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.649+0000 7f53cc890700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c80ca150 0x7f53c80ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.649+0000 7f53cc890700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80b77f0 con 0x7f53b0004850 2023-11-26T07:12:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.649+0000 7f53cc890700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c81f4eb0 con 0x7f53c8148c30 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.649+0000 7f53cc890700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80b4c80 con 0x7f53c80ca150 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.649+0000 7f53c659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 0x7f53c81f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.649+0000 7f53c659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 0x7f53c81f4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35908/0 (socket says 172.21.15.83:35908) 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53c659c700 1 -- 172.21.15.83:0/1509983891 learned_addr learned my addr 172.21.15.83:0/1509983891 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53c5d9b700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c8148c30 0x7f53c81490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53c6d9d700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c80ca150 0x7f53c80ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53c659c700 1 -- 172.21.15.83:0/1509983891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c80ca150 msgr2=0x7f53c80ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53c659c700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c80ca150 0x7f53c80ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53c659c700 1 -- 172.21.15.83:0/1509983891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c8148c30 msgr2=0x7f53c81490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53c659c700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c8148c30 0x7f53c81490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53c659c700 1 -- 172.21.15.83:0/1509983891 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c815eec0 con 0x7f53b0004850 2023-11-26T07:12:18.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53c5d9b700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c8148c30 0x7f53c81490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:18.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53c659c700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 0x7f53c81f4970 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f53b80119a0 tx=0x7f53b8011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:18.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53b6ffd700 1 -- 172.21.15.83:0/1509983891 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b800f940 con 0x7f53b0004850 2023-11-26T07:12:18.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53b6ffd700 1 -- 172.21.15.83:0/1509983891 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b800faa0 con 0x7f53b0004850 2023-11-26T07:12:18.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53b6ffd700 1 -- 172.21.15.83:0/1509983891 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b800fe30 con 0x7f53b0004850 2023-11-26T07:12:18.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53cc890700 1 -- 172.21.15.83:0/1509983891 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c815f1a0 con 0x7f53b0004850 2023-11-26T07:12:18.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.650+0000 7f53cc890700 1 -- 172.21.15.83:0/1509983891 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c80c9910 con 0x7f53b0004850 2023-11-26T07:12:18.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.651+0000 7f53cc890700 1 -- 172.21.15.83:0/1509983891 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c8067f60 con 0x7f53b0004850 2023-11-26T07:12:18.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.652+0000 7f53b6ffd700 1 -- 172.21.15.83:0/1509983891 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f53b8014440 con 0x7f53b0004850 2023-11-26T07:12:18.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.652+0000 7f53b6ffd700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f53a806aed0 0x7f53a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:18.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.652+0000 7f53b6ffd700 1 -- 172.21.15.83:0/1509983891 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f53b80898a0 con 0x7f53b0004850 2023-11-26T07:12:18.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.652+0000 7f53c5d9b700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f53a806aed0 0x7f53a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.652+0000 7f53c5d9b700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f53a806aed0 0x7f53a806d390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f53bc015040 tx=0x7f53bc00a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.656+0000 7f53b6ffd700 1 -- 172.21.15.83:0/1509983891 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53b8054d50 con 0x7f53b0004850 2023-11-26T07:12:18.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.926+0000 7f53cc890700 1 -- 172.21.15.83:0/1509983891 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f53c80b7c60 con 0x7f53b0004850 2023-11-26T07:12:18.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.928+0000 7f53b6ffd700 1 -- 172.21.15.83:0/1509983891 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f53b8058370 con 0x7f53b0004850 2023-11-26T07:12:18.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 -- 172.21.15.83:0/1509983891 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f53a806aed0 msgr2=0x7f53a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f53a806aed0 0x7f53a806d390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f53bc015040 tx=0x7f53bc00a000 comp rx=0 tx=0).stop 2023-11-26T07:12:18.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 -- 172.21.15.83:0/1509983891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 msgr2=0x7f53c81f4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:18.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 0x7f53c81f4970 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f53b80119a0 tx=0x7f53b8011d60 comp rx=0 tx=0).stop 2023-11-26T07:12:18.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 -- 172.21.15.83:0/1509983891 shutdown_connections 2023-11-26T07:12:18.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f53a806aed0 0x7f53a806d390 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53c80ca150 0x7f53c80ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b0004850 0x7f53c81f4970 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 --2- 172.21.15.83:0/1509983891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53c8148c30 0x7f53c81490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:18.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 -- 172.21.15.83:0/1509983891 >> 172.21.15.83:0/1509983891 conn(0x7f53c80731a0 msgr2=0x7f53c80cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:18.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 -- 172.21.15.83:0/1509983891 shutdown_connections 2023-11-26T07:12:18.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:18.930+0000 7f53cc890700 1 -- 172.21.15.83:0/1509983891 wait complete. 2023-11-26T07:12:18.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.063+0000 7feee99a3700 1 Processor -- start 2023-11-26T07:12:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.063+0000 7feee99a3700 1 -- start start 2023-11-26T07:12:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.063+0000 7feee99a3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee40b0fb0 0x7feee40cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.063+0000 7feee99a3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee40b1c60 con 0x7feee40d0280 2023-11-26T07:12:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.063+0000 7feee99a3700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee40d0720 con 0x7feee40b1390 2023-11-26T07:12:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.063+0000 7feee99a3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee40d0a00 con 0x7feee40b0fb0 2023-11-26T07:12:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee37fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7feee40d0280 0x7feee415d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55960/0 (socket says 172.21.15.83:55960) 2023-11-26T07:12:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee37fe700 1 -- 172.21.15.83:0/1786693318 learned_addr learned my addr 172.21.15.83:0/1786693318 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247145778 0 0) 0x7feee40b1c60 con 0x7feee40d0280 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feecc0035b0 con 0x7feee40d0280 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee2ffd700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee40b0fb0 0x7feee40cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 931976234 0 0) 0x7feecc0035b0 con 0x7feee40d0280 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee40b0fb0 msgr2=0x7feee40cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee40b0fb0 0x7feee40cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 >> v1:172.21.15.203:6789/0 conn(0x7feee40b1390 legacy=0x7feee406b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee40d0ce0 con 0x7feee40d0280 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7feedc0019f0 con 0x7feee40d0280 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 >> v1:172.21.15.83:6789/0 conn(0x7feee40d0280 legacy=0x7feee415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 0x7feecc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feecc004850 0x7feecc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee40b0fb0 0x7feecc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee40d0a00 con 0x7feecc003a30 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feecc00bb50 con 0x7feecc004850 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feecc00bd10 con 0x7feee40b0fb0 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee2ffd700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 0x7feecc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee27fc700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feecc004850 0x7feecc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee37fe700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee40b0fb0 0x7feecc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee2ffd700 1 -- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee40b0fb0 msgr2=0x7feecc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee2ffd700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee40b0fb0 0x7feecc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee2ffd700 1 -- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feecc004850 msgr2=0x7feecc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.064+0000 7feee2ffd700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feecc004850 0x7feecc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee2ffd700 1 -- 172.21.15.83:0/1786693318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc0035b0 con 0x7feecc003a30 2023-11-26T07:12:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee37fe700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee40b0fb0 0x7feecc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee2ffd700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 0x7feecc003e50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7feed4009b30 tx=0x7feecc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2cfade6760b109 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400e6c0 con 0x7feecc003a30 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed400e820 con 0x7feecc003a30 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee1ffb700 1 -- 172.21.15.83:0/1786693318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400eb90 con 0x7feecc003a30 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee99a3700 1 -- 172.21.15.83:0/1786693318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 msgr2=0x7feecc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee99a3700 1 --2- 172.21.15.83:0/1786693318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 0x7feecc003e50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7feed4009b30 tx=0x7feecc005880 comp rx=0 tx=0).stop 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee37fe700 1 -- 172.21.15.83:0/1786693318 reap_dead start 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee99a3700 1 -- 172.21.15.83:0/1786693318 shutdown_connections 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee99a3700 1 -- 172.21.15.83:0/1786693318 >> 172.21.15.83:0/1786693318 conn(0x7feee409aa00 msgr2=0x7feee409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee99a3700 1 -- 172.21.15.83:0/1786693318 shutdown_connections 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.065+0000 7feee99a3700 1 -- 172.21.15.83:0/1786693318 wait complete. 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.066+0000 7feee99a3700 1 Processor -- start 2023-11-26T07:12:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.067+0000 7feee99a3700 1 -- start start 2023-11-26T07:12:19.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.067+0000 7feee99a3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 0x7feee40b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.067+0000 7feee99a3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee41f7690 0x7feee41f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.067+0000 7feee99a3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feee41513a0 0x7feee4151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.067+0000 7feee99a3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee41f8050 con 0x7feecc003a30 2023-11-26T07:12:19.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.067+0000 7feee99a3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee40b4dd0 con 0x7feee41513a0 2023-11-26T07:12:19.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.067+0000 7feee99a3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee40b5070 con 0x7feee41f7690 2023-11-26T07:12:19.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.067+0000 7feee2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 0x7feee40b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.067+0000 7feee2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 0x7feee40b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35928/0 (socket says 172.21.15.83:35928) 2023-11-26T07:12:19.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee2ffd700 1 -- 172.21.15.83:0/4030876481 learned_addr learned my addr 172.21.15.83:0/4030876481 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:19.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee37fe700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feee41513a0 0x7feee4151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee27fc700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee41f7690 0x7feee41f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee2ffd700 1 -- 172.21.15.83:0/4030876481 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee41f7690 msgr2=0x7feee41f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee2ffd700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee41f7690 0x7feee41f7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee2ffd700 1 -- 172.21.15.83:0/4030876481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feee41513a0 msgr2=0x7feee4151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee2ffd700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feee41513a0 0x7feee4151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee2ffd700 1 -- 172.21.15.83:0/4030876481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee40b5350 con 0x7feecc003a30 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee27fc700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee41f7690 0x7feee41f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee2ffd700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 0x7feee40b4890 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feecc0047b0 tx=0x7feed40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feecbfff700 1 -- 172.21.15.83:0/4030876481 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4021070 con 0x7feecc003a30 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feecbfff700 1 -- 172.21.15.83:0/4030876481 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed40044a0 con 0x7feecc003a30 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feecbfff700 1 -- 172.21.15.83:0/4030876481 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400e6c0 con 0x7feecc003a30 2023-11-26T07:12:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee99a3700 1 -- 172.21.15.83:0/4030876481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee40b5630 con 0x7feecc003a30 2023-11-26T07:12:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.068+0000 7feee99a3700 1 -- 172.21.15.83:0/4030876481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee40b5bf0 con 0x7feecc003a30 2023-11-26T07:12:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.069+0000 7feee99a3700 1 -- 172.21.15.83:0/4030876481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee40b5db0 con 0x7feecc003a30 2023-11-26T07:12:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.069+0000 7feecbfff700 1 -- 172.21.15.83:0/4030876481 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feed4025070 con 0x7feecc003a30 2023-11-26T07:12:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.070+0000 7feecbfff700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feed006aed0 0x7feed006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.070+0000 7feecbfff700 1 -- 172.21.15.83:0/4030876481 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feed408a2d0 con 0x7feecc003a30 2023-11-26T07:12:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.070+0000 7feee27fc700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feed006aed0 0x7feed006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.070+0000 7feee27fc700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feed006aed0 0x7feed006d390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feee4074580 tx=0x7feed8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.074+0000 7feecbfff700 1 -- 172.21.15.83:0/4030876481 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feed4055760 con 0x7feecc003a30 2023-11-26T07:12:19.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.266+0000 7feee99a3700 1 -- 172.21.15.83:0/4030876481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feee4068480 con 0x7feecc003a30 2023-11-26T07:12:19.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.267+0000 7feecbfff700 1 -- 172.21.15.83:0/4030876481 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12845 (secure 0 0 0) 0x7feed4058d80 con 0x7feecc003a30 2023-11-26T07:12:19.269 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:19.269 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":30,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:15.997218+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:19.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 -- 172.21.15.83:0/4030876481 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feed006aed0 msgr2=0x7feed006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feed006aed0 0x7feed006d390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feee4074580 tx=0x7feed8009040 comp rx=0 tx=0).stop 2023-11-26T07:12:19.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 -- 172.21.15.83:0/4030876481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 msgr2=0x7feee40b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 0x7feee40b4890 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feecc0047b0 tx=0x7feed40047c0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 -- 172.21.15.83:0/4030876481 shutdown_connections 2023-11-26T07:12:19.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feee41513a0 0x7feee4151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feecc003a30 0x7feee40b4890 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feed006aed0 0x7feed006d390 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 --2- 172.21.15.83:0/4030876481 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feee41f7690 0x7feee41f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 -- 172.21.15.83:0/4030876481 >> 172.21.15.83:0/4030876481 conn(0x7feee409aa00 msgr2=0x7feee40d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:19.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 -- 172.21.15.83:0/4030876481 shutdown_connections 2023-11-26T07:12:19.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.269+0000 7feee99a3700 1 -- 172.21.15.83:0/4030876481 wait complete. 2023-11-26T07:12:19.283 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:12:19.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:12:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.404+0000 7fb5db769700 1 Processor -- start 2023-11-26T07:12:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.405+0000 7fb5db769700 1 -- start start 2023-11-26T07:12:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.405+0000 7fb5db769700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb5d415e080 0x7fb5d406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.405+0000 7fb5db769700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d406ba50 con 0x7fb5d40ce430 2023-11-26T07:12:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5db769700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d406b120 con 0x7fb5d40b4310 2023-11-26T07:12:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5db769700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d40d1200 con 0x7fb5d415e080 2023-11-26T07:12:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5d9d06700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb5d40ce430 0x7fb5d40cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:55978/0 (socket says 172.21.15.83:55978) 2023-11-26T07:12:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5d9d06700 1 -- 172.21.15.83:0/835924375 learned_addr learned my addr 172.21.15.83:0/835924375 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846267599 0 0) 0x7fb5d406ba50 con 0x7fb5d40ce430 2023-11-26T07:12:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5d9505700 1 --2- 172.21.15.83:0/835924375 >> v2:172.21.15.203:3300/0 conn(0x7fb5d415e080 0x7fb5d406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5c00035b0 con 0x7fb5d40ce430 2023-11-26T07:12:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2435330296 0 0) 0x7fb5c00035b0 con 0x7fb5d40ce430 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 >> v2:172.21.15.203:3300/0 conn(0x7fb5d415e080 msgr2=0x7fb5d406bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 --2- 172.21.15.83:0/835924375 >> v2:172.21.15.203:3300/0 conn(0x7fb5d415e080 0x7fb5d406bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 >> v1:172.21.15.203:6789/0 conn(0x7fb5d40b4310 legacy=0x7fb5d40d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d406b400 con 0x7fb5d40ce430 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fb5cc0019f0 con 0x7fb5d40ce430 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 >> v1:172.21.15.83:6789/0 conn(0x7fb5d40ce430 legacy=0x7fb5d40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c0003a00 0x7fb5c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5c0004850 0x7fb5c0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5c0005330 0x7fb5c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40d1200 con 0x7fb5c0003a00 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c000c1e0 con 0x7fb5c0005330 2023-11-26T07:12:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c000c3a0 con 0x7fb5c0004850 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.406+0000 7fb5d9505700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c0003a00 0x7fb5c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5d9d06700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5c0005330 0x7fb5c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5d8d04700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5c0004850 0x7fb5c0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5d9505700 1 -- 172.21.15.83:0/835924375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5c0004850 msgr2=0x7fb5c0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5d9505700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5c0004850 0x7fb5c0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5d9505700 1 -- 172.21.15.83:0/835924375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5c0005330 msgr2=0x7fb5c000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5d9505700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5c0005330 0x7fb5c000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5d9505700 1 -- 172.21.15.83:0/835924375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c00035b0 con 0x7fb5c0003a00 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5d9d06700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5c0005330 0x7fb5c000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5d9d06700 1 -- 172.21.15.83:0/835924375 reap_dead start 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5d9505700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c0003a00 0x7fb5c0003e00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009860 tx=0x7fb5c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af153e3c7b2567 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c400c070 con 0x7fb5c0003a00 2023-11-26T07:12:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c4013610 con 0x7fb5c0003a00 2023-11-26T07:12:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5cbfff700 1 -- 172.21.15.83:0/835924375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40139a0 con 0x7fb5c0003a00 2023-11-26T07:12:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5db769700 1 -- 172.21.15.83:0/835924375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c0003a00 msgr2=0x7fb5c0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.407+0000 7fb5db769700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c0003a00 0x7fb5c0003e00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009860 tx=0x7fb5c0005880 comp rx=0 tx=0).stop 2023-11-26T07:12:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.408+0000 7fb5db769700 1 -- 172.21.15.83:0/835924375 shutdown_connections 2023-11-26T07:12:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.408+0000 7fb5db769700 1 --2- 172.21.15.83:0/835924375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5c0003a00 0x7fb5c0003e00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.408+0000 7fb5db769700 1 -- 172.21.15.83:0/835924375 >> 172.21.15.83:0/835924375 conn(0x7fb5d40b3780 msgr2=0x7fb5d40b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.408+0000 7fb5db769700 1 -- 172.21.15.83:0/835924375 shutdown_connections 2023-11-26T07:12:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.408+0000 7fb5db769700 1 -- 172.21.15.83:0/835924375 wait complete. 2023-11-26T07:12:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.409+0000 7fb5db769700 1 Processor -- start 2023-11-26T07:12:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.409+0000 7fb5db769700 1 -- start start 2023-11-26T07:12:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5db769700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5c0003a00 0x7fb5d415e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5db769700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d40cbb20 0x7fb5d4159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5db769700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5d415bc20 0x7fb5d415c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5db769700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d4160b90 con 0x7fb5d40cbb20 2023-11-26T07:12:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5db769700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d415c5e0 con 0x7fb5d415bc20 2023-11-26T07:12:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5db769700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40c9dc0 con 0x7fb5c0003a00 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5d8d04700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d40cbb20 0x7fb5d4159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5d8d04700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d40cbb20 0x7fb5d4159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35948/0 (socket says 172.21.15.83:35948) 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5d8d04700 1 -- 172.21.15.83:0/4084747720 learned_addr learned my addr 172.21.15.83:0/4084747720 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5d9505700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5c0003a00 0x7fb5d415e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5d9d06700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5d415bc20 0x7fb5d415c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5d8d04700 1 -- 172.21.15.83:0/4084747720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5c0003a00 msgr2=0x7fb5d415e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.410+0000 7fb5d8d04700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5c0003a00 0x7fb5d415e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.411+0000 7fb5d8d04700 1 -- 172.21.15.83:0/4084747720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5d415bc20 msgr2=0x7fb5d415c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.411+0000 7fb5d8d04700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5d415bc20 0x7fb5d415c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.411+0000 7fb5d8d04700 1 -- 172.21.15.83:0/4084747720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40cbfa0 con 0x7fb5d40cbb20 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.411+0000 7fb5d9505700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5c0003a00 0x7fb5d415e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.411+0000 7fb5d9d06700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5d415bc20 0x7fb5d415c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.411+0000 7fb5d8d04700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d40cbb20 0x7fb5d4159130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000e5d0 tx=0x7fb5d000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.411+0000 7fb5c9ffb700 1 -- 172.21.15.83:0/4084747720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00135f0 con 0x7fb5d40cbb20 2023-11-26T07:12:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.411+0000 7fb5c9ffb700 1 -- 172.21.15.83:0/4084747720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d0013770 con 0x7fb5d40cbb20 2023-11-26T07:12:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.411+0000 7fb5db769700 1 -- 172.21.15.83:0/4084747720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d4159670 con 0x7fb5d40cbb20 2023-11-26T07:12:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.411+0000 7fb5c9ffb700 1 -- 172.21.15.83:0/4084747720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0013b20 con 0x7fb5d40cbb20 2023-11-26T07:12:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.411+0000 7fb5db769700 1 -- 172.21.15.83:0/4084747720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d4160da0 con 0x7fb5d40cbb20 2023-11-26T07:12:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.412+0000 7fb5db769700 1 -- 172.21.15.83:0/4084747720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d4068fb0 con 0x7fb5d40cbb20 2023-11-26T07:12:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.412+0000 7fb5c9ffb700 1 -- 172.21.15.83:0/4084747720 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb5d0013c80 con 0x7fb5d40cbb20 2023-11-26T07:12:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.413+0000 7fb5c9ffb700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb5b806aed0 0x7fb5b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.413+0000 7fb5d9505700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb5b806aed0 0x7fb5b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.413+0000 7fb5c9ffb700 1 -- 172.21.15.83:0/4084747720 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb5d00893a0 con 0x7fb5d40cbb20 2023-11-26T07:12:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.413+0000 7fb5d9505700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb5b806aed0 0x7fb5b806d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4000c00 tx=0x7fb5c400c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.417+0000 7fb5c9ffb700 1 -- 172.21.15.83:0/4084747720 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb5d00547a0 con 0x7fb5d40cbb20 2023-11-26T07:12:19.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.605+0000 7fb5db769700 1 -- 172.21.15.83:0/4084747720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5d40c9f00 con 0x7fb5d40cbb20 2023-11-26T07:12:19.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.606+0000 7fb5c9ffb700 1 -- 172.21.15.83:0/4084747720 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5d0057dc0 con 0x7fb5d40cbb20 2023-11-26T07:12:19.607 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:12:19.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 -- 172.21.15.83:0/4084747720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb5b806aed0 msgr2=0x7fb5b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb5b806aed0 0x7fb5b806d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4000c00 tx=0x7fb5c400c000 comp rx=0 tx=0).stop 2023-11-26T07:12:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 -- 172.21.15.83:0/4084747720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d40cbb20 msgr2=0x7fb5d4159130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d40cbb20 0x7fb5d4159130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000e5d0 tx=0x7fb5d000e990 comp rx=0 tx=0).stop 2023-11-26T07:12:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 -- 172.21.15.83:0/4084747720 shutdown_connections 2023-11-26T07:12:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5d415bc20 0x7fb5d415c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb5b806aed0 0x7fb5b806d390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb5c0003a00 0x7fb5d415e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 --2- 172.21.15.83:0/4084747720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5d40cbb20 0x7fb5d4159130 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 -- 172.21.15.83:0/4084747720 >> 172.21.15.83:0/4084747720 conn(0x7fb5d40b3780 msgr2=0x7fb5d40cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:19.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 -- 172.21.15.83:0/4084747720 shutdown_connections 2023-11-26T07:12:19.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.608+0000 7fb5db769700 1 -- 172.21.15.83:0/4084747720 wait complete. 2023-11-26T07:12:19.621 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:12:19.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.739+0000 7f13de1fa700 1 Processor -- start 2023-11-26T07:12:19.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.739+0000 7f13de1fa700 1 -- start start 2023-11-26T07:12:19.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.740+0000 7f13de1fa700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13d8152cc0 0x7f13d806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.740+0000 7f13de1fa700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13d80b49a0 0x7f13d806c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.740+0000 7f13de1fa700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d815c4b0 con 0x7f13d80cff20 2023-11-26T07:12:19.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.740+0000 7f13de1fa700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d815c790 con 0x7f13d8152cc0 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.740+0000 7f13de1fa700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d815ca70 con 0x7f13d80b49a0 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.740+0000 7f13d77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13d8152cc0 0x7f13d806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.740+0000 7f13d77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f13d8152cc0 0x7f13d806c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35956/0 (socket says 172.21.15.83:35956) 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.740+0000 7f13d77fe700 1 -- 172.21.15.83:0/376185856 learned_addr learned my addr 172.21.15.83:0/376185856 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.740+0000 7f13d6ffd700 1 --2- 172.21.15.83:0/376185856 >> v2:172.21.15.203:3300/0 conn(0x7f13d80b49a0 0x7f13d806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13d77fe700 1 -- 172.21.15.83:0/376185856 >> v2:172.21.15.203:3300/0 conn(0x7f13d80b49a0 msgr2=0x7f13d806c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13d77fe700 1 --2- 172.21.15.83:0/376185856 >> v2:172.21.15.203:3300/0 conn(0x7f13d80b49a0 0x7f13d806c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13d77fe700 1 -- 172.21.15.83:0/376185856 >> v1:172.21.15.203:6789/0 conn(0x7f13d80cff20 legacy=0x7f13d80d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13d77fe700 1 -- 172.21.15.83:0/376185856 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d815cd50 con 0x7f13d8152cc0 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13d6ffd700 1 --2- 172.21.15.83:0/376185856 >> v2:172.21.15.203:3300/0 conn(0x7f13d80b49a0 0x7f13d806c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13d77fe700 1 --2- 172.21.15.83:0/376185856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d8152cc0 0x7f13d806c000 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f13d80b4e50 tx=0x7f13c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b05cc282727d5b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:19.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13d67fc700 1 -- 172.21.15.83:0/376185856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c000b870 con 0x7f13d8152cc0 2023-11-26T07:12:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13d67fc700 1 -- 172.21.15.83:0/376185856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c000b9d0 con 0x7f13d8152cc0 2023-11-26T07:12:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13d67fc700 1 -- 172.21.15.83:0/376185856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c000bd60 con 0x7f13d8152cc0 2023-11-26T07:12:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13de1fa700 1 -- 172.21.15.83:0/376185856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d8152cc0 msgr2=0x7f13d806c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13de1fa700 1 --2- 172.21.15.83:0/376185856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d8152cc0 0x7f13d806c000 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f13d80b4e50 tx=0x7f13c0009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13de1fa700 1 -- 172.21.15.83:0/376185856 shutdown_connections 2023-11-26T07:12:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13de1fa700 1 --2- 172.21.15.83:0/376185856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d8152cc0 0x7f13d806c000 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13de1fa700 1 --2- 172.21.15.83:0/376185856 >> v2:172.21.15.203:3300/0 conn(0x7f13d80b49a0 0x7f13d806c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.741+0000 7f13de1fa700 1 -- 172.21.15.83:0/376185856 >> 172.21.15.83:0/376185856 conn(0x7f13d8071b90 msgr2=0x7f13d8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.742+0000 7f13de1fa700 1 -- 172.21.15.83:0/376185856 shutdown_connections 2023-11-26T07:12:19.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.742+0000 7f13de1fa700 1 -- 172.21.15.83:0/376185856 wait complete. 2023-11-26T07:12:19.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.743+0000 7f13de1fa700 1 Processor -- start 2023-11-26T07:12:19.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.743+0000 7f13de1fa700 1 -- start start 2023-11-26T07:12:19.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13de1fa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d80b49a0 0x7f13d8155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13de1fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d80cff20 0x7f13d81476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13de1fa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d8152cc0 0x7f13d815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13de1fa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d815e4d0 con 0x7f13d80b49a0 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13de1fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8147c30 con 0x7f13d80cff20 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13de1fa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d815dc30 con 0x7f13d8152cc0 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d80b49a0 0x7f13d8155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d80b49a0 0x7f13d8155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35962/0 (socket says 172.21.15.83:35962) 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d77fe700 1 -- 172.21.15.83:0/2994585294 learned_addr learned my addr 172.21.15.83:0/2994585294 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d7fff700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d8152cc0 0x7f13d815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d6ffd700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d80cff20 0x7f13d81476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d77fe700 1 -- 172.21.15.83:0/2994585294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d8152cc0 msgr2=0x7f13d815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d77fe700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d8152cc0 0x7f13d815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d77fe700 1 -- 172.21.15.83:0/2994585294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d80cff20 msgr2=0x7f13d81476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d77fe700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d80cff20 0x7f13d81476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d77fe700 1 -- 172.21.15.83:0/2994585294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c0009550 con 0x7f13d80b49a0 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d7fff700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d8152cc0 0x7f13d815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d6ffd700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d80cff20 0x7f13d81476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13d77fe700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d80b49a0 0x7f13d8155760 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f13c0000c00 tx=0x7f13c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13cffff700 1 -- 172.21.15.83:0/2994585294 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0004520 con 0x7f13d80b49a0 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13cffff700 1 -- 172.21.15.83:0/2994585294 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c001e070 con 0x7f13d80b49a0 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13de1fa700 1 -- 172.21.15.83:0/2994585294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d815df10 con 0x7f13d80b49a0 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13cffff700 1 -- 172.21.15.83:0/2994585294 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00038a0 con 0x7f13d80b49a0 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.744+0000 7f13de1fa700 1 -- 172.21.15.83:0/2994585294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d81533e0 con 0x7f13d80b49a0 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.745+0000 7f13de1fa700 1 -- 172.21.15.83:0/2994585294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d8153750 con 0x7f13d80b49a0 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.746+0000 7f13cffff700 1 -- 172.21.15.83:0/2994585294 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f13c0003a00 con 0x7f13d80b49a0 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.746+0000 7f13cffff700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13b806aed0 0x7f13b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:19.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.746+0000 7f13cffff700 1 -- 172.21.15.83:0/2994585294 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f13c0020070 con 0x7f13d80b49a0 2023-11-26T07:12:19.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.746+0000 7f13d6ffd700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13b806aed0 0x7f13b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:19.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.746+0000 7f13d6ffd700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13b806aed0 0x7f13b806d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13d81563d0 tx=0x7f13c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:19.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.751+0000 7f13cffff700 1 -- 172.21.15.83:0/2994585294 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13c0055d30 con 0x7f13d80b49a0 2023-11-26T07:12:19.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.943+0000 7f13de1fa700 1 -- 172.21.15.83:0/2994585294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f13d80753f0 con 0x7f13d80b49a0 2023-11-26T07:12:19.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.944+0000 7f13cffff700 1 -- 172.21.15.83:0/2994585294 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f13c0015090 con 0x7f13d80b49a0 2023-11-26T07:12:19.946 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:19.946 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:12:19.946 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:19.946 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2023-11-26T07:12:19.946 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2023-11-26T07:12:19.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:19.946 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:12:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 -- 172.21.15.83:0/2994585294 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13b806aed0 msgr2=0x7f13b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13b806aed0 0x7f13b806d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f13d81563d0 tx=0x7f13c8008040 comp rx=0 tx=0).stop 2023-11-26T07:12:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 -- 172.21.15.83:0/2994585294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d80b49a0 msgr2=0x7f13d8155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d80b49a0 0x7f13d8155760 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f13c0000c00 tx=0x7f13c0004750 comp rx=0 tx=0).stop 2023-11-26T07:12:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 -- 172.21.15.83:0/2994585294 shutdown_connections 2023-11-26T07:12:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13b806aed0 0x7f13b806d390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13d8152cc0 0x7f13d815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13d80b49a0 0x7f13d8155760 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 --2- 172.21.15.83:0/2994585294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13d80cff20 0x7f13d81476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:19.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 -- 172.21.15.83:0/2994585294 >> 172.21.15.83:0/2994585294 conn(0x7f13d8071b90 msgr2=0x7f13d815b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:19.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 -- 172.21.15.83:0/2994585294 shutdown_connections 2023-11-26T07:12:19.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:19.946+0000 7f13de1fa700 1 -- 172.21.15.83:0/2994585294 wait complete. 2023-11-26T07:12:19.959 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:12:19.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:12:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.082+0000 7f3c30e15700 1 Processor -- start 2023-11-26T07:12:20.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.082+0000 7f3c30e15700 1 -- start start 2023-11-26T07:12:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c30e15700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c2c06e830 0x7f3c2c06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c30e15700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2c06f1c0 con 0x7f3c2c06f480 2023-11-26T07:12:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c30e15700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2c06c9b0 con 0x7f3c2c144600 2023-11-26T07:12:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c30e15700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c2c06aff0 con 0x7f3c2c06e830 2023-11-26T07:12:20.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c29d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3c2c06f480 0x7f3c2c06c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56012/0 (socket says 172.21.15.83:56012) 2023-11-26T07:12:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c29d9b700 1 -- 172.21.15.83:0/1926329943 learned_addr learned my addr 172.21.15.83:0/1926329943 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2ad9d700 1 --2- 172.21.15.83:0/1926329943 >> v2:172.21.15.203:3300/0 conn(0x7f3c2c06e830 0x7f3c2c06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661081072 0 0) 0x7f3c20001110 con 0x7f3c2c06f480 2023-11-26T07:12:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c140035b0 con 0x7f3c2c06f480 2023-11-26T07:12:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 824150796 0 0) 0x7f3c140035b0 con 0x7f3c2c06f480 2023-11-26T07:12:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 >> v2:172.21.15.203:3300/0 conn(0x7f3c2c06e830 msgr2=0x7f3c2c06ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 --2- 172.21.15.83:0/1926329943 >> v2:172.21.15.203:3300/0 conn(0x7f3c2c06e830 0x7f3c2c06ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 >> v1:172.21.15.203:6789/0 conn(0x7f3c2c144600 legacy=0x7f3c2c0b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c06b200 con 0x7f3c2c06f480 2023-11-26T07:12:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f3c20001800 con 0x7f3c2c06f480 2023-11-26T07:12:20.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 >> v1:172.21.15.83:6789/0 conn(0x7f3c2c06f480 legacy=0x7f3c2c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c14003a00 0x7f3c14003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 0x7f3c14004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.083+0000 7f3c2959a700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c14005210 0x7f3c1400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c06aff0 con 0x7f3c14004850 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1400c1b0 con 0x7f3c14005210 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1400c370 con 0x7f3c14003a00 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c29d9b700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 0x7f3c14004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c2ad9d700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c14005210 0x7f3c1400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c2a59c700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c14003a00 0x7f3c14003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c29d9b700 1 -- 172.21.15.83:0/1926329943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c14003a00 msgr2=0x7f3c14003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c29d9b700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c14003a00 0x7f3c14003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:20.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c29d9b700 1 -- 172.21.15.83:0/1926329943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c14005210 msgr2=0x7f3c1400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c29d9b700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c14005210 0x7f3c1400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c2ad9d700 1 -- 172.21.15.83:0/1926329943 reap_dead start 2023-11-26T07:12:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c29d9b700 1 -- 172.21.15.83:0/1926329943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c140035b0 con 0x7f3c14004850 2023-11-26T07:12:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c2a59c700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c14003a00 0x7f3c14003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c29d9b700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 0x7f3c14004cd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c240 tx=0x7f3c14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e289e25e0ad8659 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20007db0 con 0x7f3c14004850 2023-11-26T07:12:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2000cd80 con 0x7f3c14004850 2023-11-26T07:12:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.084+0000 7f3c2959a700 1 -- 172.21.15.83:0/1926329943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20005ba0 con 0x7f3c14004850 2023-11-26T07:12:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.085+0000 7f3c30e15700 1 -- 172.21.15.83:0/1926329943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 msgr2=0x7f3c14004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.085+0000 7f3c30e15700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 0x7f3c14004cd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c240 tx=0x7f3c14005880 comp rx=0 tx=0).stop 2023-11-26T07:12:20.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.085+0000 7f3c30e15700 1 -- 172.21.15.83:0/1926329943 shutdown_connections 2023-11-26T07:12:20.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.085+0000 7f3c30e15700 1 --2- 172.21.15.83:0/1926329943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 0x7f3c14004cd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:20.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.085+0000 7f3c30e15700 1 -- 172.21.15.83:0/1926329943 >> 172.21.15.83:0/1926329943 conn(0x7f3c2c0d0b40 msgr2=0x7f3c2c0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:20.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.085+0000 7f3c30e15700 1 -- 172.21.15.83:0/1926329943 shutdown_connections 2023-11-26T07:12:20.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.085+0000 7f3c30e15700 1 -- 172.21.15.83:0/1926329943 wait complete. 2023-11-26T07:12:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.086+0000 7f3c30e15700 1 Processor -- start 2023-11-26T07:12:20.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.086+0000 7f3c30e15700 1 -- start start 2023-11-26T07:12:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.087+0000 7f3c30e15700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 0x7f3c2c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.087+0000 7f3c30e15700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c2c15c290 0x7f3c2c0aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.087+0000 7f3c30e15700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c2c0a9cd0 0x7f3c2c0aa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.087+0000 7f3c30e15700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c15d410 con 0x7f3c14004850 2023-11-26T07:12:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.087+0000 7f3c30e15700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c0aa6c0 con 0x7f3c2c15c290 2023-11-26T07:12:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.087+0000 7f3c30e15700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c0ab270 con 0x7f3c2c0a9cd0 2023-11-26T07:12:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.087+0000 7f3c2a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 0x7f3c2c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.087+0000 7f3c2a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 0x7f3c2c15d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35982/0 (socket says 172.21.15.83:35982) 2023-11-26T07:12:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c2a59c700 1 -- 172.21.15.83:0/1889459157 learned_addr learned my addr 172.21.15.83:0/1889459157 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c29d9b700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c2c15c290 0x7f3c2c0aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c2ad9d700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c2c0a9cd0 0x7f3c2c0aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c2a59c700 1 -- 172.21.15.83:0/1889459157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c2c0a9cd0 msgr2=0x7f3c2c0aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c2a59c700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c2c0a9cd0 0x7f3c2c0aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c2a59c700 1 -- 172.21.15.83:0/1889459157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c2c15c290 msgr2=0x7f3c2c0aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c2a59c700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c2c15c290 0x7f3c2c0aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c2a59c700 1 -- 172.21.15.83:0/1889459157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c15c790 con 0x7f3c14004850 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c2ad9d700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c2c0a9cd0 0x7f3c2c0aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c29d9b700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c2c15c290 0x7f3c2c0aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c2a59c700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 0x7f3c2c15d830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00e550 tx=0x7f3c1c00e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c1affd700 1 -- 172.21.15.83:0/1889459157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0052e0 con 0x7f3c14004850 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c1affd700 1 -- 172.21.15.83:0/1889459157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1c005440 con 0x7f3c14004850 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c30e15700 1 -- 172.21.15.83:0/1889459157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c15ca70 con 0x7f3c14004850 2023-11-26T07:12:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c1affd700 1 -- 172.21.15.83:0/1889459157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0038c0 con 0x7f3c14004850 2023-11-26T07:12:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.088+0000 7f3c30e15700 1 -- 172.21.15.83:0/1889459157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2c15d030 con 0x7f3c14004850 2023-11-26T07:12:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.089+0000 7f3c30e15700 1 -- 172.21.15.83:0/1889459157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2c15d680 con 0x7f3c14004850 2023-11-26T07:12:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.089+0000 7f3c1affd700 1 -- 172.21.15.83:0/1889459157 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c1c003a20 con 0x7f3c14004850 2023-11-26T07:12:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.090+0000 7f3c1affd700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c0c06aed0 0x7f3c0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.090+0000 7f3c1affd700 1 -- 172.21.15.83:0/1889459157 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3c1c0888e0 con 0x7f3c14004850 2023-11-26T07:12:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.090+0000 7f3c29d9b700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c0c06aed0 0x7f3c0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.090+0000 7f3c29d9b700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c0c06aed0 0x7f3c0c06d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0cd4a0 tx=0x7f3c20007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.096+0000 7f3c1affd700 1 -- 172.21.15.83:0/1889459157 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c1c053e20 con 0x7f3c14004850 2023-11-26T07:12:20.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:20.284+0000 7f3c30e15700 1 -- 172.21.15.83:0/1889459157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3c2c067f60 con 0x7f3c14004850 2023-11-26T07:12:21.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.095+0000 7f3c1affd700 1 -- 172.21.15.83:0/1889459157 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3c1c0190b0 con 0x7f3c14004850 2023-11-26T07:12:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 -- 172.21.15.83:0/1889459157 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c0c06aed0 msgr2=0x7f3c0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c0c06aed0 0x7f3c0c06d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0cd4a0 tx=0x7f3c20007000 comp rx=0 tx=0).stop 2023-11-26T07:12:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 -- 172.21.15.83:0/1889459157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 msgr2=0x7f3c2c15d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:21.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 0x7f3c2c15d830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00e550 tx=0x7f3c1c00e910 comp rx=0 tx=0).stop 2023-11-26T07:12:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 -- 172.21.15.83:0/1889459157 shutdown_connections 2023-11-26T07:12:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c0c06aed0 0x7f3c0c06d390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:21.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c2c0a9cd0 0x7f3c2c0aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c14004850 0x7f3c2c15d830 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 --2- 172.21.15.83:0/1889459157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c2c15c290 0x7f3c2c0aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 -- 172.21.15.83:0/1889459157 >> 172.21.15.83:0/1889459157 conn(0x7f3c2c0d0b40 msgr2=0x7f3c2c0b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 -- 172.21.15.83:0/1889459157 shutdown_connections 2023-11-26T07:12:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.098+0000 7f3c30e15700 1 -- 172.21.15.83:0/1889459157 wait complete. 2023-11-26T07:12:21.101 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2023-11-26T07:12:21.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:12:21.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.249+0000 7f182e809700 1 Processor -- start 2023-11-26T07:12:21.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.249+0000 7f182e809700 1 -- start start 2023-11-26T07:12:21.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.250+0000 7f182e809700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f182815f500 0x7f182806fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:21.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.250+0000 7f182e809700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1828070b40 0x7f182806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:21.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.250+0000 7f182e809700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18280703a0 con 0x7f18280cc580 2023-11-26T07:12:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.250+0000 7f182e809700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1828070820 con 0x7f182815f500 2023-11-26T07:12:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.250+0000 7f182e809700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182806fb80 con 0x7f1828070b40 2023-11-26T07:12:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.250+0000 7f1827fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f182815f500 0x7f182806fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.250+0000 7f182cda6700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f18280cc580 0x7f182806f2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56034/0 (socket says 172.21.15.83:56034) 2023-11-26T07:12:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.250+0000 7f1827fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f182815f500 0x7f182806fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35990/0 (socket says 172.21.15.83:35990) 2023-11-26T07:12:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.250+0000 7f182cda6700 1 -- 172.21.15.83:0/1561657479 learned_addr learned my addr 172.21.15.83:0/1561657479 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.250+0000 7f18277fe700 1 --2- 172.21.15.83:0/1561657479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1828070b40 0x7f182806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f1827fff700 1 -- 172.21.15.83:0/1561657479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1828070b40 msgr2=0x7f182806ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f1827fff700 1 --2- 172.21.15.83:0/1561657479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1828070b40 0x7f182806ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:21.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f1827fff700 1 -- 172.21.15.83:0/1561657479 >> v1:172.21.15.83:6789/0 conn(0x7f18280cc580 legacy=0x7f182806f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f1827fff700 1 -- 172.21.15.83:0/1561657479 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18280d0260 con 0x7f182815f500 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f1826ffd700 1 -- 172.21.15.83:0/1561657479 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587029848 0 0) 0x7f18280703a0 con 0x7f18280cc580 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f18277fe700 1 --2- 172.21.15.83:0/1561657479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1828070b40 0x7f182806ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f1827fff700 1 --2- 172.21.15.83:0/1561657479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f182815f500 0x7f182806fe60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1810009840 tx=0x7f1810009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb4b0e178c1f789 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f1826ffd700 1 -- 172.21.15.83:0/1561657479 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810011830 con 0x7f182815f500 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f1826ffd700 1 -- 172.21.15.83:0/1561657479 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1810011990 con 0x7f182815f500 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f1826ffd700 1 -- 172.21.15.83:0/1561657479 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810011d20 con 0x7f182815f500 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f182e809700 1 -- 172.21.15.83:0/1561657479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f182815f500 msgr2=0x7f182806fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f182e809700 1 --2- 172.21.15.83:0/1561657479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f182815f500 0x7f182806fe60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1810009840 tx=0x7f1810009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f182e809700 1 -- 172.21.15.83:0/1561657479 shutdown_connections 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f182e809700 1 --2- 172.21.15.83:0/1561657479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f182815f500 0x7f182806fe60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f182e809700 1 --2- 172.21.15.83:0/1561657479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1828070b40 0x7f182806ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:21.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f182e809700 1 -- 172.21.15.83:0/1561657479 >> 172.21.15.83:0/1561657479 conn(0x7f18280b3710 msgr2=0x7f18280b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:21.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.251+0000 7f182e809700 1 -- 172.21.15.83:0/1561657479 shutdown_connections 2023-11-26T07:12:21.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.252+0000 7f182e809700 1 -- 172.21.15.83:0/1561657479 wait complete. 2023-11-26T07:12:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.253+0000 7f182e809700 1 Processor -- start 2023-11-26T07:12:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.253+0000 7f182e809700 1 -- start start 2023-11-26T07:12:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f182e809700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1828070b40 0x7f18280b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f182e809700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18280cc580 0x7f18280b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:21.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f182e809700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f182815f500 0x7f1828153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f182e809700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18280b0b00 con 0x7f1828070b40 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f182e809700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18280b0dd0 con 0x7f182815f500 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f182e809700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18280b10b0 con 0x7f18280cc580 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1827fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1828070b40 0x7f18280b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1827fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1828070b40 0x7f18280b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35996/0 (socket says 172.21.15.83:35996) 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1827fff700 1 -- 172.21.15.83:0/1166535227 learned_addr learned my addr 172.21.15.83:0/1166535227 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f18277fe700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18280cc580 0x7f18280b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f182cda6700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f182815f500 0x7f1828153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1827fff700 1 -- 172.21.15.83:0/1166535227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18280cc580 msgr2=0x7f18280b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1827fff700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18280cc580 0x7f18280b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1827fff700 1 -- 172.21.15.83:0/1166535227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f182815f500 msgr2=0x7f1828153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1827fff700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f182815f500 0x7f1828153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:21.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1827fff700 1 -- 172.21.15.83:0/1166535227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18100094a0 con 0x7f1828070b40 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f182cda6700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f182815f500 0x7f1828153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1827fff700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1828070b40 0x7f18280b1e70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1810014040 tx=0x7f1810004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1824ff9700 1 -- 172.21.15.83:0/1166535227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18100038a0 con 0x7f1828070b40 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1824ff9700 1 -- 172.21.15.83:0/1166535227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181001e070 con 0x7f1828070b40 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f182e809700 1 -- 172.21.15.83:0/1166535227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182815fa60 con 0x7f1828070b40 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f1824ff9700 1 -- 172.21.15.83:0/1166535227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810003c40 con 0x7f1828070b40 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.254+0000 7f182e809700 1 -- 172.21.15.83:0/1166535227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18280b1670 con 0x7f1828070b40 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.255+0000 7f182e809700 1 -- 172.21.15.83:0/1166535227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18280b23b0 con 0x7f1828070b40 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.256+0000 7f1824ff9700 1 -- 172.21.15.83:0/1166535227 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f181002d450 con 0x7f1828070b40 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.256+0000 7f1824ff9700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f180806aed0 0x7f180806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.256+0000 7f1824ff9700 1 -- 172.21.15.83:0/1166535227 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f181008a560 con 0x7f1828070b40 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.256+0000 7f18277fe700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f180806aed0 0x7f180806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:21.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.257+0000 7f18277fe700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f180806aed0 0x7f180806d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f18281f5080 tx=0x7f1818008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:21.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.261+0000 7f1824ff9700 1 -- 172.21.15.83:0/1166535227 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1810055880 con 0x7f1828070b40 2023-11-26T07:12:21.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:21.450+0000 7f182e809700 1 -- 172.21.15.83:0/1166535227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1828068fb0 con 0x7f1828070b40 2023-11-26T07:12:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.100+0000 7f1824ff9700 1 -- 172.21.15.83:0/1166535227 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f18100200d0 con 0x7f1828070b40 2023-11-26T07:12:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.105+0000 7f182e809700 1 -- 172.21.15.83:0/1166535227 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f180806aed0 msgr2=0x7f180806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.105+0000 7f182e809700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f180806aed0 0x7f180806d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f18281f5080 tx=0x7f1818008040 comp rx=0 tx=0).stop 2023-11-26T07:12:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.105+0000 7f182e809700 1 -- 172.21.15.83:0/1166535227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1828070b40 msgr2=0x7f18280b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.105+0000 7f182e809700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1828070b40 0x7f18280b1e70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1810014040 tx=0x7f1810004750 comp rx=0 tx=0).stop 2023-11-26T07:12:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.105+0000 7f182e809700 1 -- 172.21.15.83:0/1166535227 shutdown_connections 2023-11-26T07:12:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.105+0000 7f182e809700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f182815f500 0x7f1828153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.105+0000 7f182e809700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1828070b40 0x7f18280b1e70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.105+0000 7f182e809700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f180806aed0 0x7f180806d390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.105+0000 7f182e809700 1 --2- 172.21.15.83:0/1166535227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18280cc580 0x7f18280b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.105+0000 7f182e809700 1 -- 172.21.15.83:0/1166535227 >> 172.21.15.83:0/1166535227 conn(0x7f18280b3710 msgr2=0x7f182815c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.106+0000 7f182e809700 1 -- 172.21.15.83:0/1166535227 shutdown_connections 2023-11-26T07:12:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.106+0000 7f182e809700 1 -- 172.21.15.83:0/1166535227 wait complete. 2023-11-26T07:12:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2023-11-26T07:12:22.132 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:12:22.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.258+0000 7fcd39662700 1 Processor -- start 2023-11-26T07:12:22.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.258+0000 7fcd39662700 1 -- start start 2023-11-26T07:12:22.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.259+0000 7fcd39662700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcd3415e080 0x7fcd3406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:22.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.259+0000 7fcd39662700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd340b4310 0x7fcd340d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:22.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.259+0000 7fcd39662700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3406ba50 con 0x7fcd340ce430 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.259+0000 7fcd39662700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd340d1200 con 0x7fcd340b4310 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.259+0000 7fcd39662700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3415e460 con 0x7fcd3415e080 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.259+0000 7fcd327fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd340b4310 0x7fcd340d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.259+0000 7fcd327fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd340b4310 0x7fcd340d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36004/0 (socket says 172.21.15.83:36004) 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.259+0000 7fcd327fc700 1 -- 172.21.15.83:0/1067275752 learned_addr learned my addr 172.21.15.83:0/1067275752 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd327fc700 1 -- 172.21.15.83:0/1067275752 >> v2:172.21.15.203:3300/0 conn(0x7fcd3415e080 msgr2=0x7fcd3406bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd327fc700 1 --2- 172.21.15.83:0/1067275752 >> v2:172.21.15.203:3300/0 conn(0x7fcd3415e080 0x7fcd3406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd327fc700 1 -- 172.21.15.83:0/1067275752 >> v1:172.21.15.203:6789/0 conn(0x7fcd340ce430 legacy=0x7fcd340cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd327fc700 1 -- 172.21.15.83:0/1067275752 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3406b120 con 0x7fcd340b4310 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd327fc700 1 --2- 172.21.15.83:0/1067275752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd340b4310 0x7fcd340d0900 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcd340715e0 tx=0x7fcd24009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead3b68d37a0aec4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd31ffb700 1 -- 172.21.15.83:0/1067275752 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24004230 con 0x7fcd340b4310 2023-11-26T07:12:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd31ffb700 1 -- 172.21.15.83:0/1067275752 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd24004390 con 0x7fcd340b4310 2023-11-26T07:12:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd31ffb700 1 -- 172.21.15.83:0/1067275752 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd240036a0 con 0x7fcd340b4310 2023-11-26T07:12:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd39662700 1 -- 172.21.15.83:0/1067275752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd340b4310 msgr2=0x7fcd340d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd39662700 1 --2- 172.21.15.83:0/1067275752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd340b4310 0x7fcd340d0900 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcd340715e0 tx=0x7fcd24009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd39662700 1 -- 172.21.15.83:0/1067275752 shutdown_connections 2023-11-26T07:12:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd39662700 1 --2- 172.21.15.83:0/1067275752 >> v2:172.21.15.203:3300/0 conn(0x7fcd3415e080 0x7fcd3406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd39662700 1 --2- 172.21.15.83:0/1067275752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd340b4310 0x7fcd340d0900 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.260+0000 7fcd39662700 1 -- 172.21.15.83:0/1067275752 >> 172.21.15.83:0/1067275752 conn(0x7fcd340b3780 msgr2=0x7fcd340b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.261+0000 7fcd39662700 1 -- 172.21.15.83:0/1067275752 shutdown_connections 2023-11-26T07:12:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.261+0000 7fcd39662700 1 -- 172.21.15.83:0/1067275752 wait complete. 2023-11-26T07:12:22.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.262+0000 7fcd39662700 1 Processor -- start 2023-11-26T07:12:22.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.262+0000 7fcd39662700 1 -- start start 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd39662700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd340b4310 0x7fcd34160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd39662700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd340ce430 0x7fcd340b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd39662700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3415e080 0x7fcd340b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd39662700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd34158340 con 0x7fcd3415e080 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd39662700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd34161420 con 0x7fcd340ce430 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd39662700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3415f2f0 con 0x7fcd340b4310 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd337fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3415e080 0x7fcd340b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd337fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3415e080 0x7fcd340b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36014/0 (socket says 172.21.15.83:36014) 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd337fe700 1 -- 172.21.15.83:0/2902826080 learned_addr learned my addr 172.21.15.83:0/2902826080 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd327fc700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd340ce430 0x7fcd340b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd32ffd700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd340b4310 0x7fcd34160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd337fe700 1 -- 172.21.15.83:0/2902826080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd340b4310 msgr2=0x7fcd34160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd337fe700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd340b4310 0x7fcd34160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd337fe700 1 -- 172.21.15.83:0/2902826080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd340ce430 msgr2=0x7fcd340b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd337fe700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd340ce430 0x7fcd340b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd337fe700 1 -- 172.21.15.83:0/2902826080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd24009550 con 0x7fcd3415e080 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd337fe700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3415e080 0x7fcd340b5db0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c7a0 tx=0x7fcd2c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd1b7fe700 1 -- 172.21.15.83:0/2902826080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c003e10 con 0x7fcd3415e080 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd1b7fe700 1 -- 172.21.15.83:0/2902826080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c005070 con 0x7fcd3415e080 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd39662700 1 -- 172.21.15.83:0/2902826080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3415f5d0 con 0x7fcd3415e080 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd1b7fe700 1 -- 172.21.15.83:0/2902826080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c005420 con 0x7fcd3415e080 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.263+0000 7fcd39662700 1 -- 172.21.15.83:0/2902826080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3415fb90 con 0x7fcd3415e080 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.264+0000 7fcd39662700 1 -- 172.21.15.83:0/2902826080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd340ceb90 con 0x7fcd3415e080 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.265+0000 7fcd1b7fe700 1 -- 172.21.15.83:0/2902826080 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd2c005580 con 0x7fcd3415e080 2023-11-26T07:12:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.265+0000 7fcd1b7fe700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcd2006aed0 0x7fcd2006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:22.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.265+0000 7fcd1b7fe700 1 -- 172.21.15.83:0/2902826080 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fcd2c088ad0 con 0x7fcd3415e080 2023-11-26T07:12:22.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.265+0000 7fcd32ffd700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcd2006aed0 0x7fcd2006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:22.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.266+0000 7fcd32ffd700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcd2006aed0 0x7fcd2006d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009970 tx=0x7fcd28009110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:22.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.270+0000 7fcd1b7fe700 1 -- 172.21.15.83:0/2902826080 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd2c053c70 con 0x7fcd3415e080 2023-11-26T07:12:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:22.504+0000 7fcd39662700 1 -- 172.21.15.83:0/2902826080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcd34068fb0 con 0x7fcd3415e080 2023-11-26T07:12:23.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.502+0000 7fcd1b7fe700 1 -- 172.21.15.83:0/2902826080 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fcd2c017070 con 0x7fcd3415e080 2023-11-26T07:12:23.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 -- 172.21.15.83:0/2902826080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcd2006aed0 msgr2=0x7fcd2006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcd2006aed0 0x7fcd2006d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009970 tx=0x7fcd28009110 comp rx=0 tx=0).stop 2023-11-26T07:12:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 -- 172.21.15.83:0/2902826080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3415e080 msgr2=0x7fcd340b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3415e080 0x7fcd340b5db0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c7a0 tx=0x7fcd2c00cab0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 -- 172.21.15.83:0/2902826080 shutdown_connections 2023-11-26T07:12:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd3415e080 0x7fcd340b5db0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcd2006aed0 0x7fcd2006d390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd340b4310 0x7fcd34160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 --2- 172.21.15.83:0/2902826080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd340ce430 0x7fcd340b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 -- 172.21.15.83:0/2902826080 >> 172.21.15.83:0/2902826080 conn(0x7fcd340b3780 msgr2=0x7fcd340cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 -- 172.21.15.83:0/2902826080 shutdown_connections 2023-11-26T07:12:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.505+0000 7fcd39662700 1 -- 172.21.15.83:0/2902826080 wait complete. 2023-11-26T07:12:23.509 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 4 and data pool 5 2023-11-26T07:12:23.525 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:23.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.650+0000 7f1110548700 1 Processor -- start 2023-11-26T07:12:23.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.650+0000 7f1110548700 1 -- start start 2023-11-26T07:12:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f1110548700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f110815bdd0 0x7f11081604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f1110548700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f11080b3370 0x7f11080b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f1110548700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f11080d1c70 0x7f11080d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f1110548700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11080b2f50 con 0x7f11080b3370 2023-11-26T07:12:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f1110548700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11080d2660 con 0x7f11080d1c70 2023-11-26T07:12:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f1110548700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11080b2030 con 0x7f110815bdd0 2023-11-26T07:12:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110dae3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f11080b3370 0x7f11080b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110dae3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f11080b3370 0x7f11080b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36018/0 (socket says 172.21.15.83:36018) 2023-11-26T07:12:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110dae3700 1 -- 172.21.15.83:0/1176614217 learned_addr learned my addr 172.21.15.83:0/1176614217 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110e2e4700 1 --2- 172.21.15.83:0/1176614217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f110815bdd0 0x7f11081604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110eae5700 1 --2- 172.21.15.83:0/1176614217 >> v2:172.21.15.203:3300/0 conn(0x7f11080d1c70 0x7f11080d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110dae3700 1 -- 172.21.15.83:0/1176614217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f110815bdd0 msgr2=0x7f11081604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110dae3700 1 --2- 172.21.15.83:0/1176614217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f110815bdd0 0x7f11081604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110dae3700 1 -- 172.21.15.83:0/1176614217 >> v2:172.21.15.203:3300/0 conn(0x7f11080d1c70 msgr2=0x7f11080d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110dae3700 1 --2- 172.21.15.83:0/1176614217 >> v2:172.21.15.203:3300/0 conn(0x7f11080d1c70 0x7f11080d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110dae3700 1 -- 172.21.15.83:0/1176614217 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11080b22d0 con 0x7f11080b3370 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110dae3700 1 --2- 172.21.15.83:0/1176614217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080b3370 0x7f11080b1af0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1108160a20 tx=0x7f10f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17ef4c625f7ccac server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110d2e2700 1 -- 172.21.15.83:0/1176614217 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f800b870 con 0x7f11080b3370 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110d2e2700 1 -- 172.21.15.83:0/1176614217 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f800b9d0 con 0x7f11080b3370 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.651+0000 7f110d2e2700 1 -- 172.21.15.83:0/1176614217 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f800bd60 con 0x7f11080b3370 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.652+0000 7f1110548700 1 -- 172.21.15.83:0/1176614217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080b3370 msgr2=0x7f11080b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.652+0000 7f1110548700 1 --2- 172.21.15.83:0/1176614217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080b3370 0x7f11080b1af0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1108160a20 tx=0x7f10f8009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.652+0000 7f1110548700 1 -- 172.21.15.83:0/1176614217 shutdown_connections 2023-11-26T07:12:23.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.652+0000 7f1110548700 1 --2- 172.21.15.83:0/1176614217 >> v2:172.21.15.203:3300/0 conn(0x7f11080d1c70 0x7f11080d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.652+0000 7f1110548700 1 --2- 172.21.15.83:0/1176614217 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f110815bdd0 0x7f11081604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.652+0000 7f1110548700 1 --2- 172.21.15.83:0/1176614217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11080b3370 0x7f11080b1af0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.652+0000 7f1110548700 1 -- 172.21.15.83:0/1176614217 >> 172.21.15.83:0/1176614217 conn(0x7f11080731a0 msgr2=0x7f110809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:23.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.652+0000 7f1110548700 1 -- 172.21.15.83:0/1176614217 shutdown_connections 2023-11-26T07:12:23.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.652+0000 7f1110548700 1 -- 172.21.15.83:0/1176614217 wait complete. 2023-11-26T07:12:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.653+0000 7f1110548700 1 Processor -- start 2023-11-26T07:12:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.653+0000 7f1110548700 1 -- start start 2023-11-26T07:12:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f1110548700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11080b3370 0x7f11081571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f1110548700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11080d1c70 0x7f11080cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f1110548700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f110815bdd0 0x7f11080cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f1110548700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11080c83d0 con 0x7f110815bdd0 2023-11-26T07:12:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f1110548700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11080c7b30 con 0x7f11080d1c70 2023-11-26T07:12:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f1110548700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11080c7e10 con 0x7f11080b3370 2023-11-26T07:12:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110eae5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f110815bdd0 0x7f11080cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110eae5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f110815bdd0 0x7f11080cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36028/0 (socket says 172.21.15.83:36028) 2023-11-26T07:12:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110eae5700 1 -- 172.21.15.83:0/2103435582 learned_addr learned my addr 172.21.15.83:0/2103435582 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110e2e4700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11080b3370 0x7f11081571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110dae3700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11080d1c70 0x7f11080cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110eae5700 1 -- 172.21.15.83:0/2103435582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11080b3370 msgr2=0x7f11081571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110eae5700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11080b3370 0x7f11081571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110eae5700 1 -- 172.21.15.83:0/2103435582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11080d1c70 msgr2=0x7f11080cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110eae5700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11080d1c70 0x7f11080cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110eae5700 1 -- 172.21.15.83:0/2103435582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f8009550 con 0x7f110815bdd0 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110e2e4700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11080b3370 0x7f11081571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.654+0000 7f110dae3700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11080d1c70 0x7f11080cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.655+0000 7f110eae5700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f110815bdd0 0x7f11080cde40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f110000ccf0 tx=0x7f110000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.655+0000 7f10feffd700 1 -- 172.21.15.83:0/2103435582 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100003a20 con 0x7f110815bdd0 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.655+0000 7f10feffd700 1 -- 172.21.15.83:0/2103435582 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1100003b80 con 0x7f110815bdd0 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.655+0000 7f1110548700 1 -- 172.21.15.83:0/2103435582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11080c80f0 con 0x7f110815bdd0 2023-11-26T07:12:23.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.655+0000 7f10feffd700 1 -- 172.21.15.83:0/2103435582 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100004020 con 0x7f110815bdd0 2023-11-26T07:12:23.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.655+0000 7f1110548700 1 -- 172.21.15.83:0/2103435582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1108157710 con 0x7f110815bdd0 2023-11-26T07:12:23.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.656+0000 7f1110548700 1 -- 172.21.15.83:0/2103435582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1108067f60 con 0x7f110815bdd0 2023-11-26T07:12:23.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.656+0000 7f10feffd700 1 -- 172.21.15.83:0/2103435582 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1100004180 con 0x7f110815bdd0 2023-11-26T07:12:23.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.656+0000 7f10feffd700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10ec06aed0 0x7f10ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.656+0000 7f10feffd700 1 -- 172.21.15.83:0/2103435582 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1100012070 con 0x7f110815bdd0 2023-11-26T07:12:23.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.657+0000 7f110e2e4700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10ec06aed0 0x7f10ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.657+0000 7f110e2e4700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10ec06aed0 0x7f10ec06d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f110400aa50 tx=0x7f1104008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:23.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.661+0000 7f10feffd700 1 -- 172.21.15.83:0/2103435582 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1100054e30 con 0x7f110815bdd0 2023-11-26T07:12:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.852+0000 7f1110548700 1 -- 172.21.15.83:0/2103435582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11080b7c60 con 0x7f110815bdd0 2023-11-26T07:12:23.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.853+0000 7f10feffd700 1 -- 172.21.15.83:0/2103435582 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16570 (secure 0 0 0) 0x7f1100058450 con 0x7f110815bdd0 2023-11-26T07:12:23.854 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:23.855 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":33,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:22.505119+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:20.302489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:21.468154+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:23.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 -- 172.21.15.83:0/2103435582 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10ec06aed0 msgr2=0x7f10ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10ec06aed0 0x7f10ec06d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f110400aa50 tx=0x7f1104008040 comp rx=0 tx=0).stop 2023-11-26T07:12:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 -- 172.21.15.83:0/2103435582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f110815bdd0 msgr2=0x7f11080cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f110815bdd0 0x7f11080cde40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f110000ccf0 tx=0x7f110000a570 comp rx=0 tx=0).stop 2023-11-26T07:12:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 -- 172.21.15.83:0/2103435582 shutdown_connections 2023-11-26T07:12:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f110815bdd0 0x7f11080cde40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10ec06aed0 0x7f10ec06d390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11080b3370 0x7f11081571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 --2- 172.21.15.83:0/2103435582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11080d1c70 0x7f11080cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 -- 172.21.15.83:0/2103435582 >> 172.21.15.83:0/2103435582 conn(0x7f11080731a0 msgr2=0x7f1108159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 -- 172.21.15.83:0/2103435582 shutdown_connections 2023-11-26T07:12:23.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.855+0000 7f1110548700 1 -- 172.21.15.83:0/2103435582 wait complete. 2023-11-26T07:12:23.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.987+0000 7f688288f700 1 Processor -- start 2023-11-26T07:12:23.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.987+0000 7f688288f700 1 -- start start 2023-11-26T07:12:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f688288f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f687c06c710 0x7f687c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f688288f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c0af4d0 0x7f687c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f688288f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c0ae430 con 0x7f687c15e080 2023-11-26T07:12:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f688288f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687c0af0b0 con 0x7f687c06c710 2023-11-26T07:12:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f688288f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0b05a0 con 0x7f687c0af4d0 2023-11-26T07:12:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f687b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f687c06c710 0x7f687c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f687b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f687c06c710 0x7f687c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36034/0 (socket says 172.21.15.83:36034) 2023-11-26T07:12:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f687b7fe700 1 -- 172.21.15.83:0/655322318 learned_addr learned my addr 172.21.15.83:0/655322318 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f6880e2c700 1 --2- 172.21.15.83:0/655322318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c0af4d0 0x7f687c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f687b7fe700 1 -- 172.21.15.83:0/655322318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c0af4d0 msgr2=0x7f687c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:23.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f687b7fe700 1 --2- 172.21.15.83:0/655322318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c0af4d0 0x7f687c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f687b7fe700 1 -- 172.21.15.83:0/655322318 >> v1:172.21.15.203:6789/0 conn(0x7f687c15e080 legacy=0x7f687c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f687b7fe700 1 -- 172.21.15.83:0/655322318 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c0ae710 con 0x7f687c06c710 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f6880e2c700 1 --2- 172.21.15.83:0/655322318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c0af4d0 0x7f687c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.988+0000 7f687b7fe700 1 --2- 172.21.15.83:0/655322318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f687c06c710 0x7f687c0b5050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f686c0098f0 tx=0x7f686c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9f5b8f3bee6a36 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.989+0000 7f687affd700 1 -- 172.21.15.83:0/655322318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c004230 con 0x7f687c06c710 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.989+0000 7f687affd700 1 -- 172.21.15.83:0/655322318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c004390 con 0x7f687c06c710 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.989+0000 7f687affd700 1 -- 172.21.15.83:0/655322318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c0036a0 con 0x7f687c06c710 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.989+0000 7f688288f700 1 -- 172.21.15.83:0/655322318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f687c06c710 msgr2=0x7f687c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.989+0000 7f688288f700 1 --2- 172.21.15.83:0/655322318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f687c06c710 0x7f687c0b5050 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f686c0098f0 tx=0x7f686c009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.989+0000 7f688288f700 1 -- 172.21.15.83:0/655322318 shutdown_connections 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.989+0000 7f688288f700 1 --2- 172.21.15.83:0/655322318 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c0af4d0 0x7f687c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.989+0000 7f688288f700 1 --2- 172.21.15.83:0/655322318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f687c06c710 0x7f687c0b5050 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.989+0000 7f688288f700 1 -- 172.21.15.83:0/655322318 >> 172.21.15.83:0/655322318 conn(0x7f687c071b90 msgr2=0x7f687c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:23.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.989+0000 7f688288f700 1 -- 172.21.15.83:0/655322318 shutdown_connections 2023-11-26T07:12:23.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.989+0000 7f688288f700 1 -- 172.21.15.83:0/655322318 wait complete. 2023-11-26T07:12:23.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.990+0000 7f688288f700 1 Processor -- start 2023-11-26T07:12:23.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f688288f700 1 -- start start 2023-11-26T07:12:23.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f688288f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f687c06c710 0x7f687c14c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f688288f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f687c0af4d0 0x7f687c1f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f688288f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c15e080 0x7f687c14e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f688288f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0c9d60 con 0x7f687c0af4d0 2023-11-26T07:12:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f688288f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0ce670 con 0x7f687c06c710 2023-11-26T07:12:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f688288f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0cb550 con 0x7f687c15e080 2023-11-26T07:12:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f687b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f687c0af4d0 0x7f687c1f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f687b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f687c0af4d0 0x7f687c1f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36040/0 (socket says 172.21.15.83:36040) 2023-11-26T07:12:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f687b7fe700 1 -- 172.21.15.83:0/3603333254 learned_addr learned my addr 172.21.15.83:0/3603333254 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f687bfff700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f687c06c710 0x7f687c14c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.991+0000 7f6880e2c700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c15e080 0x7f687c14e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f687b7fe700 1 -- 172.21.15.83:0/3603333254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c15e080 msgr2=0x7f687c14e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f687b7fe700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c15e080 0x7f687c14e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f687b7fe700 1 -- 172.21.15.83:0/3603333254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f687c06c710 msgr2=0x7f687c14c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f687b7fe700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f687c06c710 0x7f687c14c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f687b7fe700 1 -- 172.21.15.83:0/3603333254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c009550 con 0x7f687c0af4d0 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f6880e2c700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c15e080 0x7f687c14e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f687b7fe700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f687c0af4d0 0x7f687c1f4100 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f687c09eed0 tx=0x7f686c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f6878ff9700 1 -- 172.21.15.83:0/3603333254 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c003dc0 con 0x7f687c0af4d0 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f6878ff9700 1 -- 172.21.15.83:0/3603333254 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c01f070 con 0x7f687c0af4d0 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f688288f700 1 -- 172.21.15.83:0/3603333254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c14f060 con 0x7f687c0af4d0 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f6878ff9700 1 -- 172.21.15.83:0/3603333254 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c00b8a0 con 0x7f687c0af4d0 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.992+0000 7f688288f700 1 -- 172.21.15.83:0/3603333254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c14f620 con 0x7f687c0af4d0 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.993+0000 7f688288f700 1 -- 172.21.15.83:0/3603333254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687c0753f0 con 0x7f687c0af4d0 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.993+0000 7f6878ff9700 1 -- 172.21.15.83:0/3603333254 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f686c021030 con 0x7f687c0af4d0 2023-11-26T07:12:23.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.994+0000 7f6878ff9700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686806aed0 0x7f686806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:23.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.994+0000 7f6878ff9700 1 -- 172.21.15.83:0/3603333254 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f686c08aba0 con 0x7f687c0af4d0 2023-11-26T07:12:23.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.994+0000 7f687bfff700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686806aed0 0x7f686806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:23.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.994+0000 7f687bfff700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686806aed0 0x7f686806d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6870009960 tx=0x7f68700090e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:23.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:23.998+0000 7f6878ff9700 1 -- 172.21.15.83:0/3603333254 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f686c055ce0 con 0x7f687c0af4d0 2023-11-26T07:12:24.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.188+0000 7f688288f700 1 -- 172.21.15.83:0/3603333254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f687c0753f0 con 0x7f687c0af4d0 2023-11-26T07:12:24.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.189+0000 7f6878ff9700 1 -- 172.21.15.83:0/3603333254 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f686c059300 con 0x7f687c0af4d0 2023-11-26T07:12:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:24.192 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:24.102531+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:20.302489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:21.468154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:24.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 -- 172.21.15.83:0/3603333254 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686806aed0 msgr2=0x7f686806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:24.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686806aed0 0x7f686806d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6870009960 tx=0x7f68700090e0 comp rx=0 tx=0).stop 2023-11-26T07:12:24.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 -- 172.21.15.83:0/3603333254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f687c0af4d0 msgr2=0x7f687c1f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:24.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f687c0af4d0 0x7f687c1f4100 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f687c09eed0 tx=0x7f686c004750 comp rx=0 tx=0).stop 2023-11-26T07:12:24.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 -- 172.21.15.83:0/3603333254 shutdown_connections 2023-11-26T07:12:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686806aed0 0x7f686806d390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687c15e080 0x7f687c14e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f687c06c710 0x7f687c14c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 --2- 172.21.15.83:0/3603333254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f687c0af4d0 0x7f687c1f4100 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 -- 172.21.15.83:0/3603333254 >> 172.21.15.83:0/3603333254 conn(0x7f687c071b90 msgr2=0x7f687c06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 -- 172.21.15.83:0/3603333254 shutdown_connections 2023-11-26T07:12:24.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.191+0000 7f688288f700 1 -- 172.21.15.83:0/3603333254 wait complete. 2023-11-26T07:12:24.206 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:12:24.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.327+0000 7f5ee8dd9700 1 Processor -- start 2023-11-26T07:12:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.327+0000 7f5ee8dd9700 1 -- start start 2023-11-26T07:12:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee8dd9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ee40cc540 0x7f5ee406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee8dd9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee406eb20 0x7f5ee40b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee8dd9700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee40b2f20 con 0x7f5ee415a620 2023-11-26T07:12:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee8dd9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee406ef90 con 0x7f5ee40cc540 2023-11-26T07:12:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee8dd9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee415aa00 con 0x7f5ee406eb20 2023-11-26T07:12:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee1d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ee40cc540 0x7f5ee406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee1d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ee40cc540 0x7f5ee406e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36048/0 (socket says 172.21.15.83:36048) 2023-11-26T07:12:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee1d9b700 1 -- 172.21.15.83:0/979785058 learned_addr learned my addr 172.21.15.83:0/979785058 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee1d9b700 1 -- 172.21.15.83:0/979785058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee406eb20 msgr2=0x7f5ee40b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee2d9d700 1 --2- 172.21.15.83:0/979785058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee406eb20 0x7f5ee40b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee1d9b700 1 --2- 172.21.15.83:0/979785058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee406eb20 0x7f5ee40b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee1d9b700 1 -- 172.21.15.83:0/979785058 >> v1:172.21.15.203:6789/0 conn(0x7f5ee415a620 legacy=0x7f5ee40ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.328+0000 7f5ee1d9b700 1 -- 172.21.15.83:0/979785058 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee40ce960 con 0x7f5ee40cc540 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee1d9b700 1 --2- 172.21.15.83:0/979785058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee40cc540 0x7f5ee406e5e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40098f0 tx=0x7f5ed4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86049e451a8b7fd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee159a700 1 -- 172.21.15.83:0/979785058 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4004230 con 0x7f5ee40cc540 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee159a700 1 -- 172.21.15.83:0/979785058 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed4004390 con 0x7f5ee40cc540 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee159a700 1 -- 172.21.15.83:0/979785058 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40036a0 con 0x7f5ee40cc540 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/979785058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee40cc540 msgr2=0x7f5ee406e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee8dd9700 1 --2- 172.21.15.83:0/979785058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee40cc540 0x7f5ee406e5e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40098f0 tx=0x7f5ed4009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/979785058 shutdown_connections 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee8dd9700 1 --2- 172.21.15.83:0/979785058 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee406eb20 0x7f5ee40b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee8dd9700 1 --2- 172.21.15.83:0/979785058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee40cc540 0x7f5ee406e5e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/979785058 >> 172.21.15.83:0/979785058 conn(0x7f5ee40b2050 msgr2=0x7f5ee40b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/979785058 shutdown_connections 2023-11-26T07:12:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.329+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/979785058 wait complete. 2023-11-26T07:12:24.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.331+0000 7f5ee8dd9700 1 Processor -- start 2023-11-26T07:12:24.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.331+0000 7f5ee8dd9700 1 -- start start 2023-11-26T07:12:24.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee8dd9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee406eb20 0x7f5ee415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:24.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee8dd9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee40cc540 0x7f5ee4154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:24.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee8dd9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ee415a620 0x7f5ee413a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:24.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee8dd9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41f4df0 con 0x7f5ee406eb20 2023-11-26T07:12:24.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee8dd9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41f50d0 con 0x7f5ee415a620 2023-11-26T07:12:24.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee8dd9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41f53b0 con 0x7f5ee40cc540 2023-11-26T07:12:24.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee406eb20 0x7f5ee415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:24.336 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee406eb20 0x7f5ee415bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36052/0 (socket says 172.21.15.83:36052) 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee259c700 1 -- 172.21.15.83:0/1356181925 learned_addr learned my addr 172.21.15.83:0/1356181925 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee1d9b700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee40cc540 0x7f5ee4154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee2d9d700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ee415a620 0x7f5ee413a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee259c700 1 -- 172.21.15.83:0/1356181925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee40cc540 msgr2=0x7f5ee4154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee259c700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee40cc540 0x7f5ee4154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee259c700 1 -- 172.21.15.83:0/1356181925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ee415a620 msgr2=0x7f5ee413a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee259c700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ee415a620 0x7f5ee413a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee259c700 1 -- 172.21.15.83:0/1356181925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed4009550 con 0x7f5ee406eb20 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee259c700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee406eb20 0x7f5ee415bc10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c4d0 tx=0x7f5ed800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ed2ffd700 1 -- 172.21.15.83:0/1356181925 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8011790 con 0x7f5ee406eb20 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ed2ffd700 1 -- 172.21.15.83:0/1356181925 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed80118f0 con 0x7f5ee406eb20 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ed2ffd700 1 -- 172.21.15.83:0/1356181925 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8011ca0 con 0x7f5ee406eb20 2023-11-26T07:12:24.337 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/1356181925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee41f5690 con 0x7f5ee406eb20 2023-11-26T07:12:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.332+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/1356181925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee415c150 con 0x7f5ee406eb20 2023-11-26T07:12:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.333+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/1356181925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee41528d0 con 0x7f5ee406eb20 2023-11-26T07:12:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.334+0000 7f5ed2ffd700 1 -- 172.21.15.83:0/1356181925 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ed801c460 con 0x7f5ee406eb20 2023-11-26T07:12:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.334+0000 7f5ed2ffd700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ec406aed0 0x7f5ec406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.334+0000 7f5ed2ffd700 1 -- 172.21.15.83:0/1356181925 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ed8089300 con 0x7f5ee406eb20 2023-11-26T07:12:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.334+0000 7f5ee1d9b700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ec406aed0 0x7f5ec406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:24.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.335+0000 7f5ee1d9b700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ec406aed0 0x7f5ec406d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009130 tx=0x7f5ed4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:24.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.339+0000 7f5ed2ffd700 1 -- 172.21.15.83:0/1356181925 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ed80543b0 con 0x7f5ee406eb20 2023-11-26T07:12:24.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:24.572+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/1356181925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5ee4068d90 con 0x7f5ee406eb20 2023-11-26T07:12:25.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.507+0000 7f5ed2ffd700 1 -- 172.21.15.83:0/1356181925 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5ed80579d0 con 0x7f5ee406eb20 2023-11-26T07:12:25.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/1356181925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ec406aed0 msgr2=0x7f5ec406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ec406aed0 0x7f5ec406d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009130 tx=0x7f5ed4008030 comp rx=0 tx=0).stop 2023-11-26T07:12:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/1356181925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee406eb20 msgr2=0x7f5ee415bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee406eb20 0x7f5ee415bc10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c4d0 tx=0x7f5ed800c7e0 comp rx=0 tx=0).stop 2023-11-26T07:12:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/1356181925 shutdown_connections 2023-11-26T07:12:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ee415a620 0x7f5ee413a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ee406eb20 0x7f5ee415bc10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:25.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5ec406aed0 0x7f5ec406d390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 --2- 172.21.15.83:0/1356181925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ee40cc540 0x7f5ee4154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/1356181925 >> 172.21.15.83:0/1356181925 conn(0x7f5ee40b2050 msgr2=0x7f5ee40b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/1356181925 shutdown_connections 2023-11-26T07:12:25.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.509+0000 7f5ee8dd9700 1 -- 172.21.15.83:0/1356181925 wait complete. 2023-11-26T07:12:25.524 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:12:25.525 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:12:25.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.647+0000 7f2abab61700 1 Processor -- start 2023-11-26T07:12:25.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.647+0000 7f2abab61700 1 -- start start 2023-11-26T07:12:25.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.648+0000 7f2abab61700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ab40b3bf0 0x7f2ab406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:25.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.648+0000 7f2abab61700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 0x7f2ab4074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.648+0000 7f2abab61700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab40b0590 con 0x7f2ab415e150 2023-11-26T07:12:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.648+0000 7f2abab61700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab40b4360 con 0x7f2ab40b3bf0 2023-11-26T07:12:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.648+0000 7f2abab61700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab4074960 con 0x7f2ab4073f90 2023-11-26T07:12:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.648+0000 7f2ab3fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ab40b3bf0 0x7f2ab406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.648+0000 7f2ab90fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 0x7f2ab4074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.648+0000 7f2ab3fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ab40b3bf0 0x7f2ab406e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:35514/0 (socket says 172.21.15.83:35514) 2023-11-26T07:12:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.648+0000 7f2ab3fff700 1 -- 172.21.15.83:0/605611203 learned_addr learned my addr 172.21.15.83:0/605611203 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.648+0000 7f2ab90fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 0x7f2ab4074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:46522/0 (socket says 172.21.15.83:46522) 2023-11-26T07:12:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.648+0000 7f2ab88fd700 1 --1- 172.21.15.83:0/605611203 >> v1:172.21.15.203:6789/0 conn(0x7f2ab415e150 0x7f2ab406f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:40718/0 (socket says 172.21.15.83:40718) 2023-11-26T07:12:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2ab90fe700 1 -- 172.21.15.83:0/605611203 >> v2:172.21.15.203:3300/0 conn(0x7f2ab40b3bf0 msgr2=0x7f2ab406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2ab90fe700 1 --2- 172.21.15.83:0/605611203 >> v2:172.21.15.203:3300/0 conn(0x7f2ab40b3bf0 0x7f2ab406e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2ab90fe700 1 -- 172.21.15.83:0/605611203 >> v1:172.21.15.203:6789/0 conn(0x7f2ab415e150 legacy=0x7f2ab406f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2ab37fe700 1 -- 172.21.15.83:0/605611203 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380974139 0 0) 0x7f2ab40b0590 con 0x7f2ab415e150 2023-11-26T07:12:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2ab90fe700 1 -- 172.21.15.83:0/605611203 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab40b4080 con 0x7f2ab4073f90 2023-11-26T07:12:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2ab3fff700 1 --2- 172.21.15.83:0/605611203 >> v2:172.21.15.203:3300/0 conn(0x7f2ab40b3bf0 0x7f2ab406e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:12:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2ab90fe700 1 --2- 172.21.15.83:0/605611203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 0x7f2ab4074420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009d90 tx=0x7f2aac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe7d7f82c88abc0c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2ab37fe700 1 -- 172.21.15.83:0/605611203 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00e070 con 0x7f2ab4073f90 2023-11-26T07:12:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2ab37fe700 1 -- 172.21.15.83:0/605611203 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac00bb20 con 0x7f2ab4073f90 2023-11-26T07:12:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2ab37fe700 1 -- 172.21.15.83:0/605611203 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac012430 con 0x7f2ab4073f90 2023-11-26T07:12:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2abab61700 1 -- 172.21.15.83:0/605611203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 msgr2=0x7f2ab4074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2abab61700 1 --2- 172.21.15.83:0/605611203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 0x7f2ab4074420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009d90 tx=0x7f2aac00b330 comp rx=0 tx=0).stop 2023-11-26T07:12:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2abab61700 1 -- 172.21.15.83:0/605611203 shutdown_connections 2023-11-26T07:12:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2abab61700 1 --2- 172.21.15.83:0/605611203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 0x7f2ab4074420 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.649+0000 7f2abab61700 1 --2- 172.21.15.83:0/605611203 >> v2:172.21.15.203:3300/0 conn(0x7f2ab40b3bf0 0x7f2ab406e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.650+0000 7f2abab61700 1 -- 172.21.15.83:0/605611203 >> 172.21.15.83:0/605611203 conn(0x7f2ab40d0be0 msgr2=0x7f2ab40d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.650+0000 7f2abab61700 1 -- 172.21.15.83:0/605611203 shutdown_connections 2023-11-26T07:12:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.650+0000 7f2abab61700 1 -- 172.21.15.83:0/605611203 wait complete. 2023-11-26T07:12:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.651+0000 7f2abab61700 1 Processor -- start 2023-11-26T07:12:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.651+0000 7f2abab61700 1 -- start start 2023-11-26T07:12:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2abab61700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 0x7f2ab4155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2abab61700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ab415e150 0x7f2ab415f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2abab61700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab41611b0 0x7f2ab41475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2abab61700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab4160110 con 0x7f2ab41611b0 2023-11-26T07:12:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2abab61700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab41603f0 con 0x7f2ab415e150 2023-11-26T07:12:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2abab61700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab41606d0 con 0x7f2ab4073f90 2023-11-26T07:12:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab90fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab41611b0 0x7f2ab41475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab88fd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 0x7f2ab4155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab90fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab41611b0 0x7f2ab41475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36070/0 (socket says 172.21.15.83:36070) 2023-11-26T07:12:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab90fe700 1 -- 172.21.15.83:0/788746649 learned_addr learned my addr 172.21.15.83:0/788746649 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab3fff700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ab415e150 0x7f2ab415f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab90fe700 1 -- 172.21.15.83:0/788746649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 msgr2=0x7f2ab4155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab90fe700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 0x7f2ab4155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab90fe700 1 -- 172.21.15.83:0/788746649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ab415e150 msgr2=0x7f2ab415f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab90fe700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ab415e150 0x7f2ab415f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab90fe700 1 -- 172.21.15.83:0/788746649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa0009550 con 0x7f2ab41611b0 2023-11-26T07:12:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab90fe700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab41611b0 0x7f2ab41475d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2aac014040 tx=0x7f2aac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:25.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab17fa700 1 -- 172.21.15.83:0/788746649 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac012430 con 0x7f2ab41611b0 2023-11-26T07:12:25.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab17fa700 1 -- 172.21.15.83:0/788746649 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac020070 con 0x7f2ab41611b0 2023-11-26T07:12:25.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2abab61700 1 -- 172.21.15.83:0/788746649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aac009a70 con 0x7f2ab41611b0 2023-11-26T07:12:25.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2ab17fa700 1 -- 172.21.15.83:0/788746649 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac004280 con 0x7f2ab41611b0 2023-11-26T07:12:25.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.652+0000 7f2abab61700 1 -- 172.21.15.83:0/788746649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab4160c90 con 0x7f2ab41611b0 2023-11-26T07:12:25.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.654+0000 7f2abab61700 1 -- 172.21.15.83:0/788746649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab4160e20 con 0x7f2ab41611b0 2023-11-26T07:12:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.654+0000 7f2ab17fa700 1 -- 172.21.15.83:0/788746649 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2aac0043e0 con 0x7f2ab41611b0 2023-11-26T07:12:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.654+0000 7f2ab17fa700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2a9806aed0 0x7f2a9806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.654+0000 7f2ab17fa700 1 -- 172.21.15.83:0/788746649 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2aac022070 con 0x7f2ab41611b0 2023-11-26T07:12:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.654+0000 7f2ab88fd700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2a9806aed0 0x7f2a9806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.655+0000 7f2ab88fd700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2a9806aed0 0x7f2a9806d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8000f60 tx=0x7f2aa8007e70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.659+0000 7f2ab17fa700 1 -- 172.21.15.83:0/788746649 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2aac056650 con 0x7f2ab41611b0 2023-11-26T07:12:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:25.892+0000 7f2abab61700 1 -- 172.21.15.83:0/788746649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2ab4068fb0 con 0x7f2ab41611b0 2023-11-26T07:12:26.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.508+0000 7f2ab17fa700 1 -- 172.21.15.83:0/788746649 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2aac089e60 con 0x7f2ab41611b0 2023-11-26T07:12:26.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 -- 172.21.15.83:0/788746649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2a9806aed0 msgr2=0x7f2a9806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:26.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2a9806aed0 0x7f2a9806d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8000f60 tx=0x7f2aa8007e70 comp rx=0 tx=0).stop 2023-11-26T07:12:26.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 -- 172.21.15.83:0/788746649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab41611b0 msgr2=0x7f2ab41475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:26.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab41611b0 0x7f2ab41475d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2aac014040 tx=0x7f2aac004ad0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 -- 172.21.15.83:0/788746649 shutdown_connections 2023-11-26T07:12:26.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab41611b0 0x7f2ab41475d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2a9806aed0 0x7f2a9806d390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab4073f90 0x7f2ab4155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 --2- 172.21.15.83:0/788746649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ab415e150 0x7f2ab415f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 -- 172.21.15.83:0/788746649 >> 172.21.15.83:0/788746649 conn(0x7f2ab40d0be0 msgr2=0x7f2ab40b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:26.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 -- 172.21.15.83:0/788746649 shutdown_connections 2023-11-26T07:12:26.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.512+0000 7f2abab61700 1 -- 172.21.15.83:0/788746649 wait complete. 2023-11-26T07:12:26.528 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:12:26.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:26.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.649+0000 7f35cb564700 1 Processor -- start 2023-11-26T07:12:26.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.650+0000 7f35cb564700 1 -- start start 2023-11-26T07:12:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.650+0000 7f35cb564700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35c4158a50 0x7f35c40c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:26.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.650+0000 7f35cb564700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f35c4162a30 0x7f35c40c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.650+0000 7f35cb564700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c40cfff0 con 0x7f35c4158670 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.650+0000 7f35cb564700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c40629b0 con 0x7f35c4158a50 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.650+0000 7f35cb564700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c40cfd10 con 0x7f35c4162a30 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.650+0000 7f35c8aff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35c4158a50 0x7f35c40c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35c8aff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35c4158a50 0x7f35c40c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36076/0 (socket says 172.21.15.83:36076) 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35c8aff700 1 -- 172.21.15.83:0/3214242985 learned_addr learned my addr 172.21.15.83:0/3214242985 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35c9b01700 1 --2- 172.21.15.83:0/3214242985 >> v2:172.21.15.203:3300/0 conn(0x7f35c4162a30 0x7f35c40c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35c8aff700 1 -- 172.21.15.83:0/3214242985 >> v2:172.21.15.203:3300/0 conn(0x7f35c4162a30 msgr2=0x7f35c40c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35c8aff700 1 --2- 172.21.15.83:0/3214242985 >> v2:172.21.15.203:3300/0 conn(0x7f35c4162a30 0x7f35c40c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35c8aff700 1 -- 172.21.15.83:0/3214242985 >> v1:172.21.15.203:6789/0 conn(0x7f35c4158670 legacy=0x7f35c40622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35c8aff700 1 -- 172.21.15.83:0/3214242985 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c4161f50 con 0x7f35c4158a50 2023-11-26T07:12:26.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35c9b01700 1 --2- 172.21.15.83:0/3214242985 >> v2:172.21.15.203:3300/0 conn(0x7f35c4162a30 0x7f35c40c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35c8aff700 1 --2- 172.21.15.83:0/3214242985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4158a50 0x7f35c40c98b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f35c40b0070 tx=0x7f35c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa71174a439fae10 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35bbfff700 1 -- 172.21.15.83:0/3214242985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0004230 con 0x7f35c4158a50 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35bbfff700 1 -- 172.21.15.83:0/3214242985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c0004390 con 0x7f35c4158a50 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.651+0000 7f35bbfff700 1 -- 172.21.15.83:0/3214242985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c00036a0 con 0x7f35c4158a50 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.652+0000 7f35cb564700 1 -- 172.21.15.83:0/3214242985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4158a50 msgr2=0x7f35c40c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.652+0000 7f35cb564700 1 --2- 172.21.15.83:0/3214242985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4158a50 0x7f35c40c98b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f35c40b0070 tx=0x7f35c0009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.652+0000 7f35cb564700 1 -- 172.21.15.83:0/3214242985 shutdown_connections 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.652+0000 7f35cb564700 1 --2- 172.21.15.83:0/3214242985 >> v2:172.21.15.203:3300/0 conn(0x7f35c4162a30 0x7f35c40c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.652+0000 7f35cb564700 1 --2- 172.21.15.83:0/3214242985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4158a50 0x7f35c40c98b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.652+0000 7f35cb564700 1 -- 172.21.15.83:0/3214242985 >> 172.21.15.83:0/3214242985 conn(0x7f35c40b57b0 msgr2=0x7f35c40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.652+0000 7f35cb564700 1 -- 172.21.15.83:0/3214242985 shutdown_connections 2023-11-26T07:12:26.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.652+0000 7f35cb564700 1 -- 172.21.15.83:0/3214242985 wait complete. 2023-11-26T07:12:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.653+0000 7f35cb564700 1 Processor -- start 2023-11-26T07:12:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.653+0000 7f35cb564700 1 -- start start 2023-11-26T07:12:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35cb564700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4158670 0x7f35c41590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35cb564700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35c4158a50 0x7f35c41f4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35cb564700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35c4162a30 0x7f35c4146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:26.659 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35cb564700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c41492b0 con 0x7f35c4158670 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35cb564700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c4149590 con 0x7f35c4162a30 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35cb564700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c4149870 con 0x7f35c4158a50 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c9300700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4158670 0x7f35c41590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c9300700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4158670 0x7f35c41590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36080/0 (socket says 172.21.15.83:36080) 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c9300700 1 -- 172.21.15.83:0/3684171077 learned_addr learned my addr 172.21.15.83:0/3684171077 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c9b01700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35c4162a30 0x7f35c4146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c8aff700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35c4158a50 0x7f35c41f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c9300700 1 -- 172.21.15.83:0/3684171077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35c4158a50 msgr2=0x7f35c41f4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c9300700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35c4158a50 0x7f35c41f4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c9300700 1 -- 172.21.15.83:0/3684171077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35c4162a30 msgr2=0x7f35c4146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c9300700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35c4162a30 0x7f35c4146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c9300700 1 -- 172.21.15.83:0/3684171077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c0009550 con 0x7f35c4158670 2023-11-26T07:12:26.660 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c8aff700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35c4158a50 0x7f35c41f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c9b01700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35c4162a30 0x7f35c4146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35c9300700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4158670 0x7f35c41590a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f35b400c4d0 tx=0x7f35b400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35b9ffb700 1 -- 172.21.15.83:0/3684171077 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4011790 con 0x7f35c4158670 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35b9ffb700 1 -- 172.21.15.83:0/3684171077 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b40118f0 con 0x7f35c4158670 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35cb564700 1 -- 172.21.15.83:0/3684171077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c4149b50 con 0x7f35c4158670 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.654+0000 7f35b9ffb700 1 -- 172.21.15.83:0/3684171077 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4011ca0 con 0x7f35c4158670 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.655+0000 7f35cb564700 1 -- 172.21.15.83:0/3684171077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c41595e0 con 0x7f35c4158670 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.656+0000 7f35cb564700 1 -- 172.21.15.83:0/3684171077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c41597d0 con 0x7f35c4158670 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.656+0000 7f35b9ffb700 1 -- 172.21.15.83:0/3684171077 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f35b401c460 con 0x7f35c4158670 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.656+0000 7f35b9ffb700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f35a806aed0 0x7f35a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.656+0000 7f35b9ffb700 1 -- 172.21.15.83:0/3684171077 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35b4089440 con 0x7f35c4158670 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.656+0000 7f35c8aff700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f35a806aed0 0x7f35a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:26.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.656+0000 7f35c8aff700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f35a806aed0 0x7f35a806d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f35c40b0070 tx=0x7f35c0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:26.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.661+0000 7f35b9ffb700 1 -- 172.21.15.83:0/3684171077 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35b40544f0 con 0x7f35c4158670 2023-11-26T07:12:26.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:12:26.737 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:12:26.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.901+0000 7f35cb564700 1 -- 172.21.15.83:0/3684171077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35c4149ec0 con 0x7f35c4158670 2023-11-26T07:12:26.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.901+0000 7f35b9ffb700 1 -- 172.21.15.83:0/3684171077 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f35b4057b10 con 0x7f35c4158670 2023-11-26T07:12:26.904 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:26.904 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/486975373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":486975373},{"type":"v1","addr":"172.21.15.203:6834","nonce":486975373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3188343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3188343240},{"type":"v1","addr":"172.21.15.83:6836","nonce":3188343240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4271462644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4271462644},{"type":"v1","addr":"172.21.15.83:6837","nonce":4271462644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:22.505426+0000","modified":"2023-11-26T07:12:25.894494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6835/995330476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":995330476},{"type":"v1","addr":"172.21.15.203:6835","nonce":995330476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:12:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 -- 172.21.15.83:0/3684171077 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f35a806aed0 msgr2=0x7f35a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f35a806aed0 0x7f35a806d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f35c40b0070 tx=0x7f35c0008030 comp rx=0 tx=0).stop 2023-11-26T07:12:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 -- 172.21.15.83:0/3684171077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4158670 msgr2=0x7f35c41590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4158670 0x7f35c41590a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f35b400c4d0 tx=0x7f35b400c890 comp rx=0 tx=0).stop 2023-11-26T07:12:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 -- 172.21.15.83:0/3684171077 shutdown_connections 2023-11-26T07:12:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35c4162a30 0x7f35c4146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35c4158670 0x7f35c41590a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f35a806aed0 0x7f35a806d390 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 --2- 172.21.15.83:0/3684171077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35c4158a50 0x7f35c41f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 -- 172.21.15.83:0/3684171077 >> 172.21.15.83:0/3684171077 conn(0x7f35c40b57b0 msgr2=0x7f35c406e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 -- 172.21.15.83:0/3684171077 shutdown_connections 2023-11-26T07:12:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:26.903+0000 7f35cb564700 1 -- 172.21.15.83:0/3684171077 wait complete. 2023-11-26T07:12:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2023-11-26T07:12:26.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:26.991 INFO:tasks.ceph.mds.d.smithi203.stderr:2023-11-26T07:12:26.989+0000 7f1cbf5e5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:12:27.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.036+0000 7fa3e4622700 1 Processor -- start 2023-11-26T07:12:27.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.036+0000 7fa3e4622700 1 -- start start 2023-11-26T07:12:27.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e4622700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa3dc0cf2b0 0x7fa3dc0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e4622700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc0cc780 con 0x7fa3dc15bd60 2023-11-26T07:12:27.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e4622700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc0cc9b0 con 0x7fa3dc15c140 2023-11-26T07:12:27.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e4622700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc0ccc90 con 0x7fa3dc0cf2b0 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e23be700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa3dc15bd60 0x7fa3dc0d1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56128/0 (socket says 172.21.15.83:56128) 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e23be700 1 -- 172.21.15.83:0/2922608283 learned_addr learned my addr 172.21.15.83:0/2922608283 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437894150 0 0) 0x7fa3dc0cc780 con 0x7fa3dc15bd60 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c80035b0 con 0x7fa3dc15bd60 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e2bbf700 1 --2- 172.21.15.83:0/2922608283 >> v2:172.21.15.203:3300/0 conn(0x7fa3dc0cf2b0 0x7fa3dc0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3129405774 0 0) 0x7fa3c80035b0 con 0x7fa3dc15bd60 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 >> v2:172.21.15.203:3300/0 conn(0x7fa3dc0cf2b0 msgr2=0x7fa3dc0c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 --2- 172.21.15.83:0/2922608283 >> v2:172.21.15.203:3300/0 conn(0x7fa3dc0cf2b0 0x7fa3dc0c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 >> v1:172.21.15.203:6789/0 conn(0x7fa3dc15c140 legacy=0x7fa3dc0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0ccf70 con 0x7fa3dc15bd60 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fa3cc001670 con 0x7fa3dc15bd60 2023-11-26T07:12:27.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 >> v1:172.21.15.83:6789/0 conn(0x7fa3dc15bd60 legacy=0x7fa3dc0d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c8003a00 0x7fa3c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3c8004850 0x7fa3c8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c8005210 0x7fa3c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0ccc90 con 0x7fa3c8005210 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c800c0b0 con 0x7fa3c8004850 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.038+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c800c270 con 0x7fa3c8003a00 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e2bbf700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c8005210 0x7fa3c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e1bbd700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3c8004850 0x7fa3c8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e23be700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c8003a00 0x7fa3c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e2bbf700 1 -- 172.21.15.83:0/2922608283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c8003a00 msgr2=0x7fa3c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e2bbf700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c8003a00 0x7fa3c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e2bbf700 1 -- 172.21.15.83:0/2922608283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3c8004850 msgr2=0x7fa3c8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:27.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e2bbf700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3c8004850 0x7fa3c8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e2bbf700 1 -- 172.21.15.83:0/2922608283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80035b0 con 0x7fa3c8005210 2023-11-26T07:12:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e23be700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c8003a00 0x7fa3c8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e2bbf700 1 -- 172.21.15.83:0/2922608283 reap_dead start 2023-11-26T07:12:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e2bbf700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c8005210 0x7fa3c800ba10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009fc0 tx=0x7fa3c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8327b183b951ec96 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4003a20 con 0x7fa3c8005210 2023-11-26T07:12:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d400f040 con 0x7fa3c8005210 2023-11-26T07:12:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.039+0000 7fa3e13bc700 1 -- 172.21.15.83:0/2922608283 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4003de0 con 0x7fa3c8005210 2023-11-26T07:12:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.040+0000 7fa3e4622700 1 -- 172.21.15.83:0/2922608283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c8005210 msgr2=0x7fa3c800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.040+0000 7fa3e4622700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c8005210 0x7fa3c800ba10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009fc0 tx=0x7fa3c8005880 comp rx=0 tx=0).stop 2023-11-26T07:12:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.040+0000 7fa3e4622700 1 -- 172.21.15.83:0/2922608283 shutdown_connections 2023-11-26T07:12:27.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.040+0000 7fa3e4622700 1 --2- 172.21.15.83:0/2922608283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c8005210 0x7fa3c800ba10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.040+0000 7fa3e4622700 1 -- 172.21.15.83:0/2922608283 >> 172.21.15.83:0/2922608283 conn(0x7fa3dc073130 msgr2=0x7fa3dc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.040+0000 7fa3e4622700 1 -- 172.21.15.83:0/2922608283 shutdown_connections 2023-11-26T07:12:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.040+0000 7fa3e4622700 1 -- 172.21.15.83:0/2922608283 wait complete. 2023-11-26T07:12:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.041+0000 7fa3e4622700 1 Processor -- start 2023-11-26T07:12:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.041+0000 7fa3e4622700 1 -- start start 2023-11-26T07:12:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e4622700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c8005210 0x7fa3dc151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e4622700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3dc153d50 0x7fa3dc1f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e4622700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3dc1541d0 0x7fa3dc157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e4622700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc1560a0 con 0x7fa3dc1541d0 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e4622700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc155490 con 0x7fa3dc153d50 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e4622700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc155770 con 0x7fa3c8005210 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e2bbf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3dc1541d0 0x7fa3dc157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e2bbf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3dc1541d0 0x7fa3dc157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36104/0 (socket says 172.21.15.83:36104) 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e2bbf700 1 -- 172.21.15.83:0/2668168094 learned_addr learned my addr 172.21.15.83:0/2668168094 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e23be700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c8005210 0x7fa3dc151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e1bbd700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3dc153d50 0x7fa3dc1f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e2bbf700 1 -- 172.21.15.83:0/2668168094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c8005210 msgr2=0x7fa3dc151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e2bbf700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c8005210 0x7fa3dc151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e2bbf700 1 -- 172.21.15.83:0/2668168094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3dc153d50 msgr2=0x7fa3dc1f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e2bbf700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3dc153d50 0x7fa3dc1f2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e2bbf700 1 -- 172.21.15.83:0/2668168094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc155a50 con 0x7fa3dc1541d0 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e23be700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c8005210 0x7fa3dc151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e2bbf700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3dc1541d0 0x7fa3dc157ac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0d0da0 tx=0x7fa3d4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3d2ffd700 1 -- 172.21.15.83:0/2668168094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4003e10 con 0x7fa3dc1541d0 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3d2ffd700 1 -- 172.21.15.83:0/2668168094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d4004230 con 0x7fa3dc1541d0 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e4622700 1 -- 172.21.15.83:0/2668168094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc155d30 con 0x7fa3dc1541d0 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3d2ffd700 1 -- 172.21.15.83:0/2668168094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d40045e0 con 0x7fa3dc1541d0 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.042+0000 7fa3e4622700 1 -- 172.21.15.83:0/2668168094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc1f3040 con 0x7fa3dc1541d0 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.043+0000 7fa3e4622700 1 -- 172.21.15.83:0/2668168094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc067f60 con 0x7fa3dc1541d0 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.044+0000 7fa3d2ffd700 1 -- 172.21.15.83:0/2668168094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3d4004740 con 0x7fa3dc1541d0 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.044+0000 7fa3d2ffd700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c006aed0 0x7fa3c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.044+0000 7fa3d2ffd700 1 -- 172.21.15.83:0/2668168094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3d408a570 con 0x7fa3dc1541d0 2023-11-26T07:12:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.044+0000 7fa3e23be700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c006aed0 0x7fa3c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.045+0000 7fa3e23be700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c006aed0 0x7fa3c006d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000c00 tx=0x7fa3cc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.049+0000 7fa3d2ffd700 1 -- 172.21.15.83:0/2668168094 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3d40556b0 con 0x7fa3dc1541d0 2023-11-26T07:12:27.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.239+0000 7fa3e4622700 1 -- 172.21.15.83:0/2668168094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3dc1f3040 con 0x7fa3dc1541d0 2023-11-26T07:12:27.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.241+0000 7fa3d2ffd700 1 -- 172.21.15.83:0/2668168094 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fa3dc1f3040 con 0x7fa3dc1541d0 2023-11-26T07:12:27.243 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:27.243 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:24.102531+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:20.302489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:21.468154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 -- 172.21.15.83:0/2668168094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c006aed0 msgr2=0x7fa3c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c006aed0 0x7fa3c006d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000c00 tx=0x7fa3cc009040 comp rx=0 tx=0).stop 2023-11-26T07:12:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 -- 172.21.15.83:0/2668168094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3dc1541d0 msgr2=0x7fa3dc157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3dc1541d0 0x7fa3dc157ac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0d0da0 tx=0x7fa3d4009750 comp rx=0 tx=0).stop 2023-11-26T07:12:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 -- 172.21.15.83:0/2668168094 shutdown_connections 2023-11-26T07:12:27.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3dc1541d0 0x7fa3dc157ac0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c006aed0 0x7fa3c006d390 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c8005210 0x7fa3dc151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 --2- 172.21.15.83:0/2668168094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3dc153d50 0x7fa3dc1f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 -- 172.21.15.83:0/2668168094 >> 172.21.15.83:0/2668168094 conn(0x7fa3dc073130 msgr2=0x7fa3dc0703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 -- 172.21.15.83:0/2668168094 shutdown_connections 2023-11-26T07:12:27.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.243+0000 7fa3e4622700 1 -- 172.21.15.83:0/2668168094 wait complete. 2023-11-26T07:12:27.258 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:12:27.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:12:27.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.376+0000 7f6d3ea2b700 1 Processor -- start 2023-11-26T07:12:27.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.376+0000 7f6d3ea2b700 1 -- start start 2023-11-26T07:12:27.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d3ea2b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6d3815e080 0x7f6d380b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d3ea2b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d380af4d0 0x7f6d380afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d3ea2b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6d3806c000 0x7f6d3806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d3ea2b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d380af0b0 con 0x7f6d3815e080 2023-11-26T07:12:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d3ea2b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3806c9f0 con 0x7f6d3806c000 2023-11-26T07:12:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d3ea2b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380b05a0 con 0x7f6d380af4d0 2023-11-26T07:12:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d37fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6d3815e080 0x7f6d380b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d37fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6d3815e080 0x7f6d380b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36108/0 (socket says 172.21.15.83:36108) 2023-11-26T07:12:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d37fff700 1 -- 172.21.15.83:0/2584452426 learned_addr learned my addr 172.21.15.83:0/2584452426 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d377fe700 1 --2- 172.21.15.83:0/2584452426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d380af4d0 0x7f6d380afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d3cfc8700 1 --2- 172.21.15.83:0/2584452426 >> v2:172.21.15.203:3300/0 conn(0x7f6d3806c000 0x7f6d3806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d37fff700 1 -- 172.21.15.83:0/2584452426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d380af4d0 msgr2=0x7f6d380afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d37fff700 1 --2- 172.21.15.83:0/2584452426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d380af4d0 0x7f6d380afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d37fff700 1 -- 172.21.15.83:0/2584452426 >> v2:172.21.15.203:3300/0 conn(0x7f6d3806c000 msgr2=0x7f6d3806c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:27.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d37fff700 1 --2- 172.21.15.83:0/2584452426 >> v2:172.21.15.203:3300/0 conn(0x7f6d3806c000 0x7f6d3806c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d37fff700 1 -- 172.21.15.83:0/2584452426 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380ae430 con 0x7f6d3815e080 2023-11-26T07:12:27.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d37fff700 1 --2- 172.21.15.83:0/2584452426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d3815e080 0x7f6d380b5050 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6d3809eea0 tx=0x7f6d28009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d31c32b477aea4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:27.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d36ffd700 1 -- 172.21.15.83:0/2584452426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800b860 con 0x7f6d3815e080 2023-11-26T07:12:27.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d36ffd700 1 -- 172.21.15.83:0/2584452426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2800b9c0 con 0x7f6d3815e080 2023-11-26T07:12:27.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.377+0000 7f6d36ffd700 1 -- 172.21.15.83:0/2584452426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800bd50 con 0x7f6d3815e080 2023-11-26T07:12:27.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.378+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/2584452426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d3815e080 msgr2=0x7f6d380b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.378+0000 7f6d3ea2b700 1 --2- 172.21.15.83:0/2584452426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d3815e080 0x7f6d380b5050 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6d3809eea0 tx=0x7f6d28009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:27.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.378+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/2584452426 shutdown_connections 2023-11-26T07:12:27.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.378+0000 7f6d3ea2b700 1 --2- 172.21.15.83:0/2584452426 >> v2:172.21.15.203:3300/0 conn(0x7f6d3806c000 0x7f6d3806c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.378+0000 7f6d3ea2b700 1 --2- 172.21.15.83:0/2584452426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d3815e080 0x7f6d380b5050 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.378+0000 7f6d3ea2b700 1 --2- 172.21.15.83:0/2584452426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d380af4d0 0x7f6d380afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.378+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/2584452426 >> 172.21.15.83:0/2584452426 conn(0x7f6d38071b90 msgr2=0x7f6d38071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:27.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.378+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/2584452426 shutdown_connections 2023-11-26T07:12:27.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.378+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/2584452426 wait complete. 2023-11-26T07:12:27.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.379+0000 7f6d3ea2b700 1 Processor -- start 2023-11-26T07:12:27.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.379+0000 7f6d3ea2b700 1 -- start start 2023-11-26T07:12:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d3ea2b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3806c000 0x7f6d38146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d3ea2b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d380af4d0 0x7f6d381551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d3ea2b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3815e080 0x7f6d38155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d3ea2b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38153190 con 0x7f6d380af4d0 2023-11-26T07:12:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d3ea2b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3815f7c0 con 0x7f6d3806c000 2023-11-26T07:12:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d3ea2b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38146a90 con 0x7f6d3815e080 2023-11-26T07:12:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d380af4d0 0x7f6d381551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d377fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d380af4d0 0x7f6d381551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36116/0 (socket says 172.21.15.83:36116) 2023-11-26T07:12:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d377fe700 1 -- 172.21.15.83:0/1623987296 learned_addr learned my addr 172.21.15.83:0/1623987296 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:27.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d37fff700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3806c000 0x7f6d38146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d3cfc8700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3815e080 0x7f6d38155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d377fe700 1 -- 172.21.15.83:0/1623987296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3815e080 msgr2=0x7f6d38155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d377fe700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3815e080 0x7f6d38155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d377fe700 1 -- 172.21.15.83:0/1623987296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3806c000 msgr2=0x7f6d38146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d377fe700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3806c000 0x7f6d38146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d377fe700 1 -- 172.21.15.83:0/1623987296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d28009550 con 0x7f6d380af4d0 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d3cfc8700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3815e080 0x7f6d38155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.380+0000 7f6d37fff700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3806c000 0x7f6d38146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.381+0000 7f6d377fe700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d380af4d0 0x7f6d381551c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00e7e0 tx=0x7f6d2c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.381+0000 7f6d34ff9700 1 -- 172.21.15.83:0/1623987296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c013610 con 0x7f6d380af4d0 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.381+0000 7f6d34ff9700 1 -- 172.21.15.83:0/1623987296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c013770 con 0x7f6d380af4d0 2023-11-26T07:12:27.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.381+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/1623987296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d38157f30 con 0x7f6d380af4d0 2023-11-26T07:12:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.381+0000 7f6d34ff9700 1 -- 172.21.15.83:0/1623987296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c013ae0 con 0x7f6d380af4d0 2023-11-26T07:12:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.381+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/1623987296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d381584f0 con 0x7f6d380af4d0 2023-11-26T07:12:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.382+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/1623987296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d380753f0 con 0x7f6d380af4d0 2023-11-26T07:12:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.382+0000 7f6d34ff9700 1 -- 172.21.15.83:0/1623987296 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d2c013c40 con 0x7f6d380af4d0 2023-11-26T07:12:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.382+0000 7f6d34ff9700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6d2406aed0 0x7f6d2406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.383+0000 7f6d34ff9700 1 -- 172.21.15.83:0/1623987296 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d2c089200 con 0x7f6d380af4d0 2023-11-26T07:12:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.383+0000 7f6d37fff700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6d2406aed0 0x7f6d2406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.383+0000 7f6d37fff700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6d2406aed0 0x7f6d2406d390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6d28000c00 tx=0x7f6d28008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:27.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.387+0000 7f6d34ff9700 1 -- 172.21.15.83:0/1623987296 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6d2c054340 con 0x7f6d380af4d0 2023-11-26T07:12:27.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.575+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/1623987296 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d380ae9f0 con 0x7f6d2406aed0 2023-11-26T07:12:27.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.583+0000 7f6d34ff9700 1 -- 172.21.15.83:0/1623987296 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289112 (secure 0 0 0) 0x7f6d2800b710 con 0x7f6d2406aed0 2023-11-26T07:12:27.589 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/1623987296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6d2406aed0 msgr2=0x7f6d2406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6d2406aed0 0x7f6d2406d390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6d28000c00 tx=0x7f6d28008030 comp rx=0 tx=0).stop 2023-11-26T07:12:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/1623987296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d380af4d0 msgr2=0x7f6d381551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d380af4d0 0x7f6d381551c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00e7e0 tx=0x7f6d2c00eba0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/1623987296 shutdown_connections 2023-11-26T07:12:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6d2406aed0 0x7f6d2406d390 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3815e080 0x7f6d38155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3806c000 0x7f6d38146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 --2- 172.21.15.83:0/1623987296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d380af4d0 0x7f6d381551c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/1623987296 >> 172.21.15.83:0/1623987296 conn(0x7f6d38071b90 msgr2=0x7f6d380b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:27.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/1623987296 shutdown_connections 2023-11-26T07:12:27.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.586+0000 7f6d3ea2b700 1 -- 172.21.15.83:0/1623987296 wait complete. 2023-11-26T07:12:27.592 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:12:27.601 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-26T07:12:26.319298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220380,"kb_used_data":4184,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754340,"statfs":{"total":773094113280,"available":772868444160,"internally_reserved":0,"allocated":4284416,"data_stored":2370460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220688"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783586+0000","last_change":"2023-11-26T07:12:23.132616+0000","last_active":"2023-11-26T07:12:24.783586+0000","last_peered":"2023-11-26T07:12:24.783586+0000","last_clean":"2023-11-26T07:12:24.783586+0000","last_became_active":"2023-11-26T07:12:23.132367+0000","last_became_peered":"2023-11-26T07:12:23.132367+0000","last_unstale":"2023-11-26T07:12:24.783586+0000","last_undegraded":"2023-11-26T07:12:24.783586+0000","last_fullsized":"2023-11-26T07:12:24.783586+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:34.579111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106038+0000","last_change":"2023-11-26T07:12:23.118515+0000","last_active":"2023-11-26T07:12:24.106038+0000","last_peered":"2023-11-26T07:12:24.106038+0000","last_clean":"2023-11-26T07:12:24.106038+0000","last_became_active":"2023-11-26T07:12:23.118256+0000","last_became_peered":"2023-11-26T07:12:23.118256+0000","last_unstale":"2023-11-26T07:12:24.106038+0000","last_undegraded":"2023-11-26T07:12:24.106038+0000","last_fullsized":"2023-11-26T07:12:24.106038+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:04:25.608116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107904+0000","last_change":"2023-11-26T07:12:23.123345+0000","last_active":"2023-11-26T07:12:24.107904+0000","last_peered":"2023-11-26T07:12:24.107904+0000","last_clean":"2023-11-26T07:12:24.107904+0000","last_became_active":"2023-11-26T07:12:23.122918+0000","last_became_peered":"2023-11-26T07:12:23.122918+0000","last_unstale":"2023-11-26T07:12:24.107904+0000","last_undegraded":"2023-11-26T07:12:24.107904+0000","last_fullsized":"2023-11-26T07:12:24.107904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:50.833212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783446+0000","last_change":"2023-11-26T07:12:23.121519+0000","last_active":"2023-11-26T07:12:24.783446+0000","last_peered":"2023-11-26T07:12:24.783446+0000","last_clean":"2023-11-26T07:12:24.783446+0000","last_became_active":"2023-11-26T07:12:23.121130+0000","last_became_peered":"2023-11-26T07:12:23.121130+0000","last_unstale":"2023-11-26T07:12:24.783446+0000","last_undegraded":"2023-11-26T07:12:24.783446+0000","last_fullsized":"2023-11-26T07:12:24.783446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:10:21.444998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783207+0000","last_change":"2023-11-26T07:12:23.119945+0000","last_active":"2023-11-26T07:12:24.783207+0000","last_peered":"2023-11-26T07:12:24.783207+0000","last_clean":"2023-11-26T07:12:24.783207+0000","last_became_active":"2023-11-26T07:12:23.119678+0000","last_became_peered":"2023-11-26T07:12:23.119678+0000","last_unstale":"2023-11-26T07:12:24.783207+0000","last_undegraded":"2023-11-26T07:12:24.783207+0000","last_fullsized":"2023-11-26T07:12:24.783207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:46:57.969869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105704+0000","last_change":"2023-11-26T07:12:23.122604+0000","last_active":"2023-11-26T07:12:24.105704+0000","last_peered":"2023-11-26T07:12:24.105704+0000","last_clean":"2023-11-26T07:12:24.105704+0000","last_became_active":"2023-11-26T07:12:23.122314+0000","last_became_peered":"2023-11-26T07:12:23.122314+0000","last_unstale":"2023-11-26T07:12:24.105704+0000","last_undegraded":"2023-11-26T07:12:24.105704+0000","last_fullsized":"2023-11-26T07:12:24.105704+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:32.714331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107189+0000","last_change":"2023-11-26T07:12:23.128631+0000","last_active":"2023-11-26T07:12:24.107189+0000","last_peered":"2023-11-26T07:12:24.107189+0000","last_clean":"2023-11-26T07:12:24.107189+0000","last_became_active":"2023-11-26T07:12:23.128031+0000","last_became_peered":"2023-11-26T07:12:23.128031+0000","last_unstale":"2023-11-26T07:12:24.107189+0000","last_undegraded":"2023-11-26T07:12:24.107189+0000","last_fullsized":"2023-11-26T07:12:24.107189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:27.812069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106880+0000","last_change":"2023-11-26T07:12:23.115564+0000","last_active":"2023-11-26T07:12:24.106880+0000","last_peered":"2023-11-26T07:12:24.106880+0000","last_clean":"2023-11-26T07:12:24.106880+0000","last_became_active":"2023-11-26T07:12:23.115246+0000","last_became_peered":"2023-11-26T07:12:23.115246+0000","last_unstale":"2023-11-26T07:12:24.106880+0000","last_undegraded":"2023-11-26T07:12:24.106880+0000","last_fullsized":"2023-11-26T07:12:24.106880+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:48:14.958935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436354+0000","last_change":"2023-11-26T07:12:23.117620+0000","last_active":"2023-11-26T07:12:24.436354+0000","last_peered":"2023-11-26T07:12:24.436354+0000","last_clean":"2023-11-26T07:12:24.436354+0000","last_became_active":"2023-11-26T07:12:23.117216+0000","last_became_peered":"2023-11-26T07:12:23.117216+0000","last_unstale":"2023-11-26T07:12:24.436354+0000","last_undegraded":"2023-11-26T07:12:24.436354+0000","last_fullsized":"2023-11-26T07:12:24.436354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:21.308653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783973+0000","last_change":"2023-11-26T07:12:23.119946+0000","last_active":"2023-11-26T07:12:24.783973+0000","last_peered":"2023-11-26T07:12:24.783973+0000","last_clean":"2023-11-26T07:12:24.783973+0000","last_became_active":"2023-11-26T07:12:23.119137+0000","last_became_peered":"2023-11-26T07:12:23.119137+0000","last_unstale":"2023-11-26T07:12:24.783973+0000","last_undegraded":"2023-11-26T07:12:24.783973+0000","last_fullsized":"2023-11-26T07:12:24.783973+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:10:14.957581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107682+0000","last_change":"2023-11-26T07:12:23.125267+0000","last_active":"2023-11-26T07:12:24.107682+0000","last_peered":"2023-11-26T07:12:24.107682+0000","last_clean":"2023-11-26T07:12:24.107682+0000","last_became_active":"2023-11-26T07:12:23.125006+0000","last_became_peered":"2023-11-26T07:12:23.125006+0000","last_unstale":"2023-11-26T07:12:24.107682+0000","last_undegraded":"2023-11-26T07:12:24.107682+0000","last_fullsized":"2023-11-26T07:12:24.107682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:36:26.407162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107892+0000","last_change":"2023-11-26T07:12:23.127770+0000","last_active":"2023-11-26T07:12:24.107892+0000","last_peered":"2023-11-26T07:12:24.107892+0000","last_clean":"2023-11-26T07:12:24.107892+0000","last_became_active":"2023-11-26T07:12:23.127454+0000","last_became_peered":"2023-11-26T07:12:23.127454+0000","last_unstale":"2023-11-26T07:12:24.107892+0000","last_undegraded":"2023-11-26T07:12:24.107892+0000","last_fullsized":"2023-11-26T07:12:24.107892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:32.714331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783492+0000","last_change":"2023-11-26T07:12:23.115485+0000","last_active":"2023-11-26T07:12:24.783492+0000","last_peered":"2023-11-26T07:12:24.783492+0000","last_clean":"2023-11-26T07:12:24.783492+0000","last_became_active":"2023-11-26T07:12:23.114828+0000","last_became_peered":"2023-11-26T07:12:23.114828+0000","last_unstale":"2023-11-26T07:12:24.783492+0000","last_undegraded":"2023-11-26T07:12:24.783492+0000","last_fullsized":"2023-11-26T07:12:24.783492+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:19:25.018398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105787+0000","last_change":"2023-11-26T07:12:23.116816+0000","last_active":"2023-11-26T07:12:24.105787+0000","last_peered":"2023-11-26T07:12:24.105787+0000","last_clean":"2023-11-26T07:12:24.105787+0000","last_became_active":"2023-11-26T07:12:23.116552+0000","last_became_peered":"2023-11-26T07:12:23.116552+0000","last_unstale":"2023-11-26T07:12:24.105787+0000","last_undegraded":"2023-11-26T07:12:24.105787+0000","last_fullsized":"2023-11-26T07:12:24.105787+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:27.812069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107519+0000","last_change":"2023-11-26T07:12:23.127578+0000","last_active":"2023-11-26T07:12:24.107519+0000","last_peered":"2023-11-26T07:12:24.107519+0000","last_clean":"2023-11-26T07:12:24.107519+0000","last_became_active":"2023-11-26T07:12:23.127267+0000","last_became_peered":"2023-11-26T07:12:23.127267+0000","last_unstale":"2023-11-26T07:12:24.107519+0000","last_undegraded":"2023-11-26T07:12:24.107519+0000","last_fullsized":"2023-11-26T07:12:24.107519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:48:14.958935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107065+0000","last_change":"2023-11-26T07:12:23.120766+0000","last_active":"2023-11-26T07:12:24.107065+0000","last_peered":"2023-11-26T07:12:24.107065+0000","last_clean":"2023-11-26T07:12:24.107065+0000","last_became_active":"2023-11-26T07:12:23.119779+0000","last_became_peered":"2023-11-26T07:12:23.119779+0000","last_unstale":"2023-11-26T07:12:24.107065+0000","last_undegraded":"2023-11-26T07:12:24.107065+0000","last_fullsized":"2023-11-26T07:12:24.107065+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:32.714331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107583+0000","last_change":"2023-11-26T07:12:23.126537+0000","last_active":"2023-11-26T07:12:24.107583+0000","last_peered":"2023-11-26T07:12:24.107583+0000","last_clean":"2023-11-26T07:12:24.107583+0000","last_became_active":"2023-11-26T07:12:23.126278+0000","last_became_peered":"2023-11-26T07:12:23.126278+0000","last_unstale":"2023-11-26T07:12:24.107583+0000","last_undegraded":"2023-11-26T07:12:24.107583+0000","last_fullsized":"2023-11-26T07:12:24.107583+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:56:23.400841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106158+0000","last_change":"2023-11-26T07:12:23.119297+0000","last_active":"2023-11-26T07:12:24.106158+0000","last_peered":"2023-11-26T07:12:24.106158+0000","last_clean":"2023-11-26T07:12:24.106158+0000","last_became_active":"2023-11-26T07:12:23.117145+0000","last_became_peered":"2023-11-26T07:12:23.117145+0000","last_unstale":"2023-11-26T07:12:24.106158+0000","last_undegraded":"2023-11-26T07:12:24.106158+0000","last_fullsized":"2023-11-26T07:12:24.106158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:35:30.734856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105692+0000","last_change":"2023-11-26T07:12:23.120019+0000","last_active":"2023-11-26T07:12:24.105692+0000","last_peered":"2023-11-26T07:12:24.105692+0000","last_clean":"2023-11-26T07:12:24.105692+0000","last_became_active":"2023-11-26T07:12:23.119749+0000","last_became_peered":"2023-11-26T07:12:23.119749+0000","last_unstale":"2023-11-26T07:12:24.105692+0000","last_undegraded":"2023-11-26T07:12:24.105692+0000","last_fullsized":"2023-11-26T07:12:24.105692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:48:14.958935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783344+0000","last_change":"2023-11-26T07:12:23.127994+0000","last_active":"2023-11-26T07:12:24.783344+0000","last_peered":"2023-11-26T07:12:24.783344+0000","last_clean":"2023-11-26T07:12:24.783344+0000","last_became_active":"2023-11-26T07:12:23.127720+0000","last_became_peered":"2023-11-26T07:12:23.127720+0000","last_unstale":"2023-11-26T07:12:24.783344+0000","last_undegraded":"2023-11-26T07:12:24.783344+0000","last_fullsized":"2023-11-26T07:12:24.783344+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:56:03.522118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436107+0000","last_change":"2023-11-26T07:12:23.119253+0000","last_active":"2023-11-26T07:12:24.436107+0000","last_peered":"2023-11-26T07:12:24.436107+0000","last_clean":"2023-11-26T07:12:24.436107+0000","last_became_active":"2023-11-26T07:12:23.118964+0000","last_became_peered":"2023-11-26T07:12:23.118964+0000","last_unstale":"2023-11-26T07:12:24.436107+0000","last_undegraded":"2023-11-26T07:12:24.436107+0000","last_fullsized":"2023-11-26T07:12:24.436107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:00.647047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435924+0000","last_change":"2023-11-26T07:12:23.123099+0000","last_active":"2023-11-26T07:12:24.435924+0000","last_peered":"2023-11-26T07:12:24.435924+0000","last_clean":"2023-11-26T07:12:24.435924+0000","last_became_active":"2023-11-26T07:12:23.122828+0000","last_became_peered":"2023-11-26T07:12:23.122828+0000","last_unstale":"2023-11-26T07:12:24.435924+0000","last_undegraded":"2023-11-26T07:12:24.435924+0000","last_fullsized":"2023-11-26T07:12:24.435924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:39.433005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.782947+0000","last_change":"2023-11-26T07:12:23.120827+0000","last_active":"2023-11-26T07:12:24.782947+0000","last_peered":"2023-11-26T07:12:24.782947+0000","last_clean":"2023-11-26T07:12:24.782947+0000","last_became_active":"2023-11-26T07:12:23.120527+0000","last_became_peered":"2023-11-26T07:12:23.120527+0000","last_unstale":"2023-11-26T07:12:24.782947+0000","last_undegraded":"2023-11-26T07:12:24.782947+0000","last_fullsized":"2023-11-26T07:12:24.782947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:34:09.475730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107221+0000","last_change":"2023-11-26T07:12:23.127310+0000","last_active":"2023-11-26T07:12:24.107221+0000","last_peered":"2023-11-26T07:12:24.107221+0000","last_clean":"2023-11-26T07:12:24.107221+0000","last_became_active":"2023-11-26T07:12:23.126937+0000","last_became_peered":"2023-11-26T07:12:23.126937+0000","last_unstale":"2023-11-26T07:12:24.107221+0000","last_undegraded":"2023-11-26T07:12:24.107221+0000","last_fullsized":"2023-11-26T07:12:24.107221+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:16.181329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783931+0000","last_change":"2023-11-26T07:12:23.130299+0000","last_active":"2023-11-26T07:12:24.783931+0000","last_peered":"2023-11-26T07:12:24.783931+0000","last_clean":"2023-11-26T07:12:24.783931+0000","last_became_active":"2023-11-26T07:12:23.130010+0000","last_became_peered":"2023-11-26T07:12:23.130010+0000","last_unstale":"2023-11-26T07:12:24.783931+0000","last_undegraded":"2023-11-26T07:12:24.783931+0000","last_fullsized":"2023-11-26T07:12:24.783931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:08:24.809552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106182+0000","last_change":"2023-11-26T07:12:23.125361+0000","last_active":"2023-11-26T07:12:24.106182+0000","last_peered":"2023-11-26T07:12:24.106182+0000","last_clean":"2023-11-26T07:12:24.106182+0000","last_became_active":"2023-11-26T07:12:23.125087+0000","last_became_peered":"2023-11-26T07:12:23.125087+0000","last_unstale":"2023-11-26T07:12:24.106182+0000","last_undegraded":"2023-11-26T07:12:24.106182+0000","last_fullsized":"2023-11-26T07:12:24.106182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:04:40.263224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783755+0000","last_change":"2023-11-26T07:12:23.118477+0000","last_active":"2023-11-26T07:12:24.783755+0000","last_peered":"2023-11-26T07:12:24.783755+0000","last_clean":"2023-11-26T07:12:24.783755+0000","last_became_active":"2023-11-26T07:12:23.118189+0000","last_became_peered":"2023-11-26T07:12:23.118189+0000","last_unstale":"2023-11-26T07:12:24.783755+0000","last_undegraded":"2023-11-26T07:12:24.783755+0000","last_fullsized":"2023-11-26T07:12:24.783755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:21.308653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107901+0000","last_change":"2023-11-26T07:12:23.124357+0000","last_active":"2023-11-26T07:12:24.107901+0000","last_peered":"2023-11-26T07:12:24.107901+0000","last_clean":"2023-11-26T07:12:24.107901+0000","last_became_active":"2023-11-26T07:12:23.123664+0000","last_became_peered":"2023-11-26T07:12:23.123664+0000","last_unstale":"2023-11-26T07:12:24.107901+0000","last_undegraded":"2023-11-26T07:12:24.107901+0000","last_fullsized":"2023-11-26T07:12:24.107901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:27.812069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435935+0000","last_change":"2023-11-26T07:12:23.123725+0000","last_active":"2023-11-26T07:12:24.435935+0000","last_peered":"2023-11-26T07:12:24.435935+0000","last_clean":"2023-11-26T07:12:24.435935+0000","last_became_active":"2023-11-26T07:12:23.123368+0000","last_became_peered":"2023-11-26T07:12:23.123368+0000","last_unstale":"2023-11-26T07:12:24.435935+0000","last_undegraded":"2023-11-26T07:12:24.435935+0000","last_fullsized":"2023-11-26T07:12:24.435935+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:10:21.444998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107036+0000","last_change":"2023-11-26T07:12:23.128368+0000","last_active":"2023-11-26T07:12:24.107036+0000","last_peered":"2023-11-26T07:12:24.107036+0000","last_clean":"2023-11-26T07:12:24.107036+0000","last_became_active":"2023-11-26T07:12:23.128059+0000","last_became_peered":"2023-11-26T07:12:23.128059+0000","last_unstale":"2023-11-26T07:12:24.107036+0000","last_undegraded":"2023-11-26T07:12:24.107036+0000","last_fullsized":"2023-11-26T07:12:24.107036+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:44:46.385542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783560+0000","last_change":"2023-11-26T07:12:23.124902+0000","last_active":"2023-11-26T07:12:24.783560+0000","last_peered":"2023-11-26T07:12:24.783560+0000","last_clean":"2023-11-26T07:12:24.783560+0000","last_became_active":"2023-11-26T07:12:23.124619+0000","last_became_peered":"2023-11-26T07:12:23.124619+0000","last_unstale":"2023-11-26T07:12:24.783560+0000","last_undegraded":"2023-11-26T07:12:24.783560+0000","last_fullsized":"2023-11-26T07:12:24.783560+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:59:53.542038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107219+0000","last_change":"2023-11-26T07:12:23.121278+0000","last_active":"2023-11-26T07:12:24.107219+0000","last_peered":"2023-11-26T07:12:24.107219+0000","last_clean":"2023-11-26T07:12:24.107219+0000","last_became_active":"2023-11-26T07:12:23.120953+0000","last_became_peered":"2023-11-26T07:12:23.120953+0000","last_unstale":"2023-11-26T07:12:24.107219+0000","last_undegraded":"2023-11-26T07:12:24.107219+0000","last_fullsized":"2023-11-26T07:12:24.107219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:31:12.988370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783408+0000","last_change":"2023-11-26T07:12:23.117615+0000","last_active":"2023-11-26T07:12:24.783408+0000","last_peered":"2023-11-26T07:12:24.783408+0000","last_clean":"2023-11-26T07:12:24.783408+0000","last_became_active":"2023-11-26T07:12:23.117272+0000","last_became_peered":"2023-11-26T07:12:23.117272+0000","last_unstale":"2023-11-26T07:12:24.783408+0000","last_undegraded":"2023-11-26T07:12:24.783408+0000","last_fullsized":"2023-11-26T07:12:24.783408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:34.579111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783648+0000","last_change":"2023-11-26T07:12:22.120139+0000","last_active":"2023-11-26T07:12:24.783648+0000","last_peered":"2023-11-26T07:12:24.783648+0000","last_clean":"2023-11-26T07:12:24.783648+0000","last_became_active":"2023-11-26T07:12:22.119877+0000","last_became_peered":"2023-11-26T07:12:22.119877+0000","last_unstale":"2023-11-26T07:12:24.783648+0000","last_undegraded":"2023-11-26T07:12:24.783648+0000","last_fullsized":"2023-11-26T07:12:24.783648+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:26:37.066177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783604+0000","last_change":"2023-11-26T07:12:23.128289+0000","last_active":"2023-11-26T07:12:24.783604+0000","last_peered":"2023-11-26T07:12:24.783604+0000","last_clean":"2023-11-26T07:12:24.783604+0000","last_became_active":"2023-11-26T07:12:23.127752+0000","last_became_peered":"2023-11-26T07:12:23.127752+0000","last_unstale":"2023-11-26T07:12:24.783604+0000","last_undegraded":"2023-11-26T07:12:24.783604+0000","last_fullsized":"2023-11-26T07:12:24.783604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:36.670009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783480+0000","last_change":"2023-11-26T07:12:22.116096+0000","last_active":"2023-11-26T07:12:24.783480+0000","last_peered":"2023-11-26T07:12:24.783480+0000","last_clean":"2023-11-26T07:12:24.783480+0000","last_became_active":"2023-11-26T07:12:22.115803+0000","last_became_peered":"2023-11-26T07:12:22.115803+0000","last_unstale":"2023-11-26T07:12:24.783480+0000","last_undegraded":"2023-11-26T07:12:24.783480+0000","last_fullsized":"2023-11-26T07:12:24.783480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:19:24.013733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436074+0000","last_change":"2023-11-26T07:12:23.123058+0000","last_active":"2023-11-26T07:12:24.436074+0000","last_peered":"2023-11-26T07:12:24.436074+0000","last_clean":"2023-11-26T07:12:24.436074+0000","last_became_active":"2023-11-26T07:12:23.122789+0000","last_became_peered":"2023-11-26T07:12:23.122789+0000","last_unstale":"2023-11-26T07:12:24.436074+0000","last_undegraded":"2023-11-26T07:12:24.436074+0000","last_fullsized":"2023-11-26T07:12:24.436074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:08.151898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106204+0000","last_change":"2023-11-26T07:12:22.119031+0000","last_active":"2023-11-26T07:12:24.106204+0000","last_peered":"2023-11-26T07:12:24.106204+0000","last_clean":"2023-11-26T07:12:24.106204+0000","last_became_active":"2023-11-26T07:12:22.118581+0000","last_became_peered":"2023-11-26T07:12:22.118581+0000","last_unstale":"2023-11-26T07:12:24.106204+0000","last_undegraded":"2023-11-26T07:12:24.106204+0000","last_fullsized":"2023-11-26T07:12:24.106204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:42:59.642382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783744+0000","last_change":"2023-11-26T07:12:23.118863+0000","last_active":"2023-11-26T07:12:24.783744+0000","last_peered":"2023-11-26T07:12:24.783744+0000","last_clean":"2023-11-26T07:12:24.783744+0000","last_became_active":"2023-11-26T07:12:23.118567+0000","last_became_peered":"2023-11-26T07:12:23.118567+0000","last_unstale":"2023-11-26T07:12:24.783744+0000","last_undegraded":"2023-11-26T07:12:24.783744+0000","last_fullsized":"2023-11-26T07:12:24.783744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:20:06.180831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783439+0000","last_change":"2023-11-26T07:12:22.122912+0000","last_active":"2023-11-26T07:12:24.783439+0000","last_peered":"2023-11-26T07:12:24.783439+0000","last_clean":"2023-11-26T07:12:24.783439+0000","last_became_active":"2023-11-26T07:12:22.122591+0000","last_became_peered":"2023-11-26T07:12:22.122591+0000","last_unstale":"2023-11-26T07:12:24.783439+0000","last_undegraded":"2023-11-26T07:12:24.783439+0000","last_fullsized":"2023-11-26T07:12:24.783439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:18:23.467389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107234+0000","last_change":"2023-11-26T07:12:23.120487+0000","last_active":"2023-11-26T07:12:24.107234+0000","last_peered":"2023-11-26T07:12:24.107234+0000","last_clean":"2023-11-26T07:12:24.107234+0000","last_became_active":"2023-11-26T07:12:23.120182+0000","last_became_peered":"2023-11-26T07:12:23.120182+0000","last_unstale":"2023-11-26T07:12:24.107234+0000","last_undegraded":"2023-11-26T07:12:24.107234+0000","last_fullsized":"2023-11-26T07:12:24.107234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:04:25.608116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436058+0000","last_change":"2023-11-26T07:12:22.121764+0000","last_active":"2023-11-26T07:12:24.436058+0000","last_peered":"2023-11-26T07:12:24.436058+0000","last_clean":"2023-11-26T07:12:24.436058+0000","last_became_active":"2023-11-26T07:12:22.121534+0000","last_became_peered":"2023-11-26T07:12:22.121534+0000","last_unstale":"2023-11-26T07:12:24.436058+0000","last_undegraded":"2023-11-26T07:12:24.436058+0000","last_fullsized":"2023-11-26T07:12:24.436058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:13.437694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107674+0000","last_change":"2023-11-26T07:12:23.123986+0000","last_active":"2023-11-26T07:12:24.107674+0000","last_peered":"2023-11-26T07:12:24.107674+0000","last_clean":"2023-11-26T07:12:24.107674+0000","last_became_active":"2023-11-26T07:12:23.123625+0000","last_became_peered":"2023-11-26T07:12:23.123625+0000","last_unstale":"2023-11-26T07:12:24.107674+0000","last_undegraded":"2023-11-26T07:12:24.107674+0000","last_fullsized":"2023-11-26T07:12:24.107674+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:44:46.385542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435982+0000","last_change":"2023-11-26T07:12:22.116348+0000","last_active":"2023-11-26T07:12:24.435982+0000","last_peered":"2023-11-26T07:12:24.435982+0000","last_clean":"2023-11-26T07:12:24.435982+0000","last_became_active":"2023-11-26T07:12:22.116075+0000","last_became_peered":"2023-11-26T07:12:22.116075+0000","last_unstale":"2023-11-26T07:12:24.435982+0000","last_undegraded":"2023-11-26T07:12:24.435982+0000","last_fullsized":"2023-11-26T07:12:24.435982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:26:37.066177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435821+0000","last_change":"2023-11-26T07:12:23.126362+0000","last_active":"2023-11-26T07:12:24.435821+0000","last_peered":"2023-11-26T07:12:24.435821+0000","last_clean":"2023-11-26T07:12:24.435821+0000","last_became_active":"2023-11-26T07:12:23.125921+0000","last_became_peered":"2023-11-26T07:12:23.125921+0000","last_unstale":"2023-11-26T07:12:24.435821+0000","last_undegraded":"2023-11-26T07:12:24.435821+0000","last_fullsized":"2023-11-26T07:12:24.435821+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:34.579111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105381+0000","last_change":"2023-11-26T07:12:22.117803+0000","last_active":"2023-11-26T07:12:24.105381+0000","last_peered":"2023-11-26T07:12:24.105381+0000","last_clean":"2023-11-26T07:12:24.105381+0000","last_became_active":"2023-11-26T07:12:22.117526+0000","last_became_peered":"2023-11-26T07:12:22.117526+0000","last_unstale":"2023-11-26T07:12:24.105381+0000","last_undegraded":"2023-11-26T07:12:24.105381+0000","last_fullsized":"2023-11-26T07:12:24.105381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:33.574446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783248+0000","last_change":"2023-11-26T07:12:23.127956+0000","last_active":"2023-11-26T07:12:24.783248+0000","last_peered":"2023-11-26T07:12:24.783248+0000","last_clean":"2023-11-26T07:12:24.783248+0000","last_became_active":"2023-11-26T07:12:23.127669+0000","last_became_peered":"2023-11-26T07:12:23.127669+0000","last_unstale":"2023-11-26T07:12:24.783248+0000","last_undegraded":"2023-11-26T07:12:24.783248+0000","last_fullsized":"2023-11-26T07:12:24.783248+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:39.433005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435728+0000","last_change":"2023-11-26T07:12:22.118871+0000","last_active":"2023-11-26T07:12:24.435728+0000","last_peered":"2023-11-26T07:12:24.435728+0000","last_clean":"2023-11-26T07:12:24.435728+0000","last_became_active":"2023-11-26T07:12:22.118542+0000","last_became_peered":"2023-11-26T07:12:22.118542+0000","last_unstale":"2023-11-26T07:12:24.435728+0000","last_undegraded":"2023-11-26T07:12:24.435728+0000","last_fullsized":"2023-11-26T07:12:24.435728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:47.565317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.108128+0000","last_change":"2023-11-26T07:12:23.128059+0000","last_active":"2023-11-26T07:12:24.108128+0000","last_peered":"2023-11-26T07:12:24.108128+0000","last_clean":"2023-11-26T07:12:24.108128+0000","last_became_active":"2023-11-26T07:12:23.126648+0000","last_became_peered":"2023-11-26T07:12:23.126648+0000","last_unstale":"2023-11-26T07:12:24.108128+0000","last_undegraded":"2023-11-26T07:12:24.108128+0000","last_fullsized":"2023-11-26T07:12:24.108128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:04:25.608116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105840+0000","last_change":"2023-11-26T07:12:22.113988+0000","last_active":"2023-11-26T07:12:24.105840+0000","last_peered":"2023-11-26T07:12:24.105840+0000","last_clean":"2023-11-26T07:12:24.105840+0000","last_became_active":"2023-11-26T07:12:22.113682+0000","last_became_peered":"2023-11-26T07:12:22.113682+0000","last_unstale":"2023-11-26T07:12:24.105840+0000","last_undegraded":"2023-11-26T07:12:24.105840+0000","last_fullsized":"2023-11-26T07:12:24.105840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:38.428339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106012+0000","last_change":"2023-11-26T07:12:23.119095+0000","last_active":"2023-11-26T07:12:24.106012+0000","last_peered":"2023-11-26T07:12:24.106012+0000","last_clean":"2023-11-26T07:12:24.106012+0000","last_became_active":"2023-11-26T07:12:23.118820+0000","last_became_peered":"2023-11-26T07:12:23.118820+0000","last_unstale":"2023-11-26T07:12:24.106012+0000","last_undegraded":"2023-11-26T07:12:24.106012+0000","last_fullsized":"2023-11-26T07:12:24.106012+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:31:12.988370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783558+0000","last_change":"2023-11-26T07:12:22.117436+0000","last_active":"2023-11-26T07:12:24.783558+0000","last_peered":"2023-11-26T07:12:24.783558+0000","last_clean":"2023-11-26T07:12:24.783558+0000","last_became_active":"2023-11-26T07:12:22.117146+0000","last_became_peered":"2023-11-26T07:12:22.117146+0000","last_unstale":"2023-11-26T07:12:24.783558+0000","last_undegraded":"2023-11-26T07:12:24.783558+0000","last_fullsized":"2023-11-26T07:12:24.783558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:16.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105704+0000","last_change":"2023-11-26T07:12:23.123102+0000","last_active":"2023-11-26T07:12:24.105704+0000","last_peered":"2023-11-26T07:12:24.105704+0000","last_clean":"2023-11-26T07:12:24.105704+0000","last_became_active":"2023-11-26T07:12:23.122882+0000","last_became_peered":"2023-11-26T07:12:23.122882+0000","last_unstale":"2023-11-26T07:12:24.105704+0000","last_undegraded":"2023-11-26T07:12:24.105704+0000","last_fullsized":"2023-11-26T07:12:24.105704+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:50.833212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.108005+0000","last_change":"2023-11-26T07:12:22.117919+0000","last_active":"2023-11-26T07:12:24.108005+0000","last_peered":"2023-11-26T07:12:24.108005+0000","last_clean":"2023-11-26T07:12:24.108005+0000","last_became_active":"2023-11-26T07:12:22.117629+0000","last_became_peered":"2023-11-26T07:12:22.117629+0000","last_unstale":"2023-11-26T07:12:24.108005+0000","last_undegraded":"2023-11-26T07:12:24.108005+0000","last_fullsized":"2023-11-26T07:12:24.108005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:20:31.048080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107951+0000","last_change":"2023-11-26T07:12:23.127121+0000","last_active":"2023-11-26T07:12:24.107951+0000","last_peered":"2023-11-26T07:12:24.107951+0000","last_clean":"2023-11-26T07:12:24.107951+0000","last_became_active":"2023-11-26T07:12:23.126833+0000","last_became_peered":"2023-11-26T07:12:23.126833+0000","last_unstale":"2023-11-26T07:12:24.107951+0000","last_undegraded":"2023-11-26T07:12:24.107951+0000","last_fullsized":"2023-11-26T07:12:24.107951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:31:12.988370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105854+0000","last_change":"2023-11-26T07:12:22.120536+0000","last_active":"2023-11-26T07:12:24.105854+0000","last_peered":"2023-11-26T07:12:24.105854+0000","last_clean":"2023-11-26T07:12:24.105854+0000","last_became_active":"2023-11-26T07:12:22.120245+0000","last_became_peered":"2023-11-26T07:12:22.120245+0000","last_unstale":"2023-11-26T07:12:24.105854+0000","last_undegraded":"2023-11-26T07:12:24.105854+0000","last_fullsized":"2023-11-26T07:12:24.105854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:20:58.962312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783630+0000","last_change":"2023-11-26T07:12:23.128085+0000","last_active":"2023-11-26T07:12:24.783630+0000","last_peered":"2023-11-26T07:12:24.783630+0000","last_clean":"2023-11-26T07:12:24.783630+0000","last_became_active":"2023-11-26T07:12:23.127767+0000","last_became_peered":"2023-11-26T07:12:23.127767+0000","last_unstale":"2023-11-26T07:12:24.783630+0000","last_undegraded":"2023-11-26T07:12:24.783630+0000","last_fullsized":"2023-11-26T07:12:24.783630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:00.647047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436178+0000","last_change":"2023-11-26T07:12:22.119236+0000","last_active":"2023-11-26T07:12:24.436178+0000","last_peered":"2023-11-26T07:12:24.436178+0000","last_clean":"2023-11-26T07:12:24.436178+0000","last_became_active":"2023-11-26T07:12:22.118790+0000","last_became_peered":"2023-11-26T07:12:22.118790+0000","last_unstale":"2023-11-26T07:12:24.436178+0000","last_undegraded":"2023-11-26T07:12:24.436178+0000","last_fullsized":"2023-11-26T07:12:24.436178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:34:08.471064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105685+0000","last_change":"2023-11-26T07:12:23.116580+0000","last_active":"2023-11-26T07:12:24.105685+0000","last_peered":"2023-11-26T07:12:24.105685+0000","last_clean":"2023-11-26T07:12:24.105685+0000","last_became_active":"2023-11-26T07:12:23.116112+0000","last_became_peered":"2023-11-26T07:12:23.116112+0000","last_unstale":"2023-11-26T07:12:24.105685+0000","last_undegraded":"2023-11-26T07:12:24.105685+0000","last_fullsized":"2023-11-26T07:12:24.105685+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:44:46.385542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783730+0000","last_change":"2023-11-26T07:12:22.122113+0000","last_active":"2023-11-26T07:12:24.783730+0000","last_peered":"2023-11-26T07:12:24.783730+0000","last_clean":"2023-11-26T07:12:24.783730+0000","last_became_active":"2023-11-26T07:12:22.121809+0000","last_became_peered":"2023-11-26T07:12:22.121809+0000","last_unstale":"2023-11-26T07:12:24.783730+0000","last_undegraded":"2023-11-26T07:12:24.783730+0000","last_fullsized":"2023-11-26T07:12:24.783730+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:13.437694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107371+0000","last_change":"2023-11-26T07:12:23.117028+0000","last_active":"2023-11-26T07:12:24.107371+0000","last_peered":"2023-11-26T07:12:24.107371+0000","last_clean":"2023-11-26T07:12:24.107371+0000","last_became_active":"2023-11-26T07:12:23.116733+0000","last_became_peered":"2023-11-26T07:12:23.116733+0000","last_unstale":"2023-11-26T07:12:24.107371+0000","last_undegraded":"2023-11-26T07:12:24.107371+0000","last_fullsized":"2023-11-26T07:12:24.107371+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:04:40.263224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105463+0000","last_change":"2023-11-26T07:12:22.114756+0000","last_active":"2023-11-26T07:12:24.105463+0000","last_peered":"2023-11-26T07:12:24.105463+0000","last_clean":"2023-11-26T07:12:24.105463+0000","last_became_active":"2023-11-26T07:12:22.114070+0000","last_became_peered":"2023-11-26T07:12:22.114070+0000","last_unstale":"2023-11-26T07:12:24.105463+0000","last_undegraded":"2023-11-26T07:12:24.105463+0000","last_fullsized":"2023-11-26T07:12:24.105463+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:35.665344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783391+0000","last_change":"2023-11-26T07:12:23.129786+0000","last_active":"2023-11-26T07:12:24.783391+0000","last_peered":"2023-11-26T07:12:24.783391+0000","last_clean":"2023-11-26T07:12:24.783391+0000","last_became_active":"2023-11-26T07:12:23.129012+0000","last_became_peered":"2023-11-26T07:12:23.129012+0000","last_unstale":"2023-11-26T07:12:24.783391+0000","last_undegraded":"2023-11-26T07:12:24.783391+0000","last_fullsized":"2023-11-26T07:12:24.783391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:14.442359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105342+0000","last_change":"2023-11-26T07:12:22.118424+0000","last_active":"2023-11-26T07:12:24.105342+0000","last_peered":"2023-11-26T07:12:24.105342+0000","last_clean":"2023-11-26T07:12:24.105342+0000","last_became_active":"2023-11-26T07:12:22.118156+0000","last_became_peered":"2023-11-26T07:12:22.118156+0000","last_unstale":"2023-11-26T07:12:24.105342+0000","last_undegraded":"2023-11-26T07:12:24.105342+0000","last_fullsized":"2023-11-26T07:12:24.105342+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:29:52.714251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436283+0000","last_change":"2023-11-26T07:12:23.129656+0000","last_active":"2023-11-26T07:12:24.436283+0000","last_peered":"2023-11-26T07:12:24.436283+0000","last_clean":"2023-11-26T07:12:24.436283+0000","last_became_active":"2023-11-26T07:12:23.129396+0000","last_became_peered":"2023-11-26T07:12:23.129396+0000","last_unstale":"2023-11-26T07:12:24.436283+0000","last_undegraded":"2023-11-26T07:12:24.436283+0000","last_fullsized":"2023-11-26T07:12:24.436283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:20:06.180831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783905+0000","last_change":"2023-11-26T07:12:22.123326+0000","last_active":"2023-11-26T07:12:24.783905+0000","last_peered":"2023-11-26T07:12:24.783905+0000","last_clean":"2023-11-26T07:12:24.783905+0000","last_became_active":"2023-11-26T07:12:22.123112+0000","last_became_peered":"2023-11-26T07:12:22.123112+0000","last_unstale":"2023-11-26T07:12:24.783905+0000","last_undegraded":"2023-11-26T07:12:24.783905+0000","last_fullsized":"2023-11-26T07:12:24.783905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:04:18.883671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.108193+0000","last_change":"2023-11-26T07:12:23.127302+0000","last_active":"2023-11-26T07:12:24.108193+0000","last_peered":"2023-11-26T07:12:24.108193+0000","last_clean":"2023-11-26T07:12:24.108193+0000","last_became_active":"2023-11-26T07:12:23.126917+0000","last_became_peered":"2023-11-26T07:12:23.126917+0000","last_unstale":"2023-11-26T07:12:24.108193+0000","last_undegraded":"2023-11-26T07:12:24.108193+0000","last_fullsized":"2023-11-26T07:12:24.108193+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:04:40.263224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436273+0000","last_change":"2023-11-26T07:12:22.119703+0000","last_active":"2023-11-26T07:12:24.436273+0000","last_peered":"2023-11-26T07:12:24.436273+0000","last_clean":"2023-11-26T07:12:24.436273+0000","last_became_active":"2023-11-26T07:12:22.118932+0000","last_became_peered":"2023-11-26T07:12:22.118932+0000","last_unstale":"2023-11-26T07:12:24.436273+0000","last_undegraded":"2023-11-26T07:12:24.436273+0000","last_fullsized":"2023-11-26T07:12:24.436273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:43:37.623929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783582+0000","last_change":"2023-11-26T07:12:23.121769+0000","last_active":"2023-11-26T07:12:24.783582+0000","last_peered":"2023-11-26T07:12:24.783582+0000","last_clean":"2023-11-26T07:12:24.783582+0000","last_became_active":"2023-11-26T07:12:23.121486+0000","last_became_peered":"2023-11-26T07:12:23.121486+0000","last_unstale":"2023-11-26T07:12:24.783582+0000","last_undegraded":"2023-11-26T07:12:24.783582+0000","last_fullsized":"2023-11-26T07:12:24.783582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:08.151898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436168+0000","last_change":"2023-11-26T07:12:22.119541+0000","last_active":"2023-11-26T07:12:24.436168+0000","last_peered":"2023-11-26T07:12:24.436168+0000","last_clean":"2023-11-26T07:12:24.436168+0000","last_became_active":"2023-11-26T07:12:22.119135+0000","last_became_peered":"2023-11-26T07:12:22.119135+0000","last_unstale":"2023-11-26T07:12:24.436168+0000","last_undegraded":"2023-11-26T07:12:24.436168+0000","last_fullsized":"2023-11-26T07:12:24.436168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:40:03.997115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783117+0000","last_change":"2023-11-26T07:12:23.120751+0000","last_active":"2023-11-26T07:12:24.783117+0000","last_peered":"2023-11-26T07:12:24.783117+0000","last_clean":"2023-11-26T07:12:24.783117+0000","last_became_active":"2023-11-26T07:12:23.120488+0000","last_became_peered":"2023-11-26T07:12:23.120488+0000","last_unstale":"2023-11-26T07:12:24.783117+0000","last_undegraded":"2023-11-26T07:12:24.783117+0000","last_fullsized":"2023-11-26T07:12:24.783117+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:08.514156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436187+0000","last_change":"2023-11-26T07:12:22.115783+0000","last_active":"2023-11-26T07:12:24.436187+0000","last_peered":"2023-11-26T07:12:24.436187+0000","last_clean":"2023-11-26T07:12:24.436187+0000","last_became_active":"2023-11-26T07:12:22.115497+0000","last_became_peered":"2023-11-26T07:12:22.115497+0000","last_unstale":"2023-11-26T07:12:24.436187+0000","last_undegraded":"2023-11-26T07:12:24.436187+0000","last_fullsized":"2023-11-26T07:12:24.436187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:16.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783186+0000","last_change":"2023-11-26T07:12:23.120509+0000","last_active":"2023-11-26T07:12:24.783186+0000","last_peered":"2023-11-26T07:12:24.783186+0000","last_clean":"2023-11-26T07:12:24.783186+0000","last_became_active":"2023-11-26T07:12:23.120223+0000","last_became_peered":"2023-11-26T07:12:23.120223+0000","last_unstale":"2023-11-26T07:12:24.783186+0000","last_undegraded":"2023-11-26T07:12:24.783186+0000","last_fullsized":"2023-11-26T07:12:24.783186+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:43:38.628594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783110+0000","last_change":"2023-11-26T07:12:22.120233+0000","last_active":"2023-11-26T07:12:24.783110+0000","last_peered":"2023-11-26T07:12:24.783110+0000","last_clean":"2023-11-26T07:12:24.783110+0000","last_became_active":"2023-11-26T07:12:22.119963+0000","last_became_peered":"2023-11-26T07:12:22.119963+0000","last_unstale":"2023-11-26T07:12:24.783110+0000","last_undegraded":"2023-11-26T07:12:24.783110+0000","last_fullsized":"2023-11-26T07:12:24.783110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:56:02.517453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783444+0000","last_change":"2023-11-26T07:12:23.112011+0000","last_active":"2023-11-26T07:12:24.783444+0000","last_peered":"2023-11-26T07:12:24.783444+0000","last_clean":"2023-11-26T07:12:24.783444+0000","last_became_active":"2023-11-26T07:12:23.111720+0000","last_became_peered":"2023-11-26T07:12:23.111720+0000","last_unstale":"2023-11-26T07:12:24.783444+0000","last_undegraded":"2023-11-26T07:12:24.783444+0000","last_fullsized":"2023-11-26T07:12:24.783444+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:53:56.870495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783426+0000","last_change":"2023-11-26T07:12:22.125191+0000","last_active":"2023-11-26T07:12:24.783426+0000","last_peered":"2023-11-26T07:12:24.783426+0000","last_clean":"2023-11-26T07:12:24.783426+0000","last_became_active":"2023-11-26T07:12:22.124982+0000","last_became_peered":"2023-11-26T07:12:22.124982+0000","last_unstale":"2023-11-26T07:12:24.783426+0000","last_undegraded":"2023-11-26T07:12:24.783426+0000","last_fullsized":"2023-11-26T07:12:24.783426+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:16.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.782971+0000","last_change":"2023-11-26T07:12:23.112066+0000","last_active":"2023-11-26T07:12:24.782971+0000","last_peered":"2023-11-26T07:12:24.782971+0000","last_clean":"2023-11-26T07:12:24.782971+0000","last_became_active":"2023-11-26T07:12:23.111771+0000","last_became_peered":"2023-11-26T07:12:23.111771+0000","last_unstale":"2023-11-26T07:12:24.782971+0000","last_undegraded":"2023-11-26T07:12:24.782971+0000","last_fullsized":"2023-11-26T07:12:24.782971+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:40:05.001780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436045+0000","last_change":"2023-11-26T07:12:22.120958+0000","last_active":"2023-11-26T07:12:24.436045+0000","last_peered":"2023-11-26T07:12:24.436045+0000","last_clean":"2023-11-26T07:12:24.436045+0000","last_became_active":"2023-11-26T07:12:22.120624+0000","last_became_peered":"2023-11-26T07:12:22.120624+0000","last_unstale":"2023-11-26T07:12:24.436045+0000","last_undegraded":"2023-11-26T07:12:24.436045+0000","last_fullsized":"2023-11-26T07:12:24.436045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:19:24.013733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107307+0000","last_change":"2023-11-26T07:12:23.128800+0000","last_active":"2023-11-26T07:12:24.107307+0000","last_peered":"2023-11-26T07:12:24.107307+0000","last_clean":"2023-11-26T07:12:24.107307+0000","last_became_active":"2023-11-26T07:12:23.128396+0000","last_became_peered":"2023-11-26T07:12:23.128396+0000","last_unstale":"2023-11-26T07:12:24.107307+0000","last_undegraded":"2023-11-26T07:12:24.107307+0000","last_fullsized":"2023-11-26T07:12:24.107307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:35:30.734856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107333+0000","last_change":"2023-11-26T07:12:22.117840+0000","last_active":"2023-11-26T07:12:24.107333+0000","last_peered":"2023-11-26T07:12:24.107333+0000","last_clean":"2023-11-26T07:12:24.107333+0000","last_became_active":"2023-11-26T07:12:22.117572+0000","last_became_peered":"2023-11-26T07:12:22.117572+0000","last_unstale":"2023-11-26T07:12:24.107333+0000","last_undegraded":"2023-11-26T07:12:24.107333+0000","last_fullsized":"2023-11-26T07:12:24.107333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:40:48.702292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783429+0000","last_change":"2023-11-26T07:12:23.129557+0000","last_active":"2023-11-26T07:12:24.783429+0000","last_peered":"2023-11-26T07:12:24.783429+0000","last_clean":"2023-11-26T07:12:24.783429+0000","last_became_active":"2023-11-26T07:12:23.129286+0000","last_became_peered":"2023-11-26T07:12:23.129286+0000","last_unstale":"2023-11-26T07:12:24.783429+0000","last_undegraded":"2023-11-26T07:12:24.783429+0000","last_fullsized":"2023-11-26T07:12:24.783429+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:17:02.369629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783745+0000","last_change":"2023-11-26T07:12:22.120133+0000","last_active":"2023-11-26T07:12:24.783745+0000","last_peered":"2023-11-26T07:12:24.783745+0000","last_clean":"2023-11-26T07:12:24.783745+0000","last_became_active":"2023-11-26T07:12:22.119839+0000","last_became_peered":"2023-11-26T07:12:22.119839+0000","last_unstale":"2023-11-26T07:12:24.783745+0000","last_undegraded":"2023-11-26T07:12:24.783745+0000","last_fullsized":"2023-11-26T07:12:24.783745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:45:48.365800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.108120+0000","last_change":"2023-11-26T07:12:23.127529+0000","last_active":"2023-11-26T07:12:24.108120+0000","last_peered":"2023-11-26T07:12:24.108120+0000","last_clean":"2023-11-26T07:12:24.108120+0000","last_became_active":"2023-11-26T07:12:23.127220+0000","last_became_peered":"2023-11-26T07:12:23.127220+0000","last_unstale":"2023-11-26T07:12:24.108120+0000","last_undegraded":"2023-11-26T07:12:24.108120+0000","last_fullsized":"2023-11-26T07:12:24.108120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:35:30.734856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:25.138902+0000","last_change":"2023-11-26T07:12:22.120219+0000","last_active":"2023-11-26T07:12:25.138902+0000","last_peered":"2023-11-26T07:12:25.138902+0000","last_clean":"2023-11-26T07:12:25.138902+0000","last_became_active":"2023-11-26T07:12:22.119933+0000","last_became_peered":"2023-11-26T07:12:22.119933+0000","last_unstale":"2023-11-26T07:12:25.138902+0000","last_undegraded":"2023-11-26T07:12:25.138902+0000","last_fullsized":"2023-11-26T07:12:25.138902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:26:37.066177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783628+0000","last_change":"2023-11-26T07:12:23.115136+0000","last_active":"2023-11-26T07:12:24.783628+0000","last_peered":"2023-11-26T07:12:24.783628+0000","last_clean":"2023-11-26T07:12:24.783628+0000","last_became_active":"2023-11-26T07:12:23.114814+0000","last_became_peered":"2023-11-26T07:12:23.114814+0000","last_unstale":"2023-11-26T07:12:24.783628+0000","last_undegraded":"2023-11-26T07:12:24.783628+0000","last_fullsized":"2023-11-26T07:12:24.783628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:36.670009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783769+0000","last_change":"2023-11-26T07:12:22.121582+0000","last_active":"2023-11-26T07:12:24.783769+0000","last_peered":"2023-11-26T07:12:24.783769+0000","last_clean":"2023-11-26T07:12:24.783769+0000","last_became_active":"2023-11-26T07:12:22.121235+0000","last_became_peered":"2023-11-26T07:12:22.121235+0000","last_unstale":"2023-11-26T07:12:24.783769+0000","last_undegraded":"2023-11-26T07:12:24.783769+0000","last_fullsized":"2023-11-26T07:12:24.783769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:16.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783508+0000","last_change":"2023-11-26T07:12:23.126859+0000","last_active":"2023-11-26T07:12:24.783508+0000","last_peered":"2023-11-26T07:12:24.783508+0000","last_clean":"2023-11-26T07:12:24.783508+0000","last_became_active":"2023-11-26T07:12:23.126578+0000","last_became_peered":"2023-11-26T07:12:23.126578+0000","last_unstale":"2023-11-26T07:12:24.783508+0000","last_undegraded":"2023-11-26T07:12:24.783508+0000","last_fullsized":"2023-11-26T07:12:24.783508+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:34:09.475730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783069+0000","last_change":"2023-11-26T07:12:22.116056+0000","last_active":"2023-11-26T07:12:24.783069+0000","last_peered":"2023-11-26T07:12:24.783069+0000","last_clean":"2023-11-26T07:12:24.783069+0000","last_became_active":"2023-11-26T07:12:22.115779+0000","last_became_peered":"2023-11-26T07:12:22.115779+0000","last_unstale":"2023-11-26T07:12:24.783069+0000","last_undegraded":"2023-11-26T07:12:24.783069+0000","last_fullsized":"2023-11-26T07:12:24.783069+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:46:56.965203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783194+0000","last_change":"2023-11-26T07:12:23.128745+0000","last_active":"2023-11-26T07:12:24.783194+0000","last_peered":"2023-11-26T07:12:24.783194+0000","last_clean":"2023-11-26T07:12:24.783194+0000","last_became_active":"2023-11-26T07:12:23.128388+0000","last_became_peered":"2023-11-26T07:12:23.128388+0000","last_unstale":"2023-11-26T07:12:24.783194+0000","last_undegraded":"2023-11-26T07:12:24.783194+0000","last_fullsized":"2023-11-26T07:12:24.783194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:59:53.542038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783740+0000","last_change":"2023-11-26T07:12:22.121254+0000","last_active":"2023-11-26T07:12:24.783740+0000","last_peered":"2023-11-26T07:12:24.783740+0000","last_clean":"2023-11-26T07:12:24.783740+0000","last_became_active":"2023-11-26T07:12:22.120394+0000","last_became_peered":"2023-11-26T07:12:22.120394+0000","last_unstale":"2023-11-26T07:12:24.783740+0000","last_undegraded":"2023-11-26T07:12:24.783740+0000","last_fullsized":"2023-11-26T07:12:24.783740+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:34:08.471064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105410+0000","last_change":"2023-11-26T07:12:23.118888+0000","last_active":"2023-11-26T07:12:24.105410+0000","last_peered":"2023-11-26T07:12:24.105410+0000","last_clean":"2023-11-26T07:12:24.105410+0000","last_became_active":"2023-11-26T07:12:23.118579+0000","last_became_peered":"2023-11-26T07:12:23.118579+0000","last_unstale":"2023-11-26T07:12:24.105410+0000","last_undegraded":"2023-11-26T07:12:24.105410+0000","last_fullsized":"2023-11-26T07:12:24.105410+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:16.181329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435956+0000","last_change":"2023-11-26T07:12:22.115573+0000","last_active":"2023-11-26T07:12:24.435956+0000","last_peered":"2023-11-26T07:12:24.435956+0000","last_clean":"2023-11-26T07:12:24.435956+0000","last_became_active":"2023-11-26T07:12:22.115247+0000","last_became_peered":"2023-11-26T07:12:22.115247+0000","last_unstale":"2023-11-26T07:12:24.435956+0000","last_undegraded":"2023-11-26T07:12:24.435956+0000","last_fullsized":"2023-11-26T07:12:24.435956+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:45:48.365800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435858+0000","last_change":"2023-11-26T07:12:23.120415+0000","last_active":"2023-11-26T07:12:24.435858+0000","last_peered":"2023-11-26T07:12:24.435858+0000","last_clean":"2023-11-26T07:12:24.435858+0000","last_became_active":"2023-11-26T07:12:23.119601+0000","last_became_peered":"2023-11-26T07:12:23.119601+0000","last_unstale":"2023-11-26T07:12:24.435858+0000","last_undegraded":"2023-11-26T07:12:24.435858+0000","last_fullsized":"2023-11-26T07:12:24.435858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:36.670009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105513+0000","last_change":"2023-11-26T07:12:22.119523+0000","last_active":"2023-11-26T07:12:24.105513+0000","last_peered":"2023-11-26T07:12:24.105513+0000","last_clean":"2023-11-26T07:12:24.105513+0000","last_became_active":"2023-11-26T07:12:22.119260+0000","last_became_peered":"2023-11-26T07:12:22.119260+0000","last_unstale":"2023-11-26T07:12:24.105513+0000","last_undegraded":"2023-11-26T07:12:24.105513+0000","last_fullsized":"2023-11-26T07:12:24.105513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:20:31.048080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107087+0000","last_change":"2023-11-26T07:12:23.120541+0000","last_active":"2023-11-26T07:12:24.107087+0000","last_peered":"2023-11-26T07:12:24.107087+0000","last_clean":"2023-11-26T07:12:24.107087+0000","last_became_active":"2023-11-26T07:12:23.120224+0000","last_became_peered":"2023-11-26T07:12:23.120224+0000","last_unstale":"2023-11-26T07:12:24.107087+0000","last_undegraded":"2023-11-26T07:12:24.107087+0000","last_fullsized":"2023-11-26T07:12:24.107087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:50.833212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783438+0000","last_change":"2023-11-26T07:12:22.124427+0000","last_active":"2023-11-26T07:12:24.783438+0000","last_peered":"2023-11-26T07:12:24.783438+0000","last_clean":"2023-11-26T07:12:24.783438+0000","last_became_active":"2023-11-26T07:12:22.124140+0000","last_became_peered":"2023-11-26T07:12:22.124140+0000","last_unstale":"2023-11-26T07:12:24.783438+0000","last_undegraded":"2023-11-26T07:12:24.783438+0000","last_fullsized":"2023-11-26T07:12:24.783438+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:04:18.883671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:12:23.110367+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:12:23.110367+0000","last_peered":"2023-11-26T07:12:23.110367+0000","last_clean":"2023-11-26T07:12:23.110367+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:12:23.110367+0000","last_undegraded":"2023-11-26T07:12:23.110367+0000","last_fullsized":"2023-11-26T07:12:23.110367+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":741721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.47199999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68300000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.629},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.40999999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.442},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.58699999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.72499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64800000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60099999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.499}]},{"osd":4,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.67000000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":146597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.36699999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.39800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.42099999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59499999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68600000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.78400000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":735331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.26200000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.32200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.30499999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.47699999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.441}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59299999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.39600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":151333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.376}]},{"osd":1,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.30499999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39000000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72099999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.68700000000000006}]},{"osd":6,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.75900000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.70099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":152897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.442}]},{"osd":1,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.9109999999999996,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.38400000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.71899999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8129999999999997,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.83199999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.873},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.883,"5min":4.883,"15min":4.883},"last":0.79600000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.8499999999999996,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.85099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.48199999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.434},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.40999999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.53000000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55900000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.433},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.66700000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.623},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.59799999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.56100000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.68700000000000006}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.41299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48499999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:12:27.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:12:27.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.759+0000 7ff3b70fa700 1 Processor -- start 2023-11-26T07:12:27.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.759+0000 7ff3b70fa700 1 -- start start 2023-11-26T07:12:27.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.759+0000 7ff3b70fa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3b015bdd0 0x7ff3b01604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.759+0000 7ff3b70fa700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff3b00d1c70 0x7ff3b00d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.759+0000 7ff3b70fa700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b00b22d0 con 0x7ff3b00b3370 2023-11-26T07:12:27.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.759+0000 7ff3b70fa700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b00b2f50 con 0x7ff3b00d1c70 2023-11-26T07:12:27.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.759+0000 7ff3b70fa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00d2640 con 0x7ff3b015bdd0 2023-11-26T07:12:27.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3affff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff3b00b3370 0x7ff3b00b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56166/0 (socket says 172.21.15.83:56166) 2023-11-26T07:12:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3affff700 1 -- 172.21.15.83:0/2692064768 learned_addr learned my addr 172.21.15.83:0/2692064768 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3b4e96700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3b015bdd0 0x7ff3b01604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3b5697700 1 --2- 172.21.15.83:0/2692064768 >> v2:172.21.15.203:3300/0 conn(0x7ff3b00d1c70 0x7ff3b00d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976254910 0 0) 0x7ff3b00b22d0 con 0x7ff3b00b3370 2023-11-26T07:12:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a00035b0 con 0x7ff3b00b3370 2023-11-26T07:12:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 68489888 0 0) 0x7ff3a00035b0 con 0x7ff3b00b3370 2023-11-26T07:12:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3b015bdd0 msgr2=0x7ff3b01604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3b015bdd0 0x7ff3b01604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 >> v2:172.21.15.203:3300/0 conn(0x7ff3b00d1c70 msgr2=0x7ff3b00d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 --2- 172.21.15.83:0/2692064768 >> v2:172.21.15.203:3300/0 conn(0x7ff3b00d1c70 0x7ff3b00d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b00b25b0 con 0x7ff3b00b3370 2023-11-26T07:12:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7ff39c001670 con 0x7ff3b00b3370 2023-11-26T07:12:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 >> v1:172.21.15.83:6789/0 conn(0x7ff3b00b3370 legacy=0x7ff3b00b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a0003a00 0x7ff3a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3b015bdd0 0x7ff3a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a0004d90 0x7ff3a0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00b2f50 con 0x7ff3a0003a00 2023-11-26T07:12:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00d2640 con 0x7ff3a0004d90 2023-11-26T07:12:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a000ba10 con 0x7ff3b015bdd0 2023-11-26T07:12:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3b4e96700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a0003a00 0x7ff3a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3b5697700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a0004d90 0x7ff3a0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.760+0000 7ff3affff700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3b015bdd0 0x7ff3a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b4e96700 1 -- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3b015bdd0 msgr2=0x7ff3a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b4e96700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3b015bdd0 0x7ff3a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b4e96700 1 -- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a0004d90 msgr2=0x7ff3a0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b4e96700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a0004d90 0x7ff3a0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b4e96700 1 -- 172.21.15.83:0/2692064768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00035b0 con 0x7ff3a0003a00 2023-11-26T07:12:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b5697700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a0004d90 0x7ff3a0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b4e96700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a0003a00 0x7ff3a0003e20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009b60 tx=0x7ff3a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997e0aef5e0a20b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400e6c0 con 0x7ff3a0003a00 2023-11-26T07:12:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a400e820 con 0x7ff3a0003a00 2023-11-26T07:12:27.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3af7fe700 1 -- 172.21.15.83:0/2692064768 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400eb90 con 0x7ff3a0003a00 2023-11-26T07:12:27.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2692064768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a0003a00 msgr2=0x7ff3a0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b70fa700 1 --2- 172.21.15.83:0/2692064768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a0003a00 0x7ff3a0003e20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009b60 tx=0x7ff3a0005880 comp rx=0 tx=0).stop 2023-11-26T07:12:27.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b5697700 1 -- 172.21.15.83:0/2692064768 reap_dead start 2023-11-26T07:12:27.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2692064768 shutdown_connections 2023-11-26T07:12:27.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.761+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2692064768 >> 172.21.15.83:0/2692064768 conn(0x7ff3b00731a0 msgr2=0x7ff3b009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:27.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.762+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2692064768 shutdown_connections 2023-11-26T07:12:27.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.762+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2692064768 wait complete. 2023-11-26T07:12:27.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.763+0000 7ff3b70fa700 1 Processor -- start 2023-11-26T07:12:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.763+0000 7ff3b70fa700 1 -- start start 2023-11-26T07:12:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b70fa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3a0003a00 0x7ff3b00c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b70fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3b014c2b0 0x7ff3b00c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b70fa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b00c9730 0x7ff3b00c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b70fa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b01e5760 con 0x7ff3b00c9730 2023-11-26T07:12:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b70fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b01e62e0 con 0x7ff3b014c2b0 2023-11-26T07:12:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b70fa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b01e59b0 con 0x7ff3a0003a00 2023-11-26T07:12:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b5697700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b00c9730 0x7ff3b00c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b5697700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b00c9730 0x7ff3b00c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36138/0 (socket says 172.21.15.83:36138) 2023-11-26T07:12:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b5697700 1 -- 172.21.15.83:0/2373613936 learned_addr learned my addr 172.21.15.83:0/2373613936 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:27.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b4e96700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3a0003a00 0x7ff3b00c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3affff700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3b014c2b0 0x7ff3b00c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b5697700 1 -- 172.21.15.83:0/2373613936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3a0003a00 msgr2=0x7ff3b00c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b5697700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3a0003a00 0x7ff3b00c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b5697700 1 -- 172.21.15.83:0/2373613936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3b014c2b0 msgr2=0x7ff3b00c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b5697700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3b014c2b0 0x7ff3b00c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b5697700 1 -- 172.21.15.83:0/2373613936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b01e5c90 con 0x7ff3b00c9730 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b4e96700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3a0003a00 0x7ff3b00c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b5697700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b00c9730 0x7ff3b00c9bb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff3b006ec10 tx=0x7ff3a80104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3ad7fa700 1 -- 172.21.15.83:0/2373613936 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800e050 con 0x7ff3b00c9730 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3ad7fa700 1 -- 172.21.15.83:0/2373613936 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a800f040 con 0x7ff3b00c9730 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2373613936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b01e5f70 con 0x7ff3b00c9730 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3ad7fa700 1 -- 172.21.15.83:0/2373613936 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8010d50 con 0x7ff3b00c9730 2023-11-26T07:12:27.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.764+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2373613936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b00c6fa0 con 0x7ff3b00c9730 2023-11-26T07:12:27.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.765+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2373613936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b01e6100 con 0x7ff3b00c9730 2023-11-26T07:12:27.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.766+0000 7ff3ad7fa700 1 -- 172.21.15.83:0/2373613936 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff3a8015440 con 0x7ff3b00c9730 2023-11-26T07:12:27.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.766+0000 7ff3ad7fa700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff39406aed0 0x7ff39406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:27.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.766+0000 7ff3ad7fa700 1 -- 172.21.15.83:0/2373613936 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3a8089a20 con 0x7ff3b00c9730 2023-11-26T07:12:27.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.766+0000 7ff3b4e96700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff39406aed0 0x7ff39406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:27.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.767+0000 7ff3b4e96700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff39406aed0 0x7ff39406d390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40090b0 tx=0x7ff3a4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:27.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.771+0000 7ff3ad7fa700 1 -- 172.21.15.83:0/2373613936 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff3a8054b60 con 0x7ff3b00c9730 2023-11-26T07:12:27.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.957+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2373613936 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3b00d2170 con 0x7ff39406aed0 2023-11-26T07:12:27.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.965+0000 7ff3ad7fa700 1 -- 172.21.15.83:0/2373613936 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289112 (secure 0 0 0) 0x7ff3b00d2170 con 0x7ff39406aed0 2023-11-26T07:12:27.971 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:27.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2373613936 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff39406aed0 msgr2=0x7ff39406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff39406aed0 0x7ff39406d390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40090b0 tx=0x7ff3a4008030 comp rx=0 tx=0).stop 2023-11-26T07:12:27.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2373613936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b00c9730 msgr2=0x7ff3b00c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:27.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b00c9730 0x7ff3b00c9bb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff3b006ec10 tx=0x7ff3a80104a0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2373613936 shutdown_connections 2023-11-26T07:12:27.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3b00c9730 0x7ff3b00c9bb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff39406aed0 0x7ff39406d390 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3a0003a00 0x7ff3b00c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 --2- 172.21.15.83:0/2373613936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3b014c2b0 0x7ff3b00c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:27.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2373613936 >> 172.21.15.83:0/2373613936 conn(0x7ff3b00731a0 msgr2=0x7ff3b015a200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:27.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2373613936 shutdown_connections 2023-11-26T07:12:27.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:27.968+0000 7ff3b70fa700 1 -- 172.21.15.83:0/2373613936 wait complete. 2023-11-26T07:12:27.974 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:12:27.983 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-26T07:12:26.319298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220380,"kb_used_data":4184,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754340,"statfs":{"total":773094113280,"available":772868444160,"internally_reserved":0,"allocated":4284416,"data_stored":2370460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220688"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783586+0000","last_change":"2023-11-26T07:12:23.132616+0000","last_active":"2023-11-26T07:12:24.783586+0000","last_peered":"2023-11-26T07:12:24.783586+0000","last_clean":"2023-11-26T07:12:24.783586+0000","last_became_active":"2023-11-26T07:12:23.132367+0000","last_became_peered":"2023-11-26T07:12:23.132367+0000","last_unstale":"2023-11-26T07:12:24.783586+0000","last_undegraded":"2023-11-26T07:12:24.783586+0000","last_fullsized":"2023-11-26T07:12:24.783586+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:34.579111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106038+0000","last_change":"2023-11-26T07:12:23.118515+0000","last_active":"2023-11-26T07:12:24.106038+0000","last_peered":"2023-11-26T07:12:24.106038+0000","last_clean":"2023-11-26T07:12:24.106038+0000","last_became_active":"2023-11-26T07:12:23.118256+0000","last_became_peered":"2023-11-26T07:12:23.118256+0000","last_unstale":"2023-11-26T07:12:24.106038+0000","last_undegraded":"2023-11-26T07:12:24.106038+0000","last_fullsized":"2023-11-26T07:12:24.106038+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:04:25.608116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107904+0000","last_change":"2023-11-26T07:12:23.123345+0000","last_active":"2023-11-26T07:12:24.107904+0000","last_peered":"2023-11-26T07:12:24.107904+0000","last_clean":"2023-11-26T07:12:24.107904+0000","last_became_active":"2023-11-26T07:12:23.122918+0000","last_became_peered":"2023-11-26T07:12:23.122918+0000","last_unstale":"2023-11-26T07:12:24.107904+0000","last_undegraded":"2023-11-26T07:12:24.107904+0000","last_fullsized":"2023-11-26T07:12:24.107904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:50.833212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783446+0000","last_change":"2023-11-26T07:12:23.121519+0000","last_active":"2023-11-26T07:12:24.783446+0000","last_peered":"2023-11-26T07:12:24.783446+0000","last_clean":"2023-11-26T07:12:24.783446+0000","last_became_active":"2023-11-26T07:12:23.121130+0000","last_became_peered":"2023-11-26T07:12:23.121130+0000","last_unstale":"2023-11-26T07:12:24.783446+0000","last_undegraded":"2023-11-26T07:12:24.783446+0000","last_fullsized":"2023-11-26T07:12:24.783446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:10:21.444998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783207+0000","last_change":"2023-11-26T07:12:23.119945+0000","last_active":"2023-11-26T07:12:24.783207+0000","last_peered":"2023-11-26T07:12:24.783207+0000","last_clean":"2023-11-26T07:12:24.783207+0000","last_became_active":"2023-11-26T07:12:23.119678+0000","last_became_peered":"2023-11-26T07:12:23.119678+0000","last_unstale":"2023-11-26T07:12:24.783207+0000","last_undegraded":"2023-11-26T07:12:24.783207+0000","last_fullsized":"2023-11-26T07:12:24.783207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:46:57.969869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105704+0000","last_change":"2023-11-26T07:12:23.122604+0000","last_active":"2023-11-26T07:12:24.105704+0000","last_peered":"2023-11-26T07:12:24.105704+0000","last_clean":"2023-11-26T07:12:24.105704+0000","last_became_active":"2023-11-26T07:12:23.122314+0000","last_became_peered":"2023-11-26T07:12:23.122314+0000","last_unstale":"2023-11-26T07:12:24.105704+0000","last_undegraded":"2023-11-26T07:12:24.105704+0000","last_fullsized":"2023-11-26T07:12:24.105704+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:32.714331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107189+0000","last_change":"2023-11-26T07:12:23.128631+0000","last_active":"2023-11-26T07:12:24.107189+0000","last_peered":"2023-11-26T07:12:24.107189+0000","last_clean":"2023-11-26T07:12:24.107189+0000","last_became_active":"2023-11-26T07:12:23.128031+0000","last_became_peered":"2023-11-26T07:12:23.128031+0000","last_unstale":"2023-11-26T07:12:24.107189+0000","last_undegraded":"2023-11-26T07:12:24.107189+0000","last_fullsized":"2023-11-26T07:12:24.107189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:27.812069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106880+0000","last_change":"2023-11-26T07:12:23.115564+0000","last_active":"2023-11-26T07:12:24.106880+0000","last_peered":"2023-11-26T07:12:24.106880+0000","last_clean":"2023-11-26T07:12:24.106880+0000","last_became_active":"2023-11-26T07:12:23.115246+0000","last_became_peered":"2023-11-26T07:12:23.115246+0000","last_unstale":"2023-11-26T07:12:24.106880+0000","last_undegraded":"2023-11-26T07:12:24.106880+0000","last_fullsized":"2023-11-26T07:12:24.106880+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:48:14.958935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436354+0000","last_change":"2023-11-26T07:12:23.117620+0000","last_active":"2023-11-26T07:12:24.436354+0000","last_peered":"2023-11-26T07:12:24.436354+0000","last_clean":"2023-11-26T07:12:24.436354+0000","last_became_active":"2023-11-26T07:12:23.117216+0000","last_became_peered":"2023-11-26T07:12:23.117216+0000","last_unstale":"2023-11-26T07:12:24.436354+0000","last_undegraded":"2023-11-26T07:12:24.436354+0000","last_fullsized":"2023-11-26T07:12:24.436354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:21.308653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783973+0000","last_change":"2023-11-26T07:12:23.119946+0000","last_active":"2023-11-26T07:12:24.783973+0000","last_peered":"2023-11-26T07:12:24.783973+0000","last_clean":"2023-11-26T07:12:24.783973+0000","last_became_active":"2023-11-26T07:12:23.119137+0000","last_became_peered":"2023-11-26T07:12:23.119137+0000","last_unstale":"2023-11-26T07:12:24.783973+0000","last_undegraded":"2023-11-26T07:12:24.783973+0000","last_fullsized":"2023-11-26T07:12:24.783973+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:10:14.957581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107682+0000","last_change":"2023-11-26T07:12:23.125267+0000","last_active":"2023-11-26T07:12:24.107682+0000","last_peered":"2023-11-26T07:12:24.107682+0000","last_clean":"2023-11-26T07:12:24.107682+0000","last_became_active":"2023-11-26T07:12:23.125006+0000","last_became_peered":"2023-11-26T07:12:23.125006+0000","last_unstale":"2023-11-26T07:12:24.107682+0000","last_undegraded":"2023-11-26T07:12:24.107682+0000","last_fullsized":"2023-11-26T07:12:24.107682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:36:26.407162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107892+0000","last_change":"2023-11-26T07:12:23.127770+0000","last_active":"2023-11-26T07:12:24.107892+0000","last_peered":"2023-11-26T07:12:24.107892+0000","last_clean":"2023-11-26T07:12:24.107892+0000","last_became_active":"2023-11-26T07:12:23.127454+0000","last_became_peered":"2023-11-26T07:12:23.127454+0000","last_unstale":"2023-11-26T07:12:24.107892+0000","last_undegraded":"2023-11-26T07:12:24.107892+0000","last_fullsized":"2023-11-26T07:12:24.107892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:32.714331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783492+0000","last_change":"2023-11-26T07:12:23.115485+0000","last_active":"2023-11-26T07:12:24.783492+0000","last_peered":"2023-11-26T07:12:24.783492+0000","last_clean":"2023-11-26T07:12:24.783492+0000","last_became_active":"2023-11-26T07:12:23.114828+0000","last_became_peered":"2023-11-26T07:12:23.114828+0000","last_unstale":"2023-11-26T07:12:24.783492+0000","last_undegraded":"2023-11-26T07:12:24.783492+0000","last_fullsized":"2023-11-26T07:12:24.783492+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:19:25.018398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105787+0000","last_change":"2023-11-26T07:12:23.116816+0000","last_active":"2023-11-26T07:12:24.105787+0000","last_peered":"2023-11-26T07:12:24.105787+0000","last_clean":"2023-11-26T07:12:24.105787+0000","last_became_active":"2023-11-26T07:12:23.116552+0000","last_became_peered":"2023-11-26T07:12:23.116552+0000","last_unstale":"2023-11-26T07:12:24.105787+0000","last_undegraded":"2023-11-26T07:12:24.105787+0000","last_fullsized":"2023-11-26T07:12:24.105787+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:27.812069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107519+0000","last_change":"2023-11-26T07:12:23.127578+0000","last_active":"2023-11-26T07:12:24.107519+0000","last_peered":"2023-11-26T07:12:24.107519+0000","last_clean":"2023-11-26T07:12:24.107519+0000","last_became_active":"2023-11-26T07:12:23.127267+0000","last_became_peered":"2023-11-26T07:12:23.127267+0000","last_unstale":"2023-11-26T07:12:24.107519+0000","last_undegraded":"2023-11-26T07:12:24.107519+0000","last_fullsized":"2023-11-26T07:12:24.107519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:48:14.958935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107065+0000","last_change":"2023-11-26T07:12:23.120766+0000","last_active":"2023-11-26T07:12:24.107065+0000","last_peered":"2023-11-26T07:12:24.107065+0000","last_clean":"2023-11-26T07:12:24.107065+0000","last_became_active":"2023-11-26T07:12:23.119779+0000","last_became_peered":"2023-11-26T07:12:23.119779+0000","last_unstale":"2023-11-26T07:12:24.107065+0000","last_undegraded":"2023-11-26T07:12:24.107065+0000","last_fullsized":"2023-11-26T07:12:24.107065+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:32.714331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107583+0000","last_change":"2023-11-26T07:12:23.126537+0000","last_active":"2023-11-26T07:12:24.107583+0000","last_peered":"2023-11-26T07:12:24.107583+0000","last_clean":"2023-11-26T07:12:24.107583+0000","last_became_active":"2023-11-26T07:12:23.126278+0000","last_became_peered":"2023-11-26T07:12:23.126278+0000","last_unstale":"2023-11-26T07:12:24.107583+0000","last_undegraded":"2023-11-26T07:12:24.107583+0000","last_fullsized":"2023-11-26T07:12:24.107583+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:56:23.400841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106158+0000","last_change":"2023-11-26T07:12:23.119297+0000","last_active":"2023-11-26T07:12:24.106158+0000","last_peered":"2023-11-26T07:12:24.106158+0000","last_clean":"2023-11-26T07:12:24.106158+0000","last_became_active":"2023-11-26T07:12:23.117145+0000","last_became_peered":"2023-11-26T07:12:23.117145+0000","last_unstale":"2023-11-26T07:12:24.106158+0000","last_undegraded":"2023-11-26T07:12:24.106158+0000","last_fullsized":"2023-11-26T07:12:24.106158+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:35:30.734856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105692+0000","last_change":"2023-11-26T07:12:23.120019+0000","last_active":"2023-11-26T07:12:24.105692+0000","last_peered":"2023-11-26T07:12:24.105692+0000","last_clean":"2023-11-26T07:12:24.105692+0000","last_became_active":"2023-11-26T07:12:23.119749+0000","last_became_peered":"2023-11-26T07:12:23.119749+0000","last_unstale":"2023-11-26T07:12:24.105692+0000","last_undegraded":"2023-11-26T07:12:24.105692+0000","last_fullsized":"2023-11-26T07:12:24.105692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:48:14.958935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783344+0000","last_change":"2023-11-26T07:12:23.127994+0000","last_active":"2023-11-26T07:12:24.783344+0000","last_peered":"2023-11-26T07:12:24.783344+0000","last_clean":"2023-11-26T07:12:24.783344+0000","last_became_active":"2023-11-26T07:12:23.127720+0000","last_became_peered":"2023-11-26T07:12:23.127720+0000","last_unstale":"2023-11-26T07:12:24.783344+0000","last_undegraded":"2023-11-26T07:12:24.783344+0000","last_fullsized":"2023-11-26T07:12:24.783344+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:56:03.522118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436107+0000","last_change":"2023-11-26T07:12:23.119253+0000","last_active":"2023-11-26T07:12:24.436107+0000","last_peered":"2023-11-26T07:12:24.436107+0000","last_clean":"2023-11-26T07:12:24.436107+0000","last_became_active":"2023-11-26T07:12:23.118964+0000","last_became_peered":"2023-11-26T07:12:23.118964+0000","last_unstale":"2023-11-26T07:12:24.436107+0000","last_undegraded":"2023-11-26T07:12:24.436107+0000","last_fullsized":"2023-11-26T07:12:24.436107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:00.647047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435924+0000","last_change":"2023-11-26T07:12:23.123099+0000","last_active":"2023-11-26T07:12:24.435924+0000","last_peered":"2023-11-26T07:12:24.435924+0000","last_clean":"2023-11-26T07:12:24.435924+0000","last_became_active":"2023-11-26T07:12:23.122828+0000","last_became_peered":"2023-11-26T07:12:23.122828+0000","last_unstale":"2023-11-26T07:12:24.435924+0000","last_undegraded":"2023-11-26T07:12:24.435924+0000","last_fullsized":"2023-11-26T07:12:24.435924+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:39.433005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.782947+0000","last_change":"2023-11-26T07:12:23.120827+0000","last_active":"2023-11-26T07:12:24.782947+0000","last_peered":"2023-11-26T07:12:24.782947+0000","last_clean":"2023-11-26T07:12:24.782947+0000","last_became_active":"2023-11-26T07:12:23.120527+0000","last_became_peered":"2023-11-26T07:12:23.120527+0000","last_unstale":"2023-11-26T07:12:24.782947+0000","last_undegraded":"2023-11-26T07:12:24.782947+0000","last_fullsized":"2023-11-26T07:12:24.782947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:34:09.475730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107221+0000","last_change":"2023-11-26T07:12:23.127310+0000","last_active":"2023-11-26T07:12:24.107221+0000","last_peered":"2023-11-26T07:12:24.107221+0000","last_clean":"2023-11-26T07:12:24.107221+0000","last_became_active":"2023-11-26T07:12:23.126937+0000","last_became_peered":"2023-11-26T07:12:23.126937+0000","last_unstale":"2023-11-26T07:12:24.107221+0000","last_undegraded":"2023-11-26T07:12:24.107221+0000","last_fullsized":"2023-11-26T07:12:24.107221+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:16.181329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783931+0000","last_change":"2023-11-26T07:12:23.130299+0000","last_active":"2023-11-26T07:12:24.783931+0000","last_peered":"2023-11-26T07:12:24.783931+0000","last_clean":"2023-11-26T07:12:24.783931+0000","last_became_active":"2023-11-26T07:12:23.130010+0000","last_became_peered":"2023-11-26T07:12:23.130010+0000","last_unstale":"2023-11-26T07:12:24.783931+0000","last_undegraded":"2023-11-26T07:12:24.783931+0000","last_fullsized":"2023-11-26T07:12:24.783931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:08:24.809552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106182+0000","last_change":"2023-11-26T07:12:23.125361+0000","last_active":"2023-11-26T07:12:24.106182+0000","last_peered":"2023-11-26T07:12:24.106182+0000","last_clean":"2023-11-26T07:12:24.106182+0000","last_became_active":"2023-11-26T07:12:23.125087+0000","last_became_peered":"2023-11-26T07:12:23.125087+0000","last_unstale":"2023-11-26T07:12:24.106182+0000","last_undegraded":"2023-11-26T07:12:24.106182+0000","last_fullsized":"2023-11-26T07:12:24.106182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:04:40.263224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783755+0000","last_change":"2023-11-26T07:12:23.118477+0000","last_active":"2023-11-26T07:12:24.783755+0000","last_peered":"2023-11-26T07:12:24.783755+0000","last_clean":"2023-11-26T07:12:24.783755+0000","last_became_active":"2023-11-26T07:12:23.118189+0000","last_became_peered":"2023-11-26T07:12:23.118189+0000","last_unstale":"2023-11-26T07:12:24.783755+0000","last_undegraded":"2023-11-26T07:12:24.783755+0000","last_fullsized":"2023-11-26T07:12:24.783755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:21.308653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107901+0000","last_change":"2023-11-26T07:12:23.124357+0000","last_active":"2023-11-26T07:12:24.107901+0000","last_peered":"2023-11-26T07:12:24.107901+0000","last_clean":"2023-11-26T07:12:24.107901+0000","last_became_active":"2023-11-26T07:12:23.123664+0000","last_became_peered":"2023-11-26T07:12:23.123664+0000","last_unstale":"2023-11-26T07:12:24.107901+0000","last_undegraded":"2023-11-26T07:12:24.107901+0000","last_fullsized":"2023-11-26T07:12:24.107901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:27.812069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435935+0000","last_change":"2023-11-26T07:12:23.123725+0000","last_active":"2023-11-26T07:12:24.435935+0000","last_peered":"2023-11-26T07:12:24.435935+0000","last_clean":"2023-11-26T07:12:24.435935+0000","last_became_active":"2023-11-26T07:12:23.123368+0000","last_became_peered":"2023-11-26T07:12:23.123368+0000","last_unstale":"2023-11-26T07:12:24.435935+0000","last_undegraded":"2023-11-26T07:12:24.435935+0000","last_fullsized":"2023-11-26T07:12:24.435935+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:10:21.444998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107036+0000","last_change":"2023-11-26T07:12:23.128368+0000","last_active":"2023-11-26T07:12:24.107036+0000","last_peered":"2023-11-26T07:12:24.107036+0000","last_clean":"2023-11-26T07:12:24.107036+0000","last_became_active":"2023-11-26T07:12:23.128059+0000","last_became_peered":"2023-11-26T07:12:23.128059+0000","last_unstale":"2023-11-26T07:12:24.107036+0000","last_undegraded":"2023-11-26T07:12:24.107036+0000","last_fullsized":"2023-11-26T07:12:24.107036+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:44:46.385542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783560+0000","last_change":"2023-11-26T07:12:23.124902+0000","last_active":"2023-11-26T07:12:24.783560+0000","last_peered":"2023-11-26T07:12:24.783560+0000","last_clean":"2023-11-26T07:12:24.783560+0000","last_became_active":"2023-11-26T07:12:23.124619+0000","last_became_peered":"2023-11-26T07:12:23.124619+0000","last_unstale":"2023-11-26T07:12:24.783560+0000","last_undegraded":"2023-11-26T07:12:24.783560+0000","last_fullsized":"2023-11-26T07:12:24.783560+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:59:53.542038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107219+0000","last_change":"2023-11-26T07:12:23.121278+0000","last_active":"2023-11-26T07:12:24.107219+0000","last_peered":"2023-11-26T07:12:24.107219+0000","last_clean":"2023-11-26T07:12:24.107219+0000","last_became_active":"2023-11-26T07:12:23.120953+0000","last_became_peered":"2023-11-26T07:12:23.120953+0000","last_unstale":"2023-11-26T07:12:24.107219+0000","last_undegraded":"2023-11-26T07:12:24.107219+0000","last_fullsized":"2023-11-26T07:12:24.107219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:31:12.988370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783408+0000","last_change":"2023-11-26T07:12:23.117615+0000","last_active":"2023-11-26T07:12:24.783408+0000","last_peered":"2023-11-26T07:12:24.783408+0000","last_clean":"2023-11-26T07:12:24.783408+0000","last_became_active":"2023-11-26T07:12:23.117272+0000","last_became_peered":"2023-11-26T07:12:23.117272+0000","last_unstale":"2023-11-26T07:12:24.783408+0000","last_undegraded":"2023-11-26T07:12:24.783408+0000","last_fullsized":"2023-11-26T07:12:24.783408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:34.579111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783648+0000","last_change":"2023-11-26T07:12:22.120139+0000","last_active":"2023-11-26T07:12:24.783648+0000","last_peered":"2023-11-26T07:12:24.783648+0000","last_clean":"2023-11-26T07:12:24.783648+0000","last_became_active":"2023-11-26T07:12:22.119877+0000","last_became_peered":"2023-11-26T07:12:22.119877+0000","last_unstale":"2023-11-26T07:12:24.783648+0000","last_undegraded":"2023-11-26T07:12:24.783648+0000","last_fullsized":"2023-11-26T07:12:24.783648+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:26:37.066177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783604+0000","last_change":"2023-11-26T07:12:23.128289+0000","last_active":"2023-11-26T07:12:24.783604+0000","last_peered":"2023-11-26T07:12:24.783604+0000","last_clean":"2023-11-26T07:12:24.783604+0000","last_became_active":"2023-11-26T07:12:23.127752+0000","last_became_peered":"2023-11-26T07:12:23.127752+0000","last_unstale":"2023-11-26T07:12:24.783604+0000","last_undegraded":"2023-11-26T07:12:24.783604+0000","last_fullsized":"2023-11-26T07:12:24.783604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:36.670009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783480+0000","last_change":"2023-11-26T07:12:22.116096+0000","last_active":"2023-11-26T07:12:24.783480+0000","last_peered":"2023-11-26T07:12:24.783480+0000","last_clean":"2023-11-26T07:12:24.783480+0000","last_became_active":"2023-11-26T07:12:22.115803+0000","last_became_peered":"2023-11-26T07:12:22.115803+0000","last_unstale":"2023-11-26T07:12:24.783480+0000","last_undegraded":"2023-11-26T07:12:24.783480+0000","last_fullsized":"2023-11-26T07:12:24.783480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:19:24.013733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436074+0000","last_change":"2023-11-26T07:12:23.123058+0000","last_active":"2023-11-26T07:12:24.436074+0000","last_peered":"2023-11-26T07:12:24.436074+0000","last_clean":"2023-11-26T07:12:24.436074+0000","last_became_active":"2023-11-26T07:12:23.122789+0000","last_became_peered":"2023-11-26T07:12:23.122789+0000","last_unstale":"2023-11-26T07:12:24.436074+0000","last_undegraded":"2023-11-26T07:12:24.436074+0000","last_fullsized":"2023-11-26T07:12:24.436074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:08.151898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106204+0000","last_change":"2023-11-26T07:12:22.119031+0000","last_active":"2023-11-26T07:12:24.106204+0000","last_peered":"2023-11-26T07:12:24.106204+0000","last_clean":"2023-11-26T07:12:24.106204+0000","last_became_active":"2023-11-26T07:12:22.118581+0000","last_became_peered":"2023-11-26T07:12:22.118581+0000","last_unstale":"2023-11-26T07:12:24.106204+0000","last_undegraded":"2023-11-26T07:12:24.106204+0000","last_fullsized":"2023-11-26T07:12:24.106204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:42:59.642382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783744+0000","last_change":"2023-11-26T07:12:23.118863+0000","last_active":"2023-11-26T07:12:24.783744+0000","last_peered":"2023-11-26T07:12:24.783744+0000","last_clean":"2023-11-26T07:12:24.783744+0000","last_became_active":"2023-11-26T07:12:23.118567+0000","last_became_peered":"2023-11-26T07:12:23.118567+0000","last_unstale":"2023-11-26T07:12:24.783744+0000","last_undegraded":"2023-11-26T07:12:24.783744+0000","last_fullsized":"2023-11-26T07:12:24.783744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:20:06.180831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783439+0000","last_change":"2023-11-26T07:12:22.122912+0000","last_active":"2023-11-26T07:12:24.783439+0000","last_peered":"2023-11-26T07:12:24.783439+0000","last_clean":"2023-11-26T07:12:24.783439+0000","last_became_active":"2023-11-26T07:12:22.122591+0000","last_became_peered":"2023-11-26T07:12:22.122591+0000","last_unstale":"2023-11-26T07:12:24.783439+0000","last_undegraded":"2023-11-26T07:12:24.783439+0000","last_fullsized":"2023-11-26T07:12:24.783439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:18:23.467389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107234+0000","last_change":"2023-11-26T07:12:23.120487+0000","last_active":"2023-11-26T07:12:24.107234+0000","last_peered":"2023-11-26T07:12:24.107234+0000","last_clean":"2023-11-26T07:12:24.107234+0000","last_became_active":"2023-11-26T07:12:23.120182+0000","last_became_peered":"2023-11-26T07:12:23.120182+0000","last_unstale":"2023-11-26T07:12:24.107234+0000","last_undegraded":"2023-11-26T07:12:24.107234+0000","last_fullsized":"2023-11-26T07:12:24.107234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:04:25.608116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436058+0000","last_change":"2023-11-26T07:12:22.121764+0000","last_active":"2023-11-26T07:12:24.436058+0000","last_peered":"2023-11-26T07:12:24.436058+0000","last_clean":"2023-11-26T07:12:24.436058+0000","last_became_active":"2023-11-26T07:12:22.121534+0000","last_became_peered":"2023-11-26T07:12:22.121534+0000","last_unstale":"2023-11-26T07:12:24.436058+0000","last_undegraded":"2023-11-26T07:12:24.436058+0000","last_fullsized":"2023-11-26T07:12:24.436058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:13.437694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107674+0000","last_change":"2023-11-26T07:12:23.123986+0000","last_active":"2023-11-26T07:12:24.107674+0000","last_peered":"2023-11-26T07:12:24.107674+0000","last_clean":"2023-11-26T07:12:24.107674+0000","last_became_active":"2023-11-26T07:12:23.123625+0000","last_became_peered":"2023-11-26T07:12:23.123625+0000","last_unstale":"2023-11-26T07:12:24.107674+0000","last_undegraded":"2023-11-26T07:12:24.107674+0000","last_fullsized":"2023-11-26T07:12:24.107674+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:44:46.385542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435982+0000","last_change":"2023-11-26T07:12:22.116348+0000","last_active":"2023-11-26T07:12:24.435982+0000","last_peered":"2023-11-26T07:12:24.435982+0000","last_clean":"2023-11-26T07:12:24.435982+0000","last_became_active":"2023-11-26T07:12:22.116075+0000","last_became_peered":"2023-11-26T07:12:22.116075+0000","last_unstale":"2023-11-26T07:12:24.435982+0000","last_undegraded":"2023-11-26T07:12:24.435982+0000","last_fullsized":"2023-11-26T07:12:24.435982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:26:37.066177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435821+0000","last_change":"2023-11-26T07:12:23.126362+0000","last_active":"2023-11-26T07:12:24.435821+0000","last_peered":"2023-11-26T07:12:24.435821+0000","last_clean":"2023-11-26T07:12:24.435821+0000","last_became_active":"2023-11-26T07:12:23.125921+0000","last_became_peered":"2023-11-26T07:12:23.125921+0000","last_unstale":"2023-11-26T07:12:24.435821+0000","last_undegraded":"2023-11-26T07:12:24.435821+0000","last_fullsized":"2023-11-26T07:12:24.435821+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:34.579111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105381+0000","last_change":"2023-11-26T07:12:22.117803+0000","last_active":"2023-11-26T07:12:24.105381+0000","last_peered":"2023-11-26T07:12:24.105381+0000","last_clean":"2023-11-26T07:12:24.105381+0000","last_became_active":"2023-11-26T07:12:22.117526+0000","last_became_peered":"2023-11-26T07:12:22.117526+0000","last_unstale":"2023-11-26T07:12:24.105381+0000","last_undegraded":"2023-11-26T07:12:24.105381+0000","last_fullsized":"2023-11-26T07:12:24.105381+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:33.574446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783248+0000","last_change":"2023-11-26T07:12:23.127956+0000","last_active":"2023-11-26T07:12:24.783248+0000","last_peered":"2023-11-26T07:12:24.783248+0000","last_clean":"2023-11-26T07:12:24.783248+0000","last_became_active":"2023-11-26T07:12:23.127669+0000","last_became_peered":"2023-11-26T07:12:23.127669+0000","last_unstale":"2023-11-26T07:12:24.783248+0000","last_undegraded":"2023-11-26T07:12:24.783248+0000","last_fullsized":"2023-11-26T07:12:24.783248+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:39.433005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435728+0000","last_change":"2023-11-26T07:12:22.118871+0000","last_active":"2023-11-26T07:12:24.435728+0000","last_peered":"2023-11-26T07:12:24.435728+0000","last_clean":"2023-11-26T07:12:24.435728+0000","last_became_active":"2023-11-26T07:12:22.118542+0000","last_became_peered":"2023-11-26T07:12:22.118542+0000","last_unstale":"2023-11-26T07:12:24.435728+0000","last_undegraded":"2023-11-26T07:12:24.435728+0000","last_fullsized":"2023-11-26T07:12:24.435728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:26:47.565317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.108128+0000","last_change":"2023-11-26T07:12:23.128059+0000","last_active":"2023-11-26T07:12:24.108128+0000","last_peered":"2023-11-26T07:12:24.108128+0000","last_clean":"2023-11-26T07:12:24.108128+0000","last_became_active":"2023-11-26T07:12:23.126648+0000","last_became_peered":"2023-11-26T07:12:23.126648+0000","last_unstale":"2023-11-26T07:12:24.108128+0000","last_undegraded":"2023-11-26T07:12:24.108128+0000","last_fullsized":"2023-11-26T07:12:24.108128+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:04:25.608116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105840+0000","last_change":"2023-11-26T07:12:22.113988+0000","last_active":"2023-11-26T07:12:24.105840+0000","last_peered":"2023-11-26T07:12:24.105840+0000","last_clean":"2023-11-26T07:12:24.105840+0000","last_became_active":"2023-11-26T07:12:22.113682+0000","last_became_peered":"2023-11-26T07:12:22.113682+0000","last_unstale":"2023-11-26T07:12:24.105840+0000","last_undegraded":"2023-11-26T07:12:24.105840+0000","last_fullsized":"2023-11-26T07:12:24.105840+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:38.428339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.106012+0000","last_change":"2023-11-26T07:12:23.119095+0000","last_active":"2023-11-26T07:12:24.106012+0000","last_peered":"2023-11-26T07:12:24.106012+0000","last_clean":"2023-11-26T07:12:24.106012+0000","last_became_active":"2023-11-26T07:12:23.118820+0000","last_became_peered":"2023-11-26T07:12:23.118820+0000","last_unstale":"2023-11-26T07:12:24.106012+0000","last_undegraded":"2023-11-26T07:12:24.106012+0000","last_fullsized":"2023-11-26T07:12:24.106012+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:31:12.988370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783558+0000","last_change":"2023-11-26T07:12:22.117436+0000","last_active":"2023-11-26T07:12:24.783558+0000","last_peered":"2023-11-26T07:12:24.783558+0000","last_clean":"2023-11-26T07:12:24.783558+0000","last_became_active":"2023-11-26T07:12:22.117146+0000","last_became_peered":"2023-11-26T07:12:22.117146+0000","last_unstale":"2023-11-26T07:12:24.783558+0000","last_undegraded":"2023-11-26T07:12:24.783558+0000","last_fullsized":"2023-11-26T07:12:24.783558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:16.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105704+0000","last_change":"2023-11-26T07:12:23.123102+0000","last_active":"2023-11-26T07:12:24.105704+0000","last_peered":"2023-11-26T07:12:24.105704+0000","last_clean":"2023-11-26T07:12:24.105704+0000","last_became_active":"2023-11-26T07:12:23.122882+0000","last_became_peered":"2023-11-26T07:12:23.122882+0000","last_unstale":"2023-11-26T07:12:24.105704+0000","last_undegraded":"2023-11-26T07:12:24.105704+0000","last_fullsized":"2023-11-26T07:12:24.105704+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:50.833212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.108005+0000","last_change":"2023-11-26T07:12:22.117919+0000","last_active":"2023-11-26T07:12:24.108005+0000","last_peered":"2023-11-26T07:12:24.108005+0000","last_clean":"2023-11-26T07:12:24.108005+0000","last_became_active":"2023-11-26T07:12:22.117629+0000","last_became_peered":"2023-11-26T07:12:22.117629+0000","last_unstale":"2023-11-26T07:12:24.108005+0000","last_undegraded":"2023-11-26T07:12:24.108005+0000","last_fullsized":"2023-11-26T07:12:24.108005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:20:31.048080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107951+0000","last_change":"2023-11-26T07:12:23.127121+0000","last_active":"2023-11-26T07:12:24.107951+0000","last_peered":"2023-11-26T07:12:24.107951+0000","last_clean":"2023-11-26T07:12:24.107951+0000","last_became_active":"2023-11-26T07:12:23.126833+0000","last_became_peered":"2023-11-26T07:12:23.126833+0000","last_unstale":"2023-11-26T07:12:24.107951+0000","last_undegraded":"2023-11-26T07:12:24.107951+0000","last_fullsized":"2023-11-26T07:12:24.107951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:31:12.988370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105854+0000","last_change":"2023-11-26T07:12:22.120536+0000","last_active":"2023-11-26T07:12:24.105854+0000","last_peered":"2023-11-26T07:12:24.105854+0000","last_clean":"2023-11-26T07:12:24.105854+0000","last_became_active":"2023-11-26T07:12:22.120245+0000","last_became_peered":"2023-11-26T07:12:22.120245+0000","last_unstale":"2023-11-26T07:12:24.105854+0000","last_undegraded":"2023-11-26T07:12:24.105854+0000","last_fullsized":"2023-11-26T07:12:24.105854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:20:58.962312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783630+0000","last_change":"2023-11-26T07:12:23.128085+0000","last_active":"2023-11-26T07:12:24.783630+0000","last_peered":"2023-11-26T07:12:24.783630+0000","last_clean":"2023-11-26T07:12:24.783630+0000","last_became_active":"2023-11-26T07:12:23.127767+0000","last_became_peered":"2023-11-26T07:12:23.127767+0000","last_unstale":"2023-11-26T07:12:24.783630+0000","last_undegraded":"2023-11-26T07:12:24.783630+0000","last_fullsized":"2023-11-26T07:12:24.783630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:00.647047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436178+0000","last_change":"2023-11-26T07:12:22.119236+0000","last_active":"2023-11-26T07:12:24.436178+0000","last_peered":"2023-11-26T07:12:24.436178+0000","last_clean":"2023-11-26T07:12:24.436178+0000","last_became_active":"2023-11-26T07:12:22.118790+0000","last_became_peered":"2023-11-26T07:12:22.118790+0000","last_unstale":"2023-11-26T07:12:24.436178+0000","last_undegraded":"2023-11-26T07:12:24.436178+0000","last_fullsized":"2023-11-26T07:12:24.436178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:34:08.471064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105685+0000","last_change":"2023-11-26T07:12:23.116580+0000","last_active":"2023-11-26T07:12:24.105685+0000","last_peered":"2023-11-26T07:12:24.105685+0000","last_clean":"2023-11-26T07:12:24.105685+0000","last_became_active":"2023-11-26T07:12:23.116112+0000","last_became_peered":"2023-11-26T07:12:23.116112+0000","last_unstale":"2023-11-26T07:12:24.105685+0000","last_undegraded":"2023-11-26T07:12:24.105685+0000","last_fullsized":"2023-11-26T07:12:24.105685+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:44:46.385542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783730+0000","last_change":"2023-11-26T07:12:22.122113+0000","last_active":"2023-11-26T07:12:24.783730+0000","last_peered":"2023-11-26T07:12:24.783730+0000","last_clean":"2023-11-26T07:12:24.783730+0000","last_became_active":"2023-11-26T07:12:22.121809+0000","last_became_peered":"2023-11-26T07:12:22.121809+0000","last_unstale":"2023-11-26T07:12:24.783730+0000","last_undegraded":"2023-11-26T07:12:24.783730+0000","last_fullsized":"2023-11-26T07:12:24.783730+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:13.437694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107371+0000","last_change":"2023-11-26T07:12:23.117028+0000","last_active":"2023-11-26T07:12:24.107371+0000","last_peered":"2023-11-26T07:12:24.107371+0000","last_clean":"2023-11-26T07:12:24.107371+0000","last_became_active":"2023-11-26T07:12:23.116733+0000","last_became_peered":"2023-11-26T07:12:23.116733+0000","last_unstale":"2023-11-26T07:12:24.107371+0000","last_undegraded":"2023-11-26T07:12:24.107371+0000","last_fullsized":"2023-11-26T07:12:24.107371+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:04:40.263224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105463+0000","last_change":"2023-11-26T07:12:22.114756+0000","last_active":"2023-11-26T07:12:24.105463+0000","last_peered":"2023-11-26T07:12:24.105463+0000","last_clean":"2023-11-26T07:12:24.105463+0000","last_became_active":"2023-11-26T07:12:22.114070+0000","last_became_peered":"2023-11-26T07:12:22.114070+0000","last_unstale":"2023-11-26T07:12:24.105463+0000","last_undegraded":"2023-11-26T07:12:24.105463+0000","last_fullsized":"2023-11-26T07:12:24.105463+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:35.665344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783391+0000","last_change":"2023-11-26T07:12:23.129786+0000","last_active":"2023-11-26T07:12:24.783391+0000","last_peered":"2023-11-26T07:12:24.783391+0000","last_clean":"2023-11-26T07:12:24.783391+0000","last_became_active":"2023-11-26T07:12:23.129012+0000","last_became_peered":"2023-11-26T07:12:23.129012+0000","last_unstale":"2023-11-26T07:12:24.783391+0000","last_undegraded":"2023-11-26T07:12:24.783391+0000","last_fullsized":"2023-11-26T07:12:24.783391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:14.442359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105342+0000","last_change":"2023-11-26T07:12:22.118424+0000","last_active":"2023-11-26T07:12:24.105342+0000","last_peered":"2023-11-26T07:12:24.105342+0000","last_clean":"2023-11-26T07:12:24.105342+0000","last_became_active":"2023-11-26T07:12:22.118156+0000","last_became_peered":"2023-11-26T07:12:22.118156+0000","last_unstale":"2023-11-26T07:12:24.105342+0000","last_undegraded":"2023-11-26T07:12:24.105342+0000","last_fullsized":"2023-11-26T07:12:24.105342+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:29:52.714251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436283+0000","last_change":"2023-11-26T07:12:23.129656+0000","last_active":"2023-11-26T07:12:24.436283+0000","last_peered":"2023-11-26T07:12:24.436283+0000","last_clean":"2023-11-26T07:12:24.436283+0000","last_became_active":"2023-11-26T07:12:23.129396+0000","last_became_peered":"2023-11-26T07:12:23.129396+0000","last_unstale":"2023-11-26T07:12:24.436283+0000","last_undegraded":"2023-11-26T07:12:24.436283+0000","last_fullsized":"2023-11-26T07:12:24.436283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:20:06.180831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783905+0000","last_change":"2023-11-26T07:12:22.123326+0000","last_active":"2023-11-26T07:12:24.783905+0000","last_peered":"2023-11-26T07:12:24.783905+0000","last_clean":"2023-11-26T07:12:24.783905+0000","last_became_active":"2023-11-26T07:12:22.123112+0000","last_became_peered":"2023-11-26T07:12:22.123112+0000","last_unstale":"2023-11-26T07:12:24.783905+0000","last_undegraded":"2023-11-26T07:12:24.783905+0000","last_fullsized":"2023-11-26T07:12:24.783905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:04:18.883671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.108193+0000","last_change":"2023-11-26T07:12:23.127302+0000","last_active":"2023-11-26T07:12:24.108193+0000","last_peered":"2023-11-26T07:12:24.108193+0000","last_clean":"2023-11-26T07:12:24.108193+0000","last_became_active":"2023-11-26T07:12:23.126917+0000","last_became_peered":"2023-11-26T07:12:23.126917+0000","last_unstale":"2023-11-26T07:12:24.108193+0000","last_undegraded":"2023-11-26T07:12:24.108193+0000","last_fullsized":"2023-11-26T07:12:24.108193+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:04:40.263224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436273+0000","last_change":"2023-11-26T07:12:22.119703+0000","last_active":"2023-11-26T07:12:24.436273+0000","last_peered":"2023-11-26T07:12:24.436273+0000","last_clean":"2023-11-26T07:12:24.436273+0000","last_became_active":"2023-11-26T07:12:22.118932+0000","last_became_peered":"2023-11-26T07:12:22.118932+0000","last_unstale":"2023-11-26T07:12:24.436273+0000","last_undegraded":"2023-11-26T07:12:24.436273+0000","last_fullsized":"2023-11-26T07:12:24.436273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:43:37.623929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783582+0000","last_change":"2023-11-26T07:12:23.121769+0000","last_active":"2023-11-26T07:12:24.783582+0000","last_peered":"2023-11-26T07:12:24.783582+0000","last_clean":"2023-11-26T07:12:24.783582+0000","last_became_active":"2023-11-26T07:12:23.121486+0000","last_became_peered":"2023-11-26T07:12:23.121486+0000","last_unstale":"2023-11-26T07:12:24.783582+0000","last_undegraded":"2023-11-26T07:12:24.783582+0000","last_fullsized":"2023-11-26T07:12:24.783582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:08.151898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436168+0000","last_change":"2023-11-26T07:12:22.119541+0000","last_active":"2023-11-26T07:12:24.436168+0000","last_peered":"2023-11-26T07:12:24.436168+0000","last_clean":"2023-11-26T07:12:24.436168+0000","last_became_active":"2023-11-26T07:12:22.119135+0000","last_became_peered":"2023-11-26T07:12:22.119135+0000","last_unstale":"2023-11-26T07:12:24.436168+0000","last_undegraded":"2023-11-26T07:12:24.436168+0000","last_fullsized":"2023-11-26T07:12:24.436168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:40:03.997115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783117+0000","last_change":"2023-11-26T07:12:23.120751+0000","last_active":"2023-11-26T07:12:24.783117+0000","last_peered":"2023-11-26T07:12:24.783117+0000","last_clean":"2023-11-26T07:12:24.783117+0000","last_became_active":"2023-11-26T07:12:23.120488+0000","last_became_peered":"2023-11-26T07:12:23.120488+0000","last_unstale":"2023-11-26T07:12:24.783117+0000","last_undegraded":"2023-11-26T07:12:24.783117+0000","last_fullsized":"2023-11-26T07:12:24.783117+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:08.514156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436187+0000","last_change":"2023-11-26T07:12:22.115783+0000","last_active":"2023-11-26T07:12:24.436187+0000","last_peered":"2023-11-26T07:12:24.436187+0000","last_clean":"2023-11-26T07:12:24.436187+0000","last_became_active":"2023-11-26T07:12:22.115497+0000","last_became_peered":"2023-11-26T07:12:22.115497+0000","last_unstale":"2023-11-26T07:12:24.436187+0000","last_undegraded":"2023-11-26T07:12:24.436187+0000","last_fullsized":"2023-11-26T07:12:24.436187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:16.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783186+0000","last_change":"2023-11-26T07:12:23.120509+0000","last_active":"2023-11-26T07:12:24.783186+0000","last_peered":"2023-11-26T07:12:24.783186+0000","last_clean":"2023-11-26T07:12:24.783186+0000","last_became_active":"2023-11-26T07:12:23.120223+0000","last_became_peered":"2023-11-26T07:12:23.120223+0000","last_unstale":"2023-11-26T07:12:24.783186+0000","last_undegraded":"2023-11-26T07:12:24.783186+0000","last_fullsized":"2023-11-26T07:12:24.783186+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:43:38.628594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783110+0000","last_change":"2023-11-26T07:12:22.120233+0000","last_active":"2023-11-26T07:12:24.783110+0000","last_peered":"2023-11-26T07:12:24.783110+0000","last_clean":"2023-11-26T07:12:24.783110+0000","last_became_active":"2023-11-26T07:12:22.119963+0000","last_became_peered":"2023-11-26T07:12:22.119963+0000","last_unstale":"2023-11-26T07:12:24.783110+0000","last_undegraded":"2023-11-26T07:12:24.783110+0000","last_fullsized":"2023-11-26T07:12:24.783110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:56:02.517453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783444+0000","last_change":"2023-11-26T07:12:23.112011+0000","last_active":"2023-11-26T07:12:24.783444+0000","last_peered":"2023-11-26T07:12:24.783444+0000","last_clean":"2023-11-26T07:12:24.783444+0000","last_became_active":"2023-11-26T07:12:23.111720+0000","last_became_peered":"2023-11-26T07:12:23.111720+0000","last_unstale":"2023-11-26T07:12:24.783444+0000","last_undegraded":"2023-11-26T07:12:24.783444+0000","last_fullsized":"2023-11-26T07:12:24.783444+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:53:56.870495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783426+0000","last_change":"2023-11-26T07:12:22.125191+0000","last_active":"2023-11-26T07:12:24.783426+0000","last_peered":"2023-11-26T07:12:24.783426+0000","last_clean":"2023-11-26T07:12:24.783426+0000","last_became_active":"2023-11-26T07:12:22.124982+0000","last_became_peered":"2023-11-26T07:12:22.124982+0000","last_unstale":"2023-11-26T07:12:24.783426+0000","last_undegraded":"2023-11-26T07:12:24.783426+0000","last_fullsized":"2023-11-26T07:12:24.783426+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:16.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.782971+0000","last_change":"2023-11-26T07:12:23.112066+0000","last_active":"2023-11-26T07:12:24.782971+0000","last_peered":"2023-11-26T07:12:24.782971+0000","last_clean":"2023-11-26T07:12:24.782971+0000","last_became_active":"2023-11-26T07:12:23.111771+0000","last_became_peered":"2023-11-26T07:12:23.111771+0000","last_unstale":"2023-11-26T07:12:24.782971+0000","last_undegraded":"2023-11-26T07:12:24.782971+0000","last_fullsized":"2023-11-26T07:12:24.782971+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:40:05.001780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.436045+0000","last_change":"2023-11-26T07:12:22.120958+0000","last_active":"2023-11-26T07:12:24.436045+0000","last_peered":"2023-11-26T07:12:24.436045+0000","last_clean":"2023-11-26T07:12:24.436045+0000","last_became_active":"2023-11-26T07:12:22.120624+0000","last_became_peered":"2023-11-26T07:12:22.120624+0000","last_unstale":"2023-11-26T07:12:24.436045+0000","last_undegraded":"2023-11-26T07:12:24.436045+0000","last_fullsized":"2023-11-26T07:12:24.436045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:19:24.013733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107307+0000","last_change":"2023-11-26T07:12:23.128800+0000","last_active":"2023-11-26T07:12:24.107307+0000","last_peered":"2023-11-26T07:12:24.107307+0000","last_clean":"2023-11-26T07:12:24.107307+0000","last_became_active":"2023-11-26T07:12:23.128396+0000","last_became_peered":"2023-11-26T07:12:23.128396+0000","last_unstale":"2023-11-26T07:12:24.107307+0000","last_undegraded":"2023-11-26T07:12:24.107307+0000","last_fullsized":"2023-11-26T07:12:24.107307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:35:30.734856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107333+0000","last_change":"2023-11-26T07:12:22.117840+0000","last_active":"2023-11-26T07:12:24.107333+0000","last_peered":"2023-11-26T07:12:24.107333+0000","last_clean":"2023-11-26T07:12:24.107333+0000","last_became_active":"2023-11-26T07:12:22.117572+0000","last_became_peered":"2023-11-26T07:12:22.117572+0000","last_unstale":"2023-11-26T07:12:24.107333+0000","last_undegraded":"2023-11-26T07:12:24.107333+0000","last_fullsized":"2023-11-26T07:12:24.107333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:40:48.702292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783429+0000","last_change":"2023-11-26T07:12:23.129557+0000","last_active":"2023-11-26T07:12:24.783429+0000","last_peered":"2023-11-26T07:12:24.783429+0000","last_clean":"2023-11-26T07:12:24.783429+0000","last_became_active":"2023-11-26T07:12:23.129286+0000","last_became_peered":"2023-11-26T07:12:23.129286+0000","last_unstale":"2023-11-26T07:12:24.783429+0000","last_undegraded":"2023-11-26T07:12:24.783429+0000","last_fullsized":"2023-11-26T07:12:24.783429+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:17:02.369629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783745+0000","last_change":"2023-11-26T07:12:22.120133+0000","last_active":"2023-11-26T07:12:24.783745+0000","last_peered":"2023-11-26T07:12:24.783745+0000","last_clean":"2023-11-26T07:12:24.783745+0000","last_became_active":"2023-11-26T07:12:22.119839+0000","last_became_peered":"2023-11-26T07:12:22.119839+0000","last_unstale":"2023-11-26T07:12:24.783745+0000","last_undegraded":"2023-11-26T07:12:24.783745+0000","last_fullsized":"2023-11-26T07:12:24.783745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:45:48.365800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.108120+0000","last_change":"2023-11-26T07:12:23.127529+0000","last_active":"2023-11-26T07:12:24.108120+0000","last_peered":"2023-11-26T07:12:24.108120+0000","last_clean":"2023-11-26T07:12:24.108120+0000","last_became_active":"2023-11-26T07:12:23.127220+0000","last_became_peered":"2023-11-26T07:12:23.127220+0000","last_unstale":"2023-11-26T07:12:24.108120+0000","last_undegraded":"2023-11-26T07:12:24.108120+0000","last_fullsized":"2023-11-26T07:12:24.108120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:35:30.734856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:25.138902+0000","last_change":"2023-11-26T07:12:22.120219+0000","last_active":"2023-11-26T07:12:25.138902+0000","last_peered":"2023-11-26T07:12:25.138902+0000","last_clean":"2023-11-26T07:12:25.138902+0000","last_became_active":"2023-11-26T07:12:22.119933+0000","last_became_peered":"2023-11-26T07:12:22.119933+0000","last_unstale":"2023-11-26T07:12:25.138902+0000","last_undegraded":"2023-11-26T07:12:25.138902+0000","last_fullsized":"2023-11-26T07:12:25.138902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:26:37.066177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783628+0000","last_change":"2023-11-26T07:12:23.115136+0000","last_active":"2023-11-26T07:12:24.783628+0000","last_peered":"2023-11-26T07:12:24.783628+0000","last_clean":"2023-11-26T07:12:24.783628+0000","last_became_active":"2023-11-26T07:12:23.114814+0000","last_became_peered":"2023-11-26T07:12:23.114814+0000","last_unstale":"2023-11-26T07:12:24.783628+0000","last_undegraded":"2023-11-26T07:12:24.783628+0000","last_fullsized":"2023-11-26T07:12:24.783628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:36.670009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783769+0000","last_change":"2023-11-26T07:12:22.121582+0000","last_active":"2023-11-26T07:12:24.783769+0000","last_peered":"2023-11-26T07:12:24.783769+0000","last_clean":"2023-11-26T07:12:24.783769+0000","last_became_active":"2023-11-26T07:12:22.121235+0000","last_became_peered":"2023-11-26T07:12:22.121235+0000","last_unstale":"2023-11-26T07:12:24.783769+0000","last_undegraded":"2023-11-26T07:12:24.783769+0000","last_fullsized":"2023-11-26T07:12:24.783769+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:16.763232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783508+0000","last_change":"2023-11-26T07:12:23.126859+0000","last_active":"2023-11-26T07:12:24.783508+0000","last_peered":"2023-11-26T07:12:24.783508+0000","last_clean":"2023-11-26T07:12:24.783508+0000","last_became_active":"2023-11-26T07:12:23.126578+0000","last_became_peered":"2023-11-26T07:12:23.126578+0000","last_unstale":"2023-11-26T07:12:24.783508+0000","last_undegraded":"2023-11-26T07:12:24.783508+0000","last_fullsized":"2023-11-26T07:12:24.783508+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:34:09.475730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783069+0000","last_change":"2023-11-26T07:12:22.116056+0000","last_active":"2023-11-26T07:12:24.783069+0000","last_peered":"2023-11-26T07:12:24.783069+0000","last_clean":"2023-11-26T07:12:24.783069+0000","last_became_active":"2023-11-26T07:12:22.115779+0000","last_became_peered":"2023-11-26T07:12:22.115779+0000","last_unstale":"2023-11-26T07:12:24.783069+0000","last_undegraded":"2023-11-26T07:12:24.783069+0000","last_fullsized":"2023-11-26T07:12:24.783069+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:46:56.965203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783194+0000","last_change":"2023-11-26T07:12:23.128745+0000","last_active":"2023-11-26T07:12:24.783194+0000","last_peered":"2023-11-26T07:12:24.783194+0000","last_clean":"2023-11-26T07:12:24.783194+0000","last_became_active":"2023-11-26T07:12:23.128388+0000","last_became_peered":"2023-11-26T07:12:23.128388+0000","last_unstale":"2023-11-26T07:12:24.783194+0000","last_undegraded":"2023-11-26T07:12:24.783194+0000","last_fullsized":"2023-11-26T07:12:24.783194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:59:53.542038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783740+0000","last_change":"2023-11-26T07:12:22.121254+0000","last_active":"2023-11-26T07:12:24.783740+0000","last_peered":"2023-11-26T07:12:24.783740+0000","last_clean":"2023-11-26T07:12:24.783740+0000","last_became_active":"2023-11-26T07:12:22.120394+0000","last_became_peered":"2023-11-26T07:12:22.120394+0000","last_unstale":"2023-11-26T07:12:24.783740+0000","last_undegraded":"2023-11-26T07:12:24.783740+0000","last_fullsized":"2023-11-26T07:12:24.783740+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:34:08.471064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105410+0000","last_change":"2023-11-26T07:12:23.118888+0000","last_active":"2023-11-26T07:12:24.105410+0000","last_peered":"2023-11-26T07:12:24.105410+0000","last_clean":"2023-11-26T07:12:24.105410+0000","last_became_active":"2023-11-26T07:12:23.118579+0000","last_became_peered":"2023-11-26T07:12:23.118579+0000","last_unstale":"2023-11-26T07:12:24.105410+0000","last_undegraded":"2023-11-26T07:12:24.105410+0000","last_fullsized":"2023-11-26T07:12:24.105410+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:16.181329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435956+0000","last_change":"2023-11-26T07:12:22.115573+0000","last_active":"2023-11-26T07:12:24.435956+0000","last_peered":"2023-11-26T07:12:24.435956+0000","last_clean":"2023-11-26T07:12:24.435956+0000","last_became_active":"2023-11-26T07:12:22.115247+0000","last_became_peered":"2023-11-26T07:12:22.115247+0000","last_unstale":"2023-11-26T07:12:24.435956+0000","last_undegraded":"2023-11-26T07:12:24.435956+0000","last_fullsized":"2023-11-26T07:12:24.435956+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:45:48.365800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.435858+0000","last_change":"2023-11-26T07:12:23.120415+0000","last_active":"2023-11-26T07:12:24.435858+0000","last_peered":"2023-11-26T07:12:24.435858+0000","last_clean":"2023-11-26T07:12:24.435858+0000","last_became_active":"2023-11-26T07:12:23.119601+0000","last_became_peered":"2023-11-26T07:12:23.119601+0000","last_unstale":"2023-11-26T07:12:24.435858+0000","last_undegraded":"2023-11-26T07:12:24.435858+0000","last_fullsized":"2023-11-26T07:12:24.435858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:36.670009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.105513+0000","last_change":"2023-11-26T07:12:22.119523+0000","last_active":"2023-11-26T07:12:24.105513+0000","last_peered":"2023-11-26T07:12:24.105513+0000","last_clean":"2023-11-26T07:12:24.105513+0000","last_became_active":"2023-11-26T07:12:22.119260+0000","last_became_peered":"2023-11-26T07:12:22.119260+0000","last_unstale":"2023-11-26T07:12:24.105513+0000","last_undegraded":"2023-11-26T07:12:24.105513+0000","last_fullsized":"2023-11-26T07:12:24.105513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:20:31.048080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.107087+0000","last_change":"2023-11-26T07:12:23.120541+0000","last_active":"2023-11-26T07:12:24.107087+0000","last_peered":"2023-11-26T07:12:24.107087+0000","last_clean":"2023-11-26T07:12:24.107087+0000","last_became_active":"2023-11-26T07:12:23.120224+0000","last_became_peered":"2023-11-26T07:12:23.120224+0000","last_unstale":"2023-11-26T07:12:24.107087+0000","last_undegraded":"2023-11-26T07:12:24.107087+0000","last_fullsized":"2023-11-26T07:12:24.107087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:22.095591+0000","last_clean_scrub_stamp":"2023-11-26T07:12:22.095591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:50.833212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-26T07:12:24.783438+0000","last_change":"2023-11-26T07:12:22.124427+0000","last_active":"2023-11-26T07:12:24.783438+0000","last_peered":"2023-11-26T07:12:24.783438+0000","last_clean":"2023-11-26T07:12:24.783438+0000","last_became_active":"2023-11-26T07:12:22.124140+0000","last_became_peered":"2023-11-26T07:12:22.124140+0000","last_unstale":"2023-11-26T07:12:24.783438+0000","last_undegraded":"2023-11-26T07:12:24.783438+0000","last_fullsized":"2023-11-26T07:12:24.783438+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:21.090926+0000","last_clean_scrub_stamp":"2023-11-26T07:12:21.090926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:04:18.883671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:12:23.110367+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:12:23.110367+0000","last_peered":"2023-11-26T07:12:23.110367+0000","last_clean":"2023-11-26T07:12:23.110367+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:12:23.110367+0000","last_undegraded":"2023-11-26T07:12:23.110367+0000","last_fullsized":"2023-11-26T07:12:23.110367+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":741721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.47199999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68300000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.629},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.40999999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.442},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.58699999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.72499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64800000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60099999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.499}]},{"osd":4,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.67000000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":146597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.36699999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.39800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.42099999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59499999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68600000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.78400000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":735331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.26200000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.32200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.30499999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.47699999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.441}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59299999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.39600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":151333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.376}]},{"osd":1,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.30499999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39000000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72099999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.68700000000000006}]},{"osd":6,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.75900000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.70099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":152897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.442}]},{"osd":1,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.9109999999999996,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.38400000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.71899999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8129999999999997,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.83199999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.873},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.883,"5min":4.883,"15min":4.883},"last":0.79600000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.8499999999999996,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.85099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":145067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.48199999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.434},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.40999999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.53000000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55900000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.433},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.66700000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.623},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.59799999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.56100000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.68700000000000006}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.41299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.48499999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:12:27.987 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:12:27.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:12:28.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.148+0000 7f0e593c7700 1 Processor -- start 2023-11-26T07:12:28.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.148+0000 7f0e593c7700 1 -- start start 2023-11-26T07:12:28.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.149+0000 7f0e593c7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0e5415ada0 0x7f0e5406f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.149+0000 7f0e593c7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540afb40 0x7f0e540cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.149+0000 7f0e593c7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e540cccd0 con 0x7f0e540cfa80 2023-11-26T07:12:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.149+0000 7f0e593c7700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e540cd2e0 con 0x7f0e5415ada0 2023-11-26T07:12:28.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.149+0000 7f0e593c7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e540cd5c0 con 0x7f0e540afb40 2023-11-26T07:12:28.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.149+0000 7f0e527fc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540afb40 0x7f0e540cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e52ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0e5415ada0 0x7f0e5406f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e537fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0e540cfa80 0x7f0e5406d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:40804/0 (socket says 172.21.15.83:40804) 2023-11-26T07:12:28.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e527fc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540afb40 0x7f0e540cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:46600/0 (socket says 172.21.15.83:46600) 2023-11-26T07:12:28.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e537fe700 1 -- 172.21.15.83:0/4217147091 learned_addr learned my addr 172.21.15.83:0/4217147091 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:28.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e527fc700 1 -- 172.21.15.83:0/4217147091 >> v2:172.21.15.203:3300/0 conn(0x7f0e5415ada0 msgr2=0x7f0e5406f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e527fc700 1 --2- 172.21.15.83:0/4217147091 >> v2:172.21.15.203:3300/0 conn(0x7f0e5415ada0 0x7f0e5406f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e51ffb700 1 -- 172.21.15.83:0/4217147091 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784427248 0 0) 0x7f0e540cccd0 con 0x7f0e540cfa80 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e527fc700 1 -- 172.21.15.83:0/4217147091 >> v1:172.21.15.203:6789/0 conn(0x7f0e540cfa80 legacy=0x7f0e5406d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e527fc700 1 -- 172.21.15.83:0/4217147091 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5406fd50 con 0x7f0e540afb40 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e527fc700 1 --2- 172.21.15.83:0/4217147091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540afb40 0x7f0e540cf540 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009840 tx=0x7f0e44009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a2936ff671069e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e51ffb700 1 -- 172.21.15.83:0/4217147091 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44011830 con 0x7f0e540afb40 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e51ffb700 1 -- 172.21.15.83:0/4217147091 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e44011990 con 0x7f0e540afb40 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.150+0000 7f0e51ffb700 1 -- 172.21.15.83:0/4217147091 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e44011d20 con 0x7f0e540afb40 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.151+0000 7f0e593c7700 1 -- 172.21.15.83:0/4217147091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540afb40 msgr2=0x7f0e540cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.151+0000 7f0e593c7700 1 --2- 172.21.15.83:0/4217147091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540afb40 0x7f0e540cf540 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009840 tx=0x7f0e44009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.151+0000 7f0e593c7700 1 -- 172.21.15.83:0/4217147091 shutdown_connections 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.151+0000 7f0e593c7700 1 --2- 172.21.15.83:0/4217147091 >> v2:172.21.15.203:3300/0 conn(0x7f0e5415ada0 0x7f0e5406f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.151+0000 7f0e593c7700 1 --2- 172.21.15.83:0/4217147091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540afb40 0x7f0e540cf540 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.151+0000 7f0e593c7700 1 -- 172.21.15.83:0/4217147091 >> 172.21.15.83:0/4217147091 conn(0x7f0e540b2050 msgr2=0x7f0e540b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.151+0000 7f0e593c7700 1 -- 172.21.15.83:0/4217147091 shutdown_connections 2023-11-26T07:12:28.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.151+0000 7f0e593c7700 1 -- 172.21.15.83:0/4217147091 wait complete. 2023-11-26T07:12:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.152+0000 7f0e593c7700 1 Processor -- start 2023-11-26T07:12:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.152+0000 7f0e593c7700 1 -- start start 2023-11-26T07:12:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e593c7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e540afb40 0x7f0e541477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e593c7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540cfa80 0x7f0e541457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e593c7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e5415ada0 0x7f0e54145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e593c7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e54147cf0 con 0x7f0e540afb40 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e593c7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e541467a0 con 0x7f0e5415ada0 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e593c7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e541469f0 con 0x7f0e540cfa80 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e52ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e540afb40 0x7f0e541477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e52ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e540afb40 0x7f0e541477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36148/0 (socket says 172.21.15.83:36148) 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e52ffd700 1 -- 172.21.15.83:0/2372587774 learned_addr learned my addr 172.21.15.83:0/2372587774 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e527fc700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540cfa80 0x7f0e541457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e537fe700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e5415ada0 0x7f0e54145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e52ffd700 1 -- 172.21.15.83:0/2372587774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540cfa80 msgr2=0x7f0e541457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e52ffd700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540cfa80 0x7f0e541457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e52ffd700 1 -- 172.21.15.83:0/2372587774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e5415ada0 msgr2=0x7f0e54145d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e52ffd700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e5415ada0 0x7f0e54145d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e52ffd700 1 -- 172.21.15.83:0/2372587774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e440094a0 con 0x7f0e540afb40 2023-11-26T07:12:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e537fe700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e5415ada0 0x7f0e54145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.153+0000 7f0e52ffd700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e540afb40 0x7f0e541477b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800e9b0 tx=0x7f0e4800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.154+0000 7f0e3b7fe700 1 -- 172.21.15.83:0/2372587774 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48009710 con 0x7f0e540afb40 2023-11-26T07:12:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.154+0000 7f0e3b7fe700 1 -- 172.21.15.83:0/2372587774 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4800c850 con 0x7f0e540afb40 2023-11-26T07:12:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.154+0000 7f0e593c7700 1 -- 172.21.15.83:0/2372587774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e54146cd0 con 0x7f0e540afb40 2023-11-26T07:12:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.154+0000 7f0e3b7fe700 1 -- 172.21.15.83:0/2372587774 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4800cbc0 con 0x7f0e540afb40 2023-11-26T07:12:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.154+0000 7f0e593c7700 1 -- 172.21.15.83:0/2372587774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e54147290 con 0x7f0e540afb40 2023-11-26T07:12:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.155+0000 7f0e593c7700 1 -- 172.21.15.83:0/2372587774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e54068d90 con 0x7f0e540afb40 2023-11-26T07:12:28.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.155+0000 7f0e3b7fe700 1 -- 172.21.15.83:0/2372587774 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e4800cd20 con 0x7f0e540afb40 2023-11-26T07:12:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.156+0000 7f0e3b7fe700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0e4006aed0 0x7f0e4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.156+0000 7f0e527fc700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0e4006aed0 0x7f0e4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.156+0000 7f0e3b7fe700 1 -- 172.21.15.83:0/2372587774 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e480896f0 con 0x7f0e540afb40 2023-11-26T07:12:28.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.156+0000 7f0e527fc700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0e4006aed0 0x7f0e4006d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009fd0 tx=0x7f0e4401c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:28.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.160+0000 7f0e3b7fe700 1 -- 172.21.15.83:0/2372587774 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e48054830 con 0x7f0e540afb40 2023-11-26T07:12:28.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.431+0000 7f0e593c7700 1 -- 172.21.15.83:0/2372587774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0e54068fb0 con 0x7f0e540afb40 2023-11-26T07:12:28.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.433+0000 7f0e3b7fe700 1 -- 172.21.15.83:0/2372587774 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0e480210b0 con 0x7f0e540afb40 2023-11-26T07:12:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2023-11-26T07:12:28.435 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ== 2023-11-26T07:12:28.435 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2023-11-26T07:12:28.436 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2023-11-26T07:12:28.436 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:12:28.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 -- 172.21.15.83:0/2372587774 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0e4006aed0 msgr2=0x7f0e4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:28.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0e4006aed0 0x7f0e4006d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009fd0 tx=0x7f0e4401c040 comp rx=0 tx=0).stop 2023-11-26T07:12:28.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 -- 172.21.15.83:0/2372587774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e540afb40 msgr2=0x7f0e541477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:28.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e540afb40 0x7f0e541477b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800e9b0 tx=0x7f0e4800ed70 comp rx=0 tx=0).stop 2023-11-26T07:12:28.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 -- 172.21.15.83:0/2372587774 shutdown_connections 2023-11-26T07:12:28.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e5415ada0 0x7f0e54145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e540afb40 0x7f0e541477b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0e4006aed0 0x7f0e4006d390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 --2- 172.21.15.83:0/2372587774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e540cfa80 0x7f0e541457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 -- 172.21.15.83:0/2372587774 >> 172.21.15.83:0/2372587774 conn(0x7f0e540b2050 msgr2=0x7f0e540b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:28.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 -- 172.21.15.83:0/2372587774 shutdown_connections 2023-11-26T07:12:28.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.435+0000 7f0e593c7700 1 -- 172.21.15.83:0/2372587774 wait complete. 2023-11-26T07:12:28.439 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2023-11-26T07:12:28.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.568+0000 7fdf8ad5b700 1 Processor -- start 2023-11-26T07:12:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.568+0000 7fdf8ad5b700 1 -- start start 2023-11-26T07:12:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.569+0000 7fdf8ad5b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf841446a0 0x7fdf840b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.569+0000 7fdf8ad5b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf840adf00 0x7fdf8406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.569+0000 7fdf8ad5b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdf8406c540 0x7fdf8406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.569+0000 7fdf8ad5b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf84144a80 con 0x7fdf841446a0 2023-11-26T07:12:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.569+0000 7fdf8ad5b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8406c9b0 con 0x7fdf8406c540 2023-11-26T07:12:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.569+0000 7fdf8ad5b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf840af950 con 0x7fdf840adf00 2023-11-26T07:12:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.569+0000 7fdf88af7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf841446a0 0x7fdf840b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.569+0000 7fdf88af7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdf841446a0 0x7fdf840b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36156/0 (socket says 172.21.15.83:36156) 2023-11-26T07:12:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.569+0000 7fdf88af7700 1 -- 172.21.15.83:0/1822149118 learned_addr learned my addr 172.21.15.83:0/1822149118 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:28.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf83fff700 1 --2- 172.21.15.83:0/1822149118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf840adf00 0x7fdf8406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf892f8700 1 --2- 172.21.15.83:0/1822149118 >> v2:172.21.15.203:3300/0 conn(0x7fdf8406c540 0x7fdf8406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf88af7700 1 -- 172.21.15.83:0/1822149118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf840adf00 msgr2=0x7fdf8406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf88af7700 1 --2- 172.21.15.83:0/1822149118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf840adf00 0x7fdf8406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf88af7700 1 -- 172.21.15.83:0/1822149118 >> v2:172.21.15.203:3300/0 conn(0x7fdf8406c540 msgr2=0x7fdf8406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf88af7700 1 --2- 172.21.15.83:0/1822149118 >> v2:172.21.15.203:3300/0 conn(0x7fdf8406c540 0x7fdf8406da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf88af7700 1 -- 172.21.15.83:0/1822149118 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf840afc30 con 0x7fdf841446a0 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf892f8700 1 --2- 172.21.15.83:0/1822149118 >> v2:172.21.15.203:3300/0 conn(0x7fdf8406c540 0x7fdf8406da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf88af7700 1 --2- 172.21.15.83:0/1822149118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf841446a0 0x7fdf840b5ae0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdf8406f530 tx=0x7fdf78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ce564ca5cebe9d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf837fe700 1 -- 172.21.15.83:0/1822149118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800b860 con 0x7fdf841446a0 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf837fe700 1 -- 172.21.15.83:0/1822149118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7800b9c0 con 0x7fdf841446a0 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf837fe700 1 -- 172.21.15.83:0/1822149118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800bd50 con 0x7fdf841446a0 2023-11-26T07:12:28.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/1822149118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf841446a0 msgr2=0x7fdf840b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.570+0000 7fdf8ad5b700 1 --2- 172.21.15.83:0/1822149118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf841446a0 0x7fdf840b5ae0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdf8406f530 tx=0x7fdf78009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.571+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/1822149118 shutdown_connections 2023-11-26T07:12:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.571+0000 7fdf8ad5b700 1 --2- 172.21.15.83:0/1822149118 >> v2:172.21.15.203:3300/0 conn(0x7fdf8406c540 0x7fdf8406da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.571+0000 7fdf8ad5b700 1 --2- 172.21.15.83:0/1822149118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf841446a0 0x7fdf840b5ae0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.571+0000 7fdf8ad5b700 1 --2- 172.21.15.83:0/1822149118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf840adf00 0x7fdf8406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.571+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/1822149118 >> 172.21.15.83:0/1822149118 conn(0x7fdf84071b90 msgr2=0x7fdf84071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.571+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/1822149118 shutdown_connections 2023-11-26T07:12:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.571+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/1822149118 wait complete. 2023-11-26T07:12:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.572+0000 7fdf8ad5b700 1 Processor -- start 2023-11-26T07:12:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.572+0000 7fdf8ad5b700 1 -- start start 2023-11-26T07:12:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf8ad5b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf8406c540 0x7fdf8413f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf8ad5b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf840adf00 0x7fdf8413a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf8ad5b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf841446a0 0x7fdf84141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf8ad5b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8414a550 con 0x7fdf841446a0 2023-11-26T07:12:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf8ad5b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8414a830 con 0x7fdf8406c540 2023-11-26T07:12:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf8ad5b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8414ab10 con 0x7fdf840adf00 2023-11-26T07:12:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf892f8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf841446a0 0x7fdf84141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf892f8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf841446a0 0x7fdf84141fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36166/0 (socket says 172.21.15.83:36166) 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf892f8700 1 -- 172.21.15.83:0/3994649636 learned_addr learned my addr 172.21.15.83:0/3994649636 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf88af7700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf8406c540 0x7fdf8413f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf83fff700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf840adf00 0x7fdf8413a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf892f8700 1 -- 172.21.15.83:0/3994649636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf840adf00 msgr2=0x7fdf8413a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf892f8700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf840adf00 0x7fdf8413a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf892f8700 1 -- 172.21.15.83:0/3994649636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf8406c540 msgr2=0x7fdf8413f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf892f8700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf8406c540 0x7fdf8413f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf892f8700 1 -- 172.21.15.83:0/3994649636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78009550 con 0x7fdf841446a0 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf892f8700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf841446a0 0x7fdf84141fd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00cef0 tx=0x7fdf7c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf817fa700 1 -- 172.21.15.83:0/3994649636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00add0 con 0x7fdf841446a0 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf817fa700 1 -- 172.21.15.83:0/3994649636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c015440 con 0x7fdf841446a0 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/3994649636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8414adf0 con 0x7fdf841446a0 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.573+0000 7fdf817fa700 1 -- 172.21.15.83:0/3994649636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0157f0 con 0x7fdf841446a0 2023-11-26T07:12:28.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.574+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/3994649636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf84142510 con 0x7fdf841446a0 2023-11-26T07:12:28.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.574+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/3994649636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf840753f0 con 0x7fdf841446a0 2023-11-26T07:12:28.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.575+0000 7fdf817fa700 1 -- 172.21.15.83:0/3994649636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf7c00d430 con 0x7fdf841446a0 2023-11-26T07:12:28.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.575+0000 7fdf817fa700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf6806aed0 0x7fdf6806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:28.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.575+0000 7fdf817fa700 1 -- 172.21.15.83:0/3994649636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf7c089b70 con 0x7fdf841446a0 2023-11-26T07:12:28.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.575+0000 7fdf88af7700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf6806aed0 0x7fdf6806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:28.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.576+0000 7fdf88af7700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf6806aed0 0x7fdf6806d390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdf78000c00 tx=0x7fdf78008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:28.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.580+0000 7fdf817fa700 1 -- 172.21.15.83:0/3994649636 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf7c054c20 con 0x7fdf841446a0 2023-11-26T07:12:28.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.813+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/3994649636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf8414b310 con 0x7fdf841446a0 2023-11-26T07:12:28.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.814+0000 7fdf817fa700 1 -- 172.21.15.83:0/3994649636 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fdf8414b310 con 0x7fdf841446a0 2023-11-26T07:12:28.816 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:28.817 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/486975373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":486975373},{"type":"v1","addr":"172.21.15.203:6834","nonce":486975373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3188343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3188343240},{"type":"v1","addr":"172.21.15.83:6836","nonce":3188343240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4271462644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4271462644},{"type":"v1","addr":"172.21.15.83:6837","nonce":4271462644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:22.505426+0000","modified":"2023-11-26T07:12:25.894494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6835/995330476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":995330476},{"type":"v1","addr":"172.21.15.203:6835","nonce":995330476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:12:28.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/3994649636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf6806aed0 msgr2=0x7fdf6806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:28.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf6806aed0 0x7fdf6806d390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdf78000c00 tx=0x7fdf78008030 comp rx=0 tx=0).stop 2023-11-26T07:12:28.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/3994649636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf841446a0 msgr2=0x7fdf84141fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:28.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf841446a0 0x7fdf84141fd0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00cef0 tx=0x7fdf7c00a340 comp rx=0 tx=0).stop 2023-11-26T07:12:28.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/3994649636 shutdown_connections 2023-11-26T07:12:28.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf841446a0 0x7fdf84141fd0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf8406c540 0x7fdf8413f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdf6806aed0 0x7fdf6806d390 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 --2- 172.21.15.83:0/3994649636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf840adf00 0x7fdf8413a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:28.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/3994649636 >> 172.21.15.83:0/3994649636 conn(0x7fdf84071b90 msgr2=0x7fdf840b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:28.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/3994649636 shutdown_connections 2023-11-26T07:12:28.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:28.816+0000 7fdf8ad5b700 1 -- 172.21.15.83:0/3994649636 wait complete. 2023-11-26T07:12:28.820 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2023-11-26T07:12:29.233 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:12:22.505426+0000', 'modified': '2023-11-26T07:12:25.894494+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5100}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5100': {'gid': 5100, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.203:6835/995330476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6833', 'nonce': 995330476}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 995330476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:12:29.233 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:12:29.233 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-26T07:12:29.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.357+0000 7f7fecd91700 1 Processor -- start 2023-11-26T07:12:29.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.357+0000 7f7fecd91700 1 -- start start 2023-11-26T07:12:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.358+0000 7f7fecd91700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7fe81446a0 0x7f7fe80b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.358+0000 7f7fecd91700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fe80adf00 0x7f7fe806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.358+0000 7f7fecd91700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe80af950 con 0x7f7fe806c540 2023-11-26T07:12:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.358+0000 7f7fecd91700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe8144a80 con 0x7f7fe81446a0 2023-11-26T07:12:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.358+0000 7f7fecd91700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe806c9b0 con 0x7f7fe80adf00 2023-11-26T07:12:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.358+0000 7f7fe659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7fe81446a0 0x7f7fe80b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.358+0000 7f7fe659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7fe81446a0 0x7f7fe80b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36170/0 (socket says 172.21.15.83:36170) 2023-11-26T07:12:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.358+0000 7f7fe659c700 1 -- 172.21.15.83:0/862825784 learned_addr learned my addr 172.21.15.83:0/862825784 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.358+0000 7f7fe659c700 1 -- 172.21.15.83:0/862825784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fe80adf00 msgr2=0x7f7fe806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.358+0000 7f7fe5d9b700 1 --2- 172.21.15.83:0/862825784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fe80adf00 0x7f7fe806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.358+0000 7f7fe659c700 1 --2- 172.21.15.83:0/862825784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fe80adf00 0x7f7fe806c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fe659c700 1 -- 172.21.15.83:0/862825784 >> v1:172.21.15.203:6789/0 conn(0x7f7fe806c540 legacy=0x7f7fe806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fe659c700 1 -- 172.21.15.83:0/862825784 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe80afc30 con 0x7f7fe81446a0 2023-11-26T07:12:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fe659c700 1 --2- 172.21.15.83:0/862825784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe81446a0 0x7f7fe80b5ae0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009920 tx=0x7f7fdc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e433e1628411f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fe559a700 1 -- 172.21.15.83:0/862825784 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc004230 con 0x7f7fe81446a0 2023-11-26T07:12:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fe559a700 1 -- 172.21.15.83:0/862825784 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc004390 con 0x7f7fe81446a0 2023-11-26T07:12:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fe559a700 1 -- 172.21.15.83:0/862825784 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc009e70 con 0x7f7fe81446a0 2023-11-26T07:12:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fecd91700 1 -- 172.21.15.83:0/862825784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe81446a0 msgr2=0x7f7fe80b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fecd91700 1 --2- 172.21.15.83:0/862825784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe81446a0 0x7f7fe80b5ae0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009920 tx=0x7f7fdc009c30 comp rx=0 tx=0).stop 2023-11-26T07:12:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fecd91700 1 -- 172.21.15.83:0/862825784 shutdown_connections 2023-11-26T07:12:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fecd91700 1 --2- 172.21.15.83:0/862825784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe81446a0 0x7f7fe80b5ae0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fecd91700 1 --2- 172.21.15.83:0/862825784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fe80adf00 0x7f7fe806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fecd91700 1 -- 172.21.15.83:0/862825784 >> 172.21.15.83:0/862825784 conn(0x7f7fe8071b90 msgr2=0x7f7fe8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.359+0000 7f7fecd91700 1 -- 172.21.15.83:0/862825784 shutdown_connections 2023-11-26T07:12:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.360+0000 7f7fecd91700 1 -- 172.21.15.83:0/862825784 wait complete. 2023-11-26T07:12:29.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.361+0000 7f7fecd91700 1 Processor -- start 2023-11-26T07:12:29.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.361+0000 7f7fecd91700 1 -- start start 2023-11-26T07:12:29.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fecd91700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fe806c540 0x7f7fe81430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fecd91700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fe80adf00 0x7f7fe813f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fecd91700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe81446a0 0x7f7fe813a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fecd91700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe809b610 con 0x7f7fe81446a0 2023-11-26T07:12:29.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fecd91700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe809aa00 con 0x7f7fe80adf00 2023-11-26T07:12:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fecd91700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe809ace0 con 0x7f7fe806c540 2023-11-26T07:12:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fe6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe81446a0 0x7f7fe813a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fe6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe81446a0 0x7f7fe813a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36180/0 (socket says 172.21.15.83:36180) 2023-11-26T07:12:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fe6d9d700 1 -- 172.21.15.83:0/1539862280 learned_addr learned my addr 172.21.15.83:0/1539862280 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fe659c700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fe806c540 0x7f7fe81430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fe5d9b700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fe80adf00 0x7f7fe813f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fe6d9d700 1 -- 172.21.15.83:0/1539862280 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fe806c540 msgr2=0x7f7fe81430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fe6d9d700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fe806c540 0x7f7fe81430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fe6d9d700 1 -- 172.21.15.83:0/1539862280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fe80adf00 msgr2=0x7f7fe813f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fe6d9d700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fe80adf00 0x7f7fe813f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fe6d9d700 1 -- 172.21.15.83:0/1539862280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc009550 con 0x7f7fe81446a0 2023-11-26T07:12:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fe6d9d700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe81446a0 0x7f7fe813a010 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000c7a0 tx=0x7f7fe000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fd6ffd700 1 -- 172.21.15.83:0/1539862280 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0003e10 con 0x7f7fe81446a0 2023-11-26T07:12:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fd6ffd700 1 -- 172.21.15.83:0/1539862280 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe0005070 con 0x7f7fe81446a0 2023-11-26T07:12:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fecd91700 1 -- 172.21.15.83:0/1539862280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe809afc0 con 0x7f7fe81446a0 2023-11-26T07:12:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fd6ffd700 1 -- 172.21.15.83:0/1539862280 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0005420 con 0x7f7fe81446a0 2023-11-26T07:12:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.362+0000 7f7fecd91700 1 -- 172.21.15.83:0/1539862280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe8143620 con 0x7f7fe81446a0 2023-11-26T07:12:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.364+0000 7f7fd6ffd700 1 -- 172.21.15.83:0/1539862280 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7fe0005580 con 0x7f7fe81446a0 2023-11-26T07:12:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.364+0000 7f7fd6ffd700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7fc806aed0 0x7f7fc806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.364+0000 7f7fd6ffd700 1 -- 172.21.15.83:0/1539862280 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fe0088b60 con 0x7f7fe81446a0 2023-11-26T07:12:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.364+0000 7f7fe659c700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7fc806aed0 0x7f7fc806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.364+0000 7f7fe659c700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7fc806aed0 0x7f7fc806d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009870 tx=0x7f7fdc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.366+0000 7f7ff8f8ab80 1 Processor -- start 2023-11-26T07:12:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.366+0000 7f7ff8f8ab80 1 -- start start 2023-11-26T07:12:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x56280681a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x56280681ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d1490 0x5628067fde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562806747770 con 0x5628067d4080 2023-11-26T07:12:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56280672e750 con 0x5628067e9ed0 2023-11-26T07:12:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56280672c070 con 0x5628067d1490 2023-11-26T07:12:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7fe659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x56280681a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7fe659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x56280681a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36184/0 (socket says 172.21.15.83:36184) 2023-11-26T07:12:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7fe659c700 1 -- 172.21.15.83:0/2214916641 learned_addr learned my addr 172.21.15.83:0/2214916641 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7fe659c700 1 -- 172.21.15.83:0/2214916641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d1490 msgr2=0x5628067fde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7fe659c700 1 --2- 172.21.15.83:0/2214916641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d1490 0x5628067fde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7fe659c700 1 -- 172.21.15.83:0/2214916641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 msgr2=0x56280681ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:29.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7fe659c700 1 --2- 172.21.15.83:0/2214916641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x56280681ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.367+0000 7f7fe659c700 1 -- 172.21.15.83:0/2214916641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628065a45e0 con 0x5628067d4080 2023-11-26T07:12:29.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.368+0000 7f7fe659c700 1 --2- 172.21.15.83:0/2214916641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x56280681a840 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc004360 tx=0x7f7fdc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e550b7e8979198b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:29.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.368+0000 7f7fc7fff700 1 -- 172.21.15.83:0/2214916641 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc006d10 con 0x5628067d4080 2023-11-26T07:12:29.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.368+0000 7f7fc7fff700 1 -- 172.21.15.83:0/2214916641 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc0036a0 con 0x5628067d4080 2023-11-26T07:12:29.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.368+0000 7f7fc7fff700 1 -- 172.21.15.83:0/2214916641 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc003a30 con 0x5628067d4080 2023-11-26T07:12:29.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.368+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2214916641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 msgr2=0x56280681a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.368+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/2214916641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x56280681a840 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc004360 tx=0x7f7fdc010000 comp rx=0 tx=0).stop 2023-11-26T07:12:29.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.368+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2214916641 shutdown_connections 2023-11-26T07:12:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.368+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/2214916641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d1490 0x5628067fde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.368+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/2214916641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x56280681a840 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.368+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/2214916641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x56280681ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.368+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2214916641 >> 172.21.15.83:0/2214916641 conn(0x562806702310 msgr2=0x562806818e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.369+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2214916641 shutdown_connections 2023-11-26T07:12:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.369+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2214916641 wait complete. 2023-11-26T07:12:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.369+0000 7f7ff8f8ab80 1 -- start start 2023-11-26T07:12:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.369+0000 7f7ff8f8ab80 1 Processor -- start 2023-11-26T07:12:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d1490 0x5628067c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d4080 0x5628067c8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x562806814610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562806727960 con 0x5628067d1490 2023-11-26T07:12:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628066e1510 con 0x5628067e9ed0 2023-11-26T07:12:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628066e11b0 con 0x5628067d4080 2023-11-26T07:12:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7ff8f8ab80 10 client.0 _ll_register_callbacks cb 0x56280660ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:12:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7fe659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d1490 0x5628067c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7fe659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d1490 0x5628067c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36190/0 (socket says 172.21.15.83:36190) 2023-11-26T07:12:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7fe659c700 1 -- 172.21.15.83:0/3391888648 learned_addr learned my addr 172.21.15.83:0/3391888648 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7fe5d9b700 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d4080 0x5628067c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7fe6d9d700 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x562806814610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7fe659c700 1 -- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d4080 msgr2=0x5628067c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7fe659c700 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d4080 0x5628067c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7fe659c700 1 -- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 msgr2=0x562806814610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7fe659c700 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x562806814610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7fe659c700 1 -- 172.21.15.83:0/3391888648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe80afc30 con 0x5628067d1490 2023-11-26T07:12:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.370+0000 7f7fe6d9d700 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x562806814610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7fe659c700 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d1490 0x5628067c8330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc000c00 tx=0x7f7fdc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3763a9ba82aa3b86 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7fc57fa700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:12:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7fc57fa700 1 -- 172.21.15.83:0/3391888648 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc003c70 con 0x5628067d1490 2023-11-26T07:12:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7fc57fa700 1 -- 172.21.15.83:0/3391888648 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fdc003dd0 con 0x5628067d1490 2023-11-26T07:12:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3391888648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628067c6a40 con 0x5628067d1490 2023-11-26T07:12:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7fc57fa700 1 -- 172.21.15.83:0/3391888648 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fdc025460 con 0x5628067d1490 2023-11-26T07:12:29.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7ff8f8ab80 10 client.5259 fetch_fsmap learned FSMap version 17 2023-11-26T07:12:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3391888648 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x562806550d90 con 0x5628067d1490 2023-11-26T07:12:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7fc57fa700 1 -- 172.21.15.83:0/3391888648 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5628067c6a40 con 0x5628067d1490 2023-11-26T07:12:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7fc57fa700 1 -- 172.21.15.83:0/3391888648 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7fdc0255c0 con 0x5628067d1490 2023-11-26T07:12:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7ff8f8ab80 10 client.5259 fetch_fsmap finished waiting for FSMap version 17 2023-11-26T07:12:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7ff8f8ab80 10 client.5259 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-26T07:12:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7ff8f8ab80 20 client.5259 populate_metadata read hostname 'smithi083' 2023-11-26T07:12:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:6833/995330476,v1:172.21.15.203:6835/995330476] conn(0x562806801a20 0x562806800060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7ff8f8ab80 4 client.5259 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:12:29.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3391888648 --> [v2:172.21.15.203:6833/995330476,v1:172.21.15.203:6835/995330476] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628067a2840 con 0x562806801a20 2023-11-26T07:12:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.371+0000 7f7fe5d9b700 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:6833/995330476,v1:172.21.15.203:6835/995330476] conn(0x562806801a20 0x562806800060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.372+0000 7f7fc57fa700 10 client.5259 ms_handle_connect on v2:172.21.15.203:6833/995330476 2023-11-26T07:12:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.372+0000 7f7fe5d9b700 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:6833/995330476,v1:172.21.15.203:6835/995330476] conn(0x562806801a20 0x562806800060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35902c9242b1261 server_cookie=b0736f749369975c in_seq=0 out_seq=0 2023-11-26T07:12:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.372+0000 7f7fc57fa700 1 -- 172.21.15.83:0/3391888648 <== mds.0 v2:172.21.15.203:6833/995330476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5628067a2840 con 0x562806801a20 2023-11-26T07:12:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.372+0000 7f7fc57fa700 10 client.5259 handle_command_reply: tid=0 2023-11-26T07:12:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.372+0000 7f7ff8f8ab80 1 client.5259 shutdown 2023-11-26T07:12:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3391888648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d1490 msgr2=0x5628067c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d1490 0x5628067c8330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc000c00 tx=0x7f7fdc0047c0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3391888648 shutdown_connections 2023-11-26T07:12:29.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:6833/995330476,v1:172.21.15.203:6835/995330476] conn(0x562806801a20 0x562806800060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x562806814610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d1490 0x5628067c8330 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7fc57fa700 0 client.5259 ms_handle_reset on v2:172.21.15.203:6833/995330476 2023-11-26T07:12:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/3391888648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d4080 0x5628067c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3391888648 >> 172.21.15.83:0/3391888648 conn(0x562806702310 msgr2=0x56280681cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3391888648 shutdown_connections 2023-11-26T07:12:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3391888648 wait complete. 2023-11-26T07:12:29.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.373+0000 7f7ff8f8ab80 20 client.5259 trim_cache size 0 max 16384 2023-11-26T07:12:29.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.391+0000 7f7ff8f8ab80 1 Processor -- start 2023-11-26T07:12:29.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.391+0000 7f7ff8f8ab80 1 -- start start 2023-11-26T07:12:29.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067d1490 0x562806827570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d4080 0x562806827ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067e9ed0 0x56280680e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628065a68d0 con 0x5628067e9ed0 2023-11-26T07:12:29.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628067d3cf0 con 0x5628067d1490 2023-11-26T07:12:29.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56280682b700 con 0x5628067d4080 2023-11-26T07:12:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067e9ed0 0x56280680e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067e9ed0 0x56280680e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36202/0 (socket says 172.21.15.83:36202) 2023-11-26T07:12:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe6d9d700 1 -- 172.21.15.83:0/3076223932 learned_addr learned my addr 172.21.15.83:0/3076223932 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe6d9d700 1 -- 172.21.15.83:0/3076223932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d4080 msgr2=0x562806827ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe5d9b700 1 --2- 172.21.15.83:0/3076223932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d4080 0x562806827ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe659c700 1 --2- 172.21.15.83:0/3076223932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067d1490 0x562806827570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe6d9d700 1 --2- 172.21.15.83:0/3076223932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d4080 0x562806827ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe6d9d700 1 -- 172.21.15.83:0/3076223932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067d1490 msgr2=0x562806827570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe6d9d700 1 --2- 172.21.15.83:0/3076223932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067d1490 0x562806827570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe6d9d700 1 -- 172.21.15.83:0/3076223932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562806539c10 con 0x5628067e9ed0 2023-11-26T07:12:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe5d9b700 1 --2- 172.21.15.83:0/3076223932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d4080 0x562806827ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe659c700 1 --2- 172.21.15.83:0/3076223932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067d1490 0x562806827570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fe6d9d700 1 --2- 172.21.15.83:0/3076223932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067e9ed0 0x56280680e800 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000ce00 tx=0x7f7fe0058050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d8d4349d4488a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fd4ff9700 1 -- 172.21.15.83:0/3076223932 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe006f1c0 con 0x5628067e9ed0 2023-11-26T07:12:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fd4ff9700 1 -- 172.21.15.83:0/3076223932 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe005ba20 con 0x5628067e9ed0 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.392+0000 7f7fd4ff9700 1 -- 172.21.15.83:0/3076223932 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe005fbd0 con 0x5628067e9ed0 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.393+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3076223932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067e9ed0 msgr2=0x56280680e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.393+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/3076223932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067e9ed0 0x56280680e800 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000ce00 tx=0x7f7fe0058050 comp rx=0 tx=0).stop 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.393+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3076223932 shutdown_connections 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.393+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/3076223932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067e9ed0 0x56280680e800 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.393+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/3076223932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067d1490 0x562806827570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.393+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/3076223932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d4080 0x562806827ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.393+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3076223932 >> 172.21.15.83:0/3076223932 conn(0x562806702310 msgr2=0x5628067cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.393+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3076223932 shutdown_connections 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.393+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/3076223932 wait complete. 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.393+0000 7f7ff8f8ab80 1 -- start start 2023-11-26T07:12:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.393+0000 7f7ff8f8ab80 1 Processor -- start 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.394+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d1490 0x5628068814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.394+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x5628067e83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.394+0000 7f7ff8f8ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x5628067e8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.394+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562806881a20 con 0x5628067d4080 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.394+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562806886090 con 0x5628067e9ed0 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.394+0000 7f7ff8f8ab80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628068178e0 con 0x5628067d1490 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.394+0000 7f7ff8f8ab80 10 client.0 _ll_register_callbacks cb 0x56280660ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.394+0000 7f7fe5d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x5628067e83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.394+0000 7f7fe5d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x5628067e83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36206/0 (socket says 172.21.15.83:36206) 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.394+0000 7f7fe5d9b700 1 -- 172.21.15.83:0/2037644487 learned_addr learned my addr 172.21.15.83:0/2037644487 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fe659c700 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d1490 0x5628068814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fe6d9d700 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x5628067e8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fe5d9b700 1 -- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d1490 msgr2=0x5628068814e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fe5d9b700 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d1490 0x5628068814e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fe5d9b700 1 -- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 msgr2=0x5628067e8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fe5d9b700 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x5628067e8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fe5d9b700 1 -- 172.21.15.83:0/2037644487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe8143620 con 0x5628067d4080 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fe5d9b700 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x5628067e83e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7fe806eef0 tx=0x7f7fd8004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992a0201452dc196 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fc57fa700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fc57fa700 1 -- 172.21.15.83:0/2037644487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd801a040 con 0x5628067d4080 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fc57fa700 1 -- 172.21.15.83:0/2037644487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fd801bd00 con 0x5628067d4080 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2037644487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fdc0036a0 con 0x5628067d4080 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fc57fa700 1 -- 172.21.15.83:0/2037644487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fd8003ab0 con 0x5628067d4080 2023-11-26T07:12:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7ff8f8ab80 10 client.5265 fetch_fsmap learned FSMap version 17 2023-11-26T07:12:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2037644487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x562806577200 con 0x5628067d4080 2023-11-26T07:12:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fc57fa700 1 -- 172.21.15.83:0/2037644487 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc0036a0 con 0x5628067d4080 2023-11-26T07:12:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7fc57fa700 1 -- 172.21.15.83:0/2037644487 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7fd8003c10 con 0x5628067d4080 2023-11-26T07:12:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7ff8f8ab80 10 client.5265 fetch_fsmap finished waiting for FSMap version 17 2023-11-26T07:12:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7ff8f8ab80 10 client.5265 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-26T07:12:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7ff8f8ab80 20 client.5265 populate_metadata read hostname 'smithi083' 2023-11-26T07:12:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:6833/995330476,v1:172.21.15.203:6835/995330476] conn(0x562806885400 0x5628068857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7ff8f8ab80 4 client.5265 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:12:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.395+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2037644487 --> [v2:172.21.15.203:6833/995330476,v1:172.21.15.203:6835/995330476] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5628066edf80 con 0x562806885400 2023-11-26T07:12:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7fe659c700 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:6833/995330476,v1:172.21.15.203:6835/995330476] conn(0x562806885400 0x5628068857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7fc57fa700 10 client.5265 ms_handle_connect on v2:172.21.15.203:6833/995330476 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7fe659c700 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:6833/995330476,v1:172.21.15.203:6835/995330476] conn(0x562806885400 0x5628068857e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b867947f4ef0f5b3 server_cookie=4535ba0cffbdbc59 in_seq=0 out_seq=0 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7fc57fa700 1 -- 172.21.15.83:0/2037644487 <== mds.0 v2:172.21.15.203:6833/995330476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x5628066edf80 con 0x562806885400 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7fc57fa700 10 client.5265 handle_command_reply: tid=0 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7ff8f8ab80 1 client.5265 shutdown 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2037644487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 msgr2=0x5628067e83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x5628067e83e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7fe806eef0 tx=0x7f7fd8004320 comp rx=0 tx=0).stop 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2037644487 shutdown_connections 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:6833/995330476,v1:172.21.15.203:6835/995330476] conn(0x562806885400 0x5628068857e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628067e9ed0 0x5628067e8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5628067d1490 0x5628068814e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7fc57fa700 0 client.5265 ms_handle_reset on v2:172.21.15.203:6833/995330476 2023-11-26T07:12:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7ff8f8ab80 1 --2- 172.21.15.83:0/2037644487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628067d4080 0x5628067e83e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2037644487 >> 172.21.15.83:0/2037644487 conn(0x562806702310 msgr2=0x562806815f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2037644487 shutdown_connections 2023-11-26T07:12:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.396+0000 7f7ff8f8ab80 1 -- 172.21.15.83:0/2037644487 wait complete. 2023-11-26T07:12:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7ff8f8ab80 20 client.5265 trim_cache size 0 max 16384 2023-11-26T07:12:29.408 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:12:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "0aae1049-cc8c-4011-a01f-378673e57a96", 2023-11-26T07:12:29.408 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-26T07:12:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5100, 2023-11-26T07:12:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-26T07:12:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-26T07:12:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-26T07:12:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.8948132739999997, 2023-11-26T07:12:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 17, 2023-11-26T07:12:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 34, 2023-11-26T07:12:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 34, 2023-11-26T07:12:29.409 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 14.2629506 2023-11-26T07:12:29.409 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:12:29.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 -- 172.21.15.83:0/1539862280 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7fc806aed0 msgr2=0x7f7fc806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7fc806aed0 0x7f7fc806d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009870 tx=0x7f7fdc008030 comp rx=0 tx=0).stop 2023-11-26T07:12:29.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 -- 172.21.15.83:0/1539862280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe81446a0 msgr2=0x7f7fe813a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe81446a0 0x7f7fe813a010 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000c7a0 tx=0x7f7fe000cab0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 -- 172.21.15.83:0/1539862280 shutdown_connections 2023-11-26T07:12:29.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fe81446a0 0x7f7fe813a010 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7fc806aed0 0x7f7fc806d390 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7fe806c540 0x7f7fe81430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 --2- 172.21.15.83:0/1539862280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fe80adf00 0x7f7fe813f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 -- 172.21.15.83:0/1539862280 >> 172.21.15.83:0/1539862280 conn(0x7f7fe8071b90 msgr2=0x7f7fe806bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:29.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 -- 172.21.15.83:0/1539862280 shutdown_connections 2023-11-26T07:12:29.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.397+0000 7f7fecd91700 1 -- 172.21.15.83:0/1539862280 wait complete. 2023-11-26T07:12:29.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:12:29.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.530+0000 7f3c9873e700 1 Processor -- start 2023-11-26T07:12:29.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.530+0000 7f3c9873e700 1 -- start start 2023-11-26T07:12:29.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.531+0000 7f3c9873e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c901446a0 0x7f3c900b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.531+0000 7f3c9873e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c9006c710 0x7f3c900b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.531+0000 7f3c9873e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c900af950 con 0x7f3c900adf00 2023-11-26T07:12:29.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.531+0000 7f3c9873e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c90144a80 con 0x7f3c9006c710 2023-11-26T07:12:29.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.531+0000 7f3c9873e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900afc30 con 0x7f3c901446a0 2023-11-26T07:12:29.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.531+0000 7f3c96cdb700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c9006c710 0x7f3c900b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.531+0000 7f3c964da700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c901446a0 0x7f3c900b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c96cdb700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c9006c710 0x7f3c900b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:35668/0 (socket says 172.21.15.83:35668) 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c964da700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c901446a0 0x7f3c900b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:46670/0 (socket says 172.21.15.83:46670) 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c95cd9700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3c900adf00 0x7f3c9006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:40872/0 (socket says 172.21.15.83:40872) 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c96cdb700 1 -- 172.21.15.83:0/2520224829 learned_addr learned my addr 172.21.15.83:0/2520224829 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c964da700 1 -- 172.21.15.83:0/2520224829 >> v2:172.21.15.203:3300/0 conn(0x7f3c9006c710 msgr2=0x7f3c900b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c964da700 1 --2- 172.21.15.83:0/2520224829 >> v2:172.21.15.203:3300/0 conn(0x7f3c9006c710 0x7f3c900b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c964da700 1 -- 172.21.15.83:0/2520224829 >> v1:172.21.15.203:6789/0 conn(0x7f3c900adf00 legacy=0x7f3c9006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c964da700 1 -- 172.21.15.83:0/2520224829 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c900aff10 con 0x7f3c901446a0 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c964da700 1 --2- 172.21.15.83:0/2520224829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c901446a0 0x7f3c900b5ae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009840 tx=0x7f3c80009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad739488486d466b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c954d8700 1 -- 172.21.15.83:0/2520224829 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c80011830 con 0x7f3c901446a0 2023-11-26T07:12:29.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c954d8700 1 -- 172.21.15.83:0/2520224829 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c80011990 con 0x7f3c901446a0 2023-11-26T07:12:29.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.532+0000 7f3c954d8700 1 -- 172.21.15.83:0/2520224829 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c80011d20 con 0x7f3c901446a0 2023-11-26T07:12:29.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.533+0000 7f3c9873e700 1 -- 172.21.15.83:0/2520224829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c901446a0 msgr2=0x7f3c900b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.533+0000 7f3c9873e700 1 --2- 172.21.15.83:0/2520224829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c901446a0 0x7f3c900b5ae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009840 tx=0x7f3c80009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:29.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.533+0000 7f3c9873e700 1 -- 172.21.15.83:0/2520224829 shutdown_connections 2023-11-26T07:12:29.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.533+0000 7f3c9873e700 1 --2- 172.21.15.83:0/2520224829 >> v2:172.21.15.203:3300/0 conn(0x7f3c9006c710 0x7f3c900b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.533+0000 7f3c9873e700 1 --2- 172.21.15.83:0/2520224829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c901446a0 0x7f3c900b5ae0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.533+0000 7f3c9873e700 1 -- 172.21.15.83:0/2520224829 >> 172.21.15.83:0/2520224829 conn(0x7f3c90071b90 msgr2=0x7f3c90071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:29.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.533+0000 7f3c9873e700 1 -- 172.21.15.83:0/2520224829 shutdown_connections 2023-11-26T07:12:29.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.533+0000 7f3c9873e700 1 -- 172.21.15.83:0/2520224829 wait complete. 2023-11-26T07:12:29.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.534+0000 7f3c9873e700 1 Processor -- start 2023-11-26T07:12:29.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.534+0000 7f3c9873e700 1 -- start start 2023-11-26T07:12:29.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c9873e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c9006c710 0x7f3c9014f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c9873e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c900adf00 0x7f3c9014c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c9873e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c901446a0 0x7f3c9014c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c9873e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9014ce90 con 0x7f3c9006c710 2023-11-26T07:12:29.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c9873e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9015dcb0 con 0x7f3c901446a0 2023-11-26T07:12:29.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c9873e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9014d460 con 0x7f3c900adf00 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c964da700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c9006c710 0x7f3c9014f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c964da700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c9006c710 0x7f3c9014f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36218/0 (socket says 172.21.15.83:36218) 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c964da700 1 -- 172.21.15.83:0/1744568506 learned_addr learned my addr 172.21.15.83:0/1744568506 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c95cd9700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c900adf00 0x7f3c9014c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c96cdb700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c901446a0 0x7f3c9014c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c964da700 1 -- 172.21.15.83:0/1744568506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c900adf00 msgr2=0x7f3c9014c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c964da700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c900adf00 0x7f3c9014c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c964da700 1 -- 172.21.15.83:0/1744568506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c901446a0 msgr2=0x7f3c9014c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c964da700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c901446a0 0x7f3c9014c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c964da700 1 -- 172.21.15.83:0/1744568506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c800094a0 con 0x7f3c9006c710 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c96cdb700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c901446a0 0x7f3c9014c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:29.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c964da700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c9006c710 0x7f3c9014f750 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009fd0 tx=0x7f3c80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:29.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c86ffd700 1 -- 172.21.15.83:0/1744568506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c800043c0 con 0x7f3c9006c710 2023-11-26T07:12:29.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.535+0000 7f3c86ffd700 1 -- 172.21.15.83:0/1744568506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c80004540 con 0x7f3c9006c710 2023-11-26T07:12:29.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.536+0000 7f3c9873e700 1 -- 172.21.15.83:0/1744568506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9014d740 con 0x7f3c9006c710 2023-11-26T07:12:29.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.536+0000 7f3c86ffd700 1 -- 172.21.15.83:0/1744568506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c800038e0 con 0x7f3c9006c710 2023-11-26T07:12:29.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.536+0000 7f3c9873e700 1 -- 172.21.15.83:0/1744568506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9014dd00 con 0x7f3c9006c710 2023-11-26T07:12:29.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.537+0000 7f3c9873e700 1 -- 172.21.15.83:0/1744568506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c900753f0 con 0x7f3c9006c710 2023-11-26T07:12:29.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.537+0000 7f3c86ffd700 1 -- 172.21.15.83:0/1744568506 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c80023070 con 0x7f3c9006c710 2023-11-26T07:12:29.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.537+0000 7f3c86ffd700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c7406aed0 0x7f3c7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:29.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.537+0000 7f3c86ffd700 1 -- 172.21.15.83:0/1744568506 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c8008c460 con 0x7f3c9006c710 2023-11-26T07:12:29.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.537+0000 7f3c95cd9700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c7406aed0 0x7f3c7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:29.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.538+0000 7f3c95cd9700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c7406aed0 0x7f3c7406d390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c900b5080 tx=0x7f3c8c007f10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:29.544 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.542+0000 7f3c86ffd700 1 -- 172.21.15.83:0/1744568506 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c80057500 con 0x7f3c9006c710 2023-11-26T07:12:29.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.779+0000 7f3c9873e700 1 -- 172.21.15.83:0/1744568506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c9014de90 con 0x7f3c9006c710 2023-11-26T07:12:29.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.780+0000 7f3c86ffd700 1 -- 172.21.15.83:0/1744568506 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3c8005ab20 con 0x7f3c9006c710 2023-11-26T07:12:29.782 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:12:29.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 -- 172.21.15.83:0/1744568506 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c7406aed0 msgr2=0x7f3c7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c7406aed0 0x7f3c7406d390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c900b5080 tx=0x7f3c8c007f10 comp rx=0 tx=0).stop 2023-11-26T07:12:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 -- 172.21.15.83:0/1744568506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c9006c710 msgr2=0x7f3c9014f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c9006c710 0x7f3c9014f750 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009fd0 tx=0x7f3c80004750 comp rx=0 tx=0).stop 2023-11-26T07:12:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 -- 172.21.15.83:0/1744568506 shutdown_connections 2023-11-26T07:12:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c901446a0 0x7f3c9014c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c9006c710 0x7f3c9014f750 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c7406aed0 0x7f3c7406d390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 --2- 172.21.15.83:0/1744568506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c900adf00 0x7f3c9014c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 -- 172.21.15.83:0/1744568506 >> 172.21.15.83:0/1744568506 conn(0x7f3c90071b90 msgr2=0x7f3c900b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 -- 172.21.15.83:0/1744568506 shutdown_connections 2023-11-26T07:12:29.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:29.782+0000 7f3c9873e700 1 -- 172.21.15.83:0/1744568506 wait complete. 2023-11-26T07:12:29.796 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:12:29.796 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:12:29.796 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-26T07:12:29.796 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:12:29.796 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:12:29.796 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:12:29.797 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:12:29.797 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:12:29.797 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:12:29.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:29.797 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-26T07:12:29.814 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:12:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:12:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:12:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-26T07:12:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:29.814 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:12:29.814 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42336sec preferred_lft 42336sec 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::3a41:fab:adc5:5d6c/64 scope link noprefixroute 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:12:29.815 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::54b0:6eff:fe33:bb15/64 scope link 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether fe:46:b0:72:7b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::fc46:b0ff:fe72:7b3f/64 scope link 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::7808:fdff:fe1b:770c/64 scope link 2023-11-26T07:12:29.816 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:29.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:29.817 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:12:29.879 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-26T07:12:29.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:29.881 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-26T07:12:29.939 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-26T07:12:29.940 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:12:29.940 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:12:29.940 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:12:29.940 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:12:29.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:29.940 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:12:29.940 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:12:29.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:12:29.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:12:29.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:12:29.941 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:12:30.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:30.044 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:12:30.060 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:12:30.060 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:12:30.060 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:12:30.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:30.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-26T07:12:30.127 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:12:30.128 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:12:30.183 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:12:30.184 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:12:30.184 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:12:30.240 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:12:30.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:30.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:12:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:12:30.327 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:12:30.350 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:12:30.350 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:12:30.351 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:12:30.351 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:12:30.351 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:12:30.351 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:12:30.351 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:12:30.351 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:12:30.378 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:12:30.378 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:12:30.379 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:12:30.379 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:12:30.379 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:12:30.379 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:12:30.379 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:12:30.379 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:12:30.379 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:12:30.379 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:12:30.379 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:12:30.382 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:12:30.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:30.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:12:30.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:30.417 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:12:30.505 INFO:teuthology.orchestra.run.smithi083.stdout:client5271 2023-11-26T07:12:30.506 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:30.506 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:30.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:12:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.680+0000 7fec34412700 1 Processor -- start 2023-11-26T07:12:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.680+0000 7fec34412700 1 -- start start 2023-11-26T07:12:30.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.681+0000 7fec34412700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2c15bed0 0x7fec2c0b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:30.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.681+0000 7fec34412700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fec2c1603c0 0x7fec2c0d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:30.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.681+0000 7fec34412700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2c0d0770 con 0x7fec2c0d1a50 2023-11-26T07:12:30.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.681+0000 7fec34412700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c15c2b0 con 0x7fec2c1603c0 2023-11-26T07:12:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.681+0000 7fec34412700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0cd100 con 0x7fec2c15bed0 2023-11-26T07:12:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.681+0000 7fec319ad700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fec2c1603c0 0x7fec2c0d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.681+0000 7fec329af700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fec2c0d1a50 0x7fec2c0d1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56274/0 (socket says 172.21.15.83:56274) 2023-11-26T07:12:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec329af700 1 -- 172.21.15.83:0/601809188 learned_addr learned my addr 172.21.15.83:0/601809188 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec319ad700 1 -- 172.21.15.83:0/601809188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2c15bed0 msgr2=0x7fec2c0b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec321ae700 1 --2- 172.21.15.83:0/601809188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2c15bed0 0x7fec2c0b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec311ac700 1 -- 172.21.15.83:0/601809188 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265896568 0 0) 0x7fec2c0d0770 con 0x7fec2c0d1a50 2023-11-26T07:12:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec319ad700 1 --2- 172.21.15.83:0/601809188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2c15bed0 0x7fec2c0b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec319ad700 1 -- 172.21.15.83:0/601809188 >> v1:172.21.15.83:6789/0 conn(0x7fec2c0d1a50 legacy=0x7fec2c0d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec319ad700 1 -- 172.21.15.83:0/601809188 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c0d09a0 con 0x7fec2c1603c0 2023-11-26T07:12:30.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec321ae700 1 --2- 172.21.15.83:0/601809188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2c15bed0 0x7fec2c0b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec319ad700 1 --2- 172.21.15.83:0/601809188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2c1603c0 0x7fec2c0d0230 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fec28009840 tx=0x7fec28009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3608a84e0e1f64 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec311ac700 1 -- 172.21.15.83:0/601809188 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28011830 con 0x7fec2c1603c0 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec311ac700 1 -- 172.21.15.83:0/601809188 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec28011990 con 0x7fec2c1603c0 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.682+0000 7fec311ac700 1 -- 172.21.15.83:0/601809188 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28011d20 con 0x7fec2c1603c0 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.683+0000 7fec34412700 1 -- 172.21.15.83:0/601809188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2c1603c0 msgr2=0x7fec2c0d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.683+0000 7fec34412700 1 --2- 172.21.15.83:0/601809188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2c1603c0 0x7fec2c0d0230 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fec28009840 tx=0x7fec28009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.683+0000 7fec34412700 1 -- 172.21.15.83:0/601809188 shutdown_connections 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.683+0000 7fec34412700 1 --2- 172.21.15.83:0/601809188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2c15bed0 0x7fec2c0b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.683+0000 7fec34412700 1 --2- 172.21.15.83:0/601809188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2c1603c0 0x7fec2c0d0230 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.683+0000 7fec34412700 1 -- 172.21.15.83:0/601809188 >> 172.21.15.83:0/601809188 conn(0x7fec2c073130 msgr2=0x7fec2c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.683+0000 7fec34412700 1 -- 172.21.15.83:0/601809188 shutdown_connections 2023-11-26T07:12:30.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.683+0000 7fec34412700 1 -- 172.21.15.83:0/601809188 wait complete. 2023-11-26T07:12:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.684+0000 7fec34412700 1 Processor -- start 2023-11-26T07:12:30.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.684+0000 7fec34412700 1 -- start start 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec34412700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2c0d1a50 0x7fec2c14ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec34412700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2c15bed0 0x7fec2c09d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec34412700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec2c1603c0 0x7fec2c0c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec34412700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c09f2f0 con 0x7fec2c15bed0 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec34412700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c09e250 con 0x7fec2c1603c0 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec34412700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c09e530 con 0x7fec2c0d1a50 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec319ad700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2c15bed0 0x7fec2c09d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec319ad700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2c15bed0 0x7fec2c09d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36238/0 (socket says 172.21.15.83:36238) 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec319ad700 1 -- 172.21.15.83:0/3908842907 learned_addr learned my addr 172.21.15.83:0/3908842907 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec321ae700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2c0d1a50 0x7fec2c14ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec329af700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec2c1603c0 0x7fec2c0c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec319ad700 1 -- 172.21.15.83:0/3908842907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2c0d1a50 msgr2=0x7fec2c14ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:30.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec319ad700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2c0d1a50 0x7fec2c14ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec319ad700 1 -- 172.21.15.83:0/3908842907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec2c1603c0 msgr2=0x7fec2c0c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec319ad700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec2c1603c0 0x7fec2c0c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec319ad700 1 -- 172.21.15.83:0/3908842907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec280094a0 con 0x7fec2c15bed0 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec329af700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec2c1603c0 0x7fec2c0c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec319ad700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2c15bed0 0x7fec2c09d790 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fec280090b0 tx=0x7fec28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec22ffd700 1 -- 172.21.15.83:0/3908842907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec280036a0 con 0x7fec2c15bed0 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec22ffd700 1 -- 172.21.15.83:0/3908842907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec28003800 con 0x7fec2c15bed0 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec22ffd700 1 -- 172.21.15.83:0/3908842907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28003bb0 con 0x7fec2c15bed0 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.685+0000 7fec34412700 1 -- 172.21.15.83:0/3908842907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2c09e810 con 0x7fec2c15bed0 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.686+0000 7fec34412700 1 -- 172.21.15.83:0/3908842907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c09edd0 con 0x7fec2c15bed0 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.686+0000 7fec34412700 1 -- 172.21.15.83:0/3908842907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2c067f60 con 0x7fec2c15bed0 2023-11-26T07:12:30.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.687+0000 7fec22ffd700 1 -- 172.21.15.83:0/3908842907 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec28003d10 con 0x7fec2c15bed0 2023-11-26T07:12:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.687+0000 7fec22ffd700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1006aed0 0x7fec1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.687+0000 7fec22ffd700 1 -- 172.21.15.83:0/3908842907 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec2808aca0 con 0x7fec2c15bed0 2023-11-26T07:12:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.687+0000 7fec321ae700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1006aed0 0x7fec1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:30.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.688+0000 7fec321ae700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1006aed0 0x7fec1006d390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec1c006fd0 tx=0x7fec1c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:30.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.692+0000 7fec22ffd700 1 -- 172.21.15.83:0/3908842907 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec28055dd0 con 0x7fec2c15bed0 2023-11-26T07:12:30.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.926+0000 7fec34412700 1 -- 172.21.15.83:0/3908842907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fec2c0b7bd0 con 0x7fec2c15bed0 2023-11-26T07:12:30.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.926+0000 7fec22ffd700 1 -- 172.21.15.83:0/3908842907 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fec28015020 con 0x7fec2c15bed0 2023-11-26T07:12:30.929 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:12:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.928+0000 7fec34412700 1 -- 172.21.15.83:0/3908842907 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1006aed0 msgr2=0x7fec1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.928+0000 7fec34412700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1006aed0 0x7fec1006d390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec1c006fd0 tx=0x7fec1c008040 comp rx=0 tx=0).stop 2023-11-26T07:12:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.928+0000 7fec34412700 1 -- 172.21.15.83:0/3908842907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2c15bed0 msgr2=0x7fec2c09d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.928+0000 7fec34412700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2c15bed0 0x7fec2c09d790 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fec280090b0 tx=0x7fec28004750 comp rx=0 tx=0).stop 2023-11-26T07:12:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.928+0000 7fec34412700 1 -- 172.21.15.83:0/3908842907 shutdown_connections 2023-11-26T07:12:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.929+0000 7fec34412700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec2c1603c0 0x7fec2c0c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.929+0000 7fec34412700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1006aed0 0x7fec1006d390 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.929+0000 7fec34412700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2c0d1a50 0x7fec2c14ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.929+0000 7fec34412700 1 --2- 172.21.15.83:0/3908842907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2c15bed0 0x7fec2c09d790 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.929+0000 7fec34412700 1 -- 172.21.15.83:0/3908842907 >> 172.21.15.83:0/3908842907 conn(0x7fec2c073130 msgr2=0x7fec2c15f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.929+0000 7fec34412700 1 -- 172.21.15.83:0/3908842907 shutdown_connections 2023-11-26T07:12:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:30.929+0000 7fec34412700 1 -- 172.21.15.83:0/3908842907 wait complete. 2023-11-26T07:12:30.943 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:12:30.943 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:12:30.943 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2023-11-26T07:12:30.943 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:12:30.943 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:12:30.943 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:12:30.943 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:12:30.943 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:12:30.943 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:12:30.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:30.944 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2023-11-26T07:12:30.965 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:12:30.965 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:12:30.965 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:12:30.965 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:30.965 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-11-26T07:12:30.965 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:30.966 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:12:30.966 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:12:30.966 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:12:30.966 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:12:30.966 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:12:30.966 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:12:30.966 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:12:30.966 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:12:30.966 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:12:30.966 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42338sec preferred_lft 42338sec 2023-11-26T07:12:30.966 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::64b0:e04:3aa8:a78c/64 scope link noprefixroute 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1092:a6ff:fe63:3da7/64 scope link 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::b88a:cff:fe60:2b6f/64 scope link 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:30.967 INFO:teuthology.orchestra.run.smithi203.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:12:30.968 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether de:b3:89:ce:09:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:12:30.968 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::dcb3:89ff:fece:9cd/64 scope link 2023-11-26T07:12:30.968 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:12:30.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:30.968 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:12:31.034 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-11-26T07:12:31.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:31.036 DEBUG:teuthology.orchestra.run.smithi203:> route 2023-11-26T07:12:31.098 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2023-11-26T07:12:31.098 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:12:31.099 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:12:31.099 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:12:31.099 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:12:31.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:31.100 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:12:31.100 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:12:31.100 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:12:31.100 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:12:31.100 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:12:31.100 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:12:31.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:31.205 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:12:31.221 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:12:31.221 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:12:31.221 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:12:31.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:31.221 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2023-11-26T07:12:31.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:12:31.288 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:12:31.345 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:12:31.345 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:12:31.345 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:12:31.401 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:12:31.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:31.402 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:12:31.486 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:12:31.487 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:12:31.510 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:12:31.511 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:12:31.511 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:12:31.511 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:12:31.511 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:12:31.511 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:12:31.511 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:12:31.511 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:12:31.536 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:12:31.536 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:12:31.537 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:12:31.537 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:12:31.537 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:12:31.537 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:12:31.537 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:12:31.537 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:12:31.537 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:12:31.537 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:12:31.537 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:12:31.540 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:12:31.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:31.540 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:12:31.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:31.570 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:12:31.642 INFO:teuthology.orchestra.run.smithi203.stdout:client4742 2023-11-26T07:12:31.643 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:31.643 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:31.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:12:31.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.827+0000 7f2ea7107700 1 Processor -- start 2023-11-26T07:12:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.827+0000 7f2ea7107700 1 -- start start 2023-11-26T07:12:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea7107700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ea00b01e0 0x7f2ea006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea7107700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ea00746a0 0x7f2ea00b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea7107700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea00b05c0 con 0x7f2ea00b59c0 2023-11-26T07:12:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea7107700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea01470a0 con 0x7f2ea00b01e0 2023-11-26T07:12:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea7107700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea00b5e70 con 0x7f2ea00746a0 2023-11-26T07:12:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea4ea3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ea00b01e0 0x7f2ea006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2e9ffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2ea00b59c0 0x7f2ea0073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56284/0 (socket says 172.21.15.83:56284) 2023-11-26T07:12:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2e9ffff700 1 -- 172.21.15.83:0/216923831 learned_addr learned my addr 172.21.15.83:0/216923831 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea4ea3700 1 -- 172.21.15.83:0/216923831 >> v2:172.21.15.203:3300/0 conn(0x7f2ea00746a0 msgr2=0x7f2ea00b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea56a4700 1 --2- 172.21.15.83:0/216923831 >> v2:172.21.15.203:3300/0 conn(0x7f2ea00746a0 0x7f2ea00b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2e9f7fe700 1 -- 172.21.15.83:0/216923831 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761476120 0 0) 0x7f2ea00b05c0 con 0x7f2ea00b59c0 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea4ea3700 1 --2- 172.21.15.83:0/216923831 >> v2:172.21.15.203:3300/0 conn(0x7f2ea00746a0 0x7f2ea00b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea4ea3700 1 -- 172.21.15.83:0/216923831 >> v1:172.21.15.83:6789/0 conn(0x7f2ea00b59c0 legacy=0x7f2ea0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea4ea3700 1 -- 172.21.15.83:0/216923831 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea00b4250 con 0x7f2ea00b01e0 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea56a4700 1 --2- 172.21.15.83:0/216923831 >> v2:172.21.15.203:3300/0 conn(0x7f2ea00746a0 0x7f2ea00b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.828+0000 7f2ea4ea3700 1 --2- 172.21.15.83:0/216923831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ea00b01e0 0x7f2ea006e9c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00d1970 tx=0x7f2e94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f37afbf2518a78 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.829+0000 7f2e9f7fe700 1 -- 172.21.15.83:0/216923831 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94011830 con 0x7f2ea00b01e0 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.829+0000 7f2e9f7fe700 1 -- 172.21.15.83:0/216923831 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e94011990 con 0x7f2ea00b01e0 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.829+0000 7f2e9f7fe700 1 -- 172.21.15.83:0/216923831 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94011d20 con 0x7f2ea00b01e0 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.829+0000 7f2ea7107700 1 -- 172.21.15.83:0/216923831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ea00b01e0 msgr2=0x7f2ea006e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.829+0000 7f2ea7107700 1 --2- 172.21.15.83:0/216923831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ea00b01e0 0x7f2ea006e9c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00d1970 tx=0x7f2e94009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.829+0000 7f2ea7107700 1 -- 172.21.15.83:0/216923831 shutdown_connections 2023-11-26T07:12:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.829+0000 7f2ea7107700 1 --2- 172.21.15.83:0/216923831 >> v2:172.21.15.203:3300/0 conn(0x7f2ea00746a0 0x7f2ea00b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.829+0000 7f2ea7107700 1 --2- 172.21.15.83:0/216923831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ea00b01e0 0x7f2ea006e9c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.829+0000 7f2ea7107700 1 -- 172.21.15.83:0/216923831 >> 172.21.15.83:0/216923831 conn(0x7f2ea00d0b40 msgr2=0x7f2ea00d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.829+0000 7f2ea7107700 1 -- 172.21.15.83:0/216923831 shutdown_connections 2023-11-26T07:12:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.829+0000 7f2ea7107700 1 -- 172.21.15.83:0/216923831 wait complete. 2023-11-26T07:12:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.831+0000 7f2ea7107700 1 Processor -- start 2023-11-26T07:12:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.831+0000 7f2ea7107700 1 -- start start 2023-11-26T07:12:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea7107700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ea00746a0 0x7f2ea01f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea7107700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ea00b01e0 0x7f2ea013f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea7107700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ea00b59c0 0x7f2ea013fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea7107700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea01401e0 con 0x7f2ea00746a0 2023-11-26T07:12:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea7107700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea01404c0 con 0x7f2ea00b59c0 2023-11-26T07:12:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea7107700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea01407a0 con 0x7f2ea00b01e0 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea4ea3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ea00746a0 0x7f2ea01f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea4ea3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ea00746a0 0x7f2ea01f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36250/0 (socket says 172.21.15.83:36250) 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea4ea3700 1 -- 172.21.15.83:0/3133029644 learned_addr learned my addr 172.21.15.83:0/3133029644 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea4ea3700 1 -- 172.21.15.83:0/3133029644 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ea00b01e0 msgr2=0x7f2ea013f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea4ea3700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ea00b01e0 0x7f2ea013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea4ea3700 1 -- 172.21.15.83:0/3133029644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ea00b59c0 msgr2=0x7f2ea013fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea4ea3700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ea00b59c0 0x7f2ea013fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea4ea3700 1 -- 172.21.15.83:0/3133029644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e940094a0 con 0x7f2ea00746a0 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2ea4ea3700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ea00746a0 0x7f2ea01f3260 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2e940090b0 tx=0x7f2e94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.832+0000 7f2e9d7fa700 1 -- 172.21.15.83:0/3133029644 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e940038a0 con 0x7f2ea00746a0 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.833+0000 7f2e9d7fa700 1 -- 172.21.15.83:0/3133029644 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9401f070 con 0x7f2ea00746a0 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.833+0000 7f2ea7107700 1 -- 172.21.15.83:0/3133029644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea0140a80 con 0x7f2ea00746a0 2023-11-26T07:12:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.833+0000 7f2e9d7fa700 1 -- 172.21.15.83:0/3133029644 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94003c40 con 0x7f2ea00746a0 2023-11-26T07:12:31.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.833+0000 7f2ea7107700 1 -- 172.21.15.83:0/3133029644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea0157080 con 0x7f2ea00746a0 2023-11-26T07:12:31.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.833+0000 7f2ea7107700 1 -- 172.21.15.83:0/3133029644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea0067f60 con 0x7f2ea00746a0 2023-11-26T07:12:31.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.834+0000 7f2e9d7fa700 1 -- 172.21.15.83:0/3133029644 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e94026480 con 0x7f2ea00746a0 2023-11-26T07:12:31.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.834+0000 7f2e9d7fa700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e8406aed0 0x7f2e8406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:31.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.834+0000 7f2e9ffff700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e8406aed0 0x7f2e8406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:31.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.835+0000 7f2e9d7fa700 1 -- 172.21.15.83:0/3133029644 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e94021070 con 0x7f2ea00746a0 2023-11-26T07:12:31.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.835+0000 7f2e9ffff700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e8406aed0 0x7f2e8406d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ea006ecd0 tx=0x7f2e8c0079f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:31.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:31.839+0000 7f2e9d7fa700 1 -- 172.21.15.83:0/3133029644 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e940560f0 con 0x7f2ea00746a0 2023-11-26T07:12:32.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.073+0000 7f2ea7107700 1 -- 172.21.15.83:0/3133029644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2ea006c980 con 0x7f2ea00746a0 2023-11-26T07:12:32.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.536+0000 7f2e9d7fa700 1 -- 172.21.15.83:0/3133029644 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7f2e94059710 con 0x7f2ea00746a0 2023-11-26T07:12:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.538+0000 7f2ea7107700 1 -- 172.21.15.83:0/3133029644 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e8406aed0 msgr2=0x7f2e8406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.538+0000 7f2ea7107700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e8406aed0 0x7f2e8406d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ea006ecd0 tx=0x7f2e8c0079f0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.538+0000 7f2ea7107700 1 -- 172.21.15.83:0/3133029644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ea00746a0 msgr2=0x7f2ea01f3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.538+0000 7f2ea7107700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ea00746a0 0x7f2ea01f3260 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2e940090b0 tx=0x7f2e94004750 comp rx=0 tx=0).stop 2023-11-26T07:12:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.538+0000 7f2ea7107700 1 -- 172.21.15.83:0/3133029644 shutdown_connections 2023-11-26T07:12:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.538+0000 7f2ea7107700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ea00b59c0 0x7f2ea013fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.539+0000 7f2ea7107700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ea00746a0 0x7f2ea01f3260 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.539+0000 7f2ea7107700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e8406aed0 0x7f2e8406d390 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.539+0000 7f2ea7107700 1 --2- 172.21.15.83:0/3133029644 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ea00b01e0 0x7f2ea013f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.539+0000 7f2ea7107700 1 -- 172.21.15.83:0/3133029644 >> 172.21.15.83:0/3133029644 conn(0x7f2ea00d0b40 msgr2=0x7f2ea0145060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.539+0000 7f2ea7107700 1 -- 172.21.15.83:0/3133029644 shutdown_connections 2023-11-26T07:12:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.539+0000 7f2ea7107700 1 -- 172.21.15.83:0/3133029644 wait complete. 2023-11-26T07:12:32.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:12:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.672+0000 7f7cae3d5700 1 Processor -- start 2023-11-26T07:12:32.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.672+0000 7f7cae3d5700 1 -- start start 2023-11-26T07:12:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7cae3d5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ca806c710 0x7f7ca80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7cae3d5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7ca80af4d0 0x7f7ca80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7cae3d5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca80ae430 con 0x7f7ca815e080 2023-11-26T07:12:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7cae3d5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80af0b0 con 0x7f7ca80af4d0 2023-11-26T07:12:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7cae3d5700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80b05a0 con 0x7f7ca806c710 2023-11-26T07:12:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7cac972700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7ca80af4d0 0x7f7ca80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7ca7fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7ca815e080 0x7f7ca806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56298/0 (socket says 172.21.15.83:56298) 2023-11-26T07:12:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7cac972700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7ca80af4d0 0x7f7ca80afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36262/0 (socket says 172.21.15.83:36262) 2023-11-26T07:12:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7ca7fff700 1 -- 172.21.15.83:0/634181364 learned_addr learned my addr 172.21.15.83:0/634181364 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7ca77fe700 1 --2- 172.21.15.83:0/634181364 >> v2:172.21.15.203:3300/0 conn(0x7f7ca806c710 0x7f7ca80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7cac972700 1 -- 172.21.15.83:0/634181364 >> v2:172.21.15.203:3300/0 conn(0x7f7ca806c710 msgr2=0x7f7ca80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7cac972700 1 --2- 172.21.15.83:0/634181364 >> v2:172.21.15.203:3300/0 conn(0x7f7ca806c710 0x7f7ca80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7cac972700 1 -- 172.21.15.83:0/634181364 >> v1:172.21.15.83:6789/0 conn(0x7f7ca815e080 legacy=0x7f7ca806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7cac972700 1 -- 172.21.15.83:0/634181364 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca80ae710 con 0x7f7ca80af4d0 2023-11-26T07:12:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.673+0000 7f7ca6ffd700 1 -- 172.21.15.83:0/634181364 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290841658 0 0) 0x7f7ca80ae430 con 0x7f7ca815e080 2023-11-26T07:12:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7cac972700 1 --2- 172.21.15.83:0/634181364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ca80af4d0 0x7f7ca80afb90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7ca806e4e0 tx=0x7f7ca000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb4271d501b3e6d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7ca6ffd700 1 -- 172.21.15.83:0/634181364 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000e070 con 0x7f7ca80af4d0 2023-11-26T07:12:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7ca6ffd700 1 -- 172.21.15.83:0/634181364 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca000bb90 con 0x7f7ca80af4d0 2023-11-26T07:12:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7ca6ffd700 1 -- 172.21.15.83:0/634181364 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0012430 con 0x7f7ca80af4d0 2023-11-26T07:12:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7cae3d5700 1 -- 172.21.15.83:0/634181364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ca80af4d0 msgr2=0x7f7ca80afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7cae3d5700 1 --2- 172.21.15.83:0/634181364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ca80af4d0 0x7f7ca80afb90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7ca806e4e0 tx=0x7f7ca000b330 comp rx=0 tx=0).stop 2023-11-26T07:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7cae3d5700 1 -- 172.21.15.83:0/634181364 shutdown_connections 2023-11-26T07:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7cae3d5700 1 --2- 172.21.15.83:0/634181364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ca80af4d0 0x7f7ca80afb90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7cae3d5700 1 --2- 172.21.15.83:0/634181364 >> v2:172.21.15.203:3300/0 conn(0x7f7ca806c710 0x7f7ca80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7cae3d5700 1 -- 172.21.15.83:0/634181364 >> 172.21.15.83:0/634181364 conn(0x7f7ca8071b90 msgr2=0x7f7ca8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7cae3d5700 1 -- 172.21.15.83:0/634181364 shutdown_connections 2023-11-26T07:12:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.674+0000 7f7cae3d5700 1 -- 172.21.15.83:0/634181364 wait complete. 2023-11-26T07:12:32.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.675+0000 7f7cae3d5700 1 Processor -- start 2023-11-26T07:12:32.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.676+0000 7f7cae3d5700 1 -- start start 2023-11-26T07:12:32.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.676+0000 7f7cae3d5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ca806c710 0x7f7ca8144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:32.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.676+0000 7f7cae3d5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ca80af4d0 0x7f7ca8160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.676+0000 7f7cae3d5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ca815e080 0x7f7ca8151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.676+0000 7f7cae3d5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8145280 con 0x7f7ca806c710 2023-11-26T07:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.676+0000 7f7cae3d5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca815fd40 con 0x7f7ca80af4d0 2023-11-26T07:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.676+0000 7f7cae3d5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8160020 con 0x7f7ca815e080 2023-11-26T07:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ca806c710 0x7f7ca8144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ca806c710 0x7f7ca8144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36264/0 (socket says 172.21.15.83:36264) 2023-11-26T07:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca7fff700 1 -- 172.21.15.83:0/781888065 learned_addr learned my addr 172.21.15.83:0/781888065 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca7fff700 1 -- 172.21.15.83:0/781888065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ca815e080 msgr2=0x7f7ca8151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca77fe700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ca80af4d0 0x7f7ca8160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca7fff700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ca815e080 0x7f7ca8151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca7fff700 1 -- 172.21.15.83:0/781888065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ca80af4d0 msgr2=0x7f7ca8160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca7fff700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ca80af4d0 0x7f7ca8160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca7fff700 1 -- 172.21.15.83:0/781888065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca0009a70 con 0x7f7ca806c710 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca77fe700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ca80af4d0 0x7f7ca8160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca7fff700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ca806c710 0x7f7ca8144d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00c550 tx=0x7f7c9c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca4ff9700 1 -- 172.21.15.83:0/781888065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c004c80 con 0x7f7ca806c710 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca4ff9700 1 -- 172.21.15.83:0/781888065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c004e00 con 0x7f7ca806c710 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7cae3d5700 1 -- 172.21.15.83:0/781888065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca8160300 con 0x7f7ca806c710 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7ca4ff9700 1 -- 172.21.15.83:0/781888065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c007310 con 0x7f7ca806c710 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.677+0000 7f7cae3d5700 1 -- 172.21.15.83:0/781888065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca81608c0 con 0x7f7ca806c710 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.678+0000 7f7cae3d5700 1 -- 172.21.15.83:0/781888065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca80753f0 con 0x7f7ca806c710 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.679+0000 7f7ca4ff9700 1 -- 172.21.15.83:0/781888065 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c9c007470 con 0x7f7ca806c710 2023-11-26T07:12:32.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.679+0000 7f7ca4ff9700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c9406aed0 0x7f7c9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:32.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.679+0000 7f7ca4ff9700 1 -- 172.21.15.83:0/781888065 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c9c0893d0 con 0x7f7ca806c710 2023-11-26T07:12:32.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.679+0000 7f7ca77fe700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c9406aed0 0x7f7c9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:32.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.679+0000 7f7ca77fe700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c9406aed0 0x7f7c9406d390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ca815c020 tx=0x7f7c98008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:32.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.683+0000 7f7ca4ff9700 1 -- 172.21.15.83:0/781888065 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c9c0543f0 con 0x7f7ca806c710 2023-11-26T07:12:32.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.905+0000 7f7cae3d5700 1 -- 172.21.15.83:0/781888065 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ca80aecd0 con 0x7f7c9406aed0 2023-11-26T07:12:32.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.906+0000 7f7ca4ff9700 1 -- 172.21.15.83:0/781888065 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f7c98009610 con 0x7f7c9406aed0 2023-11-26T07:12:32.909 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:12:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephfs" 2023-11-26T07:12:32.909 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:32.909 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:12:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.908+0000 7f7cae3d5700 1 -- 172.21.15.83:0/781888065 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c9406aed0 msgr2=0x7f7c9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.908+0000 7f7cae3d5700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c9406aed0 0x7f7c9406d390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ca815c020 tx=0x7f7c98008040 comp rx=0 tx=0).stop 2023-11-26T07:12:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.908+0000 7f7cae3d5700 1 -- 172.21.15.83:0/781888065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ca806c710 msgr2=0x7f7ca8144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.908+0000 7f7cae3d5700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ca806c710 0x7f7ca8144d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00c550 tx=0x7f7c9c00c910 comp rx=0 tx=0).stop 2023-11-26T07:12:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.908+0000 7f7cae3d5700 1 -- 172.21.15.83:0/781888065 shutdown_connections 2023-11-26T07:12:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.908+0000 7f7cae3d5700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7c9406aed0 0x7f7c9406d390 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.908+0000 7f7cae3d5700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ca815e080 0x7f7ca8151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.908+0000 7f7cae3d5700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ca806c710 0x7f7ca8144d40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.908+0000 7f7cae3d5700 1 --2- 172.21.15.83:0/781888065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ca80af4d0 0x7f7ca8160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.908+0000 7f7cae3d5700 1 -- 172.21.15.83:0/781888065 >> 172.21.15.83:0/781888065 conn(0x7f7ca8071b90 msgr2=0x7f7ca80b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.909+0000 7f7cae3d5700 1 -- 172.21.15.83:0/781888065 shutdown_connections 2023-11-26T07:12:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:32.909+0000 7f7cae3d5700 1 -- 172.21.15.83:0/781888065 wait complete. 2023-11-26T07:12:32.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:12:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.044+0000 7ff636030700 1 Processor -- start 2023-11-26T07:12:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.044+0000 7ff636030700 1 -- start start 2023-11-26T07:12:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.045+0000 7ff636030700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff6300cf610 0x7ff6300add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.045+0000 7ff636030700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 0x7ff6300b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.045+0000 7ff636030700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63006b080 con 0x7ff630073f90 2023-11-26T07:12:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.045+0000 7ff636030700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63015c290 con 0x7ff6300cf610 2023-11-26T07:12:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.045+0000 7ff636030700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6300cf9f0 con 0x7ff63015f9e0 2023-11-26T07:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.045+0000 7ff62f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff6300cf610 0x7ff6300add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.045+0000 7ff62effd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 0x7ff6300b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.045+0000 7ff62effd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 0x7ff6300b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:46732/0 (socket says 172.21.15.83:46732) 2023-11-26T07:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.045+0000 7ff62f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff6300cf610 0x7ff6300add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:35724/0 (socket says 172.21.15.83:35724) 2023-11-26T07:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.045+0000 7ff62effd700 1 -- 172.21.15.83:0/1434462096 learned_addr learned my addr 172.21.15.83:0/1434462096 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.045+0000 7ff62ffff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ff630073f90 0x7ff63006b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:40934/0 (socket says 172.21.15.83:40934) 2023-11-26T07:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.046+0000 7ff62f7fe700 1 -- 172.21.15.83:0/1434462096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 msgr2=0x7ff6300b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.046+0000 7ff62f7fe700 1 --2- 172.21.15.83:0/1434462096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 0x7ff6300b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.046+0000 7ff62f7fe700 1 -- 172.21.15.83:0/1434462096 >> v1:172.21.15.203:6789/0 conn(0x7ff630073f90 legacy=0x7ff63006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.046+0000 7ff62f7fe700 1 -- 172.21.15.83:0/1434462096 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63015ff80 con 0x7ff6300cf610 2023-11-26T07:12:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.046+0000 7ff62e7fc700 1 -- 172.21.15.83:0/1434462096 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690290717 0 0) 0x7ff63006b080 con 0x7ff630073f90 2023-11-26T07:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.046+0000 7ff62effd700 1 --2- 172.21.15.83:0/1434462096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 0x7ff6300b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.046+0000 7ff62f7fe700 1 --2- 172.21.15.83:0/1434462096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6300cf610 0x7ff6300add40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff624009840 tx=0x7ff624009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e88b47e7fe1241bb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.046+0000 7ff62e7fc700 1 -- 172.21.15.83:0/1434462096 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624011830 con 0x7ff6300cf610 2023-11-26T07:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.046+0000 7ff62e7fc700 1 -- 172.21.15.83:0/1434462096 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff624011990 con 0x7ff6300cf610 2023-11-26T07:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.046+0000 7ff62e7fc700 1 -- 172.21.15.83:0/1434462096 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624011d20 con 0x7ff6300cf610 2023-11-26T07:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.047+0000 7ff636030700 1 -- 172.21.15.83:0/1434462096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6300cf610 msgr2=0x7ff6300add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.047+0000 7ff636030700 1 --2- 172.21.15.83:0/1434462096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6300cf610 0x7ff6300add40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff624009840 tx=0x7ff624009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.047+0000 7ff636030700 1 -- 172.21.15.83:0/1434462096 shutdown_connections 2023-11-26T07:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.047+0000 7ff636030700 1 --2- 172.21.15.83:0/1434462096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6300cf610 0x7ff6300add40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.047+0000 7ff636030700 1 --2- 172.21.15.83:0/1434462096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 0x7ff6300b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.047+0000 7ff636030700 1 -- 172.21.15.83:0/1434462096 >> 172.21.15.83:0/1434462096 conn(0x7ff6300d0bf0 msgr2=0x7ff6300d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.047+0000 7ff636030700 1 -- 172.21.15.83:0/1434462096 shutdown_connections 2023-11-26T07:12:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.047+0000 7ff636030700 1 -- 172.21.15.83:0/1434462096 wait complete. 2023-11-26T07:12:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.048+0000 7ff636030700 1 Processor -- start 2023-11-26T07:12:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.048+0000 7ff636030700 1 -- start start 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff636030700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff630073f90 0x7ff63014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff636030700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6300cf610 0x7ff63015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff636030700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 0x7ff63015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff636030700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63015cee0 con 0x7ff6300cf610 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff636030700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63006dca0 con 0x7ff630073f90 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff636030700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63014fbe0 con 0x7ff63015f9e0 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6300cf610 0x7ff63015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6300cf610 0x7ff63015c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36280/0 (socket says 172.21.15.83:36280) 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62effd700 1 -- 172.21.15.83:0/2056917218 learned_addr learned my addr 172.21.15.83:0/2056917218 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62f7fe700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff630073f90 0x7ff63014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62ffff700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 0x7ff63015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62effd700 1 -- 172.21.15.83:0/2056917218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 msgr2=0x7ff63015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62effd700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 0x7ff63015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62effd700 1 -- 172.21.15.83:0/2056917218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff630073f90 msgr2=0x7ff63014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62effd700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff630073f90 0x7ff63014fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62effd700 1 -- 172.21.15.83:0/2056917218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff620009550 con 0x7ff6300cf610 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62f7fe700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff630073f90 0x7ff63014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff62effd700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6300cf610 0x7ff63015c460 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff62000ebe0 tx=0x7ff620009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff617fff700 1 -- 172.21.15.83:0/2056917218 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62000c8b0 con 0x7ff6300cf610 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff617fff700 1 -- 172.21.15.83:0/2056917218 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62000ca10 con 0x7ff6300cf610 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff636030700 1 -- 172.21.15.83:0/2056917218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6240094a0 con 0x7ff6300cf610 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.049+0000 7ff617fff700 1 -- 172.21.15.83:0/2056917218 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62000cd80 con 0x7ff6300cf610 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.050+0000 7ff636030700 1 -- 172.21.15.83:0/2056917218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63014efb0 con 0x7ff6300cf610 2023-11-26T07:12:33.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.051+0000 7ff636030700 1 -- 172.21.15.83:0/2056917218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff630068fb0 con 0x7ff6300cf610 2023-11-26T07:12:33.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.051+0000 7ff617fff700 1 -- 172.21.15.83:0/2056917218 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff620013470 con 0x7ff6300cf610 2023-11-26T07:12:33.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.051+0000 7ff617fff700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff61c06aed0 0x7ff61c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.051+0000 7ff62f7fe700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff61c06aed0 0x7ff61c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:33.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.051+0000 7ff617fff700 1 -- 172.21.15.83:0/2056917218 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff620089b20 con 0x7ff6300cf610 2023-11-26T07:12:33.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.052+0000 7ff62f7fe700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff61c06aed0 0x7ff61c06d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff6240090b0 tx=0x7ff624008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:33.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.056+0000 7ff617fff700 1 -- 172.21.15.83:0/2056917218 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff620054bd0 con 0x7ff6300cf610 2023-11-26T07:12:33.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.248+0000 7ff636030700 1 -- 172.21.15.83:0/2056917218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff63006dde0 con 0x7ff6300cf610 2023-11-26T07:12:33.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.249+0000 7ff617fff700 1 -- 172.21.15.83:0/2056917218 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff63006dde0 con 0x7ff6300cf610 2023-11-26T07:12:33.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 -- 172.21.15.83:0/2056917218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff61c06aed0 msgr2=0x7ff61c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff61c06aed0 0x7ff61c06d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff6240090b0 tx=0x7ff624008030 comp rx=0 tx=0).stop 2023-11-26T07:12:33.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 -- 172.21.15.83:0/2056917218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6300cf610 msgr2=0x7ff63015c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6300cf610 0x7ff63015c460 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff62000ebe0 tx=0x7ff620009e70 comp rx=0 tx=0).stop 2023-11-26T07:12:33.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 -- 172.21.15.83:0/2056917218 shutdown_connections 2023-11-26T07:12:33.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff61c06aed0 0x7ff61c06d390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff63015f9e0 0x7ff63015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff630073f90 0x7ff63014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 --2- 172.21.15.83:0/2056917218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6300cf610 0x7ff63015c460 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 -- 172.21.15.83:0/2056917218 >> 172.21.15.83:0/2056917218 conn(0x7ff6300d0bf0 msgr2=0x7ff6300d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:33.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 -- 172.21.15.83:0/2056917218 shutdown_connections 2023-11-26T07:12:33.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.251+0000 7ff636030700 1 -- 172.21.15.83:0/2056917218 wait complete. 2023-11-26T07:12:33.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:33.265 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp19700/subvol46630) 2023-11-26T07:12:33.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:33.299 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-11-26T07:12:33.358 INFO:teuthology.orchestra.run.smithi083.stdout:cephfs_data 2023-11-26T07:12:33.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:33.358 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp19700/subvol46630) 2023-11-26T07:12:33.445 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:33.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp19700/subvol46630/.meta 2023-11-26T07:12:33.512 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol46630 subvol_grp19700 2023-11-26T07:12:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.674+0000 7f5c3a254700 1 Processor -- start 2023-11-26T07:12:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.674+0000 7f5c3a254700 1 -- start start 2023-11-26T07:12:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c3a254700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c340ce5a0 0x7f5c340ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c3a254700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5c340ad720 0x7f5c340afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c3a254700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c340aeff0 con 0x7f5c3415a510 2023-11-26T07:12:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c3a254700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c340aebd0 con 0x7f5c340ad720 2023-11-26T07:12:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c3a254700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c340adf50 con 0x7f5c340ce5a0 2023-11-26T07:12:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c337fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5c3415a510 0x7f5c340cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56328/0 (socket says 172.21.15.83:56328) 2023-11-26T07:12:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c33fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5c340ad720 0x7f5c340afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c337fe700 1 -- 172.21.15.83:0/2365837273 learned_addr learned my addr 172.21.15.83:0/2365837273 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c33fff700 1 -- 172.21.15.83:0/2365837273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c340ce5a0 msgr2=0x7f5c340ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c33fff700 1 --2- 172.21.15.83:0/2365837273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c340ce5a0 0x7f5c340ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c33fff700 1 -- 172.21.15.83:0/2365837273 >> v1:172.21.15.83:6789/0 conn(0x7f5c3415a510 legacy=0x7f5c340cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c33fff700 1 -- 172.21.15.83:0/2365837273 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c340ae230 con 0x7f5c340ad720 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c327fc700 1 -- 172.21.15.83:0/2365837273 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588781618 0 0) 0x7f5c28001110 con 0x7f5c3415a510 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.675+0000 7f5c33fff700 1 --2- 172.21.15.83:0/2365837273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c340ad720 0x7f5c340afb80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009d90 tx=0x7f5c2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cbcc3d2a6de2ef server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.676+0000 7f5c327fc700 1 -- 172.21.15.83:0/2365837273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00e070 con 0x7f5c340ad720 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.676+0000 7f5c327fc700 1 -- 172.21.15.83:0/2365837273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c00bbd0 con 0x7f5c340ad720 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.676+0000 7f5c327fc700 1 -- 172.21.15.83:0/2365837273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c012430 con 0x7f5c340ad720 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.676+0000 7f5c3a254700 1 -- 172.21.15.83:0/2365837273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c340ad720 msgr2=0x7f5c340afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.676+0000 7f5c3a254700 1 --2- 172.21.15.83:0/2365837273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c340ad720 0x7f5c340afb80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009d90 tx=0x7f5c2c00b330 comp rx=0 tx=0).stop 2023-11-26T07:12:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.676+0000 7f5c3a254700 1 -- 172.21.15.83:0/2365837273 shutdown_connections 2023-11-26T07:12:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.676+0000 7f5c3a254700 1 --2- 172.21.15.83:0/2365837273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c340ad720 0x7f5c340afb80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.676+0000 7f5c3a254700 1 --2- 172.21.15.83:0/2365837273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c340ce5a0 0x7f5c340ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.676+0000 7f5c3a254700 1 -- 172.21.15.83:0/2365837273 >> 172.21.15.83:0/2365837273 conn(0x7f5c34072710 msgr2=0x7f5c34072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.676+0000 7f5c3a254700 1 -- 172.21.15.83:0/2365837273 shutdown_connections 2023-11-26T07:12:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.676+0000 7f5c3a254700 1 -- 172.21.15.83:0/2365837273 wait complete. 2023-11-26T07:12:33.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.677+0000 7f5c3a254700 1 Processor -- start 2023-11-26T07:12:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.678+0000 7f5c3a254700 1 -- start start 2023-11-26T07:12:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.678+0000 7f5c3a254700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c340ad720 0x7f5c3414f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.678+0000 7f5c3a254700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c340ce5a0 0x7f5c3415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.678+0000 7f5c3a254700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3415a510 0x7f5c3415e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.678+0000 7f5c3a254700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c340c8640 con 0x7f5c3415a510 2023-11-26T07:12:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.678+0000 7f5c3a254700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34153070 con 0x7f5c340ce5a0 2023-11-26T07:12:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.678+0000 7f5c3a254700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3414f940 con 0x7f5c340ad720 2023-11-26T07:12:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.678+0000 7f5c33fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3415a510 0x7f5c3415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c33fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3415a510 0x7f5c3415e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36298/0 (socket says 172.21.15.83:36298) 2023-11-26T07:12:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c33fff700 1 -- 172.21.15.83:0/4234582334 learned_addr learned my addr 172.21.15.83:0/4234582334 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c337fe700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c340ad720 0x7f5c3414f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c32ffd700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c340ce5a0 0x7f5c3415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c33fff700 1 -- 172.21.15.83:0/4234582334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c340ad720 msgr2=0x7f5c3414f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c33fff700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c340ad720 0x7f5c3414f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c33fff700 1 -- 172.21.15.83:0/4234582334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c340ce5a0 msgr2=0x7f5c3415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c33fff700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c340ce5a0 0x7f5c3415dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c33fff700 1 -- 172.21.15.83:0/4234582334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c009a70 con 0x7f5c3415a510 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c33fff700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3415a510 0x7f5c3415e530 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5c34072320 tx=0x7f5c2c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c1bfff700 1 -- 172.21.15.83:0/4234582334 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c012430 con 0x7f5c3415a510 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c1bfff700 1 -- 172.21.15.83:0/4234582334 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c020070 con 0x7f5c3415a510 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c3a254700 1 -- 172.21.15.83:0/4234582334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c340c8080 con 0x7f5c3415a510 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c1bfff700 1 -- 172.21.15.83:0/4234582334 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c004280 con 0x7f5c3415a510 2023-11-26T07:12:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.679+0000 7f5c3a254700 1 -- 172.21.15.83:0/4234582334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c340c8960 con 0x7f5c3415a510 2023-11-26T07:12:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.680+0000 7f5c3a254700 1 -- 172.21.15.83:0/4234582334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c340c87c0 con 0x7f5c3415a510 2023-11-26T07:12:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.680+0000 7f5c1bfff700 1 -- 172.21.15.83:0/4234582334 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c2c0043e0 con 0x7f5c3415a510 2023-11-26T07:12:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.681+0000 7f5c1bfff700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5c2006aed0 0x7f5c2006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.681+0000 7f5c1bfff700 1 -- 172.21.15.83:0/4234582334 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c2c022070 con 0x7f5c3415a510 2023-11-26T07:12:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.681+0000 7f5c337fe700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5c2006aed0 0x7f5c2006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.681+0000 7f5c337fe700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5c2006aed0 0x7f5c2006d390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5c28000f60 tx=0x7f5c28009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.685+0000 7f5c1bfff700 1 -- 172.21.15.83:0/4234582334 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c340c87c0 con 0x7f5c3415a510 2023-11-26T07:12:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.907+0000 7f5c3a254700 1 -- 172.21.15.83:0/4234582334 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol46630", "group_name": "subvol_grp19700", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c3415a8f0 con 0x7f5c2006aed0 2023-11-26T07:12:33.913 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:33.911+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:33.913 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:33.911+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:33.913 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:33.911+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:33.914 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:33.911+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:33.914 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:33.911+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.946+0000 7f5c1bfff700 1 -- 172.21.15.83:0/4234582334 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5c28004870 con 0x7f5c2006aed0 2023-11-26T07:12:33.948 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/subvol_grp19700/subvol46630 2023-11-26T07:12:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 -- 172.21.15.83:0/4234582334 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5c2006aed0 msgr2=0x7f5c2006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5c2006aed0 0x7f5c2006d390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5c28000f60 tx=0x7f5c28009040 comp rx=0 tx=0).stop 2023-11-26T07:12:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 -- 172.21.15.83:0/4234582334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3415a510 msgr2=0x7f5c3415e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3415a510 0x7f5c3415e530 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5c34072320 tx=0x7f5c2c004ad0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 -- 172.21.15.83:0/4234582334 shutdown_connections 2023-11-26T07:12:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3415a510 0x7f5c3415e530 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f5c2006aed0 0x7f5c2006d390 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c340ad720 0x7f5c3414f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 --2- 172.21.15.83:0/4234582334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c340ce5a0 0x7f5c3415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:33.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 -- 172.21.15.83:0/4234582334 >> 172.21.15.83:0/4234582334 conn(0x7f5c34072710 msgr2=0x7f5c340b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:33.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 -- 172.21.15.83:0/4234582334 shutdown_connections 2023-11-26T07:12:33.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:33.948+0000 7f5c3a254700 1 -- 172.21.15.83:0/4234582334 wait complete. 2023-11-26T07:12:33.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol46630 subvol_grp19700 2023-11-26T07:12:34.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.085+0000 7f7a8c244700 1 Processor -- start 2023-11-26T07:12:34.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.085+0000 7f7a8c244700 1 -- start start 2023-11-26T07:12:34.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.086+0000 7f7a8c244700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7a840cd810 0x7f7a8406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.086+0000 7f7a8c244700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a84073f90 0x7f7a84074390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.086+0000 7f7a8c244700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a840cf530 con 0x7f7a84145620 2023-11-26T07:12:34.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.086+0000 7f7a8c244700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8406b700 con 0x7f7a84073f90 2023-11-26T07:12:34.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.086+0000 7f7a8c244700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a840748d0 con 0x7f7a840cd810 2023-11-26T07:12:34.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.086+0000 7f7a8a7e1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a84073f90 0x7f7a84074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.086+0000 7f7a8a7e1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7a84073f90 0x7f7a84074390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36322/0 (socket says 172.21.15.83:36322) 2023-11-26T07:12:34.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.086+0000 7f7a8a7e1700 1 -- 172.21.15.83:0/2719359379 learned_addr learned my addr 172.21.15.83:0/2719359379 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:34.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.086+0000 7f7a897df700 1 --2- 172.21.15.83:0/2719359379 >> v2:172.21.15.203:3300/0 conn(0x7f7a840cd810 0x7f7a8406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.087+0000 7f7a8a7e1700 1 -- 172.21.15.83:0/2719359379 >> v2:172.21.15.203:3300/0 conn(0x7f7a840cd810 msgr2=0x7f7a8406dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:34.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.087+0000 7f7a8a7e1700 1 --2- 172.21.15.83:0/2719359379 >> v2:172.21.15.203:3300/0 conn(0x7f7a840cd810 0x7f7a8406dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.087+0000 7f7a8a7e1700 1 -- 172.21.15.83:0/2719359379 >> v1:172.21.15.203:6789/0 conn(0x7f7a84145620 legacy=0x7f7a8406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.087+0000 7f7a8a7e1700 1 -- 172.21.15.83:0/2719359379 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8406b8f0 con 0x7f7a84073f90 2023-11-26T07:12:34.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.087+0000 7f7a897df700 1 --2- 172.21.15.83:0/2719359379 >> v2:172.21.15.203:3300/0 conn(0x7f7a840cd810 0x7f7a8406dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:34.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.087+0000 7f7a8a7e1700 1 --2- 172.21.15.83:0/2719359379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a84073f90 0x7f7a84074390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7a840cce30 tx=0x7f7a7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520b3c5b77239fb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:34.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.087+0000 7f7a88fde700 1 -- 172.21.15.83:0/2719359379 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00e070 con 0x7f7a84073f90 2023-11-26T07:12:34.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.087+0000 7f7a88fde700 1 -- 172.21.15.83:0/2719359379 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c00b9c0 con 0x7f7a84073f90 2023-11-26T07:12:34.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.087+0000 7f7a88fde700 1 -- 172.21.15.83:0/2719359379 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00bd70 con 0x7f7a84073f90 2023-11-26T07:12:34.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.087+0000 7f7a8c244700 1 -- 172.21.15.83:0/2719359379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a84073f90 msgr2=0x7f7a84074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.087+0000 7f7a8c244700 1 --2- 172.21.15.83:0/2719359379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a84073f90 0x7f7a84074390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7a840cce30 tx=0x7f7a7c00b330 comp rx=0 tx=0).stop 2023-11-26T07:12:34.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.088+0000 7f7a8c244700 1 -- 172.21.15.83:0/2719359379 shutdown_connections 2023-11-26T07:12:34.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.088+0000 7f7a8c244700 1 --2- 172.21.15.83:0/2719359379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a84073f90 0x7f7a84074390 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.088+0000 7f7a8c244700 1 --2- 172.21.15.83:0/2719359379 >> v2:172.21.15.203:3300/0 conn(0x7f7a840cd810 0x7f7a8406dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.088+0000 7f7a8c244700 1 -- 172.21.15.83:0/2719359379 >> 172.21.15.83:0/2719359379 conn(0x7f7a840d0b40 msgr2=0x7f7a840d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:34.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.088+0000 7f7a8c244700 1 -- 172.21.15.83:0/2719359379 shutdown_connections 2023-11-26T07:12:34.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.088+0000 7f7a8c244700 1 -- 172.21.15.83:0/2719359379 wait complete. 2023-11-26T07:12:34.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.089+0000 7f7a8c244700 1 Processor -- start 2023-11-26T07:12:34.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.089+0000 7f7a8c244700 1 -- start start 2023-11-26T07:12:34.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a8c244700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a84073f90 0x7f7a84137270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a8c244700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a840cd810 0x7f7a841f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a8c244700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a84145620 0x7f7a841f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a8c244700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8413ce70 con 0x7f7a84073f90 2023-11-26T07:12:34.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a8c244700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a841377b0 con 0x7f7a840cd810 2023-11-26T07:12:34.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a8c244700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8413a450 con 0x7f7a84145620 2023-11-26T07:12:34.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a89fe0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a84073f90 0x7f7a84137270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a89fe0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a84073f90 0x7f7a84137270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36324/0 (socket says 172.21.15.83:36324) 2023-11-26T07:12:34.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a89fe0700 1 -- 172.21.15.83:0/2624589733 learned_addr learned my addr 172.21.15.83:0/2624589733 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a897df700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a840cd810 0x7f7a841f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a8a7e1700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a84145620 0x7f7a841f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a89fe0700 1 -- 172.21.15.83:0/2624589733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a84145620 msgr2=0x7f7a841f5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a89fe0700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a84145620 0x7f7a841f5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a89fe0700 1 -- 172.21.15.83:0/2624589733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a840cd810 msgr2=0x7f7a841f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a89fe0700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a840cd810 0x7f7a841f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a89fe0700 1 -- 172.21.15.83:0/2624589733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c009b30 con 0x7f7a84073f90 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a89fe0700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a84073f90 0x7f7a84137270 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400c450 tx=0x7f7a7400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a7a7fc700 1 -- 172.21.15.83:0/2624589733 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a740052d0 con 0x7f7a84073f90 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a7a7fc700 1 -- 172.21.15.83:0/2624589733 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a74005430 con 0x7f7a84073f90 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a7a7fc700 1 -- 172.21.15.83:0/2624589733 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a740038c0 con 0x7f7a84073f90 2023-11-26T07:12:34.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a8c244700 1 -- 172.21.15.83:0/2624589733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8413c820 con 0x7f7a84073f90 2023-11-26T07:12:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.090+0000 7f7a8c244700 1 -- 172.21.15.83:0/2624589733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a841f5d10 con 0x7f7a84073f90 2023-11-26T07:12:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.091+0000 7f7a8c244700 1 -- 172.21.15.83:0/2624589733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a84067f60 con 0x7f7a84073f90 2023-11-26T07:12:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.092+0000 7f7a7a7fc700 1 -- 172.21.15.83:0/2624589733 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a74003a20 con 0x7f7a84073f90 2023-11-26T07:12:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.092+0000 7f7a7a7fc700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7a6806aed0 0x7f7a6806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.092+0000 7f7a7a7fc700 1 -- 172.21.15.83:0/2624589733 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a74088980 con 0x7f7a84073f90 2023-11-26T07:12:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.092+0000 7f7a897df700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7a6806aed0 0x7f7a6806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.092+0000 7f7a897df700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7a6806aed0 0x7f7a6806d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7a84136430 tx=0x7f7a80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:34.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.096+0000 7f7a7a7fc700 1 -- 172.21.15.83:0/2624589733 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a74053a30 con 0x7f7a84073f90 2023-11-26T07:12:34.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.322+0000 7f7a8c244700 1 -- 172.21.15.83:0/2624589733 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol46630", "group_name": "subvol_grp19700", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a841608e0 con 0x7f7a6806aed0 2023-11-26T07:12:34.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.326+0000 7f7a7a7fc700 1 -- 172.21.15.83:0/2624589733 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7a841608e0 con 0x7f7a6806aed0 2023-11-26T07:12:34.329 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/subvol_grp19700/subvol46630 2023-11-26T07:12:34.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 -- 172.21.15.83:0/2624589733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7a6806aed0 msgr2=0x7f7a6806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7a6806aed0 0x7f7a6806d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7a84136430 tx=0x7f7a80008040 comp rx=0 tx=0).stop 2023-11-26T07:12:34.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 -- 172.21.15.83:0/2624589733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a84073f90 msgr2=0x7f7a84137270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a84073f90 0x7f7a84137270 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400c450 tx=0x7f7a7400c760 comp rx=0 tx=0).stop 2023-11-26T07:12:34.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 -- 172.21.15.83:0/2624589733 shutdown_connections 2023-11-26T07:12:34.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f7a6806aed0 0x7f7a6806d390 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a84145620 0x7f7a841f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a84073f90 0x7f7a84137270 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 --2- 172.21.15.83:0/2624589733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a840cd810 0x7f7a841f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 -- 172.21.15.83:0/2624589733 >> 172.21.15.83:0/2624589733 conn(0x7f7a840d0b40 msgr2=0x7f7a8413fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:34.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 -- 172.21.15.83:0/2624589733 shutdown_connections 2023-11-26T07:12:34.332 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.328+0000 7f7a8c244700 1 -- 172.21.15.83:0/2624589733 wait complete. 2023-11-26T07:12:34.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:34.343 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/ecbf3b65ea1f2e70c35b5e3bbcec8a04.meta) 2023-11-26T07:12:34.374 INFO:teuthology.orchestra.run.smithi083.stdout:[GLOBAL] 2023-11-26T07:12:34.375 INFO:teuthology.orchestra.run.smithi083.stdout:version = 1 2023-11-26T07:12:34.375 INFO:teuthology.orchestra.run.smithi083.stdout:type = subvolume 2023-11-26T07:12:34.375 INFO:teuthology.orchestra.run.smithi083.stdout:path = /volumes/subvol_grp19700/subvol46630 2023-11-26T07:12:34.375 INFO:teuthology.orchestra.run.smithi083.stdout:state = complete 2023-11-26T07:12:34.375 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:34.376 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol46630 subvol_grp19700 2023-11-26T07:12:34.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.538+0000 7f9d89dea700 1 Processor -- start 2023-11-26T07:12:34.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.538+0000 7f9d89dea700 1 -- start start 2023-11-26T07:12:34.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d89dea700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d84145950 0x7f9d840d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d89dea700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d8406c710 0x7f9d8406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d89dea700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d840cf100 con 0x7f9d8406e5c0 2023-11-26T07:12:34.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d89dea700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d840d0910 con 0x7f9d8406c710 2023-11-26T07:12:34.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d89dea700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840cf330 con 0x7f9d84145950 2023-11-26T07:12:34.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d83fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d8406c710 0x7f9d8406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d83fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d8406c710 0x7f9d8406ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36338/0 (socket says 172.21.15.83:36338) 2023-11-26T07:12:34.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d83fff700 1 -- 172.21.15.83:0/2862442858 learned_addr learned my addr 172.21.15.83:0/2862442858 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:34.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d837fe700 1 --2- 172.21.15.83:0/2862442858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d84145950 0x7f9d840d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d83fff700 1 -- 172.21.15.83:0/2862442858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d84145950 msgr2=0x7f9d840d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:34.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d83fff700 1 --2- 172.21.15.83:0/2862442858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d84145950 0x7f9d840d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d83fff700 1 -- 172.21.15.83:0/2862442858 >> v1:172.21.15.203:6789/0 conn(0x7f9d8406e5c0 legacy=0x7f9d8406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.539+0000 7f9d83fff700 1 -- 172.21.15.83:0/2862442858 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d840cf610 con 0x7f9d8406c710 2023-11-26T07:12:34.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.540+0000 7f9d83fff700 1 --2- 172.21.15.83:0/2862442858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8406c710 0x7f9d8406ea20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009e80 tx=0x7f9d7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300283ffaa469ae6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:34.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.541+0000 7f9d827fc700 1 -- 172.21.15.83:0/2862442858 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c00e070 con 0x7f9d8406c710 2023-11-26T07:12:34.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.541+0000 7f9d827fc700 1 -- 172.21.15.83:0/2862442858 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c00b980 con 0x7f9d8406c710 2023-11-26T07:12:34.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.541+0000 7f9d827fc700 1 -- 172.21.15.83:0/2862442858 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c00bd30 con 0x7f9d8406c710 2023-11-26T07:12:34.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.541+0000 7f9d89dea700 1 -- 172.21.15.83:0/2862442858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8406c710 msgr2=0x7f9d8406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.541+0000 7f9d89dea700 1 --2- 172.21.15.83:0/2862442858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8406c710 0x7f9d8406ea20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009e80 tx=0x7f9d7c00b330 comp rx=0 tx=0).stop 2023-11-26T07:12:34.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.541+0000 7f9d89dea700 1 -- 172.21.15.83:0/2862442858 shutdown_connections 2023-11-26T07:12:34.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.541+0000 7f9d89dea700 1 --2- 172.21.15.83:0/2862442858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8406c710 0x7f9d8406ea20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.541+0000 7f9d89dea700 1 --2- 172.21.15.83:0/2862442858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d84145950 0x7f9d840d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.541+0000 7f9d89dea700 1 -- 172.21.15.83:0/2862442858 >> 172.21.15.83:0/2862442858 conn(0x7f9d84071b50 msgr2=0x7f9d84071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:34.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.541+0000 7f9d89dea700 1 -- 172.21.15.83:0/2862442858 shutdown_connections 2023-11-26T07:12:34.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.541+0000 7f9d89dea700 1 -- 172.21.15.83:0/2862442858 wait complete. 2023-11-26T07:12:34.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.542+0000 7f9d89dea700 1 Processor -- start 2023-11-26T07:12:34.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.543+0000 7f9d89dea700 1 -- start start 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.543+0000 7f9d89dea700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8406c710 0x7f9d841f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.543+0000 7f9d89dea700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d8406e5c0 0x7f9d8413c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.543+0000 7f9d89dea700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d84145950 0x7f9d8413c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.543+0000 7f9d89dea700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840b3cc0 con 0x7f9d8406c710 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.543+0000 7f9d89dea700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d841f6b70 con 0x7f9d8406e5c0 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.543+0000 7f9d89dea700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840b4cd0 con 0x7f9d84145950 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d837fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8406c710 0x7f9d841f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d837fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8406c710 0x7f9d841f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36340/0 (socket says 172.21.15.83:36340) 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d837fe700 1 -- 172.21.15.83:0/870777218 learned_addr learned my addr 172.21.15.83:0/870777218 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d83fff700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d84145950 0x7f9d8413c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d82ffd700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d8406e5c0 0x7f9d8413c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d837fe700 1 -- 172.21.15.83:0/870777218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d84145950 msgr2=0x7f9d8413c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d837fe700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d84145950 0x7f9d8413c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d837fe700 1 -- 172.21.15.83:0/870777218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d8406e5c0 msgr2=0x7f9d8413c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d837fe700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d8406e5c0 0x7f9d8413c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d837fe700 1 -- 172.21.15.83:0/870777218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c009b30 con 0x7f9d8406c710 2023-11-26T07:12:34.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d837fe700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8406c710 0x7f9d841f6630 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800d800 tx=0x7f9d7800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:34.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d6bfff700 1 -- 172.21.15.83:0/870777218 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d780125f0 con 0x7f9d8406c710 2023-11-26T07:12:34.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d6bfff700 1 -- 172.21.15.83:0/870777218 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d78012750 con 0x7f9d8406c710 2023-11-26T07:12:34.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d89dea700 1 -- 172.21.15.83:0/870777218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d840b3f10 con 0x7f9d8406c710 2023-11-26T07:12:34.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d6bfff700 1 -- 172.21.15.83:0/870777218 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78012b00 con 0x7f9d8406c710 2023-11-26T07:12:34.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.544+0000 7f9d89dea700 1 -- 172.21.15.83:0/870777218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d840b44d0 con 0x7f9d8406c710 2023-11-26T07:12:34.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.545+0000 7f9d89dea700 1 -- 172.21.15.83:0/870777218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d840624c0 con 0x7f9d8406c710 2023-11-26T07:12:34.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.546+0000 7f9d6bfff700 1 -- 172.21.15.83:0/870777218 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d78012c60 con 0x7f9d8406c710 2023-11-26T07:12:34.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.546+0000 7f9d6bfff700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9d7006aed0 0x7f9d7006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:34.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.546+0000 7f9d6bfff700 1 -- 172.21.15.83:0/870777218 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d780895c0 con 0x7f9d8406c710 2023-11-26T07:12:34.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.546+0000 7f9d82ffd700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9d7006aed0 0x7f9d7006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:34.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.547+0000 7f9d82ffd700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9d7006aed0 0x7f9d7006d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9d8415edb0 tx=0x7f9d74009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:34.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.551+0000 7f9d6bfff700 1 -- 172.21.15.83:0/870777218 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d78054700 con 0x7f9d8406c710 2023-11-26T07:12:34.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.772+0000 7f9d89dea700 1 -- 172.21.15.83:0/870777218 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol46630", "group_name": "subvol_grp19700", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d840cfbd0 con 0x7f9d7006aed0 2023-11-26T07:12:34.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.787+0000 7f9d6bfff700 1 -- 172.21.15.83:0/870777218 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9d74001d60 con 0x7f9d7006aed0 2023-11-26T07:12:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.789+0000 7f9d89dea700 1 -- 172.21.15.83:0/870777218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9d7006aed0 msgr2=0x7f9d7006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.789+0000 7f9d89dea700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9d7006aed0 0x7f9d7006d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9d8415edb0 tx=0x7f9d74009040 comp rx=0 tx=0).stop 2023-11-26T07:12:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.789+0000 7f9d89dea700 1 -- 172.21.15.83:0/870777218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8406c710 msgr2=0x7f9d841f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.789+0000 7f9d89dea700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8406c710 0x7f9d841f6630 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800d800 tx=0x7f9d7800db10 comp rx=0 tx=0).stop 2023-11-26T07:12:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.789+0000 7f9d89dea700 1 -- 172.21.15.83:0/870777218 shutdown_connections 2023-11-26T07:12:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.789+0000 7f9d89dea700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9d7006aed0 0x7f9d7006d390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.789+0000 7f9d89dea700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d84145950 0x7f9d8413c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.789+0000 7f9d89dea700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d8406c710 0x7f9d841f6630 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.789+0000 7f9d89dea700 1 --2- 172.21.15.83:0/870777218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d8406e5c0 0x7f9d8413c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.789+0000 7f9d89dea700 1 -- 172.21.15.83:0/870777218 >> 172.21.15.83:0/870777218 conn(0x7f9d84071b50 msgr2=0x7f9d84143f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.790+0000 7f9d89dea700 1 -- 172.21.15.83:0/870777218 shutdown_connections 2023-11-26T07:12:34.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:34.790+0000 7f9d89dea700 1 -- 172.21.15.83:0/870777218 wait complete. 2023-11-26T07:12:34.793 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:34.790+0000 7f3781b6c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:34.793 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:34.790+0000 7f3781b6c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:34.793 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:34.790+0000 7f3781b6c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:34.794 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:34.790+0000 7f3781b6c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:34.794 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:34.790+0000 7f3781b6c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:34.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:34.806 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-26T07:12:34.807 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:34.805+0000 7f3780369700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:34.807 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:34.805+0000 7f3780369700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:34.807 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:34.805+0000 7f3780369700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:34.807 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:34.805+0000 7f3780369700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:34.807 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:12:34.805+0000 7f3780369700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:12:34.839 INFO:teuthology.orchestra.run.smithi083.stdout:2 2023-11-26T07:12:34.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp19700 2023-11-26T07:12:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.000+0000 7f1c8c931700 1 Processor -- start 2023-11-26T07:12:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.000+0000 7f1c8c931700 1 -- start start 2023-11-26T07:12:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.001+0000 7f1c8c931700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c880cc480 0x7f1c8806d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.001+0000 7f1c8c931700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c8806dc60 0x7f1c8806e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.001+0000 7f1c8c931700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c880c5b10 con 0x7f1c880ccae0 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.001+0000 7f1c8c931700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c880c60d0 con 0x7f1c8806dc60 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.001+0000 7f1c8c931700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c880cd790 con 0x7f1c880cc480 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.001+0000 7f1c86d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c8806dc60 0x7f1c8806e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.001+0000 7f1c86d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1c8806dc60 0x7f1c8806e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36380/0 (socket says 172.21.15.83:36380) 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.001+0000 7f1c86d9d700 1 -- 172.21.15.83:0/575250841 learned_addr learned my addr 172.21.15.83:0/575250841 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.002+0000 7f1c85d9b700 1 --2- 172.21.15.83:0/575250841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c880cc480 0x7f1c8806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.002+0000 7f1c86d9d700 1 -- 172.21.15.83:0/575250841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c880cc480 msgr2=0x7f1c8806d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.002+0000 7f1c86d9d700 1 --2- 172.21.15.83:0/575250841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c880cc480 0x7f1c8806d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.002+0000 7f1c86d9d700 1 -- 172.21.15.83:0/575250841 >> v1:172.21.15.203:6789/0 conn(0x7f1c880ccae0 legacy=0x7f1c880ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.002+0000 7f1c86d9d700 1 -- 172.21.15.83:0/575250841 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c880cda30 con 0x7f1c8806dc60 2023-11-26T07:12:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.002+0000 7f1c86d9d700 1 --2- 172.21.15.83:0/575250841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8806dc60 0x7f1c8806e110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009e50 tx=0x7f1c8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac16880f462f7db server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.002+0000 7f1c8559a700 1 -- 172.21.15.83:0/575250841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000e070 con 0x7f1c8806dc60 2023-11-26T07:12:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.002+0000 7f1c8559a700 1 -- 172.21.15.83:0/575250841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8000b910 con 0x7f1c8806dc60 2023-11-26T07:12:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.002+0000 7f1c8559a700 1 -- 172.21.15.83:0/575250841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000bcc0 con 0x7f1c8806dc60 2023-11-26T07:12:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.003+0000 7f1c8c931700 1 -- 172.21.15.83:0/575250841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8806dc60 msgr2=0x7f1c8806e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.003+0000 7f1c8c931700 1 --2- 172.21.15.83:0/575250841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8806dc60 0x7f1c8806e110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009e50 tx=0x7f1c8000b330 comp rx=0 tx=0).stop 2023-11-26T07:12:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.003+0000 7f1c8c931700 1 -- 172.21.15.83:0/575250841 shutdown_connections 2023-11-26T07:12:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.003+0000 7f1c8c931700 1 --2- 172.21.15.83:0/575250841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8806dc60 0x7f1c8806e110 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.003+0000 7f1c8c931700 1 --2- 172.21.15.83:0/575250841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c880cc480 0x7f1c8806d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.003+0000 7f1c8c931700 1 -- 172.21.15.83:0/575250841 >> 172.21.15.83:0/575250841 conn(0x7f1c880726e0 msgr2=0x7f1c88072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.003+0000 7f1c8c931700 1 -- 172.21.15.83:0/575250841 shutdown_connections 2023-11-26T07:12:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.003+0000 7f1c8c931700 1 -- 172.21.15.83:0/575250841 wait complete. 2023-11-26T07:12:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.004+0000 7f1c8c931700 1 Processor -- start 2023-11-26T07:12:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.004+0000 7f1c8c931700 1 -- start start 2023-11-26T07:12:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8c931700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8806dc60 0x7f1c880b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8c931700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c880cc480 0x7f1c8814b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8c931700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c880ccae0 0x7f1c880b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8c931700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8814bd80 con 0x7f1c8806dc60 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8c931700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c880b5500 con 0x7f1c880ccae0 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8c931700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c880b48f0 con 0x7f1c880cc480 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8806dc60 0x7f1c880b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8806dc60 0x7f1c880b4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36382/0 (socket says 172.21.15.83:36382) 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8659c700 1 -- 172.21.15.83:0/1123427941 learned_addr learned my addr 172.21.15.83:0/1123427941 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c85d9b700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c880cc480 0x7f1c8814b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c86d9d700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c880ccae0 0x7f1c880b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8659c700 1 -- 172.21.15.83:0/1123427941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c880cc480 msgr2=0x7f1c8814b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8659c700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c880cc480 0x7f1c8814b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8659c700 1 -- 172.21.15.83:0/1123427941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c880ccae0 msgr2=0x7f1c880b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8659c700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c880ccae0 0x7f1c880b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8659c700 1 -- 172.21.15.83:0/1123427941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c80009b30 con 0x7f1c8806dc60 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c85d9b700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c880cc480 0x7f1c8814b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8659c700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8806dc60 0x7f1c880b4280 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00c4e0 tx=0x7f1c7c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c76ffd700 1 -- 172.21.15.83:0/1123427941 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0052d0 con 0x7f1c8806dc60 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c76ffd700 1 -- 172.21.15.83:0/1123427941 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7c005430 con 0x7f1c8806dc60 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c76ffd700 1 -- 172.21.15.83:0/1123427941 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0038c0 con 0x7f1c8806dc60 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8c931700 1 -- 172.21.15.83:0/1123427941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c880b4bd0 con 0x7f1c8806dc60 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.005+0000 7f1c8c931700 1 -- 172.21.15.83:0/1123427941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c880b5190 con 0x7f1c8806dc60 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.006+0000 7f1c8c931700 1 -- 172.21.15.83:0/1123427941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c88068fb0 con 0x7f1c8806dc60 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.007+0000 7f1c76ffd700 1 -- 172.21.15.83:0/1123427941 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c7c003a20 con 0x7f1c8806dc60 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.007+0000 7f1c76ffd700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1c6806aed0 0x7f1c6806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.007+0000 7f1c76ffd700 1 -- 172.21.15.83:0/1123427941 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c7c088a20 con 0x7f1c8806dc60 2023-11-26T07:12:35.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.007+0000 7f1c85d9b700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1c6806aed0 0x7f1c6806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.008+0000 7f1c85d9b700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1c6806aed0 0x7f1c6806d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1c880cb0e0 tx=0x7f1c78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.012+0000 7f1c76ffd700 1 -- 172.21.15.83:0/1123427941 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c7c053b60 con 0x7f1c8806dc60 2023-11-26T07:12:35.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.232+0000 7f1c8c931700 1 -- 172.21.15.83:0/1123427941 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp19700", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c88062540 con 0x7f1c6806aed0 2023-11-26T07:12:35.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.237+0000 7f1c76ffd700 1 -- 172.21.15.83:0/1123427941 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1c780097e0 con 0x7f1c6806aed0 2023-11-26T07:12:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.238+0000 7f1c8c931700 1 -- 172.21.15.83:0/1123427941 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1c6806aed0 msgr2=0x7f1c6806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.238+0000 7f1c8c931700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1c6806aed0 0x7f1c6806d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1c880cb0e0 tx=0x7f1c78008040 comp rx=0 tx=0).stop 2023-11-26T07:12:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.238+0000 7f1c8c931700 1 -- 172.21.15.83:0/1123427941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8806dc60 msgr2=0x7f1c880b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.238+0000 7f1c8c931700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8806dc60 0x7f1c880b4280 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00c4e0 tx=0x7f1c7c00c8a0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.239+0000 7f1c8c931700 1 -- 172.21.15.83:0/1123427941 shutdown_connections 2023-11-26T07:12:35.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.239+0000 7f1c8c931700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c880ccae0 0x7f1c880b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.239+0000 7f1c8c931700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c8806dc60 0x7f1c880b4280 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.239+0000 7f1c8c931700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1c6806aed0 0x7f1c6806d390 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.239+0000 7f1c8c931700 1 --2- 172.21.15.83:0/1123427941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c880cc480 0x7f1c8814b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.239+0000 7f1c8c931700 1 -- 172.21.15.83:0/1123427941 >> 172.21.15.83:0/1123427941 conn(0x7f1c880726e0 msgr2=0x7f1c880adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:35.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.239+0000 7f1c8c931700 1 -- 172.21.15.83:0/1123427941 shutdown_connections 2023-11-26T07:12:35.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.239+0000 7f1c8c931700 1 -- 172.21.15.83:0/1123427941 wait complete. 2023-11-26T07:12:35.253 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:12:35.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:12:35.292 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:12:35.341 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:12:35.341 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:35.341 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:35.360 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:35.361 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:35.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:35.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.579+0000 7fd4ed459700 1 Processor -- start 2023-11-26T07:12:35.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.579+0000 7fd4ed459700 1 -- start start 2023-11-26T07:12:35.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.579+0000 7fd4ed459700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e80cf360 0x7fd4e80b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.579+0000 7fd4ed459700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e815a790 con 0x7fd4e815a3b0 2023-11-26T07:12:35.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.579+0000 7fd4ed459700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e80b4c40 con 0x7fd4e815deb0 2023-11-26T07:12:35.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.579+0000 7fd4ed459700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80cf8b0 con 0x7fd4e80cf360 2023-11-26T07:12:35.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e6ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd4e815a3b0 0x7fd4e80b3320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56430/0 (socket says 172.21.15.83:56430) 2023-11-26T07:12:35.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e6ffd700 1 -- 172.21.15.83:0/3535850603 learned_addr learned my addr 172.21.15.83:0/3535850603 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:35.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881833012 0 0) 0x7fd4e815a790 con 0x7fd4e815a3b0 2023-11-26T07:12:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e67fc700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e80cf360 0x7fd4e80b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4d00035b0 con 0x7fd4e815a3b0 2023-11-26T07:12:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1913178809 0 0) 0x7fd4d00035b0 con 0x7fd4e815a3b0 2023-11-26T07:12:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e80cf360 msgr2=0x7fd4e80b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e80cf360 0x7fd4e80b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 >> v1:172.21.15.203:6789/0 conn(0x7fd4e815deb0 legacy=0x7fd4e815e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e8070c60 con 0x7fd4e815a3b0 2023-11-26T07:12:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fd4dc001670 con 0x7fd4e815a3b0 2023-11-26T07:12:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 >> v1:172.21.15.83:6789/0 conn(0x7fd4e815a3b0 legacy=0x7fd4e80b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4d0003a30 0x7fd4d0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d0004850 0x7fd4d0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e80cf360 0x7fd4d0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80cf8b0 con 0x7fd4d0003a30 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d000cbd0 con 0x7fd4d0004850 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d000bb40 con 0x7fd4e80cf360 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e6ffd700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4d0003a30 0x7fd4d0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e67fc700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d0004850 0x7fd4d0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.580+0000 7fd4e77fe700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e80cf360 0x7fd4d0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4e6ffd700 1 -- 172.21.15.83:0/3535850603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e80cf360 msgr2=0x7fd4d0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4e6ffd700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e80cf360 0x7fd4d0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4e6ffd700 1 -- 172.21.15.83:0/3535850603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d0004850 msgr2=0x7fd4d0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4e6ffd700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d0004850 0x7fd4d0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4e6ffd700 1 -- 172.21.15.83:0/3535850603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00035b0 con 0x7fd4d0003a30 2023-11-26T07:12:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4e6ffd700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4d0003a30 0x7fd4d0003e50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc001eb0 tx=0x7fd4d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc269ea56665fef server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00cd50 con 0x7fd4d0003a30 2023-11-26T07:12:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc005960 con 0x7fd4d0003a30 2023-11-26T07:12:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4e5ffb700 1 -- 172.21.15.83:0/3535850603 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc005d10 con 0x7fd4d0003a30 2023-11-26T07:12:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4ed459700 1 -- 172.21.15.83:0/3535850603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4d0003a30 msgr2=0x7fd4d0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4ed459700 1 --2- 172.21.15.83:0/3535850603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4d0003a30 0x7fd4d0003e50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc001eb0 tx=0x7fd4d0005880 comp rx=0 tx=0).stop 2023-11-26T07:12:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4e77fe700 1 -- 172.21.15.83:0/3535850603 reap_dead start 2023-11-26T07:12:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4ed459700 1 -- 172.21.15.83:0/3535850603 shutdown_connections 2023-11-26T07:12:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4ed459700 1 -- 172.21.15.83:0/3535850603 >> 172.21.15.83:0/3535850603 conn(0x7fd4e80b1740 msgr2=0x7fd4e80b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.581+0000 7fd4ed459700 1 -- 172.21.15.83:0/3535850603 shutdown_connections 2023-11-26T07:12:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.582+0000 7fd4ed459700 1 -- 172.21.15.83:0/3535850603 wait complete. 2023-11-26T07:12:35.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.583+0000 7fd4ed459700 1 Processor -- start 2023-11-26T07:12:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.583+0000 7fd4ed459700 1 -- start start 2023-11-26T07:12:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.583+0000 7fd4ed459700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4d0003a30 0x7fd4e8142f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4ed459700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4e80ca640 0x7fd4e80c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4ed459700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4e8158be0 0x7fd4e81f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4ed459700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c9300 con 0x7fd4e80ca640 2023-11-26T07:12:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4ed459700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c95a0 con 0x7fd4e8158be0 2023-11-26T07:12:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4ed459700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e80c9880 con 0x7fd4d0003a30 2023-11-26T07:12:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4e67fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4e80ca640 0x7fd4e80c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4e67fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4e80ca640 0x7fd4e80c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36402/0 (socket says 172.21.15.83:36402) 2023-11-26T07:12:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4e67fc700 1 -- 172.21.15.83:0/2294532469 learned_addr learned my addr 172.21.15.83:0/2294532469 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4e6ffd700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4d0003a30 0x7fd4e8142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4e77fe700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4e8158be0 0x7fd4e81f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4e67fc700 1 -- 172.21.15.83:0/2294532469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4d0003a30 msgr2=0x7fd4e8142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4e67fc700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4d0003a30 0x7fd4e8142f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4e67fc700 1 -- 172.21.15.83:0/2294532469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4e8158be0 msgr2=0x7fd4e81f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4e67fc700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4e8158be0 0x7fd4e81f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4e67fc700 1 -- 172.21.15.83:0/2294532469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e80c9b60 con 0x7fd4e80ca640 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4e67fc700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4e80ca640 0x7fd4e80c8dc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80119c0 tx=0x7fd4d8011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4cf7fe700 1 -- 172.21.15.83:0/2294532469 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8011e60 con 0x7fd4e80ca640 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4cf7fe700 1 -- 172.21.15.83:0/2294532469 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d800f940 con 0x7fd4e80ca640 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4ed459700 1 -- 172.21.15.83:0/2294532469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e80c9e40 con 0x7fd4e80ca640 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4cf7fe700 1 -- 172.21.15.83:0/2294532469 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d800fcf0 con 0x7fd4e80ca640 2023-11-26T07:12:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.584+0000 7fd4ed459700 1 -- 172.21.15.83:0/2294532469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e8159070 con 0x7fd4e80ca640 2023-11-26T07:12:35.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.585+0000 7fd4ed459700 1 -- 172.21.15.83:0/2294532469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e80751e0 con 0x7fd4e80ca640 2023-11-26T07:12:35.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.586+0000 7fd4cf7fe700 1 -- 172.21.15.83:0/2294532469 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fd4d8014440 con 0x7fd4e80ca640 2023-11-26T07:12:35.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.586+0000 7fd4cf7fe700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4d406af60 0x7fd4d406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.586+0000 7fd4e6ffd700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4d406af60 0x7fd4d406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.586+0000 7fd4cf7fe700 1 -- 172.21.15.83:0/2294532469 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4d8089bb0 con 0x7fd4e80ca640 2023-11-26T07:12:35.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.587+0000 7fd4e6ffd700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4d406af60 0x7fd4d406d420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc001930 tx=0x7fd4dc00a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:35.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.591+0000 7fd4cf7fe700 1 -- 172.21.15.83:0/2294532469 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4d8090080 con 0x7fd4e80ca640 2023-11-26T07:12:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.782+0000 7fd4ed459700 1 -- 172.21.15.83:0/2294532469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4e80cb890 con 0x7fd4e80ca640 2023-11-26T07:12:35.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.783+0000 7fd4cf7fe700 1 -- 172.21.15.83:0/2294532469 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fd4d8054c70 con 0x7fd4e80ca640 2023-11-26T07:12:35.786 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:35.787 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:24.102531+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:20.302489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:21.468154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:35.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 -- 172.21.15.83:0/2294532469 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4d406af60 msgr2=0x7fd4d406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4d406af60 0x7fd4d406d420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc001930 tx=0x7fd4dc00a000 comp rx=0 tx=0).stop 2023-11-26T07:12:35.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 -- 172.21.15.83:0/2294532469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4e80ca640 msgr2=0x7fd4e80c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4e80ca640 0x7fd4e80c8dc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80119c0 tx=0x7fd4d8011cd0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 -- 172.21.15.83:0/2294532469 shutdown_connections 2023-11-26T07:12:35.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4e8158be0 0x7fd4e81f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd4d406af60 0x7fd4d406d420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4d0003a30 0x7fd4e8142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 --2- 172.21.15.83:0/2294532469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4e80ca640 0x7fd4e80c8dc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 -- 172.21.15.83:0/2294532469 >> 172.21.15.83:0/2294532469 conn(0x7fd4e80b1740 msgr2=0x7fd4e81546b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:35.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 -- 172.21.15.83:0/2294532469 shutdown_connections 2023-11-26T07:12:35.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.786+0000 7fd4ed459700 1 -- 172.21.15.83:0/2294532469 wait complete. 2023-11-26T07:12:35.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:12:20.302489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:12:21.468154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:12:35.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:12:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.921+0000 7f1e7e409700 1 Processor -- start 2023-11-26T07:12:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.922+0000 7f1e7e409700 1 -- start start 2023-11-26T07:12:35.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.922+0000 7f1e7e409700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1e7815e160 0x7f1e780b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.922+0000 7f1e7e409700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e780afc90 0x7f1e78073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.922+0000 7f1e7e409700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e780b3c60 con 0x7f1e780744d0 2023-11-26T07:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.922+0000 7f1e7e409700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e780b4500 con 0x7f1e780afc90 2023-11-26T07:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.922+0000 7f1e7e409700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e78074940 con 0x7f1e7815e160 2023-11-26T07:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.922+0000 7f1e777fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e780afc90 0x7f1e78073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.922+0000 7f1e777fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e780afc90 0x7f1e78073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36408/0 (socket says 172.21.15.83:36408) 2023-11-26T07:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.923+0000 7f1e777fe700 1 -- 172.21.15.83:0/2205920412 learned_addr learned my addr 172.21.15.83:0/2205920412 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.923+0000 7f1e77fff700 1 --2- 172.21.15.83:0/2205920412 >> v2:172.21.15.203:3300/0 conn(0x7f1e7815e160 0x7f1e780b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.923+0000 7f1e777fe700 1 -- 172.21.15.83:0/2205920412 >> v2:172.21.15.203:3300/0 conn(0x7f1e7815e160 msgr2=0x7f1e780b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.923+0000 7f1e777fe700 1 --2- 172.21.15.83:0/2205920412 >> v2:172.21.15.203:3300/0 conn(0x7f1e7815e160 0x7f1e780b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.923+0000 7f1e777fe700 1 -- 172.21.15.83:0/2205920412 >> v1:172.21.15.203:6789/0 conn(0x7f1e780744d0 legacy=0x7f1e7806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.923+0000 7f1e777fe700 1 -- 172.21.15.83:0/2205920412 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e780b58c0 con 0x7f1e780afc90 2023-11-26T07:12:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.923+0000 7f1e777fe700 1 --2- 172.21.15.83:0/2205920412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e780afc90 0x7f1e78073f90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1e7809efa0 tx=0x7f1e60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7d599817175a9f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.923+0000 7f1e76ffd700 1 -- 172.21.15.83:0/2205920412 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60004230 con 0x7f1e780afc90 2023-11-26T07:12:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.923+0000 7f1e76ffd700 1 -- 172.21.15.83:0/2205920412 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e60004390 con 0x7f1e780afc90 2023-11-26T07:12:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.923+0000 7f1e76ffd700 1 -- 172.21.15.83:0/2205920412 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600036a0 con 0x7f1e780afc90 2023-11-26T07:12:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.924+0000 7f1e7e409700 1 -- 172.21.15.83:0/2205920412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e780afc90 msgr2=0x7f1e78073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.924+0000 7f1e7e409700 1 --2- 172.21.15.83:0/2205920412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e780afc90 0x7f1e78073f90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1e7809efa0 tx=0x7f1e60009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:35.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.924+0000 7f1e7e409700 1 -- 172.21.15.83:0/2205920412 shutdown_connections 2023-11-26T07:12:35.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.924+0000 7f1e7e409700 1 --2- 172.21.15.83:0/2205920412 >> v2:172.21.15.203:3300/0 conn(0x7f1e7815e160 0x7f1e780b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.924+0000 7f1e7e409700 1 --2- 172.21.15.83:0/2205920412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e780afc90 0x7f1e78073f90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.924+0000 7f1e7e409700 1 -- 172.21.15.83:0/2205920412 >> 172.21.15.83:0/2205920412 conn(0x7f1e780d0bf0 msgr2=0x7f1e780d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:35.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.924+0000 7f1e7e409700 1 -- 172.21.15.83:0/2205920412 shutdown_connections 2023-11-26T07:12:35.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.924+0000 7f1e7e409700 1 -- 172.21.15.83:0/2205920412 wait complete. 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.925+0000 7f1e7e409700 1 Processor -- start 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.925+0000 7f1e7e409700 1 -- start start 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7e409700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e780744d0 0x7f1e78144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7e409700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e780afc90 0x7f1e7815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7e409700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7815e160 0x7f1e7815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7e409700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e781611f0 con 0x7f1e7815e160 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7e409700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e78161610 con 0x7f1e780744d0 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7e409700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780c5c40 con 0x7f1e780afc90 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7c9a6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7815e160 0x7f1e7815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7c9a6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7815e160 0x7f1e7815cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36418/0 (socket says 172.21.15.83:36418) 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7c9a6700 1 -- 172.21.15.83:0/2436641893 learned_addr learned my addr 172.21.15.83:0/2436641893 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e777fe700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e780afc90 0x7f1e7815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7c9a6700 1 -- 172.21.15.83:0/2436641893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e780afc90 msgr2=0x7f1e7815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7c9a6700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e780afc90 0x7f1e7815c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e77fff700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e780744d0 0x7f1e78144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7c9a6700 1 -- 172.21.15.83:0/2436641893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e780744d0 msgr2=0x7f1e78144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7c9a6700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e780744d0 0x7f1e78144640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7c9a6700 1 -- 172.21.15.83:0/2436641893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60009550 con 0x7f1e7815e160 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e777fe700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e780afc90 0x7f1e7815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.926+0000 7f1e7c9a6700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7815e160 0x7f1e7815cde0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000cc40 tx=0x7f1e7000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.927+0000 7f1e74ff9700 1 -- 172.21.15.83:0/2436641893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000aba0 con 0x7f1e7815e160 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.927+0000 7f1e74ff9700 1 -- 172.21.15.83:0/2436641893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7000ad00 con 0x7f1e7815e160 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.927+0000 7f1e7e409700 1 -- 172.21.15.83:0/2436641893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e78160600 con 0x7f1e7815e160 2023-11-26T07:12:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.927+0000 7f1e74ff9700 1 -- 172.21.15.83:0/2436641893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70012660 con 0x7f1e7815e160 2023-11-26T07:12:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.927+0000 7f1e7e409700 1 -- 172.21.15.83:0/2436641893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e78160850 con 0x7f1e7815e160 2023-11-26T07:12:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.928+0000 7f1e7e409700 1 -- 172.21.15.83:0/2436641893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e78068fb0 con 0x7f1e7815e160 2023-11-26T07:12:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.928+0000 7f1e74ff9700 1 -- 172.21.15.83:0/2436641893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1e700127c0 con 0x7f1e7815e160 2023-11-26T07:12:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.928+0000 7f1e74ff9700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1e5806af60 0x7f1e5806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.929+0000 7f1e74ff9700 1 -- 172.21.15.83:0/2436641893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e70089cf0 con 0x7f1e7815e160 2023-11-26T07:12:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.929+0000 7f1e77fff700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1e5806af60 0x7f1e5806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.929+0000 7f1e77fff700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1e5806af60 0x7f1e5806d420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009770 tx=0x7f1e68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:35.933+0000 7f1e74ff9700 1 -- 172.21.15.83:0/2436641893 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1e70054e40 con 0x7f1e7815e160 2023-11-26T07:12:36.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.122+0000 7f1e7e409700 1 -- 172.21.15.83:0/2436641893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e7806c140 con 0x7f1e7815e160 2023-11-26T07:12:36.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.122+0000 7f1e74ff9700 1 -- 172.21.15.83:0/2436641893 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e70016070 con 0x7f1e7815e160 2023-11-26T07:12:36.125 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:12:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.124+0000 7f1e7e409700 1 -- 172.21.15.83:0/2436641893 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1e5806af60 msgr2=0x7f1e5806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.124+0000 7f1e7e409700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1e5806af60 0x7f1e5806d420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009770 tx=0x7f1e68008040 comp rx=0 tx=0).stop 2023-11-26T07:12:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.125+0000 7f1e7e409700 1 -- 172.21.15.83:0/2436641893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7815e160 msgr2=0x7f1e7815cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.125+0000 7f1e7e409700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7815e160 0x7f1e7815cde0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000cc40 tx=0x7f1e7000a060 comp rx=0 tx=0).stop 2023-11-26T07:12:36.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.125+0000 7f1e7e409700 1 -- 172.21.15.83:0/2436641893 shutdown_connections 2023-11-26T07:12:36.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.125+0000 7f1e7e409700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7815e160 0x7f1e7815cde0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.125+0000 7f1e7e409700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e780744d0 0x7f1e78144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.125+0000 7f1e7e409700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1e5806af60 0x7f1e5806d420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.125+0000 7f1e7e409700 1 --2- 172.21.15.83:0/2436641893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e780afc90 0x7f1e7815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.125+0000 7f1e7e409700 1 -- 172.21.15.83:0/2436641893 >> 172.21.15.83:0/2436641893 conn(0x7f1e780d0bf0 msgr2=0x7f1e781619f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:36.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.125+0000 7f1e7e409700 1 -- 172.21.15.83:0/2436641893 shutdown_connections 2023-11-26T07:12:36.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.125+0000 7f1e7e409700 1 -- 172.21.15.83:0/2436641893 wait complete. 2023-11-26T07:12:36.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:12:36.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.260+0000 7f473c6b4700 1 Processor -- start 2023-11-26T07:12:36.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.260+0000 7f473c6b4700 1 -- start start 2023-11-26T07:12:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.261+0000 7f473c6b4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f47340c6c30 0x7f47340cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.261+0000 7f473c6b4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47340cef40 0x7f47340cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.261+0000 7f473c6b4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47340d0140 con 0x7f473415bf50 2023-11-26T07:12:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.261+0000 7f473c6b4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473415c330 con 0x7f47340c6c30 2023-11-26T07:12:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.261+0000 7f473c6b4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47340cf3d0 con 0x7f47340cef40 2023-11-26T07:12:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.261+0000 7f473a450700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f473415bf50 0x7f47340ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56462/0 (socket says 172.21.15.83:56462) 2023-11-26T07:12:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.261+0000 7f473a450700 1 -- 172.21.15.83:0/1882165205 learned_addr learned my addr 172.21.15.83:0/1882165205 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473ac51700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47340cef40 0x7f47340cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f4739c4f700 1 --2- 172.21.15.83:0/1882165205 >> v2:172.21.15.203:3300/0 conn(0x7f47340c6c30 0x7f47340cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87939320 0 0) 0x7f47340d0140 con 0x7f473415bf50 2023-11-26T07:12:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47200035b0 con 0x7f473415bf50 2023-11-26T07:12:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 979886544 0 0) 0x7f47200035b0 con 0x7f473415bf50 2023-11-26T07:12:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47340cef40 msgr2=0x7f47340cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47340cef40 0x7f47340cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 >> v2:172.21.15.203:3300/0 conn(0x7f47340c6c30 msgr2=0x7f47340cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 --2- 172.21.15.83:0/1882165205 >> v2:172.21.15.203:3300/0 conn(0x7f47340c6c30 0x7f47340cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47340d03e0 con 0x7f473415bf50 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f4724001670 con 0x7f473415bf50 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 >> v1:172.21.15.83:6789/0 conn(0x7f473415bf50 legacy=0x7f47340ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4720003a00 0x7f4720003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4720004850 0x7f4720004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4720005210 0x7f472000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473415c330 con 0x7f4720003a00 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47340cf3d0 con 0x7f4720005210 2023-11-26T07:12:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472000bf50 con 0x7f4720004850 2023-11-26T07:12:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473ac51700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47340cef40 0x7f47340cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:12:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473a450700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4720003a00 0x7f4720003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f4739c4f700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4720004850 0x7f4720004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473ac51700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4720005210 0x7f472000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473a450700 1 -- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4720004850 msgr2=0x7f4720004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473a450700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4720004850 0x7f4720004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473a450700 1 -- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4720005210 msgr2=0x7f472000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473a450700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4720005210 0x7f472000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.262+0000 7f473a450700 1 -- 172.21.15.83:0/1882165205 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47200035b0 con 0x7f4720003a00 2023-11-26T07:12:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.263+0000 7f473a450700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4720003a00 0x7f4720003e20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f472400c350 tx=0x7f4720005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bed29f98f0a8592 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.263+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472400ce50 con 0x7f4720003a00 2023-11-26T07:12:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.263+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4724005960 con 0x7f4720003a00 2023-11-26T07:12:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.263+0000 7f473944e700 1 -- 172.21.15.83:0/1882165205 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724005d10 con 0x7f4720003a00 2023-11-26T07:12:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.263+0000 7f473c6b4700 1 -- 172.21.15.83:0/1882165205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4720003a00 msgr2=0x7f4720003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.263+0000 7f473c6b4700 1 --2- 172.21.15.83:0/1882165205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4720003a00 0x7f4720003e20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f472400c350 tx=0x7f4720005880 comp rx=0 tx=0).stop 2023-11-26T07:12:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.263+0000 7f473ac51700 1 -- 172.21.15.83:0/1882165205 reap_dead start 2023-11-26T07:12:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.263+0000 7f473c6b4700 1 -- 172.21.15.83:0/1882165205 shutdown_connections 2023-11-26T07:12:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.263+0000 7f473c6b4700 1 -- 172.21.15.83:0/1882165205 >> 172.21.15.83:0/1882165205 conn(0x7f47340b3780 msgr2=0x7f47340b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.263+0000 7f473c6b4700 1 -- 172.21.15.83:0/1882165205 shutdown_connections 2023-11-26T07:12:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.263+0000 7f473c6b4700 1 -- 172.21.15.83:0/1882165205 wait complete. 2023-11-26T07:12:36.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.265+0000 7f473c6b4700 1 Processor -- start 2023-11-26T07:12:36.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.265+0000 7f473c6b4700 1 -- start start 2023-11-26T07:12:36.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.265+0000 7f473c6b4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4720003a00 0x7f47341f1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.265+0000 7f473c6b4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47340ca270 0x7f47340ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.265+0000 7f473c6b4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47341574f0 0x7f4734157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.265+0000 7f473c6b4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4734141950 con 0x7f47341574f0 2023-11-26T07:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.265+0000 7f473c6b4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47340cac30 con 0x7f47340ca270 2023-11-26T07:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.265+0000 7f473c6b4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4734158570 con 0x7f4720003a00 2023-11-26T07:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.265+0000 7f473ac51700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47341574f0 0x7f4734157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.265+0000 7f473ac51700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47341574f0 0x7f4734157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36438/0 (socket says 172.21.15.83:36438) 2023-11-26T07:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.265+0000 7f473ac51700 1 -- 172.21.15.83:0/1915101201 learned_addr learned my addr 172.21.15.83:0/1915101201 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f473a450700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4720003a00 0x7f47341f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f4739c4f700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47340ca270 0x7f47340ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f473ac51700 1 -- 172.21.15.83:0/1915101201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4720003a00 msgr2=0x7f47341f1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f473ac51700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4720003a00 0x7f47341f1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f473ac51700 1 -- 172.21.15.83:0/1915101201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47340ca270 msgr2=0x7f47340ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f473ac51700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47340ca270 0x7f47340ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f473ac51700 1 -- 172.21.15.83:0/1915101201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4734158290 con 0x7f47341574f0 2023-11-26T07:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f4739c4f700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47340ca270 0x7f47340ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f473ac51700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47341574f0 0x7f4734157980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f47340cdc60 tx=0x7f472c0116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f472affd700 1 -- 172.21.15.83:0/1915101201 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c00e070 con 0x7f47341574f0 2023-11-26T07:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f472affd700 1 -- 172.21.15.83:0/1915101201 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c011be0 con 0x7f47341574f0 2023-11-26T07:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f473c6b4700 1 -- 172.21.15.83:0/1915101201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4734141290 con 0x7f47341574f0 2023-11-26T07:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f472affd700 1 -- 172.21.15.83:0/1915101201 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c011d90 con 0x7f47341574f0 2023-11-26T07:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.266+0000 7f473c6b4700 1 -- 172.21.15.83:0/1915101201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4734157ec0 con 0x7f47341574f0 2023-11-26T07:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.267+0000 7f473c6b4700 1 -- 172.21.15.83:0/1915101201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47340c5680 con 0x7f47341574f0 2023-11-26T07:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.267+0000 7f472affd700 1 -- 172.21.15.83:0/1915101201 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f472c003a20 con 0x7f47341574f0 2023-11-26T07:12:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.268+0000 7f472affd700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f471806af60 0x7f471806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.268+0000 7f472affd700 1 -- 172.21.15.83:0/1915101201 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f472c089da0 con 0x7f47341574f0 2023-11-26T07:12:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.268+0000 7f473a450700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f471806af60 0x7f471806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.268+0000 7f473a450700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f471806af60 0x7f471806d420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4724008fd0 tx=0x7f472400a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:36.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.272+0000 7f472affd700 1 -- 172.21.15.83:0/1915101201 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f472c054de0 con 0x7f47341574f0 2023-11-26T07:12:36.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.463+0000 7f473c6b4700 1 -- 172.21.15.83:0/1915101201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4734068fb0 con 0x7f47341574f0 2023-11-26T07:12:36.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.463+0000 7f472affd700 1 -- 172.21.15.83:0/1915101201 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f472c058400 con 0x7f47341574f0 2023-11-26T07:12:36.466 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:12:36.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 -- 172.21.15.83:0/1915101201 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f471806af60 msgr2=0x7f471806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f471806af60 0x7f471806d420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4724008fd0 tx=0x7f472400a000 comp rx=0 tx=0).stop 2023-11-26T07:12:36.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 -- 172.21.15.83:0/1915101201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47341574f0 msgr2=0x7f4734157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47341574f0 0x7f4734157980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f47340cdc60 tx=0x7f472c0116e0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 -- 172.21.15.83:0/1915101201 shutdown_connections 2023-11-26T07:12:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47341574f0 0x7f4734157980 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f471806af60 0x7f471806d420 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4720003a00 0x7f47341f1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 --2- 172.21.15.83:0/1915101201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47340ca270 0x7f47340ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 -- 172.21.15.83:0/1915101201 >> 172.21.15.83:0/1915101201 conn(0x7f47340b3780 msgr2=0x7f47340d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 -- 172.21.15.83:0/1915101201 shutdown_connections 2023-11-26T07:12:36.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.465+0000 7f473c6b4700 1 -- 172.21.15.83:0/1915101201 wait complete. 2023-11-26T07:12:36.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:12:36.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.603+0000 7f8ee2b60700 1 Processor -- start 2023-11-26T07:12:36.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.603+0000 7f8ee2b60700 1 -- start start 2023-11-26T07:12:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee2b60700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8edc15be70 0x7f8edc0cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee2b60700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8edc0746a0 0x7f8edc0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee2b60700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8edc0b5ee0 con 0x7f8edc0add40 2023-11-26T07:12:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee2b60700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8edc06bfc0 con 0x7f8edc0746a0 2023-11-26T07:12:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee2b60700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8edc06ba00 con 0x7f8edc15be70 2023-11-26T07:12:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee10fd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8edc0746a0 0x7f8edc0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8edbfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8edc0add40 0x7f8edc073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56484/0 (socket says 172.21.15.83:56484) 2023-11-26T07:12:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8edbfff700 1 -- 172.21.15.83:0/3051195912 learned_addr learned my addr 172.21.15.83:0/3051195912 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee08fc700 1 --2- 172.21.15.83:0/3051195912 >> v2:172.21.15.203:3300/0 conn(0x7f8edc15be70 0x7f8edc0cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee10fd700 1 -- 172.21.15.83:0/3051195912 >> v2:172.21.15.203:3300/0 conn(0x7f8edc15be70 msgr2=0x7f8edc0cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee10fd700 1 --2- 172.21.15.83:0/3051195912 >> v2:172.21.15.203:3300/0 conn(0x7f8edc15be70 0x7f8edc0cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee10fd700 1 -- 172.21.15.83:0/3051195912 >> v1:172.21.15.83:6789/0 conn(0x7f8edc0add40 legacy=0x7f8edc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8edb7fe700 1 -- 172.21.15.83:0/3051195912 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676852927 0 0) 0x7f8edc0b5ee0 con 0x7f8edc0add40 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee10fd700 1 -- 172.21.15.83:0/3051195912 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc06bce0 con 0x7f8edc0746a0 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8ee10fd700 1 --2- 172.21.15.83:0/3051195912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc0746a0 0x7f8edc0b3c60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8edc073020 tx=0x7f8ed400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362f3c51d588b12 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8edb7fe700 1 -- 172.21.15.83:0/3051195912 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400e070 con 0x7f8edc0746a0 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8edb7fe700 1 -- 172.21.15.83:0/3051195912 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed400bbd0 con 0x7f8edc0746a0 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.604+0000 7f8edb7fe700 1 -- 172.21.15.83:0/3051195912 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4012430 con 0x7f8edc0746a0 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.605+0000 7f8ee2b60700 1 -- 172.21.15.83:0/3051195912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc0746a0 msgr2=0x7f8edc0b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.605+0000 7f8ee2b60700 1 --2- 172.21.15.83:0/3051195912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc0746a0 0x7f8edc0b3c60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8edc073020 tx=0x7f8ed400b330 comp rx=0 tx=0).stop 2023-11-26T07:12:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.605+0000 7f8ee2b60700 1 -- 172.21.15.83:0/3051195912 shutdown_connections 2023-11-26T07:12:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.605+0000 7f8ee2b60700 1 --2- 172.21.15.83:0/3051195912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc0746a0 0x7f8edc0b3c60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.605+0000 7f8ee2b60700 1 --2- 172.21.15.83:0/3051195912 >> v2:172.21.15.203:3300/0 conn(0x7f8edc15be70 0x7f8edc0cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.605+0000 7f8ee2b60700 1 -- 172.21.15.83:0/3051195912 >> 172.21.15.83:0/3051195912 conn(0x7f8edc0d0bf0 msgr2=0x7f8edc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.605+0000 7f8ee2b60700 1 -- 172.21.15.83:0/3051195912 shutdown_connections 2023-11-26T07:12:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.605+0000 7f8ee2b60700 1 -- 172.21.15.83:0/3051195912 wait complete. 2023-11-26T07:12:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.606+0000 7f8ee2b60700 1 Processor -- start 2023-11-26T07:12:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.606+0000 7f8ee2b60700 1 -- start start 2023-11-26T07:12:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ee2b60700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8edc0746a0 0x7f8edc1f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ee2b60700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc0add40 0x7f8edc14a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ee2b60700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8edc15be70 0x7f8edc09e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ee2b60700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc14aab0 con 0x7f8edc0add40 2023-11-26T07:12:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ee2b60700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc15f290 con 0x7f8edc15be70 2023-11-26T07:12:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ee2b60700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc09ad70 con 0x7f8edc0746a0 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8edbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc0add40 0x7f8edc14a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8edbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc0add40 0x7f8edc14a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36450/0 (socket says 172.21.15.83:36450) 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8edbfff700 1 -- 172.21.15.83:0/2313361939 learned_addr learned my addr 172.21.15.83:0/2313361939 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ee08fc700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8edc0746a0 0x7f8edc1f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ee10fd700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8edc15be70 0x7f8edc09e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8edbfff700 1 -- 172.21.15.83:0/2313361939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8edc0746a0 msgr2=0x7f8edc1f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8edbfff700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8edc0746a0 0x7f8edc1f1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8edbfff700 1 -- 172.21.15.83:0/2313361939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8edc15be70 msgr2=0x7f8edc09e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8edbfff700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8edc15be70 0x7f8edc09e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8edbfff700 1 -- 172.21.15.83:0/2313361939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed4009a70 con 0x7f8edc0add40 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8edbfff700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc0add40 0x7f8edc14a570 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80011e0 tx=0x7f8ec800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ed97fa700 1 -- 172.21.15.83:0/2313361939 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8004c80 con 0x7f8edc0add40 2023-11-26T07:12:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ed97fa700 1 -- 172.21.15.83:0/2313361939 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec8004e00 con 0x7f8edc0add40 2023-11-26T07:12:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ee2b60700 1 -- 172.21.15.83:0/2313361939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc09b050 con 0x7f8edc0add40 2023-11-26T07:12:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ed97fa700 1 -- 172.21.15.83:0/2313361939 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8007310 con 0x7f8edc0add40 2023-11-26T07:12:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.608+0000 7f8ee2b60700 1 -- 172.21.15.83:0/2313361939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc09b610 con 0x7f8edc0add40 2023-11-26T07:12:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.609+0000 7f8ee2b60700 1 -- 172.21.15.83:0/2313361939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8edc068fb0 con 0x7f8edc0add40 2023-11-26T07:12:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.610+0000 7f8ed97fa700 1 -- 172.21.15.83:0/2313361939 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8ec8007470 con 0x7f8edc0add40 2023-11-26T07:12:36.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.610+0000 7f8ed97fa700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ec006af60 0x7f8ec006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.610+0000 7f8ed97fa700 1 -- 172.21.15.83:0/2313361939 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ec8089240 con 0x7f8edc0add40 2023-11-26T07:12:36.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.610+0000 7f8ee08fc700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ec006af60 0x7f8ec006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.610+0000 7f8ee08fc700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ec006af60 0x7f8ec006d420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0006b20 tx=0x7f8ed0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.614+0000 7f8ed97fa700 1 -- 172.21.15.83:0/2313361939 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ec8054300 con 0x7f8edc0add40 2023-11-26T07:12:36.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.803+0000 7f8ee2b60700 1 -- 172.21.15.83:0/2313361939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8edc09ee60 con 0x7f8edc0add40 2023-11-26T07:12:36.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.804+0000 7f8ed97fa700 1 -- 172.21.15.83:0/2313361939 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8ec8057920 con 0x7f8edc0add40 2023-11-26T07:12:36.806 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:12:36.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 -- 172.21.15.83:0/2313361939 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ec006af60 msgr2=0x7f8ec006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ec006af60 0x7f8ec006d420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0006b20 tx=0x7f8ed0008040 comp rx=0 tx=0).stop 2023-11-26T07:12:36.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 -- 172.21.15.83:0/2313361939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc0add40 msgr2=0x7f8edc14a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc0add40 0x7f8edc14a570 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80011e0 tx=0x7f8ec800ca30 comp rx=0 tx=0).stop 2023-11-26T07:12:36.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 -- 172.21.15.83:0/2313361939 shutdown_connections 2023-11-26T07:12:36.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8edc15be70 0x7f8edc09e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ec006af60 0x7f8ec006d420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8edc0746a0 0x7f8edc1f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 --2- 172.21.15.83:0/2313361939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc0add40 0x7f8edc14a570 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 -- 172.21.15.83:0/2313361939 >> 172.21.15.83:0/2313361939 conn(0x7f8edc0d0bf0 msgr2=0x7f8edc1589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:36.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 -- 172.21.15.83:0/2313361939 shutdown_connections 2023-11-26T07:12:36.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.806+0000 7f8ee2b60700 1 -- 172.21.15.83:0/2313361939 wait complete. 2023-11-26T07:12:36.820 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:12:36.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.939+0000 7feddbfaa700 1 Processor -- start 2023-11-26T07:12:36.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.939+0000 7feddbfaa700 1 -- start start 2023-11-26T07:12:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7feddbfaa700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fedd40ce140 0x7fedd415f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7feddbfaa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedd415f630 0x7fedd406b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7feddbfaa700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd41414c0 con 0x7fedd415af10 2023-11-26T07:12:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7feddbfaa700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd415c300 con 0x7fedd40ce140 2023-11-26T07:12:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7feddbfaa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd40ce9f0 con 0x7fedd415f630 2023-11-26T07:12:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7fedd9545700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fedd40ce140 0x7fedd415f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7fedda547700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedd415f630 0x7fedd406b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7fedd9545700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fedd40ce140 0x7fedd415f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:35910/0 (socket says 172.21.15.83:35910) 2023-11-26T07:12:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7fedd9d46700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fedd415af10 0x7fedd415e9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:41114/0 (socket says 172.21.15.83:41114) 2023-11-26T07:12:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7fedd9545700 1 -- 172.21.15.83:0/3236564688 learned_addr learned my addr 172.21.15.83:0/3236564688 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7fedda547700 1 -- 172.21.15.83:0/3236564688 >> v2:172.21.15.203:3300/0 conn(0x7fedd40ce140 msgr2=0x7fedd415f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7fedda547700 1 --2- 172.21.15.83:0/3236564688 >> v2:172.21.15.203:3300/0 conn(0x7fedd40ce140 0x7fedd415f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7fedda547700 1 -- 172.21.15.83:0/3236564688 >> v1:172.21.15.203:6789/0 conn(0x7fedd415af10 legacy=0x7fedd415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.940+0000 7fedda547700 1 -- 172.21.15.83:0/3236564688 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd40cecd0 con 0x7fedd415f630 2023-11-26T07:12:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7fedda547700 1 --2- 172.21.15.83:0/3236564688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedd415f630 0x7fedd406b640 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009d90 tx=0x7fedcc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b54abd51267df390 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7fedd8d44700 1 -- 172.21.15.83:0/3236564688 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc00e070 con 0x7fedd415f630 2023-11-26T07:12:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7fedd8d44700 1 -- 172.21.15.83:0/3236564688 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedcc00bb20 con 0x7fedd415f630 2023-11-26T07:12:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7fedd8d44700 1 -- 172.21.15.83:0/3236564688 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc012430 con 0x7fedd415f630 2023-11-26T07:12:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7feddbfaa700 1 -- 172.21.15.83:0/3236564688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedd415f630 msgr2=0x7fedd406b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7feddbfaa700 1 --2- 172.21.15.83:0/3236564688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedd415f630 0x7fedd406b640 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009d90 tx=0x7fedcc00b330 comp rx=0 tx=0).stop 2023-11-26T07:12:36.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7feddbfaa700 1 -- 172.21.15.83:0/3236564688 shutdown_connections 2023-11-26T07:12:36.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7feddbfaa700 1 --2- 172.21.15.83:0/3236564688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedd415f630 0x7fedd406b640 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7feddbfaa700 1 --2- 172.21.15.83:0/3236564688 >> v2:172.21.15.203:3300/0 conn(0x7fedd40ce140 0x7fedd415f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7feddbfaa700 1 -- 172.21.15.83:0/3236564688 >> 172.21.15.83:0/3236564688 conn(0x7fedd40cd570 msgr2=0x7fedd40cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:36.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7feddbfaa700 1 -- 172.21.15.83:0/3236564688 shutdown_connections 2023-11-26T07:12:36.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.941+0000 7feddbfaa700 1 -- 172.21.15.83:0/3236564688 wait complete. 2023-11-26T07:12:36.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.942+0000 7feddbfaa700 1 Processor -- start 2023-11-26T07:12:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.942+0000 7feddbfaa700 1 -- start start 2023-11-26T07:12:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.943+0000 7feddbfaa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd40ce140 0x7fedd41f2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.943+0000 7feddbfaa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedd415af10 0x7fedd4143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.943+0000 7feddbfaa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedd413fe90 0x7fedd40b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.943+0000 7feddbfaa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd415ca30 con 0x7fedd40ce140 2023-11-26T07:12:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.943+0000 7feddbfaa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd41f2740 con 0x7fedd413fe90 2023-11-26T07:12:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.943+0000 7feddbfaa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd4143e80 con 0x7fedd415af10 2023-11-26T07:12:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.943+0000 7fedd9d46700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd40ce140 0x7fedd41f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.943+0000 7fedd9d46700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd40ce140 0x7fedd41f2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36462/0 (socket says 172.21.15.83:36462) 2023-11-26T07:12:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.943+0000 7fedd9d46700 1 -- 172.21.15.83:0/2845961236 learned_addr learned my addr 172.21.15.83:0/2845961236 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.943+0000 7fedd9545700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedd415af10 0x7fedd4143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7fedda547700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedd413fe90 0x7fedd40b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7fedd9d46700 1 -- 172.21.15.83:0/2845961236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedd415af10 msgr2=0x7fedd4143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7fedd9d46700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedd415af10 0x7fedd4143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7fedd9d46700 1 -- 172.21.15.83:0/2845961236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedd413fe90 msgr2=0x7fedd40b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7fedd9d46700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedd413fe90 0x7fedd40b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7fedd9d46700 1 -- 172.21.15.83:0/2845961236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc009a70 con 0x7fedd40ce140 2023-11-26T07:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7fedda547700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedd413fe90 0x7fedd40b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7fedd9d46700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd40ce140 0x7fedd41f2200 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fedd000c4c0 tx=0x7fedd000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:36.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7fedca7fc700 1 -- 172.21.15.83:0/2845961236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd0005cb0 con 0x7fedd40ce140 2023-11-26T07:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7fedca7fc700 1 -- 172.21.15.83:0/2845961236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd0005e10 con 0x7fedd40ce140 2023-11-26T07:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7feddbfaa700 1 -- 172.21.15.83:0/2845961236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd4144670 con 0x7fedd40ce140 2023-11-26T07:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7fedca7fc700 1 -- 172.21.15.83:0/2845961236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00072d0 con 0x7fedd40ce140 2023-11-26T07:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.944+0000 7feddbfaa700 1 -- 172.21.15.83:0/2845961236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd4144390 con 0x7fedd40ce140 2023-11-26T07:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.945+0000 7feddbfaa700 1 -- 172.21.15.83:0/2845961236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd4067f60 con 0x7fedd40ce140 2023-11-26T07:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.945+0000 7fedca7fc700 1 -- 172.21.15.83:0/2845961236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fedd0007430 con 0x7fedd40ce140 2023-11-26T07:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.946+0000 7fedca7fc700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fedb806af60 0x7fedb806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:36.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.946+0000 7fedca7fc700 1 -- 172.21.15.83:0/2845961236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedd00889a0 con 0x7fedd40ce140 2023-11-26T07:12:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.946+0000 7fedd9545700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fedb806af60 0x7fedb806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.946+0000 7fedd9545700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fedb806af60 0x7fedb806d420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fedd40cf4c0 tx=0x7fedc4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:36.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:36.950+0000 7fedca7fc700 1 -- 172.21.15.83:0/2845961236 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fedd0053ba0 con 0x7fedd40ce140 2023-11-26T07:12:37.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.154+0000 7feddbfaa700 1 -- 172.21.15.83:0/2845961236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fedd40d0700 con 0x7fedd40ce140 2023-11-26T07:12:37.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.155+0000 7fedca7fc700 1 -- 172.21.15.83:0/2845961236 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7fedd0017080 con 0x7fedd40ce140 2023-11-26T07:12:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.156+0000 7feddbfaa700 1 -- 172.21.15.83:0/2845961236 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fedb806af60 msgr2=0x7fedb806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.156+0000 7feddbfaa700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fedb806af60 0x7fedb806d420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fedd40cf4c0 tx=0x7fedc4008040 comp rx=0 tx=0).stop 2023-11-26T07:12:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.156+0000 7feddbfaa700 1 -- 172.21.15.83:0/2845961236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd40ce140 msgr2=0x7fedd41f2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.156+0000 7feddbfaa700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd40ce140 0x7fedd41f2200 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fedd000c4c0 tx=0x7fedd000c880 comp rx=0 tx=0).stop 2023-11-26T07:12:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.157+0000 7feddbfaa700 1 -- 172.21.15.83:0/2845961236 shutdown_connections 2023-11-26T07:12:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.157+0000 7feddbfaa700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedd413fe90 0x7fedd40b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.157+0000 7feddbfaa700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd40ce140 0x7fedd41f2200 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.157+0000 7feddbfaa700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fedb806af60 0x7fedb806d420 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.157+0000 7feddbfaa700 1 --2- 172.21.15.83:0/2845961236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedd415af10 0x7fedd4143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.157+0000 7feddbfaa700 1 -- 172.21.15.83:0/2845961236 >> 172.21.15.83:0/2845961236 conn(0x7fedd40cd570 msgr2=0x7fedd406dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.157+0000 7feddbfaa700 1 -- 172.21.15.83:0/2845961236 shutdown_connections 2023-11-26T07:12:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.157+0000 7feddbfaa700 1 -- 172.21.15.83:0/2845961236 wait complete. 2023-11-26T07:12:37.161 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:12:37.171 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:12:37.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:37.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:12:37.223 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:12:37.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:37.224 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:12:37.255 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:37.256 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:37.274 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:37.275 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:37.334 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:37.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.453+0000 7fef9691a700 1 Processor -- start 2023-11-26T07:12:37.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.453+0000 7fef9691a700 1 -- start start 2023-11-26T07:12:37.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.454+0000 7fef9691a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fef901446a0 0x7fef900b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.454+0000 7fef9691a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef900adf00 0x7fef9006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.454+0000 7fef9691a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef900af950 con 0x7fef9006c540 2023-11-26T07:12:37.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.454+0000 7fef9691a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef90144a80 con 0x7fef901446a0 2023-11-26T07:12:37.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.454+0000 7fef9691a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9006c9b0 con 0x7fef900adf00 2023-11-26T07:12:37.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8f7fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef900adf00 0x7fef9006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8ffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fef901446a0 0x7fef900b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef94eb7700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fef9006c540 0x7fef9006da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:41132/0 (socket says 172.21.15.83:41132) 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8f7fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef900adf00 0x7fef9006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:46928/0 (socket says 172.21.15.83:46928) 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8ffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fef901446a0 0x7fef900b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:35922/0 (socket says 172.21.15.83:35922) 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef94eb7700 1 -- 172.21.15.83:0/1525205492 learned_addr learned my addr 172.21.15.83:0/1525205492 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8f7fe700 1 -- 172.21.15.83:0/1525205492 >> v2:172.21.15.203:3300/0 conn(0x7fef901446a0 msgr2=0x7fef900b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8f7fe700 1 --2- 172.21.15.83:0/1525205492 >> v2:172.21.15.203:3300/0 conn(0x7fef901446a0 0x7fef900b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8f7fe700 1 -- 172.21.15.83:0/1525205492 >> v1:172.21.15.203:6789/0 conn(0x7fef9006c540 legacy=0x7fef9006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8f7fe700 1 -- 172.21.15.83:0/1525205492 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900afc30 con 0x7fef900adf00 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8ffff700 1 --2- 172.21.15.83:0/1525205492 >> v2:172.21.15.203:3300/0 conn(0x7fef901446a0 0x7fef900b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8f7fe700 1 --2- 172.21.15.83:0/1525205492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef900adf00 0x7fef9006c000 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fef80009840 tx=0x7fef80009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed90b78f39c70227 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8effd700 1 -- 172.21.15.83:0/1525205492 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80011830 con 0x7fef900adf00 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.455+0000 7fef8effd700 1 -- 172.21.15.83:0/1525205492 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef80011990 con 0x7fef900adf00 2023-11-26T07:12:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.456+0000 7fef8effd700 1 -- 172.21.15.83:0/1525205492 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80011d20 con 0x7fef900adf00 2023-11-26T07:12:37.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.456+0000 7fef9691a700 1 -- 172.21.15.83:0/1525205492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef900adf00 msgr2=0x7fef9006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.456+0000 7fef9691a700 1 --2- 172.21.15.83:0/1525205492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef900adf00 0x7fef9006c000 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fef80009840 tx=0x7fef80009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:37.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.456+0000 7fef9691a700 1 -- 172.21.15.83:0/1525205492 shutdown_connections 2023-11-26T07:12:37.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.456+0000 7fef9691a700 1 --2- 172.21.15.83:0/1525205492 >> v2:172.21.15.203:3300/0 conn(0x7fef901446a0 0x7fef900b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.456+0000 7fef9691a700 1 --2- 172.21.15.83:0/1525205492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef900adf00 0x7fef9006c000 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fef80009840 tx=0x7fef80009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:37.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.456+0000 7fef9691a700 1 -- 172.21.15.83:0/1525205492 >> 172.21.15.83:0/1525205492 conn(0x7fef90071b90 msgr2=0x7fef90071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:37.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.456+0000 7fef9691a700 1 -- 172.21.15.83:0/1525205492 shutdown_connections 2023-11-26T07:12:37.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.456+0000 7fef9691a700 1 -- 172.21.15.83:0/1525205492 wait complete. 2023-11-26T07:12:37.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.457+0000 7fef9691a700 1 Processor -- start 2023-11-26T07:12:37.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.457+0000 7fef9691a700 1 -- start start 2023-11-26T07:12:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.458+0000 7fef9691a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9006c540 0x7fef9015a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.458+0000 7fef9691a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef901446a0 0x7fef90159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.458+0000 7fef9691a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9015a030 0x7fef9006da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.458+0000 7fef9691a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef900ce650 con 0x7fef9006c540 2023-11-26T07:12:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.458+0000 7fef9691a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9013d800 con 0x7fef9015a030 2023-11-26T07:12:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.458+0000 7fef9691a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9013dae0 con 0x7fef901446a0 2023-11-26T07:12:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.458+0000 7fef8ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9006c540 0x7fef9015a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.458+0000 7fef8ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9006c540 0x7fef9015a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36476/0 (socket says 172.21.15.83:36476) 2023-11-26T07:12:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.458+0000 7fef8ffff700 1 -- 172.21.15.83:0/562866083 learned_addr learned my addr 172.21.15.83:0/562866083 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.458+0000 7fef8f7fe700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef901446a0 0x7fef90159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.458+0000 7fef94eb7700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9015a030 0x7fef9006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.459+0000 7fef8ffff700 1 -- 172.21.15.83:0/562866083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef901446a0 msgr2=0x7fef90159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.459+0000 7fef8ffff700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef901446a0 0x7fef90159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.459+0000 7fef8ffff700 1 -- 172.21.15.83:0/562866083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9015a030 msgr2=0x7fef9006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.459+0000 7fef8ffff700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9015a030 0x7fef9006da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.459+0000 7fef8ffff700 1 -- 172.21.15.83:0/562866083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840094a0 con 0x7fef9006c540 2023-11-26T07:12:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.459+0000 7fef8ffff700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9006c540 0x7fef9015a560 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fef8400ebe0 tx=0x7fef8400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.459+0000 7fef8cff9700 1 -- 172.21.15.83:0/562866083 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84009760 con 0x7fef9006c540 2023-11-26T07:12:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.459+0000 7fef8cff9700 1 -- 172.21.15.83:0/562866083 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8400c850 con 0x7fef9006c540 2023-11-26T07:12:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.459+0000 7fef9691a700 1 -- 172.21.15.83:0/562866083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef800094a0 con 0x7fef9006c540 2023-11-26T07:12:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.459+0000 7fef8cff9700 1 -- 172.21.15.83:0/562866083 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8400cbe0 con 0x7fef9006c540 2023-11-26T07:12:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.459+0000 7fef9691a700 1 -- 172.21.15.83:0/562866083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef900ce910 con 0x7fef9006c540 2023-11-26T07:12:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.460+0000 7fef9691a700 1 -- 172.21.15.83:0/562866083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef900cef60 con 0x7fef9006c540 2023-11-26T07:12:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.460+0000 7fef8cff9700 1 -- 172.21.15.83:0/562866083 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fef8400cd40 con 0x7fef9006c540 2023-11-26T07:12:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.461+0000 7fef8cff9700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06af60 0x7fef7c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.461+0000 7fef8cff9700 1 -- 172.21.15.83:0/562866083 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef84089630 con 0x7fef9006c540 2023-11-26T07:12:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.461+0000 7fef8f7fe700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06af60 0x7fef7c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.461+0000 7fef8f7fe700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06af60 0x7fef7c06d420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fef900ae2e0 tx=0x7fef8001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:37.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.465+0000 7fef8cff9700 1 -- 172.21.15.83:0/562866083 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef840546f0 con 0x7fef9006c540 2023-11-26T07:12:37.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.566+0000 7fef8cff9700 1 -- 172.21.15.83:0/562866083 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fef84054ac0 con 0x7fef9006c540 2023-11-26T07:12:37.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.660+0000 7fef9691a700 1 -- 172.21.15.83:0/562866083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef900753f0 con 0x7fef9006c540 2023-11-26T07:12:37.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.661+0000 7fef8cff9700 1 -- 172.21.15.83:0/562866083 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fef840736a0 con 0x7fef9006c540 2023-11-26T07:12:37.664 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:37.664 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:24.102531+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:20.302489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:21.468154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 -- 172.21.15.83:0/562866083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06af60 msgr2=0x7fef7c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06af60 0x7fef7c06d420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fef900ae2e0 tx=0x7fef8001c040 comp rx=0 tx=0).stop 2023-11-26T07:12:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 -- 172.21.15.83:0/562866083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9006c540 msgr2=0x7fef9015a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9006c540 0x7fef9015a560 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fef8400ebe0 tx=0x7fef8400eef0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 -- 172.21.15.83:0/562866083 shutdown_connections 2023-11-26T07:12:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9015a030 0x7fef9006da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9006c540 0x7fef9015a560 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef7c06af60 0x7fef7c06d420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 --2- 172.21.15.83:0/562866083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef901446a0 0x7fef90159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 -- 172.21.15.83:0/562866083 >> 172.21.15.83:0/562866083 conn(0x7fef90071b90 msgr2=0x7fef90071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:37.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 -- 172.21.15.83:0/562866083 shutdown_connections 2023-11-26T07:12:37.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.664+0000 7fef9691a700 1 -- 172.21.15.83:0/562866083 wait complete. 2023-11-26T07:12:37.678 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:12:20.302489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:12:21.468154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:12:37.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:12:37.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.799+0000 7fde72cf1700 1 Processor -- start 2023-11-26T07:12:37.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.799+0000 7fde72cf1700 1 -- start start 2023-11-26T07:12:37.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.800+0000 7fde72cf1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde6c15f9e0 0x7fde6c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.800+0000 7fde72cf1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c073f90 0x7fde6c074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.800+0000 7fde72cf1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6c06b080 con 0x7fde6c0cf610 2023-11-26T07:12:37.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.800+0000 7fde72cf1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6c074960 con 0x7fde6c15f9e0 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.800+0000 7fde72cf1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c15c290 con 0x7fde6c073f90 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde70a8d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fde6c0cf610 0x7fde6c0b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56524/0 (socket says 172.21.15.83:56524) 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde70a8d700 1 -- 172.21.15.83:0/2802528678 learned_addr learned my addr 172.21.15.83:0/2802528678 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde6bfff700 1 --2- 172.21.15.83:0/2802528678 >> v2:172.21.15.83:3300/0 conn(0x7fde6c15f9e0 0x7fde6c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde6b7fe700 1 -- 172.21.15.83:0/2802528678 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610427444 0 0) 0x7fde6c06b080 con 0x7fde6c0cf610 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde6b7fe700 1 -- 172.21.15.83:0/2802528678 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde5c0035b0 con 0x7fde6c0cf610 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde7128e700 1 --2- 172.21.15.83:0/2802528678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c073f90 0x7fde6c074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde6bfff700 1 -- 172.21.15.83:0/2802528678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c073f90 msgr2=0x7fde6c074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde6bfff700 1 --2- 172.21.15.83:0/2802528678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c073f90 0x7fde6c074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde6bfff700 1 -- 172.21.15.83:0/2802528678 >> v1:172.21.15.83:6789/0 conn(0x7fde6c0cf610 legacy=0x7fde6c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde6bfff700 1 -- 172.21.15.83:0/2802528678 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c0cf9f0 con 0x7fde6c15f9e0 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde7128e700 1 --2- 172.21.15.83:0/2802528678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c073f90 0x7fde6c074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:37.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde6bfff700 1 --2- 172.21.15.83:0/2802528678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c15f9e0 0x7fde6c0add40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fde58009840 tx=0x7fde58009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aecb8656ac99b1c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde6b7fe700 1 -- 172.21.15.83:0/2802528678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58011830 con 0x7fde6c15f9e0 2023-11-26T07:12:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde6b7fe700 1 -- 172.21.15.83:0/2802528678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58011990 con 0x7fde6c15f9e0 2023-11-26T07:12:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.801+0000 7fde6b7fe700 1 -- 172.21.15.83:0/2802528678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58011d00 con 0x7fde6c15f9e0 2023-11-26T07:12:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.802+0000 7fde72cf1700 1 -- 172.21.15.83:0/2802528678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c15f9e0 msgr2=0x7fde6c0add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.802+0000 7fde72cf1700 1 --2- 172.21.15.83:0/2802528678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c15f9e0 0x7fde6c0add40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fde58009840 tx=0x7fde58009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.802+0000 7fde72cf1700 1 -- 172.21.15.83:0/2802528678 shutdown_connections 2023-11-26T07:12:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.802+0000 7fde72cf1700 1 --2- 172.21.15.83:0/2802528678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c073f90 0x7fde6c074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.802+0000 7fde72cf1700 1 --2- 172.21.15.83:0/2802528678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c15f9e0 0x7fde6c0add40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.802+0000 7fde72cf1700 1 -- 172.21.15.83:0/2802528678 >> 172.21.15.83:0/2802528678 conn(0x7fde6c0d0bf0 msgr2=0x7fde6c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.802+0000 7fde72cf1700 1 -- 172.21.15.83:0/2802528678 shutdown_connections 2023-11-26T07:12:37.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.802+0000 7fde72cf1700 1 -- 172.21.15.83:0/2802528678 wait complete. 2023-11-26T07:12:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.803+0000 7fde72cf1700 1 Processor -- start 2023-11-26T07:12:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.803+0000 7fde72cf1700 1 -- start start 2023-11-26T07:12:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.804+0000 7fde72cf1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c073f90 0x7fde6c1f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.804+0000 7fde72cf1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0cf610 0x7fde6c0b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.804+0000 7fde72cf1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c15f9e0 0x7fde6c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.804+0000 7fde72cf1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c157ba0 con 0x7fde6c0cf610 2023-11-26T07:12:37.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.804+0000 7fde72cf1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1f13a0 con 0x7fde6c073f90 2023-11-26T07:12:37.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.804+0000 7fde72cf1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1575e0 con 0x7fde6c15f9e0 2023-11-26T07:12:37.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde6bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0cf610 0x7fde6c0b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde6bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0cf610 0x7fde6c0b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36492/0 (socket says 172.21.15.83:36492) 2023-11-26T07:12:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde6bfff700 1 -- 172.21.15.83:0/2983610676 learned_addr learned my addr 172.21.15.83:0/2983610676 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde7128e700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c15f9e0 0x7fde6c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde70a8d700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c073f90 0x7fde6c1f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde6bfff700 1 -- 172.21.15.83:0/2983610676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c15f9e0 msgr2=0x7fde6c0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde6bfff700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c15f9e0 0x7fde6c0b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde6bfff700 1 -- 172.21.15.83:0/2983610676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c073f90 msgr2=0x7fde6c1f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde6bfff700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c073f90 0x7fde6c1f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde6bfff700 1 -- 172.21.15.83:0/2983610676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde580094a0 con 0x7fde6c0cf610 2023-11-26T07:12:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde70a8d700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c073f90 0x7fde6c1f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde6bfff700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0cf610 0x7fde6c0b1810 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde58009b90 tx=0x7fde58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:37.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde697fa700 1 -- 172.21.15.83:0/2983610676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde580036a0 con 0x7fde6c0cf610 2023-11-26T07:12:37.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde697fa700 1 -- 172.21.15.83:0/2983610676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5801e070 con 0x7fde6c0cf610 2023-11-26T07:12:37.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde72cf1700 1 -- 172.21.15.83:0/2983610676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c1578c0 con 0x7fde6c0cf610 2023-11-26T07:12:37.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde697fa700 1 -- 172.21.15.83:0/2983610676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58003a60 con 0x7fde6c0cf610 2023-11-26T07:12:37.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.805+0000 7fde72cf1700 1 -- 172.21.15.83:0/2983610676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c15d6a0 con 0x7fde6c0cf610 2023-11-26T07:12:37.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.806+0000 7fde72cf1700 1 -- 172.21.15.83:0/2983610676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6c15d220 con 0x7fde6c0cf610 2023-11-26T07:12:37.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.807+0000 7fde697fa700 1 -- 172.21.15.83:0/2983610676 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fde58003bc0 con 0x7fde6c0cf610 2023-11-26T07:12:37.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.807+0000 7fde697fa700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5006b080 0x7fde5006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:37.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.807+0000 7fde697fa700 1 -- 172.21.15.83:0/2983610676 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde5808a810 con 0x7fde6c0cf610 2023-11-26T07:12:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.807+0000 7fde70a8d700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5006b080 0x7fde5006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.807+0000 7fde70a8d700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5006b080 0x7fde5006d540 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fde60005910 tx=0x7fde60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:37.812+0000 7fde697fa700 1 -- 172.21.15.83:0/2983610676 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde6c15d220 con 0x7fde6c0cf610 2023-11-26T07:12:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.000+0000 7fde72cf1700 1 -- 172.21.15.83:0/2983610676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde6c15d220 con 0x7fde6c0cf610 2023-11-26T07:12:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.000+0000 7fde697fa700 1 -- 172.21.15.83:0/2983610676 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde6c15d220 con 0x7fde6c0cf610 2023-11-26T07:12:38.003 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:12:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 -- 172.21.15.83:0/2983610676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5006b080 msgr2=0x7fde5006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5006b080 0x7fde5006d540 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fde60005910 tx=0x7fde60009040 comp rx=0 tx=0).stop 2023-11-26T07:12:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 -- 172.21.15.83:0/2983610676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0cf610 msgr2=0x7fde6c0b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0cf610 0x7fde6c0b1810 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde58009b90 tx=0x7fde58004750 comp rx=0 tx=0).stop 2023-11-26T07:12:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 -- 172.21.15.83:0/2983610676 shutdown_connections 2023-11-26T07:12:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde5006b080 0x7fde5006d540 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde6c15f9e0 0x7fde6c0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde6c073f90 0x7fde6c1f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 --2- 172.21.15.83:0/2983610676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde6c0cf610 0x7fde6c0b1810 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 -- 172.21.15.83:0/2983610676 >> 172.21.15.83:0/2983610676 conn(0x7fde6c0d0bf0 msgr2=0x7fde6c0b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 -- 172.21.15.83:0/2983610676 shutdown_connections 2023-11-26T07:12:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.002+0000 7fde72cf1700 1 -- 172.21.15.83:0/2983610676 wait complete. 2023-11-26T07:12:38.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:12:38.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.138+0000 7f4d979b8700 1 Processor -- start 2023-11-26T07:12:38.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.138+0000 7f4d979b8700 1 -- start start 2023-11-26T07:12:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d979b8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4d9015bf50 0x7f4d900ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d979b8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4d900ce830 0x7f4d900cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d979b8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d900d0140 con 0x7f4d900cf1f0 2023-11-26T07:12:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d979b8700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d9015c330 con 0x7f4d9015bf50 2023-11-26T07:12:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d979b8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d900d03e0 con 0x7f4d900ce830 2023-11-26T07:12:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d95754700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4d9015bf50 0x7f4d900ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d95754700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4d9015bf50 0x7f4d900ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36498/0 (socket says 172.21.15.83:36498) 2023-11-26T07:12:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d95754700 1 -- 172.21.15.83:0/1466820626 learned_addr learned my addr 172.21.15.83:0/1466820626 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d94f53700 1 --2- 172.21.15.83:0/1466820626 >> v2:172.21.15.203:3300/0 conn(0x7f4d900ce830 0x7f4d900cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d95754700 1 -- 172.21.15.83:0/1466820626 >> v2:172.21.15.203:3300/0 conn(0x7f4d900ce830 msgr2=0x7f4d900cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d95754700 1 --2- 172.21.15.83:0/1466820626 >> v2:172.21.15.203:3300/0 conn(0x7f4d900ce830 0x7f4d900cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d95754700 1 -- 172.21.15.83:0/1466820626 >> v1:172.21.15.83:6789/0 conn(0x7f4d900cf1f0 legacy=0x7f4d900c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.139+0000 7f4d95754700 1 -- 172.21.15.83:0/1466820626 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d900d06c0 con 0x7f4d9015bf50 2023-11-26T07:12:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d95754700 1 --2- 172.21.15.83:0/1466820626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d9015bf50 0x7f4d900ce2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4d90072420 tx=0x7f4d80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706d19045358445e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d87fff700 1 -- 172.21.15.83:0/1466820626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000b870 con 0x7f4d9015bf50 2023-11-26T07:12:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d87fff700 1 -- 172.21.15.83:0/1466820626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8000b9d0 con 0x7f4d9015bf50 2023-11-26T07:12:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d87fff700 1 -- 172.21.15.83:0/1466820626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000bd60 con 0x7f4d9015bf50 2023-11-26T07:12:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d979b8700 1 -- 172.21.15.83:0/1466820626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d9015bf50 msgr2=0x7f4d900ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d979b8700 1 --2- 172.21.15.83:0/1466820626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d9015bf50 0x7f4d900ce2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4d90072420 tx=0x7f4d80009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d979b8700 1 -- 172.21.15.83:0/1466820626 shutdown_connections 2023-11-26T07:12:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d979b8700 1 --2- 172.21.15.83:0/1466820626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d9015bf50 0x7f4d900ce2f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d979b8700 1 --2- 172.21.15.83:0/1466820626 >> v2:172.21.15.203:3300/0 conn(0x7f4d900ce830 0x7f4d900cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d979b8700 1 -- 172.21.15.83:0/1466820626 >> 172.21.15.83:0/1466820626 conn(0x7f4d900b3780 msgr2=0x7f4d900b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d979b8700 1 -- 172.21.15.83:0/1466820626 shutdown_connections 2023-11-26T07:12:38.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.140+0000 7f4d979b8700 1 -- 172.21.15.83:0/1466820626 wait complete. 2023-11-26T07:12:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.141+0000 7f4d979b8700 1 Processor -- start 2023-11-26T07:12:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.142+0000 7f4d979b8700 1 -- start start 2023-11-26T07:12:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.142+0000 7f4d979b8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d900ce830 0x7f4d9014b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.142+0000 7f4d979b8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d900cf1f0 0x7f4d900c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.142+0000 7f4d979b8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d9015bf50 0x7f4d900c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.142+0000 7f4d979b8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d900c6e00 con 0x7f4d900cf1f0 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.142+0000 7f4d979b8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d900c91c0 con 0x7f4d900ce830 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.142+0000 7f4d979b8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d900ca000 con 0x7f4d9015bf50 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.142+0000 7f4d94f53700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d900cf1f0 0x7f4d900c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.142+0000 7f4d94f53700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d900cf1f0 0x7f4d900c8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36506/0 (socket says 172.21.15.83:36506) 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.142+0000 7f4d94f53700 1 -- 172.21.15.83:0/381003117 learned_addr learned my addr 172.21.15.83:0/381003117 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.142+0000 7f4d95754700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d900ce830 0x7f4d9014b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d95f55700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d9015bf50 0x7f4d900c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d94f53700 1 -- 172.21.15.83:0/381003117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d9015bf50 msgr2=0x7f4d900c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d94f53700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d9015bf50 0x7f4d900c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d94f53700 1 -- 172.21.15.83:0/381003117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d900ce830 msgr2=0x7f4d9014b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d94f53700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d900ce830 0x7f4d9014b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d94f53700 1 -- 172.21.15.83:0/381003117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d80009550 con 0x7f4d900cf1f0 2023-11-26T07:12:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d95754700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d900ce830 0x7f4d9014b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d94f53700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d900cf1f0 0x7f4d900c8c80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00e810 tx=0x7f4d8c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d85ffb700 1 -- 172.21.15.83:0/381003117 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c013610 con 0x7f4d900cf1f0 2023-11-26T07:12:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d85ffb700 1 -- 172.21.15.83:0/381003117 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c013770 con 0x7f4d900cf1f0 2023-11-26T07:12:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d85ffb700 1 -- 172.21.15.83:0/381003117 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c013b20 con 0x7f4d900cf1f0 2023-11-26T07:12:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d979b8700 1 -- 172.21.15.83:0/381003117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d900c93f0 con 0x7f4d900cf1f0 2023-11-26T07:12:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.143+0000 7f4d979b8700 1 -- 172.21.15.83:0/381003117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d900c99b0 con 0x7f4d900cf1f0 2023-11-26T07:12:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.144+0000 7f4d979b8700 1 -- 172.21.15.83:0/381003117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d90068fb0 con 0x7f4d900cf1f0 2023-11-26T07:12:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.145+0000 7f4d85ffb700 1 -- 172.21.15.83:0/381003117 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4d8c013c80 con 0x7f4d900cf1f0 2023-11-26T07:12:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.145+0000 7f4d85ffb700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4d7406b080 0x7f4d7406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.145+0000 7f4d85ffb700 1 -- 172.21.15.83:0/381003117 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d8c0896d0 con 0x7f4d900cf1f0 2023-11-26T07:12:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.145+0000 7f4d95754700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4d7406b080 0x7f4d7406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.145+0000 7f4d95754700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4d7406b080 0x7f4d7406d540 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4d80014040 tx=0x7f4d80008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.149+0000 7f4d85ffb700 1 -- 172.21.15.83:0/381003117 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d8c054400 con 0x7f4d900cf1f0 2023-11-26T07:12:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.339+0000 7f4d979b8700 1 -- 172.21.15.83:0/381003117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d900ca140 con 0x7f4d900cf1f0 2023-11-26T07:12:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.339+0000 7f4d85ffb700 1 -- 172.21.15.83:0/381003117 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d8c057a20 con 0x7f4d900cf1f0 2023-11-26T07:12:38.342 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:12:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 -- 172.21.15.83:0/381003117 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4d7406b080 msgr2=0x7f4d7406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4d7406b080 0x7f4d7406d540 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4d80014040 tx=0x7f4d80008030 comp rx=0 tx=0).stop 2023-11-26T07:12:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 -- 172.21.15.83:0/381003117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d900cf1f0 msgr2=0x7f4d900c8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d900cf1f0 0x7f4d900c8c80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00e810 tx=0x7f4d8c00ebd0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 -- 172.21.15.83:0/381003117 shutdown_connections 2023-11-26T07:12:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4d7406b080 0x7f4d7406d540 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d9015bf50 0x7f4d900c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d900ce830 0x7f4d9014b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 --2- 172.21.15.83:0/381003117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d900cf1f0 0x7f4d900c8c80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 -- 172.21.15.83:0/381003117 >> 172.21.15.83:0/381003117 conn(0x7f4d900b3780 msgr2=0x7f4d900b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 -- 172.21.15.83:0/381003117 shutdown_connections 2023-11-26T07:12:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.342+0000 7f4d979b8700 1 -- 172.21.15.83:0/381003117 wait complete. 2023-11-26T07:12:38.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:12:38.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.477+0000 7f4fec693700 1 Processor -- start 2023-11-26T07:12:38.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.477+0000 7f4fec693700 1 -- start start 2023-11-26T07:12:38.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fec693700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4fe415a620 0x7f4fe40ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fec693700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4fe40cc540 0x7f4fe406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fec693700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe406eb20 0x7f4fe406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fec693700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe406ef90 con 0x7f4fe40cc540 2023-11-26T07:12:38.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fec693700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40b2f20 con 0x7f4fe415a620 2023-11-26T07:12:38.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fec693700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40ce690 con 0x7f4fe406eb20 2023-11-26T07:12:38.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fe9c2e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4fe40cc540 0x7f4fe406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fe9c2e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4fe40cc540 0x7f4fe406e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36514/0 (socket says 172.21.15.83:36514) 2023-11-26T07:12:38.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fe9c2e700 1 -- 172.21.15.83:0/945628306 learned_addr learned my addr 172.21.15.83:0/945628306 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:38.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4feac30700 1 --2- 172.21.15.83:0/945628306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe406eb20 0x7f4fe406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fea42f700 1 --2- 172.21.15.83:0/945628306 >> v2:172.21.15.203:3300/0 conn(0x7f4fe415a620 0x7f4fe40ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fe9c2e700 1 -- 172.21.15.83:0/945628306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe406eb20 msgr2=0x7f4fe406d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:38.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fe9c2e700 1 --2- 172.21.15.83:0/945628306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe406eb20 0x7f4fe406d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fe9c2e700 1 -- 172.21.15.83:0/945628306 >> v2:172.21.15.203:3300/0 conn(0x7f4fe415a620 msgr2=0x7f4fe40ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:38.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fe9c2e700 1 --2- 172.21.15.83:0/945628306 >> v2:172.21.15.203:3300/0 conn(0x7f4fe415a620 0x7f4fe40ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fe9c2e700 1 -- 172.21.15.83:0/945628306 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe415aa00 con 0x7f4fe40cc540 2023-11-26T07:12:38.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fe9c2e700 1 --2- 172.21.15.83:0/945628306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe40cc540 0x7f4fe406e5e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4072800 tx=0x7f4fd4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae43176eb488b69 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.478+0000 7f4fe942d700 1 -- 172.21.15.83:0/945628306 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400b870 con 0x7f4fe40cc540 2023-11-26T07:12:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.479+0000 7f4fe942d700 1 -- 172.21.15.83:0/945628306 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd400b9d0 con 0x7f4fe40cc540 2023-11-26T07:12:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.479+0000 7f4fe942d700 1 -- 172.21.15.83:0/945628306 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400bd60 con 0x7f4fe40cc540 2023-11-26T07:12:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.479+0000 7f4fec693700 1 -- 172.21.15.83:0/945628306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe40cc540 msgr2=0x7f4fe406e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.479+0000 7f4fec693700 1 --2- 172.21.15.83:0/945628306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe40cc540 0x7f4fe406e5e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4072800 tx=0x7f4fd4009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.479+0000 7f4fec693700 1 -- 172.21.15.83:0/945628306 shutdown_connections 2023-11-26T07:12:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.479+0000 7f4fec693700 1 --2- 172.21.15.83:0/945628306 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe406eb20 0x7f4fe406d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.479+0000 7f4fec693700 1 --2- 172.21.15.83:0/945628306 >> v2:172.21.15.203:3300/0 conn(0x7f4fe415a620 0x7f4fe40ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.479+0000 7f4fec693700 1 --2- 172.21.15.83:0/945628306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe40cc540 0x7f4fe406e5e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.479+0000 7f4fec693700 1 -- 172.21.15.83:0/945628306 >> 172.21.15.83:0/945628306 conn(0x7f4fe40b2050 msgr2=0x7f4fe40b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:38.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.479+0000 7f4fec693700 1 -- 172.21.15.83:0/945628306 shutdown_connections 2023-11-26T07:12:38.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.479+0000 7f4fec693700 1 -- 172.21.15.83:0/945628306 wait complete. 2023-11-26T07:12:38.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.480+0000 7f4fec693700 1 Processor -- start 2023-11-26T07:12:38.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.480+0000 7f4fec693700 1 -- start start 2023-11-26T07:12:38.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fec693700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fe406eb20 0x7f4fe415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fec693700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe40cc540 0x7f4fe4152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fec693700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe415a620 0x7f4fe415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fec693700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40ad870 con 0x7f4fe40cc540 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fec693700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe415b140 con 0x7f4fe406eb20 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fec693700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4153310 con 0x7f4fe415a620 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fe9c2e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe40cc540 0x7f4fe4152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fe9c2e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe40cc540 0x7f4fe4152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36520/0 (socket says 172.21.15.83:36520) 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fe9c2e700 1 -- 172.21.15.83:0/115635256 learned_addr learned my addr 172.21.15.83:0/115635256 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fe9c2e700 1 -- 172.21.15.83:0/115635256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe415a620 msgr2=0x7f4fe415cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fe9c2e700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe415a620 0x7f4fe415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fe9c2e700 1 -- 172.21.15.83:0/115635256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fe406eb20 msgr2=0x7f4fe415ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fe9c2e700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fe406eb20 0x7f4fe415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.481+0000 7f4fe9c2e700 1 -- 172.21.15.83:0/115635256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4009550 con 0x7f4fe40cc540 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.482+0000 7f4fe9c2e700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe40cc540 0x7f4fe4152dd0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4014040 tx=0x7f4fd4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.482+0000 7f4fdaffd700 1 -- 172.21.15.83:0/115635256 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4004520 con 0x7f4fe40cc540 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.482+0000 7f4fdaffd700 1 -- 172.21.15.83:0/115635256 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd401e070 con 0x7f4fe40cc540 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.482+0000 7f4fdaffd700 1 -- 172.21.15.83:0/115635256 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40038c0 con 0x7f4fe40cc540 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.482+0000 7f4fec693700 1 -- 172.21.15.83:0/115635256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe41f4140 con 0x7f4fe40cc540 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.482+0000 7f4fec693700 1 -- 172.21.15.83:0/115635256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe41f4700 con 0x7f4fe40cc540 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.483+0000 7f4fec693700 1 -- 172.21.15.83:0/115635256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe4068d90 con 0x7f4fe40cc540 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.483+0000 7f4fdaffd700 1 -- 172.21.15.83:0/115635256 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4fd4003a20 con 0x7f4fe40cc540 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.483+0000 7f4fdaffd700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4fc806b080 0x7f4fc806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.483+0000 7f4fdaffd700 1 -- 172.21.15.83:0/115635256 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fd4020070 con 0x7f4fe40cc540 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.483+0000 7f4fea42f700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4fc806b080 0x7f4fc806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.484+0000 7f4fea42f700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4fc806b080 0x7f4fc806d540 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40aeb30 tx=0x7f4fe0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:38.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.488+0000 7f4fdaffd700 1 -- 172.21.15.83:0/115635256 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4fd4055d10 con 0x7f4fe40cc540 2023-11-26T07:12:38.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.678+0000 7f4fec693700 1 -- 172.21.15.83:0/115635256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fe40ad9e0 con 0x7f4fe40cc540 2023-11-26T07:12:38.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.678+0000 7f4fdaffd700 1 -- 172.21.15.83:0/115635256 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fe40ad9e0 con 0x7f4fe40cc540 2023-11-26T07:12:38.681 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:12:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.680+0000 7f4fec693700 1 -- 172.21.15.83:0/115635256 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4fc806b080 msgr2=0x7f4fc806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.680+0000 7f4fec693700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4fc806b080 0x7f4fc806d540 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40aeb30 tx=0x7f4fe0008040 comp rx=0 tx=0).stop 2023-11-26T07:12:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.680+0000 7f4fec693700 1 -- 172.21.15.83:0/115635256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe40cc540 msgr2=0x7f4fe4152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.680+0000 7f4fec693700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe40cc540 0x7f4fe4152dd0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4014040 tx=0x7f4fd4004750 comp rx=0 tx=0).stop 2023-11-26T07:12:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.681+0000 7f4fec693700 1 -- 172.21.15.83:0/115635256 shutdown_connections 2023-11-26T07:12:38.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.681+0000 7f4fec693700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4fc806b080 0x7f4fc806d540 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.681+0000 7f4fec693700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4fe415a620 0x7f4fe415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.681+0000 7f4fec693700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fe406eb20 0x7f4fe415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.681+0000 7f4fec693700 1 --2- 172.21.15.83:0/115635256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe40cc540 0x7f4fe4152dd0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.681+0000 7f4fec693700 1 -- 172.21.15.83:0/115635256 >> 172.21.15.83:0/115635256 conn(0x7f4fe40b2050 msgr2=0x7f4fe40b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.681+0000 7f4fec693700 1 -- 172.21.15.83:0/115635256 shutdown_connections 2023-11-26T07:12:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.681+0000 7f4fec693700 1 -- 172.21.15.83:0/115635256 wait complete. 2023-11-26T07:12:38.695 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:12:38.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.815+0000 7f79e3881700 1 Processor -- start 2023-11-26T07:12:38.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.815+0000 7f79e3881700 1 -- start start 2023-11-26T07:12:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79e3881700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f79dc15e080 0x7f79dc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79e3881700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79dc0ae430 con 0x7f79dc0af4d0 2023-11-26T07:12:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79e3881700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79dc0ae710 con 0x7f79dc06c710 2023-11-26T07:12:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79e3881700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc0af0b0 con 0x7f79dc15e080 2023-11-26T07:12:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79e0e1c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f79dc0af4d0 0x7f79dc06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56566/0 (socket says 172.21.15.83:56566) 2023-11-26T07:12:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79e0e1c700 1 -- 172.21.15.83:0/600723663 learned_addr learned my addr 172.21.15.83:0/600723663 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79e161d700 1 --2- 172.21.15.83:0/600723663 >> v2:172.21.15.203:3300/0 conn(0x7f79dc15e080 0x7f79dc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596920584 0 0) 0x7f79dc0ae430 con 0x7f79dc0af4d0 2023-11-26T07:12:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c80035b0 con 0x7f79dc0af4d0 2023-11-26T07:12:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2379038890 0 0) 0x7f79c80035b0 con 0x7f79dc0af4d0 2023-11-26T07:12:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 >> v2:172.21.15.203:3300/0 conn(0x7f79dc15e080 msgr2=0x7f79dc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79d3fff700 1 --2- 172.21.15.83:0/600723663 >> v2:172.21.15.203:3300/0 conn(0x7f79dc15e080 0x7f79dc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 >> v1:172.21.15.203:6789/0 conn(0x7f79dc06c710 legacy=0x7f79dc06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc0ae9f0 con 0x7f79dc0af4d0 2023-11-26T07:12:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f79cc001670 con 0x7f79dc0af4d0 2023-11-26T07:12:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.816+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 >> v1:172.21.15.83:6789/0 conn(0x7f79dc0af4d0 legacy=0x7f79dc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79d3fff700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c8003a00 0x7f79c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79d3fff700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c8004850 0x7f79c8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79d3fff700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c8005210 0x7f79c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0af0b0 con 0x7f79c8003a00 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c800c0b0 con 0x7f79c8004850 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c800c270 con 0x7f79c8005210 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79e161d700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c8003a00 0x7f79c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79e0e1c700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c8004850 0x7f79c8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79e1e1e700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c8005210 0x7f79c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79e161d700 1 -- 172.21.15.83:0/600723663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c8005210 msgr2=0x7f79c800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79e161d700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79c8005210 0x7f79c800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79e161d700 1 -- 172.21.15.83:0/600723663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c8004850 msgr2=0x7f79c8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79e161d700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c8004850 0x7f79c8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79e161d700 1 -- 172.21.15.83:0/600723663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c80035b0 con 0x7f79c8003a00 2023-11-26T07:12:38.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79e0e1c700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c8004850 0x7f79c8004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79e1e1e700 1 -- 172.21.15.83:0/600723663 reap_dead start 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79e161d700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c8003a00 0x7f79c8003e20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009b60 tx=0x7f79c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5ca339184e0d5b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d800e6c0 con 0x7f79c8003a00 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d800e820 con 0x7f79c8003a00 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.817+0000 7f79d3fff700 1 -- 172.21.15.83:0/600723663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d800eb90 con 0x7f79c8003a00 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.818+0000 7f79e3881700 1 -- 172.21.15.83:0/600723663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c8003a00 msgr2=0x7f79c8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.818+0000 7f79e3881700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c8003a00 0x7f79c8003e20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009b60 tx=0x7f79c8005880 comp rx=0 tx=0).stop 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.818+0000 7f79e3881700 1 -- 172.21.15.83:0/600723663 shutdown_connections 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.818+0000 7f79e3881700 1 --2- 172.21.15.83:0/600723663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79c8003a00 0x7f79c8003e20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.818+0000 7f79e3881700 1 -- 172.21.15.83:0/600723663 >> 172.21.15.83:0/600723663 conn(0x7f79dc071b90 msgr2=0x7f79dc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.818+0000 7f79e3881700 1 -- 172.21.15.83:0/600723663 shutdown_connections 2023-11-26T07:12:38.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.818+0000 7f79e3881700 1 -- 172.21.15.83:0/600723663 wait complete. 2023-11-26T07:12:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.819+0000 7f79e3881700 1 Processor -- start 2023-11-26T07:12:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.819+0000 7f79e3881700 1 -- start start 2023-11-26T07:12:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.820+0000 7f79e3881700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c8003a00 0x7f79dc146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.820+0000 7f79e3881700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79dc155b00 0x7f79dc153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.820+0000 7f79e3881700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79dc154430 0x7f79dc0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.820+0000 7f79e3881700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc1556e0 con 0x7f79dc154430 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.820+0000 7f79e3881700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc1572e0 con 0x7f79c8003a00 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.820+0000 7f79e3881700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc160170 con 0x7f79dc155b00 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.820+0000 7f79e1e1e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79dc154430 0x7f79dc0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.820+0000 7f79e1e1e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79dc154430 0x7f79dc0b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36542/0 (socket says 172.21.15.83:36542) 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.820+0000 7f79e1e1e700 1 -- 172.21.15.83:0/2428476870 learned_addr learned my addr 172.21.15.83:0/2428476870 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79e0e1c700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79dc155b00 0x7f79dc153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79e161d700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c8003a00 0x7f79dc146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79e1e1e700 1 -- 172.21.15.83:0/2428476870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79dc155b00 msgr2=0x7f79dc153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79e1e1e700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79dc155b00 0x7f79dc153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79e1e1e700 1 -- 172.21.15.83:0/2428476870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c8003a00 msgr2=0x7f79dc146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79e1e1e700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c8003a00 0x7f79dc146700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:38.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79e1e1e700 1 -- 172.21.15.83:0/2428476870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc154a60 con 0x7f79dc154430 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79e1e1e700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79dc154430 0x7f79dc0b0fb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f79dc161a70 tx=0x7f79d400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79d1ffb700 1 -- 172.21.15.83:0/2428476870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4003c10 con 0x7f79dc154430 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79d1ffb700 1 -- 172.21.15.83:0/2428476870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d4003d70 con 0x7f79dc154430 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79e3881700 1 -- 172.21.15.83:0/2428476870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc154d40 con 0x7f79dc154430 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79d1ffb700 1 -- 172.21.15.83:0/2428476870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4005310 con 0x7f79dc154430 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.821+0000 7f79e3881700 1 -- 172.21.15.83:0/2428476870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc155300 con 0x7f79dc154430 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.822+0000 7f79e3881700 1 -- 172.21.15.83:0/2428476870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc0753f0 con 0x7f79dc154430 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.823+0000 7f79d1ffb700 1 -- 172.21.15.83:0/2428476870 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f79d4005470 con 0x7f79dc154430 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.823+0000 7f79d1ffb700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79c006b080 0x7f79c006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.823+0000 7f79d1ffb700 1 -- 172.21.15.83:0/2428476870 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79d4089750 con 0x7f79dc154430 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.823+0000 7f79e161d700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79c006b080 0x7f79c006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:38.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.823+0000 7f79e161d700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79c006b080 0x7f79c006d540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f79d80090b0 tx=0x7f79d801e040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:38.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:38.827+0000 7f79d1ffb700 1 -- 172.21.15.83:0/2428476870 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79d4054610 con 0x7f79dc154430 2023-11-26T07:12:39.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.032+0000 7f79e3881700 1 -- 172.21.15.83:0/2428476870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f79dc06c580 con 0x7f79dc154430 2023-11-26T07:12:39.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.032+0000 7f79d1ffb700 1 -- 172.21.15.83:0/2428476870 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7f79d4057c30 con 0x7f79dc154430 2023-11-26T07:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 -- 172.21.15.83:0/2428476870 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79c006b080 msgr2=0x7f79c006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79c006b080 0x7f79c006d540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f79d80090b0 tx=0x7f79d801e040 comp rx=0 tx=0).stop 2023-11-26T07:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 -- 172.21.15.83:0/2428476870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79dc154430 msgr2=0x7f79dc0b0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79dc154430 0x7f79dc0b0fb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f79dc161a70 tx=0x7f79d400edd0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 -- 172.21.15.83:0/2428476870 shutdown_connections 2023-11-26T07:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79dc154430 0x7f79dc0b0fb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79c8003a00 0x7f79dc146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79c006b080 0x7f79c006d540 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 --2- 172.21.15.83:0/2428476870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79dc155b00 0x7f79dc153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 -- 172.21.15.83:0/2428476870 >> 172.21.15.83:0/2428476870 conn(0x7f79dc071b90 msgr2=0x7f79dc073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 -- 172.21.15.83:0/2428476870 shutdown_connections 2023-11-26T07:12:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.034+0000 7f79e3881700 1 -- 172.21.15.83:0/2428476870 wait complete. 2023-11-26T07:12:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:12:39.048 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:12:39.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:39.049 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:12:39.100 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:12:39.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:12:39.100 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:12:39.120 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:39.120 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:39.137 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:12:39.137 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:39.137 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:39.179 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:12:39.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.297+0000 7fbd8cd9a700 1 Processor -- start 2023-11-26T07:12:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.297+0000 7fbd8cd9a700 1 -- start start 2023-11-26T07:12:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.298+0000 7fbd8cd9a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd880d2240 0x7fbd880b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.298+0000 7fbd8cd9a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd880b1d40 con 0x7fbd8815ddd0 2023-11-26T07:12:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.298+0000 7fbd8cd9a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd880b28c0 con 0x7fbd8815e1b0 2023-11-26T07:12:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.298+0000 7fbd8cd9a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd880b1f90 con 0x7fbd880d2240 2023-11-26T07:12:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8659c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbd8815ddd0 0x7fbd880d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56586/0 (socket says 172.21.15.83:56586) 2023-11-26T07:12:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8659c700 1 -- 172.21.15.83:0/2755237154 learned_addr learned my addr 172.21.15.83:0/2755237154 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801915924 0 0) 0x7fbd880b1d40 con 0x7fbd8815ddd0 2023-11-26T07:12:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd85d9b700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd880d2240 0x7fbd880b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd740035b0 con 0x7fbd8815ddd0 2023-11-26T07:12:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2465069716 0 0) 0x7fbd740035b0 con 0x7fbd8815ddd0 2023-11-26T07:12:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd880d2240 msgr2=0x7fbd880b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd880d2240 0x7fbd880b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 >> v1:172.21.15.203:6789/0 conn(0x7fbd8815e1b0 legacy=0x7fbd8806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd880b2270 con 0x7fbd8815ddd0 2023-11-26T07:12:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fbd70001670 con 0x7fbd8815ddd0 2023-11-26T07:12:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 >> v1:172.21.15.83:6789/0 conn(0x7fbd8815ddd0 legacy=0x7fbd880d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd880d2240 0x7fbd74003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd74003fb0 0x7fbd74004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd74004d90 0x7fbd74005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd880b1f90 con 0x7fbd74003fb0 2023-11-26T07:12:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7400bc50 con 0x7fbd74004d90 2023-11-26T07:12:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7400be10 con 0x7fbd880d2240 2023-11-26T07:12:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd85d9b700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd74003fb0 0x7fbd74004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd8659c700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd880d2240 0x7fbd74003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.299+0000 7fbd86d9d700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd74004d90 0x7fbd74005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd85d9b700 1 -- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd880d2240 msgr2=0x7fbd74003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd85d9b700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd880d2240 0x7fbd74003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd85d9b700 1 -- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd74004d90 msgr2=0x7fbd74005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd85d9b700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd74004d90 0x7fbd74005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd85d9b700 1 -- 172.21.15.83:0/2755237154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd740035b0 con 0x7fbd74003fb0 2023-11-26T07:12:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd86d9d700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd74004d90 0x7fbd74005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd85d9b700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd74003fb0 0x7fbd74004850 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009b60 tx=0x7fbd74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b388c30ce15ace1b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7800e6c0 con 0x7fbd74003fb0 2023-11-26T07:12:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7800e820 con 0x7fbd74003fb0 2023-11-26T07:12:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd8559a700 1 -- 172.21.15.83:0/2755237154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7800eb90 con 0x7fbd74003fb0 2023-11-26T07:12:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/2755237154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd74003fb0 msgr2=0x7fbd74004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd8cd9a700 1 --2- 172.21.15.83:0/2755237154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd74003fb0 0x7fbd74004850 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009b60 tx=0x7fbd74005880 comp rx=0 tx=0).stop 2023-11-26T07:12:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd86d9d700 1 -- 172.21.15.83:0/2755237154 reap_dead start 2023-11-26T07:12:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/2755237154 shutdown_connections 2023-11-26T07:12:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/2755237154 >> 172.21.15.83:0/2755237154 conn(0x7fbd880731a0 msgr2=0x7fbd8809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.300+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/2755237154 shutdown_connections 2023-11-26T07:12:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.301+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/2755237154 wait complete. 2023-11-26T07:12:39.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.302+0000 7fbd8cd9a700 1 Processor -- start 2023-11-26T07:12:39.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.302+0000 7fbd8cd9a700 1 -- start start 2023-11-26T07:12:39.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd8cd9a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd74003fb0 0x7fbd881552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd8cd9a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd881565b0 0x7fbd8806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd8cd9a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd8806b530 0x7fbd88160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd8cd9a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8806b9e0 con 0x7fbd8806b530 2023-11-26T07:12:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd8cd9a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd881f7c70 con 0x7fbd74003fb0 2023-11-26T07:12:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd8cd9a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd881f7f50 con 0x7fbd881565b0 2023-11-26T07:12:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd86d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd8806b530 0x7fbd88160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd86d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd8806b530 0x7fbd88160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36560/0 (socket says 172.21.15.83:36560) 2023-11-26T07:12:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd86d9d700 1 -- 172.21.15.83:0/344108151 learned_addr learned my addr 172.21.15.83:0/344108151 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd8659c700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd74003fb0 0x7fbd881552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd85d9b700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd881565b0 0x7fbd8806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd86d9d700 1 -- 172.21.15.83:0/344108151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd881565b0 msgr2=0x7fbd8806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.303+0000 7fbd86d9d700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd881565b0 0x7fbd8806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.304+0000 7fbd86d9d700 1 -- 172.21.15.83:0/344108151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd74003fb0 msgr2=0x7fbd881552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.304+0000 7fbd86d9d700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd74003fb0 0x7fbd881552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.304+0000 7fbd86d9d700 1 -- 172.21.15.83:0/344108151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd880b2ed0 con 0x7fbd8806b530 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.304+0000 7fbd86d9d700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd8806b530 0x7fbd88160ba0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbd8815e590 tx=0x7fbd8000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.304+0000 7fbd7effd700 1 -- 172.21.15.83:0/344108151 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80003c10 con 0x7fbd8806b530 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.304+0000 7fbd7effd700 1 -- 172.21.15.83:0/344108151 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd80003d90 con 0x7fbd8806b530 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.304+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/344108151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd881f8230 con 0x7fbd8806b530 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.304+0000 7fbd7effd700 1 -- 172.21.15.83:0/344108151 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80005310 con 0x7fbd8806b530 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.304+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/344108151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd881f87f0 con 0x7fbd8806b530 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.305+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/344108151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd881f89b0 con 0x7fbd8806b530 2023-11-26T07:12:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.305+0000 7fbd7effd700 1 -- 172.21.15.83:0/344108151 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fbd80005470 con 0x7fbd8806b530 2023-11-26T07:12:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.306+0000 7fbd7effd700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbd6806b080 0x7fbd6806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.306+0000 7fbd7effd700 1 -- 172.21.15.83:0/344108151 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd800897d0 con 0x7fbd8806b530 2023-11-26T07:12:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.306+0000 7fbd8659c700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbd6806b080 0x7fbd6806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.306+0000 7fbd8659c700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbd6806b080 0x7fbd6806d540 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd70001090 tx=0x7fbd70001020 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.310+0000 7fbd7effd700 1 -- 172.21.15.83:0/344108151 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd80054690 con 0x7fbd8806b530 2023-11-26T07:12:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.542+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/344108151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd880b1a10 con 0x7fbd8806b530 2023-11-26T07:12:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.543+0000 7fbd7effd700 1 -- 172.21.15.83:0/344108151 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fbd80001b50 con 0x7fbd8806b530 2023-11-26T07:12:39.546 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:39.546 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/486975373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":486975373},{"type":"v1","addr":"172.21.15.203:6834","nonce":486975373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3188343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3188343240},{"type":"v1","addr":"172.21.15.83:6836","nonce":3188343240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4271462644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4271462644},{"type":"v1","addr":"172.21.15.83:6837","nonce":4271462644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:22.505426+0000","modified":"2023-11-26T07:12:25.894494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6835/995330476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":995330476},{"type":"v1","addr":"172.21.15.203:6835","nonce":995330476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:12:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/344108151 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbd6806b080 msgr2=0x7fbd6806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbd6806b080 0x7fbd6806d540 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd70001090 tx=0x7fbd70001020 comp rx=0 tx=0).stop 2023-11-26T07:12:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/344108151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd8806b530 msgr2=0x7fbd88160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd8806b530 0x7fbd88160ba0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbd8815e590 tx=0x7fbd8000edc0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/344108151 shutdown_connections 2023-11-26T07:12:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd8806b530 0x7fbd88160ba0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd74003fb0 0x7fbd881552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbd6806b080 0x7fbd6806d540 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 --2- 172.21.15.83:0/344108151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbd881565b0 0x7fbd8806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/344108151 >> 172.21.15.83:0/344108151 conn(0x7fbd880731a0 msgr2=0x7fbd8806be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/344108151 shutdown_connections 2023-11-26T07:12:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.545+0000 7fbd8cd9a700 1 -- 172.21.15.83:0/344108151 wait complete. 2023-11-26T07:12:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2023-11-26T07:12:39.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:39.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.677+0000 7fd3f765e700 1 Processor -- start 2023-11-26T07:12:39.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.677+0000 7fd3f765e700 1 -- start start 2023-11-26T07:12:39.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.678+0000 7fd3f765e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd3f00d1570 0x7fd3f00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.678+0000 7fd3f765e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3f00b2fc0 0x7fd3f00d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.678+0000 7fd3f765e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f00d2d10 con 0x7fd3f015dfc0 2023-11-26T07:12:39.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.678+0000 7fd3f765e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f00cefd0 con 0x7fd3f00d1570 2023-11-26T07:12:39.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.678+0000 7fd3f765e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00cf2b0 con 0x7fd3f00b2fc0 2023-11-26T07:12:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.678+0000 7fd3f53fa700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd3f015dfc0 0x7fd3f00d0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56606/0 (socket says 172.21.15.83:56606) 2023-11-26T07:12:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.678+0000 7fd3f53fa700 1 -- 172.21.15.83:0/3060944508 learned_addr learned my addr 172.21.15.83:0/3060944508 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f5bfb700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3f00b2fc0 0x7fd3f00d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f4bf9700 1 --2- 172.21.15.83:0/3060944508 >> v2:172.21.15.203:3300/0 conn(0x7fd3f00d1570 0x7fd3f00d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324686481 0 0) 0x7fd3f00d2d10 con 0x7fd3f015dfc0 2023-11-26T07:12:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3dc0035b0 con 0x7fd3f015dfc0 2023-11-26T07:12:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1301659351 0 0) 0x7fd3dc0035b0 con 0x7fd3f015dfc0 2023-11-26T07:12:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3f00b2fc0 msgr2=0x7fd3f00d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3f00b2fc0 0x7fd3f00d3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 >> v2:172.21.15.203:3300/0 conn(0x7fd3f00d1570 msgr2=0x7fd3f00d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:39.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 --2- 172.21.15.83:0/3060944508 >> v2:172.21.15.203:3300/0 conn(0x7fd3f00d1570 0x7fd3f00d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f00d2fb0 con 0x7fd3f015dfc0 2023-11-26T07:12:39.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fd3e0001670 con 0x7fd3f015dfc0 2023-11-26T07:12:39.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 >> v1:172.21.15.83:6789/0 conn(0x7fd3f015dfc0 legacy=0x7fd3f00d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc003a00 0x7fd3dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3f00b2fc0 0x7fd3dc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3dc004d90 0x7fd3dc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00cefd0 con 0x7fd3dc003a00 2023-11-26T07:12:39.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00cf2b0 con 0x7fd3dc004d90 2023-11-26T07:12:39.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc00ba40 con 0x7fd3f00b2fc0 2023-11-26T07:12:39.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f53fa700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc003a00 0x7fd3dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f4bf9700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3f00b2fc0 0x7fd3dc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f5bfb700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3dc004d90 0x7fd3dc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f53fa700 1 -- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3f00b2fc0 msgr2=0x7fd3dc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:39.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f53fa700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3f00b2fc0 0x7fd3dc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f53fa700 1 -- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3dc004d90 msgr2=0x7fd3dc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:39.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f53fa700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3dc004d90 0x7fd3dc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f53fa700 1 -- 172.21.15.83:0/3060944508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc0035b0 con 0x7fd3dc003a00 2023-11-26T07:12:39.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f5bfb700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3dc004d90 0x7fd3dc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:39.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.679+0000 7fd3f53fa700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc003a00 0x7fd3dc003e20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000c370 tx=0x7fd3dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7ce24e4feb6944 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:39.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.680+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0007c60 con 0x7fd3dc003a00 2023-11-26T07:12:39.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.680+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e0005960 con 0x7fd3dc003a00 2023-11-26T07:12:39.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.680+0000 7fd3e7fff700 1 -- 172.21.15.83:0/3060944508 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0005d10 con 0x7fd3dc003a00 2023-11-26T07:12:39.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.680+0000 7fd3f765e700 1 -- 172.21.15.83:0/3060944508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc003a00 msgr2=0x7fd3dc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.680+0000 7fd3f765e700 1 --2- 172.21.15.83:0/3060944508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc003a00 0x7fd3dc003e20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000c370 tx=0x7fd3dc005880 comp rx=0 tx=0).stop 2023-11-26T07:12:39.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.680+0000 7fd3f5bfb700 1 -- 172.21.15.83:0/3060944508 reap_dead start 2023-11-26T07:12:39.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.680+0000 7fd3f765e700 1 -- 172.21.15.83:0/3060944508 shutdown_connections 2023-11-26T07:12:39.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.680+0000 7fd3f765e700 1 -- 172.21.15.83:0/3060944508 >> 172.21.15.83:0/3060944508 conn(0x7fd3f006b410 msgr2=0x7fd3f006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:39.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.680+0000 7fd3f765e700 1 -- 172.21.15.83:0/3060944508 shutdown_connections 2023-11-26T07:12:39.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.680+0000 7fd3f765e700 1 -- 172.21.15.83:0/3060944508 wait complete. 2023-11-26T07:12:39.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f765e700 1 Processor -- start 2023-11-26T07:12:39.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f765e700 1 -- start start 2023-11-26T07:12:39.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f765e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3dc003a00 0x7fd3f0158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f765e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3f015b710 0x7fd3f015bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f765e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3f0159da0 0x7fd3f015a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f765e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00ca8c0 con 0x7fd3f0159da0 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f765e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f015b350 con 0x7fd3f015b710 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f765e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f015c0d0 con 0x7fd3dc003a00 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f5bfb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3f0159da0 0x7fd3f015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f5bfb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3f0159da0 0x7fd3f015a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36582/0 (socket says 172.21.15.83:36582) 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f5bfb700 1 -- 172.21.15.83:0/4078544346 learned_addr learned my addr 172.21.15.83:0/4078544346 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f53fa700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3dc003a00 0x7fd3f0158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.682+0000 7fd3f4bf9700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3f015b710 0x7fd3f015bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.683+0000 7fd3f5bfb700 1 -- 172.21.15.83:0/4078544346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3dc003a00 msgr2=0x7fd3f0158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.683+0000 7fd3f5bfb700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3dc003a00 0x7fd3f0158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.683+0000 7fd3f5bfb700 1 -- 172.21.15.83:0/4078544346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3f015b710 msgr2=0x7fd3f015bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.683+0000 7fd3f5bfb700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3f015b710 0x7fd3f015bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.683+0000 7fd3f5bfb700 1 -- 172.21.15.83:0/4078544346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f015aab0 con 0x7fd3f0159da0 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.683+0000 7fd3f5bfb700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3f0159da0 0x7fd3f015a220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00b3470 tx=0x7fd3e8007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.683+0000 7fd3e5ffb700 1 -- 172.21.15.83:0/4078544346 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800e030 con 0x7fd3f0159da0 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.683+0000 7fd3e5ffb700 1 -- 172.21.15.83:0/4078544346 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e8011930 con 0x7fd3f0159da0 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.683+0000 7fd3e5ffb700 1 -- 172.21.15.83:0/4078544346 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8011ce0 con 0x7fd3f0159da0 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.683+0000 7fd3f765e700 1 -- 172.21.15.83:0/4078544346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f015ad90 con 0x7fd3f0159da0 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.683+0000 7fd3f765e700 1 -- 172.21.15.83:0/4078544346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f00c7b30 con 0x7fd3f0159da0 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.684+0000 7fd3f765e700 1 -- 172.21.15.83:0/4078544346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f00c7ea0 con 0x7fd3f0159da0 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.685+0000 7fd3e5ffb700 1 -- 172.21.15.83:0/4078544346 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fd3e8016470 con 0x7fd3f0159da0 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.685+0000 7fd3e5ffb700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd3d406b080 0x7fd3d406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.685+0000 7fd3e5ffb700 1 -- 172.21.15.83:0/4078544346 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3e800f070 con 0x7fd3f0159da0 2023-11-26T07:12:39.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.685+0000 7fd3f53fa700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd3d406b080 0x7fd3d406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:39.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.685+0000 7fd3f53fa700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd3d406b080 0x7fd3d406d540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd3f01614d0 tx=0x7fd3e000a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:39.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.689+0000 7fd3e5ffb700 1 -- 172.21.15.83:0/4078544346 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3e8054f00 con 0x7fd3f0159da0 2023-11-26T07:12:39.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.883+0000 7fd3f765e700 1 -- 172.21.15.83:0/4078544346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3f0067f60 con 0x7fd3f0159da0 2023-11-26T07:12:39.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.884+0000 7fd3e5ffb700 1 -- 172.21.15.83:0/4078544346 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fd3e8058520 con 0x7fd3f0159da0 2023-11-26T07:12:39.886 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:39.887 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:24.102531+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:20.302489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:21.468154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 -- 172.21.15.83:0/4078544346 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd3d406b080 msgr2=0x7fd3d406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd3d406b080 0x7fd3d406d540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd3f01614d0 tx=0x7fd3e000a000 comp rx=0 tx=0).stop 2023-11-26T07:12:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 -- 172.21.15.83:0/4078544346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3f0159da0 msgr2=0x7fd3f015a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3f0159da0 0x7fd3f015a220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00b3470 tx=0x7fd3e8007f60 comp rx=0 tx=0).stop 2023-11-26T07:12:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 -- 172.21.15.83:0/4078544346 shutdown_connections 2023-11-26T07:12:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3f0159da0 0x7fd3f015a220 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd3d406b080 0x7fd3d406d540 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3dc003a00 0x7fd3f0158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 --2- 172.21.15.83:0/4078544346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3f015b710 0x7fd3f015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:39.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 -- 172.21.15.83:0/4078544346 >> 172.21.15.83:0/4078544346 conn(0x7fd3f006b410 msgr2=0x7fd3f00b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:39.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 -- 172.21.15.83:0/4078544346 shutdown_connections 2023-11-26T07:12:39.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:39.886+0000 7fd3f765e700 1 -- 172.21.15.83:0/4078544346 wait complete. 2023-11-26T07:12:39.901 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:12:20.302489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:12:21.468154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:12:39.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:12:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.021+0000 7f01044e3700 1 Processor -- start 2023-11-26T07:12:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.021+0000 7f01044e3700 1 -- start start 2023-11-26T07:12:40.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.022+0000 7f01044e3700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f00fc15bf40 0x7f00fc0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.022+0000 7f01044e3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc06fd30 con 0x7f00fc0cccd0 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.022+0000 7f01044e3700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc06f770 con 0x7f00fc15ca30 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.022+0000 7f01044e3700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00fc06fa50 con 0x7f00fc15bf40 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f0101a7e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f00fc0cccd0 0x7f00fc15c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56628/0 (socket says 172.21.15.83:56628) 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f0101a7e700 1 -- 172.21.15.83:0/2737783186 learned_addr learned my addr 172.21.15.83:0/2737783186 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010227f700 1 --2- 172.21.15.83:0/2737783186 >> v2:172.21.15.203:3300/0 conn(0x7f00fc15bf40 0x7f00fc0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055508315 0 0) 0x7f00fc06fd30 con 0x7f00fc0cccd0 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e80035b0 con 0x7f00fc0cccd0 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 248070468 0 0) 0x7f00e80035b0 con 0x7f00fc0cccd0 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 >> v2:172.21.15.203:3300/0 conn(0x7f00fc15bf40 msgr2=0x7f00fc0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 --2- 172.21.15.83:0/2737783186 >> v2:172.21.15.203:3300/0 conn(0x7f00fc15bf40 0x7f00fc0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 >> v1:172.21.15.203:6789/0 conn(0x7f00fc15ca30 legacy=0x7f00fc0b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc0b4490 con 0x7f00fc0cccd0 2023-11-26T07:12:40.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f00f8001670 con 0x7f00fc0cccd0 2023-11-26T07:12:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 >> v1:172.21.15.83:6789/0 conn(0x7f00fc0cccd0 legacy=0x7f00fc15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 --2- 172.21.15.83:0/2737783186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8003a00 0x7f00e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 --2- 172.21.15.83:0/2737783186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e8004850 0x7f00e8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 --2- 172.21.15.83:0/2737783186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e8005210 0x7f00e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc06fa50 con 0x7f00e8003a00 2023-11-26T07:12:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e800c0b0 con 0x7f00e8005210 2023-11-26T07:12:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e800c270 con 0x7f00e8004850 2023-11-26T07:12:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010227f700 1 --2- 172.21.15.83:0/2737783186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8003a00 0x7f00e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f0102a80700 1 --2- 172.21.15.83:0/2737783186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e8005210 0x7f00e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f0101a7e700 1 --2- 172.21.15.83:0/2737783186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e8004850 0x7f00e8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010227f700 1 -- 172.21.15.83:0/2737783186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e8004850 msgr2=0x7f00e8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010227f700 1 --2- 172.21.15.83:0/2737783186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e8004850 0x7f00e8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010227f700 1 -- 172.21.15.83:0/2737783186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e8005210 msgr2=0x7f00e800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010227f700 1 --2- 172.21.15.83:0/2737783186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e8005210 0x7f00e800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f010227f700 1 -- 172.21.15.83:0/2737783186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e80035b0 con 0x7f00e8003a00 2023-11-26T07:12:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.023+0000 7f0102a80700 1 -- 172.21.15.83:0/2737783186 reap_dead start 2023-11-26T07:12:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.024+0000 7f010227f700 1 --2- 172.21.15.83:0/2737783186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8003a00 0x7f00e8003e20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009b60 tx=0x7f00e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774004a7bae51321 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.024+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec00e6c0 con 0x7f00e8003a00 2023-11-26T07:12:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.024+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec00e820 con 0x7f00e8003a00 2023-11-26T07:12:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.024+0000 7f010127d700 1 -- 172.21.15.83:0/2737783186 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec00eb90 con 0x7f00e8003a00 2023-11-26T07:12:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.024+0000 7f01044e3700 1 -- 172.21.15.83:0/2737783186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8003a00 msgr2=0x7f00e8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.024+0000 7f01044e3700 1 --2- 172.21.15.83:0/2737783186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8003a00 0x7f00e8003e20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009b60 tx=0x7f00e8005880 comp rx=0 tx=0).stop 2023-11-26T07:12:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.024+0000 7f01044e3700 1 -- 172.21.15.83:0/2737783186 shutdown_connections 2023-11-26T07:12:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.024+0000 7f01044e3700 1 --2- 172.21.15.83:0/2737783186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e8003a00 0x7f00e8003e20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.024+0000 7f01044e3700 1 -- 172.21.15.83:0/2737783186 >> 172.21.15.83:0/2737783186 conn(0x7f00fc0b2050 msgr2=0x7f00fc0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.024+0000 7f01044e3700 1 -- 172.21.15.83:0/2737783186 shutdown_connections 2023-11-26T07:12:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.024+0000 7f01044e3700 1 -- 172.21.15.83:0/2737783186 wait complete. 2023-11-26T07:12:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.025+0000 7f01044e3700 1 Processor -- start 2023-11-26T07:12:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f01044e3700 1 -- start start 2023-11-26T07:12:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f01044e3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e8003a00 0x7f00fc13dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f01044e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc152540 0x7f00fc1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f01044e3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc153180 0x7f00fc153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f01044e3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc152f00 con 0x7f00fc153180 2023-11-26T07:12:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f01044e3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc153b40 con 0x7f00fc152540 2023-11-26T07:12:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f01044e3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc153db0 con 0x7f00e8003a00 2023-11-26T07:12:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f0102a80700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc153180 0x7f00fc153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f0102a80700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc153180 0x7f00fc153600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36602/0 (socket says 172.21.15.83:36602) 2023-11-26T07:12:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f0102a80700 1 -- 172.21.15.83:0/1721468772 learned_addr learned my addr 172.21.15.83:0/1721468772 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f010227f700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e8003a00 0x7f00fc13dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.026+0000 7f0101a7e700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc152540 0x7f00fc1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.027+0000 7f0102a80700 1 -- 172.21.15.83:0/1721468772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e8003a00 msgr2=0x7f00fc13dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.027+0000 7f0102a80700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e8003a00 0x7f00fc13dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.027+0000 7f0102a80700 1 -- 172.21.15.83:0/1721468772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc152540 msgr2=0x7f00fc1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.027+0000 7f0102a80700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc152540 0x7f00fc1529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.027+0000 7f0102a80700 1 -- 172.21.15.83:0/1721468772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc154090 con 0x7f00fc153180 2023-11-26T07:12:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.027+0000 7f0102a80700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc153180 0x7f00fc153600 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f00fc072c70 tx=0x7f00f400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.027+0000 7f00f2ffd700 1 -- 172.21.15.83:0/1721468772 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4003a00 con 0x7f00fc153180 2023-11-26T07:12:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.027+0000 7f00f2ffd700 1 -- 172.21.15.83:0/1721468772 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f4003b80 con 0x7f00fc153180 2023-11-26T07:12:40.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.027+0000 7f01044e3700 1 -- 172.21.15.83:0/1721468772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc154370 con 0x7f00fc153180 2023-11-26T07:12:40.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.027+0000 7f00f2ffd700 1 -- 172.21.15.83:0/1721468772 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f40050d0 con 0x7f00fc153180 2023-11-26T07:12:40.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.027+0000 7f01044e3700 1 -- 172.21.15.83:0/1721468772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc154930 con 0x7f00fc153180 2023-11-26T07:12:40.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.028+0000 7f01044e3700 1 -- 172.21.15.83:0/1721468772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc14fd30 con 0x7f00fc153180 2023-11-26T07:12:40.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.028+0000 7f00f2ffd700 1 -- 172.21.15.83:0/1721468772 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f00f4005230 con 0x7f00fc153180 2023-11-26T07:12:40.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.029+0000 7f00f2ffd700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f00e006b0b0 0x7f00e006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.029+0000 7f00f2ffd700 1 -- 172.21.15.83:0/1721468772 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00f4089670 con 0x7f00fc153180 2023-11-26T07:12:40.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.029+0000 7f010227f700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f00e006b0b0 0x7f00e006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.029+0000 7f010227f700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f00e006b0b0 0x7f00e006d570 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0090b0 tx=0x7f00ec01e040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:40.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.033+0000 7f00f2ffd700 1 -- 172.21.15.83:0/1721468772 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00f4054530 con 0x7f00fc153180 2023-11-26T07:12:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.228+0000 7f01044e3700 1 -- 172.21.15.83:0/1721468772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00fc068d90 con 0x7f00fc153180 2023-11-26T07:12:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.228+0000 7f00f2ffd700 1 -- 172.21.15.83:0/1721468772 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00f4057b50 con 0x7f00fc153180 2023-11-26T07:12:40.231 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:12:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 -- 172.21.15.83:0/1721468772 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f00e006b0b0 msgr2=0x7f00e006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f00e006b0b0 0x7f00e006d570 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0090b0 tx=0x7f00ec01e040 comp rx=0 tx=0).stop 2023-11-26T07:12:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 -- 172.21.15.83:0/1721468772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc153180 msgr2=0x7f00fc153600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc153180 0x7f00fc153600 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f00fc072c70 tx=0x7f00f400ed50 comp rx=0 tx=0).stop 2023-11-26T07:12:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 -- 172.21.15.83:0/1721468772 shutdown_connections 2023-11-26T07:12:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00fc153180 0x7f00fc153600 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f00e006b0b0 0x7f00e006d570 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e8003a00 0x7f00fc13dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 --2- 172.21.15.83:0/1721468772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00fc152540 0x7f00fc1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 -- 172.21.15.83:0/1721468772 >> 172.21.15.83:0/1721468772 conn(0x7f00fc0b2050 msgr2=0x7f00fc15c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 -- 172.21.15.83:0/1721468772 shutdown_connections 2023-11-26T07:12:40.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.230+0000 7f01044e3700 1 -- 172.21.15.83:0/1721468772 wait complete. 2023-11-26T07:12:40.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:12:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.366+0000 7f72d6ccf700 1 Processor -- start 2023-11-26T07:12:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.366+0000 7f72d6ccf700 1 -- start start 2023-11-26T07:12:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.367+0000 7f72d6ccf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f72d015be40 0x7f72d0160550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.367+0000 7f72d6ccf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d00b5880 0x7f72d015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.367+0000 7f72d6ccf700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d01600c0 con 0x7f72d0073f90 2023-11-26T07:12:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.367+0000 7f72d6ccf700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d00cd8d0 con 0x7f72d015be40 2023-11-26T07:12:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.367+0000 7f72d6ccf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d015c220 con 0x7f72d00b5880 2023-11-26T07:12:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.367+0000 7f72d4a6b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f72d015be40 0x7f72d0160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.367+0000 7f72d4a6b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f72d015be40 0x7f72d0160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36606/0 (socket says 172.21.15.83:36606) 2023-11-26T07:12:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.367+0000 7f72d4a6b700 1 -- 172.21.15.83:0/3886627445 learned_addr learned my addr 172.21.15.83:0/3886627445 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.368+0000 7f72cffff700 1 --2- 172.21.15.83:0/3886627445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d00b5880 0x7f72d015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.368+0000 7f72d4a6b700 1 -- 172.21.15.83:0/3886627445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d00b5880 msgr2=0x7f72d015cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.368+0000 7f72d4a6b700 1 --2- 172.21.15.83:0/3886627445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d00b5880 0x7f72d015cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.368+0000 7f72d4a6b700 1 -- 172.21.15.83:0/3886627445 >> v1:172.21.15.203:6789/0 conn(0x7f72d0073f90 legacy=0x7f72d00b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.368+0000 7f72d4a6b700 1 -- 172.21.15.83:0/3886627445 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d015d300 con 0x7f72d015be40 2023-11-26T07:12:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.368+0000 7f72d4a6b700 1 --2- 172.21.15.83:0/3886627445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d015be40 0x7f72d0160550 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f72c40098f0 tx=0x7f72c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644ecf00d2b6c6c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.368+0000 7f72cf7fe700 1 -- 172.21.15.83:0/3886627445 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4004230 con 0x7f72d015be40 2023-11-26T07:12:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.368+0000 7f72cf7fe700 1 -- 172.21.15.83:0/3886627445 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c4004390 con 0x7f72d015be40 2023-11-26T07:12:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.368+0000 7f72cf7fe700 1 -- 172.21.15.83:0/3886627445 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c40036a0 con 0x7f72d015be40 2023-11-26T07:12:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.369+0000 7f72d6ccf700 1 -- 172.21.15.83:0/3886627445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d015be40 msgr2=0x7f72d0160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.369+0000 7f72d6ccf700 1 --2- 172.21.15.83:0/3886627445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d015be40 0x7f72d0160550 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f72c40098f0 tx=0x7f72c4009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.369+0000 7f72d6ccf700 1 -- 172.21.15.83:0/3886627445 shutdown_connections 2023-11-26T07:12:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.369+0000 7f72d6ccf700 1 --2- 172.21.15.83:0/3886627445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d015be40 0x7f72d0160550 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.369+0000 7f72d6ccf700 1 --2- 172.21.15.83:0/3886627445 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d00b5880 0x7f72d015cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.369+0000 7f72d6ccf700 1 -- 172.21.15.83:0/3886627445 >> 172.21.15.83:0/3886627445 conn(0x7f72d00d0b80 msgr2=0x7f72d00d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.369+0000 7f72d6ccf700 1 -- 172.21.15.83:0/3886627445 shutdown_connections 2023-11-26T07:12:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.369+0000 7f72d6ccf700 1 -- 172.21.15.83:0/3886627445 wait complete. 2023-11-26T07:12:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.370+0000 7f72d6ccf700 1 Processor -- start 2023-11-26T07:12:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.370+0000 7f72d6ccf700 1 -- start start 2023-11-26T07:12:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d6ccf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d0073f90 0x7f72d013ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d6ccf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d00b5880 0x7f72d015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d6ccf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72d015be40 0x7f72d00cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d6ccf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d013e140 con 0x7f72d0073f90 2023-11-26T07:12:40.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d6ccf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d013d830 con 0x7f72d015be40 2023-11-26T07:12:40.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d6ccf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d0140d20 con 0x7f72d00b5880 2023-11-26T07:12:40.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d4a6b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d0073f90 0x7f72d013ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d4a6b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d0073f90 0x7f72d013ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36610/0 (socket says 172.21.15.83:36610) 2023-11-26T07:12:40.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d4a6b700 1 -- 172.21.15.83:0/1688869639 learned_addr learned my addr 172.21.15.83:0/1688869639 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72cffff700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d00b5880 0x7f72d015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d526c700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72d015be40 0x7f72d00cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d4a6b700 1 -- 172.21.15.83:0/1688869639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d00b5880 msgr2=0x7f72d015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d4a6b700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d00b5880 0x7f72d015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d4a6b700 1 -- 172.21.15.83:0/1688869639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72d015be40 msgr2=0x7f72d00cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d4a6b700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72d015be40 0x7f72d00cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d4a6b700 1 -- 172.21.15.83:0/1688869639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c4009550 con 0x7f72d0073f90 2023-11-26T07:12:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72cffff700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d00b5880 0x7f72d015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d4a6b700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d0073f90 0x7f72d013ce90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f72c400b0e0 tx=0x7f72c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:40.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72cd7fa700 1 -- 172.21.15.83:0/1688869639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400b860 con 0x7f72d0073f90 2023-11-26T07:12:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72cd7fa700 1 -- 172.21.15.83:0/1688869639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c400b9c0 con 0x7f72d0073f90 2023-11-26T07:12:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72d6ccf700 1 -- 172.21.15.83:0/1688869639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d015b4b0 con 0x7f72d0073f90 2023-11-26T07:12:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.371+0000 7f72cd7fa700 1 -- 172.21.15.83:0/1688869639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400bd70 con 0x7f72d0073f90 2023-11-26T07:12:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.372+0000 7f72d6ccf700 1 -- 172.21.15.83:0/1688869639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d013dd60 con 0x7f72d0073f90 2023-11-26T07:12:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.372+0000 7f72d6ccf700 1 -- 172.21.15.83:0/1688869639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d0068fb0 con 0x7f72d0073f90 2023-11-26T07:12:40.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.373+0000 7f72cd7fa700 1 -- 172.21.15.83:0/1688869639 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f72c401b440 con 0x7f72d0073f90 2023-11-26T07:12:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.373+0000 7f72cd7fa700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72b406b080 0x7f72b406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.373+0000 7f72cd7fa700 1 -- 172.21.15.83:0/1688869639 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72c408b2b0 con 0x7f72d0073f90 2023-11-26T07:12:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.373+0000 7f72cffff700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72b406b080 0x7f72b406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.374+0000 7f72cffff700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72b406b080 0x7f72b406d540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f72d01f50b0 tx=0x7f72bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:40.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.377+0000 7f72cd7fa700 1 -- 172.21.15.83:0/1688869639 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f72c4056170 con 0x7f72d0073f90 2023-11-26T07:12:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.566+0000 7f72d6ccf700 1 -- 172.21.15.83:0/1688869639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72d013e280 con 0x7f72d0073f90 2023-11-26T07:12:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.567+0000 7f72cd7fa700 1 -- 172.21.15.83:0/1688869639 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f72d013e280 con 0x7f72d0073f90 2023-11-26T07:12:40.569 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:12:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 -- 172.21.15.83:0/1688869639 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72b406b080 msgr2=0x7f72b406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72b406b080 0x7f72b406d540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f72d01f50b0 tx=0x7f72bc008040 comp rx=0 tx=0).stop 2023-11-26T07:12:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 -- 172.21.15.83:0/1688869639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d0073f90 msgr2=0x7f72d013ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d0073f90 0x7f72d013ce90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f72c400b0e0 tx=0x7f72c4004750 comp rx=0 tx=0).stop 2023-11-26T07:12:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 -- 172.21.15.83:0/1688869639 shutdown_connections 2023-11-26T07:12:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72d015be40 0x7f72d00cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72d0073f90 0x7f72d013ce90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f72b406b080 0x7f72b406d540 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 --2- 172.21.15.83:0/1688869639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d00b5880 0x7f72d015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 -- 172.21.15.83:0/1688869639 >> 172.21.15.83:0/1688869639 conn(0x7f72d00d0b80 msgr2=0x7f72d015e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 -- 172.21.15.83:0/1688869639 shutdown_connections 2023-11-26T07:12:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.569+0000 7f72d6ccf700 1 -- 172.21.15.83:0/1688869639 wait complete. 2023-11-26T07:12:40.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:12:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.704+0000 7f0dc172c700 1 Processor -- start 2023-11-26T07:12:40.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.704+0000 7f0dc172c700 1 -- start start 2023-11-26T07:12:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.705+0000 7f0dc172c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0dbc15b370 0x7f0dbc15b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.705+0000 7f0dc172c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 0x7f0dbc0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.705+0000 7f0dc172c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc0ae5e0 con 0x7f0dbc1587f0 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.705+0000 7f0dc172c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dbc0b4630 con 0x7f0dbc15b370 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.705+0000 7f0dc172c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc15be30 con 0x7f0dbc0b3c60 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.705+0000 7f0dbb7fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 0x7f0dbc0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.705+0000 7f0dba7fc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0dbc15b370 0x7f0dbc15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.705+0000 7f0dbb7fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 0x7f0dbc0b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:47080/0 (socket says 172.21.15.83:47080) 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.705+0000 7f0dbb7fe700 1 -- 172.21.15.83:0/993761010 learned_addr learned my addr 172.21.15.83:0/993761010 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.706+0000 7f0dbb7fe700 1 -- 172.21.15.83:0/993761010 >> v2:172.21.15.203:3300/0 conn(0x7f0dbc15b370 msgr2=0x7f0dbc15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.706+0000 7f0dbb7fe700 1 --2- 172.21.15.83:0/993761010 >> v2:172.21.15.203:3300/0 conn(0x7f0dbc15b370 0x7f0dbc15b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.706+0000 7f0dbb7fe700 1 -- 172.21.15.83:0/993761010 >> v1:172.21.15.203:6789/0 conn(0x7f0dbc1587f0 legacy=0x7f0dbc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.706+0000 7f0dbb7fe700 1 -- 172.21.15.83:0/993761010 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc15c110 con 0x7f0dbc0b3c60 2023-11-26T07:12:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.706+0000 7f0dbb7fe700 1 --2- 172.21.15.83:0/993761010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 0x7f0dbc0b40f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009e80 tx=0x7f0db400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c49f857ff770712d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.706+0000 7f0db9ffb700 1 -- 172.21.15.83:0/993761010 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db400e070 con 0x7f0dbc0b3c60 2023-11-26T07:12:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.706+0000 7f0db9ffb700 1 -- 172.21.15.83:0/993761010 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db400b8d0 con 0x7f0dbc0b3c60 2023-11-26T07:12:40.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.706+0000 7f0db9ffb700 1 -- 172.21.15.83:0/993761010 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db400bc80 con 0x7f0dbc0b3c60 2023-11-26T07:12:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.707+0000 7f0dc172c700 1 -- 172.21.15.83:0/993761010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 msgr2=0x7f0dbc0b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.707+0000 7f0dc172c700 1 --2- 172.21.15.83:0/993761010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 0x7f0dbc0b40f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009e80 tx=0x7f0db400b330 comp rx=0 tx=0).stop 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.707+0000 7f0dc172c700 1 -- 172.21.15.83:0/993761010 shutdown_connections 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.707+0000 7f0dc172c700 1 --2- 172.21.15.83:0/993761010 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 0x7f0dbc0b40f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.707+0000 7f0dc172c700 1 --2- 172.21.15.83:0/993761010 >> v2:172.21.15.203:3300/0 conn(0x7f0dbc15b370 0x7f0dbc15b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.707+0000 7f0dc172c700 1 -- 172.21.15.83:0/993761010 >> 172.21.15.83:0/993761010 conn(0x7f0dbc0d0bf0 msgr2=0x7f0dbc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.707+0000 7f0dc172c700 1 -- 172.21.15.83:0/993761010 shutdown_connections 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.707+0000 7f0dc172c700 1 -- 172.21.15.83:0/993761010 wait complete. 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.708+0000 7f0dc172c700 1 Processor -- start 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.708+0000 7f0dc172c700 1 -- start start 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dc172c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 0x7f0dbc1f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dc172c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dbc1587f0 0x7f0dbc09d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dc172c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dbc15b370 0x7f0dbc074140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dc172c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc09b8a0 con 0x7f0dbc15b370 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dc172c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc09bb80 con 0x7f0dbc1587f0 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dc172c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc09be60 con 0x7f0dbc0b3c60 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dbb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dbc15b370 0x7f0dbc074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dbb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dbc15b370 0x7f0dbc074140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36628/0 (socket says 172.21.15.83:36628) 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dbb7fe700 1 -- 172.21.15.83:0/3880243827 learned_addr learned my addr 172.21.15.83:0/3880243827 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dbaffd700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 0x7f0dbc1f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dba7fc700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dbc1587f0 0x7f0dbc09d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dbb7fe700 1 -- 172.21.15.83:0/3880243827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 msgr2=0x7f0dbc1f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dbb7fe700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 0x7f0dbc1f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dbb7fe700 1 -- 172.21.15.83:0/3880243827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dbc1587f0 msgr2=0x7f0dbc09d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dbb7fe700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dbc1587f0 0x7f0dbc09d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dbb7fe700 1 -- 172.21.15.83:0/3880243827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db4009b30 con 0x7f0dbc15b370 2023-11-26T07:12:40.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dba7fc700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dbc1587f0 0x7f0dbc09d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0dbb7fe700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dbc15b370 0x7f0dbc074140 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc15bf70 tx=0x7f0db4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.709+0000 7f0da37fe700 1 -- 172.21.15.83:0/3880243827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4004960 con 0x7f0dbc15b370 2023-11-26T07:12:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.710+0000 7f0da37fe700 1 -- 172.21.15.83:0/3880243827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db40099b0 con 0x7f0dbc15b370 2023-11-26T07:12:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.710+0000 7f0dc172c700 1 -- 172.21.15.83:0/3880243827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc09c140 con 0x7f0dbc15b370 2023-11-26T07:12:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.710+0000 7f0da37fe700 1 -- 172.21.15.83:0/3880243827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db401c680 con 0x7f0dbc15b370 2023-11-26T07:12:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.710+0000 7f0dc172c700 1 -- 172.21.15.83:0/3880243827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc1f2ff0 con 0x7f0dbc15b370 2023-11-26T07:12:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.710+0000 7f0dc172c700 1 -- 172.21.15.83:0/3880243827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc068fb0 con 0x7f0dbc15b370 2023-11-26T07:12:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.711+0000 7f0da37fe700 1 -- 172.21.15.83:0/3880243827 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0db401c7e0 con 0x7f0dbc15b370 2023-11-26T07:12:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.711+0000 7f0da37fe700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0da806b080 0x7f0da806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.711+0000 7f0da37fe700 1 -- 172.21.15.83:0/3880243827 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0db408b400 con 0x7f0dbc15b370 2023-11-26T07:12:40.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.711+0000 7f0dbaffd700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0da806b080 0x7f0da806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:40.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.712+0000 7f0dbaffd700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0da806b080 0x7f0da806d540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0db0000f60 tx=0x7f0db0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:40.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.716+0000 7f0da37fe700 1 -- 172.21.15.83:0/3880243827 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0db40561b0 con 0x7f0dbc15b370 2023-11-26T07:12:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.908+0000 7f0dc172c700 1 -- 172.21.15.83:0/3880243827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0dbc09c600 con 0x7f0dbc15b370 2023-11-26T07:12:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.908+0000 7f0da37fe700 1 -- 172.21.15.83:0/3880243827 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0db40597d0 con 0x7f0dbc15b370 2023-11-26T07:12:40.911 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:12:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.910+0000 7f0dc172c700 1 -- 172.21.15.83:0/3880243827 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0da806b080 msgr2=0x7f0da806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.910+0000 7f0dc172c700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0da806b080 0x7f0da806d540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0db0000f60 tx=0x7f0db0009040 comp rx=0 tx=0).stop 2023-11-26T07:12:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.910+0000 7f0dc172c700 1 -- 172.21.15.83:0/3880243827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dbc15b370 msgr2=0x7f0dbc074140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.910+0000 7f0dc172c700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dbc15b370 0x7f0dbc074140 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc15bf70 tx=0x7f0db4004ad0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.910+0000 7f0dc172c700 1 -- 172.21.15.83:0/3880243827 shutdown_connections 2023-11-26T07:12:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.910+0000 7f0dc172c700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dbc15b370 0x7f0dbc074140 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.910+0000 7f0dc172c700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0da806b080 0x7f0da806d540 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.910+0000 7f0dc172c700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dbc0b3c60 0x7f0dbc1f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.910+0000 7f0dc172c700 1 --2- 172.21.15.83:0/3880243827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0dbc1587f0 0x7f0dbc09d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.911+0000 7f0dc172c700 1 -- 172.21.15.83:0/3880243827 >> 172.21.15.83:0/3880243827 conn(0x7f0dbc0d0bf0 msgr2=0x7f0dbc0aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.911+0000 7f0dc172c700 1 -- 172.21.15.83:0/3880243827 shutdown_connections 2023-11-26T07:12:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:40.911+0000 7f0dc172c700 1 -- 172.21.15.83:0/3880243827 wait complete. 2023-11-26T07:12:40.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:41.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.042+0000 7fdaabfe0700 1 Processor -- start 2023-11-26T07:12:41.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.042+0000 7fdaabfe0700 1 -- start start 2023-11-26T07:12:41.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.043+0000 7fdaabfe0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdaa406c710 0x7fdaa40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.043+0000 7fdaabfe0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa40af4d0 0x7fdaa40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.043+0000 7fdaabfe0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa40ae430 con 0x7fdaa415e080 2023-11-26T07:12:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.043+0000 7fdaabfe0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa40af0b0 con 0x7fdaa406c710 2023-11-26T07:12:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.043+0000 7fdaabfe0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40b05a0 con 0x7fdaa40af4d0 2023-11-26T07:12:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.043+0000 7fdaa9d7c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdaa415e080 0x7fdaa406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56672/0 (socket says 172.21.15.83:56672) 2023-11-26T07:12:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.043+0000 7fdaa9d7c700 1 -- 172.21.15.83:0/341046079 learned_addr learned my addr 172.21.15.83:0/341046079 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991184358 0 0) 0x7fdaa40ae430 con 0x7fdaa415e080 2023-11-26T07:12:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaaa57d700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa40af4d0 0x7fdaa40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda900035b0 con 0x7fdaa415e080 2023-11-26T07:12:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa957b700 1 --2- 172.21.15.83:0/341046079 >> v2:172.21.15.203:3300/0 conn(0x7fdaa406c710 0x7fdaa40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2688192425 0 0) 0x7fda900035b0 con 0x7fdaa415e080 2023-11-26T07:12:41.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa40af4d0 msgr2=0x7fdaa40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa40af4d0 0x7fdaa40afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 >> v2:172.21.15.203:3300/0 conn(0x7fdaa406c710 msgr2=0x7fdaa40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 --2- 172.21.15.83:0/341046079 >> v2:172.21.15.203:3300/0 conn(0x7fdaa406c710 0x7fdaa40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa40ae710 con 0x7fdaa415e080 2023-11-26T07:12:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fda94001670 con 0x7fdaa415e080 2023-11-26T07:12:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa957b700 1 --2- 172.21.15.83:0/341046079 >> v2:172.21.15.203:3300/0 conn(0x7fdaa406c710 0x7fdaa40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 >> v1:172.21.15.83:6789/0 conn(0x7fdaa415e080 legacy=0x7fdaa406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda90003a00 0x7fda90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa40af4d0 0x7fda90004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 0x7fda90005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40af0b0 con 0x7fda90004d90 2023-11-26T07:12:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40b05a0 con 0x7fda90003a00 2023-11-26T07:12:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9000ba10 con 0x7fdaa40af4d0 2023-11-26T07:12:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaaa57d700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 0x7fda90005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa9d7c700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda90003a00 0x7fda90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaa957b700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa40af4d0 0x7fda90004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaaa57d700 1 -- 172.21.15.83:0/341046079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa40af4d0 msgr2=0x7fda90004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:41.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaaa57d700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa40af4d0 0x7fda90004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaaa57d700 1 -- 172.21.15.83:0/341046079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda90003a00 msgr2=0x7fda90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaaa57d700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda90003a00 0x7fda90003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaaa57d700 1 -- 172.21.15.83:0/341046079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda900035b0 con 0x7fda90004d90 2023-11-26T07:12:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.044+0000 7fdaaa57d700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 0x7fda90005210 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00e170 tx=0x7fda90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe107c756077f135 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.045+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00ec90 con 0x7fda90004d90 2023-11-26T07:12:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.045+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c00f040 con 0x7fda90004d90 2023-11-26T07:12:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.045+0000 7fdaa8d7a700 1 -- 172.21.15.83:0/341046079 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c014480 con 0x7fda90004d90 2023-11-26T07:12:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.045+0000 7fdaabfe0700 1 -- 172.21.15.83:0/341046079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 msgr2=0x7fda90005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.045+0000 7fdaabfe0700 1 --2- 172.21.15.83:0/341046079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 0x7fda90005210 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00e170 tx=0x7fda90005880 comp rx=0 tx=0).stop 2023-11-26T07:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.045+0000 7fdaaa57d700 1 -- 172.21.15.83:0/341046079 reap_dead start 2023-11-26T07:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.045+0000 7fdaabfe0700 1 -- 172.21.15.83:0/341046079 shutdown_connections 2023-11-26T07:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.045+0000 7fdaabfe0700 1 -- 172.21.15.83:0/341046079 >> 172.21.15.83:0/341046079 conn(0x7fdaa4071b90 msgr2=0x7fdaa4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.045+0000 7fdaabfe0700 1 -- 172.21.15.83:0/341046079 shutdown_connections 2023-11-26T07:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.045+0000 7fdaabfe0700 1 -- 172.21.15.83:0/341046079 wait complete. 2023-11-26T07:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.046+0000 7fdaabfe0700 1 Processor -- start 2023-11-26T07:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.047+0000 7fdaabfe0700 1 -- start start 2023-11-26T07:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.047+0000 7fdaabfe0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 0x7fdaa406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.047+0000 7fdaabfe0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa415f8f0 0x7fdaa415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.047+0000 7fdaabfe0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa4151370 0x7fdaa41517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.047+0000 7fdaabfe0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa4151d30 con 0x7fda90004d90 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.047+0000 7fdaabfe0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa4153c40 con 0x7fdaa4151370 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.047+0000 7fdaabfe0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa415b4e0 con 0x7fdaa415f8f0 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.047+0000 7fdaa9d7c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 0x7fdaa406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.047+0000 7fdaa9d7c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 0x7fdaa406ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36642/0 (socket says 172.21.15.83:36642) 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.047+0000 7fdaa9d7c700 1 -- 172.21.15.83:0/1300730541 learned_addr learned my addr 172.21.15.83:0/1300730541 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.047+0000 7fdaa957b700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa415f8f0 0x7fdaa415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fdaaa57d700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa4151370 0x7fdaa41517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fdaa9d7c700 1 -- 172.21.15.83:0/1300730541 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa415f8f0 msgr2=0x7fdaa415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fdaa9d7c700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa415f8f0 0x7fdaa415fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fdaa9d7c700 1 -- 172.21.15.83:0/1300730541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa4151370 msgr2=0x7fdaa41517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fdaa9d7c700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa4151370 0x7fdaa41517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fdaa9d7c700 1 -- 172.21.15.83:0/1300730541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa4153e90 con 0x7fda90004d90 2023-11-26T07:12:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fdaa9d7c700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 0x7fdaa406ee20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fda94007340 tx=0x7fda94007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fda9a7fc700 1 -- 172.21.15.83:0/1300730541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9400f6c0 con 0x7fda90004d90 2023-11-26T07:12:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fda9a7fc700 1 -- 172.21.15.83:0/1300730541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9400f840 con 0x7fda90004d90 2023-11-26T07:12:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fda9a7fc700 1 -- 172.21.15.83:0/1300730541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9400fbf0 con 0x7fda90004d90 2023-11-26T07:12:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fdaabfe0700 1 -- 172.21.15.83:0/1300730541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa4154170 con 0x7fda90004d90 2023-11-26T07:12:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.048+0000 7fdaabfe0700 1 -- 172.21.15.83:0/1300730541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa406f360 con 0x7fda90004d90 2023-11-26T07:12:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.049+0000 7fdaabfe0700 1 -- 172.21.15.83:0/1300730541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa40753f0 con 0x7fda90004d90 2023-11-26T07:12:41.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.049+0000 7fda9a7fc700 1 -- 172.21.15.83:0/1300730541 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fda9400fd50 con 0x7fda90004d90 2023-11-26T07:12:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.050+0000 7fda9a7fc700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda8806b080 0x7fda8806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.050+0000 7fda9a7fc700 1 -- 172.21.15.83:0/1300730541 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda940897f0 con 0x7fda90004d90 2023-11-26T07:12:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.050+0000 7fdaa957b700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda8806b080 0x7fda8806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.050+0000 7fdaa957b700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda8806b080 0x7fda8806d540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009a80 tx=0x7fdaa0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:41.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.054+0000 7fda9a7fc700 1 -- 172.21.15.83:0/1300730541 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda940546b0 con 0x7fda90004d90 2023-11-26T07:12:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.287+0000 7fdaabfe0700 1 -- 172.21.15.83:0/1300730541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdaa4154690 con 0x7fda90004d90 2023-11-26T07:12:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.287+0000 7fda9a7fc700 1 -- 172.21.15.83:0/1300730541 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fdaa4154690 con 0x7fda90004d90 2023-11-26T07:12:41.290 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:41.290 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/486975373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":486975373},{"type":"v1","addr":"172.21.15.203:6834","nonce":486975373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3188343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3188343240},{"type":"v1","addr":"172.21.15.83:6836","nonce":3188343240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4271462644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4271462644},{"type":"v1","addr":"172.21.15.83:6837","nonce":4271462644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:22.505426+0000","modified":"2023-11-26T07:12:25.894494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6835/995330476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":995330476},{"type":"v1","addr":"172.21.15.203:6835","nonce":995330476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:12:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 -- 172.21.15.83:0/1300730541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda8806b080 msgr2=0x7fda8806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda8806b080 0x7fda8806d540 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009a80 tx=0x7fdaa0008040 comp rx=0 tx=0).stop 2023-11-26T07:12:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 -- 172.21.15.83:0/1300730541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 msgr2=0x7fdaa406ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 0x7fdaa406ee20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fda94007340 tx=0x7fda94007650 comp rx=0 tx=0).stop 2023-11-26T07:12:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 -- 172.21.15.83:0/1300730541 shutdown_connections 2023-11-26T07:12:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdaa4151370 0x7fdaa41517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda90004d90 0x7fdaa406ee20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda8806b080 0x7fda8806d540 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 --2- 172.21.15.83:0/1300730541 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdaa415f8f0 0x7fdaa415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 -- 172.21.15.83:0/1300730541 >> 172.21.15.83:0/1300730541 conn(0x7fdaa4071b90 msgr2=0x7fdaa40b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 -- 172.21.15.83:0/1300730541 shutdown_connections 2023-11-26T07:12:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.289+0000 7fdaabfe0700 1 -- 172.21.15.83:0/1300730541 wait complete. 2023-11-26T07:12:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2023-11-26T07:12:41.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.422+0000 7f1f60a31700 1 Processor -- start 2023-11-26T07:12:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.422+0000 7f1f60a31700 1 -- start start 2023-11-26T07:12:41.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.423+0000 7f1f60a31700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c158670 0x7f1f5c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.423+0000 7f1f60a31700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c0cfff0 con 0x7f1f5c0c9fc0 2023-11-26T07:12:41.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.423+0000 7f1f60a31700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c0cfd10 con 0x7f1f5c158a50 2023-11-26T07:12:41.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.423+0000 7f1f60a31700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c161f50 con 0x7f1f5c158670 2023-11-26T07:12:41.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.423+0000 7f1f5ad9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1f5c0c9fc0 0x7f1f5c0ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56692/0 (socket says 172.21.15.83:56692) 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.423+0000 7f1f5ad9d700 1 -- 172.21.15.83:0/2549778375 learned_addr learned my addr 172.21.15.83:0/2549778375 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.423+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459596530 0 0) 0x7f1f5c0cfff0 con 0x7f1f5c0c9fc0 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.423+0000 7f1f5a59c700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c158670 0x7f1f5c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.423+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f440035b0 con 0x7f1f5c0c9fc0 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1181248982 0 0) 0x7f1f440035b0 con 0x7f1f5c0c9fc0 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c158670 msgr2=0x7f1f5c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c158670 0x7f1f5c0622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 >> v1:172.21.15.203:6789/0 conn(0x7f1f5c158a50 legacy=0x7f1f5c0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c162230 con 0x7f1f5c0c9fc0 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f1f54001a10 con 0x7f1f5c0c9fc0 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 >> v1:172.21.15.83:6789/0 conn(0x7f1f5c0c9fc0 legacy=0x7f1f5c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f44003a30 0x7f1f44003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f44004850 0x7f1f44004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c158670 0x7f1f44005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c161f50 con 0x7f1f44004850 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4400bb40 con 0x7f1f44003a30 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4400bd00 con 0x7f1f5c158670 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f59d9b700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f44004850 0x7f1f44004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5a59c700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f44003a30 0x7f1f44003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f5ad9d700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c158670 0x7f1f44005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f59d9b700 1 -- 172.21.15.83:0/2549778375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c158670 msgr2=0x7f1f44005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f59d9b700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c158670 0x7f1f44005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f59d9b700 1 -- 172.21.15.83:0/2549778375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f44003a30 msgr2=0x7f1f44003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f59d9b700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f44003a30 0x7f1f44003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f59d9b700 1 -- 172.21.15.83:0/2549778375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f440035b0 con 0x7f1f44004850 2023-11-26T07:12:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.424+0000 7f1f59d9b700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f44004850 0x7f1f44004cd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1f500017c0 tx=0x7f1f44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1ae3a9a047e9d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.425+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f500052d0 con 0x7f1f44004850 2023-11-26T07:12:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.425+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f50005430 con 0x7f1f44004850 2023-11-26T07:12:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.425+0000 7f1f5959a700 1 -- 172.21.15.83:0/2549778375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f500038c0 con 0x7f1f44004850 2023-11-26T07:12:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.425+0000 7f1f60a31700 1 -- 172.21.15.83:0/2549778375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f44004850 msgr2=0x7f1f44004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.425+0000 7f1f60a31700 1 --2- 172.21.15.83:0/2549778375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f44004850 0x7f1f44004cd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1f500017c0 tx=0x7f1f44005880 comp rx=0 tx=0).stop 2023-11-26T07:12:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.425+0000 7f1f5ad9d700 1 -- 172.21.15.83:0/2549778375 reap_dead start 2023-11-26T07:12:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.425+0000 7f1f60a31700 1 -- 172.21.15.83:0/2549778375 shutdown_connections 2023-11-26T07:12:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.425+0000 7f1f60a31700 1 -- 172.21.15.83:0/2549778375 >> 172.21.15.83:0/2549778375 conn(0x7f1f5c0b57b0 msgr2=0x7f1f5c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.425+0000 7f1f60a31700 1 -- 172.21.15.83:0/2549778375 shutdown_connections 2023-11-26T07:12:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.425+0000 7f1f60a31700 1 -- 172.21.15.83:0/2549778375 wait complete. 2023-11-26T07:12:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.426+0000 7f1f60a31700 1 Processor -- start 2023-11-26T07:12:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.427+0000 7f1f60a31700 1 -- start start 2023-11-26T07:12:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.427+0000 7f1f60a31700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f44004850 0x7f1f5c155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.427+0000 7f1f60a31700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f5c0b51d0 0x7f1f5c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.427+0000 7f1f60a31700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0cac20 0x7f1f5c0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.427+0000 7f1f60a31700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0b4410 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.427+0000 7f1f60a31700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0b46f0 con 0x7f1f44004850 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.427+0000 7f1f60a31700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0b49d0 con 0x7f1f5c0cac20 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.427+0000 7f1f59d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f5c0b51d0 0x7f1f5c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.427+0000 7f1f59d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f5c0b51d0 0x7f1f5c0cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36664/0 (socket says 172.21.15.83:36664) 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.427+0000 7f1f59d9b700 1 -- 172.21.15.83:0/2793848937 learned_addr learned my addr 172.21.15.83:0/2793848937 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f59d9b700 1 -- 172.21.15.83:0/2793848937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0cac20 msgr2=0x7f1f5c0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f59d9b700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0cac20 0x7f1f5c0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f59d9b700 1 -- 172.21.15.83:0/2793848937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f44004850 msgr2=0x7f1f5c155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f5a59c700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f44004850 0x7f1f5c155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f59d9b700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f44004850 0x7f1f5c155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f59d9b700 1 -- 172.21.15.83:0/2793848937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c0b4cb0 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f59d9b700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f5c0b51d0 0x7f1f5c0cb5b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400bb10 tx=0x7f1f50001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f4affd700 1 -- 172.21.15.83:0/2793848937 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50007240 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f4affd700 1 -- 172.21.15.83:0/2793848937 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f500073a0 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f4affd700 1 -- 172.21.15.83:0/2793848937 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50001290 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f60a31700 1 -- 172.21.15.83:0/2793848937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c0cbaf0 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.428+0000 7f1f60a31700 1 -- 172.21.15.83:0/2793848937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c0cd620 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.429+0000 7f1f60a31700 1 -- 172.21.15.83:0/2793848937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c0b5020 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.429+0000 7f1f4affd700 1 -- 172.21.15.83:0/2793848937 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f1f50003680 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.430+0000 7f1f4affd700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f3c06b080 0x7f1f3c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.430+0000 7f1f4affd700 1 -- 172.21.15.83:0/2793848937 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f5008a7d0 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.430+0000 7f1f5a59c700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f3c06b080 0x7f1f3c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.430+0000 7f1f5a59c700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f3c06b080 0x7f1f3c06d540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0afcf0 tx=0x7f1f4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:41.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.434+0000 7f1f4affd700 1 -- 172.21.15.83:0/2793848937 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f50055690 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.625+0000 7f1f60a31700 1 -- 172.21.15.83:0/2793848937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f5c0b4e40 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.626+0000 7f1f4affd700 1 -- 172.21.15.83:0/2793848937 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16557 (secure 0 0 0) 0x7f1f50058cb0 con 0x7f1f5c0b51d0 2023-11-26T07:12:41.629 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:41.629 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:24.102531+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:20.302489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:21.468154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.628+0000 7f1f60a31700 1 -- 172.21.15.83:0/2793848937 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f3c06b080 msgr2=0x7f1f3c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.628+0000 7f1f60a31700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f3c06b080 0x7f1f3c06d540 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0afcf0 tx=0x7f1f4c008040 comp rx=0 tx=0).stop 2023-11-26T07:12:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.628+0000 7f1f60a31700 1 -- 172.21.15.83:0/2793848937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f5c0b51d0 msgr2=0x7f1f5c0cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.628+0000 7f1f60a31700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f5c0b51d0 0x7f1f5c0cb5b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400bb10 tx=0x7f1f50001120 comp rx=0 tx=0).stop 2023-11-26T07:12:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.628+0000 7f1f60a31700 1 -- 172.21.15.83:0/2793848937 shutdown_connections 2023-11-26T07:12:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.628+0000 7f1f60a31700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f3c06b080 0x7f1f3c06d540 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.628+0000 7f1f60a31700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f5c0cac20 0x7f1f5c0cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.628+0000 7f1f60a31700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f44004850 0x7f1f5c155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.628+0000 7f1f60a31700 1 --2- 172.21.15.83:0/2793848937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f5c0b51d0 0x7f1f5c0cb5b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.628+0000 7f1f60a31700 1 -- 172.21.15.83:0/2793848937 >> 172.21.15.83:0/2793848937 conn(0x7f1f5c0b57b0 msgr2=0x7f1f5c06e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.629+0000 7f1f60a31700 1 -- 172.21.15.83:0/2793848937 shutdown_connections 2023-11-26T07:12:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.629+0000 7f1f60a31700 1 -- 172.21.15.83:0/2793848937 wait complete. 2023-11-26T07:12:41.643 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:12:41.643 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.760+0000 7f955b769700 1 Processor -- start 2023-11-26T07:12:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.761+0000 7f955b769700 1 -- start start 2023-11-26T07:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.761+0000 7f955b769700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 0x7f95541604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.761+0000 7f955b769700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f95540d1c70 0x7f95540d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.761+0000 7f955b769700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95540b22d0 con 0x7f95540b3370 2023-11-26T07:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.761+0000 7f955b769700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95540b2f50 con 0x7f95540d1c70 2023-11-26T07:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.761+0000 7f955b769700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95540d2640 con 0x7f955415bdd0 2023-11-26T07:12:41.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f9558d04700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f95540b3370 0x7f95540b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56710/0 (socket says 172.21.15.83:56710) 2023-11-26T07:12:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f9558d04700 1 -- 172.21.15.83:0/1934542734 learned_addr learned my addr 172.21.15.83:0/1934542734 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f9559505700 1 --2- 172.21.15.83:0/1934542734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 0x7f95541604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f9559d06700 1 --2- 172.21.15.83:0/1934542734 >> v2:172.21.15.203:3300/0 conn(0x7f95540d1c70 0x7f95540d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f954bfff700 1 -- 172.21.15.83:0/1934542734 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511889720 0 0) 0x7f95540b22d0 con 0x7f95540b3370 2023-11-26T07:12:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f954bfff700 1 -- 172.21.15.83:0/1934542734 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95400035b0 con 0x7f95540b3370 2023-11-26T07:12:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f9559d06700 1 -- 172.21.15.83:0/1934542734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 msgr2=0x7f95541604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f9559d06700 1 --2- 172.21.15.83:0/1934542734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 0x7f95541604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f9559d06700 1 -- 172.21.15.83:0/1934542734 >> v1:172.21.15.83:6789/0 conn(0x7f95540b3370 legacy=0x7f95540b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f9559d06700 1 -- 172.21.15.83:0/1934542734 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95540b25b0 con 0x7f95540d1c70 2023-11-26T07:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f9559505700 1 --2- 172.21.15.83:0/1934542734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 0x7f95541604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f9559d06700 1 --2- 172.21.15.83:0/1934542734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95540d1c70 0x7f95540d2100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f954c009d90 tx=0x7f954c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49bc779341ac1bb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.762+0000 7f954bfff700 1 -- 172.21.15.83:0/1934542734 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c00e070 con 0x7f95540d1c70 2023-11-26T07:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.763+0000 7f954bfff700 1 -- 172.21.15.83:0/1934542734 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c00bb20 con 0x7f95540d1c70 2023-11-26T07:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.763+0000 7f954bfff700 1 -- 172.21.15.83:0/1934542734 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c012430 con 0x7f95540d1c70 2023-11-26T07:12:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.763+0000 7f955b769700 1 -- 172.21.15.83:0/1934542734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95540d1c70 msgr2=0x7f95540d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.763+0000 7f955b769700 1 --2- 172.21.15.83:0/1934542734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95540d1c70 0x7f95540d2100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f954c009d90 tx=0x7f954c00b330 comp rx=0 tx=0).stop 2023-11-26T07:12:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.763+0000 7f955b769700 1 -- 172.21.15.83:0/1934542734 shutdown_connections 2023-11-26T07:12:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.763+0000 7f955b769700 1 --2- 172.21.15.83:0/1934542734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95540d1c70 0x7f95540d2100 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.763+0000 7f955b769700 1 --2- 172.21.15.83:0/1934542734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 0x7f95541604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.763+0000 7f955b769700 1 -- 172.21.15.83:0/1934542734 >> 172.21.15.83:0/1934542734 conn(0x7f95540731a0 msgr2=0x7f955409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.763+0000 7f955b769700 1 -- 172.21.15.83:0/1934542734 shutdown_connections 2023-11-26T07:12:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.763+0000 7f955b769700 1 -- 172.21.15.83:0/1934542734 wait complete. 2023-11-26T07:12:41.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.764+0000 7f955b769700 1 Processor -- start 2023-11-26T07:12:41.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.764+0000 7f955b769700 1 -- start start 2023-11-26T07:12:41.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f955b769700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95540b3370 0x7f955414a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f955b769700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95540d1c70 0x7f95540b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f955b769700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 0x7f95540cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f955b769700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95540cabb0 con 0x7f95540d1c70 2023-11-26T07:12:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f955b769700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9554148dc0 con 0x7f95540b3370 2023-11-26T07:12:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f955b769700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955414b230 con 0x7f955415bdd0 2023-11-26T07:12:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9558d04700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95540d1c70 0x7f95540b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9558d04700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95540d1c70 0x7f95540b7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36678/0 (socket says 172.21.15.83:36678) 2023-11-26T07:12:41.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9558d04700 1 -- 172.21.15.83:0/2675025793 learned_addr learned my addr 172.21.15.83:0/2675025793 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9559505700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95540b3370 0x7f955414a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9559d06700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 0x7f95540cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9558d04700 1 -- 172.21.15.83:0/2675025793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 msgr2=0x7f95540cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9558d04700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 0x7f95540cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9558d04700 1 -- 172.21.15.83:0/2675025793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95540b3370 msgr2=0x7f955414a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9558d04700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95540b3370 0x7f955414a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9558d04700 1 -- 172.21.15.83:0/2675025793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440094a0 con 0x7f95540d1c70 2023-11-26T07:12:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9559505700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95540b3370 0x7f955414a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.765+0000 7f9559d06700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 0x7f95540cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:41.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.766+0000 7f9558d04700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95540d1c70 0x7f95540b7020 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f955000c700 tx=0x7f955000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:41.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.766+0000 7f9549ffb700 1 -- 172.21.15.83:0/2675025793 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550005d70 con 0x7f95540d1c70 2023-11-26T07:12:41.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.766+0000 7f9549ffb700 1 -- 172.21.15.83:0/2675025793 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95500070d0 con 0x7f95540d1c70 2023-11-26T07:12:41.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.766+0000 7f9549ffb700 1 -- 172.21.15.83:0/2675025793 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550007460 con 0x7f95540d1c70 2023-11-26T07:12:41.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.766+0000 7f955b769700 1 -- 172.21.15.83:0/2675025793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c009a70 con 0x7f95540d1c70 2023-11-26T07:12:41.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.766+0000 7f955b769700 1 -- 172.21.15.83:0/2675025793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955414b7f0 con 0x7f95540d1c70 2023-11-26T07:12:41.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.767+0000 7f955b769700 1 -- 172.21.15.83:0/2675025793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95540cad20 con 0x7f95540d1c70 2023-11-26T07:12:41.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.767+0000 7f9549ffb700 1 -- 172.21.15.83:0/2675025793 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f95500075c0 con 0x7f95540d1c70 2023-11-26T07:12:41.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.768+0000 7f9549ffb700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953806b080 0x7f953806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:41.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.768+0000 7f9549ffb700 1 -- 172.21.15.83:0/2675025793 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9550088f60 con 0x7f95540d1c70 2023-11-26T07:12:41.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.768+0000 7f9559505700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953806b080 0x7f953806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:41.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.768+0000 7f9559505700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953806b080 0x7f953806d540 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f95541f4ed0 tx=0x7f9544008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:41.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:41.772+0000 7f9549ffb700 1 -- 172.21.15.83:0/2675025793 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95540cad20 con 0x7f95540d1c70 2023-11-26T07:12:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.005+0000 7f955b769700 1 -- 172.21.15.83:0/2675025793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95540cacf0 con 0x7f95540d1c70 2023-11-26T07:12:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.006+0000 7f9549ffb700 1 -- 172.21.15.83:0/2675025793 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f9550053da0 con 0x7f95540d1c70 2023-11-26T07:12:42.008 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:42.009 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/486975373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":486975373},{"type":"v1","addr":"172.21.15.203:6834","nonce":486975373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3188343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3188343240},{"type":"v1","addr":"172.21.15.83:6836","nonce":3188343240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4271462644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4271462644},{"type":"v1","addr":"172.21.15.83:6837","nonce":4271462644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:22.505426+0000","modified":"2023-11-26T07:12:25.894494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6835/995330476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":995330476},{"type":"v1","addr":"172.21.15.203:6835","nonce":995330476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:12:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 -- 172.21.15.83:0/2675025793 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953806b080 msgr2=0x7f953806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953806b080 0x7f953806d540 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f95541f4ed0 tx=0x7f9544008040 comp rx=0 tx=0).stop 2023-11-26T07:12:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 -- 172.21.15.83:0/2675025793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95540d1c70 msgr2=0x7f95540b7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95540d1c70 0x7f95540b7020 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f955000c700 tx=0x7f955000cac0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 -- 172.21.15.83:0/2675025793 shutdown_connections 2023-11-26T07:12:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f953806b080 0x7f953806d540 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f955415bdd0 0x7f95540cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95540b3370 0x7f955414a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 --2- 172.21.15.83:0/2675025793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95540d1c70 0x7f95540b7020 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 -- 172.21.15.83:0/2675025793 >> 172.21.15.83:0/2675025793 conn(0x7f95540731a0 msgr2=0x7f955406ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 -- 172.21.15.83:0/2675025793 shutdown_connections 2023-11-26T07:12:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.008+0000 7f955b769700 1 -- 172.21.15.83:0/2675025793 wait complete. 2023-11-26T07:12:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2023-11-26T07:12:42.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:42.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.140+0000 7fe1b1ce4700 1 Processor -- start 2023-11-26T07:12:42.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.140+0000 7fe1b1ce4700 1 -- start start 2023-11-26T07:12:42.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1b1ce4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac15e080 0x7fe1ac0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1b1ce4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe1ac06c710 0x7fe1ac0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1b1ce4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1ac0ae430 con 0x7fe1ac0af4d0 2023-11-26T07:12:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1b1ce4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1ac0af0b0 con 0x7fe1ac06c710 2023-11-26T07:12:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1b1ce4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac0b05a0 con 0x7fe1ac15e080 2023-11-26T07:12:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aaffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe1ac0af4d0 0x7fe1ac06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56726/0 (socket says 172.21.15.83:56726) 2023-11-26T07:12:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aaffd700 1 -- 172.21.15.83:0/1794358871 learned_addr learned my addr 172.21.15.83:0/1794358871 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1ab7fe700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac15e080 0x7fe1ac0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1abfff700 1 --2- 172.21.15.83:0/1794358871 >> v2:172.21.15.203:3300/0 conn(0x7fe1ac06c710 0x7fe1ac0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686963841 0 0) 0x7fe1ac0ae430 con 0x7fe1ac0af4d0 2023-11-26T07:12:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1940035b0 con 0x7fe1ac0af4d0 2023-11-26T07:12:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3016506732 0 0) 0x7fe1940035b0 con 0x7fe1ac0af4d0 2023-11-26T07:12:42.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac15e080 msgr2=0x7fe1ac0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aa7fc700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac15e080 0x7fe1ac0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 >> v2:172.21.15.203:3300/0 conn(0x7fe1ac06c710 msgr2=0x7fe1ac0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aa7fc700 1 --2- 172.21.15.83:0/1794358871 >> v2:172.21.15.203:3300/0 conn(0x7fe1ac06c710 0x7fe1ac0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac0ae710 con 0x7fe1ac0af4d0 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fe19c001670 con 0x7fe1ac0af4d0 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.141+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 >> v1:172.21.15.83:6789/0 conn(0x7fe1ac0af4d0 legacy=0x7fe1ac06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aa7fc700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe194003a30 0x7fe194003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aa7fc700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 0x7fe194004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aa7fc700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac15e080 0x7fe194005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac0af0b0 con 0x7fe194004850 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac0b05a0 con 0x7fe194003a30 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19400ba10 con 0x7fe1ac15e080 2023-11-26T07:12:42.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aaffd700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 0x7fe194004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1ab7fe700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe194003a30 0x7fe194003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1abfff700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac15e080 0x7fe194005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aaffd700 1 -- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac15e080 msgr2=0x7fe194005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aaffd700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac15e080 0x7fe194005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aaffd700 1 -- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe194003a30 msgr2=0x7fe194003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aaffd700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe194003a30 0x7fe194003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aaffd700 1 -- 172.21.15.83:0/1794358871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1940035b0 con 0x7fe194004850 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1abfff700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac15e080 0x7fe194005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aaffd700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 0x7fe194004cd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00c210 tx=0x7fe194005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88ea5df58201b9b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c007c10 con 0x7fe194004850 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19c00cdb0 con 0x7fe194004850 2023-11-26T07:12:42.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.142+0000 7fe1aa7fc700 1 -- 172.21.15.83:0/1794358871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c005ba0 con 0x7fe194004850 2023-11-26T07:12:42.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.143+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/1794358871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 msgr2=0x7fe194004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.143+0000 7fe1b1ce4700 1 --2- 172.21.15.83:0/1794358871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 0x7fe194004cd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00c210 tx=0x7fe194005880 comp rx=0 tx=0).stop 2023-11-26T07:12:42.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.143+0000 7fe1abfff700 1 -- 172.21.15.83:0/1794358871 reap_dead start 2023-11-26T07:12:42.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.143+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/1794358871 shutdown_connections 2023-11-26T07:12:42.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.143+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/1794358871 >> 172.21.15.83:0/1794358871 conn(0x7fe1ac071b90 msgr2=0x7fe1ac071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:42.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.143+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/1794358871 shutdown_connections 2023-11-26T07:12:42.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.143+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/1794358871 wait complete. 2023-11-26T07:12:42.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.144+0000 7fe1b1ce4700 1 Processor -- start 2023-11-26T07:12:42.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.144+0000 7fe1b1ce4700 1 -- start start 2023-11-26T07:12:42.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1b1ce4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 0x7fe1ac1f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1b1ce4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1ac149610 0x7fe1ac157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1b1ce4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac1581a0 0x7fe1ac155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1b1ce4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac152fd0 con 0x7fe194004850 2023-11-26T07:12:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1b1ce4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac15f750 con 0x7fe1ac149610 2023-11-26T07:12:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1b1ce4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac15fa30 con 0x7fe1ac1581a0 2023-11-26T07:12:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1ab7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 0x7fe1ac1f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1ab7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 0x7fe1ac1f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36696/0 (socket says 172.21.15.83:36696) 2023-11-26T07:12:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1ab7fe700 1 -- 172.21.15.83:0/927896547 learned_addr learned my addr 172.21.15.83:0/927896547 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1ab7fe700 1 -- 172.21.15.83:0/927896547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac1581a0 msgr2=0x7fe1ac155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1ab7fe700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac1581a0 0x7fe1ac155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1aaffd700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1ac149610 0x7fe1ac157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1ab7fe700 1 -- 172.21.15.83:0/927896547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1ac149610 msgr2=0x7fe1ac157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1ab7fe700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1ac149610 0x7fe1ac157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1ab7fe700 1 -- 172.21.15.83:0/927896547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac15fd10 con 0x7fe194004850 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1aaffd700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1ac149610 0x7fe1ac157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.145+0000 7fe1ab7fe700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 0x7fe1ac1f3bb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0011920 tx=0x7fe1a0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.146+0000 7fe193fff700 1 -- 172.21.15.83:0/927896547 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a000f940 con 0x7fe194004850 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.146+0000 7fe193fff700 1 -- 172.21.15.83:0/927896547 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a000faa0 con 0x7fe194004850 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.146+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/927896547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ac15fff0 con 0x7fe194004850 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.146+0000 7fe193fff700 1 -- 172.21.15.83:0/927896547 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a000fe30 con 0x7fe194004850 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.146+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/927896547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ac1f40f0 con 0x7fe194004850 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.147+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/927896547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ac0753f0 con 0x7fe194004850 2023-11-26T07:12:42.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.147+0000 7fe193fff700 1 -- 172.21.15.83:0/927896547 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe1a0014440 con 0x7fe194004850 2023-11-26T07:12:42.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.147+0000 7fe193fff700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe19806b080 0x7fe19806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.147+0000 7fe193fff700 1 -- 172.21.15.83:0/927896547 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1a0089ea0 con 0x7fe194004850 2023-11-26T07:12:42.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.147+0000 7fe1aaffd700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe19806b080 0x7fe19806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.148+0000 7fe1aaffd700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe19806b080 0x7fe19806d540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe19c015040 tx=0x7fe19c007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:42.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.152+0000 7fe193fff700 1 -- 172.21.15.83:0/927896547 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe1a0054d60 con 0x7fe194004850 2023-11-26T07:12:42.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.386+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/927896547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1ac067f60 con 0x7fe194004850 2023-11-26T07:12:42.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.386+0000 7fe193fff700 1 -- 172.21.15.83:0/927896547 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fe1a0058380 con 0x7fe194004850 2023-11-26T07:12:42.389 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:42.389 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/486975373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":486975373},{"type":"v1","addr":"172.21.15.203:6834","nonce":486975373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3188343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3188343240},{"type":"v1","addr":"172.21.15.83:6836","nonce":3188343240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4271462644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4271462644},{"type":"v1","addr":"172.21.15.83:6837","nonce":4271462644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:22.505426+0000","modified":"2023-11-26T07:12:25.894494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.203:6835/995330476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":995330476},{"type":"v1","addr":"172.21.15.203:6835","nonce":995330476}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:12:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.388+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/927896547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe19806b080 msgr2=0x7fe19806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.388+0000 7fe1b1ce4700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe19806b080 0x7fe19806d540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe19c015040 tx=0x7fe19c007000 comp rx=0 tx=0).stop 2023-11-26T07:12:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.388+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/927896547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 msgr2=0x7fe1ac1f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.388+0000 7fe1b1ce4700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 0x7fe1ac1f3bb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0011920 tx=0x7fe1a0011ce0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.388+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/927896547 shutdown_connections 2023-11-26T07:12:42.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.388+0000 7fe1b1ce4700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe19806b080 0x7fe19806d540 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.388+0000 7fe1b1ce4700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1ac1581a0 0x7fe1ac155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.388+0000 7fe1b1ce4700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe194004850 0x7fe1ac1f3bb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.388+0000 7fe1b1ce4700 1 --2- 172.21.15.83:0/927896547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1ac149610 0x7fe1ac157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.388+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/927896547 >> 172.21.15.83:0/927896547 conn(0x7fe1ac071b90 msgr2=0x7fe1ac0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.389+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/927896547 shutdown_connections 2023-11-26T07:12:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.389+0000 7fe1b1ce4700 1 -- 172.21.15.83:0/927896547 wait complete. 2023-11-26T07:12:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 18 2023-11-26T07:12:42.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.523+0000 7fbf8b9d2700 1 Processor -- start 2023-11-26T07:12:42.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.523+0000 7fbf8b9d2700 1 -- start start 2023-11-26T07:12:42.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf8b9d2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 0x7fbf840622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf8b9d2700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbf840b2870 0x7fbf840afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf8b9d2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8415c540 con 0x7fbf840b02e0 2023-11-26T07:12:42.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf8b9d2700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf840b2cf0 con 0x7fbf840b2870 2023-11-26T07:12:42.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf8b9d2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf840b0770 con 0x7fbf8415c160 2023-11-26T07:12:42.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf8976e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 0x7fbf840622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf88f6d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbf840b2870 0x7fbf840afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf88f6d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbf840b2870 0x7fbf840afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:36158/0 (socket says 172.21.15.83:36158) 2023-11-26T07:12:42.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf88f6d700 1 -- 172.21.15.83:0/1710460899 learned_addr learned my addr 172.21.15.83:0/1710460899 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:42.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf89f6f700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fbf840b02e0 0x7fbf840ae1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:41366/0 (socket says 172.21.15.83:41366) 2023-11-26T07:12:42.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf8976e700 1 --2- 172.21.15.83:0/1710460899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 0x7fbf840622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:47162/0 (socket says 172.21.15.83:47162) 2023-11-26T07:12:42.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf8976e700 1 -- 172.21.15.83:0/1710460899 >> v2:172.21.15.203:3300/0 conn(0x7fbf840b2870 msgr2=0x7fbf840afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:42.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf8976e700 1 --2- 172.21.15.83:0/1710460899 >> v2:172.21.15.203:3300/0 conn(0x7fbf840b2870 0x7fbf840afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf8976e700 1 -- 172.21.15.83:0/1710460899 >> v1:172.21.15.203:6789/0 conn(0x7fbf840b02e0 legacy=0x7fbf840ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf7bfff700 1 -- 172.21.15.83:0/1710460899 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783534749 0 0) 0x7fbf8415c540 con 0x7fbf840b02e0 2023-11-26T07:12:42.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf8976e700 1 -- 172.21.15.83:0/1710460899 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf840b2ee0 con 0x7fbf8415c160 2023-11-26T07:12:42.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.524+0000 7fbf88f6d700 1 --2- 172.21.15.83:0/1710460899 >> v2:172.21.15.203:3300/0 conn(0x7fbf840b2870 0x7fbf840afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:12:42.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf8976e700 1 --2- 172.21.15.83:0/1710460899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 0x7fbf840622a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009840 tx=0x7fbf80009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9d3f187cbf8ea6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:42.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf7bfff700 1 -- 172.21.15.83:0/1710460899 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80011830 con 0x7fbf8415c160 2023-11-26T07:12:42.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf7bfff700 1 -- 172.21.15.83:0/1710460899 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf80011990 con 0x7fbf8415c160 2023-11-26T07:12:42.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf7bfff700 1 -- 172.21.15.83:0/1710460899 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80011d20 con 0x7fbf8415c160 2023-11-26T07:12:42.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/1710460899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 msgr2=0x7fbf840622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf8b9d2700 1 --2- 172.21.15.83:0/1710460899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 0x7fbf840622a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009840 tx=0x7fbf80009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:42.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/1710460899 shutdown_connections 2023-11-26T07:12:42.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf8b9d2700 1 --2- 172.21.15.83:0/1710460899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 0x7fbf840622a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf8b9d2700 1 --2- 172.21.15.83:0/1710460899 >> v2:172.21.15.203:3300/0 conn(0x7fbf840b2870 0x7fbf840afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/1710460899 >> 172.21.15.83:0/1710460899 conn(0x7fbf840b57b0 msgr2=0x7fbf840b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:42.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/1710460899 shutdown_connections 2023-11-26T07:12:42.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.525+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/1710460899 wait complete. 2023-11-26T07:12:42.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.527+0000 7fbf8b9d2700 1 Processor -- start 2023-11-26T07:12:42.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.527+0000 7fbf8b9d2700 1 -- start start 2023-11-26T07:12:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.527+0000 7fbf8b9d2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf840b02e0 0x7fbf8415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.527+0000 7fbf8b9d2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf840b2870 0x7fbf840c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.527+0000 7fbf8b9d2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 0x7fbf8414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.527+0000 7fbf8b9d2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8415d580 con 0x7fbf840b2870 2023-11-26T07:12:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.527+0000 7fbf8b9d2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8415c890 con 0x7fbf840b02e0 2023-11-26T07:12:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.527+0000 7fbf8b9d2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf840cbe10 con 0x7fbf8415c160 2023-11-26T07:12:42.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.527+0000 7fbf88f6d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf840b2870 0x7fbf840c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.527+0000 7fbf88f6d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf840b2870 0x7fbf840c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36712/0 (socket says 172.21.15.83:36712) 2023-11-26T07:12:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.527+0000 7fbf88f6d700 1 -- 172.21.15.83:0/385882515 learned_addr learned my addr 172.21.15.83:0/385882515 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf8976e700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf840b02e0 0x7fbf8415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf89f6f700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 0x7fbf8414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf88f6d700 1 -- 172.21.15.83:0/385882515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 msgr2=0x7fbf8414d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf88f6d700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 0x7fbf8414d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf88f6d700 1 -- 172.21.15.83:0/385882515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf840b02e0 msgr2=0x7fbf8415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf88f6d700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf840b02e0 0x7fbf8415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf88f6d700 1 -- 172.21.15.83:0/385882515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf74009550 con 0x7fbf840b2870 2023-11-26T07:12:42.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf88f6d700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf840b2870 0x7fbf840c9880 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400ebe0 tx=0x7fbf7400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:42.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf79ffb700 1 -- 172.21.15.83:0/385882515 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7400c850 con 0x7fbf840b2870 2023-11-26T07:12:42.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf79ffb700 1 -- 172.21.15.83:0/385882515 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7400c9b0 con 0x7fbf840b2870 2023-11-26T07:12:42.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/385882515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf800094a0 con 0x7fbf840b2870 2023-11-26T07:12:42.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf79ffb700 1 -- 172.21.15.83:0/385882515 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7400cd40 con 0x7fbf840b2870 2023-11-26T07:12:42.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.528+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/385882515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf840cc3d0 con 0x7fbf840b2870 2023-11-26T07:12:42.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.529+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/385882515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf840cca20 con 0x7fbf840b2870 2023-11-26T07:12:42.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.530+0000 7fbf79ffb700 1 -- 172.21.15.83:0/385882515 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fbf74013470 con 0x7fbf840b2870 2023-11-26T07:12:42.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.530+0000 7fbf79ffb700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf6806b080 0x7fbf6806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.530+0000 7fbf79ffb700 1 -- 172.21.15.83:0/385882515 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf74089af0 con 0x7fbf840b2870 2023-11-26T07:12:42.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.530+0000 7fbf8976e700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf6806b080 0x7fbf6806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.530+0000 7fbf8976e700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf6806b080 0x7fbf6806d540 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbf800090b0 tx=0x7fbf8001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:42.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.534+0000 7fbf79ffb700 1 -- 172.21.15.83:0/385882515 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf74054920 con 0x7fbf840b2870 2023-11-26T07:12:42.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.727+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/385882515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf84067f60 con 0x7fbf840b2870 2023-11-26T07:12:42.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.729+0000 7fbf79ffb700 1 -- 172.21.15.83:0/385882515 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16557 (secure 0 0 0) 0x7fbf74057f40 con 0x7fbf840b2870 2023-11-26T07:12:42.731 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:42.731 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":34,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:24.102531+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:20.302489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:21.468154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/385882515 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf6806b080 msgr2=0x7fbf6806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf6806b080 0x7fbf6806d540 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbf800090b0 tx=0x7fbf8001c040 comp rx=0 tx=0).stop 2023-11-26T07:12:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/385882515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf840b2870 msgr2=0x7fbf840c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf840b2870 0x7fbf840c9880 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400ebe0 tx=0x7fbf7400eef0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/385882515 shutdown_connections 2023-11-26T07:12:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf6806b080 0x7fbf6806d540 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf8415c160 0x7fbf8414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf840b02e0 0x7fbf8415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 --2- 172.21.15.83:0/385882515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf840b2870 0x7fbf840c9880 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/385882515 >> 172.21.15.83:0/385882515 conn(0x7fbf840b57b0 msgr2=0x7fbf840cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/385882515 shutdown_connections 2023-11-26T07:12:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.731+0000 7fbf8b9d2700 1 -- 172.21.15.83:0/385882515 wait complete. 2023-11-26T07:12:42.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:12:42.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.864+0000 7fd86d6c1700 1 Processor -- start 2023-11-26T07:12:42.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.864+0000 7fd86d6c1700 1 -- start start 2023-11-26T07:12:42.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.865+0000 7fd86d6c1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86815c030 0x7fd8680d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.865+0000 7fd86d6c1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd8680ccfd0 0x7fd8680d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.865+0000 7fd86d6c1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8680cd5d0 con 0x7fd8680ae910 2023-11-26T07:12:42.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.865+0000 7fd86d6c1700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8680cd870 con 0x7fd8680ccfd0 2023-11-26T07:12:42.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.865+0000 7fd86d6c1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680cdb50 con 0x7fd86815c030 2023-11-26T07:12:42.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.865+0000 7fd8667fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd8680ae910 0x7fd8680cc8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56758/0 (socket says 172.21.15.83:56758) 2023-11-26T07:12:42.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.865+0000 7fd8667fc700 1 -- 172.21.15.83:0/564720240 learned_addr learned my addr 172.21.15.83:0/564720240 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:42.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.865+0000 7fd8677fe700 1 --2- 172.21.15.83:0/564720240 >> v2:172.21.15.203:3300/0 conn(0x7fd8680ccfd0 0x7fd8680d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.865+0000 7fd866ffd700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86815c030 0x7fd8680d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.865+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118189216 0 0) 0x7fd8680cd5d0 con 0x7fd8680ae910 2023-11-26T07:12:42.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.865+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8500035b0 con 0x7fd8680ae910 2023-11-26T07:12:42.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3728874781 0 0) 0x7fd8500035b0 con 0x7fd8680ae910 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86815c030 msgr2=0x7fd8680d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86815c030 0x7fd8680d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 >> v2:172.21.15.203:3300/0 conn(0x7fd8680ccfd0 msgr2=0x7fd8680d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 --2- 172.21.15.83:0/564720240 >> v2:172.21.15.203:3300/0 conn(0x7fd8680ccfd0 0x7fd8680d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8680cde30 con 0x7fd8680ae910 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fd85c001670 con 0x7fd8680ae910 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 >> v1:172.21.15.83:6789/0 conn(0x7fd8680ae910 legacy=0x7fd8680cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd850003a00 0x7fd850003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd850004850 0x7fd850004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8500051d0 0x7fd85000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680cd870 con 0x7fd8500051d0 2023-11-26T07:12:42.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680cdb50 con 0x7fd850003a00 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd850004650 con 0x7fd850004850 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd8677fe700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8500051d0 0x7fd85000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd866ffd700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd850003a00 0x7fd850003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd8667fc700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd850004850 0x7fd850004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd8677fe700 1 -- 172.21.15.83:0/564720240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd850004850 msgr2=0x7fd850004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd8677fe700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd850004850 0x7fd850004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd8677fe700 1 -- 172.21.15.83:0/564720240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd850003a00 msgr2=0x7fd850003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd8677fe700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd850003a00 0x7fd850003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd8677fe700 1 -- 172.21.15.83:0/564720240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8500035b0 con 0x7fd8500051d0 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd8677fe700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8500051d0 0x7fd85000ba10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd860009fc0 tx=0x7fd850005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c842a2297c417a67 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86000ec30 con 0x7fd8500051d0 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86000f040 con 0x7fd8500051d0 2023-11-26T07:12:42.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.866+0000 7fd865ffb700 1 -- 172.21.15.83:0/564720240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860014480 con 0x7fd8500051d0 2023-11-26T07:12:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.867+0000 7fd86d6c1700 1 -- 172.21.15.83:0/564720240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8500051d0 msgr2=0x7fd85000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.867+0000 7fd86d6c1700 1 --2- 172.21.15.83:0/564720240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8500051d0 0x7fd85000ba10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd860009fc0 tx=0x7fd850005880 comp rx=0 tx=0).stop 2023-11-26T07:12:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.867+0000 7fd8677fe700 1 -- 172.21.15.83:0/564720240 reap_dead start 2023-11-26T07:12:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.867+0000 7fd86d6c1700 1 -- 172.21.15.83:0/564720240 shutdown_connections 2023-11-26T07:12:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.867+0000 7fd86d6c1700 1 -- 172.21.15.83:0/564720240 >> 172.21.15.83:0/564720240 conn(0x7fd8680730a0 msgr2=0x7fd86809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.867+0000 7fd86d6c1700 1 -- 172.21.15.83:0/564720240 shutdown_connections 2023-11-26T07:12:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.867+0000 7fd86d6c1700 1 -- 172.21.15.83:0/564720240 wait complete. 2023-11-26T07:12:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.868+0000 7fd86d6c1700 1 Processor -- start 2023-11-26T07:12:42.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.868+0000 7fd86d6c1700 1 -- start start 2023-11-26T07:12:42.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.869+0000 7fd86d6c1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8500051d0 0x7fd86806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.869+0000 7fd86d6c1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8680c8cf0 0x7fd8680c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.869+0000 7fd86d6c1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8680c67a0 0x7fd8680c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.869+0000 7fd86d6c1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680ca850 con 0x7fd8680c8cf0 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.869+0000 7fd86d6c1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680c97b0 con 0x7fd8500051d0 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.869+0000 7fd86d6c1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680c9a90 con 0x7fd8680c67a0 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.869+0000 7fd8667fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8680c8cf0 0x7fd8680c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.869+0000 7fd8667fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8680c8cf0 0x7fd8680c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36732/0 (socket says 172.21.15.83:36732) 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.869+0000 7fd8667fc700 1 -- 172.21.15.83:0/251749253 learned_addr learned my addr 172.21.15.83:0/251749253 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.869+0000 7fd8677fe700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8680c67a0 0x7fd8680c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.869+0000 7fd866ffd700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8500051d0 0x7fd86806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.870+0000 7fd8667fc700 1 -- 172.21.15.83:0/251749253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8680c67a0 msgr2=0x7fd8680c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.870+0000 7fd8667fc700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8680c67a0 0x7fd8680c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.870+0000 7fd8667fc700 1 -- 172.21.15.83:0/251749253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8500051d0 msgr2=0x7fd86806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.870+0000 7fd8667fc700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8500051d0 0x7fd86806c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.870+0000 7fd8667fc700 1 -- 172.21.15.83:0/251749253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8680c9d70 con 0x7fd8680c8cf0 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.870+0000 7fd8667fc700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8680c8cf0 0x7fd8680c9170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd85c000fb0 tx=0x7fd85c0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.870+0000 7fd84f7fe700 1 -- 172.21.15.83:0/251749253 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00f630 con 0x7fd8680c8cf0 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.870+0000 7fd84f7fe700 1 -- 172.21.15.83:0/251749253 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c00f7b0 con 0x7fd8680c8cf0 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.870+0000 7fd86d6c1700 1 -- 172.21.15.83:0/251749253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8680ca050 con 0x7fd8680c8cf0 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.870+0000 7fd84f7fe700 1 -- 172.21.15.83:0/251749253 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00fb60 con 0x7fd8680c8cf0 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.870+0000 7fd86d6c1700 1 -- 172.21.15.83:0/251749253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86806bdb0 con 0x7fd8680c8cf0 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.871+0000 7fd86d6c1700 1 -- 172.21.15.83:0/251749253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8680751e0 con 0x7fd8680c8cf0 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.871+0000 7fd84f7fe700 1 -- 172.21.15.83:0/251749253 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fd85c00fcc0 con 0x7fd8680c8cf0 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.872+0000 7fd84f7fe700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd85406b080 0x7fd85406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:42.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.872+0000 7fd84f7fe700 1 -- 172.21.15.83:0/251749253 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd85c089820 con 0x7fd8680c8cf0 2023-11-26T07:12:42.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.872+0000 7fd866ffd700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd85406b080 0x7fd85406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:42.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.872+0000 7fd866ffd700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd85406b080 0x7fd85406d540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd858009be0 tx=0x7fd858009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:42.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:42.876+0000 7fd84f7fe700 1 -- 172.21.15.83:0/251749253 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd85c0546e0 con 0x7fd8680c8cf0 2023-11-26T07:12:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.111+0000 7fd86d6c1700 1 -- 172.21.15.83:0/251749253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd8680ca570 con 0x7fd8680c8cf0 2023-11-26T07:12:43.210 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-11-26T07:12:43.210 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-26T07:12:43.112+0000 7fcd25823700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:12:43.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.113+0000 7fd84f7fe700 1 -- 172.21.15.83:0/251749253 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd8680ca570 con 0x7fd8680c8cf0 2023-11-26T07:12:43.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 -- 172.21.15.83:0/251749253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd85406b080 msgr2=0x7fd85406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:43.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd85406b080 0x7fd85406d540 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd858009be0 tx=0x7fd858009210 comp rx=0 tx=0).stop 2023-11-26T07:12:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 -- 172.21.15.83:0/251749253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8680c8cf0 msgr2=0x7fd8680c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8680c8cf0 0x7fd8680c9170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd85c000fb0 tx=0x7fd85c0075f0 comp rx=0 tx=0).stop 2023-11-26T07:12:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 -- 172.21.15.83:0/251749253 shutdown_connections 2023-11-26T07:12:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd85406b080 0x7fd85406d540 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8680c67a0 0x7fd8680c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8500051d0 0x7fd86806c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 --2- 172.21.15.83:0/251749253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8680c8cf0 0x7fd8680c9170 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 -- 172.21.15.83:0/251749253 >> 172.21.15.83:0/251749253 conn(0x7fd8680730a0 msgr2=0x7fd868070480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 -- 172.21.15.83:0/251749253 shutdown_connections 2023-11-26T07:12:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.115+0000 7fd86d6c1700 1 -- 172.21.15.83:0/251749253 wait complete. 2023-11-26T07:12:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:12:43.213 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:12:43.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.332+0000 7eff828af700 1 Processor -- start 2023-11-26T07:12:43.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.332+0000 7eff828af700 1 -- start start 2023-11-26T07:12:43.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.333+0000 7eff828af700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7eff7c145720 0x7eff7c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:43.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.333+0000 7eff828af700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c0b5730 0x7eff7c0b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:43.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.333+0000 7eff828af700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7c0b60d0 con 0x7eff7c15f8f0 2023-11-26T07:12:43.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.333+0000 7eff828af700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7c06b750 con 0x7eff7c145720 2023-11-26T07:12:43.338 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.333+0000 7eff828af700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c160870 con 0x7eff7c0b5730 2023-11-26T07:12:43.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff7bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7eff7c145720 0x7eff7c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:43.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff7b7fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c0b5730 0x7eff7c0b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:43.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff80e4c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7eff7c15f8f0 0x7eff7c15fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:41398/0 (socket says 172.21.15.83:41398) 2023-11-26T07:12:43.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff80e4c700 1 -- 172.21.15.83:0/3415449425 learned_addr learned my addr 172.21.15.83:0/3415449425 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:43.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff7b7fe700 1 -- 172.21.15.83:0/3415449425 >> v2:172.21.15.203:3300/0 conn(0x7eff7c145720 msgr2=0x7eff7c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:43.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff7affd700 1 -- 172.21.15.83:0/3415449425 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142579437 0 0) 0x7eff7c0b60d0 con 0x7eff7c15f8f0 2023-11-26T07:12:43.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff7b7fe700 1 --2- 172.21.15.83:0/3415449425 >> v2:172.21.15.203:3300/0 conn(0x7eff7c145720 0x7eff7c06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:43.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff7b7fe700 1 -- 172.21.15.83:0/3415449425 >> v1:172.21.15.203:6789/0 conn(0x7eff7c15f8f0 legacy=0x7eff7c15fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:43.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff7b7fe700 1 -- 172.21.15.83:0/3415449425 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7c160590 con 0x7eff7c0b5730 2023-11-26T07:12:43.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff7b7fe700 1 --2- 172.21.15.83:0/3415449425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c0b5730 0x7eff7c0b5b90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009840 tx=0x7eff6c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d9b5cfe0f8083af server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:43.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff7affd700 1 -- 172.21.15.83:0/3415449425 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c011830 con 0x7eff7c0b5730 2023-11-26T07:12:43.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff7affd700 1 -- 172.21.15.83:0/3415449425 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6c011990 con 0x7eff7c0b5730 2023-11-26T07:12:43.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.334+0000 7eff7affd700 1 -- 172.21.15.83:0/3415449425 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c011d20 con 0x7eff7c0b5730 2023-11-26T07:12:43.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.335+0000 7eff828af700 1 -- 172.21.15.83:0/3415449425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c0b5730 msgr2=0x7eff7c0b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:43.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.335+0000 7eff828af700 1 --2- 172.21.15.83:0/3415449425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c0b5730 0x7eff7c0b5b90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009840 tx=0x7eff6c009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:43.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.335+0000 7eff828af700 1 -- 172.21.15.83:0/3415449425 shutdown_connections 2023-11-26T07:12:43.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.335+0000 7eff828af700 1 --2- 172.21.15.83:0/3415449425 >> v2:172.21.15.203:3300/0 conn(0x7eff7c145720 0x7eff7c06bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:43.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.335+0000 7eff828af700 1 --2- 172.21.15.83:0/3415449425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c0b5730 0x7eff7c0b5b90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:43.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.335+0000 7eff828af700 1 -- 172.21.15.83:0/3415449425 >> 172.21.15.83:0/3415449425 conn(0x7eff7c0d0be0 msgr2=0x7eff7c0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:43.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.335+0000 7eff828af700 1 -- 172.21.15.83:0/3415449425 shutdown_connections 2023-11-26T07:12:43.342 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.335+0000 7eff828af700 1 -- 172.21.15.83:0/3415449425 wait complete. 2023-11-26T07:12:43.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.336+0000 7eff828af700 1 Processor -- start 2023-11-26T07:12:43.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.336+0000 7eff828af700 1 -- start start 2023-11-26T07:12:43.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff828af700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff7c0b5730 0x7eff7c15d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:43.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff828af700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff7c145720 0x7eff7c13fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff828af700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c15f8f0 0x7eff7c09cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff828af700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c15d870 con 0x7eff7c145720 2023-11-26T07:12:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff828af700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c15e300 con 0x7eff7c0b5730 2023-11-26T07:12:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff828af700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c09b740 con 0x7eff7c15f8f0 2023-11-26T07:12:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff7b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff7c145720 0x7eff7c13fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff7b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff7c145720 0x7eff7c13fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36746/0 (socket says 172.21.15.83:36746) 2023-11-26T07:12:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff7b7fe700 1 -- 172.21.15.83:0/2042579101 learned_addr learned my addr 172.21.15.83:0/2042579101 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff7bfff700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff7c0b5730 0x7eff7c15d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff80e4c700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c15f8f0 0x7eff7c09cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff7b7fe700 1 -- 172.21.15.83:0/2042579101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c15f8f0 msgr2=0x7eff7c09cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff7b7fe700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c15f8f0 0x7eff7c09cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:43.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff7b7fe700 1 -- 172.21.15.83:0/2042579101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff7c0b5730 msgr2=0x7eff7c15d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:43.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff7b7fe700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff7c0b5730 0x7eff7c15d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:43.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff7b7fe700 1 -- 172.21.15.83:0/2042579101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c0094a0 con 0x7eff7c145720 2023-11-26T07:12:43.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.337+0000 7eff80e4c700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c15f8f0 0x7eff7c09cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:43.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.338+0000 7eff7b7fe700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff7c145720 0x7eff7c13fa00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009fd0 tx=0x7eff6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:43.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.338+0000 7eff78ff9700 1 -- 172.21.15.83:0/2042579101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c0036a0 con 0x7eff7c145720 2023-11-26T07:12:43.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.338+0000 7eff78ff9700 1 -- 172.21.15.83:0/2042579101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6c01f070 con 0x7eff7c145720 2023-11-26T07:12:43.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.338+0000 7eff78ff9700 1 -- 172.21.15.83:0/2042579101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c003a20 con 0x7eff7c145720 2023-11-26T07:12:43.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.338+0000 7eff828af700 1 -- 172.21.15.83:0/2042579101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c15da60 con 0x7eff7c145720 2023-11-26T07:12:43.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.338+0000 7eff828af700 1 -- 172.21.15.83:0/2042579101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c15e020 con 0x7eff7c145720 2023-11-26T07:12:43.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.339+0000 7eff828af700 1 -- 172.21.15.83:0/2042579101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7c068fb0 con 0x7eff7c145720 2023-11-26T07:12:43.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.339+0000 7eff78ff9700 1 -- 172.21.15.83:0/2042579101 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7eff6c003b80 con 0x7eff7c145720 2023-11-26T07:12:43.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.340+0000 7eff78ff9700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff6806b080 0x7eff6806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:43.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.340+0000 7eff7bfff700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff6806b080 0x7eff6806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:43.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.340+0000 7eff78ff9700 1 -- 172.21.15.83:0/2042579101 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7eff6c021070 con 0x7eff7c145720 2023-11-26T07:12:43.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.340+0000 7eff7bfff700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff6806b080 0x7eff6806d540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff70009be0 tx=0x7eff70009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:43.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.344+0000 7eff78ff9700 1 -- 172.21.15.83:0/2042579101 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff6c056090 con 0x7eff7c145720 2023-11-26T07:12:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:43.578+0000 7eff828af700 1 -- 172.21.15.83:0/2042579101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7eff7c0b41d0 con 0x7eff7c145720 2023-11-26T07:12:44.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.115+0000 7eff78ff9700 1 -- 172.21.15.83:0/2042579101 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7eff6c0596b0 con 0x7eff7c145720 2023-11-26T07:12:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 -- 172.21.15.83:0/2042579101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff6806b080 msgr2=0x7eff6806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff6806b080 0x7eff6806d540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff70009be0 tx=0x7eff70009210 comp rx=0 tx=0).stop 2023-11-26T07:12:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 -- 172.21.15.83:0/2042579101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff7c145720 msgr2=0x7eff7c13fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff7c145720 0x7eff7c13fa00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009fd0 tx=0x7eff6c004750 comp rx=0 tx=0).stop 2023-11-26T07:12:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 -- 172.21.15.83:0/2042579101 shutdown_connections 2023-11-26T07:12:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff6806b080 0x7eff6806d540 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:44.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7c15f8f0 0x7eff7c09cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:44.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff7c0b5730 0x7eff7c15d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:44.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 --2- 172.21.15.83:0/2042579101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff7c145720 0x7eff7c13fa00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:44.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 -- 172.21.15.83:0/2042579101 >> 172.21.15.83:0/2042579101 conn(0x7eff7c0d0be0 msgr2=0x7eff7c0d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:44.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 -- 172.21.15.83:0/2042579101 shutdown_connections 2023-11-26T07:12:44.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.118+0000 7eff828af700 1 -- 172.21.15.83:0/2042579101 wait complete. 2023-11-26T07:12:44.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:12:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.260+0000 7fd264867700 1 Processor -- start 2023-11-26T07:12:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.260+0000 7fd264867700 1 -- start start 2023-11-26T07:12:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd264867700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd260144600 0x7fd2600b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd264867700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26006c9b0 con 0x7fd26006f480 2023-11-26T07:12:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd264867700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26006b200 con 0x7fd26006e830 2023-11-26T07:12:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd264867700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26006aff0 con 0x7fd260144600 2023-11-26T07:12:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd25dd9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd26006f480 0x7fd26006c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56794/0 (socket says 172.21.15.83:56794) 2023-11-26T07:12:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd25dd9b700 1 -- 172.21.15.83:0/3349534425 learned_addr learned my addr 172.21.15.83:0/3349534425 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd25e59c700 1 --2- 172.21.15.83:0/3349534425 >> v2:172.21.15.203:3300/0 conn(0x7fd260144600 0x7fd2600b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:44.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034135042 0 0) 0x7fd26006c9b0 con 0x7fd26006f480 2023-11-26T07:12:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2480035b0 con 0x7fd26006f480 2023-11-26T07:12:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3635987457 0 0) 0x7fd2480035b0 con 0x7fd26006f480 2023-11-26T07:12:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 >> v2:172.21.15.203:3300/0 conn(0x7fd260144600 msgr2=0x7fd2600b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd25d59a700 1 --2- 172.21.15.83:0/3349534425 >> v2:172.21.15.203:3300/0 conn(0x7fd260144600 0x7fd2600b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 >> v1:172.21.15.203:6789/0 conn(0x7fd26006e830 legacy=0x7fd2600cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26006b4e0 con 0x7fd26006f480 2023-11-26T07:12:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.261+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fd254001670 con 0x7fd26006f480 2023-11-26T07:12:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 >> v1:172.21.15.83:6789/0 conn(0x7fd26006f480 legacy=0x7fd26006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:44.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25d59a700 1 --2- 172.21.15.83:0/3349534425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 0x7fd248003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25d59a700 1 --2- 172.21.15.83:0/3349534425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd248004850 0x7fd248004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25d59a700 1 --2- 172.21.15.83:0/3349534425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd248005210 0x7fd24800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26006aff0 con 0x7fd248003a00 2023-11-26T07:12:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24800c1b0 con 0x7fd248004850 2023-11-26T07:12:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24800c370 con 0x7fd248005210 2023-11-26T07:12:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25e59c700 1 --2- 172.21.15.83:0/3349534425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 0x7fd248003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:44.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25dd9b700 1 --2- 172.21.15.83:0/3349534425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd248004850 0x7fd248004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25ed9d700 1 --2- 172.21.15.83:0/3349534425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd248005210 0x7fd24800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25e59c700 1 -- 172.21.15.83:0/3349534425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd248005210 msgr2=0x7fd24800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25e59c700 1 --2- 172.21.15.83:0/3349534425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd248005210 0x7fd24800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25e59c700 1 -- 172.21.15.83:0/3349534425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd248004850 msgr2=0x7fd248004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25e59c700 1 --2- 172.21.15.83:0/3349534425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd248004850 0x7fd248004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25e59c700 1 -- 172.21.15.83:0/3349534425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2480035b0 con 0x7fd248003a00 2023-11-26T07:12:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25ed9d700 1 -- 172.21.15.83:0/3349534425 reap_dead start 2023-11-26T07:12:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25e59c700 1 --2- 172.21.15.83:0/3349534425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 0x7fd248003e20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd250009b60 tx=0x7fd248005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459655570b217e20 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25000e6c0 con 0x7fd248003a00 2023-11-26T07:12:44.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25000e820 con 0x7fd248003a00 2023-11-26T07:12:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.262+0000 7fd25d59a700 1 -- 172.21.15.83:0/3349534425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25000ebb0 con 0x7fd248003a00 2023-11-26T07:12:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.263+0000 7fd264867700 1 -- 172.21.15.83:0/3349534425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 msgr2=0x7fd248003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.263+0000 7fd264867700 1 --2- 172.21.15.83:0/3349534425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 0x7fd248003e20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd250009b60 tx=0x7fd248005880 comp rx=0 tx=0).stop 2023-11-26T07:12:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.263+0000 7fd264867700 1 -- 172.21.15.83:0/3349534425 shutdown_connections 2023-11-26T07:12:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.263+0000 7fd264867700 1 --2- 172.21.15.83:0/3349534425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 0x7fd248003e20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.263+0000 7fd264867700 1 -- 172.21.15.83:0/3349534425 >> 172.21.15.83:0/3349534425 conn(0x7fd2600d0b40 msgr2=0x7fd2600d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.263+0000 7fd264867700 1 -- 172.21.15.83:0/3349534425 shutdown_connections 2023-11-26T07:12:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.263+0000 7fd264867700 1 -- 172.21.15.83:0/3349534425 wait complete. 2023-11-26T07:12:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.264+0000 7fd264867700 1 Processor -- start 2023-11-26T07:12:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.264+0000 7fd264867700 1 -- start start 2023-11-26T07:12:44.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd264867700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 0x7fd26015bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:44.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd264867700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd26015a610 0x7fd26015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:44.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd264867700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd26015a0c0 0x7fd2600ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:44.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd264867700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26015bb00 con 0x7fd248003a00 2023-11-26T07:12:44.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd264867700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26015ae10 con 0x7fd26015a0c0 2023-11-26T07:12:44.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd264867700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26015dcb0 con 0x7fd26015a610 2023-11-26T07:12:44.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd25e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 0x7fd26015bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:44.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd25e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 0x7fd26015bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36764/0 (socket says 172.21.15.83:36764) 2023-11-26T07:12:44.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd25e59c700 1 -- 172.21.15.83:0/1344125062 learned_addr learned my addr 172.21.15.83:0/1344125062 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:44.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd25e59c700 1 -- 172.21.15.83:0/1344125062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd26015a610 msgr2=0x7fd26015c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:44.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd25e59c700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd26015a610 0x7fd26015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:44.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd25e59c700 1 -- 172.21.15.83:0/1344125062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd26015a0c0 msgr2=0x7fd2600ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd25e59c700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd26015a0c0 0x7fd2600ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.265+0000 7fd25e59c700 1 -- 172.21.15.83:0/1344125062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26015b4f0 con 0x7fd248003a00 2023-11-26T07:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.266+0000 7fd25e59c700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 0x7fd26015bcc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd24800c180 tx=0x7fd2500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.266+0000 7fd24effd700 1 -- 172.21.15.83:0/1344125062 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25000e6c0 con 0x7fd248003a00 2023-11-26T07:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.266+0000 7fd24effd700 1 -- 172.21.15.83:0/1344125062 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250003ad0 con 0x7fd248003a00 2023-11-26T07:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.266+0000 7fd264867700 1 -- 172.21.15.83:0/1344125062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd260159e60 con 0x7fd248003a00 2023-11-26T07:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.266+0000 7fd24effd700 1 -- 172.21.15.83:0/1344125062 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500045d0 con 0x7fd248003a00 2023-11-26T07:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.266+0000 7fd264867700 1 -- 172.21.15.83:0/1344125062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26015b790 con 0x7fd248003a00 2023-11-26T07:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.267+0000 7fd264867700 1 -- 172.21.15.83:0/1344125062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2600ae9b0 con 0x7fd248003a00 2023-11-26T07:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.267+0000 7fd24effd700 1 -- 172.21.15.83:0/1344125062 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fd250013470 con 0x7fd248003a00 2023-11-26T07:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.267+0000 7fd24effd700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd24006b080 0x7fd24006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.268+0000 7fd24effd700 1 -- 172.21.15.83:0/1344125062 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd25008b0b0 con 0x7fd248003a00 2023-11-26T07:12:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.268+0000 7fd25dd9b700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd24006b080 0x7fd24006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.268+0000 7fd25dd9b700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd24006b080 0x7fd24006d540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd254001c90 tx=0x7fd254001b20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.272+0000 7fd24effd700 1 -- 172.21.15.83:0/1344125062 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd250055f10 con 0x7fd248003a00 2023-11-26T07:12:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:44.464+0000 7fd264867700 1 -- 172.21.15.83:0/1344125062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd2600cd3c0 con 0x7fd248003a00 2023-11-26T07:12:45.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.120+0000 7fd24effd700 1 -- 172.21.15.83:0/1344125062 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd250059530 con 0x7fd248003a00 2023-11-26T07:12:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.123+0000 7fd264867700 1 -- 172.21.15.83:0/1344125062 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd24006b080 msgr2=0x7fd24006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.123+0000 7fd264867700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd24006b080 0x7fd24006d540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd254001c90 tx=0x7fd254001b20 comp rx=0 tx=0).stop 2023-11-26T07:12:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.123+0000 7fd264867700 1 -- 172.21.15.83:0/1344125062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 msgr2=0x7fd26015bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.123+0000 7fd264867700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 0x7fd26015bcc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd24800c180 tx=0x7fd2500047c0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.123+0000 7fd264867700 1 -- 172.21.15.83:0/1344125062 shutdown_connections 2023-11-26T07:12:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.123+0000 7fd264867700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd26015a0c0 0x7fd2600ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.123+0000 7fd264867700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd248003a00 0x7fd26015bcc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.123+0000 7fd264867700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd24006b080 0x7fd24006d540 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.123+0000 7fd264867700 1 --2- 172.21.15.83:0/1344125062 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd26015a610 0x7fd26015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.123+0000 7fd264867700 1 -- 172.21.15.83:0/1344125062 >> 172.21.15.83:0/1344125062 conn(0x7fd2600d0b40 msgr2=0x7fd2601438d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.123+0000 7fd264867700 1 -- 172.21.15.83:0/1344125062 shutdown_connections 2023-11-26T07:12:45.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.124+0000 7fd264867700 1 -- 172.21.15.83:0/1344125062 wait complete. 2023-11-26T07:12:45.128 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:12:45.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:12:45.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.264+0000 7f97e4851700 1 Processor -- start 2023-11-26T07:12:45.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.264+0000 7f97e4851700 1 -- start start 2023-11-26T07:12:45.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97e4851700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97e0144600 0x7f97e00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:45.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97e4851700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97e006f480 0x7f97e006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:45.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97e4851700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f97e006e830 0x7f97e006eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:45.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97e4851700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e006aff0 con 0x7f97e006e830 2023-11-26T07:12:45.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97e4851700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e006f1e0 con 0x7f97e006f480 2023-11-26T07:12:45.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97e4851700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e006b200 con 0x7f97e0144600 2023-11-26T07:12:45.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97ded9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f97e006e830 0x7f97e006eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:45.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97ded9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f97e006e830 0x7f97e006eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36776/0 (socket says 172.21.15.83:36776) 2023-11-26T07:12:45.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97ded9d700 1 -- 172.21.15.83:0/3438155892 learned_addr learned my addr 172.21.15.83:0/3438155892 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:45.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97ddd9b700 1 --2- 172.21.15.83:0/3438155892 >> v2:172.21.15.203:3300/0 conn(0x7f97e006f480 0x7f97e006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:45.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97de59c700 1 --2- 172.21.15.83:0/3438155892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97e0144600 0x7f97e00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:45.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97ded9d700 1 -- 172.21.15.83:0/3438155892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97e0144600 msgr2=0x7f97e00b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:45.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97ded9d700 1 --2- 172.21.15.83:0/3438155892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97e0144600 0x7f97e00b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97ded9d700 1 -- 172.21.15.83:0/3438155892 >> v2:172.21.15.203:3300/0 conn(0x7f97e006f480 msgr2=0x7f97e006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:45.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97ded9d700 1 --2- 172.21.15.83:0/3438155892 >> v2:172.21.15.203:3300/0 conn(0x7f97e006f480 0x7f97e006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97ded9d700 1 -- 172.21.15.83:0/3438155892 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e006b4e0 con 0x7f97e006e830 2023-11-26T07:12:45.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97de59c700 1 --2- 172.21.15.83:0/3438155892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97e0144600 0x7f97e00b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.265+0000 7f97ddd9b700 1 --2- 172.21.15.83:0/3438155892 >> v2:172.21.15.203:3300/0 conn(0x7f97e006f480 0x7f97e006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97ded9d700 1 --2- 172.21.15.83:0/3438155892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97e006e830 0x7f97e006eca0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f97e00718b0 tx=0x7f97d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c149bcd09dd256 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97dd59a700 1 -- 172.21.15.83:0/3438155892 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d800e070 con 0x7f97e006e830 2023-11-26T07:12:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97dd59a700 1 -- 172.21.15.83:0/3438155892 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d800b9c0 con 0x7f97e006e830 2023-11-26T07:12:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97dd59a700 1 -- 172.21.15.83:0/3438155892 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d800bd70 con 0x7f97e006e830 2023-11-26T07:12:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97e4851700 1 -- 172.21.15.83:0/3438155892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97e006e830 msgr2=0x7f97e006eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:45.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97e4851700 1 --2- 172.21.15.83:0/3438155892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97e006e830 0x7f97e006eca0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f97e00718b0 tx=0x7f97d800b330 comp rx=0 tx=0).stop 2023-11-26T07:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97e4851700 1 -- 172.21.15.83:0/3438155892 shutdown_connections 2023-11-26T07:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97e4851700 1 --2- 172.21.15.83:0/3438155892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97e006e830 0x7f97e006eca0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97e4851700 1 --2- 172.21.15.83:0/3438155892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97e0144600 0x7f97e00b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97e4851700 1 --2- 172.21.15.83:0/3438155892 >> v2:172.21.15.203:3300/0 conn(0x7f97e006f480 0x7f97e006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97e4851700 1 -- 172.21.15.83:0/3438155892 >> 172.21.15.83:0/3438155892 conn(0x7f97e00d0b40 msgr2=0x7f97e00d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97e4851700 1 -- 172.21.15.83:0/3438155892 shutdown_connections 2023-11-26T07:12:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.266+0000 7f97e4851700 1 -- 172.21.15.83:0/3438155892 wait complete. 2023-11-26T07:12:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.268+0000 7f97e4851700 1 Processor -- start 2023-11-26T07:12:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.268+0000 7f97e4851700 1 -- start start 2023-11-26T07:12:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.268+0000 7f97e4851700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97e006e830 0x7f97e015d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.268+0000 7f97e4851700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97e006f480 0x7f97e00aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.268+0000 7f97e4851700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97e0144600 0x7f97e01f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.268+0000 7f97e4851700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e015d780 con 0x7f97e006e830 2023-11-26T07:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.268+0000 7f97e4851700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00add80 con 0x7f97e006f480 2023-11-26T07:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.268+0000 7f97e4851700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00ae060 con 0x7f97e0144600 2023-11-26T07:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97de59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97e006e830 0x7f97e015d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97de59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97e006e830 0x7f97e015d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36778/0 (socket says 172.21.15.83:36778) 2023-11-26T07:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97de59c700 1 -- 172.21.15.83:0/3096084539 learned_addr learned my addr 172.21.15.83:0/3096084539 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97ddd9b700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97e006f480 0x7f97e00aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97ded9d700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97e0144600 0x7f97e01f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97de59c700 1 -- 172.21.15.83:0/3096084539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97e0144600 msgr2=0x7f97e01f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97de59c700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97e0144600 0x7f97e01f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97de59c700 1 -- 172.21.15.83:0/3096084539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97e006f480 msgr2=0x7f97e00aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97de59c700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97e006f480 0x7f97e00aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97de59c700 1 -- 172.21.15.83:0/3096084539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d8009b30 con 0x7f97e006e830 2023-11-26T07:12:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97ddd9b700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97e006f480 0x7f97e00aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97de59c700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97e006e830 0x7f97e015d240 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f97d400c7f0 tx=0x7f97d400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97ceffd700 1 -- 172.21.15.83:0/3096084539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4011800 con 0x7f97e006e830 2023-11-26T07:12:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97ceffd700 1 -- 172.21.15.83:0/3096084539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d4011960 con 0x7f97e006e830 2023-11-26T07:12:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97ceffd700 1 -- 172.21.15.83:0/3096084539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4011d10 con 0x7f97e006e830 2023-11-26T07:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97e4851700 1 -- 172.21.15.83:0/3096084539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e00ae340 con 0x7f97e006e830 2023-11-26T07:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.269+0000 7f97e4851700 1 -- 172.21.15.83:0/3096084539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e00ae620 con 0x7f97e006e830 2023-11-26T07:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.270+0000 7f97e4851700 1 -- 172.21.15.83:0/3096084539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e0068a00 con 0x7f97e006e830 2023-11-26T07:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.270+0000 7f97ceffd700 1 -- 172.21.15.83:0/3096084539 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f97d4018360 con 0x7f97e006e830 2023-11-26T07:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.271+0000 7f97ceffd700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f97c006b080 0x7f97c006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.271+0000 7f97ceffd700 1 -- 172.21.15.83:0/3096084539 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f97d40898c0 con 0x7f97e006e830 2023-11-26T07:12:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.271+0000 7f97ddd9b700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f97c006b080 0x7f97c006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.271+0000 7f97ddd9b700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f97c006b080 0x7f97c006d540 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f97e01f7fa0 tx=0x7f97d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.275+0000 7f97ceffd700 1 -- 172.21.15.83:0/3096084539 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97d4054740 con 0x7f97e006e830 2023-11-26T07:12:45.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:45.467+0000 7f97e4851700 1 -- 172.21.15.83:0/3096084539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f97e01f3c70 con 0x7f97e006e830 2023-11-26T07:12:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.122+0000 7f97ceffd700 1 -- 172.21.15.83:0/3096084539 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7f97e01f3c70 con 0x7f97e006e830 2023-11-26T07:12:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 -- 172.21.15.83:0/3096084539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f97c006b080 msgr2=0x7f97c006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f97c006b080 0x7f97c006d540 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f97e01f7fa0 tx=0x7f97d0008040 comp rx=0 tx=0).stop 2023-11-26T07:12:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 -- 172.21.15.83:0/3096084539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97e006e830 msgr2=0x7f97e015d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97e006e830 0x7f97e015d240 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f97d400c7f0 tx=0x7f97d400cbb0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 -- 172.21.15.83:0/3096084539 shutdown_connections 2023-11-26T07:12:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f97c006b080 0x7f97c006d540 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97e0144600 0x7f97e01f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97e006e830 0x7f97e015d240 secure :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f97d400c7f0 tx=0x7f97d400cbb0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 --2- 172.21.15.83:0/3096084539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97e006f480 0x7f97e00aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 -- 172.21.15.83:0/3096084539 >> 172.21.15.83:0/3096084539 conn(0x7f97e00d0b40 msgr2=0x7f97e00d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:46.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 -- 172.21.15.83:0/3096084539 shutdown_connections 2023-11-26T07:12:46.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.126+0000 7f97e4851700 1 -- 172.21.15.83:0/3096084539 wait complete. 2023-11-26T07:12:46.130 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2023-11-26T07:12:46.144 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:12:46.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.269+0000 7f8c9ee7c700 1 Processor -- start 2023-11-26T07:12:46.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.269+0000 7f8c9ee7c700 1 -- start start 2023-11-26T07:12:46.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c9ee7c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8c9815e040 0x7f8c9815e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c9ee7c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c980b2b00 con 0x7f8c98151660 2023-11-26T07:12:46.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c9ee7c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9806e3a0 con 0x7f8c9806dec0 2023-11-26T07:12:46.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c9ee7c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9815ea30 con 0x7f8c9815e040 2023-11-26T07:12:46.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c9cc18700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8c98151660 0x7f8c980cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56826/0 (socket says 172.21.15.83:56826) 2023-11-26T07:12:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c9cc18700 1 -- 172.21.15.83:0/4179150245 learned_addr learned my addr 172.21.15.83:0/4179150245 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c9d419700 1 --2- 172.21.15.83:0/4179150245 >> v2:172.21.15.203:3300/0 conn(0x7f8c9815e040 0x7f8c9815e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 505835666 0 0) 0x7f8c980b2b00 con 0x7f8c98151660 2023-11-26T07:12:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c880046c0 con 0x7f8c98151660 2023-11-26T07:12:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3484335482 0 0) 0x7f8c880046c0 con 0x7f8c98151660 2023-11-26T07:12:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 >> v2:172.21.15.203:3300/0 conn(0x7f8c9815e040 msgr2=0x7f8c9815e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 --2- 172.21.15.83:0/4179150245 >> v2:172.21.15.203:3300/0 conn(0x7f8c9815e040 0x7f8c9815e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 >> v1:172.21.15.203:6789/0 conn(0x7f8c9806dec0 legacy=0x7f8c98070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c980b2da0 con 0x7f8c98151660 2023-11-26T07:12:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f8c8c001670 con 0x7f8c98151660 2023-11-26T07:12:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 >> v1:172.21.15.83:6789/0 conn(0x7f8c98151660 legacy=0x7f8c980cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 --2- 172.21.15.83:0/4179150245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c88004b10 0x7f8c88004f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 --2- 172.21.15.83:0/4179150245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 0x7f8c88005de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 --2- 172.21.15.83:0/4179150245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c88006320 0x7f8c8800cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9815ea30 con 0x7f8c88005960 2023-11-26T07:12:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.270+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8800d1c0 con 0x7f8c88006320 2023-11-26T07:12:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88003550 con 0x7f8c88004b10 2023-11-26T07:12:46.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c97fff700 1 --2- 172.21.15.83:0/4179150245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 0x7f8c88005de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c9d419700 1 --2- 172.21.15.83:0/4179150245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c88006320 0x7f8c8800cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c9cc18700 1 --2- 172.21.15.83:0/4179150245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c88004b10 0x7f8c88004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c97fff700 1 -- 172.21.15.83:0/4179150245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c88004b10 msgr2=0x7f8c88004f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:46.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c97fff700 1 --2- 172.21.15.83:0/4179150245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c88004b10 0x7f8c88004f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c97fff700 1 -- 172.21.15.83:0/4179150245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c88006320 msgr2=0x7f8c8800cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:46.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c97fff700 1 --2- 172.21.15.83:0/4179150245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c88006320 0x7f8c8800cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c97fff700 1 -- 172.21.15.83:0/4179150245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c880046c0 con 0x7f8c88005960 2023-11-26T07:12:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c9d419700 1 -- 172.21.15.83:0/4179150245 reap_dead start 2023-11-26T07:12:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c97fff700 1 --2- 172.21.15.83:0/4179150245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 0x7f8c88005de0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8c980cc5e0 tx=0x7f8c88006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a4fb741a60751d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840052d0 con 0x7f8c88005960 2023-11-26T07:12:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c84005450 con 0x7f8c88005960 2023-11-26T07:12:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.271+0000 7f8c977fe700 1 -- 172.21.15.83:0/4179150245 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840038c0 con 0x7f8c88005960 2023-11-26T07:12:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.272+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/4179150245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 msgr2=0x7f8c88005de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.272+0000 7f8c9ee7c700 1 --2- 172.21.15.83:0/4179150245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 0x7f8c88005de0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8c980cc5e0 tx=0x7f8c88006990 comp rx=0 tx=0).stop 2023-11-26T07:12:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.272+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/4179150245 shutdown_connections 2023-11-26T07:12:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.272+0000 7f8c9ee7c700 1 --2- 172.21.15.83:0/4179150245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 0x7f8c88005de0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.272+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/4179150245 >> 172.21.15.83:0/4179150245 conn(0x7f8c9806b410 msgr2=0x7f8c9806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.272+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/4179150245 shutdown_connections 2023-11-26T07:12:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.272+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/4179150245 wait complete. 2023-11-26T07:12:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.273+0000 7f8c9ee7c700 1 Processor -- start 2023-11-26T07:12:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.273+0000 7f8c9ee7c700 1 -- start start 2023-11-26T07:12:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9ee7c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 0x7f8c981585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9ee7c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c981f4a20 0x7f8c98143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9ee7c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c981e46f0 0x7f8c981e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9ee7c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98143ac0 con 0x7f8c88005960 2023-11-26T07:12:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9ee7c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c981e50b0 con 0x7f8c981e46f0 2023-11-26T07:12:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9ee7c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c980cd180 con 0x7f8c981f4a20 2023-11-26T07:12:46.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9cc18700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 0x7f8c981585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9cc18700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 0x7f8c981585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36798/0 (socket says 172.21.15.83:36798) 2023-11-26T07:12:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9cc18700 1 -- 172.21.15.83:0/708314147 learned_addr learned my addr 172.21.15.83:0/708314147 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9cc18700 1 -- 172.21.15.83:0/708314147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c981f4a20 msgr2=0x7f8c98143580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9cc18700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c981f4a20 0x7f8c98143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9cc18700 1 -- 172.21.15.83:0/708314147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c981e46f0 msgr2=0x7f8c981e4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9cc18700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c981e46f0 0x7f8c981e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9cc18700 1 -- 172.21.15.83:0/708314147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c981f4ea0 con 0x7f8c88005960 2023-11-26T07:12:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.274+0000 7f8c9cc18700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 0x7f8c981585e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e7b0 tx=0x7f8c8c00eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.275+0000 7f8c957fa700 1 -- 172.21.15.83:0/708314147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c012590 con 0x7f8c88005960 2023-11-26T07:12:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.275+0000 7f8c957fa700 1 -- 172.21.15.83:0/708314147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c0126f0 con 0x7f8c88005960 2023-11-26T07:12:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.275+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/708314147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c98158cc0 con 0x7f8c88005960 2023-11-26T07:12:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.275+0000 7f8c957fa700 1 -- 172.21.15.83:0/708314147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c012aa0 con 0x7f8c88005960 2023-11-26T07:12:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.275+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/708314147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c980ccb30 con 0x7f8c88005960 2023-11-26T07:12:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.276+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/708314147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c980cd460 con 0x7f8c88005960 2023-11-26T07:12:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.276+0000 7f8c957fa700 1 -- 172.21.15.83:0/708314147 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f8c8c012c00 con 0x7f8c88005960 2023-11-26T07:12:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.276+0000 7f8c957fa700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8c7c06b080 0x7f8c7c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.276+0000 7f8c957fa700 1 -- 172.21.15.83:0/708314147 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8c8c0893a0 con 0x7f8c88005960 2023-11-26T07:12:46.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.276+0000 7f8c97fff700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8c7c06b080 0x7f8c7c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.277+0000 7f8c97fff700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8c7c06b080 0x7f8c7c06d540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c980af0b0 tx=0x7f8c84011000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:46.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.281+0000 7f8c957fa700 1 -- 172.21.15.83:0/708314147 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c8c0544d0 con 0x7f8c88005960 2023-11-26T07:12:46.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.476+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/708314147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8c98068480 con 0x7f8c88005960 2023-11-26T07:12:46.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.480+0000 7f8c957fa700 1 -- 172.21.15.83:0/708314147 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8c8c057af0 con 0x7f8c88005960 2023-11-26T07:12:46.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/708314147 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8c7c06b080 msgr2=0x7f8c7c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8c7c06b080 0x7f8c7c06d540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c980af0b0 tx=0x7f8c84011000 comp rx=0 tx=0).stop 2023-11-26T07:12:46.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/708314147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 msgr2=0x7f8c981585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 0x7f8c981585e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e7b0 tx=0x7f8c8c00eb70 comp rx=0 tx=0).stop 2023-11-26T07:12:46.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/708314147 shutdown_connections 2023-11-26T07:12:46.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c981e46f0 0x7f8c981e4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88005960 0x7f8c981585e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8c7c06b080 0x7f8c7c06d540 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 --2- 172.21.15.83:0/708314147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c981f4a20 0x7f8c98143580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/708314147 >> 172.21.15.83:0/708314147 conn(0x7f8c9806b410 msgr2=0x7f8c9806b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/708314147 shutdown_connections 2023-11-26T07:12:46.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.482+0000 7f8c9ee7c700 1 -- 172.21.15.83:0/708314147 wait complete. 2023-11-26T07:12:46.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:12:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.616+0000 7f9f90c6b700 1 Processor -- start 2023-11-26T07:12:46.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.616+0000 7f9f90c6b700 1 -- start start 2023-11-26T07:12:46.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f90c6b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f8c06c710 0x7f9f8c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f90c6b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c0af4d0 0x7f9f8c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f90c6b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c0ae430 con 0x7f9f8c15e080 2023-11-26T07:12:46.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f90c6b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f8c0af0b0 con 0x7f9f8c06c710 2023-11-26T07:12:46.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f90c6b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0b05a0 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f89d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f8c06c710 0x7f9f8c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f89d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f8c06c710 0x7f9f8c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36808/0 (socket says 172.21.15.83:36808) 2023-11-26T07:12:46.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f89d9b700 1 -- 172.21.15.83:0/3777342949 learned_addr learned my addr 172.21.15.83:0/3777342949 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f8ad9d700 1 --2- 172.21.15.83:0/3777342949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c0af4d0 0x7f9f8c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f89d9b700 1 -- 172.21.15.83:0/3777342949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c0af4d0 msgr2=0x7f9f8c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f89d9b700 1 --2- 172.21.15.83:0/3777342949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c0af4d0 0x7f9f8c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f89d9b700 1 -- 172.21.15.83:0/3777342949 >> v1:172.21.15.203:6789/0 conn(0x7f9f8c15e080 legacy=0x7f9f8c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.617+0000 7f9f89d9b700 1 -- 172.21.15.83:0/3777342949 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0ae710 con 0x7f9f8c06c710 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f89d9b700 1 --2- 172.21.15.83:0/3777342949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f8c06c710 0x7f9f8c0b5050 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0098f0 tx=0x7f9f7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362c6fa065f820c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f8959a700 1 -- 172.21.15.83:0/3777342949 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c004230 con 0x7f9f8c06c710 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f8959a700 1 -- 172.21.15.83:0/3777342949 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c004390 con 0x7f9f8c06c710 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f8959a700 1 -- 172.21.15.83:0/3777342949 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c0036a0 con 0x7f9f8c06c710 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f90c6b700 1 -- 172.21.15.83:0/3777342949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f8c06c710 msgr2=0x7f9f8c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f90c6b700 1 --2- 172.21.15.83:0/3777342949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f8c06c710 0x7f9f8c0b5050 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0098f0 tx=0x7f9f7c009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:46.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f90c6b700 1 -- 172.21.15.83:0/3777342949 shutdown_connections 2023-11-26T07:12:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f90c6b700 1 --2- 172.21.15.83:0/3777342949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c0af4d0 0x7f9f8c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f90c6b700 1 --2- 172.21.15.83:0/3777342949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f8c06c710 0x7f9f8c0b5050 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f90c6b700 1 -- 172.21.15.83:0/3777342949 >> 172.21.15.83:0/3777342949 conn(0x7f9f8c071b90 msgr2=0x7f9f8c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f90c6b700 1 -- 172.21.15.83:0/3777342949 shutdown_connections 2023-11-26T07:12:46.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.618+0000 7f9f90c6b700 1 -- 172.21.15.83:0/3777342949 wait complete. 2023-11-26T07:12:46.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.620+0000 7f9f90c6b700 1 Processor -- start 2023-11-26T07:12:46.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.620+0000 7f9f90c6b700 1 -- start start 2023-11-26T07:12:46.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.620+0000 7f9f90c6b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c06c710 0x7f9f8c0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f90c6b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f8c0af4d0 0x7f9f8c0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f90c6b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f8c15e080 0x7f9f8c15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f90c6b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c1585f0 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f90c6b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0ce570 con 0x7f9f8c15e080 2023-11-26T07:12:46.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f90c6b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c0ce7f0 con 0x7f9f8c06c710 2023-11-26T07:12:46.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f89d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f8c0af4d0 0x7f9f8c0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f89d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f8c0af4d0 0x7f9f8c0ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36814/0 (socket says 172.21.15.83:36814) 2023-11-26T07:12:46.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f89d9b700 1 -- 172.21.15.83:0/1796642384 learned_addr learned my addr 172.21.15.83:0/1796642384 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:46.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f8a59c700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c06c710 0x7f9f8c0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f8ad9d700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f8c15e080 0x7f9f8c15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f89d9b700 1 -- 172.21.15.83:0/1796642384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c06c710 msgr2=0x7f9f8c0d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f89d9b700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c06c710 0x7f9f8c0d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f89d9b700 1 -- 172.21.15.83:0/1796642384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f8c15e080 msgr2=0x7f9f8c15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f89d9b700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f8c15e080 0x7f9f8c15a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f89d9b700 1 -- 172.21.15.83:0/1796642384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c009550 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f89d9b700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f8c0af4d0 0x7f9f8c0ce030 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009130 tx=0x7f9f7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f7affd700 1 -- 172.21.15.83:0/1796642384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c003aa0 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f7affd700 1 -- 172.21.15.83:0/1796642384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c01f070 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f90c6b700 1 -- 172.21.15.83:0/1796642384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8c0cead0 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f7affd700 1 -- 172.21.15.83:0/1796642384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c003e40 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.621+0000 7f9f90c6b700 1 -- 172.21.15.83:0/1796642384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c158800 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.622+0000 7f9f90c6b700 1 -- 172.21.15.83:0/1796642384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8c0753f0 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.623+0000 7f9f7affd700 1 -- 172.21.15.83:0/1796642384 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f9f7c021030 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.623+0000 7f9f7affd700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9f6c06b080 0x7f9f6c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:46.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.623+0000 7f9f7affd700 1 -- 172.21.15.83:0/1796642384 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9f7c08aa40 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.623+0000 7f9f8a59c700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9f6c06b080 0x7f9f6c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:46.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.623+0000 7f9f8a59c700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9f6c06b080 0x7f9f6c06d540 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009960 tx=0x7f9f800090e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:46.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.627+0000 7f9f7affd700 1 -- 172.21.15.83:0/1796642384 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f7c08ce60 con 0x7f9f8c0af4d0 2023-11-26T07:12:46.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:46.902+0000 7f9f90c6b700 1 -- 172.21.15.83:0/1796642384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f9f8c0753f0 con 0x7f9f8c0af4d0 2023-11-26T07:12:47.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.128+0000 7f9f7affd700 1 -- 172.21.15.83:0/1796642384 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v99) v1 ==== 158+0+0 (secure 0 0 0) 0x7f9f7c055c00 con 0x7f9f8c0af4d0 2023-11-26T07:12:47.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 -- 172.21.15.83:0/1796642384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9f6c06b080 msgr2=0x7f9f6c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9f6c06b080 0x7f9f6c06d540 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009960 tx=0x7f9f800090e0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 -- 172.21.15.83:0/1796642384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f8c0af4d0 msgr2=0x7f9f8c0ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f8c0af4d0 0x7f9f8c0ce030 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009130 tx=0x7f9f7c004750 comp rx=0 tx=0).stop 2023-11-26T07:12:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 -- 172.21.15.83:0/1796642384 shutdown_connections 2023-11-26T07:12:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f8c15e080 0x7f9f8c15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9f6c06b080 0x7f9f6c06d540 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f8c06c710 0x7f9f8c0d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 --2- 172.21.15.83:0/1796642384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f8c0af4d0 0x7f9f8c0ce030 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 -- 172.21.15.83:0/1796642384 >> 172.21.15.83:0/1796642384 conn(0x7f9f8c071b90 msgr2=0x7f9f8c06b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 -- 172.21.15.83:0/1796642384 shutdown_connections 2023-11-26T07:12:47.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.131+0000 7f9f90c6b700 1 -- 172.21.15.83:0/1796642384 wait complete. 2023-11-26T07:12:47.146 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2023-11-26T07:12:47.147 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:12:47.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:47.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.265+0000 7f4c3efe5700 1 Processor -- start 2023-11-26T07:12:47.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.265+0000 7f4c3efe5700 1 -- start start 2023-11-26T07:12:47.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3efe5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c3815ddd0 0x7f4c380b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3efe5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4c380d2240 0x7f4c380cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3efe5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c380b1d40 con 0x7f4c3815e1b0 2023-11-26T07:12:47.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3efe5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c380b28c0 con 0x7f4c3815ddd0 2023-11-26T07:12:47.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3efe5700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c3806c000 con 0x7f4c380d2240 2023-11-26T07:12:47.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3cd81700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c3815ddd0 0x7f4c380b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3cd81700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4c3815ddd0 0x7f4c380b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36820/0 (socket says 172.21.15.83:36820) 2023-11-26T07:12:47.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3cd81700 1 -- 172.21.15.83:0/1699077826 learned_addr learned my addr 172.21.15.83:0/1699077826 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:47.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3d582700 1 --2- 172.21.15.83:0/1699077826 >> v2:172.21.15.203:3300/0 conn(0x7f4c380d2240 0x7f4c380cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3cd81700 1 -- 172.21.15.83:0/1699077826 >> v2:172.21.15.203:3300/0 conn(0x7f4c380d2240 msgr2=0x7f4c380cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3cd81700 1 --2- 172.21.15.83:0/1699077826 >> v2:172.21.15.203:3300/0 conn(0x7f4c380d2240 0x7f4c380cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3cd81700 1 -- 172.21.15.83:0/1699077826 >> v1:172.21.15.83:6789/0 conn(0x7f4c3815e1b0 legacy=0x7f4c380d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3cd81700 1 -- 172.21.15.83:0/1699077826 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c380b1f90 con 0x7f4c3815ddd0 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c3cd81700 1 --2- 172.21.15.83:0/1699077826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3815ddd0 0x7f4c380b1680 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0098f0 tx=0x7f4c2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1a73c814e632d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c377fe700 1 -- 172.21.15.83:0/1699077826 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00b860 con 0x7f4c3815ddd0 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c377fe700 1 -- 172.21.15.83:0/1699077826 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c00b9c0 con 0x7f4c3815ddd0 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.266+0000 7f4c377fe700 1 -- 172.21.15.83:0/1699077826 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00bd50 con 0x7f4c3815ddd0 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.267+0000 7f4c3efe5700 1 -- 172.21.15.83:0/1699077826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3815ddd0 msgr2=0x7f4c380b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.267+0000 7f4c3efe5700 1 --2- 172.21.15.83:0/1699077826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3815ddd0 0x7f4c380b1680 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0098f0 tx=0x7f4c2c009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.267+0000 7f4c3efe5700 1 -- 172.21.15.83:0/1699077826 shutdown_connections 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.267+0000 7f4c3efe5700 1 --2- 172.21.15.83:0/1699077826 >> v2:172.21.15.203:3300/0 conn(0x7f4c380d2240 0x7f4c380cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.267+0000 7f4c3efe5700 1 --2- 172.21.15.83:0/1699077826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3815ddd0 0x7f4c380b1680 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.267+0000 7f4c3efe5700 1 -- 172.21.15.83:0/1699077826 >> 172.21.15.83:0/1699077826 conn(0x7f4c380731a0 msgr2=0x7f4c3809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:47.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.267+0000 7f4c3efe5700 1 -- 172.21.15.83:0/1699077826 shutdown_connections 2023-11-26T07:12:47.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.267+0000 7f4c3efe5700 1 -- 172.21.15.83:0/1699077826 wait complete. 2023-11-26T07:12:47.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.268+0000 7f4c3efe5700 1 Processor -- start 2023-11-26T07:12:47.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.268+0000 7f4c3efe5700 1 -- start start 2023-11-26T07:12:47.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c3efe5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c380d2240 0x7f4c380d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c3efe5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3815ddd0 0x7f4c381586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c3efe5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c3815e1b0 0x7f4c380b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c3efe5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c380b4ec0 con 0x7f4c3815ddd0 2023-11-26T07:12:47.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c3efe5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c38158c00 con 0x7f4c3815e1b0 2023-11-26T07:12:47.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c3efe5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3815f270 con 0x7f4c380d2240 2023-11-26T07:12:47.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c37fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3815ddd0 0x7f4c381586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c37fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3815ddd0 0x7f4c381586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36828/0 (socket says 172.21.15.83:36828) 2023-11-26T07:12:47.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c37fff700 1 -- 172.21.15.83:0/4042317226 learned_addr learned my addr 172.21.15.83:0/4042317226 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:47.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c3cd81700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c380d2240 0x7f4c380d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c3d582700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c3815e1b0 0x7f4c380b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c37fff700 1 -- 172.21.15.83:0/4042317226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c380d2240 msgr2=0x7f4c380d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c37fff700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c380d2240 0x7f4c380d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c37fff700 1 -- 172.21.15.83:0/4042317226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c3815e1b0 msgr2=0x7f4c380b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c37fff700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c3815e1b0 0x7f4c380b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.269+0000 7f4c37fff700 1 -- 172.21.15.83:0/4042317226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c009550 con 0x7f4c3815ddd0 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.270+0000 7f4c37fff700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3815ddd0 0x7f4c381586c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c450 tx=0x7f4c2400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.270+0000 7f4c357fa700 1 -- 172.21.15.83:0/4042317226 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c240052d0 con 0x7f4c3815ddd0 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.270+0000 7f4c357fa700 1 -- 172.21.15.83:0/4042317226 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c24005430 con 0x7f4c3815ddd0 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.270+0000 7f4c3efe5700 1 -- 172.21.15.83:0/4042317226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c380b2ed0 con 0x7f4c3815ddd0 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.270+0000 7f4c357fa700 1 -- 172.21.15.83:0/4042317226 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c240038c0 con 0x7f4c3815ddd0 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.270+0000 7f4c3efe5700 1 -- 172.21.15.83:0/4042317226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3815f830 con 0x7f4c3815ddd0 2023-11-26T07:12:47.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.271+0000 7f4c3efe5700 1 -- 172.21.15.83:0/4042317226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c38160160 con 0x7f4c3815ddd0 2023-11-26T07:12:47.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.271+0000 7f4c357fa700 1 -- 172.21.15.83:0/4042317226 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4c24003a20 con 0x7f4c3815ddd0 2023-11-26T07:12:47.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.271+0000 7f4c357fa700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4c1c06b080 0x7f4c1c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.272+0000 7f4c357fa700 1 -- 172.21.15.83:0/4042317226 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4c24088910 con 0x7f4c3815ddd0 2023-11-26T07:12:47.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.272+0000 7f4c3cd81700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4c1c06b080 0x7f4c1c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.272+0000 7f4c3cd81700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4c1c06b080 0x7f4c1c06d540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009c40 tx=0x7f4c2c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:47.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.276+0000 7f4c357fa700 1 -- 172.21.15.83:0/4042317226 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c24053a40 con 0x7f4c3815ddd0 2023-11-26T07:12:47.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.467+0000 7f4c3efe5700 1 -- 172.21.15.83:0/4042317226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c380b2d30 con 0x7f4c3815ddd0 2023-11-26T07:12:47.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.467+0000 7f4c357fa700 1 -- 172.21.15.83:0/4042317226 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+12974 (secure 0 0 0) 0x7f4c380b2d30 con 0x7f4c3815ddd0 2023-11-26T07:12:47.471 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:47.471 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":37,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:46.121871+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6835/995330476":"2023-11-27T07:12:43.113472+0000","172.21.15.203:6833/995330476":"2023-11-27T07:12:43.113472+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:47.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.469+0000 7f4c3efe5700 1 -- 172.21.15.83:0/4042317226 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4c1c06b080 msgr2=0x7f4c1c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.469+0000 7f4c3efe5700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4c1c06b080 0x7f4c1c06d540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009c40 tx=0x7f4c2c008030 comp rx=0 tx=0).stop 2023-11-26T07:12:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.469+0000 7f4c3efe5700 1 -- 172.21.15.83:0/4042317226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3815ddd0 msgr2=0x7f4c381586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.469+0000 7f4c3efe5700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3815ddd0 0x7f4c381586c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c450 tx=0x7f4c2400c760 comp rx=0 tx=0).stop 2023-11-26T07:12:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.470+0000 7f4c3efe5700 1 -- 172.21.15.83:0/4042317226 shutdown_connections 2023-11-26T07:12:47.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.470+0000 7f4c3efe5700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c3815e1b0 0x7f4c380b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.470+0000 7f4c3efe5700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4c1c06b080 0x7f4c1c06d540 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.470+0000 7f4c3efe5700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c380d2240 0x7f4c380d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.470+0000 7f4c3efe5700 1 --2- 172.21.15.83:0/4042317226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c3815ddd0 0x7f4c381586c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.470+0000 7f4c3efe5700 1 -- 172.21.15.83:0/4042317226 >> 172.21.15.83:0/4042317226 conn(0x7f4c380731a0 msgr2=0x7f4c380cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.470+0000 7f4c3efe5700 1 -- 172.21.15.83:0/4042317226 shutdown_connections 2023-11-26T07:12:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.470+0000 7f4c3efe5700 1 -- 172.21.15.83:0/4042317226 wait complete. 2023-11-26T07:12:47.485 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:12:47.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:12:47.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.607+0000 7f47d5b93700 1 Processor -- start 2023-11-26T07:12:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.607+0000 7f47d5b93700 1 -- start start 2023-11-26T07:12:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47d5b93700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47d00c6c30 0x7f47d00cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47d5b93700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f47d00cef40 0x7f47d00cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47d5b93700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d00d0140 con 0x7f47d015bf50 2023-11-26T07:12:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47d5b93700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d015c330 con 0x7f47d00cef40 2023-11-26T07:12:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47d5b93700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00cf3d0 con 0x7f47d00c6c30 2023-11-26T07:12:47.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47cffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f47d00cef40 0x7f47d00cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47cffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f47d00cef40 0x7f47d00cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36838/0 (socket says 172.21.15.83:36838) 2023-11-26T07:12:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47cffff700 1 -- 172.21.15.83:0/1340951222 learned_addr learned my addr 172.21.15.83:0/1340951222 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47ceffd700 1 --2- 172.21.15.83:0/1340951222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47d00c6c30 0x7f47d00cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47cffff700 1 -- 172.21.15.83:0/1340951222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47d00c6c30 msgr2=0x7f47d00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47cffff700 1 --2- 172.21.15.83:0/1340951222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47d00c6c30 0x7f47d00cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47cffff700 1 -- 172.21.15.83:0/1340951222 >> v1:172.21.15.203:6789/0 conn(0x7f47d015bf50 legacy=0x7f47d00ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47cffff700 1 -- 172.21.15.83:0/1340951222 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d00d03e0 con 0x7f47d00cef40 2023-11-26T07:12:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.608+0000 7f47cffff700 1 --2- 172.21.15.83:0/1340951222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d00cef40 0x7f47d00cc6c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009e50 tx=0x7f47c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783216685ff3b73e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.609+0000 7f47ce7fc700 1 -- 172.21.15.83:0/1340951222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c800e070 con 0x7f47d00cef40 2023-11-26T07:12:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.609+0000 7f47ce7fc700 1 -- 172.21.15.83:0/1340951222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c800b9c0 con 0x7f47d00cef40 2023-11-26T07:12:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.609+0000 7f47ce7fc700 1 -- 172.21.15.83:0/1340951222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c800bd70 con 0x7f47d00cef40 2023-11-26T07:12:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.609+0000 7f47d5b93700 1 -- 172.21.15.83:0/1340951222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d00cef40 msgr2=0x7f47d00cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.609+0000 7f47d5b93700 1 --2- 172.21.15.83:0/1340951222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d00cef40 0x7f47d00cc6c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009e50 tx=0x7f47c800b330 comp rx=0 tx=0).stop 2023-11-26T07:12:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.609+0000 7f47d5b93700 1 -- 172.21.15.83:0/1340951222 shutdown_connections 2023-11-26T07:12:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.609+0000 7f47d5b93700 1 --2- 172.21.15.83:0/1340951222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d00cef40 0x7f47d00cc6c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.609+0000 7f47d5b93700 1 --2- 172.21.15.83:0/1340951222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47d00c6c30 0x7f47d00cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.609+0000 7f47d5b93700 1 -- 172.21.15.83:0/1340951222 >> 172.21.15.83:0/1340951222 conn(0x7f47d00b3780 msgr2=0x7f47d00b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.609+0000 7f47d5b93700 1 -- 172.21.15.83:0/1340951222 shutdown_connections 2023-11-26T07:12:47.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.609+0000 7f47d5b93700 1 -- 172.21.15.83:0/1340951222 wait complete. 2023-11-26T07:12:47.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.611+0000 7f47d5b93700 1 Processor -- start 2023-11-26T07:12:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.611+0000 7f47d5b93700 1 -- start start 2023-11-26T07:12:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47d5b93700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d00c6c30 0x7f47d0152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47d5b93700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47d00cef40 0x7f47d006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47d5b93700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47d015bf50 0x7f47d006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47d5b93700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d015c470 con 0x7f47d00c6c30 2023-11-26T07:12:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47d5b93700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00b66c0 con 0x7f47d015bf50 2023-11-26T07:12:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47d5b93700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00b69a0 con 0x7f47d00cef40 2023-11-26T07:12:47.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47cf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d00c6c30 0x7f47d0152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47cf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d00c6c30 0x7f47d0152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36840/0 (socket says 172.21.15.83:36840) 2023-11-26T07:12:47.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47cf7fe700 1 -- 172.21.15.83:0/1365634000 learned_addr learned my addr 172.21.15.83:0/1365634000 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:47.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47cffff700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47d015bf50 0x7f47d006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47ceffd700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47d00cef40 0x7f47d006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47cf7fe700 1 -- 172.21.15.83:0/1365634000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47d00cef40 msgr2=0x7f47d006f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47cf7fe700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47d00cef40 0x7f47d006f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47cf7fe700 1 -- 172.21.15.83:0/1365634000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47d015bf50 msgr2=0x7f47d006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47cf7fe700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47d015bf50 0x7f47d006fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47cf7fe700 1 -- 172.21.15.83:0/1365634000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c8009b30 con 0x7f47d00c6c30 2023-11-26T07:12:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47cf7fe700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d00c6c30 0x7f47d0152150 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f47b800c450 tx=0x7f47b800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47c7fff700 1 -- 172.21.15.83:0/1365634000 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80052d0 con 0x7f47d00c6c30 2023-11-26T07:12:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47c7fff700 1 -- 172.21.15.83:0/1365634000 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b8005430 con 0x7f47d00c6c30 2023-11-26T07:12:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47d5b93700 1 -- 172.21.15.83:0/1365634000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d00b6c80 con 0x7f47d00c6c30 2023-11-26T07:12:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.612+0000 7f47c7fff700 1 -- 172.21.15.83:0/1365634000 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80038c0 con 0x7f47d00c6c30 2023-11-26T07:12:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.613+0000 7f47d5b93700 1 -- 172.21.15.83:0/1365634000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d0152690 con 0x7f47d00c6c30 2023-11-26T07:12:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.613+0000 7f47d5b93700 1 -- 172.21.15.83:0/1365634000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d0152850 con 0x7f47d00c6c30 2023-11-26T07:12:47.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.614+0000 7f47c7fff700 1 -- 172.21.15.83:0/1365634000 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f47b8003a20 con 0x7f47d00c6c30 2023-11-26T07:12:47.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.614+0000 7f47c7fff700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47b006b0b0 0x7f47b006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.614+0000 7f47c7fff700 1 -- 172.21.15.83:0/1365634000 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f47b8088950 con 0x7f47d00c6c30 2023-11-26T07:12:47.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.614+0000 7f47ceffd700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47b006b0b0 0x7f47b006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.615+0000 7f47ceffd700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47b006b0b0 0x7f47b006d570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f47d015b060 tx=0x7f47c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:47.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.619+0000 7f47c7fff700 1 -- 172.21.15.83:0/1365634000 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47b8053a80 con 0x7f47d00c6c30 2023-11-26T07:12:47.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.808+0000 7f47d5b93700 1 -- 172.21.15.83:0/1365634000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47d00cbf70 con 0x7f47d00c6c30 2023-11-26T07:12:47.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.809+0000 7f47c7fff700 1 -- 172.21.15.83:0/1365634000 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47b80570a0 con 0x7f47d00c6c30 2023-11-26T07:12:47.811 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:12:47.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 -- 172.21.15.83:0/1365634000 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47b006b0b0 msgr2=0x7f47b006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47b006b0b0 0x7f47b006d570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f47d015b060 tx=0x7f47c0008040 comp rx=0 tx=0).stop 2023-11-26T07:12:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 -- 172.21.15.83:0/1365634000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d00c6c30 msgr2=0x7f47d0152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d00c6c30 0x7f47d0152150 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f47b800c450 tx=0x7f47b800c810 comp rx=0 tx=0).stop 2023-11-26T07:12:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 -- 172.21.15.83:0/1365634000 shutdown_connections 2023-11-26T07:12:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47d015bf50 0x7f47d006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47d00c6c30 0x7f47d0152150 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47b006b0b0 0x7f47b006d570 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 --2- 172.21.15.83:0/1365634000 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47d00cef40 0x7f47d006f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 -- 172.21.15.83:0/1365634000 >> 172.21.15.83:0/1365634000 conn(0x7f47d00b3780 msgr2=0x7f47d00c7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 -- 172.21.15.83:0/1365634000 shutdown_connections 2023-11-26T07:12:47.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.811+0000 7f47d5b93700 1 -- 172.21.15.83:0/1365634000 wait complete. 2023-11-26T07:12:47.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-11-26T07:12:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.943+0000 7f57b4206700 1 Processor -- start 2023-11-26T07:12:47.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.943+0000 7f57b4206700 1 -- start start 2023-11-26T07:12:47.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b4206700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f57ac0b3370 0x7f57ac1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b4206700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57ac0d1c70 0x7f57ac0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b4206700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ac0b22d0 con 0x7f57ac15bdd0 2023-11-26T07:12:47.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b4206700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ac0b2f50 con 0x7f57ac0d1c70 2023-11-26T07:12:47.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b4206700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ac0d2640 con 0x7f57ac0b3370 2023-11-26T07:12:47.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b27a3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57ac0d1c70 0x7f57ac0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b27a3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57ac0d1c70 0x7f57ac0d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36852/0 (socket says 172.21.15.83:36852) 2023-11-26T07:12:47.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b27a3700 1 -- 172.21.15.83:0/1718121078 learned_addr learned my addr 172.21.15.83:0/1718121078 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:47.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b17a1700 1 --2- 172.21.15.83:0/1718121078 >> v2:172.21.15.203:3300/0 conn(0x7f57ac0b3370 0x7f57ac1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b27a3700 1 -- 172.21.15.83:0/1718121078 >> v2:172.21.15.203:3300/0 conn(0x7f57ac0b3370 msgr2=0x7f57ac1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b27a3700 1 --2- 172.21.15.83:0/1718121078 >> v2:172.21.15.203:3300/0 conn(0x7f57ac0b3370 0x7f57ac1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b27a3700 1 -- 172.21.15.83:0/1718121078 >> v1:172.21.15.203:6789/0 conn(0x7f57ac15bdd0 legacy=0x7f57ac0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.944+0000 7f57b27a3700 1 -- 172.21.15.83:0/1718121078 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac0b25b0 con 0x7f57ac0d1c70 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b27a3700 1 --2- 172.21.15.83:0/1718121078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac0d1c70 0x7f57ac0d2100 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f57ac15f910 tx=0x7f57a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b1d0d3af460145 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b0fa0700 1 -- 172.21.15.83:0/1718121078 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a8004230 con 0x7f57ac0d1c70 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b0fa0700 1 -- 172.21.15.83:0/1718121078 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a8004390 con 0x7f57ac0d1c70 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b0fa0700 1 -- 172.21.15.83:0/1718121078 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a80036a0 con 0x7f57ac0d1c70 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b4206700 1 -- 172.21.15.83:0/1718121078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac0d1c70 msgr2=0x7f57ac0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b4206700 1 --2- 172.21.15.83:0/1718121078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac0d1c70 0x7f57ac0d2100 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f57ac15f910 tx=0x7f57a8009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b4206700 1 -- 172.21.15.83:0/1718121078 shutdown_connections 2023-11-26T07:12:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b4206700 1 --2- 172.21.15.83:0/1718121078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac0d1c70 0x7f57ac0d2100 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b4206700 1 --2- 172.21.15.83:0/1718121078 >> v2:172.21.15.203:3300/0 conn(0x7f57ac0b3370 0x7f57ac1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b4206700 1 -- 172.21.15.83:0/1718121078 >> 172.21.15.83:0/1718121078 conn(0x7f57ac0731a0 msgr2=0x7f57ac09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:47.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b4206700 1 -- 172.21.15.83:0/1718121078 shutdown_connections 2023-11-26T07:12:47.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.945+0000 7f57b4206700 1 -- 172.21.15.83:0/1718121078 wait complete. 2023-11-26T07:12:47.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.946+0000 7f57b4206700 1 Processor -- start 2023-11-26T07:12:47.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.946+0000 7f57b4206700 1 -- start start 2023-11-26T07:12:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.947+0000 7f57b4206700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac0b3370 0x7f57ac156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.947+0000 7f57b4206700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57ac0d1c70 0x7f57ac0ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.947+0000 7f57b4206700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57ac15bdd0 0x7f57ac0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.947+0000 7f57b4206700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0c9ca0 con 0x7f57ac0b3370 2023-11-26T07:12:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.947+0000 7f57b4206700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0c8db0 con 0x7f57ac15bdd0 2023-11-26T07:12:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.947+0000 7f57b4206700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0c9090 con 0x7f57ac0d1c70 2023-11-26T07:12:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.947+0000 7f57b1fa2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac0b3370 0x7f57ac156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.947+0000 7f57b1fa2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac0b3370 0x7f57ac156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36854/0 (socket says 172.21.15.83:36854) 2023-11-26T07:12:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.947+0000 7f57b1fa2700 1 -- 172.21.15.83:0/2959636156 learned_addr learned my addr 172.21.15.83:0/2959636156 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.947+0000 7f57b27a3700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57ac15bdd0 0x7f57ac0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.947+0000 7f57b17a1700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57ac0d1c70 0x7f57ac0ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.948+0000 7f57b1fa2700 1 -- 172.21.15.83:0/2959636156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57ac0d1c70 msgr2=0x7f57ac0ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.948+0000 7f57b1fa2700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57ac0d1c70 0x7f57ac0ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.948+0000 7f57b1fa2700 1 -- 172.21.15.83:0/2959636156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57ac15bdd0 msgr2=0x7f57ac0cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.948+0000 7f57b1fa2700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57ac15bdd0 0x7f57ac0cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.948+0000 7f57b1fa2700 1 -- 172.21.15.83:0/2959636156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a8009550 con 0x7f57ac0b3370 2023-11-26T07:12:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.948+0000 7f57b1fa2700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac0b3370 0x7f57ac156e10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f57a400ba50 tx=0x7f57a400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.948+0000 7f57a27fc700 1 -- 172.21.15.83:0/2959636156 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a400c9b0 con 0x7f57ac0b3370 2023-11-26T07:12:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.948+0000 7f57a27fc700 1 -- 172.21.15.83:0/2959636156 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a400cb50 con 0x7f57ac0b3370 2023-11-26T07:12:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.948+0000 7f57b4206700 1 -- 172.21.15.83:0/2959636156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ac0c9370 con 0x7f57ac0b3370 2023-11-26T07:12:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.948+0000 7f57a27fc700 1 -- 172.21.15.83:0/2959636156 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4014440 con 0x7f57ac0b3370 2023-11-26T07:12:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.948+0000 7f57b4206700 1 -- 172.21.15.83:0/2959636156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ac0c9930 con 0x7f57ac0b3370 2023-11-26T07:12:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.949+0000 7f57b4206700 1 -- 172.21.15.83:0/2959636156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ac067f60 con 0x7f57ac0b3370 2023-11-26T07:12:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.949+0000 7f57a27fc700 1 -- 172.21.15.83:0/2959636156 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f57a40145a0 con 0x7f57ac0b3370 2023-11-26T07:12:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.950+0000 7f57a27fc700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f579006b080 0x7f579006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.950+0000 7f57a27fc700 1 -- 172.21.15.83:0/2959636156 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f57a40891a0 con 0x7f57ac0b3370 2023-11-26T07:12:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.950+0000 7f57b17a1700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f579006b080 0x7f579006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.950+0000 7f57b17a1700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f579006b080 0x7f579006d540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0b7b40 tx=0x7f579c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:47.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:47.954+0000 7f57a27fc700 1 -- 172.21.15.83:0/2959636156 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57a4054380 con 0x7f57ac0b3370 2023-11-26T07:12:48.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:48.225+0000 7f57b4206700 1 -- 172.21.15.83:0/2959636156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f57ac15cd50 con 0x7f57ac0b3370 2023-11-26T07:12:49.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.138+0000 7f57a27fc700 1 -- 172.21.15.83:0/2959636156 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v101) v1 ==== 143+0+0 (secure 0 0 0) 0x7f57a40579a0 con 0x7f57ac0b3370 2023-11-26T07:12:49.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.142+0000 7f57b4206700 1 -- 172.21.15.83:0/2959636156 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f579006b080 msgr2=0x7f579006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.142+0000 7f57b4206700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f579006b080 0x7f579006d540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0b7b40 tx=0x7f579c008040 comp rx=0 tx=0).stop 2023-11-26T07:12:49.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.142+0000 7f57b4206700 1 -- 172.21.15.83:0/2959636156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac0b3370 msgr2=0x7f57ac156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.142+0000 7f57b4206700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac0b3370 0x7f57ac156e10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f57a400ba50 tx=0x7f57a400bd60 comp rx=0 tx=0).stop 2023-11-26T07:12:49.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.143+0000 7f57b4206700 1 -- 172.21.15.83:0/2959636156 shutdown_connections 2023-11-26T07:12:49.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.143+0000 7f57b4206700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57ac15bdd0 0x7f57ac0cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.143+0000 7f57b4206700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57ac0b3370 0x7f57ac156e10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.143+0000 7f57b4206700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f579006b080 0x7f579006d540 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.143+0000 7f57b4206700 1 --2- 172.21.15.83:0/2959636156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57ac0d1c70 0x7f57ac0ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.143+0000 7f57b4206700 1 -- 172.21.15.83:0/2959636156 >> 172.21.15.83:0/2959636156 conn(0x7f57ac0731a0 msgr2=0x7f57ac1601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:49.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.143+0000 7f57b4206700 1 -- 172.21.15.83:0/2959636156 shutdown_connections 2023-11-26T07:12:49.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.143+0000 7f57b4206700 1 -- 172.21.15.83:0/2959636156 wait complete. 2023-11-26T07:12:49.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:12:49.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.289+0000 7fd556ad7700 1 Processor -- start 2023-11-26T07:12:49.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.290+0000 7fd556ad7700 1 -- start start 2023-11-26T07:12:49.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.290+0000 7fd556ad7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5501587f0 0x7fd55006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.290+0000 7fd556ad7700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55006b530 con 0x7fd55015b370 2023-11-26T07:12:49.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.290+0000 7fd556ad7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5500ae5e0 con 0x7fd5500b4370 2023-11-26T07:12:49.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.290+0000 7fd556ad7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55015be30 con 0x7fd5501587f0 2023-11-26T07:12:49.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54ffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd55015b370 0x7fd5500b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56904/0 (socket says 172.21.15.83:56904) 2023-11-26T07:12:49.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54ffff700 1 -- 172.21.15.83:0/242330096 learned_addr learned my addr 172.21.15.83:0/242330096 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:49.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd554873700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5501587f0 0x7fd55006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756374095 0 0) 0x7fd55006b530 con 0x7fd55015b370 2023-11-26T07:12:49.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5400035b0 con 0x7fd55015b370 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219832737 0 0) 0x7fd5500ae5e0 con 0x7fd5500b4370 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd55006b530 con 0x7fd5500b4370 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1929548221 0 0) 0x7fd5400035b0 con 0x7fd55015b370 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5501587f0 msgr2=0x7fd55006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5501587f0 0x7fd55006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 >> v1:172.21.15.203:6789/0 conn(0x7fd5500b4370 legacy=0x7fd5500af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55015c110 con 0x7fd55015b370 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fd53c001670 con 0x7fd55015b370 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 >> v1:172.21.15.83:6789/0 conn(0x7fd55015b370 legacy=0x7fd5500b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5501587f0 0x7fd540003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd540004120 0x7fd540004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd540004d90 0x7fd540005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55015be30 con 0x7fd540004d90 2023-11-26T07:12:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54000baf0 con 0x7fd540004120 2023-11-26T07:12:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54000bcb0 con 0x7fd5501587f0 2023-11-26T07:12:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd555074700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd540004d90 0x7fd540005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd554873700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5501587f0 0x7fd540003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.291+0000 7fd54ffff700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd540004120 0x7fd540004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd555074700 1 -- 172.21.15.83:0/242330096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5501587f0 msgr2=0x7fd540003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd555074700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5501587f0 0x7fd540003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd555074700 1 -- 172.21.15.83:0/242330096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd540004120 msgr2=0x7fd540004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:49.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd555074700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd540004120 0x7fd540004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd555074700 1 -- 172.21.15.83:0/242330096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400035b0 con 0x7fd540004d90 2023-11-26T07:12:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd555074700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd540004d90 0x7fd540005210 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd548002230 tx=0x7fd540005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50de94194623845 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548001f30 con 0x7fd540004d90 2023-11-26T07:12:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54800ce60 con 0x7fd540004d90 2023-11-26T07:12:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd54f7fe700 1 -- 172.21.15.83:0/242330096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548003c40 con 0x7fd540004d90 2023-11-26T07:12:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd556ad7700 1 -- 172.21.15.83:0/242330096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd540004d90 msgr2=0x7fd540005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd556ad7700 1 --2- 172.21.15.83:0/242330096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd540004d90 0x7fd540005210 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd548002230 tx=0x7fd540005880 comp rx=0 tx=0).stop 2023-11-26T07:12:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd555074700 1 -- 172.21.15.83:0/242330096 reap_dead start 2023-11-26T07:12:49.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd556ad7700 1 -- 172.21.15.83:0/242330096 shutdown_connections 2023-11-26T07:12:49.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd556ad7700 1 -- 172.21.15.83:0/242330096 >> 172.21.15.83:0/242330096 conn(0x7fd5500d0bf0 msgr2=0x7fd5500d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:49.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd556ad7700 1 -- 172.21.15.83:0/242330096 shutdown_connections 2023-11-26T07:12:49.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.292+0000 7fd556ad7700 1 -- 172.21.15.83:0/242330096 wait complete. 2023-11-26T07:12:49.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.294+0000 7fd556ad7700 1 Processor -- start 2023-11-26T07:12:49.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.294+0000 7fd556ad7700 1 -- start start 2023-11-26T07:12:49.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.294+0000 7fd556ad7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd540004d90 0x7fd550149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd556ad7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5501499a0 0x7fd55015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd556ad7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd55009b5a0 0x7fd55015c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd556ad7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55014a400 con 0x7fd55009b5a0 2023-11-26T07:12:49.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd556ad7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5500affb0 con 0x7fd5501499a0 2023-11-26T07:12:49.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd556ad7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55009ccd0 con 0x7fd540004d90 2023-11-26T07:12:49.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd555074700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd55009b5a0 0x7fd55015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd555074700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd55009b5a0 0x7fd55015c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36876/0 (socket says 172.21.15.83:36876) 2023-11-26T07:12:49.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd555074700 1 -- 172.21.15.83:0/1095276818 learned_addr learned my addr 172.21.15.83:0/1095276818 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd54ffff700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5501499a0 0x7fd55015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd554873700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd540004d90 0x7fd550149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd555074700 1 -- 172.21.15.83:0/1095276818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd540004d90 msgr2=0x7fd550149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd555074700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd540004d90 0x7fd550149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd555074700 1 -- 172.21.15.83:0/1095276818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5501499a0 msgr2=0x7fd55015c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd555074700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5501499a0 0x7fd55015c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd555074700 1 -- 172.21.15.83:0/1095276818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd550149e40 con 0x7fd55009b5a0 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd555074700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd55009b5a0 0x7fd55015c950 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd548005040 tx=0x7fd548001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd54d7fa700 1 -- 172.21.15.83:0/1095276818 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54800cc60 con 0x7fd55009b5a0 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd54d7fa700 1 -- 172.21.15.83:0/1095276818 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5480050d0 con 0x7fd55009b5a0 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd556ad7700 1 -- 172.21.15.83:0/1095276818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55014a120 con 0x7fd55009b5a0 2023-11-26T07:12:49.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd54d7fa700 1 -- 172.21.15.83:0/1095276818 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548005480 con 0x7fd55009b5a0 2023-11-26T07:12:49.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.295+0000 7fd556ad7700 1 -- 172.21.15.83:0/1095276818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55009d250 con 0x7fd55009b5a0 2023-11-26T07:12:49.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.296+0000 7fd556ad7700 1 -- 172.21.15.83:0/1095276818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55009d3e0 con 0x7fd55009b5a0 2023-11-26T07:12:49.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.297+0000 7fd54d7fa700 1 -- 172.21.15.83:0/1095276818 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fd548019070 con 0x7fd55009b5a0 2023-11-26T07:12:49.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.297+0000 7fd54d7fa700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd53406b080 0x7fd53406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.297+0000 7fd54d7fa700 1 -- 172.21.15.83:0/1095276818 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd54808a110 con 0x7fd55009b5a0 2023-11-26T07:12:49.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.297+0000 7fd554873700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd53406b080 0x7fd53406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.298+0000 7fd554873700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd53406b080 0x7fd53406d540 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd550075030 tx=0x7fd544009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:49.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.302+0000 7fd54d7fa700 1 -- 172.21.15.83:0/1095276818 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5480553f0 con 0x7fd55009b5a0 2023-11-26T07:12:49.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.495+0000 7fd556ad7700 1 -- 172.21.15.83:0/1095276818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd550068fb0 con 0x7fd55009b5a0 2023-11-26T07:12:49.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.500+0000 7fd54d7fa700 1 -- 172.21.15.83:0/1095276818 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd5480011c0 con 0x7fd55009b5a0 2023-11-26T07:12:49.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 -- 172.21.15.83:0/1095276818 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd53406b080 msgr2=0x7fd53406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd53406b080 0x7fd53406d540 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd550075030 tx=0x7fd544009210 comp rx=0 tx=0).stop 2023-11-26T07:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 -- 172.21.15.83:0/1095276818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd55009b5a0 msgr2=0x7fd55015c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd55009b5a0 0x7fd55015c950 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd548005040 tx=0x7fd548001650 comp rx=0 tx=0).stop 2023-11-26T07:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 -- 172.21.15.83:0/1095276818 shutdown_connections 2023-11-26T07:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd55009b5a0 0x7fd55015c950 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd53406b080 0x7fd53406d540 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd540004d90 0x7fd550149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 --2- 172.21.15.83:0/1095276818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5501499a0 0x7fd55015c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 -- 172.21.15.83:0/1095276818 >> 172.21.15.83:0/1095276818 conn(0x7fd5500d0bf0 msgr2=0x7fd5500b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 -- 172.21.15.83:0/1095276818 shutdown_connections 2023-11-26T07:12:49.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.503+0000 7fd556ad7700 1 -- 172.21.15.83:0/1095276818 wait complete. 2023-11-26T07:12:49.517 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:12:49.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:12:49.559 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:12:49.618 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:12:49.618 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:49.618 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:49.635 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:12:49.636 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:49.636 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:49.656 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:12:49.656 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:12:49.656 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:49.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:12:49.693 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:12:49.693 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:12:49.714 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:12:49.715 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:12:49.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.850+0000 7ff16cb1d700 1 Processor -- start 2023-11-26T07:12:49.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.850+0000 7ff16cb1d700 1 -- start start 2023-11-26T07:12:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16cb1d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff168158670 0x7ff1680622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16cb1d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff1680c9fc0 0x7ff1680ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16cb1d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1680cfff0 con 0x7ff168158a50 2023-11-26T07:12:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16cb1d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1680cfd10 con 0x7ff168158670 2023-11-26T07:12:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16cb1d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff168161f50 con 0x7ff1680c9fc0 2023-11-26T07:12:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff168158670 0x7ff1680622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff168158670 0x7ff1680622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36880/0 (socket says 172.21.15.83:36880) 2023-11-26T07:12:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16659c700 1 -- 172.21.15.83:0/2504898676 learned_addr learned my addr 172.21.15.83:0/2504898676 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:49.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff166d9d700 1 --2- 172.21.15.83:0/2504898676 >> v2:172.21.15.203:3300/0 conn(0x7ff1680c9fc0 0x7ff1680ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16659c700 1 -- 172.21.15.83:0/2504898676 >> v2:172.21.15.203:3300/0 conn(0x7ff1680c9fc0 msgr2=0x7ff1680ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16659c700 1 --2- 172.21.15.83:0/2504898676 >> v2:172.21.15.203:3300/0 conn(0x7ff1680c9fc0 0x7ff1680ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16659c700 1 -- 172.21.15.83:0/2504898676 >> v1:172.21.15.203:6789/0 conn(0x7ff168158a50 legacy=0x7ff1680c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16659c700 1 -- 172.21.15.83:0/2504898676 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff168162230 con 0x7ff168158670 2023-11-26T07:12:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff166d9d700 1 --2- 172.21.15.83:0/2504898676 >> v2:172.21.15.203:3300/0 conn(0x7ff1680c9fc0 0x7ff1680ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.851+0000 7ff16659c700 1 --2- 172.21.15.83:0/2504898676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff168158670 0x7ff1680622a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff1680b0070 tx=0x7ff158009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a363667c182fc8a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.852+0000 7ff16559a700 1 -- 172.21.15.83:0/2504898676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15800b870 con 0x7ff168158670 2023-11-26T07:12:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.852+0000 7ff16559a700 1 -- 172.21.15.83:0/2504898676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15800b9d0 con 0x7ff168158670 2023-11-26T07:12:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.852+0000 7ff16559a700 1 -- 172.21.15.83:0/2504898676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15800bd60 con 0x7ff168158670 2023-11-26T07:12:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.852+0000 7ff16cb1d700 1 -- 172.21.15.83:0/2504898676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff168158670 msgr2=0x7ff1680622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.852+0000 7ff16cb1d700 1 --2- 172.21.15.83:0/2504898676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff168158670 0x7ff1680622a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff1680b0070 tx=0x7ff158009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.852+0000 7ff16cb1d700 1 -- 172.21.15.83:0/2504898676 shutdown_connections 2023-11-26T07:12:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.852+0000 7ff16cb1d700 1 --2- 172.21.15.83:0/2504898676 >> v2:172.21.15.203:3300/0 conn(0x7ff1680c9fc0 0x7ff1680ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.852+0000 7ff16cb1d700 1 --2- 172.21.15.83:0/2504898676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff168158670 0x7ff1680622a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.852+0000 7ff16cb1d700 1 -- 172.21.15.83:0/2504898676 >> 172.21.15.83:0/2504898676 conn(0x7ff1680b57b0 msgr2=0x7ff1680b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.852+0000 7ff16cb1d700 1 -- 172.21.15.83:0/2504898676 shutdown_connections 2023-11-26T07:12:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.852+0000 7ff16cb1d700 1 -- 172.21.15.83:0/2504898676 wait complete. 2023-11-26T07:12:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.854+0000 7ff16cb1d700 1 Processor -- start 2023-11-26T07:12:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.854+0000 7ff16cb1d700 1 -- start start 2023-11-26T07:12:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.854+0000 7ff16cb1d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1680c9fc0 0x7ff168160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.854+0000 7ff16cb1d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff168158670 0x7ff1680b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.854+0000 7ff16cb1d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff168158a50 0x7ff1680b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.854+0000 7ff16cb1d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1680b4870 con 0x7ff168158670 2023-11-26T07:12:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.854+0000 7ff16cb1d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff168153f60 con 0x7ff168158a50 2023-11-26T07:12:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.854+0000 7ff16cb1d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1680b3bf0 con 0x7ff1680c9fc0 2023-11-26T07:12:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.854+0000 7ff165d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff168158670 0x7ff1680b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.854+0000 7ff165d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff168158670 0x7ff1680b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36884/0 (socket says 172.21.15.83:36884) 2023-11-26T07:12:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.854+0000 7ff165d9b700 1 -- 172.21.15.83:0/1789355542 learned_addr learned my addr 172.21.15.83:0/1789355542 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff166d9d700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff168158a50 0x7ff1680b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff16659c700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1680c9fc0 0x7ff168160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff165d9b700 1 -- 172.21.15.83:0/1789355542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1680c9fc0 msgr2=0x7ff168160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff165d9b700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1680c9fc0 0x7ff168160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff165d9b700 1 -- 172.21.15.83:0/1789355542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff168158a50 msgr2=0x7ff1680b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff165d9b700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff168158a50 0x7ff1680b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff165d9b700 1 -- 172.21.15.83:0/1789355542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff158009550 con 0x7ff168158670 2023-11-26T07:12:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff166d9d700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff168158a50 0x7ff1680b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:49.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff165d9b700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff168158670 0x7ff1680b3080 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00c430 tx=0x7ff15c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff156ffd700 1 -- 172.21.15.83:0/1789355542 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0052d0 con 0x7ff168158670 2023-11-26T07:12:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff156ffd700 1 -- 172.21.15.83:0/1789355542 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c005430 con 0x7ff168158670 2023-11-26T07:12:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff16cb1d700 1 -- 172.21.15.83:0/1789355542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1680b3ed0 con 0x7ff168158670 2023-11-26T07:12:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff156ffd700 1 -- 172.21.15.83:0/1789355542 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0038c0 con 0x7ff168158670 2023-11-26T07:12:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.855+0000 7ff16cb1d700 1 -- 172.21.15.83:0/1789355542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1680b4490 con 0x7ff168158670 2023-11-26T07:12:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.856+0000 7ff16cb1d700 1 -- 172.21.15.83:0/1789355542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff168067f60 con 0x7ff168158670 2023-11-26T07:12:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.857+0000 7ff156ffd700 1 -- 172.21.15.83:0/1789355542 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7ff15c003a20 con 0x7ff168158670 2023-11-26T07:12:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.857+0000 7ff156ffd700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff14806b080 0x7ff14806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.857+0000 7ff156ffd700 1 -- 172.21.15.83:0/1789355542 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff15c0889a0 con 0x7ff168158670 2023-11-26T07:12:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.857+0000 7ff16659c700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff14806b080 0x7ff14806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.857+0000 7ff16659c700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff14806b080 0x7ff14806d540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff158000c00 tx=0x7ff158008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:49.862+0000 7ff156ffd700 1 -- 172.21.15.83:0/1789355542 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff15c053ad0 con 0x7ff168158670 2023-11-26T07:12:50.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.091+0000 7ff16cb1d700 1 -- 172.21.15.83:0/1789355542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff168074a20 con 0x7ff168158670 2023-11-26T07:12:50.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.092+0000 7ff156ffd700 1 -- 172.21.15.83:0/1789355542 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3699 (secure 0 0 0) 0x7ff15c017090 con 0x7ff168158670 2023-11-26T07:12:50.094 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:50.094 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/486975373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":486975373},{"type":"v1","addr":"172.21.15.203:6834","nonce":486975373}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/3188343240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":3188343240},{"type":"v1","addr":"172.21.15.83:6836","nonce":3188343240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/4271462644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4271462644},{"type":"v1","addr":"172.21.15.83:6837","nonce":4271462644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5218,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/358688880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":358688880},{"type":"v1","addr":"172.21.15.203:6835","nonce":358688880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2023-11-26T07:12:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 -- 172.21.15.83:0/1789355542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff14806b080 msgr2=0x7ff14806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff14806b080 0x7ff14806d540 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff158000c00 tx=0x7ff158008030 comp rx=0 tx=0).stop 2023-11-26T07:12:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 -- 172.21.15.83:0/1789355542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff168158670 msgr2=0x7ff1680b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff168158670 0x7ff1680b3080 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00c430 tx=0x7ff15c00c7f0 comp rx=0 tx=0).stop 2023-11-26T07:12:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 -- 172.21.15.83:0/1789355542 shutdown_connections 2023-11-26T07:12:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff168158a50 0x7ff1680b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff14806b080 0x7ff14806d540 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1680c9fc0 0x7ff168160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 --2- 172.21.15.83:0/1789355542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff168158670 0x7ff1680b3080 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 -- 172.21.15.83:0/1789355542 >> 172.21.15.83:0/1789355542 conn(0x7ff1680b57b0 msgr2=0x7ff16806eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 -- 172.21.15.83:0/1789355542 shutdown_connections 2023-11-26T07:12:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.094+0000 7ff16cb1d700 1 -- 172.21.15.83:0/1789355542 wait complete. 2023-11-26T07:12:50.098 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 20 2023-11-26T07:12:50.108 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:12:50.109 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:12:50.109 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:12:50.109 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:12:50.109 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:12:50.109 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:12:50.109 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:12:50.109 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:12:50.110 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:12:50.110 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:12:50.110 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:12:50.110 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:12:50.110 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:12:50.110 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:12:50.111 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:12:50.111 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:12:50.118 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:12:50.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:12:50.119 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:12:50.119 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:12:50.124 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:12:50.124 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:12:50.125 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:12:50.125 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:12:50.160 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:12:50.160 DEBUG:teuthology.parallel:result is None 2023-11-26T07:12:50.161 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:12:50.161 DEBUG:teuthology.parallel:result is None 2023-11-26T07:12:50.167 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:12:50.167 DEBUG:teuthology.parallel:result is None 2023-11-26T07:12:50.167 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:12:50.167 DEBUG:teuthology.parallel:result is None 2023-11-26T07:12:50.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:12:50.240 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-26T07:12:50.240 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-26T07:12:50.241 INFO:tasks.ceph.mds.c.smithi083.stderr:2023-11-26T07:12:50.239+0000 7f0a5da61700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2023-11-26T07:12:50.257 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-11-26T07:12:50.257 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-11-26T07:12:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.293+0000 7f3e7f161700 1 Processor -- start 2023-11-26T07:12:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.293+0000 7f3e7f161700 1 -- start start 2023-11-26T07:12:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7f161700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780afc10 0x7f3e7815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7f161700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3e780746a0 0x7f3e78162120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7f161700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e780af090 con 0x7f3e7806fa40 2023-11-26T07:12:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7f161700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7806dda0 con 0x7f3e780746a0 2023-11-26T07:12:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7f161700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7806e060 con 0x7f3e780afc10 2023-11-26T07:12:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7cefd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780afc10 0x7f3e7815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7d6fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3e780746a0 0x7f3e78162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7cefd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780afc10 0x7f3e7815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:47380/0 (socket says 172.21.15.83:47380) 2023-11-26T07:12:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7cefd700 1 -- 172.21.15.83:0/4066892257 learned_addr learned my addr 172.21.15.83:0/4066892257 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7cefd700 1 -- 172.21.15.83:0/4066892257 >> v2:172.21.15.203:3300/0 conn(0x7f3e780746a0 msgr2=0x7f3e78162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7cefd700 1 --2- 172.21.15.83:0/4066892257 >> v2:172.21.15.203:3300/0 conn(0x7f3e780746a0 0x7f3e78162120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7cefd700 1 -- 172.21.15.83:0/4066892257 >> v1:172.21.15.203:6789/0 conn(0x7f3e7806fa40 legacy=0x7f3e78073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7cefd700 1 -- 172.21.15.83:0/4066892257 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7806ec70 con 0x7f3e780afc10 2023-11-26T07:12:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.294+0000 7f3e7d6fe700 1 --2- 172.21.15.83:0/4066892257 >> v2:172.21.15.203:3300/0 conn(0x7f3e780746a0 0x7f3e78162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:12:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e7cefd700 1 --2- 172.21.15.83:0/4066892257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780afc10 0x7f3e7815e850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0098f0 tx=0x7f3e6c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56dee2e32a895ff8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e777fe700 1 -- 172.21.15.83:0/4066892257 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c011830 con 0x7f3e780afc10 2023-11-26T07:12:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e777fe700 1 -- 172.21.15.83:0/4066892257 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c011990 con 0x7f3e780afc10 2023-11-26T07:12:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e777fe700 1 -- 172.21.15.83:0/4066892257 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c011d20 con 0x7f3e780afc10 2023-11-26T07:12:50.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e7f161700 1 -- 172.21.15.83:0/4066892257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780afc10 msgr2=0x7f3e7815e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:50.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e7f161700 1 --2- 172.21.15.83:0/4066892257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780afc10 0x7f3e7815e850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0098f0 tx=0x7f3e6c009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:50.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e7f161700 1 -- 172.21.15.83:0/4066892257 shutdown_connections 2023-11-26T07:12:50.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e7f161700 1 --2- 172.21.15.83:0/4066892257 >> v2:172.21.15.203:3300/0 conn(0x7f3e780746a0 0x7f3e78162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:50.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e7f161700 1 --2- 172.21.15.83:0/4066892257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780afc10 0x7f3e7815e850 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:50.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e7f161700 1 -- 172.21.15.83:0/4066892257 >> 172.21.15.83:0/4066892257 conn(0x7f3e7806aff0 msgr2=0x7f3e7806b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:50.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e7f161700 1 -- 172.21.15.83:0/4066892257 shutdown_connections 2023-11-26T07:12:50.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.295+0000 7f3e7f161700 1 -- 172.21.15.83:0/4066892257 wait complete. 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.296+0000 7f3e7f161700 1 Processor -- start 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.297+0000 7f3e7f161700 1 -- start start 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.297+0000 7f3e7f161700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7806fa40 0x7f3e78154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.297+0000 7f3e7f161700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780746a0 0x7f3e78144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.297+0000 7f3e7f161700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e780afc10 0x7f3e78153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.297+0000 7f3e7f161700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e781554d0 con 0x7f3e7806fa40 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.297+0000 7f3e7f161700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e78154b00 con 0x7f3e780afc10 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.297+0000 7f3e7f161700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e78151db0 con 0x7f3e780746a0 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.297+0000 7f3e7cefd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7806fa40 0x7f3e78154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.297+0000 7f3e7cefd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7806fa40 0x7f3e78154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36928/0 (socket says 172.21.15.83:36928) 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.297+0000 7f3e7cefd700 1 -- 172.21.15.83:0/2444111923 learned_addr learned my addr 172.21.15.83:0/2444111923 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e77fff700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780746a0 0x7f3e78144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e7d6fe700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e780afc10 0x7f3e78153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e7cefd700 1 -- 172.21.15.83:0/2444111923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780746a0 msgr2=0x7f3e78144440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e7cefd700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780746a0 0x7f3e78144440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e7cefd700 1 -- 172.21.15.83:0/2444111923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e780afc10 msgr2=0x7f3e78153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e7cefd700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e780afc10 0x7f3e78153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e7cefd700 1 -- 172.21.15.83:0/2444111923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e70009a70 con 0x7f3e7806fa40 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e7cefd700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7806fa40 0x7f3e78154f90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c014040 tx=0x7f3e6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e757fa700 1 -- 172.21.15.83:0/2444111923 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0043c0 con 0x7f3e7806fa40 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e757fa700 1 -- 172.21.15.83:0/2444111923 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c01f070 con 0x7f3e7806fa40 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e7f161700 1 -- 172.21.15.83:0/2444111923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c009550 con 0x7f3e7806fa40 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e757fa700 1 -- 172.21.15.83:0/2444111923 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0036a0 con 0x7f3e7806fa40 2023-11-26T07:12:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.298+0000 7f3e7f161700 1 -- 172.21.15.83:0/2444111923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e781541d0 con 0x7f3e7806fa40 2023-11-26T07:12:50.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.299+0000 7f3e7f161700 1 -- 172.21.15.83:0/2444111923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e78068f10 con 0x7f3e7806fa40 2023-11-26T07:12:50.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.300+0000 7f3e757fa700 1 -- 172.21.15.83:0/2444111923 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3e6c021030 con 0x7f3e7806fa40 2023-11-26T07:12:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.300+0000 7f3e757fa700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e5c06b080 0x7f3e5c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.300+0000 7f3e757fa700 1 -- 172.21.15.83:0/2444111923 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e6c08a930 con 0x7f3e7806fa40 2023-11-26T07:12:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.300+0000 7f3e77fff700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e5c06b080 0x7f3e5c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:50.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.300+0000 7f3e77fff700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e5c06b080 0x7f3e5c06d540 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3e781611f0 tx=0x7f3e64009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:50.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.304+0000 7f3e757fa700 1 -- 172.21.15.83:0/2444111923 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e6c055b80 con 0x7f3e7806fa40 2023-11-26T07:12:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:50.513+0000 7f3e7f161700 1 -- 172.21.15.83:0/2444111923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3e78074450 con 0x7f3e7806fa40 2023-11-26T07:12:51.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.157+0000 7f3e757fa700 1 -- 172.21.15.83:0/2444111923 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3e6c015080 con 0x7f3e7806fa40 2023-11-26T07:12:51.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.159+0000 7f3e7f161700 1 -- 172.21.15.83:0/2444111923 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e5c06b080 msgr2=0x7f3e5c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.159+0000 7f3e7f161700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e5c06b080 0x7f3e5c06d540 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3e781611f0 tx=0x7f3e64009040 comp rx=0 tx=0).stop 2023-11-26T07:12:51.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.159+0000 7f3e7f161700 1 -- 172.21.15.83:0/2444111923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7806fa40 msgr2=0x7f3e78154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.159+0000 7f3e7f161700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7806fa40 0x7f3e78154f90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c014040 tx=0x7f3e6c004750 comp rx=0 tx=0).stop 2023-11-26T07:12:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.160+0000 7f3e7f161700 1 -- 172.21.15.83:0/2444111923 shutdown_connections 2023-11-26T07:12:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.160+0000 7f3e7f161700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e780afc10 0x7f3e78153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.160+0000 7f3e7f161700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e7806fa40 0x7f3e78154f90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.160+0000 7f3e7f161700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3e5c06b080 0x7f3e5c06d540 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.160+0000 7f3e7f161700 1 --2- 172.21.15.83:0/2444111923 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e780746a0 0x7f3e78144440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.160+0000 7f3e7f161700 1 -- 172.21.15.83:0/2444111923 >> 172.21.15.83:0/2444111923 conn(0x7f3e7806aff0 msgr2=0x7f3e780b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.160+0000 7f3e7f161700 1 -- 172.21.15.83:0/2444111923 shutdown_connections 2023-11-26T07:12:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.160+0000 7f3e7f161700 1 -- 172.21.15.83:0/2444111923 wait complete. 2023-11-26T07:12:51.163 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2023-11-26T07:12:51.176 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:12:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.298+0000 7f4f45baa700 1 Processor -- start 2023-11-26T07:12:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.298+0000 7f4f45baa700 1 -- start start 2023-11-26T07:12:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f45baa700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4f4006c710 0x7f4f400b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f45baa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f400af4d0 0x7f4f400afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f45baa700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f400ae430 con 0x7f4f4015e080 2023-11-26T07:12:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f45baa700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f400af0b0 con 0x7f4f4006c710 2023-11-26T07:12:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f45baa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400b05a0 con 0x7f4f400af4d0 2023-11-26T07:12:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4f4015e080 0x7f4f4006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56978/0 (socket says 172.21.15.83:56978) 2023-11-26T07:12:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3f7fe700 1 -- 172.21.15.83:0/2813507674 learned_addr learned my addr 172.21.15.83:0/2813507674 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634239119 0 0) 0x7f4f400ae430 con 0x7f4f4015e080 2023-11-26T07:12:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f280035b0 con 0x7f4f4015e080 2023-11-26T07:12:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3effd700 1 --2- 172.21.15.83:0/2813507674 >> v2:172.21.15.203:3300/0 conn(0x7f4f4006c710 0x7f4f400b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3ffff700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f400af4d0 0x7f4f400afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 529836441 0 0) 0x7f4f280035b0 con 0x7f4f4015e080 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f400af4d0 msgr2=0x7f4f400afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f400af4d0 0x7f4f400afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 >> v2:172.21.15.203:3300/0 conn(0x7f4f4006c710 msgr2=0x7f4f400b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 --2- 172.21.15.83:0/2813507674 >> v2:172.21.15.203:3300/0 conn(0x7f4f4006c710 0x7f4f400b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f400ae710 con 0x7f4f4015e080 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f4f34001670 con 0x7f4f4015e080 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 >> v1:172.21.15.83:6789/0 conn(0x7f4f4015e080 legacy=0x7f4f4006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f28003a00 0x7f4f28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f400af4d0 0x7f4f28004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f28004d90 0x7f4f28005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400af0b0 con 0x7f4f28004d90 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400b05a0 con 0x7f4f28003a00 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.299+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2800ba10 con 0x7f4f400af4d0 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3ffff700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f28004d90 0x7f4f28005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3f7fe700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f28003a00 0x7f4f28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3effd700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f400af4d0 0x7f4f28004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3ffff700 1 -- 172.21.15.83:0/2813507674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f400af4d0 msgr2=0x7f4f28004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3ffff700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f400af4d0 0x7f4f28004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3ffff700 1 -- 172.21.15.83:0/2813507674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f28003a00 msgr2=0x7f4f28003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3ffff700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f28003a00 0x7f4f28003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3ffff700 1 -- 172.21.15.83:0/2813507674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f280035b0 con 0x7f4f28004d90 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3ffff700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f28004d90 0x7f4f28005210 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009fc0 tx=0x7f4f28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd87221f8fbc670 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800ec30 con 0x7f4f28004d90 2023-11-26T07:12:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3800f040 con 0x7f4f28004d90 2023-11-26T07:12:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3e7fc700 1 -- 172.21.15.83:0/2813507674 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38014480 con 0x7f4f28004d90 2023-11-26T07:12:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f45baa700 1 -- 172.21.15.83:0/2813507674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f28004d90 msgr2=0x7f4f28005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f45baa700 1 --2- 172.21.15.83:0/2813507674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f28004d90 0x7f4f28005210 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009fc0 tx=0x7f4f28005880 comp rx=0 tx=0).stop 2023-11-26T07:12:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.300+0000 7f4f3ffff700 1 -- 172.21.15.83:0/2813507674 reap_dead start 2023-11-26T07:12:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.301+0000 7f4f45baa700 1 -- 172.21.15.83:0/2813507674 shutdown_connections 2023-11-26T07:12:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.301+0000 7f4f45baa700 1 -- 172.21.15.83:0/2813507674 >> 172.21.15.83:0/2813507674 conn(0x7f4f40071b90 msgr2=0x7f4f40071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.301+0000 7f4f45baa700 1 -- 172.21.15.83:0/2813507674 shutdown_connections 2023-11-26T07:12:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.301+0000 7f4f45baa700 1 -- 172.21.15.83:0/2813507674 wait complete. 2023-11-26T07:12:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.302+0000 7f4f45baa700 1 Processor -- start 2023-11-26T07:12:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.302+0000 7f4f45baa700 1 -- start start 2023-11-26T07:12:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f45baa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f401493e0 0x7f4f40156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f45baa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f401497c0 0x7f4f401f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f45baa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f401f7620 0x7f4f400b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f45baa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f401f6eb0 con 0x7f4f401f7620 2023-11-26T07:12:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f45baa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f401f7190 con 0x7f4f401493e0 2023-11-26T07:12:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f45baa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f401f3e00 con 0x7f4f401497c0 2023-11-26T07:12:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f401f7620 0x7f4f400b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f401f7620 0x7f4f400b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36954/0 (socket says 172.21.15.83:36954) 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3ffff700 1 -- 172.21.15.83:0/4246297978 learned_addr learned my addr 172.21.15.83:0/4246297978 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3f7fe700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f401493e0 0x7f4f40156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3effd700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f401497c0 0x7f4f401f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3ffff700 1 -- 172.21.15.83:0/4246297978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f401497c0 msgr2=0x7f4f401f38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3ffff700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f401497c0 0x7f4f401f38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3ffff700 1 -- 172.21.15.83:0/4246297978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f401493e0 msgr2=0x7f4f40156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3ffff700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f401493e0 0x7f4f40156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3ffff700 1 -- 172.21.15.83:0/4246297978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f401f40a0 con 0x7f4f401f7620 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3effd700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f401497c0 0x7f4f401f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f3ffff700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f401f7620 0x7f4f400b17b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4f38016000 tx=0x7f4f38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f27fff700 1 -- 172.21.15.83:0/4246297978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38007000 con 0x7f4f401f7620 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f27fff700 1 -- 172.21.15.83:0/4246297978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f38022070 con 0x7f4f401f7620 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f45baa700 1 -- 172.21.15.83:0/4246297978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f401f4380 con 0x7f4f401f7620 2023-11-26T07:12:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f27fff700 1 -- 172.21.15.83:0/4246297978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38029480 con 0x7f4f401f7620 2023-11-26T07:12:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.303+0000 7f4f45baa700 1 -- 172.21.15.83:0/4246297978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f401f7ab0 con 0x7f4f401f7620 2023-11-26T07:12:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.304+0000 7f4f45baa700 1 -- 172.21.15.83:0/4246297978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f400753f0 con 0x7f4f401f7620 2023-11-26T07:12:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.305+0000 7f4f27fff700 1 -- 172.21.15.83:0/4246297978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4f3801e430 con 0x7f4f401f7620 2023-11-26T07:12:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.305+0000 7f4f27fff700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4f2c06b080 0x7f4f2c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.305+0000 7f4f27fff700 1 -- 172.21.15.83:0/4246297978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4f38024070 con 0x7f4f401f7620 2023-11-26T07:12:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.305+0000 7f4f3f7fe700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4f2c06b080 0x7f4f2c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.306+0000 7f4f3f7fe700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4f2c06b080 0x7f4f2c06d540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4f34000c00 tx=0x7f4f34007980 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:51.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.310+0000 7f4f27fff700 1 -- 172.21.15.83:0/4246297978 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f38056040 con 0x7f4f401f7620 2023-11-26T07:12:51.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.579+0000 7f4f45baa700 1 -- 172.21.15.83:0/4246297978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4f401f7300 con 0x7f4f401f7620 2023-11-26T07:12:51.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.580+0000 7f4f27fff700 1 -- 172.21.15.83:0/4246297978 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4f38018020 con 0x7f4f401f7620 2023-11-26T07:12:51.582 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlA8pxNxAAt5NuewAjm7yPThCguJ6sEA==", 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlHBgDCBAAk6egig5krY7kiMiHkMKrWg==", 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.584 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlbKEQAxAAM2tBBQwM+tt2d8JU37YURg==", 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlI3ZHDxAArtLmBVGr/vAB/YGwEFfhGw==", 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:51.585 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBo72JlhuSTKRAA6LRDOHBQVl5TL5tIHzyfzA==", 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBr72JlteU6KRAAuuHTYosYpl+KptywVmaeHA==", 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBu72JlEb6dKBAAebp4pnu9XZsSRJpojTZcvQ==", 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.587 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBx72JlbTJ3KBAA1JUxxAoyCzTtfgubUxSJEA==", 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.588 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB372JlufqqLhAAm6ZUt9juWj5yAIDlnCXqWg==", 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB672Jl1L3vLRAAX06GcsntcmB+7L1CsfnJzw==", 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:51.589 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB972Jlv41HLRAALtuyuREvGmTuDZt/RT0UcA==", 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.590 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCA72JlKCHsLBAAAXDzA7AN36wcLjhK97nlkg==", 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ==", 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2023-11-26T07:12:51.591 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JleNpIGhAAKwoQj4zB9YOxpFMD28juCA==", 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.592 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JldAHlFhAAb3u9KljU9vssDl54zgno2Q==", 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JluiQUIBAA06mqpeCzgpbSLo2/fulfmw==", 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:12:51.593 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlD4jbGhAA05xtSS5EzQTuhBjVqS2JpA==", 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlsA0VNBAAfSlnVquhI+vpLFwmgLGGVA==", 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72Jl3qkjMBAA+jcxYuDG/UHhB7W7HDwmaw==", 2023-11-26T07:12:51.595 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:12:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:12:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:12:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:12:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:51.596 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:12:51.596 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:12:51.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 -- 172.21.15.83:0/4246297978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4f2c06b080 msgr2=0x7f4f2c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4f2c06b080 0x7f4f2c06d540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4f34000c00 tx=0x7f4f34007980 comp rx=0 tx=0).stop 2023-11-26T07:12:51.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 -- 172.21.15.83:0/4246297978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f401f7620 msgr2=0x7f4f400b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f401f7620 0x7f4f400b17b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4f38016000 tx=0x7f4f38009750 comp rx=0 tx=0).stop 2023-11-26T07:12:51.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 -- 172.21.15.83:0/4246297978 shutdown_connections 2023-11-26T07:12:51.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f401f7620 0x7f4f400b17b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f401493e0 0x7f4f40156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4f2c06b080 0x7f4f2c06d540 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 --2- 172.21.15.83:0/4246297978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f401497c0 0x7f4f401f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 -- 172.21.15.83:0/4246297978 >> 172.21.15.83:0/4246297978 conn(0x7f4f40071b90 msgr2=0x7f4f4015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:51.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 -- 172.21.15.83:0/4246297978 shutdown_connections 2023-11-26T07:12:51.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.583+0000 7f4f45baa700 1 -- 172.21.15.83:0/4246297978 wait complete. 2023-11-26T07:12:51.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:51.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.716+0000 7f5777544700 1 Processor -- start 2023-11-26T07:12:51.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.716+0000 7f5777544700 1 -- start start 2023-11-26T07:12:51.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f5777544700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f57700afc10 0x7f577015e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f5777544700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57700746a0 0x7f5770162120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f5777544700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57700af090 con 0x7f577006fa40 2023-11-26T07:12:51.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f5777544700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577006dda0 con 0x7f57700746a0 2023-11-26T07:12:51.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f5777544700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577006e060 con 0x7f57700afc10 2023-11-26T07:12:51.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f5775ae1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57700746a0 0x7f5770162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f5774adf700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f577006fa40 0x7f5770073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:56998/0 (socket says 172.21.15.83:56998) 2023-11-26T07:12:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f5774adf700 1 -- 172.21.15.83:0/1422363766 learned_addr learned my addr 172.21.15.83:0/1422363766 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f57752e0700 1 --2- 172.21.15.83:0/1422363766 >> v2:172.21.15.203:3300/0 conn(0x7f57700afc10 0x7f577015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f5775ae1700 1 -- 172.21.15.83:0/1422363766 >> v2:172.21.15.203:3300/0 conn(0x7f57700afc10 msgr2=0x7f577015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f5775ae1700 1 --2- 172.21.15.83:0/1422363766 >> v2:172.21.15.203:3300/0 conn(0x7f57700afc10 0x7f577015e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.717+0000 7f5775ae1700 1 -- 172.21.15.83:0/1422363766 >> v1:172.21.15.83:6789/0 conn(0x7f577006fa40 legacy=0x7f5770073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5767fff700 1 -- 172.21.15.83:0/1422363766 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960846172 0 0) 0x7f57700af090 con 0x7f577006fa40 2023-11-26T07:12:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5775ae1700 1 -- 172.21.15.83:0/1422363766 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577006ec70 con 0x7f57700746a0 2023-11-26T07:12:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5775ae1700 1 --2- 172.21.15.83:0/1422363766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57700746a0 0x7f5770162120 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5770074b20 tx=0x7f576800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7bec06519a8e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5767fff700 1 -- 172.21.15.83:0/1422363766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576800e070 con 0x7f57700746a0 2023-11-26T07:12:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5767fff700 1 -- 172.21.15.83:0/1422363766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576800bbd0 con 0x7f57700746a0 2023-11-26T07:12:51.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5767fff700 1 -- 172.21.15.83:0/1422363766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768012430 con 0x7f57700746a0 2023-11-26T07:12:51.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5777544700 1 -- 172.21.15.83:0/1422363766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57700746a0 msgr2=0x7f5770162120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5777544700 1 --2- 172.21.15.83:0/1422363766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57700746a0 0x7f5770162120 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5770074b20 tx=0x7f576800b330 comp rx=0 tx=0).stop 2023-11-26T07:12:51.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5777544700 1 -- 172.21.15.83:0/1422363766 shutdown_connections 2023-11-26T07:12:51.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5777544700 1 --2- 172.21.15.83:0/1422363766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57700746a0 0x7f5770162120 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5777544700 1 --2- 172.21.15.83:0/1422363766 >> v2:172.21.15.203:3300/0 conn(0x7f57700afc10 0x7f577015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.718+0000 7f5777544700 1 -- 172.21.15.83:0/1422363766 >> 172.21.15.83:0/1422363766 conn(0x7f577006aff0 msgr2=0x7f577006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:51.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.719+0000 7f5777544700 1 -- 172.21.15.83:0/1422363766 shutdown_connections 2023-11-26T07:12:51.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.719+0000 7f5777544700 1 -- 172.21.15.83:0/1422363766 wait complete. 2023-11-26T07:12:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.720+0000 7f5777544700 1 Processor -- start 2023-11-26T07:12:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.720+0000 7f5777544700 1 -- start start 2023-11-26T07:12:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.720+0000 7f5777544700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f577006fa40 0x7f5770155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.720+0000 7f5777544700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57700746a0 0x7f57701f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.720+0000 7f5777544700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57700afc10 0x7f5770158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.720+0000 7f5777544700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577015a890 con 0x7f577006fa40 2023-11-26T07:12:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.720+0000 7f5777544700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577015a520 con 0x7f57700746a0 2023-11-26T07:12:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.720+0000 7f5777544700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57701599f0 con 0x7f57700afc10 2023-11-26T07:12:51.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f57752e0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f577006fa40 0x7f5770155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f57752e0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f577006fa40 0x7f5770155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36962/0 (socket says 172.21.15.83:36962) 2023-11-26T07:12:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f57752e0700 1 -- 172.21.15.83:0/2428627118 learned_addr learned my addr 172.21.15.83:0/2428627118 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f5774adf700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57700746a0 0x7f57701f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f57752e0700 1 -- 172.21.15.83:0/2428627118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57700afc10 msgr2=0x7f5770158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f57752e0700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57700afc10 0x7f5770158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f57752e0700 1 -- 172.21.15.83:0/2428627118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57700746a0 msgr2=0x7f57701f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f57752e0700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57700746a0 0x7f57701f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f57752e0700 1 -- 172.21.15.83:0/2428627118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5768009a70 con 0x7f577006fa40 2023-11-26T07:12:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f5774adf700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57700746a0 0x7f57701f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f57752e0700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f577006fa40 0x7f5770155f60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f576000d800 tx=0x7f576000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f5765ffb700 1 -- 172.21.15.83:0/2428627118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57600125f0 con 0x7f577006fa40 2023-11-26T07:12:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f5765ffb700 1 -- 172.21.15.83:0/2428627118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5760012750 con 0x7f577006fa40 2023-11-26T07:12:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f5777544700 1 -- 172.21.15.83:0/2428627118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5770159cd0 con 0x7f577006fa40 2023-11-26T07:12:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f5765ffb700 1 -- 172.21.15.83:0/2428627118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760012b00 con 0x7f577006fa40 2023-11-26T07:12:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.721+0000 7f5777544700 1 -- 172.21.15.83:0/2428627118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577015adc0 con 0x7f577006fa40 2023-11-26T07:12:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.722+0000 7f5777544700 1 -- 172.21.15.83:0/2428627118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577015af50 con 0x7f577006fa40 2023-11-26T07:12:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.723+0000 7f5765ffb700 1 -- 172.21.15.83:0/2428627118 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f5760012c60 con 0x7f577006fa40 2023-11-26T07:12:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.723+0000 7f5765ffb700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f575406b080 0x7f575406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.723+0000 7f5765ffb700 1 -- 172.21.15.83:0/2428627118 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5760089970 con 0x7f577006fa40 2023-11-26T07:12:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.723+0000 7f5774adf700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f575406b080 0x7f575406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.723+0000 7f5774adf700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f575406b080 0x7f575406d540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f577015a840 tx=0x7f576c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:51.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.728+0000 7f5765ffb700 1 -- 172.21.15.83:0/2428627118 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5760054af0 con 0x7f577006fa40 2023-11-26T07:12:51.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.918+0000 7f5777544700 1 -- 172.21.15.83:0/2428627118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5770068f10 con 0x7f577006fa40 2023-11-26T07:12:51.920 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.919+0000 7f5765ffb700 1 -- 172.21.15.83:0/2428627118 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12845 (secure 0 0 0) 0x7f5760058110 con 0x7f577006fa40 2023-11-26T07:12:51.921 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:51.921 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":41,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:51.169062+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:51.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.921+0000 7f5777544700 1 -- 172.21.15.83:0/2428627118 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f575406b080 msgr2=0x7f575406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.921+0000 7f5777544700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f575406b080 0x7f575406d540 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f577015a840 tx=0x7f576c009040 comp rx=0 tx=0).stop 2023-11-26T07:12:51.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.921+0000 7f5777544700 1 -- 172.21.15.83:0/2428627118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f577006fa40 msgr2=0x7f5770155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.921+0000 7f5777544700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f577006fa40 0x7f5770155f60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f576000d800 tx=0x7f576000dbc0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.921+0000 7f5777544700 1 -- 172.21.15.83:0/2428627118 shutdown_connections 2023-11-26T07:12:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.922+0000 7f5777544700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f575406b080 0x7f575406d540 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.922+0000 7f5777544700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f57700afc10 0x7f5770158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.922+0000 7f5777544700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f577006fa40 0x7f5770155f60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.922+0000 7f5777544700 1 --2- 172.21.15.83:0/2428627118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57700746a0 0x7f57701f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.922+0000 7f5777544700 1 -- 172.21.15.83:0/2428627118 >> 172.21.15.83:0/2428627118 conn(0x7f577006aff0 msgr2=0x7f57700aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.922+0000 7f5777544700 1 -- 172.21.15.83:0/2428627118 shutdown_connections 2023-11-26T07:12:51.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:51.922+0000 7f5777544700 1 -- 172.21.15.83:0/2428627118 wait complete. 2023-11-26T07:12:51.935 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:12:51.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:12:52.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.056+0000 7f070efcd700 1 Processor -- start 2023-11-26T07:12:52.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.056+0000 7f070efcd700 1 -- start start 2023-11-26T07:12:52.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.057+0000 7f070efcd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f07080746a0 0x7f07080b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.057+0000 7f070efcd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07080afc90 0x7f070806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.057+0000 7f070efcd700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07080b3c60 con 0x7f070815e160 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.057+0000 7f070efcd700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07080b4500 con 0x7f07080746a0 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.057+0000 7f070efcd700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07080b58c0 con 0x7f07080afc90 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.057+0000 7f070cd69700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f070815e160 0x7f0708073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57010/0 (socket says 172.21.15.83:57010) 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.057+0000 7f070cd69700 1 -- 172.21.15.83:0/953371271 learned_addr learned my addr 172.21.15.83:0/953371271 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f0707fff700 1 --2- 172.21.15.83:0/953371271 >> v2:172.21.15.203:3300/0 conn(0x7f07080746a0 0x7f07080b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575834027 0 0) 0x7f07080b3c60 con 0x7f070815e160 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f070d56a700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07080afc90 0x7f070806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06f80035b0 con 0x7f070815e160 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 747131520 0 0) 0x7f06f80035b0 con 0x7f070815e160 2023-11-26T07:12:52.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07080afc90 msgr2=0x7f070806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07080afc90 0x7f070806d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 >> v2:172.21.15.203:3300/0 conn(0x7f07080746a0 msgr2=0x7f07080b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 --2- 172.21.15.83:0/953371271 >> v2:172.21.15.203:3300/0 conn(0x7f07080746a0 0x7f07080b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07080b0120 con 0x7f070815e160 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f06fc001670 con 0x7f070815e160 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 >> v1:172.21.15.83:6789/0 conn(0x7f070815e160 legacy=0x7f0708073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07080afc90 0x7f06f8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f8003f80 0x7f06f8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f8004d90 0x7f06f8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07080b4500 con 0x7f06f8003f80 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07080b58c0 con 0x7f06f8004d90 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f800ba10 con 0x7f07080afc90 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f0707fff700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f8003f80 0x7f06f8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f070d56a700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f8004d90 0x7f06f8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f070cd69700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07080afc90 0x7f06f8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f0707fff700 1 -- 172.21.15.83:0/953371271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07080afc90 msgr2=0x7f06f8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f0707fff700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07080afc90 0x7f06f8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f0707fff700 1 -- 172.21.15.83:0/953371271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f8004d90 msgr2=0x7f06f8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.058+0000 7f0707fff700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06f8004d90 0x7f06f8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f0707fff700 1 -- 172.21.15.83:0/953371271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f80035b0 con 0x7f06f8003f80 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f0707fff700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f8003f80 0x7f06f8004850 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009b60 tx=0x7f06f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e1b96c6b232a78 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f400e660 con 0x7f06f8003f80 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f400e7c0 con 0x7f06f8003f80 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f07077fe700 1 -- 172.21.15.83:0/953371271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f400eb30 con 0x7f06f8003f80 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f070efcd700 1 -- 172.21.15.83:0/953371271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f8003f80 msgr2=0x7f06f8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f070efcd700 1 --2- 172.21.15.83:0/953371271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f8003f80 0x7f06f8004850 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009b60 tx=0x7f06f8005880 comp rx=0 tx=0).stop 2023-11-26T07:12:52.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f070d56a700 1 -- 172.21.15.83:0/953371271 reap_dead start 2023-11-26T07:12:52.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f070efcd700 1 -- 172.21.15.83:0/953371271 shutdown_connections 2023-11-26T07:12:52.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f070efcd700 1 -- 172.21.15.83:0/953371271 >> 172.21.15.83:0/953371271 conn(0x7f07080d0bf0 msgr2=0x7f07080d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:52.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f070efcd700 1 -- 172.21.15.83:0/953371271 shutdown_connections 2023-11-26T07:12:52.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.059+0000 7f070efcd700 1 -- 172.21.15.83:0/953371271 wait complete. 2023-11-26T07:12:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.061+0000 7f070efcd700 1 Processor -- start 2023-11-26T07:12:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.061+0000 7f070efcd700 1 -- start start 2023-11-26T07:12:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.061+0000 7f070efcd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06f8003f80 0x7f0708143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.061+0000 7f070efcd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07081f54d0 0x7f07081f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070efcd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0708160ba0 0x7f0708161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070efcd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0708160190 con 0x7f0708160ba0 2023-11-26T07:12:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070efcd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07081f4d60 con 0x7f07081f54d0 2023-11-26T07:12:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070efcd700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07081f5040 con 0x7f06f8003f80 2023-11-26T07:12:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070d56a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0708160ba0 0x7f0708161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070d56a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0708160ba0 0x7f0708161050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36986/0 (socket says 172.21.15.83:36986) 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070d56a700 1 -- 172.21.15.83:0/1229593627 learned_addr learned my addr 172.21.15.83:0/1229593627 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f0707fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07081f54d0 0x7f07081f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070cd69700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06f8003f80 0x7f0708143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070d56a700 1 -- 172.21.15.83:0/1229593627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06f8003f80 msgr2=0x7f0708143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070d56a700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06f8003f80 0x7f0708143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070d56a700 1 -- 172.21.15.83:0/1229593627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07081f54d0 msgr2=0x7f07081f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070d56a700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07081f54d0 0x7f07081f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070d56a700 1 -- 172.21.15.83:0/1229593627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07081603a0 con 0x7f0708160ba0 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070d56a700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0708160ba0 0x7f0708161050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f07080b5070 tx=0x7f0700011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f07057fa700 1 -- 172.21.15.83:0/1229593627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000e040 con 0x7f0708160ba0 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f07057fa700 1 -- 172.21.15.83:0/1229593627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070000f040 con 0x7f0708160ba0 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070efcd700 1 -- 172.21.15.83:0/1229593627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0708160680 con 0x7f0708160ba0 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f07057fa700 1 -- 172.21.15.83:0/1229593627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000fc40 con 0x7f0708160ba0 2023-11-26T07:12:52.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.062+0000 7f070efcd700 1 -- 172.21.15.83:0/1229593627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07080add40 con 0x7f0708160ba0 2023-11-26T07:12:52.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.063+0000 7f070efcd700 1 -- 172.21.15.83:0/1229593627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07081f5200 con 0x7f0708160ba0 2023-11-26T07:12:52.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.064+0000 7f07057fa700 1 -- 172.21.15.83:0/1229593627 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0700022460 con 0x7f0708160ba0 2023-11-26T07:12:52.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.064+0000 7f07057fa700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f06ec06b080 0x7f06ec06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.064+0000 7f07057fa700 1 -- 172.21.15.83:0/1229593627 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0700089aa0 con 0x7f0708160ba0 2023-11-26T07:12:52.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.064+0000 7f070cd69700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f06ec06b080 0x7f06ec06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.065+0000 7f070cd69700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f06ec06b080 0x7f06ec06d540 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f06fc000c00 tx=0x7f06fc007980 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:52.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.069+0000 7f07057fa700 1 -- 172.21.15.83:0/1229593627 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0700054b10 con 0x7f0708160ba0 2023-11-26T07:12:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.259+0000 7f070efcd700 1 -- 172.21.15.83:0/1229593627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07080d0fd0 con 0x7f0708160ba0 2023-11-26T07:12:52.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.259+0000 7f07057fa700 1 -- 172.21.15.83:0/1229593627 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0700058130 con 0x7f0708160ba0 2023-11-26T07:12:52.261 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:12:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.261+0000 7f070efcd700 1 -- 172.21.15.83:0/1229593627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f06ec06b080 msgr2=0x7f06ec06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.261+0000 7f070efcd700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f06ec06b080 0x7f06ec06d540 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f06fc000c00 tx=0x7f06fc007980 comp rx=0 tx=0).stop 2023-11-26T07:12:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.261+0000 7f070efcd700 1 -- 172.21.15.83:0/1229593627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0708160ba0 msgr2=0x7f0708161050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.261+0000 7f070efcd700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0708160ba0 0x7f0708161050 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f07080b5070 tx=0x7f0700011ed0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.261+0000 7f070efcd700 1 -- 172.21.15.83:0/1229593627 shutdown_connections 2023-11-26T07:12:52.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.261+0000 7f070efcd700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0708160ba0 0x7f0708161050 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.261+0000 7f070efcd700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f06ec06b080 0x7f06ec06d540 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.261+0000 7f070efcd700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f06f8003f80 0x7f0708143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.261+0000 7f070efcd700 1 --2- 172.21.15.83:0/1229593627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07081f54d0 0x7f07081f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.261+0000 7f070efcd700 1 -- 172.21.15.83:0/1229593627 >> 172.21.15.83:0/1229593627 conn(0x7f07080d0bf0 msgr2=0x7f0708073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:52.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.262+0000 7f070efcd700 1 -- 172.21.15.83:0/1229593627 shutdown_connections 2023-11-26T07:12:52.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.262+0000 7f070efcd700 1 -- 172.21.15.83:0/1229593627 wait complete. 2023-11-26T07:12:52.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:12:52.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.394+0000 7f8507de2700 1 Processor -- start 2023-11-26T07:12:52.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.394+0000 7f8507de2700 1 -- start start 2023-11-26T07:12:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8507de2700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f85000b3370 0x7f85001604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8507de2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85000b22d0 con 0x7f85000d1c70 2023-11-26T07:12:52.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8507de2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85000b25b0 con 0x7f850015bdd0 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8507de2700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85000b2f50 con 0x7f85000b3370 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f850637f700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f85000d1c70 0x7f8500165ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57036/0 (socket says 172.21.15.83:57036) 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f850637f700 1 -- 172.21.15.83:0/1229184074 learned_addr learned my addr 172.21.15.83:0/1229184074 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f850537d700 1 --2- 172.21.15.83:0/1229184074 >> v2:172.21.15.203:3300/0 conn(0x7f85000b3370 0x7f85001604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304975661 0 0) 0x7f85000b22d0 con 0x7f85000d1c70 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84ec0035b0 con 0x7f85000d1c70 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2992082636 0 0) 0x7f84ec0035b0 con 0x7f85000d1c70 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 >> v2:172.21.15.203:3300/0 conn(0x7f85000b3370 msgr2=0x7f85001604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8504b7c700 1 --2- 172.21.15.83:0/1229184074 >> v2:172.21.15.203:3300/0 conn(0x7f85000b3370 0x7f85001604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 >> v1:172.21.15.203:6789/0 conn(0x7f850015bdd0 legacy=0x7f85000b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85000b2890 con 0x7f85000d1c70 2023-11-26T07:12:52.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.395+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f84f80019f0 con 0x7f85000d1c70 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 >> v1:172.21.15.83:6789/0 conn(0x7f85000d1c70 legacy=0x7f8500165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f8504b7c700 1 --2- 172.21.15.83:0/1229184074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84ec003a00 0x7f84ec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f8504b7c700 1 --2- 172.21.15.83:0/1229184074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84ec004850 0x7f84ec004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f8504b7c700 1 --2- 172.21.15.83:0/1229184074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ec005210 0x7f84ec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85000b2f50 con 0x7f84ec005210 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec00c0b0 con 0x7f84ec003a00 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec00c270 con 0x7f84ec004850 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f850637f700 1 --2- 172.21.15.83:0/1229184074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ec005210 0x7f84ec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f8505b7e700 1 --2- 172.21.15.83:0/1229184074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84ec003a00 0x7f84ec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f850537d700 1 --2- 172.21.15.83:0/1229184074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84ec004850 0x7f84ec004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f850637f700 1 -- 172.21.15.83:0/1229184074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84ec004850 msgr2=0x7f84ec004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:52.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f850637f700 1 --2- 172.21.15.83:0/1229184074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84ec004850 0x7f84ec004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f850637f700 1 -- 172.21.15.83:0/1229184074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84ec003a00 msgr2=0x7f84ec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f850637f700 1 --2- 172.21.15.83:0/1229184074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84ec003a00 0x7f84ec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f850637f700 1 -- 172.21.15.83:0/1229184074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec0035b0 con 0x7f84ec005210 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f850637f700 1 -- 172.21.15.83:0/1229184074 reap_dead start 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f850637f700 1 --2- 172.21.15.83:0/1229184074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ec005210 0x7f84ec00ba10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f84f800a830 tx=0x7f84ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1765ce0cdbde9069 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f800a2f0 con 0x7f84ec005210 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f8012620 con 0x7f84ec005210 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.396+0000 7f8504b7c700 1 -- 172.21.15.83:0/1229184074 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f80129d0 con 0x7f84ec005210 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.397+0000 7f8507de2700 1 -- 172.21.15.83:0/1229184074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ec005210 msgr2=0x7f84ec00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.397+0000 7f8507de2700 1 --2- 172.21.15.83:0/1229184074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ec005210 0x7f84ec00ba10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f84f800a830 tx=0x7f84ec005880 comp rx=0 tx=0).stop 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.397+0000 7f8507de2700 1 -- 172.21.15.83:0/1229184074 shutdown_connections 2023-11-26T07:12:52.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.397+0000 7f8507de2700 1 --2- 172.21.15.83:0/1229184074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84ec005210 0x7f84ec00ba10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.397+0000 7f8507de2700 1 -- 172.21.15.83:0/1229184074 >> 172.21.15.83:0/1229184074 conn(0x7f85000731a0 msgr2=0x7f850009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.397+0000 7f8507de2700 1 -- 172.21.15.83:0/1229184074 shutdown_connections 2023-11-26T07:12:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.397+0000 7f8507de2700 1 -- 172.21.15.83:0/1229184074 wait complete. 2023-11-26T07:12:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.398+0000 7f8507de2700 1 Processor -- start 2023-11-26T07:12:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.398+0000 7f8507de2700 1 -- start start 2023-11-26T07:12:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f8507de2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84ec005210 0x7f85000c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f8507de2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85000c9360 0x7f85000b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f8507de2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85000c97e0 0x7f85000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f8507de2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85000b6880 con 0x7f85000c97e0 2023-11-26T07:12:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f8507de2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85000b6b20 con 0x7f85000c9360 2023-11-26T07:12:52.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f8507de2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85000b6e00 con 0x7f84ec005210 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f850637f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85000c97e0 0x7f85000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f850637f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85000c97e0 0x7f85000b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37008/0 (socket says 172.21.15.83:37008) 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f850637f700 1 -- 172.21.15.83:0/10646362 learned_addr learned my addr 172.21.15.83:0/10646362 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f850537d700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85000c9360 0x7f85000b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f8505b7e700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84ec005210 0x7f85000c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f850637f700 1 -- 172.21.15.83:0/10646362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84ec005210 msgr2=0x7f85000c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f850637f700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84ec005210 0x7f85000c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f850637f700 1 -- 172.21.15.83:0/10646362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85000c9360 msgr2=0x7f85000b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f850637f700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85000c9360 0x7f85000b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f850637f700 1 -- 172.21.15.83:0/10646362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85000b70e0 con 0x7f85000c97e0 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.399+0000 7f850637f700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85000c97e0 0x7f85000b4e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8500071560 tx=0x7f84f8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.400+0000 7f84f67fc700 1 -- 172.21.15.83:0/10646362 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8024070 con 0x7f85000c97e0 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.400+0000 7f84f67fc700 1 -- 172.21.15.83:0/10646362 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f800a210 con 0x7f85000c97e0 2023-11-26T07:12:52.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.400+0000 7f8507de2700 1 -- 172.21.15.83:0/10646362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85000b73c0 con 0x7f85000c97e0 2023-11-26T07:12:52.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.400+0000 7f84f67fc700 1 -- 172.21.15.83:0/10646362 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f80019f0 con 0x7f85000c97e0 2023-11-26T07:12:52.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.400+0000 7f8507de2700 1 -- 172.21.15.83:0/10646362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8500150690 con 0x7f85000c97e0 2023-11-26T07:12:52.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.400+0000 7f8507de2700 1 -- 172.21.15.83:0/10646362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85000c9dc0 con 0x7f85000c97e0 2023-11-26T07:12:52.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.401+0000 7f84f67fc700 1 -- 172.21.15.83:0/10646362 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f84f801d420 con 0x7f85000c97e0 2023-11-26T07:12:52.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.401+0000 7f84f67fc700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f84e406b080 0x7f84e406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.402+0000 7f8505b7e700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f84e406b080 0x7f84e406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.402+0000 7f84f67fc700 1 -- 172.21.15.83:0/10646362 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f84f808a880 con 0x7f85000c97e0 2023-11-26T07:12:52.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.402+0000 7f8505b7e700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f84e406b080 0x7f84e406d540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f84f0007fd0 tx=0x7f84f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:52.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.406+0000 7f84f67fc700 1 -- 172.21.15.83:0/10646362 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84f8055a90 con 0x7f85000c97e0 2023-11-26T07:12:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.599+0000 7f8507de2700 1 -- 172.21.15.83:0/10646362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8500067f60 con 0x7f85000c97e0 2023-11-26T07:12:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.599+0000 7f84f67fc700 1 -- 172.21.15.83:0/10646362 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7f84f80590b0 con 0x7f85000c97e0 2023-11-26T07:12:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:52.601 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:12:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:12:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2023-11-26T07:12:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2023-11-26T07:12:52.602 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:12:52.602 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:12:52.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 -- 172.21.15.83:0/10646362 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f84e406b080 msgr2=0x7f84e406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f84e406b080 0x7f84e406d540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f84f0007fd0 tx=0x7f84f0009040 comp rx=0 tx=0).stop 2023-11-26T07:12:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 -- 172.21.15.83:0/10646362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85000c97e0 msgr2=0x7f85000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85000c97e0 0x7f85000b4e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8500071560 tx=0x7f84f8001d80 comp rx=0 tx=0).stop 2023-11-26T07:12:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 -- 172.21.15.83:0/10646362 shutdown_connections 2023-11-26T07:12:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85000c97e0 0x7f85000b4e10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f84e406b080 0x7f84e406d540 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84ec005210 0x7f85000c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 --2- 172.21.15.83:0/10646362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85000c9360 0x7f85000b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 -- 172.21.15.83:0/10646362 >> 172.21.15.83:0/10646362 conn(0x7f85000731a0 msgr2=0x7f850015f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 -- 172.21.15.83:0/10646362 shutdown_connections 2023-11-26T07:12:52.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.602+0000 7f8507de2700 1 -- 172.21.15.83:0/10646362 wait complete. 2023-11-26T07:12:52.615 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:12:52.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:12:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.738+0000 7f3122585700 1 Processor -- start 2023-11-26T07:12:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.738+0000 7f3122585700 1 -- start start 2023-11-26T07:12:52.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f3122585700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311c0aefc0 0x7f311c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f3122585700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f311c0b19c0 0x7f311c0ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f3122585700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c0b0e50 con 0x7f311c0af3a0 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f3122585700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c15ea90 con 0x7f311c0b19c0 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f3122585700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0b02b0 con 0x7f311c0aefc0 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f311c0af3a0 0x7f311c0ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57052/0 (socket says 172.21.15.83:57052) 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311b7fe700 1 -- 172.21.15.83:0/2561994015 learned_addr learned my addr 172.21.15.83:0/2561994015 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 954236959 0 0) 0x7f311c0b0e50 con 0x7f311c0af3a0 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f3120b22700 1 --2- 172.21.15.83:0/2561994015 >> v2:172.21.15.203:3300/0 conn(0x7f311c0b19c0 0x7f311c0ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31040035b0 con 0x7f311c0af3a0 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311bfff700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311c0aefc0 0x7f311c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 56782588 0 0) 0x7f31040035b0 con 0x7f311c0af3a0 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311c0aefc0 msgr2=0x7f311c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311c0aefc0 0x7f311c0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 >> v2:172.21.15.203:3300/0 conn(0x7f311c0b19c0 msgr2=0x7f311c0ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 --2- 172.21.15.83:0/2561994015 >> v2:172.21.15.203:3300/0 conn(0x7f311c0b19c0 0x7f311c0ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0b0590 con 0x7f311c0af3a0 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f310c001670 con 0x7f311c0af3a0 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 >> v1:172.21.15.83:6789/0 conn(0x7f311c0af3a0 legacy=0x7f311c0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3104003a00 0x7f3104003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3104004850 0x7f3104004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311c0aefc0 0x7f3104005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c15ea90 con 0x7f3104003a00 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0b02b0 con 0x7f3104004850 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310400ba10 con 0x7f311c0aefc0 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.739+0000 7f311bfff700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3104003a00 0x7f3104003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f311b7fe700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3104004850 0x7f3104004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f3120b22700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311c0aefc0 0x7f3104005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f311bfff700 1 -- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311c0aefc0 msgr2=0x7f3104005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f311bfff700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f311c0aefc0 0x7f3104005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f311bfff700 1 -- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3104004850 msgr2=0x7f3104004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f311bfff700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3104004850 0x7f3104004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f311bfff700 1 -- 172.21.15.83:0/2561994015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31040035b0 con 0x7f3104003a00 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f311b7fe700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3104004850 0x7f3104004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f311bfff700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3104003a00 0x7f3104003e20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3110009b60 tx=0x7f3104005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31249fc9f25b4419 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311000e6c0 con 0x7f3104003a00 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311000e820 con 0x7f3104003a00 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f311affd700 1 -- 172.21.15.83:0/2561994015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311000eb90 con 0x7f3104003a00 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f3122585700 1 -- 172.21.15.83:0/2561994015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3104003a00 msgr2=0x7f3104003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f3122585700 1 --2- 172.21.15.83:0/2561994015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3104003a00 0x7f3104003e20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3110009b60 tx=0x7f3104005880 comp rx=0 tx=0).stop 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.740+0000 7f3120b22700 1 -- 172.21.15.83:0/2561994015 reap_dead start 2023-11-26T07:12:52.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.741+0000 7f3122585700 1 -- 172.21.15.83:0/2561994015 shutdown_connections 2023-11-26T07:12:52.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.741+0000 7f3122585700 1 -- 172.21.15.83:0/2561994015 >> 172.21.15.83:0/2561994015 conn(0x7f311c072710 msgr2=0x7f311c072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:52.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.741+0000 7f3122585700 1 -- 172.21.15.83:0/2561994015 shutdown_connections 2023-11-26T07:12:52.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.741+0000 7f3122585700 1 -- 172.21.15.83:0/2561994015 wait complete. 2023-11-26T07:12:52.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.742+0000 7f3122585700 1 Processor -- start 2023-11-26T07:12:52.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.742+0000 7f3122585700 1 -- start start 2023-11-26T07:12:52.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3122585700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3104003a00 0x7f311c150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3122585700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311c1f1100 0x7f311c0c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3122585700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0b4f50 0x7f311c0b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3122585700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0b5910 con 0x7f311c0b4f50 2023-11-26T07:12:52.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3122585700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c1512b0 con 0x7f311c1f1100 2023-11-26T07:12:52.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3122585700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c14e650 con 0x7f3104003a00 2023-11-26T07:12:52.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3120b22700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0b4f50 0x7f311c0b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3120b22700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0b4f50 0x7f311c0b53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37026/0 (socket says 172.21.15.83:37026) 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3120b22700 1 -- 172.21.15.83:0/1930600309 learned_addr learned my addr 172.21.15.83:0/1930600309 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f311bfff700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3104003a00 0x7f311c150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f311b7fe700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311c1f1100 0x7f311c0c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3120b22700 1 -- 172.21.15.83:0/1930600309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3104003a00 msgr2=0x7f311c150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3120b22700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3104003a00 0x7f311c150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3120b22700 1 -- 172.21.15.83:0/1930600309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311c1f1100 msgr2=0x7f311c0c7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3120b22700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311c1f1100 0x7f311c0c7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3120b22700 1 -- 172.21.15.83:0/1930600309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c14e930 con 0x7f311c0b4f50 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3120b22700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0b4f50 0x7f311c0b53d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f311c071010 tx=0x7f31140111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3118ff9700 1 -- 172.21.15.83:0/1930600309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311400e070 con 0x7f311c0b4f50 2023-11-26T07:12:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3118ff9700 1 -- 172.21.15.83:0/1930600309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3114003cf0 con 0x7f311c0b4f50 2023-11-26T07:12:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3122585700 1 -- 172.21.15.83:0/1930600309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c14ec10 con 0x7f311c0b4f50 2023-11-26T07:12:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.743+0000 7f3118ff9700 1 -- 172.21.15.83:0/1930600309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31140117a0 con 0x7f311c0b4f50 2023-11-26T07:12:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.744+0000 7f3122585700 1 -- 172.21.15.83:0/1930600309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c14f1d0 con 0x7f311c0b4f50 2023-11-26T07:12:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.744+0000 7f3122585700 1 -- 172.21.15.83:0/1930600309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c14f390 con 0x7f311c0b4f50 2023-11-26T07:12:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.745+0000 7f3118ff9700 1 -- 172.21.15.83:0/1930600309 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3114011900 con 0x7f311c0b4f50 2023-11-26T07:12:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.745+0000 7f3118ff9700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f310806b080 0x7f310806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.745+0000 7f311bfff700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f310806b080 0x7f310806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.745+0000 7f3118ff9700 1 -- 172.21.15.83:0/1930600309 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3114089990 con 0x7f311c0b4f50 2023-11-26T07:12:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.746+0000 7f311bfff700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f310806b080 0x7f310806d540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f31100090b0 tx=0x7f311001f040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:52.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.750+0000 7f3118ff9700 1 -- 172.21.15.83:0/1930600309 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3114054a00 con 0x7f311c0b4f50 2023-11-26T07:12:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:52.940+0000 7f3122585700 1 -- 172.21.15.83:0/1930600309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f311c062340 con 0x7f311c0b4f50 2023-11-26T07:12:53.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.507+0000 7f3118ff9700 1 -- 172.21.15.83:0/1930600309 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3114058020 con 0x7f311c0b4f50 2023-11-26T07:12:53.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 -- 172.21.15.83:0/1930600309 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f310806b080 msgr2=0x7f310806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:53.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f310806b080 0x7f310806d540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f31100090b0 tx=0x7f311001f040 comp rx=0 tx=0).stop 2023-11-26T07:12:53.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 -- 172.21.15.83:0/1930600309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0b4f50 msgr2=0x7f311c0b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0b4f50 0x7f311c0b53d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f311c071010 tx=0x7f31140111d0 comp rx=0 tx=0).stop 2023-11-26T07:12:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 -- 172.21.15.83:0/1930600309 shutdown_connections 2023-11-26T07:12:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0b4f50 0x7f311c0b53d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:53.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f310806b080 0x7f310806d540 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:53.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3104003a00 0x7f311c150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:53.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 --2- 172.21.15.83:0/1930600309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f311c1f1100 0x7f311c0c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:53.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 -- 172.21.15.83:0/1930600309 >> 172.21.15.83:0/1930600309 conn(0x7f311c072710 msgr2=0x7f311c0cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 -- 172.21.15.83:0/1930600309 shutdown_connections 2023-11-26T07:12:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.510+0000 7f3122585700 1 -- 172.21.15.83:0/1930600309 wait complete. 2023-11-26T07:12:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2023-11-26T07:12:53.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:12:53.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.656+0000 7f6fcb06d700 1 Processor -- start 2023-11-26T07:12:53.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.656+0000 7f6fcb06d700 1 -- start start 2023-11-26T07:12:53.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fcb06d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6fc415a620 0x7f6fc40ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:53.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fcb06d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc40cc540 0x7f6fc406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:53.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fcb06d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc40b2f20 con 0x7f6fc406eb20 2023-11-26T07:12:53.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fcb06d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc406efb0 con 0x7f6fc415a620 2023-11-26T07:12:53.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fcb06d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40ce690 con 0x7f6fc40cc540 2023-11-26T07:12:53.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc960a700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6fc406eb20 0x7f6fc406d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57072/0 (socket says 172.21.15.83:57072) 2023-11-26T07:12:53.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc960a700 1 -- 172.21.15.83:0/930221656 learned_addr learned my addr 172.21.15.83:0/930221656 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:53.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc8e09700 1 --2- 172.21.15.83:0/930221656 >> v2:172.21.15.203:3300/0 conn(0x7f6fc415a620 0x7f6fc40ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:53.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc3fff700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc40cc540 0x7f6fc406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:53.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084891956 0 0) 0x7f6fc40b2f20 con 0x7f6fc406eb20 2023-11-26T07:12:53.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb40046c0 con 0x7f6fc406eb20 2023-11-26T07:12:53.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1096087406 0 0) 0x7f6fb40046c0 con 0x7f6fc406eb20 2023-11-26T07:12:53.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc40cc540 msgr2=0x7f6fc406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:53.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc40cc540 0x7f6fc406e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:53.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 >> v2:172.21.15.203:3300/0 conn(0x7f6fc415a620 msgr2=0x7f6fc40ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:53.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 --2- 172.21.15.83:0/930221656 >> v2:172.21.15.203:3300/0 conn(0x7f6fc415a620 0x7f6fc40ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:53.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc415aa00 con 0x7f6fc406eb20 2023-11-26T07:12:53.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f6fbc0019f0 con 0x7f6fc406eb20 2023-11-26T07:12:53.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 >> v1:172.21.15.83:6789/0 conn(0x7f6fc406eb20 legacy=0x7f6fc406d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:53.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc40cc540 0x7f6fb4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fb4005090 0x7f6fb4005960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb4005ea0 0x7f6fb4006320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc406efb0 con 0x7f6fb4005ea0 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40ce690 con 0x7f6fb4005090 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb400cb20 con 0x7f6fc40cc540 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.657+0000 7f6fc960a700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb4005ea0 0x7f6fb4006320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fc3fff700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fb4005090 0x7f6fb4005960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fc8e09700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc40cc540 0x7f6fb4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fc960a700 1 -- 172.21.15.83:0/930221656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc40cc540 msgr2=0x7f6fb4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fc960a700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc40cc540 0x7f6fb4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fc960a700 1 -- 172.21.15.83:0/930221656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fb4005090 msgr2=0x7f6fb4005960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:53.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fc960a700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fb4005090 0x7f6fb4005960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fc960a700 1 -- 172.21.15.83:0/930221656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40046c0 con 0x7f6fb4005ea0 2023-11-26T07:12:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fc960a700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb4005ea0 0x7f6fb4006320 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00a8d0 tx=0x7f6fb4006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f60616dc3c4560 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00a340 con 0x7f6fb4005ea0 2023-11-26T07:12:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc012600 con 0x7f6fb4005ea0 2023-11-26T07:12:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fc37fe700 1 -- 172.21.15.83:0/930221656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc0129b0 con 0x7f6fb4005ea0 2023-11-26T07:12:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fcb06d700 1 -- 172.21.15.83:0/930221656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb4005ea0 msgr2=0x7f6fb4006320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.658+0000 7f6fcb06d700 1 --2- 172.21.15.83:0/930221656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fb4005ea0 0x7f6fb4006320 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00a8d0 tx=0x7f6fb4006990 comp rx=0 tx=0).stop 2023-11-26T07:12:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.659+0000 7f6fc960a700 1 -- 172.21.15.83:0/930221656 reap_dead start 2023-11-26T07:12:53.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.659+0000 7f6fcb06d700 1 -- 172.21.15.83:0/930221656 shutdown_connections 2023-11-26T07:12:53.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.659+0000 7f6fcb06d700 1 -- 172.21.15.83:0/930221656 >> 172.21.15.83:0/930221656 conn(0x7f6fc40b2050 msgr2=0x7f6fc40b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:53.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.659+0000 7f6fcb06d700 1 -- 172.21.15.83:0/930221656 shutdown_connections 2023-11-26T07:12:53.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.659+0000 7f6fcb06d700 1 -- 172.21.15.83:0/930221656 wait complete. 2023-11-26T07:12:53.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.660+0000 7f6fcb06d700 1 Processor -- start 2023-11-26T07:12:53.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.660+0000 7f6fcb06d700 1 -- start start 2023-11-26T07:12:53.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fcb06d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc414c650 0x7f6fc415d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fcb06d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc414ca30 0x7f6fc415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fcb06d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fc4155ff0 0x7f6fc4156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fcb06d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc41569b0 con 0x7f6fc414c650 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fcb06d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc415c780 con 0x7f6fc4155ff0 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fcb06d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40ad870 con 0x7f6fc414ca30 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc8e09700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc414c650 0x7f6fc415d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc8e09700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc414c650 0x7f6fc415d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37040/0 (socket says 172.21.15.83:37040) 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc8e09700 1 -- 172.21.15.83:0/1204835892 learned_addr learned my addr 172.21.15.83:0/1204835892 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc960a700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fc4155ff0 0x7f6fc4156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc3fff700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc414ca30 0x7f6fc415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc8e09700 1 -- 172.21.15.83:0/1204835892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc414ca30 msgr2=0x7f6fc415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:53.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc8e09700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc414ca30 0x7f6fc415c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:53.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc8e09700 1 -- 172.21.15.83:0/1204835892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fc4155ff0 msgr2=0x7f6fc4156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:53.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc8e09700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fc4155ff0 0x7f6fc4156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:53.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc8e09700 1 -- 172.21.15.83:0/1204835892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc40ae110 con 0x7f6fc414c650 2023-11-26T07:12:53.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc960a700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fc4155ff0 0x7f6fc4156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:53.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc8e09700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc414c650 0x7f6fc415d660 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8011aa0 tx=0x7f6fb8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:53.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc17fa700 1 -- 172.21.15.83:0/1204835892 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb800f9e0 con 0x7f6fc414c650 2023-11-26T07:12:53.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc17fa700 1 -- 172.21.15.83:0/1204835892 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb800fb40 con 0x7f6fc414c650 2023-11-26T07:12:53.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fcb06d700 1 -- 172.21.15.83:0/1204835892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc415c9b0 con 0x7f6fc414c650 2023-11-26T07:12:53.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fc17fa700 1 -- 172.21.15.83:0/1204835892 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8014440 con 0x7f6fc414c650 2023-11-26T07:12:53.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.661+0000 7f6fcb06d700 1 -- 172.21.15.83:0/1204835892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc40adda0 con 0x7f6fc414c650 2023-11-26T07:12:53.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.662+0000 7f6fcb06d700 1 -- 172.21.15.83:0/1204835892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc40ae3f0 con 0x7f6fc414c650 2023-11-26T07:12:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.663+0000 7f6fc17fa700 1 -- 172.21.15.83:0/1204835892 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6fb80145a0 con 0x7f6fc414c650 2023-11-26T07:12:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.663+0000 7f6fc17fa700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6fa806b080 0x7f6fa806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.663+0000 7f6fc17fa700 1 -- 172.21.15.83:0/1204835892 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6fb8089fc0 con 0x7f6fc414c650 2023-11-26T07:12:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.663+0000 7f6fc3fff700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6fa806b080 0x7f6fa806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.664+0000 7f6fc3fff700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6fa806b080 0x7f6fa806d540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6fc41f6e10 tx=0x7f6fb0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:53.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.668+0000 7f6fc17fa700 1 -- 172.21.15.83:0/1204835892 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fb80bce90 con 0x7f6fc414c650 2023-11-26T07:12:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:53.857+0000 7f6fcb06d700 1 -- 172.21.15.83:0/1204835892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6fc4068d90 con 0x7f6fc414c650 2023-11-26T07:12:54.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.513+0000 7f6fc17fa700 1 -- 172.21.15.83:0/1204835892 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6fb8054fd0 con 0x7f6fc414c650 2023-11-26T07:12:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 -- 172.21.15.83:0/1204835892 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6fa806b080 msgr2=0x7f6fa806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6fa806b080 0x7f6fa806d540 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6fc41f6e10 tx=0x7f6fb0008040 comp rx=0 tx=0).stop 2023-11-26T07:12:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 -- 172.21.15.83:0/1204835892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc414c650 msgr2=0x7f6fc415d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc414c650 0x7f6fc415d660 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8011aa0 tx=0x7f6fb8011e60 comp rx=0 tx=0).stop 2023-11-26T07:12:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 -- 172.21.15.83:0/1204835892 shutdown_connections 2023-11-26T07:12:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fc4155ff0 0x7f6fc4156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fc414c650 0x7f6fc415d660 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6fa806b080 0x7f6fa806d540 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 --2- 172.21.15.83:0/1204835892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fc414ca30 0x7f6fc415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:54.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 -- 172.21.15.83:0/1204835892 >> 172.21.15.83:0/1204835892 conn(0x7f6fc40b2050 msgr2=0x7f6fc40b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:54.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 -- 172.21.15.83:0/1204835892 shutdown_connections 2023-11-26T07:12:54.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.516+0000 7f6fcb06d700 1 -- 172.21.15.83:0/1204835892 wait complete. 2023-11-26T07:12:54.525 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2023-11-26T07:12:54.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:12:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.671+0000 7f31296a7700 1 Processor -- start 2023-11-26T07:12:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.671+0000 7f31296a7700 1 -- start start 2023-11-26T07:12:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.672+0000 7f31296a7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f312415bf50 0x7f31240ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.672+0000 7f31296a7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f31240c6c30 0x7f31240cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.672+0000 7f31296a7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31240d0140 con 0x7f31240ce830 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.672+0000 7f31296a7700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312415c330 con 0x7f312415bf50 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.672+0000 7f31296a7700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31240cf3c0 con 0x7f31240c6c30 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.672+0000 7f3122ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f312415bf50 0x7f31240ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.672+0000 7f3122ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f312415bf50 0x7f31240ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37048/0 (socket says 172.21.15.83:37048) 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.672+0000 7f3122ffd700 1 -- 172.21.15.83:0/2630938804 learned_addr learned my addr 172.21.15.83:0/2630938804 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.673+0000 7f3122ffd700 1 -- 172.21.15.83:0/2630938804 >> v2:172.21.15.203:3300/0 conn(0x7f31240c6c30 msgr2=0x7f31240cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.673+0000 7f31237fe700 1 --2- 172.21.15.83:0/2630938804 >> v2:172.21.15.203:3300/0 conn(0x7f31240c6c30 0x7f31240cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.673+0000 7f3122ffd700 1 --2- 172.21.15.83:0/2630938804 >> v2:172.21.15.203:3300/0 conn(0x7f31240c6c30 0x7f31240cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.673+0000 7f3122ffd700 1 -- 172.21.15.83:0/2630938804 >> v1:172.21.15.203:6789/0 conn(0x7f31240ce830 legacy=0x7f31240cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.673+0000 7f3122ffd700 1 -- 172.21.15.83:0/2630938804 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31240d03e0 con 0x7f312415bf50 2023-11-26T07:12:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.673+0000 7f3122ffd700 1 --2- 172.21.15.83:0/2630938804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312415bf50 0x7f31240ce2f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f31240c8c00 tx=0x7f3118009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79009be649f5c87e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.673+0000 7f3121ffb700 1 -- 172.21.15.83:0/2630938804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118004230 con 0x7f312415bf50 2023-11-26T07:12:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.673+0000 7f3121ffb700 1 -- 172.21.15.83:0/2630938804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3118004390 con 0x7f312415bf50 2023-11-26T07:12:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.673+0000 7f3121ffb700 1 -- 172.21.15.83:0/2630938804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180036a0 con 0x7f312415bf50 2023-11-26T07:12:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.674+0000 7f31296a7700 1 -- 172.21.15.83:0/2630938804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312415bf50 msgr2=0x7f31240ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.674+0000 7f31296a7700 1 --2- 172.21.15.83:0/2630938804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312415bf50 0x7f31240ce2f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f31240c8c00 tx=0x7f3118009bd0 comp rx=0 tx=0).stop 2023-11-26T07:12:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.674+0000 7f31296a7700 1 -- 172.21.15.83:0/2630938804 shutdown_connections 2023-11-26T07:12:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.674+0000 7f31296a7700 1 --2- 172.21.15.83:0/2630938804 >> v2:172.21.15.203:3300/0 conn(0x7f31240c6c30 0x7f31240cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.674+0000 7f31296a7700 1 --2- 172.21.15.83:0/2630938804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312415bf50 0x7f31240ce2f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.674+0000 7f31296a7700 1 -- 172.21.15.83:0/2630938804 >> 172.21.15.83:0/2630938804 conn(0x7f31240b3780 msgr2=0x7f31240b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.674+0000 7f31296a7700 1 -- 172.21.15.83:0/2630938804 shutdown_connections 2023-11-26T07:12:54.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.674+0000 7f31296a7700 1 -- 172.21.15.83:0/2630938804 wait complete. 2023-11-26T07:12:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.675+0000 7f31296a7700 1 Processor -- start 2023-11-26T07:12:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.675+0000 7f31296a7700 1 -- start start 2023-11-26T07:12:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31296a7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31240c6c30 0x7f31240cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31296a7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31240ce830 0x7f31241549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31296a7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312415bf50 0x7f31241e5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31296a7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31241e7d70 con 0x7f312415bf50 2023-11-26T07:12:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31296a7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312406b290 con 0x7f31240c6c30 2023-11-26T07:12:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31296a7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31241e74d0 con 0x7f31240ce830 2023-11-26T07:12:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31237fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312415bf50 0x7f31241e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:54.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31237fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312415bf50 0x7f31241e5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37058/0 (socket says 172.21.15.83:37058) 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31237fe700 1 -- 172.21.15.83:0/2347974576 learned_addr learned my addr 172.21.15.83:0/2347974576 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f3122ffd700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31240c6c30 0x7f31240cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31237fe700 1 -- 172.21.15.83:0/2347974576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31240ce830 msgr2=0x7f31241549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31227fc700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31240ce830 0x7f31241549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31237fe700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31240ce830 0x7f31241549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31237fe700 1 -- 172.21.15.83:0/2347974576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31240c6c30 msgr2=0x7f31240cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31237fe700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31240c6c30 0x7f31240cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31237fe700 1 -- 172.21.15.83:0/2347974576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118009550 con 0x7f312415bf50 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31237fe700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312415bf50 0x7f31241e5590 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f311c00cd90 tx=0x7f311c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f310b7fe700 1 -- 172.21.15.83:0/2347974576 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c003a20 con 0x7f312415bf50 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f310b7fe700 1 -- 172.21.15.83:0/2347974576 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c003ba0 con 0x7f312415bf50 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31296a7700 1 -- 172.21.15.83:0/2347974576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31241e77b0 con 0x7f312415bf50 2023-11-26T07:12:54.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f310b7fe700 1 -- 172.21.15.83:0/2347974576 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c004020 con 0x7f312415bf50 2023-11-26T07:12:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.676+0000 7f31296a7700 1 -- 172.21.15.83:0/2347974576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31240cbd90 con 0x7f312415bf50 2023-11-26T07:12:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.677+0000 7f31296a7700 1 -- 172.21.15.83:0/2347974576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312406b500 con 0x7f312415bf50 2023-11-26T07:12:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.678+0000 7f310b7fe700 1 -- 172.21.15.83:0/2347974576 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f311c004180 con 0x7f312415bf50 2023-11-26T07:12:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.678+0000 7f310b7fe700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f311006b080 0x7f311006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.678+0000 7f310b7fe700 1 -- 172.21.15.83:0/2347974576 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f311c012070 con 0x7f312415bf50 2023-11-26T07:12:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.678+0000 7f3122ffd700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f311006b080 0x7f311006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:54.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.679+0000 7f3122ffd700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f311006b080 0x7f311006d540 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f3118008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:54.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.683+0000 7f310b7fe700 1 -- 172.21.15.83:0/2347974576 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f311c054ef0 con 0x7f312415bf50 2023-11-26T07:12:54.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:54.919+0000 7f31296a7700 1 -- 172.21.15.83:0/2347974576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f312406aff0 con 0x7f312415bf50 2023-11-26T07:12:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.514+0000 7f310b7fe700 1 -- 172.21.15.83:0/2347974576 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) v1 ==== 172+0+0 (secure 0 0 0) 0x7f312406aff0 con 0x7f312415bf50 2023-11-26T07:12:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.518+0000 7f31296a7700 1 -- 172.21.15.83:0/2347974576 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f311006b080 msgr2=0x7f311006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.518+0000 7f31296a7700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f311006b080 0x7f311006d540 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f3118008030 comp rx=0 tx=0).stop 2023-11-26T07:12:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.518+0000 7f31296a7700 1 -- 172.21.15.83:0/2347974576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312415bf50 msgr2=0x7f31241e5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.518+0000 7f31296a7700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312415bf50 0x7f31241e5590 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f311c00cd90 tx=0x7f311c00a570 comp rx=0 tx=0).stop 2023-11-26T07:12:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.519+0000 7f31296a7700 1 -- 172.21.15.83:0/2347974576 shutdown_connections 2023-11-26T07:12:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.519+0000 7f31296a7700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312415bf50 0x7f31241e5590 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.519+0000 7f31296a7700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31240c6c30 0x7f31240cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.519+0000 7f31296a7700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f311006b080 0x7f311006d540 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f3118008030 comp rx=0 tx=0).stop 2023-11-26T07:12:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.519+0000 7f31296a7700 1 --2- 172.21.15.83:0/2347974576 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31240ce830 0x7f31241549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.519+0000 7f31296a7700 1 -- 172.21.15.83:0/2347974576 >> 172.21.15.83:0/2347974576 conn(0x7f31240b3780 msgr2=0x7f312406ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.519+0000 7f31296a7700 1 -- 172.21.15.83:0/2347974576 shutdown_connections 2023-11-26T07:12:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.519+0000 7f31296a7700 1 -- 172.21.15.83:0/2347974576 wait complete. 2023-11-26T07:12:55.522 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 6 and data pool 7 2023-11-26T07:12:55.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:55.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.668+0000 7fe8b0a87700 1 Processor -- start 2023-11-26T07:12:55.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.668+0000 7fe8b0a87700 1 -- start start 2023-11-26T07:12:55.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8b0a87700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ac15bdd0 0x7fe8ac1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8b0a87700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8ac0b3370 0x7fe8ac0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8b0a87700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac0b2030 con 0x7fe8ac0d1c70 2023-11-26T07:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8b0a87700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b2f50 con 0x7fe8ac0b3370 2023-11-26T07:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8b0a87700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b22d0 con 0x7fe8ac15bdd0 2023-11-26T07:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8a9d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8ac0b3370 0x7fe8ac0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8aad9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe8ac0d1c70 0x7fe8ac0ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57120/0 (socket says 172.21.15.83:57120) 2023-11-26T07:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8aad9d700 1 -- 172.21.15.83:0/517562697 learned_addr learned my addr 172.21.15.83:0/517562697 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8aa59c700 1 --2- 172.21.15.83:0/517562697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ac15bdd0 0x7fe8ac1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8a9d9b700 1 -- 172.21.15.83:0/517562697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ac15bdd0 msgr2=0x7fe8ac1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8a9d9b700 1 --2- 172.21.15.83:0/517562697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ac15bdd0 0x7fe8ac1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8a9d9b700 1 -- 172.21.15.83:0/517562697 >> v1:172.21.15.83:6789/0 conn(0x7fe8ac0d1c70 legacy=0x7fe8ac0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8a9d9b700 1 -- 172.21.15.83:0/517562697 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0b25b0 con 0x7fe8ac0b3370 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8a959a700 1 -- 172.21.15.83:0/517562697 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536511087 0 0) 0x7fe8ac0b2030 con 0x7fe8ac0d1c70 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8a9d9b700 1 --2- 172.21.15.83:0/517562697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac0b3370 0x7fe8ac0b1af0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009840 tx=0x7fe8a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5af73834cd67611 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.669+0000 7fe8a959a700 1 -- 172.21.15.83:0/517562697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0011830 con 0x7fe8ac0b3370 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.670+0000 7fe8a959a700 1 -- 172.21.15.83:0/517562697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0011990 con 0x7fe8ac0b3370 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.670+0000 7fe8a959a700 1 -- 172.21.15.83:0/517562697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0011d20 con 0x7fe8ac0b3370 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.670+0000 7fe8b0a87700 1 -- 172.21.15.83:0/517562697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac0b3370 msgr2=0x7fe8ac0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.670+0000 7fe8b0a87700 1 --2- 172.21.15.83:0/517562697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac0b3370 0x7fe8ac0b1af0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009840 tx=0x7fe8a0009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.670+0000 7fe8b0a87700 1 -- 172.21.15.83:0/517562697 shutdown_connections 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.670+0000 7fe8b0a87700 1 --2- 172.21.15.83:0/517562697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ac15bdd0 0x7fe8ac1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.670+0000 7fe8b0a87700 1 --2- 172.21.15.83:0/517562697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac0b3370 0x7fe8ac0b1af0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.670+0000 7fe8b0a87700 1 -- 172.21.15.83:0/517562697 >> 172.21.15.83:0/517562697 conn(0x7fe8ac0731a0 msgr2=0x7fe8ac09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:55.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.670+0000 7fe8b0a87700 1 -- 172.21.15.83:0/517562697 shutdown_connections 2023-11-26T07:12:55.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.670+0000 7fe8b0a87700 1 -- 172.21.15.83:0/517562697 wait complete. 2023-11-26T07:12:55.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.671+0000 7fe8b0a87700 1 Processor -- start 2023-11-26T07:12:55.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.671+0000 7fe8b0a87700 1 -- start start 2023-11-26T07:12:55.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.672+0000 7fe8b0a87700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac0b3370 0x7fe8ac14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:55.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.672+0000 7fe8b0a87700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ac0d1c70 0x7fe8ac15c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:55.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.672+0000 7fe8b0a87700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ac15bdd0 0x7fe8ac15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.672+0000 7fe8b0a87700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1503e0 con 0x7fe8ac0b3370 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.672+0000 7fe8b0a87700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b51e0 con 0x7fe8ac0d1c70 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.672+0000 7fe8b0a87700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b48d0 con 0x7fe8ac15bdd0 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.672+0000 7fe8aa59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac0b3370 0x7fe8ac14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.672+0000 7fe8aa59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac0b3370 0x7fe8ac14fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37082/0 (socket says 172.21.15.83:37082) 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.672+0000 7fe8aa59c700 1 -- 172.21.15.83:0/3532156947 learned_addr learned my addr 172.21.15.83:0/3532156947 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe8aa59c700 1 -- 172.21.15.83:0/3532156947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ac15bdd0 msgr2=0x7fe8ac15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe8aa59c700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ac15bdd0 0x7fe8ac15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe8aa59c700 1 -- 172.21.15.83:0/3532156947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ac0d1c70 msgr2=0x7fe8ac15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe8a9d9b700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ac0d1c70 0x7fe8ac15c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe8aa59c700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ac0d1c70 0x7fe8ac15c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe8aa59c700 1 -- 172.21.15.83:0/3532156947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a00094a0 con 0x7fe8ac0b3370 2023-11-26T07:12:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe8aa59c700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac0b3370 0x7fe8ac14fea0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c7f0 tx=0x7fe89c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe89affd700 1 -- 172.21.15.83:0/3532156947 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c011800 con 0x7fe8ac0b3370 2023-11-26T07:12:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe89affd700 1 -- 172.21.15.83:0/3532156947 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c011960 con 0x7fe8ac0b3370 2023-11-26T07:12:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe8b0a87700 1 -- 172.21.15.83:0/3532156947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac0cb640 con 0x7fe8ac0b3370 2023-11-26T07:12:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe89affd700 1 -- 172.21.15.83:0/3532156947 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c011cf0 con 0x7fe8ac0b3370 2023-11-26T07:12:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.673+0000 7fe8b0a87700 1 -- 172.21.15.83:0/3532156947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac0b4e70 con 0x7fe8ac0b3370 2023-11-26T07:12:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.674+0000 7fe8b0a87700 1 -- 172.21.15.83:0/3532156947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac067f60 con 0x7fe8ac0b3370 2023-11-26T07:12:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.674+0000 7fe89affd700 1 -- 172.21.15.83:0/3532156947 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe89c018360 con 0x7fe8ac0b3370 2023-11-26T07:12:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.675+0000 7fe89affd700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe88c06b080 0x7fe88c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.675+0000 7fe8a9d9b700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe88c06b080 0x7fe88c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.675+0000 7fe89affd700 1 -- 172.21.15.83:0/3532156947 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe89c089990 con 0x7fe8ac0b3370 2023-11-26T07:12:55.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.675+0000 7fe8a9d9b700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe88c06b080 0x7fe88c06d540 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac1f4bc0 tx=0x7fe8a0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:55.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.679+0000 7fe89affd700 1 -- 172.21.15.83:0/3532156947 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe89c0546b0 con 0x7fe8ac0b3370 2023-11-26T07:12:55.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.873+0000 7fe8b0a87700 1 -- 172.21.15.83:0/3532156947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8ac0b7c60 con 0x7fe8ac0b3370 2023-11-26T07:12:55.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.874+0000 7fe89affd700 1 -- 172.21.15.83:0/3532156947 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16514 (secure 0 0 0) 0x7fe89c01f090 con 0x7fe8ac0b3370 2023-11-26T07:12:55.876 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:55.876 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:54.920639+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:52.958067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:53.875459+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:55.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.876+0000 7fe8b0a87700 1 -- 172.21.15.83:0/3532156947 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe88c06b080 msgr2=0x7fe88c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:55.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.876+0000 7fe8b0a87700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe88c06b080 0x7fe88c06d540 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac1f4bc0 tx=0x7fe8a0008030 comp rx=0 tx=0).stop 2023-11-26T07:12:55.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.876+0000 7fe8b0a87700 1 -- 172.21.15.83:0/3532156947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac0b3370 msgr2=0x7fe8ac14fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:55.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.876+0000 7fe8b0a87700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac0b3370 0x7fe8ac14fea0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c7f0 tx=0x7fe89c00cbb0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.876+0000 7fe8b0a87700 1 -- 172.21.15.83:0/3532156947 shutdown_connections 2023-11-26T07:12:55.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.876+0000 7fe8b0a87700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe88c06b080 0x7fe88c06d540 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.876+0000 7fe8b0a87700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ac15bdd0 0x7fe8ac15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.876+0000 7fe8b0a87700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ac0b3370 0x7fe8ac14fea0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.876+0000 7fe8b0a87700 1 --2- 172.21.15.83:0/3532156947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ac0d1c70 0x7fe8ac15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:55.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.876+0000 7fe8b0a87700 1 -- 172.21.15.83:0/3532156947 >> 172.21.15.83:0/3532156947 conn(0x7fe8ac0731a0 msgr2=0x7fe8ac0afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:55.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.877+0000 7fe8b0a87700 1 -- 172.21.15.83:0/3532156947 shutdown_connections 2023-11-26T07:12:55.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:55.877+0000 7fe8b0a87700 1 -- 172.21.15.83:0/3532156947 wait complete. 2023-11-26T07:12:55.890 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:56.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.008+0000 7f46e3e0d700 1 Processor -- start 2023-11-26T07:12:56.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.009+0000 7f46e3e0d700 1 -- start start 2023-11-26T07:12:56.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.009+0000 7f46e3e0d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f46dc06c710 0x7f46dc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.009+0000 7f46e3e0d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46dc0af4d0 0x7f46dc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e3e0d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46dc0ae430 con 0x7f46dc15e080 2023-11-26T07:12:56.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e3e0d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46dc0af0b0 con 0x7f46dc06c710 2023-11-26T07:12:56.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e3e0d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0b05a0 con 0x7f46dc0af4d0 2023-11-26T07:12:56.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e13a8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f46dc06c710 0x7f46dc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:56.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e13a8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f46dc06c710 0x7f46dc0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37090/0 (socket says 172.21.15.83:37090) 2023-11-26T07:12:56.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e1ba9700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f46dc15e080 0x7f46dc06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57130/0 (socket says 172.21.15.83:57130) 2023-11-26T07:12:56.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e13a8700 1 -- 172.21.15.83:0/190999652 learned_addr learned my addr 172.21.15.83:0/190999652 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:56.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e23aa700 1 --2- 172.21.15.83:0/190999652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46dc0af4d0 0x7f46dc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:56.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e13a8700 1 -- 172.21.15.83:0/190999652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46dc0af4d0 msgr2=0x7f46dc0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e13a8700 1 --2- 172.21.15.83:0/190999652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46dc0af4d0 0x7f46dc0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e13a8700 1 -- 172.21.15.83:0/190999652 >> v1:172.21.15.83:6789/0 conn(0x7f46dc15e080 legacy=0x7f46dc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e13a8700 1 -- 172.21.15.83:0/190999652 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46dc0ae710 con 0x7f46dc06c710 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e0ba7700 1 -- 172.21.15.83:0/190999652 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679381461 0 0) 0x7f46dc0ae430 con 0x7f46dc15e080 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e13a8700 1 --2- 172.21.15.83:0/190999652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46dc06c710 0x7f46dc0b5050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f46d80098f0 tx=0x7f46d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc80d9efaceba59 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e0ba7700 1 -- 172.21.15.83:0/190999652 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8011830 con 0x7f46dc06c710 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e0ba7700 1 -- 172.21.15.83:0/190999652 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d8011990 con 0x7f46dc06c710 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.010+0000 7f46e0ba7700 1 -- 172.21.15.83:0/190999652 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8011d20 con 0x7f46dc06c710 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.011+0000 7f46e3e0d700 1 -- 172.21.15.83:0/190999652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46dc06c710 msgr2=0x7f46dc0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.011+0000 7f46e3e0d700 1 --2- 172.21.15.83:0/190999652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46dc06c710 0x7f46dc0b5050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f46d80098f0 tx=0x7f46d8009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.011+0000 7f46e3e0d700 1 -- 172.21.15.83:0/190999652 shutdown_connections 2023-11-26T07:12:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.011+0000 7f46e3e0d700 1 --2- 172.21.15.83:0/190999652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46dc0af4d0 0x7f46dc0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.011+0000 7f46e3e0d700 1 --2- 172.21.15.83:0/190999652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46dc06c710 0x7f46dc0b5050 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.011+0000 7f46e3e0d700 1 -- 172.21.15.83:0/190999652 >> 172.21.15.83:0/190999652 conn(0x7f46dc071b90 msgr2=0x7f46dc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:56.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.011+0000 7f46e3e0d700 1 -- 172.21.15.83:0/190999652 shutdown_connections 2023-11-26T07:12:56.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.011+0000 7f46e3e0d700 1 -- 172.21.15.83:0/190999652 wait complete. 2023-11-26T07:12:56.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.012+0000 7f46e3e0d700 1 Processor -- start 2023-11-26T07:12:56.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.013+0000 7f46e3e0d700 1 -- start start 2023-11-26T07:12:56.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.013+0000 7f46e3e0d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46dc06c710 0x7f46dc151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.013+0000 7f46e3e0d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46dc0af4d0 0x7f46dc15f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.013+0000 7f46e3e0d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46dc15e080 0x7f46dc15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.013+0000 7f46e3e0d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc15ed70 con 0x7f46dc0af4d0 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.013+0000 7f46e3e0d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc151c60 con 0x7f46dc06c710 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.013+0000 7f46e3e0d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc15e460 con 0x7f46dc15e080 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e13a8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46dc0af4d0 0x7f46dc15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e13a8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46dc0af4d0 0x7f46dc15f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37098/0 (socket says 172.21.15.83:37098) 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e13a8700 1 -- 172.21.15.83:0/3468300613 learned_addr learned my addr 172.21.15.83:0/3468300613 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e1ba9700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46dc06c710 0x7f46dc151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e23aa700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46dc15e080 0x7f46dc15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e13a8700 1 -- 172.21.15.83:0/3468300613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46dc15e080 msgr2=0x7f46dc15f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e13a8700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46dc15e080 0x7f46dc15f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e13a8700 1 -- 172.21.15.83:0/3468300613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46dc06c710 msgr2=0x7f46dc151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e13a8700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46dc06c710 0x7f46dc151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e13a8700 1 -- 172.21.15.83:0/3468300613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d8009550 con 0x7f46dc0af4d0 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e13a8700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46dc0af4d0 0x7f46dc15f190 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f46dc09eed0 tx=0x7f46d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46ce7fc700 1 -- 172.21.15.83:0/3468300613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8004480 con 0x7f46dc0af4d0 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46ce7fc700 1 -- 172.21.15.83:0/3468300613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d801f070 con 0x7f46dc0af4d0 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e3e0d700 1 -- 172.21.15.83:0/3468300613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46dc1f7a80 con 0x7f46dc0af4d0 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46ce7fc700 1 -- 172.21.15.83:0/3468300613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d80038c0 con 0x7f46dc0af4d0 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.014+0000 7f46e3e0d700 1 -- 172.21.15.83:0/3468300613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46dc15e990 con 0x7f46dc0af4d0 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.015+0000 7f46e3e0d700 1 -- 172.21.15.83:0/3468300613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46dc0753f0 con 0x7f46dc0af4d0 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.016+0000 7f46ce7fc700 1 -- 172.21.15.83:0/3468300613 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f46d8021030 con 0x7f46dc0af4d0 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.016+0000 7f46ce7fc700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f46c006b080 0x7f46c006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.016+0000 7f46ce7fc700 1 -- 172.21.15.83:0/3468300613 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46d808aca0 con 0x7f46dc0af4d0 2023-11-26T07:12:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.016+0000 7f46e1ba9700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f46c006b080 0x7f46c006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.017+0000 7f46e1ba9700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f46c006b080 0x7f46c006d540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f46d0000f60 tx=0x7f46d0007c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:56.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.020+0000 7f46ce7fc700 1 -- 172.21.15.83:0/3468300613 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f46d8055b60 con 0x7f46dc0af4d0 2023-11-26T07:12:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.210+0000 7f46e3e0d700 1 -- 172.21.15.83:0/3468300613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46dc15eeb0 con 0x7f46dc0af4d0 2023-11-26T07:12:56.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.211+0000 7f46ce7fc700 1 -- 172.21.15.83:0/3468300613 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16514 (secure 0 0 0) 0x7f46dc15eeb0 con 0x7f46dc0af4d0 2023-11-26T07:12:56.213 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:56.213 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":44,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:54.920639+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:52.958067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:53.875459+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:56.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.213+0000 7f46e3e0d700 1 -- 172.21.15.83:0/3468300613 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f46c006b080 msgr2=0x7f46c006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:56.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.213+0000 7f46e3e0d700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f46c006b080 0x7f46c006d540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f46d0000f60 tx=0x7f46d0007c90 comp rx=0 tx=0).stop 2023-11-26T07:12:56.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.213+0000 7f46e3e0d700 1 -- 172.21.15.83:0/3468300613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46dc0af4d0 msgr2=0x7f46dc15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:56.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.213+0000 7f46e3e0d700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46dc0af4d0 0x7f46dc15f190 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f46dc09eed0 tx=0x7f46d8004750 comp rx=0 tx=0).stop 2023-11-26T07:12:56.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.213+0000 7f46e3e0d700 1 -- 172.21.15.83:0/3468300613 shutdown_connections 2023-11-26T07:12:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.213+0000 7f46e3e0d700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f46c006b080 0x7f46c006d540 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.213+0000 7f46e3e0d700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46dc15e080 0x7f46dc15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.213+0000 7f46e3e0d700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46dc06c710 0x7f46dc151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.213+0000 7f46e3e0d700 1 --2- 172.21.15.83:0/3468300613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46dc0af4d0 0x7f46dc15f190 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.213+0000 7f46e3e0d700 1 -- 172.21.15.83:0/3468300613 >> 172.21.15.83:0/3468300613 conn(0x7f46dc071b90 msgr2=0x7f46dc0b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.214+0000 7f46e3e0d700 1 -- 172.21.15.83:0/3468300613 shutdown_connections 2023-11-26T07:12:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.214+0000 7f46e3e0d700 1 -- 172.21.15.83:0/3468300613 wait complete. 2023-11-26T07:12:56.227 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:12:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.349+0000 7f04afa96700 1 Processor -- start 2023-11-26T07:12:56.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.349+0000 7f04afa96700 1 -- start start 2023-11-26T07:12:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.350+0000 7f04afa96700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04a80c6c30 0x7f04a80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.350+0000 7f04afa96700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f04a80cef40 0x7f04a80cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.350+0000 7f04afa96700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a80d0140 con 0x7f04a815bf50 2023-11-26T07:12:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.350+0000 7f04afa96700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a815c330 con 0x7f04a80cef40 2023-11-26T07:12:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.350+0000 7f04afa96700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80cf3d0 con 0x7f04a80c6c30 2023-11-26T07:12:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.350+0000 7f04ad832700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f04a815bf50 0x7f04a80ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57144/0 (socket says 172.21.15.83:57144) 2023-11-26T07:12:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.350+0000 7f04ad832700 1 -- 172.21.15.83:0/1035149790 learned_addr learned my addr 172.21.15.83:0/1035149790 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.350+0000 7f04ae033700 1 --2- 172.21.15.83:0/1035149790 >> v2:172.21.15.83:3300/0 conn(0x7f04a80cef40 0x7f04a80cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.351+0000 7f04ad031700 1 --2- 172.21.15.83:0/1035149790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04a80c6c30 0x7f04a80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.351+0000 7f04ae033700 1 -- 172.21.15.83:0/1035149790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04a80c6c30 msgr2=0x7f04a80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.351+0000 7f04ae033700 1 --2- 172.21.15.83:0/1035149790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04a80c6c30 0x7f04a80cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.351+0000 7f04ae033700 1 -- 172.21.15.83:0/1035149790 >> v1:172.21.15.83:6789/0 conn(0x7f04a815bf50 legacy=0x7f04a80ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.351+0000 7f04ae033700 1 -- 172.21.15.83:0/1035149790 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a80d03e0 con 0x7f04a80cef40 2023-11-26T07:12:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.351+0000 7f04ac830700 1 -- 172.21.15.83:0/1035149790 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365939992 0 0) 0x7f04a80d0140 con 0x7f04a815bf50 2023-11-26T07:12:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.351+0000 7f04ae033700 1 --2- 172.21.15.83:0/1035149790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a80cef40 0x7f04a80cc6c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009d90 tx=0x7f04a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8629d4dfb0083c44 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.351+0000 7f04ac830700 1 -- 172.21.15.83:0/1035149790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a000e070 con 0x7f04a80cef40 2023-11-26T07:12:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.351+0000 7f04ac830700 1 -- 172.21.15.83:0/1035149790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a000bbd0 con 0x7f04a80cef40 2023-11-26T07:12:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.351+0000 7f04ac830700 1 -- 172.21.15.83:0/1035149790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0012430 con 0x7f04a80cef40 2023-11-26T07:12:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.352+0000 7f04afa96700 1 -- 172.21.15.83:0/1035149790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a80cef40 msgr2=0x7f04a80cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.352+0000 7f04afa96700 1 --2- 172.21.15.83:0/1035149790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a80cef40 0x7f04a80cc6c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009d90 tx=0x7f04a000b330 comp rx=0 tx=0).stop 2023-11-26T07:12:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.352+0000 7f04afa96700 1 -- 172.21.15.83:0/1035149790 shutdown_connections 2023-11-26T07:12:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.352+0000 7f04afa96700 1 --2- 172.21.15.83:0/1035149790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a80cef40 0x7f04a80cc6c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.352+0000 7f04afa96700 1 --2- 172.21.15.83:0/1035149790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04a80c6c30 0x7f04a80cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.352+0000 7f04afa96700 1 -- 172.21.15.83:0/1035149790 >> 172.21.15.83:0/1035149790 conn(0x7f04a80b3780 msgr2=0x7f04a80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.352+0000 7f04afa96700 1 -- 172.21.15.83:0/1035149790 shutdown_connections 2023-11-26T07:12:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.352+0000 7f04afa96700 1 -- 172.21.15.83:0/1035149790 wait complete. 2023-11-26T07:12:56.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.353+0000 7f04afa96700 1 Processor -- start 2023-11-26T07:12:56.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.353+0000 7f04afa96700 1 -- start start 2023-11-26T07:12:56.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04afa96700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a80c6c30 0x7f04a80b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04afa96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04a80cef40 0x7f04a814cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04afa96700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04a815bf50 0x7f04a815e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04afa96700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a81564e0 con 0x7f04a80c6c30 2023-11-26T07:12:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04afa96700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a814d4b0 con 0x7f04a80cef40 2023-11-26T07:12:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04afa96700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80b5b20 con 0x7f04a815bf50 2023-11-26T07:12:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04ad832700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a80c6c30 0x7f04a80b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04ad832700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a80c6c30 0x7f04a80b70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37110/0 (socket says 172.21.15.83:37110) 2023-11-26T07:12:56.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04ad832700 1 -- 172.21.15.83:0/974790214 learned_addr learned my addr 172.21.15.83:0/974790214 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04ad031700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04a80cef40 0x7f04a814cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04ad832700 1 -- 172.21.15.83:0/974790214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04a815bf50 msgr2=0x7f04a815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04ad832700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04a815bf50 0x7f04a815e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04ad832700 1 -- 172.21.15.83:0/974790214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04a80cef40 msgr2=0x7f04a814cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04ad832700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04a80cef40 0x7f04a814cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04ad832700 1 -- 172.21.15.83:0/974790214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a0009a70 con 0x7f04a80c6c30 2023-11-26T07:12:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04ad832700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a80c6c30 0x7f04a80b70c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f049800c550 tx=0x7f049800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f049e7fc700 1 -- 172.21.15.83:0/974790214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498004c80 con 0x7f04a80c6c30 2023-11-26T07:12:56.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f049e7fc700 1 -- 172.21.15.83:0/974790214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0498004e00 con 0x7f04a80c6c30 2023-11-26T07:12:56.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04afa96700 1 -- 172.21.15.83:0/974790214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a80b5250 con 0x7f04a80c6c30 2023-11-26T07:12:56.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f049e7fc700 1 -- 172.21.15.83:0/974790214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498007310 con 0x7f04a80c6c30 2023-11-26T07:12:56.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.354+0000 7f04afa96700 1 -- 172.21.15.83:0/974790214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a80b54d0 con 0x7f04a80c6c30 2023-11-26T07:12:56.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.355+0000 7f04afa96700 1 -- 172.21.15.83:0/974790214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a8068fb0 con 0x7f04a80c6c30 2023-11-26T07:12:56.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.356+0000 7f049e7fc700 1 -- 172.21.15.83:0/974790214 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0498007470 con 0x7f04a80c6c30 2023-11-26T07:12:56.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.356+0000 7f049e7fc700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f048c06b080 0x7f048c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:56.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.356+0000 7f049e7fc700 1 -- 172.21.15.83:0/974790214 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04980895e0 con 0x7f04a80c6c30 2023-11-26T07:12:56.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.356+0000 7f04ad031700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f048c06b080 0x7f048c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:56.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.356+0000 7f04ad031700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f048c06b080 0x7f048c06d540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f04a4008850 tx=0x7f04a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:56.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.360+0000 7f049e7fc700 1 -- 172.21.15.83:0/974790214 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f04980544a0 con 0x7f04a80c6c30 2023-11-26T07:12:56.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:56.598+0000 7f04afa96700 1 -- 172.21.15.83:0/974790214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f04a80cdd00 con 0x7f04a80c6c30 2023-11-26T07:12:56.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:12:56.810 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:12:57.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.552+0000 7f049e7fc700 1 -- 172.21.15.83:0/974790214 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7f04a80cdd00 con 0x7f04a80c6c30 2023-11-26T07:12:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.555+0000 7f04afa96700 1 -- 172.21.15.83:0/974790214 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f048c06b080 msgr2=0x7f048c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.555+0000 7f04afa96700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f048c06b080 0x7f048c06d540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f04a4008850 tx=0x7f04a4008040 comp rx=0 tx=0).stop 2023-11-26T07:12:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.555+0000 7f04afa96700 1 -- 172.21.15.83:0/974790214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a80c6c30 msgr2=0x7f04a80b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.555+0000 7f04afa96700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a80c6c30 0x7f04a80b70c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f049800c550 tx=0x7f049800c910 comp rx=0 tx=0).stop 2023-11-26T07:12:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.555+0000 7f04afa96700 1 -- 172.21.15.83:0/974790214 shutdown_connections 2023-11-26T07:12:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.555+0000 7f04afa96700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f048c06b080 0x7f048c06d540 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:57.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.555+0000 7f04afa96700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04a815bf50 0x7f04a815e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.555+0000 7f04afa96700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04a80c6c30 0x7f04a80b70c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.555+0000 7f04afa96700 1 --2- 172.21.15.83:0/974790214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04a80cef40 0x7f04a814cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.555+0000 7f04afa96700 1 -- 172.21.15.83:0/974790214 >> 172.21.15.83:0/974790214 conn(0x7f04a80b3780 msgr2=0x7f04a80c7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.555+0000 7f04afa96700 1 -- 172.21.15.83:0/974790214 shutdown_connections 2023-11-26T07:12:57.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.556+0000 7f04afa96700 1 -- 172.21.15.83:0/974790214 wait complete. 2023-11-26T07:12:57.572 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:12:57.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:12:57.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.699+0000 7fa185cb6700 1 Processor -- start 2023-11-26T07:12:57.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.699+0000 7fa185cb6700 1 -- start start 2023-11-26T07:12:57.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.700+0000 7fa185cb6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa18015bf50 0x7fa1800ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:57.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.700+0000 7fa185cb6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1800d0140 con 0x7fa1800c6c30 2023-11-26T07:12:57.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.700+0000 7fa185cb6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1800cf3c0 con 0x7fa1800ce830 2023-11-26T07:12:57.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.700+0000 7fa185cb6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18015c330 con 0x7fa18015bf50 2023-11-26T07:12:57.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.700+0000 7fa17f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa18015bf50 0x7fa1800ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:57.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.700+0000 7fa17ffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa1800c6c30 0x7fa1800cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57158/0 (socket says 172.21.15.83:57158) 2023-11-26T07:12:57.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.700+0000 7fa17ffff700 1 -- 172.21.15.83:0/1666136540 learned_addr learned my addr 172.21.15.83:0/1666136540 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:57.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.701+0000 7fa17f7fe700 1 -- 172.21.15.83:0/1666136540 >> v1:172.21.15.203:6789/0 conn(0x7fa1800ce830 legacy=0x7fa1800cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:57.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.701+0000 7fa17f7fe700 1 -- 172.21.15.83:0/1666136540 >> v1:172.21.15.83:6789/0 conn(0x7fa1800c6c30 legacy=0x7fa1800cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:57.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.701+0000 7fa17f7fe700 1 -- 172.21.15.83:0/1666136540 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1800d03e0 con 0x7fa18015bf50 2023-11-26T07:12:57.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.701+0000 7fa17e7fc700 1 -- 172.21.15.83:0/1666136540 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947561763 0 0) 0x7fa1800d0140 con 0x7fa1800c6c30 2023-11-26T07:12:57.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.701+0000 7fa17f7fe700 1 --2- 172.21.15.83:0/1666136540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa18015bf50 0x7fa1800ce2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa174009840 tx=0x7fa174009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568e632be1ea94c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:57.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.701+0000 7fa17e7fc700 1 -- 172.21.15.83:0/1666136540 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174011830 con 0x7fa18015bf50 2023-11-26T07:12:57.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.701+0000 7fa17e7fc700 1 -- 172.21.15.83:0/1666136540 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174011990 con 0x7fa18015bf50 2023-11-26T07:12:57.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.701+0000 7fa17e7fc700 1 -- 172.21.15.83:0/1666136540 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174011d20 con 0x7fa18015bf50 2023-11-26T07:12:57.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.702+0000 7fa185cb6700 1 -- 172.21.15.83:0/1666136540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa18015bf50 msgr2=0x7fa1800ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:57.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.702+0000 7fa185cb6700 1 --2- 172.21.15.83:0/1666136540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa18015bf50 0x7fa1800ce2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa174009840 tx=0x7fa174009b50 comp rx=0 tx=0).stop 2023-11-26T07:12:57.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.702+0000 7fa185cb6700 1 -- 172.21.15.83:0/1666136540 shutdown_connections 2023-11-26T07:12:57.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.702+0000 7fa185cb6700 1 --2- 172.21.15.83:0/1666136540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa18015bf50 0x7fa1800ce2f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:57.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.702+0000 7fa185cb6700 1 -- 172.21.15.83:0/1666136540 >> 172.21.15.83:0/1666136540 conn(0x7fa1800b3780 msgr2=0x7fa1800b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:57.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.702+0000 7fa185cb6700 1 -- 172.21.15.83:0/1666136540 shutdown_connections 2023-11-26T07:12:57.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.702+0000 7fa185cb6700 1 -- 172.21.15.83:0/1666136540 wait complete. 2023-11-26T07:12:57.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.703+0000 7fa185cb6700 1 Processor -- start 2023-11-26T07:12:57.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.703+0000 7fa185cb6700 1 -- start start 2023-11-26T07:12:57.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa185cb6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1800c6c30 0x7fa18006ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:57.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa185cb6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1800ce830 0x7fa1801458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:57.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa185cb6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa18015bf50 0x7fa1801f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:57.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa185cb6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801468e0 con 0x7fa18015bf50 2023-11-26T07:12:57.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa185cb6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa180145e20 con 0x7fa1800c6c30 2023-11-26T07:12:57.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa185cb6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800c94a0 con 0x7fa1800ce830 2023-11-26T07:12:57.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa17ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa18015bf50 0x7fa1801f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:57.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa17ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa18015bf50 0x7fa1801f38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37126/0 (socket says 172.21.15.83:37126) 2023-11-26T07:12:57.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa17ffff700 1 -- 172.21.15.83:0/3374909376 learned_addr learned my addr 172.21.15.83:0/3374909376 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa17f7fe700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1800c6c30 0x7fa18006ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa17effd700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1800ce830 0x7fa1801458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa17ffff700 1 -- 172.21.15.83:0/3374909376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1800ce830 msgr2=0x7fa1801458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa17ffff700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1800ce830 0x7fa1801458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa17ffff700 1 -- 172.21.15.83:0/3374909376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1800c6c30 msgr2=0x7fa18006ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa17ffff700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1800c6c30 0x7fa18006ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa17ffff700 1 -- 172.21.15.83:0/3374909376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1740094a0 con 0x7fa18015bf50 2023-11-26T07:12:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.704+0000 7fa17ffff700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa18015bf50 0x7fa1801f38d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa17800cca0 tx=0x7fa17800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.705+0000 7fa167fff700 1 -- 172.21.15.83:0/3374909376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17800ab80 con 0x7fa18015bf50 2023-11-26T07:12:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.705+0000 7fa167fff700 1 -- 172.21.15.83:0/3374909376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17800ad00 con 0x7fa18015bf50 2023-11-26T07:12:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.705+0000 7fa167fff700 1 -- 172.21.15.83:0/3374909376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178007310 con 0x7fa18015bf50 2023-11-26T07:12:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.705+0000 7fa185cb6700 1 -- 172.21.15.83:0/3374909376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa180146040 con 0x7fa18015bf50 2023-11-26T07:12:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.705+0000 7fa185cb6700 1 -- 172.21.15.83:0/3374909376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa180146600 con 0x7fa18015bf50 2023-11-26T07:12:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.706+0000 7fa185cb6700 1 -- 172.21.15.83:0/3374909376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1800c9270 con 0x7fa18015bf50 2023-11-26T07:12:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.706+0000 7fa167fff700 1 -- 172.21.15.83:0/3374909376 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fa178007470 con 0x7fa18015bf50 2023-11-26T07:12:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.706+0000 7fa167fff700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa16c06b080 0x7fa16c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.706+0000 7fa167fff700 1 -- 172.21.15.83:0/3374909376 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa178089330 con 0x7fa18015bf50 2023-11-26T07:12:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.706+0000 7fa17f7fe700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa16c06b080 0x7fa16c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.707+0000 7fa17f7fe700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa16c06b080 0x7fa16c06d540 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa1740090b0 tx=0x7fa17401c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.711+0000 7fa167fff700 1 -- 172.21.15.83:0/3374909376 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1800c9270 con 0x7fa18015bf50 2023-11-26T07:12:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:57.942+0000 7fa185cb6700 1 -- 172.21.15.83:0/3374909376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa1800c9270 con 0x7fa18015bf50 2023-11-26T07:12:58.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.562+0000 7fa167fff700 1 -- 172.21.15.83:0/3374909376 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa1800c9270 con 0x7fa18015bf50 2023-11-26T07:12:58.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 -- 172.21.15.83:0/3374909376 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa16c06b080 msgr2=0x7fa16c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:58.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa16c06b080 0x7fa16c06d540 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa1740090b0 tx=0x7fa17401c040 comp rx=0 tx=0).stop 2023-11-26T07:12:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 -- 172.21.15.83:0/3374909376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa18015bf50 msgr2=0x7fa1801f38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa18015bf50 0x7fa1801f38d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa17800cca0 tx=0x7fa17800a060 comp rx=0 tx=0).stop 2023-11-26T07:12:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 -- 172.21.15.83:0/3374909376 shutdown_connections 2023-11-26T07:12:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa18015bf50 0x7fa1801f38d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1800c6c30 0x7fa18006ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa16c06b080 0x7fa16c06d540 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 --2- 172.21.15.83:0/3374909376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1800ce830 0x7fa1801458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 -- 172.21.15.83:0/3374909376 >> 172.21.15.83:0/3374909376 conn(0x7fa1800b3780 msgr2=0x7fa1800c7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 -- 172.21.15.83:0/3374909376 shutdown_connections 2023-11-26T07:12:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.565+0000 7fa185cb6700 1 -- 172.21.15.83:0/3374909376 wait complete. 2023-11-26T07:12:58.581 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:12:58.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:12:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.705+0000 7f3cb2534700 1 Processor -- start 2023-11-26T07:12:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.705+0000 7f3cb2534700 1 -- start start 2023-11-26T07:12:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb2534700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cac15e080 0x7f3cac0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb2534700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3cac0af4d0 0x7f3cac0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb2534700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cac06c000 0x7f3cac06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb2534700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac0af0b0 con 0x7f3cac06c000 2023-11-26T07:12:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb2534700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac06c9f0 con 0x7f3cac0af4d0 2023-11-26T07:12:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb2534700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0b05a0 con 0x7f3cac15e080 2023-11-26T07:12:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb0ad1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cac06c000 0x7f3cac06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb0ad1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cac06c000 0x7f3cac06c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37134/0 (socket says 172.21.15.83:37134) 2023-11-26T07:12:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb0ad1700 1 -- 172.21.15.83:0/1792134697 learned_addr learned my addr 172.21.15.83:0/1792134697 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb0ad1700 1 -- 172.21.15.83:0/1792134697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cac15e080 msgr2=0x7f3cac0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb0ad1700 1 --2- 172.21.15.83:0/1792134697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cac15e080 0x7f3cac0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb0ad1700 1 -- 172.21.15.83:0/1792134697 >> v2:172.21.15.203:3300/0 conn(0x7f3cac0af4d0 msgr2=0x7f3cac0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cab7fe700 1 --2- 172.21.15.83:0/1792134697 >> v2:172.21.15.203:3300/0 conn(0x7f3cac0af4d0 0x7f3cac0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb0ad1700 1 --2- 172.21.15.83:0/1792134697 >> v2:172.21.15.203:3300/0 conn(0x7f3cac0af4d0 0x7f3cac0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb0ad1700 1 -- 172.21.15.83:0/1792134697 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac0ae430 con 0x7f3cac06c000 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cab7fe700 1 --2- 172.21.15.83:0/1792134697 >> v2:172.21.15.203:3300/0 conn(0x7f3cac0af4d0 0x7f3cac0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.706+0000 7f3cb0ad1700 1 --2- 172.21.15.83:0/1792134697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cac06c000 0x7f3cac06c4b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3cac06d380 tx=0x7f3ca400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476a40b503d4a32c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3caaffd700 1 -- 172.21.15.83:0/1792134697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400e070 con 0x7f3cac06c000 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3caaffd700 1 -- 172.21.15.83:0/1792134697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca400b980 con 0x7f3cac06c000 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3caaffd700 1 -- 172.21.15.83:0/1792134697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400bd30 con 0x7f3cac06c000 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3cb2534700 1 -- 172.21.15.83:0/1792134697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cac06c000 msgr2=0x7f3cac06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3cb2534700 1 --2- 172.21.15.83:0/1792134697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cac06c000 0x7f3cac06c4b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3cac06d380 tx=0x7f3ca400b330 comp rx=0 tx=0).stop 2023-11-26T07:12:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3cb2534700 1 -- 172.21.15.83:0/1792134697 shutdown_connections 2023-11-26T07:12:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3cb2534700 1 --2- 172.21.15.83:0/1792134697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cac06c000 0x7f3cac06c4b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3cb2534700 1 --2- 172.21.15.83:0/1792134697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cac15e080 0x7f3cac0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3cb2534700 1 --2- 172.21.15.83:0/1792134697 >> v2:172.21.15.203:3300/0 conn(0x7f3cac0af4d0 0x7f3cac0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3cb2534700 1 -- 172.21.15.83:0/1792134697 >> 172.21.15.83:0/1792134697 conn(0x7f3cac071b90 msgr2=0x7f3cac071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3cb2534700 1 -- 172.21.15.83:0/1792134697 shutdown_connections 2023-11-26T07:12:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.707+0000 7f3cb2534700 1 -- 172.21.15.83:0/1792134697 wait complete. 2023-11-26T07:12:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.708+0000 7f3cb2534700 1 Processor -- start 2023-11-26T07:12:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.709+0000 7f3cb2534700 1 -- start start 2023-11-26T07:12:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.709+0000 7f3cb2534700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cac06c000 0x7f3cac155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.709+0000 7f3cb2534700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cac0af4d0 0x7f3cac0cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.709+0000 7f3cb2534700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cac15e080 0x7f3cac0d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.709+0000 7f3cb2534700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0cdfc0 con 0x7f3cac06c000 2023-11-26T07:12:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.709+0000 7f3cb2534700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0ce260 con 0x7f3cac15e080 2023-11-26T07:12:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.709+0000 7f3cb2534700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0ce540 con 0x7f3cac0af4d0 2023-11-26T07:12:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.709+0000 7f3cabfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cac06c000 0x7f3cac155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:58.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.709+0000 7f3cabfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cac06c000 0x7f3cac155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37136/0 (socket says 172.21.15.83:37136) 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.709+0000 7f3cabfff700 1 -- 172.21.15.83:0/3934028347 learned_addr learned my addr 172.21.15.83:0/3934028347 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3cb0ad1700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cac15e080 0x7f3cac0d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3cab7fe700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cac0af4d0 0x7f3cac0cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3cabfff700 1 -- 172.21.15.83:0/3934028347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cac0af4d0 msgr2=0x7f3cac0cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3cabfff700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cac0af4d0 0x7f3cac0cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3cabfff700 1 -- 172.21.15.83:0/3934028347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cac15e080 msgr2=0x7f3cac0d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3cabfff700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cac15e080 0x7f3cac0d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3cabfff700 1 -- 172.21.15.83:0/3934028347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca4009b30 con 0x7f3cac06c000 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3cb0ad1700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cac15e080 0x7f3cac0d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3cabfff700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cac06c000 0x7f3cac155f90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000b4a0 tx=0x7f3ca000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3ca8ff9700 1 -- 172.21.15.83:0/3934028347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0011830 con 0x7f3cac06c000 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3ca8ff9700 1 -- 172.21.15.83:0/3934028347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca0011990 con 0x7f3cac06c000 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3cb2534700 1 -- 172.21.15.83:0/3934028347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac0ce820 con 0x7f3cac06c000 2023-11-26T07:12:58.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3ca8ff9700 1 -- 172.21.15.83:0/3934028347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0011d40 con 0x7f3cac06c000 2023-11-26T07:12:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.710+0000 7f3cb2534700 1 -- 172.21.15.83:0/3934028347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac0cede0 con 0x7f3cac06c000 2023-11-26T07:12:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.711+0000 7f3cb2534700 1 -- 172.21.15.83:0/3934028347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac0753f0 con 0x7f3cac06c000 2023-11-26T07:12:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.711+0000 7f3ca8ff9700 1 -- 172.21.15.83:0/3934028347 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3ca001a400 con 0x7f3cac06c000 2023-11-26T07:12:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.712+0000 7f3ca8ff9700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c9806b080 0x7f3c9806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.712+0000 7f3ca8ff9700 1 -- 172.21.15.83:0/3934028347 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ca0089980 con 0x7f3cac06c000 2023-11-26T07:12:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.712+0000 7f3cab7fe700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c9806b080 0x7f3c9806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:58.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.712+0000 7f3cab7fe700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c9806b080 0x7f3c9806d540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c006fd0 tx=0x7f3c9c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:58.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.716+0000 7f3ca8ff9700 1 -- 172.21.15.83:0/3934028347 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ca0054730 con 0x7f3cac06c000 2023-11-26T07:12:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.949+0000 7f3cb2534700 1 -- 172.21.15.83:0/3934028347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cac15dbe0 con 0x7f3cac06c000 2023-11-26T07:12:58.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.950+0000 7f3ca8ff9700 1 -- 172.21.15.83:0/3934028347 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f3ca0015090 con 0x7f3cac06c000 2023-11-26T07:12:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:58.952 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:12:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 -- 172.21.15.83:0/3934028347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c9806b080 msgr2=0x7f3c9806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c9806b080 0x7f3c9806d540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c006fd0 tx=0x7f3c9c008040 comp rx=0 tx=0).stop 2023-11-26T07:12:58.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 -- 172.21.15.83:0/3934028347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cac06c000 msgr2=0x7f3cac155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cac06c000 0x7f3cac155f90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000b4a0 tx=0x7f3ca000b860 comp rx=0 tx=0).stop 2023-11-26T07:12:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 -- 172.21.15.83:0/3934028347 shutdown_connections 2023-11-26T07:12:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cac15e080 0x7f3cac0d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cac06c000 0x7f3cac155f90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3c9806b080 0x7f3c9806d540 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 --2- 172.21.15.83:0/3934028347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cac0af4d0 0x7f3cac0cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 -- 172.21.15.83:0/3934028347 >> 172.21.15.83:0/3934028347 conn(0x7f3cac071b90 msgr2=0x7f3cac06e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 -- 172.21.15.83:0/3934028347 shutdown_connections 2023-11-26T07:12:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:58.952+0000 7f3cb2534700 1 -- 172.21.15.83:0/3934028347 wait complete. 2023-11-26T07:12:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2023-11-26T07:12:58.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:12:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.085+0000 7fa1bc013700 1 Processor -- start 2023-11-26T07:12:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.085+0000 7fa1bc013700 1 -- start start 2023-11-26T07:12:59.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1bc013700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b40adf00 0x7fa1b406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1bc013700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa1b406c540 0x7fa1b40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1bc013700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b40af950 con 0x7fa1b41446a0 2023-11-26T07:12:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1bc013700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b4144a80 con 0x7fa1b406c540 2023-11-26T07:12:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1bc013700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b406c9b0 con 0x7fa1b40adf00 2023-11-26T07:12:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1b95ae700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b40adf00 0x7fa1b406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1ba5b0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa1b406c540 0x7fa1b40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1b95ae700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b40adf00 0x7fa1b406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:47604/0 (socket says 172.21.15.83:47604) 2023-11-26T07:12:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1ba5b0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa1b406c540 0x7fa1b40b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:36600/0 (socket says 172.21.15.83:36600) 2023-11-26T07:12:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1b95ae700 1 -- 172.21.15.83:0/1807859531 learned_addr learned my addr 172.21.15.83:0/1807859531 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1ba5b0700 1 -- 172.21.15.83:0/1807859531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b40adf00 msgr2=0x7fa1b406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1ba5b0700 1 --2- 172.21.15.83:0/1807859531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b40adf00 0x7fa1b406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1ba5b0700 1 -- 172.21.15.83:0/1807859531 >> v1:172.21.15.203:6789/0 conn(0x7fa1b41446a0 legacy=0x7fa1b40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1ba5b0700 1 -- 172.21.15.83:0/1807859531 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b40afc30 con 0x7fa1b406c540 2023-11-26T07:12:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.086+0000 7fa1b95ae700 1 --2- 172.21.15.83:0/1807859531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b40adf00 0x7fa1b406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:12:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.087+0000 7fa1ba5b0700 1 --2- 172.21.15.83:0/1807859531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b406c540 0x7fa1b40b3d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009dc0 tx=0x7fa1ac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3064a0cafb5f109e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.087+0000 7fa1b8dad700 1 -- 172.21.15.83:0/1807859531 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac00e070 con 0x7fa1b406c540 2023-11-26T07:12:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.087+0000 7fa1b8dad700 1 -- 172.21.15.83:0/1807859531 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ac00bae0 con 0x7fa1b406c540 2023-11-26T07:12:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.087+0000 7fa1b8dad700 1 -- 172.21.15.83:0/1807859531 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac00be90 con 0x7fa1b406c540 2023-11-26T07:12:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.087+0000 7fa1bc013700 1 -- 172.21.15.83:0/1807859531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b406c540 msgr2=0x7fa1b40b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.087+0000 7fa1bc013700 1 --2- 172.21.15.83:0/1807859531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b406c540 0x7fa1b40b3d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009dc0 tx=0x7fa1ac00b330 comp rx=0 tx=0).stop 2023-11-26T07:12:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.087+0000 7fa1bc013700 1 -- 172.21.15.83:0/1807859531 shutdown_connections 2023-11-26T07:12:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.087+0000 7fa1bc013700 1 --2- 172.21.15.83:0/1807859531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b406c540 0x7fa1b40b3d40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.087+0000 7fa1bc013700 1 --2- 172.21.15.83:0/1807859531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b40adf00 0x7fa1b406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.087+0000 7fa1bc013700 1 -- 172.21.15.83:0/1807859531 >> 172.21.15.83:0/1807859531 conn(0x7fa1b4071b90 msgr2=0x7fa1b4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.088+0000 7fa1bc013700 1 -- 172.21.15.83:0/1807859531 shutdown_connections 2023-11-26T07:12:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.088+0000 7fa1bc013700 1 -- 172.21.15.83:0/1807859531 wait complete. 2023-11-26T07:12:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.089+0000 7fa1bc013700 1 Processor -- start 2023-11-26T07:12:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.089+0000 7fa1bc013700 1 -- start start 2023-11-26T07:12:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.089+0000 7fa1bc013700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b406c540 0x7fa1b41f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.089+0000 7fa1bc013700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b40adf00 0x7fa1b4073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.089+0000 7fa1bc013700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b41446a0 0x7fa1b40744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.089+0000 7fa1bc013700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b414df80 con 0x7fa1b406c540 2023-11-26T07:12:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.089+0000 7fa1bc013700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b40d01f0 con 0x7fa1b40adf00 2023-11-26T07:12:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.089+0000 7fa1bc013700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b40d04d0 con 0x7fa1b41446a0 2023-11-26T07:12:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.089+0000 7fa1b9daf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b406c540 0x7fa1b41f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.089+0000 7fa1b9daf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b406c540 0x7fa1b41f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37150/0 (socket says 172.21.15.83:37150) 2023-11-26T07:12:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1b9daf700 1 -- 172.21.15.83:0/4017909114 learned_addr learned my addr 172.21.15.83:0/4017909114 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1b95ae700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b40adf00 0x7fa1b4073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1ba5b0700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b41446a0 0x7fa1b40744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1b9daf700 1 -- 172.21.15.83:0/4017909114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b41446a0 msgr2=0x7fa1b40744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1b9daf700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b41446a0 0x7fa1b40744d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1b9daf700 1 -- 172.21.15.83:0/4017909114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b40adf00 msgr2=0x7fa1b4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1b9daf700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b40adf00 0x7fa1b4073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1b9daf700 1 -- 172.21.15.83:0/4017909114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a4009550 con 0x7fa1b406c540 2023-11-26T07:12:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1b95ae700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b40adf00 0x7fa1b4073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:12:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1b9daf700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b406c540 0x7fa1b41f3620 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000c450 tx=0x7fa1b000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1aa7fc700 1 -- 172.21.15.83:0/4017909114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00052d0 con 0x7fa1b406c540 2023-11-26T07:12:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1aa7fc700 1 -- 172.21.15.83:0/4017909114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b0005430 con 0x7fa1b406c540 2023-11-26T07:12:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1bc013700 1 -- 172.21.15.83:0/4017909114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ac009a70 con 0x7fa1b406c540 2023-11-26T07:12:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1aa7fc700 1 -- 172.21.15.83:0/4017909114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00038c0 con 0x7fa1b406c540 2023-11-26T07:12:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.090+0000 7fa1bc013700 1 -- 172.21.15.83:0/4017909114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b414e470 con 0x7fa1b406c540 2023-11-26T07:12:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.091+0000 7fa1bc013700 1 -- 172.21.15.83:0/4017909114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b414e630 con 0x7fa1b406c540 2023-11-26T07:12:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.091+0000 7fa1aa7fc700 1 -- 172.21.15.83:0/4017909114 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fa1b0003a20 con 0x7fa1b406c540 2023-11-26T07:12:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.092+0000 7fa1aa7fc700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa19806b080 0x7fa19806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.092+0000 7fa1aa7fc700 1 -- 172.21.15.83:0/4017909114 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1b0088d90 con 0x7fa1b406c540 2023-11-26T07:12:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.092+0000 7fa1b95ae700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa19806b080 0x7fa19806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.092+0000 7fa1b95ae700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa19806b080 0x7fa19806d540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1b41f7330 tx=0x7fa1a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:59.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.096+0000 7fa1aa7fc700 1 -- 172.21.15.83:0/4017909114 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1b0053a10 con 0x7fa1b406c540 2023-11-26T07:12:59.290 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.289+0000 7fa1bc013700 1 -- 172.21.15.83:0/4017909114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1b40753f0 con 0x7fa1b406c540 2023-11-26T07:12:59.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.290+0000 7fa1aa7fc700 1 -- 172.21.15.83:0/4017909114 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16501 (secure 0 0 0) 0x7fa1b0057030 con 0x7fa1b406c540 2023-11-26T07:12:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:59.291 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":45,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:12:56.515236+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:52.958067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:53.875459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:12:59.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 -- 172.21.15.83:0/4017909114 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa19806b080 msgr2=0x7fa19806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa19806b080 0x7fa19806d540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1b41f7330 tx=0x7fa1a4008040 comp rx=0 tx=0).stop 2023-11-26T07:12:59.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 -- 172.21.15.83:0/4017909114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b406c540 msgr2=0x7fa1b41f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b406c540 0x7fa1b41f3620 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000c450 tx=0x7fa1b000c810 comp rx=0 tx=0).stop 2023-11-26T07:12:59.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 -- 172.21.15.83:0/4017909114 shutdown_connections 2023-11-26T07:12:59.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa19806b080 0x7fa19806d540 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1b41446a0 0x7fa1b40744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1b406c540 0x7fa1b41f3620 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 --2- 172.21.15.83:0/4017909114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1b40adf00 0x7fa1b4073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 -- 172.21.15.83:0/4017909114 >> 172.21.15.83:0/4017909114 conn(0x7fa1b4071b90 msgr2=0x7fa1b40b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:59.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 -- 172.21.15.83:0/4017909114 shutdown_connections 2023-11-26T07:12:59.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.292+0000 7fa1bc013700 1 -- 172.21.15.83:0/4017909114 wait complete. 2023-11-26T07:12:59.306 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:12:59.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:12:59.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.424+0000 7fb78e482700 1 Processor -- start 2023-11-26T07:12:59.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.424+0000 7fb78e482700 1 -- start start 2023-11-26T07:12:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.425+0000 7fb78e482700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb78815bd30 0x7fb7880cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.425+0000 7fb78e482700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb788161bf0 con 0x7fb788162920 2023-11-26T07:12:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.425+0000 7fb78e482700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78815f450 con 0x7fb78815edf0 2023-11-26T07:12:59.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.425+0000 7fb78e482700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb788161e40 con 0x7fb78815bd30 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.425+0000 7fb787fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb78815bd30 0x7fb7880cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.425+0000 7fb787fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb78815bd30 0x7fb7880cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37158/0 (socket says 172.21.15.83:37158) 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.425+0000 7fb787fff700 1 -- 172.21.15.83:0/3178450954 learned_addr learned my addr 172.21.15.83:0/3178450954 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.425+0000 7fb7877fe700 1 --1- 172.21.15.83:0/3178450954 >> v1:172.21.15.83:6789/0 conn(0x7fb788162920 0x7fb78815e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57198/0 (socket says 172.21.15.83:57198) 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb787fff700 1 -- 172.21.15.83:0/3178450954 >> v1:172.21.15.203:6789/0 conn(0x7fb78815edf0 legacy=0x7fb7880afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb787fff700 1 -- 172.21.15.83:0/3178450954 >> v1:172.21.15.83:6789/0 conn(0x7fb788162920 legacy=0x7fb78815e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb786ffd700 1 -- 172.21.15.83:0/3178450954 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558130756 0 0) 0x7fb788161bf0 con 0x7fb788162920 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb787fff700 1 -- 172.21.15.83:0/3178450954 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb788162120 con 0x7fb78815bd30 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb787fff700 1 --2- 172.21.15.83:0/3178450954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78815bd30 0x7fb7880cf4c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0098f0 tx=0x7fb77c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55d5b12d29a25c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb786ffd700 1 -- 172.21.15.83:0/3178450954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c004230 con 0x7fb78815bd30 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb786ffd700 1 -- 172.21.15.83:0/3178450954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c004390 con 0x7fb78815bd30 2023-11-26T07:12:59.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb786ffd700 1 -- 172.21.15.83:0/3178450954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0036a0 con 0x7fb78815bd30 2023-11-26T07:12:59.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb78e482700 1 -- 172.21.15.83:0/3178450954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78815bd30 msgr2=0x7fb7880cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb78e482700 1 --2- 172.21.15.83:0/3178450954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78815bd30 0x7fb7880cf4c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0098f0 tx=0x7fb77c009c00 comp rx=0 tx=0).stop 2023-11-26T07:12:59.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb78e482700 1 -- 172.21.15.83:0/3178450954 shutdown_connections 2023-11-26T07:12:59.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb78e482700 1 --2- 172.21.15.83:0/3178450954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78815bd30 0x7fb7880cf4c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.426+0000 7fb78e482700 1 -- 172.21.15.83:0/3178450954 >> 172.21.15.83:0/3178450954 conn(0x7fb788072710 msgr2=0x7fb788072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:59.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.427+0000 7fb78e482700 1 -- 172.21.15.83:0/3178450954 shutdown_connections 2023-11-26T07:12:59.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.427+0000 7fb78e482700 1 -- 172.21.15.83:0/3178450954 wait complete. 2023-11-26T07:12:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.428+0000 7fb78e482700 1 Processor -- start 2023-11-26T07:12:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.428+0000 7fb78e482700 1 -- start start 2023-11-26T07:12:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.429+0000 7fb78e482700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78815bd30 0x7fb788156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.429+0000 7fb78e482700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78815edf0 0x7fb7880c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.429+0000 7fb78e482700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb788162920 0x7fb7880c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.429+0000 7fb78e482700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7881e8b10 con 0x7fb78815bd30 2023-11-26T07:12:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.429+0000 7fb78e482700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb788157220 con 0x7fb78815edf0 2023-11-26T07:12:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.429+0000 7fb78e482700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7881593a0 con 0x7fb788162920 2023-11-26T07:12:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.429+0000 7fb787fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78815bd30 0x7fb788156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.429+0000 7fb787fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78815bd30 0x7fb788156ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37164/0 (socket says 172.21.15.83:37164) 2023-11-26T07:12:59.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.429+0000 7fb787fff700 1 -- 172.21.15.83:0/334705622 learned_addr learned my addr 172.21.15.83:0/334705622 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.429+0000 7fb7877fe700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78815edf0 0x7fb7880c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.429+0000 7fb78ca1f700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb788162920 0x7fb7880c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb787fff700 1 -- 172.21.15.83:0/334705622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb788162920 msgr2=0x7fb7880c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb787fff700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb788162920 0x7fb7880c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb787fff700 1 -- 172.21.15.83:0/334705622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78815edf0 msgr2=0x7fb7880c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb787fff700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78815edf0 0x7fb7880c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb787fff700 1 -- 172.21.15.83:0/334705622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c009550 con 0x7fb78815bd30 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb7877fe700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78815edf0 0x7fb7880c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb78ca1f700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb788162920 0x7fb7880c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb787fff700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78815bd30 0x7fb788156ce0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000c00 tx=0x7fb77c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb784ff9700 1 -- 172.21.15.83:0/334705622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c003db0 con 0x7fb78815bd30 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb784ff9700 1 -- 172.21.15.83:0/334705622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c01f070 con 0x7fb78815bd30 2023-11-26T07:12:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb78e482700 1 -- 172.21.15.83:0/334705622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb788159680 con 0x7fb78815bd30 2023-11-26T07:12:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb784ff9700 1 -- 172.21.15.83:0/334705622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c00b8a0 con 0x7fb78815bd30 2023-11-26T07:12:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.430+0000 7fb78e482700 1 -- 172.21.15.83:0/334705622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb788159c40 con 0x7fb78815bd30 2023-11-26T07:12:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.431+0000 7fb78e482700 1 -- 172.21.15.83:0/334705622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7880cd2f0 con 0x7fb78815bd30 2023-11-26T07:12:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.431+0000 7fb784ff9700 1 -- 172.21.15.83:0/334705622 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fb77c021030 con 0x7fb78815bd30 2023-11-26T07:12:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.432+0000 7fb784ff9700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb77406b080 0x7fb77406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.432+0000 7fb784ff9700 1 -- 172.21.15.83:0/334705622 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb77c08b050 con 0x7fb78815bd30 2023-11-26T07:12:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.432+0000 7fb7877fe700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb77406b080 0x7fb77406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.432+0000 7fb7877fe700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb77406b080 0x7fb77406d540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb7881f7690 tx=0x7fb778007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.436+0000 7fb784ff9700 1 -- 172.21.15.83:0/334705622 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb77c055f10 con 0x7fb78815bd30 2023-11-26T07:12:59.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.621+0000 7fb78e482700 1 -- 172.21.15.83:0/334705622 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7880cde80 con 0x7fb77406b080 2023-11-26T07:12:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.629+0000 7fb784ff9700 1 -- 172.21.15.83:0/334705622 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283717 (secure 0 0 0) 0x7fb7880cde80 con 0x7fb77406b080 2023-11-26T07:12:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:12:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 -- 172.21.15.83:0/334705622 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb77406b080 msgr2=0x7fb77406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb77406b080 0x7fb77406d540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb7881f7690 tx=0x7fb778007d30 comp rx=0 tx=0).stop 2023-11-26T07:12:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 -- 172.21.15.83:0/334705622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78815bd30 msgr2=0x7fb788156ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78815bd30 0x7fb788156ce0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000c00 tx=0x7fb77c004750 comp rx=0 tx=0).stop 2023-11-26T07:12:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 -- 172.21.15.83:0/334705622 shutdown_connections 2023-11-26T07:12:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb77406b080 0x7fb77406d540 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb788162920 0x7fb7880c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78815bd30 0x7fb788156ce0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 --2- 172.21.15.83:0/334705622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78815edf0 0x7fb7880c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 -- 172.21.15.83:0/334705622 >> 172.21.15.83:0/334705622 conn(0x7fb788072710 msgr2=0x7fb7880cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 -- 172.21.15.83:0/334705622 shutdown_connections 2023-11-26T07:12:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.633+0000 7fb78e482700 1 -- 172.21.15.83:0/334705622 wait complete. 2023-11-26T07:12:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:12:59.646 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-11-26T07:12:58.325188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221236,"kb_used_data":5048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753484,"statfs":{"total":773094113280,"available":772867567616,"internally_reserved":0,"allocated":5169152,"data_stored":2766708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.816391"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521879+0000","last_change":"2023-11-26T07:12:54.521879+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521879+0000","last_undegraded":"2023-11-26T07:12:54.521879+0000","last_fullsized":"2023-11-26T07:12:54.521879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521487+0000","last_change":"2023-11-26T07:12:54.521487+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521487+0000","last_undegraded":"2023-11-26T07:12:54.521487+0000","last_fullsized":"2023-11-26T07:12:54.521487+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519224+0000","last_change":"2023-11-26T07:12:54.519224+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519224+0000","last_undegraded":"2023-11-26T07:12:54.519224+0000","last_fullsized":"2023-11-26T07:12:54.519224+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523622+0000","last_change":"2023-11-26T07:12:54.523622+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523622+0000","last_undegraded":"2023-11-26T07:12:54.523622+0000","last_fullsized":"2023-11-26T07:12:54.523622+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521234+0000","last_change":"2023-11-26T07:12:54.521234+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521234+0000","last_undegraded":"2023-11-26T07:12:54.521234+0000","last_fullsized":"2023-11-26T07:12:54.521234+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529973+0000","last_change":"2023-11-26T07:12:54.529973+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529973+0000","last_undegraded":"2023-11-26T07:12:54.529973+0000","last_fullsized":"2023-11-26T07:12:54.529973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521276+0000","last_change":"2023-11-26T07:12:54.521276+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521276+0000","last_undegraded":"2023-11-26T07:12:54.521276+0000","last_fullsized":"2023-11-26T07:12:54.521276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526287+0000","last_change":"2023-11-26T07:12:54.526287+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526287+0000","last_undegraded":"2023-11-26T07:12:54.526287+0000","last_fullsized":"2023-11-26T07:12:54.526287+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523760+0000","last_change":"2023-11-26T07:12:54.523760+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523760+0000","last_undegraded":"2023-11-26T07:12:54.523760+0000","last_fullsized":"2023-11-26T07:12:54.523760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.524402+0000","last_change":"2023-11-26T07:12:54.524402+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.524402+0000","last_undegraded":"2023-11-26T07:12:54.524402+0000","last_fullsized":"2023-11-26T07:12:54.524402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523524+0000","last_change":"2023-11-26T07:12:54.523524+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523524+0000","last_undegraded":"2023-11-26T07:12:54.523524+0000","last_fullsized":"2023-11-26T07:12:54.523524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521618+0000","last_change":"2023-11-26T07:12:54.521618+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521618+0000","last_undegraded":"2023-11-26T07:12:54.521618+0000","last_fullsized":"2023-11-26T07:12:54.521618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520013+0000","last_change":"2023-11-26T07:12:54.520013+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520013+0000","last_undegraded":"2023-11-26T07:12:54.520013+0000","last_fullsized":"2023-11-26T07:12:54.520013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523982+0000","last_change":"2023-11-26T07:12:54.523982+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523982+0000","last_undegraded":"2023-11-26T07:12:54.523982+0000","last_fullsized":"2023-11-26T07:12:54.523982+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.530689+0000","last_change":"2023-11-26T07:12:54.530689+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.530689+0000","last_undegraded":"2023-11-26T07:12:54.530689+0000","last_fullsized":"2023-11-26T07:12:54.530689+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526270+0000","last_change":"2023-11-26T07:12:54.526270+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526270+0000","last_undegraded":"2023-11-26T07:12:54.526270+0000","last_fullsized":"2023-11-26T07:12:54.526270+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518831+0000","last_change":"2023-11-26T07:12:54.518831+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518831+0000","last_undegraded":"2023-11-26T07:12:54.518831+0000","last_fullsized":"2023-11-26T07:12:54.518831+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520492+0000","last_change":"2023-11-26T07:12:54.520492+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520492+0000","last_undegraded":"2023-11-26T07:12:54.520492+0000","last_fullsized":"2023-11-26T07:12:54.520492+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.528121+0000","last_change":"2023-11-26T07:12:54.528121+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.528121+0000","last_undegraded":"2023-11-26T07:12:54.528121+0000","last_fullsized":"2023-11-26T07:12:54.528121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520226+0000","last_change":"2023-11-26T07:12:54.520226+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520226+0000","last_undegraded":"2023-11-26T07:12:54.520226+0000","last_fullsized":"2023-11-26T07:12:54.520226+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529224+0000","last_change":"2023-11-26T07:12:54.529224+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529224+0000","last_undegraded":"2023-11-26T07:12:54.529224+0000","last_fullsized":"2023-11-26T07:12:54.529224+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523489+0000","last_change":"2023-11-26T07:12:54.523489+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523489+0000","last_undegraded":"2023-11-26T07:12:54.523489+0000","last_fullsized":"2023-11-26T07:12:54.523489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.530050+0000","last_change":"2023-11-26T07:12:54.530050+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.530050+0000","last_undegraded":"2023-11-26T07:12:54.530050+0000","last_fullsized":"2023-11-26T07:12:54.530050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525496+0000","last_change":"2023-11-26T07:12:54.525496+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525496+0000","last_undegraded":"2023-11-26T07:12:54.525496+0000","last_fullsized":"2023-11-26T07:12:54.525496+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525484+0000","last_change":"2023-11-26T07:12:54.525484+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525484+0000","last_undegraded":"2023-11-26T07:12:54.525484+0000","last_fullsized":"2023-11-26T07:12:54.525484+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520283+0000","last_change":"2023-11-26T07:12:54.520283+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520283+0000","last_undegraded":"2023-11-26T07:12:54.520283+0000","last_fullsized":"2023-11-26T07:12:54.520283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518411+0000","last_change":"2023-11-26T07:12:54.518411+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518411+0000","last_undegraded":"2023-11-26T07:12:54.518411+0000","last_fullsized":"2023-11-26T07:12:54.518411+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522587+0000","last_change":"2023-11-26T07:12:54.522587+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522587+0000","last_undegraded":"2023-11-26T07:12:54.522587+0000","last_fullsized":"2023-11-26T07:12:54.522587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526264+0000","last_change":"2023-11-26T07:12:54.526264+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526264+0000","last_undegraded":"2023-11-26T07:12:54.526264+0000","last_fullsized":"2023-11-26T07:12:54.526264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522172+0000","last_change":"2023-11-26T07:12:54.522172+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522172+0000","last_undegraded":"2023-11-26T07:12:54.522172+0000","last_fullsized":"2023-11-26T07:12:54.522172+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526169+0000","last_change":"2023-11-26T07:12:54.526169+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526169+0000","last_undegraded":"2023-11-26T07:12:54.526169+0000","last_fullsized":"2023-11-26T07:12:54.526169+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521324+0000","last_change":"2023-11-26T07:12:54.521324+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521324+0000","last_undegraded":"2023-11-26T07:12:54.521324+0000","last_fullsized":"2023-11-26T07:12:54.521324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521871+0000","last_change":"2023-11-26T07:12:54.521871+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521871+0000","last_undegraded":"2023-11-26T07:12:54.521871+0000","last_fullsized":"2023-11-26T07:12:54.521871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542574+0000","last_change":"2023-11-26T07:12:54.536815+0000","last_active":"2023-11-26T07:12:54.542574+0000","last_peered":"2023-11-26T07:12:54.542574+0000","last_clean":"2023-11-26T07:12:54.542574+0000","last_became_active":"2023-11-26T07:12:54.536526+0000","last_became_peered":"2023-11-26T07:12:54.536526+0000","last_unstale":"2023-11-26T07:12:54.542574+0000","last_undegraded":"2023-11-26T07:12:54.542574+0000","last_fullsized":"2023-11-26T07:12:54.542574+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:20:45.041782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526811+0000","last_change":"2023-11-26T07:12:54.526811+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526811+0000","last_undegraded":"2023-11-26T07:12:54.526811+0000","last_fullsized":"2023-11-26T07:12:54.526811+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536052+0000","last_change":"2023-11-26T07:12:54.533729+0000","last_active":"2023-11-26T07:12:54.536052+0000","last_peered":"2023-11-26T07:12:54.536052+0000","last_clean":"2023-11-26T07:12:54.536052+0000","last_became_active":"2023-11-26T07:12:54.533440+0000","last_became_peered":"2023-11-26T07:12:54.533440+0000","last_unstale":"2023-11-26T07:12:54.536052+0000","last_undegraded":"2023-11-26T07:12:54.536052+0000","last_fullsized":"2023-11-26T07:12:54.536052+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:04.125066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522115+0000","last_change":"2023-11-26T07:12:54.522115+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522115+0000","last_undegraded":"2023-11-26T07:12:54.522115+0000","last_fullsized":"2023-11-26T07:12:54.522115+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534324+0000","last_change":"2023-11-26T07:12:54.529395+0000","last_active":"2023-11-26T07:12:54.534324+0000","last_peered":"2023-11-26T07:12:54.534324+0000","last_clean":"2023-11-26T07:12:54.534324+0000","last_became_active":"2023-11-26T07:12:54.529093+0000","last_became_peered":"2023-11-26T07:12:54.529093+0000","last_unstale":"2023-11-26T07:12:54.534324+0000","last_undegraded":"2023-11-26T07:12:54.534324+0000","last_fullsized":"2023-11-26T07:12:54.534324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:59.222804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520849+0000","last_change":"2023-11-26T07:12:54.520849+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520849+0000","last_undegraded":"2023-11-26T07:12:54.520849+0000","last_fullsized":"2023-11-26T07:12:54.520849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.532244+0000","last_change":"2023-11-26T07:12:54.529608+0000","last_active":"2023-11-26T07:12:54.532244+0000","last_peered":"2023-11-26T07:12:54.532244+0000","last_clean":"2023-11-26T07:12:54.532244+0000","last_became_active":"2023-11-26T07:12:54.529334+0000","last_became_peered":"2023-11-26T07:12:54.529334+0000","last_unstale":"2023-11-26T07:12:54.532244+0000","last_undegraded":"2023-11-26T07:12:54.532244+0000","last_fullsized":"2023-11-26T07:12:54.532244+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:27:43.076531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519440+0000","last_change":"2023-11-26T07:12:54.519440+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519440+0000","last_undegraded":"2023-11-26T07:12:54.519440+0000","last_fullsized":"2023-11-26T07:12:54.519440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542468+0000","last_change":"2023-11-26T07:12:54.532202+0000","last_active":"2023-11-26T07:12:54.542468+0000","last_peered":"2023-11-26T07:12:54.542468+0000","last_clean":"2023-11-26T07:12:54.542468+0000","last_became_active":"2023-11-26T07:12:54.531692+0000","last_became_peered":"2023-11-26T07:12:54.531692+0000","last_unstale":"2023-11-26T07:12:54.542468+0000","last_undegraded":"2023-11-26T07:12:54.542468+0000","last_fullsized":"2023-11-26T07:12:54.542468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:16.433050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526413+0000","last_change":"2023-11-26T07:12:54.526413+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526413+0000","last_undegraded":"2023-11-26T07:12:54.526413+0000","last_fullsized":"2023-11-26T07:12:54.526413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534545+0000","last_change":"2023-11-26T07:12:54.529445+0000","last_active":"2023-11-26T07:12:54.534545+0000","last_peered":"2023-11-26T07:12:54.534545+0000","last_clean":"2023-11-26T07:12:54.534545+0000","last_became_active":"2023-11-26T07:12:54.529160+0000","last_became_peered":"2023-11-26T07:12:54.529160+0000","last_unstale":"2023-11-26T07:12:54.534545+0000","last_undegraded":"2023-11-26T07:12:54.534545+0000","last_fullsized":"2023-11-26T07:12:54.534545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:45:17.796276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525851+0000","last_change":"2023-11-26T07:12:54.525851+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525851+0000","last_undegraded":"2023-11-26T07:12:54.525851+0000","last_fullsized":"2023-11-26T07:12:54.525851+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534715+0000","last_change":"2023-11-26T07:12:54.531344+0000","last_active":"2023-11-26T07:12:54.534715+0000","last_peered":"2023-11-26T07:12:54.534715+0000","last_clean":"2023-11-26T07:12:54.534715+0000","last_became_active":"2023-11-26T07:12:54.530914+0000","last_became_peered":"2023-11-26T07:12:54.530914+0000","last_unstale":"2023-11-26T07:12:54.534715+0000","last_undegraded":"2023-11-26T07:12:54.534715+0000","last_fullsized":"2023-11-26T07:12:54.534715+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:58.991270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529503+0000","last_change":"2023-11-26T07:12:54.529503+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529503+0000","last_undegraded":"2023-11-26T07:12:54.529503+0000","last_fullsized":"2023-11-26T07:12:54.529503+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541649+0000","last_change":"2023-11-26T07:12:54.533250+0000","last_active":"2023-11-26T07:12:54.541649+0000","last_peered":"2023-11-26T07:12:54.541649+0000","last_clean":"2023-11-26T07:12:54.541649+0000","last_became_active":"2023-11-26T07:12:54.532934+0000","last_became_peered":"2023-11-26T07:12:54.532934+0000","last_unstale":"2023-11-26T07:12:54.541649+0000","last_undegraded":"2023-11-26T07:12:54.541649+0000","last_fullsized":"2023-11-26T07:12:54.541649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:15:29.001269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518952+0000","last_change":"2023-11-26T07:12:54.518952+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518952+0000","last_undegraded":"2023-11-26T07:12:54.518952+0000","last_fullsized":"2023-11-26T07:12:54.518952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.530096+0000","last_change":"2023-11-26T07:12:54.528493+0000","last_active":"2023-11-26T07:12:54.530096+0000","last_peered":"2023-11-26T07:12:54.530096+0000","last_clean":"2023-11-26T07:12:54.530096+0000","last_became_active":"2023-11-26T07:12:54.527916+0000","last_became_peered":"2023-11-26T07:12:54.527916+0000","last_unstale":"2023-11-26T07:12:54.530096+0000","last_undegraded":"2023-11-26T07:12:54.530096+0000","last_fullsized":"2023-11-26T07:12:54.530096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:23:22.243947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520365+0000","last_change":"2023-11-26T07:12:54.520365+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520365+0000","last_undegraded":"2023-11-26T07:12:54.520365+0000","last_fullsized":"2023-11-26T07:12:54.520365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541667+0000","last_change":"2023-11-26T07:12:54.529990+0000","last_active":"2023-11-26T07:12:54.541667+0000","last_peered":"2023-11-26T07:12:54.541667+0000","last_clean":"2023-11-26T07:12:54.541667+0000","last_became_active":"2023-11-26T07:12:54.529727+0000","last_became_peered":"2023-11-26T07:12:54.529727+0000","last_unstale":"2023-11-26T07:12:54.541667+0000","last_undegraded":"2023-11-26T07:12:54.541667+0000","last_fullsized":"2023-11-26T07:12:54.541667+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:41:21.117692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525687+0000","last_change":"2023-11-26T07:12:54.525687+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525687+0000","last_undegraded":"2023-11-26T07:12:54.525687+0000","last_fullsized":"2023-11-26T07:12:54.525687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540184+0000","last_change":"2023-11-26T07:12:54.534123+0000","last_active":"2023-11-26T07:12:54.540184+0000","last_peered":"2023-11-26T07:12:54.540184+0000","last_clean":"2023-11-26T07:12:54.540184+0000","last_became_active":"2023-11-26T07:12:54.533831+0000","last_became_peered":"2023-11-26T07:12:54.533831+0000","last_unstale":"2023-11-26T07:12:54.540184+0000","last_undegraded":"2023-11-26T07:12:54.540184+0000","last_fullsized":"2023-11-26T07:12:54.540184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:25:35.938501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522255+0000","last_change":"2023-11-26T07:12:54.522255+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522255+0000","last_undegraded":"2023-11-26T07:12:54.522255+0000","last_fullsized":"2023-11-26T07:12:54.522255+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.531202+0000","last_change":"2023-11-26T07:12:54.528248+0000","last_active":"2023-11-26T07:12:54.531202+0000","last_peered":"2023-11-26T07:12:54.531202+0000","last_clean":"2023-11-26T07:12:54.531202+0000","last_became_active":"2023-11-26T07:12:54.527887+0000","last_became_peered":"2023-11-26T07:12:54.527887+0000","last_unstale":"2023-11-26T07:12:54.531202+0000","last_undegraded":"2023-11-26T07:12:54.531202+0000","last_fullsized":"2023-11-26T07:12:54.531202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:52.719388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.527948+0000","last_change":"2023-11-26T07:12:54.527948+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.527948+0000","last_undegraded":"2023-11-26T07:12:54.527948+0000","last_fullsized":"2023-11-26T07:12:54.527948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542608+0000","last_change":"2023-11-26T07:12:54.537145+0000","last_active":"2023-11-26T07:12:54.542608+0000","last_peered":"2023-11-26T07:12:54.542608+0000","last_clean":"2023-11-26T07:12:54.542608+0000","last_became_active":"2023-11-26T07:12:54.536921+0000","last_became_peered":"2023-11-26T07:12:54.536921+0000","last_unstale":"2023-11-26T07:12:54.542608+0000","last_undegraded":"2023-11-26T07:12:54.542608+0000","last_fullsized":"2023-11-26T07:12:54.542608+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:02:54.984506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521236+0000","last_change":"2023-11-26T07:12:54.521236+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521236+0000","last_undegraded":"2023-11-26T07:12:54.521236+0000","last_fullsized":"2023-11-26T07:12:54.521236+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536376+0000","last_change":"2023-11-26T07:12:54.532202+0000","last_active":"2023-11-26T07:12:54.536376+0000","last_peered":"2023-11-26T07:12:54.536376+0000","last_clean":"2023-11-26T07:12:54.536376+0000","last_became_active":"2023-11-26T07:12:54.531841+0000","last_became_peered":"2023-11-26T07:12:54.531841+0000","last_unstale":"2023-11-26T07:12:54.536376+0000","last_undegraded":"2023-11-26T07:12:54.536376+0000","last_fullsized":"2023-11-26T07:12:54.536376+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:20:45.041782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525718+0000","last_change":"2023-11-26T07:12:54.525718+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525718+0000","last_undegraded":"2023-11-26T07:12:54.525718+0000","last_fullsized":"2023-11-26T07:12:54.525718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542408+0000","last_change":"2023-11-26T07:12:54.531548+0000","last_active":"2023-11-26T07:12:54.542408+0000","last_peered":"2023-11-26T07:12:54.542408+0000","last_clean":"2023-11-26T07:12:54.542408+0000","last_became_active":"2023-11-26T07:12:54.531133+0000","last_became_peered":"2023-11-26T07:12:54.531133+0000","last_unstale":"2023-11-26T07:12:54.542408+0000","last_undegraded":"2023-11-26T07:12:54.542408+0000","last_fullsized":"2023-11-26T07:12:54.542408+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:58.991270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519968+0000","last_change":"2023-11-26T07:12:54.519968+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519968+0000","last_undegraded":"2023-11-26T07:12:54.519968+0000","last_fullsized":"2023-11-26T07:12:54.519968+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541930+0000","last_change":"2023-11-26T07:12:54.536843+0000","last_active":"2023-11-26T07:12:54.541930+0000","last_peered":"2023-11-26T07:12:54.541930+0000","last_clean":"2023-11-26T07:12:54.541930+0000","last_became_active":"2023-11-26T07:12:54.536560+0000","last_became_peered":"2023-11-26T07:12:54.536560+0000","last_unstale":"2023-11-26T07:12:54.541930+0000","last_undegraded":"2023-11-26T07:12:54.541930+0000","last_fullsized":"2023-11-26T07:12:54.541930+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:47.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519772+0000","last_change":"2023-11-26T07:12:54.519772+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519772+0000","last_undegraded":"2023-11-26T07:12:54.519772+0000","last_fullsized":"2023-11-26T07:12:54.519772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.529907+0000","last_change":"2023-11-26T07:12:54.527254+0000","last_active":"2023-11-26T07:12:54.529907+0000","last_peered":"2023-11-26T07:12:54.529907+0000","last_clean":"2023-11-26T07:12:54.529907+0000","last_became_active":"2023-11-26T07:12:54.526949+0000","last_became_peered":"2023-11-26T07:12:54.526949+0000","last_unstale":"2023-11-26T07:12:54.529907+0000","last_undegraded":"2023-11-26T07:12:54.529907+0000","last_fullsized":"2023-11-26T07:12:54.529907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:20:37.591566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519987+0000","last_change":"2023-11-26T07:12:54.519987+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519987+0000","last_undegraded":"2023-11-26T07:12:54.519987+0000","last_fullsized":"2023-11-26T07:12:54.519987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536115+0000","last_change":"2023-11-26T07:12:54.535255+0000","last_active":"2023-11-26T07:12:54.536115+0000","last_peered":"2023-11-26T07:12:54.536115+0000","last_clean":"2023-11-26T07:12:54.536115+0000","last_became_active":"2023-11-26T07:12:54.534934+0000","last_became_peered":"2023-11-26T07:12:54.534934+0000","last_unstale":"2023-11-26T07:12:54.536115+0000","last_undegraded":"2023-11-26T07:12:54.536115+0000","last_fullsized":"2023-11-26T07:12:54.536115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:59.222804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526457+0000","last_change":"2023-11-26T07:12:54.526457+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526457+0000","last_undegraded":"2023-11-26T07:12:54.526457+0000","last_fullsized":"2023-11-26T07:12:54.526457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534475+0000","last_change":"2023-11-26T07:12:54.531509+0000","last_active":"2023-11-26T07:12:54.534475+0000","last_peered":"2023-11-26T07:12:54.534475+0000","last_clean":"2023-11-26T07:12:54.534475+0000","last_became_active":"2023-11-26T07:12:54.531168+0000","last_became_peered":"2023-11-26T07:12:54.531168+0000","last_unstale":"2023-11-26T07:12:54.534475+0000","last_undegraded":"2023-11-26T07:12:54.534475+0000","last_fullsized":"2023-11-26T07:12:54.534475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:45.828740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.527721+0000","last_change":"2023-11-26T07:12:54.527721+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.527721+0000","last_undegraded":"2023-11-26T07:12:54.527721+0000","last_fullsized":"2023-11-26T07:12:54.527721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542544+0000","last_change":"2023-11-26T07:12:54.536077+0000","last_active":"2023-11-26T07:12:54.542544+0000","last_peered":"2023-11-26T07:12:54.542544+0000","last_clean":"2023-11-26T07:12:54.542544+0000","last_became_active":"2023-11-26T07:12:54.535793+0000","last_became_peered":"2023-11-26T07:12:54.535793+0000","last_unstale":"2023-11-26T07:12:54.542544+0000","last_undegraded":"2023-11-26T07:12:54.542544+0000","last_fullsized":"2023-11-26T07:12:54.542544+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:27:43.076531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529399+0000","last_change":"2023-11-26T07:12:54.529399+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529399+0000","last_undegraded":"2023-11-26T07:12:54.529399+0000","last_fullsized":"2023-11-26T07:12:54.529399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542631+0000","last_change":"2023-11-26T07:12:54.537136+0000","last_active":"2023-11-26T07:12:54.542631+0000","last_peered":"2023-11-26T07:12:54.542631+0000","last_clean":"2023-11-26T07:12:54.542631+0000","last_became_active":"2023-11-26T07:12:54.536557+0000","last_became_peered":"2023-11-26T07:12:54.536557+0000","last_unstale":"2023-11-26T07:12:54.542631+0000","last_undegraded":"2023-11-26T07:12:54.542631+0000","last_fullsized":"2023-11-26T07:12:54.542631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:58:57.698467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519560+0000","last_change":"2023-11-26T07:12:54.519560+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519560+0000","last_undegraded":"2023-11-26T07:12:54.519560+0000","last_fullsized":"2023-11-26T07:12:54.519560+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536191+0000","last_change":"2023-11-26T07:12:54.533932+0000","last_active":"2023-11-26T07:12:54.536191+0000","last_peered":"2023-11-26T07:12:54.536191+0000","last_clean":"2023-11-26T07:12:54.536191+0000","last_became_active":"2023-11-26T07:12:54.533651+0000","last_became_peered":"2023-11-26T07:12:54.533651+0000","last_unstale":"2023-11-26T07:12:54.536191+0000","last_undegraded":"2023-11-26T07:12:54.536191+0000","last_fullsized":"2023-11-26T07:12:54.536191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:16.433050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520815+0000","last_change":"2023-11-26T07:12:54.520815+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520815+0000","last_undegraded":"2023-11-26T07:12:54.520815+0000","last_fullsized":"2023-11-26T07:12:54.520815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541674+0000","last_change":"2023-11-26T07:12:54.533722+0000","last_active":"2023-11-26T07:12:54.541674+0000","last_peered":"2023-11-26T07:12:54.541674+0000","last_clean":"2023-11-26T07:12:54.541674+0000","last_became_active":"2023-11-26T07:12:54.533207+0000","last_became_peered":"2023-11-26T07:12:54.533207+0000","last_unstale":"2023-11-26T07:12:54.541674+0000","last_undegraded":"2023-11-26T07:12:54.541674+0000","last_fullsized":"2023-11-26T07:12:54.541674+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:03.463480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521433+0000","last_change":"2023-11-26T07:12:54.521433+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521433+0000","last_undegraded":"2023-11-26T07:12:54.521433+0000","last_fullsized":"2023-11-26T07:12:54.521433+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540251+0000","last_change":"2023-11-26T07:12:54.535338+0000","last_active":"2023-11-26T07:12:54.540251+0000","last_peered":"2023-11-26T07:12:54.540251+0000","last_clean":"2023-11-26T07:12:54.540251+0000","last_became_active":"2023-11-26T07:12:54.534988+0000","last_became_peered":"2023-11-26T07:12:54.534988+0000","last_unstale":"2023-11-26T07:12:54.540251+0000","last_undegraded":"2023-11-26T07:12:54.540251+0000","last_fullsized":"2023-11-26T07:12:54.540251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:58:57.698467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526021+0000","last_change":"2023-11-26T07:12:54.526021+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526021+0000","last_undegraded":"2023-11-26T07:12:54.526021+0000","last_fullsized":"2023-11-26T07:12:54.526021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542447+0000","last_change":"2023-11-26T07:12:54.531775+0000","last_active":"2023-11-26T07:12:54.542447+0000","last_peered":"2023-11-26T07:12:54.542447+0000","last_clean":"2023-11-26T07:12:54.542447+0000","last_became_active":"2023-11-26T07:12:54.531491+0000","last_became_peered":"2023-11-26T07:12:54.531491+0000","last_unstale":"2023-11-26T07:12:54.542447+0000","last_undegraded":"2023-11-26T07:12:54.542447+0000","last_fullsized":"2023-11-26T07:12:54.542447+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:45.828740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520515+0000","last_change":"2023-11-26T07:12:54.520515+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520515+0000","last_undegraded":"2023-11-26T07:12:54.520515+0000","last_fullsized":"2023-11-26T07:12:54.520515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540202+0000","last_change":"2023-11-26T07:12:54.534426+0000","last_active":"2023-11-26T07:12:54.540202+0000","last_peered":"2023-11-26T07:12:54.540202+0000","last_clean":"2023-11-26T07:12:54.540202+0000","last_became_active":"2023-11-26T07:12:54.534154+0000","last_became_peered":"2023-11-26T07:12:54.534154+0000","last_unstale":"2023-11-26T07:12:54.540202+0000","last_undegraded":"2023-11-26T07:12:54.540202+0000","last_fullsized":"2023-11-26T07:12:54.540202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:38:16.665106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519918+0000","last_change":"2023-11-26T07:12:54.519918+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519918+0000","last_undegraded":"2023-11-26T07:12:54.519918+0000","last_fullsized":"2023-11-26T07:12:54.519918+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534342+0000","last_change":"2023-11-26T07:12:54.529524+0000","last_active":"2023-11-26T07:12:54.534342+0000","last_peered":"2023-11-26T07:12:54.534342+0000","last_clean":"2023-11-26T07:12:54.534342+0000","last_became_active":"2023-11-26T07:12:54.529161+0000","last_became_peered":"2023-11-26T07:12:54.529161+0000","last_unstale":"2023-11-26T07:12:54.534342+0000","last_undegraded":"2023-11-26T07:12:54.534342+0000","last_fullsized":"2023-11-26T07:12:54.534342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:04.125066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.528822+0000","last_change":"2023-11-26T07:12:54.528822+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.528822+0000","last_undegraded":"2023-11-26T07:12:54.528822+0000","last_fullsized":"2023-11-26T07:12:54.528822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540137+0000","last_change":"2023-11-26T07:12:54.533531+0000","last_active":"2023-11-26T07:12:54.540137+0000","last_peered":"2023-11-26T07:12:54.540137+0000","last_clean":"2023-11-26T07:12:54.540137+0000","last_became_active":"2023-11-26T07:12:54.533142+0000","last_became_peered":"2023-11-26T07:12:54.533142+0000","last_unstale":"2023-11-26T07:12:54.540137+0000","last_undegraded":"2023-11-26T07:12:54.540137+0000","last_fullsized":"2023-11-26T07:12:54.540137+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:01:33.193390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526115+0000","last_change":"2023-11-26T07:12:54.526115+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526115+0000","last_undegraded":"2023-11-26T07:12:54.526115+0000","last_fullsized":"2023-11-26T07:12:54.526115+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.528932+0000","last_change":"2023-11-26T07:12:54.525145+0000","last_active":"2023-11-26T07:12:54.528932+0000","last_peered":"2023-11-26T07:12:54.528932+0000","last_clean":"2023-11-26T07:12:54.528932+0000","last_became_active":"2023-11-26T07:12:54.524871+0000","last_became_peered":"2023-11-26T07:12:54.524871+0000","last_unstale":"2023-11-26T07:12:54.528932+0000","last_undegraded":"2023-11-26T07:12:54.528932+0000","last_fullsized":"2023-11-26T07:12:54.528932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:48:46.369669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521333+0000","last_change":"2023-11-26T07:12:54.521333+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521333+0000","last_undegraded":"2023-11-26T07:12:54.521333+0000","last_fullsized":"2023-11-26T07:12:54.521333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542409+0000","last_change":"2023-11-26T07:12:54.533682+0000","last_active":"2023-11-26T07:12:54.542409+0000","last_peered":"2023-11-26T07:12:54.542409+0000","last_clean":"2023-11-26T07:12:54.542409+0000","last_became_active":"2023-11-26T07:12:54.533409+0000","last_became_peered":"2023-11-26T07:12:54.533409+0000","last_unstale":"2023-11-26T07:12:54.542409+0000","last_undegraded":"2023-11-26T07:12:54.542409+0000","last_fullsized":"2023-11-26T07:12:54.542409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:23:22.243947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518436+0000","last_change":"2023-11-26T07:12:54.518436+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518436+0000","last_undegraded":"2023-11-26T07:12:54.518436+0000","last_fullsized":"2023-11-26T07:12:54.518436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541952+0000","last_change":"2023-11-26T07:12:54.537875+0000","last_active":"2023-11-26T07:12:54.541952+0000","last_peered":"2023-11-26T07:12:54.541952+0000","last_clean":"2023-11-26T07:12:54.541952+0000","last_became_active":"2023-11-26T07:12:54.537498+0000","last_became_peered":"2023-11-26T07:12:54.537498+0000","last_unstale":"2023-11-26T07:12:54.541952+0000","last_undegraded":"2023-11-26T07:12:54.541952+0000","last_fullsized":"2023-11-26T07:12:54.541952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:31.377712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520632+0000","last_change":"2023-11-26T07:12:54.520632+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520632+0000","last_undegraded":"2023-11-26T07:12:54.520632+0000","last_fullsized":"2023-11-26T07:12:54.520632+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.531028+0000","last_change":"2023-11-26T07:12:54.528129+0000","last_active":"2023-11-26T07:12:54.531028+0000","last_peered":"2023-11-26T07:12:54.531028+0000","last_clean":"2023-11-26T07:12:54.531028+0000","last_became_active":"2023-11-26T07:12:54.527835+0000","last_became_peered":"2023-11-26T07:12:54.527835+0000","last_unstale":"2023-11-26T07:12:54.531028+0000","last_undegraded":"2023-11-26T07:12:54.531028+0000","last_fullsized":"2023-11-26T07:12:54.531028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:02:54.984506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-26T07:12:53.513412+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:12:53.513412+0000","last_peered":"2023-11-26T07:12:53.513412+0000","last_clean":"2023-11-26T07:12:53.513412+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:12:53.513412+0000","last_undegraded":"2023-11-26T07:12:53.513412+0000","last_fullsized":"2023-11-26T07:12:53.513412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.497}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55800000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.433},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.39800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.63}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.503},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.40300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.57199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.31}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.24399999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.504}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":791748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.313}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.36699999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39800000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.621}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58399999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.253},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.313}]},{"osd":1,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.40899999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.59299999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.628}]},{"osd":6,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.48599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.34799999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.9109999999999996,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.313}]},{"osd":2,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.247},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.32800000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8129999999999997,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.71799999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.883,"5min":4.883,"15min":4.883},"last":0.60699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.8499999999999996,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.67000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47399999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.61099999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.495},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.747}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.60499999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54300000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.621},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.64200000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.67300000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57399999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:12:59.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:12:59.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.806+0000 7f05f2148700 1 Processor -- start 2023-11-26T07:12:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.807+0000 7f05f2148700 1 -- start start 2023-11-26T07:12:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.807+0000 7f05f2148700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ec0b3370 0x7f05ec1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.813 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.807+0000 7f05f2148700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f05ec0d1c70 0x7f05ec0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.807+0000 7f05f2148700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec0b22d0 con 0x7f05ec15bdd0 2023-11-26T07:12:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.807+0000 7f05f2148700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec0b2f50 con 0x7f05ec0d1c70 2023-11-26T07:12:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.807+0000 7f05f2148700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0d2640 con 0x7f05ec0b3370 2023-11-26T07:12:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.807+0000 7f05eb7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f05ec15bdd0 0x7f05ec0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57212/0 (socket says 172.21.15.83:57212) 2023-11-26T07:12:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.807+0000 7f05ebfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f05ec0d1c70 0x7f05ec0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.807+0000 7f05eb7fe700 1 -- 172.21.15.83:0/2645236658 learned_addr learned my addr 172.21.15.83:0/2645236658 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:12:59.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.808+0000 7f05eaffd700 1 --2- 172.21.15.83:0/2645236658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ec0b3370 0x7f05ec1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.808+0000 7f05ebfff700 1 -- 172.21.15.83:0/2645236658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ec0b3370 msgr2=0x7f05ec1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:12:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.808+0000 7f05ebfff700 1 --2- 172.21.15.83:0/2645236658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ec0b3370 0x7f05ec1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.808+0000 7f05ebfff700 1 -- 172.21.15.83:0/2645236658 >> v1:172.21.15.83:6789/0 conn(0x7f05ec15bdd0 legacy=0x7f05ec0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.808+0000 7f05ebfff700 1 -- 172.21.15.83:0/2645236658 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0b25b0 con 0x7f05ec0d1c70 2023-11-26T07:12:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.808+0000 7f05ea7fc700 1 -- 172.21.15.83:0/2645236658 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282682205 0 0) 0x7f05ec0b22d0 con 0x7f05ec15bdd0 2023-11-26T07:12:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.808+0000 7f05ebfff700 1 --2- 172.21.15.83:0/2645236658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec0d1c70 0x7f05ec0d2100 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009d90 tx=0x7f05e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4be3bb597a7d01 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.808+0000 7f05ea7fc700 1 -- 172.21.15.83:0/2645236658 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e400e070 con 0x7f05ec0d1c70 2023-11-26T07:12:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.808+0000 7f05ea7fc700 1 -- 172.21.15.83:0/2645236658 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e400bbd0 con 0x7f05ec0d1c70 2023-11-26T07:12:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.808+0000 7f05ea7fc700 1 -- 172.21.15.83:0/2645236658 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4012430 con 0x7f05ec0d1c70 2023-11-26T07:12:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.809+0000 7f05f2148700 1 -- 172.21.15.83:0/2645236658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec0d1c70 msgr2=0x7f05ec0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.809+0000 7f05f2148700 1 --2- 172.21.15.83:0/2645236658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec0d1c70 0x7f05ec0d2100 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009d90 tx=0x7f05e400b330 comp rx=0 tx=0).stop 2023-11-26T07:12:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.809+0000 7f05f2148700 1 -- 172.21.15.83:0/2645236658 shutdown_connections 2023-11-26T07:12:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.809+0000 7f05f2148700 1 --2- 172.21.15.83:0/2645236658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec0d1c70 0x7f05ec0d2100 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.809+0000 7f05f2148700 1 --2- 172.21.15.83:0/2645236658 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ec0b3370 0x7f05ec1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.809+0000 7f05f2148700 1 -- 172.21.15.83:0/2645236658 >> 172.21.15.83:0/2645236658 conn(0x7f05ec0731a0 msgr2=0x7f05ec09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:12:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.809+0000 7f05f2148700 1 -- 172.21.15.83:0/2645236658 shutdown_connections 2023-11-26T07:12:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.809+0000 7f05f2148700 1 -- 172.21.15.83:0/2645236658 wait complete. 2023-11-26T07:12:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.810+0000 7f05f2148700 1 Processor -- start 2023-11-26T07:12:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.810+0000 7f05f2148700 1 -- start start 2023-11-26T07:12:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05f2148700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05ec0b3370 0x7f05ec1e5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05f2148700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec0d1c70 0x7f05ec1e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05f2148700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ec15bdd0 0x7f05ec0ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05f2148700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec1e7010 con 0x7f05ec0d1c70 2023-11-26T07:12:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05f2148700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec1e6640 con 0x7f05ec0b3370 2023-11-26T07:12:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05f2148700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec1e5a30 con 0x7f05ec15bdd0 2023-11-26T07:12:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05eaffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec0d1c70 0x7f05ec1e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05eaffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec0d1c70 0x7f05ec1e6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37180/0 (socket says 172.21.15.83:37180) 2023-11-26T07:12:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05eaffd700 1 -- 172.21.15.83:0/3686308130 learned_addr learned my addr 172.21.15.83:0/3686308130 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:12:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05eb7fe700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05ec0b3370 0x7f05ec1e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05ebfff700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ec15bdd0 0x7f05ec0ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05eaffd700 1 -- 172.21.15.83:0/3686308130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ec15bdd0 msgr2=0x7f05ec0ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05eaffd700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ec15bdd0 0x7f05ec0ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05eaffd700 1 -- 172.21.15.83:0/3686308130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05ec0b3370 msgr2=0x7f05ec1e5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:12:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05eaffd700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05ec0b3370 0x7f05ec1e5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:12:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05eaffd700 1 -- 172.21.15.83:0/3686308130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4009a70 con 0x7f05ec0d1c70 2023-11-26T07:12:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05ebfff700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ec15bdd0 0x7f05ec0ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:12:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05eaffd700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec0d1c70 0x7f05ec1e6ad0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f05d400e810 tx=0x7f05d400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05e3fff700 1 -- 172.21.15.83:0/3686308130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4013610 con 0x7f05ec0d1c70 2023-11-26T07:12:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05e3fff700 1 -- 172.21.15.83:0/3686308130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d4013770 con 0x7f05ec0d1c70 2023-11-26T07:12:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05f2148700 1 -- 172.21.15.83:0/3686308130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec1e5d10 con 0x7f05ec0d1c70 2023-11-26T07:12:59.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.811+0000 7f05e3fff700 1 -- 172.21.15.83:0/3686308130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4013b20 con 0x7f05ec0d1c70 2023-11-26T07:12:59.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.812+0000 7f05f2148700 1 -- 172.21.15.83:0/3686308130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec1e62d0 con 0x7f05ec0d1c70 2023-11-26T07:12:59.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.813+0000 7f05f2148700 1 -- 172.21.15.83:0/3686308130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec1e67b0 con 0x7f05ec0d1c70 2023-11-26T07:12:59.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.813+0000 7f05e3fff700 1 -- 172.21.15.83:0/3686308130 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f05d4013c80 con 0x7f05ec0d1c70 2023-11-26T07:12:59.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.813+0000 7f05e3fff700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05cc06b080 0x7f05cc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:12:59.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.814+0000 7f05e3fff700 1 -- 172.21.15.83:0/3686308130 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05d4089620 con 0x7f05ec0d1c70 2023-11-26T07:12:59.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.814+0000 7f05eb7fe700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05cc06b080 0x7f05cc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:12:59.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.814+0000 7f05eb7fe700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05cc06b080 0x7f05cc06d540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f05dc000f60 tx=0x7f05dc007e70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:12:59.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:12:59.818+0000 7f05e3fff700 1 -- 172.21.15.83:0/3686308130 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05d4054350 con 0x7f05ec0d1c70 2023-11-26T07:13:00.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.003+0000 7f05f2148700 1 -- 172.21.15.83:0/3686308130 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05ec0b2b70 con 0x7f05cc06b080 2023-11-26T07:13:00.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.010+0000 7f05e3fff700 1 -- 172.21.15.83:0/3686308130 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283717 (secure 0 0 0) 0x7f05dc001290 con 0x7f05cc06b080 2023-11-26T07:13:00.015 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.013+0000 7f05f2148700 1 -- 172.21.15.83:0/3686308130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05cc06b080 msgr2=0x7f05cc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.014+0000 7f05f2148700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05cc06b080 0x7f05cc06d540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f05dc000f60 tx=0x7f05dc007e70 comp rx=0 tx=0).stop 2023-11-26T07:13:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.014+0000 7f05f2148700 1 -- 172.21.15.83:0/3686308130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec0d1c70 msgr2=0x7f05ec1e6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.014+0000 7f05f2148700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec0d1c70 0x7f05ec1e6ad0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f05d400e810 tx=0x7f05d400eb20 comp rx=0 tx=0).stop 2023-11-26T07:13:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.014+0000 7f05f2148700 1 -- 172.21.15.83:0/3686308130 shutdown_connections 2023-11-26T07:13:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.014+0000 7f05f2148700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05cc06b080 0x7f05cc06d540 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.014+0000 7f05f2148700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05ec15bdd0 0x7f05ec0ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.014+0000 7f05f2148700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05ec0b3370 0x7f05ec1e5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.014+0000 7f05f2148700 1 --2- 172.21.15.83:0/3686308130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec0d1c70 0x7f05ec1e6ad0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.014+0000 7f05f2148700 1 -- 172.21.15.83:0/3686308130 >> 172.21.15.83:0/3686308130 conn(0x7f05ec0731a0 msgr2=0x7f05ec0b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.014+0000 7f05f2148700 1 -- 172.21.15.83:0/3686308130 shutdown_connections 2023-11-26T07:13:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.014+0000 7f05f2148700 1 -- 172.21.15.83:0/3686308130 wait complete. 2023-11-26T07:13:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:13:00.027 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-11-26T07:12:58.325188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221236,"kb_used_data":5048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753484,"statfs":{"total":773094113280,"available":772867567616,"internally_reserved":0,"allocated":5169152,"data_stored":2766708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.816391"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521879+0000","last_change":"2023-11-26T07:12:54.521879+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521879+0000","last_undegraded":"2023-11-26T07:12:54.521879+0000","last_fullsized":"2023-11-26T07:12:54.521879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521487+0000","last_change":"2023-11-26T07:12:54.521487+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521487+0000","last_undegraded":"2023-11-26T07:12:54.521487+0000","last_fullsized":"2023-11-26T07:12:54.521487+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519224+0000","last_change":"2023-11-26T07:12:54.519224+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519224+0000","last_undegraded":"2023-11-26T07:12:54.519224+0000","last_fullsized":"2023-11-26T07:12:54.519224+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523622+0000","last_change":"2023-11-26T07:12:54.523622+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523622+0000","last_undegraded":"2023-11-26T07:12:54.523622+0000","last_fullsized":"2023-11-26T07:12:54.523622+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521234+0000","last_change":"2023-11-26T07:12:54.521234+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521234+0000","last_undegraded":"2023-11-26T07:12:54.521234+0000","last_fullsized":"2023-11-26T07:12:54.521234+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529973+0000","last_change":"2023-11-26T07:12:54.529973+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529973+0000","last_undegraded":"2023-11-26T07:12:54.529973+0000","last_fullsized":"2023-11-26T07:12:54.529973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521276+0000","last_change":"2023-11-26T07:12:54.521276+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521276+0000","last_undegraded":"2023-11-26T07:12:54.521276+0000","last_fullsized":"2023-11-26T07:12:54.521276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526287+0000","last_change":"2023-11-26T07:12:54.526287+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526287+0000","last_undegraded":"2023-11-26T07:12:54.526287+0000","last_fullsized":"2023-11-26T07:12:54.526287+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523760+0000","last_change":"2023-11-26T07:12:54.523760+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523760+0000","last_undegraded":"2023-11-26T07:12:54.523760+0000","last_fullsized":"2023-11-26T07:12:54.523760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.524402+0000","last_change":"2023-11-26T07:12:54.524402+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.524402+0000","last_undegraded":"2023-11-26T07:12:54.524402+0000","last_fullsized":"2023-11-26T07:12:54.524402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523524+0000","last_change":"2023-11-26T07:12:54.523524+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523524+0000","last_undegraded":"2023-11-26T07:12:54.523524+0000","last_fullsized":"2023-11-26T07:12:54.523524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521618+0000","last_change":"2023-11-26T07:12:54.521618+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521618+0000","last_undegraded":"2023-11-26T07:12:54.521618+0000","last_fullsized":"2023-11-26T07:12:54.521618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520013+0000","last_change":"2023-11-26T07:12:54.520013+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520013+0000","last_undegraded":"2023-11-26T07:12:54.520013+0000","last_fullsized":"2023-11-26T07:12:54.520013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523982+0000","last_change":"2023-11-26T07:12:54.523982+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523982+0000","last_undegraded":"2023-11-26T07:12:54.523982+0000","last_fullsized":"2023-11-26T07:12:54.523982+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.530689+0000","last_change":"2023-11-26T07:12:54.530689+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.530689+0000","last_undegraded":"2023-11-26T07:12:54.530689+0000","last_fullsized":"2023-11-26T07:12:54.530689+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526270+0000","last_change":"2023-11-26T07:12:54.526270+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526270+0000","last_undegraded":"2023-11-26T07:12:54.526270+0000","last_fullsized":"2023-11-26T07:12:54.526270+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518831+0000","last_change":"2023-11-26T07:12:54.518831+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518831+0000","last_undegraded":"2023-11-26T07:12:54.518831+0000","last_fullsized":"2023-11-26T07:12:54.518831+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520492+0000","last_change":"2023-11-26T07:12:54.520492+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520492+0000","last_undegraded":"2023-11-26T07:12:54.520492+0000","last_fullsized":"2023-11-26T07:12:54.520492+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.528121+0000","last_change":"2023-11-26T07:12:54.528121+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.528121+0000","last_undegraded":"2023-11-26T07:12:54.528121+0000","last_fullsized":"2023-11-26T07:12:54.528121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520226+0000","last_change":"2023-11-26T07:12:54.520226+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520226+0000","last_undegraded":"2023-11-26T07:12:54.520226+0000","last_fullsized":"2023-11-26T07:12:54.520226+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529224+0000","last_change":"2023-11-26T07:12:54.529224+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529224+0000","last_undegraded":"2023-11-26T07:12:54.529224+0000","last_fullsized":"2023-11-26T07:12:54.529224+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523489+0000","last_change":"2023-11-26T07:12:54.523489+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523489+0000","last_undegraded":"2023-11-26T07:12:54.523489+0000","last_fullsized":"2023-11-26T07:12:54.523489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.530050+0000","last_change":"2023-11-26T07:12:54.530050+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.530050+0000","last_undegraded":"2023-11-26T07:12:54.530050+0000","last_fullsized":"2023-11-26T07:12:54.530050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525496+0000","last_change":"2023-11-26T07:12:54.525496+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525496+0000","last_undegraded":"2023-11-26T07:12:54.525496+0000","last_fullsized":"2023-11-26T07:12:54.525496+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525484+0000","last_change":"2023-11-26T07:12:54.525484+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525484+0000","last_undegraded":"2023-11-26T07:12:54.525484+0000","last_fullsized":"2023-11-26T07:12:54.525484+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520283+0000","last_change":"2023-11-26T07:12:54.520283+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520283+0000","last_undegraded":"2023-11-26T07:12:54.520283+0000","last_fullsized":"2023-11-26T07:12:54.520283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518411+0000","last_change":"2023-11-26T07:12:54.518411+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518411+0000","last_undegraded":"2023-11-26T07:12:54.518411+0000","last_fullsized":"2023-11-26T07:12:54.518411+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522587+0000","last_change":"2023-11-26T07:12:54.522587+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522587+0000","last_undegraded":"2023-11-26T07:12:54.522587+0000","last_fullsized":"2023-11-26T07:12:54.522587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526264+0000","last_change":"2023-11-26T07:12:54.526264+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526264+0000","last_undegraded":"2023-11-26T07:12:54.526264+0000","last_fullsized":"2023-11-26T07:12:54.526264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522172+0000","last_change":"2023-11-26T07:12:54.522172+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522172+0000","last_undegraded":"2023-11-26T07:12:54.522172+0000","last_fullsized":"2023-11-26T07:12:54.522172+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526169+0000","last_change":"2023-11-26T07:12:54.526169+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526169+0000","last_undegraded":"2023-11-26T07:12:54.526169+0000","last_fullsized":"2023-11-26T07:12:54.526169+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521324+0000","last_change":"2023-11-26T07:12:54.521324+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521324+0000","last_undegraded":"2023-11-26T07:12:54.521324+0000","last_fullsized":"2023-11-26T07:12:54.521324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521871+0000","last_change":"2023-11-26T07:12:54.521871+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521871+0000","last_undegraded":"2023-11-26T07:12:54.521871+0000","last_fullsized":"2023-11-26T07:12:54.521871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542574+0000","last_change":"2023-11-26T07:12:54.536815+0000","last_active":"2023-11-26T07:12:54.542574+0000","last_peered":"2023-11-26T07:12:54.542574+0000","last_clean":"2023-11-26T07:12:54.542574+0000","last_became_active":"2023-11-26T07:12:54.536526+0000","last_became_peered":"2023-11-26T07:12:54.536526+0000","last_unstale":"2023-11-26T07:12:54.542574+0000","last_undegraded":"2023-11-26T07:12:54.542574+0000","last_fullsized":"2023-11-26T07:12:54.542574+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:20:45.041782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526811+0000","last_change":"2023-11-26T07:12:54.526811+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526811+0000","last_undegraded":"2023-11-26T07:12:54.526811+0000","last_fullsized":"2023-11-26T07:12:54.526811+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536052+0000","last_change":"2023-11-26T07:12:54.533729+0000","last_active":"2023-11-26T07:12:54.536052+0000","last_peered":"2023-11-26T07:12:54.536052+0000","last_clean":"2023-11-26T07:12:54.536052+0000","last_became_active":"2023-11-26T07:12:54.533440+0000","last_became_peered":"2023-11-26T07:12:54.533440+0000","last_unstale":"2023-11-26T07:12:54.536052+0000","last_undegraded":"2023-11-26T07:12:54.536052+0000","last_fullsized":"2023-11-26T07:12:54.536052+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:04.125066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522115+0000","last_change":"2023-11-26T07:12:54.522115+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522115+0000","last_undegraded":"2023-11-26T07:12:54.522115+0000","last_fullsized":"2023-11-26T07:12:54.522115+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534324+0000","last_change":"2023-11-26T07:12:54.529395+0000","last_active":"2023-11-26T07:12:54.534324+0000","last_peered":"2023-11-26T07:12:54.534324+0000","last_clean":"2023-11-26T07:12:54.534324+0000","last_became_active":"2023-11-26T07:12:54.529093+0000","last_became_peered":"2023-11-26T07:12:54.529093+0000","last_unstale":"2023-11-26T07:12:54.534324+0000","last_undegraded":"2023-11-26T07:12:54.534324+0000","last_fullsized":"2023-11-26T07:12:54.534324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:59.222804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520849+0000","last_change":"2023-11-26T07:12:54.520849+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520849+0000","last_undegraded":"2023-11-26T07:12:54.520849+0000","last_fullsized":"2023-11-26T07:12:54.520849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.532244+0000","last_change":"2023-11-26T07:12:54.529608+0000","last_active":"2023-11-26T07:12:54.532244+0000","last_peered":"2023-11-26T07:12:54.532244+0000","last_clean":"2023-11-26T07:12:54.532244+0000","last_became_active":"2023-11-26T07:12:54.529334+0000","last_became_peered":"2023-11-26T07:12:54.529334+0000","last_unstale":"2023-11-26T07:12:54.532244+0000","last_undegraded":"2023-11-26T07:12:54.532244+0000","last_fullsized":"2023-11-26T07:12:54.532244+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:27:43.076531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519440+0000","last_change":"2023-11-26T07:12:54.519440+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519440+0000","last_undegraded":"2023-11-26T07:12:54.519440+0000","last_fullsized":"2023-11-26T07:12:54.519440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542468+0000","last_change":"2023-11-26T07:12:54.532202+0000","last_active":"2023-11-26T07:12:54.542468+0000","last_peered":"2023-11-26T07:12:54.542468+0000","last_clean":"2023-11-26T07:12:54.542468+0000","last_became_active":"2023-11-26T07:12:54.531692+0000","last_became_peered":"2023-11-26T07:12:54.531692+0000","last_unstale":"2023-11-26T07:12:54.542468+0000","last_undegraded":"2023-11-26T07:12:54.542468+0000","last_fullsized":"2023-11-26T07:12:54.542468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:16.433050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526413+0000","last_change":"2023-11-26T07:12:54.526413+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526413+0000","last_undegraded":"2023-11-26T07:12:54.526413+0000","last_fullsized":"2023-11-26T07:12:54.526413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534545+0000","last_change":"2023-11-26T07:12:54.529445+0000","last_active":"2023-11-26T07:12:54.534545+0000","last_peered":"2023-11-26T07:12:54.534545+0000","last_clean":"2023-11-26T07:12:54.534545+0000","last_became_active":"2023-11-26T07:12:54.529160+0000","last_became_peered":"2023-11-26T07:12:54.529160+0000","last_unstale":"2023-11-26T07:12:54.534545+0000","last_undegraded":"2023-11-26T07:12:54.534545+0000","last_fullsized":"2023-11-26T07:12:54.534545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:45:17.796276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525851+0000","last_change":"2023-11-26T07:12:54.525851+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525851+0000","last_undegraded":"2023-11-26T07:12:54.525851+0000","last_fullsized":"2023-11-26T07:12:54.525851+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534715+0000","last_change":"2023-11-26T07:12:54.531344+0000","last_active":"2023-11-26T07:12:54.534715+0000","last_peered":"2023-11-26T07:12:54.534715+0000","last_clean":"2023-11-26T07:12:54.534715+0000","last_became_active":"2023-11-26T07:12:54.530914+0000","last_became_peered":"2023-11-26T07:12:54.530914+0000","last_unstale":"2023-11-26T07:12:54.534715+0000","last_undegraded":"2023-11-26T07:12:54.534715+0000","last_fullsized":"2023-11-26T07:12:54.534715+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:58.991270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529503+0000","last_change":"2023-11-26T07:12:54.529503+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529503+0000","last_undegraded":"2023-11-26T07:12:54.529503+0000","last_fullsized":"2023-11-26T07:12:54.529503+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541649+0000","last_change":"2023-11-26T07:12:54.533250+0000","last_active":"2023-11-26T07:12:54.541649+0000","last_peered":"2023-11-26T07:12:54.541649+0000","last_clean":"2023-11-26T07:12:54.541649+0000","last_became_active":"2023-11-26T07:12:54.532934+0000","last_became_peered":"2023-11-26T07:12:54.532934+0000","last_unstale":"2023-11-26T07:12:54.541649+0000","last_undegraded":"2023-11-26T07:12:54.541649+0000","last_fullsized":"2023-11-26T07:12:54.541649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:15:29.001269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518952+0000","last_change":"2023-11-26T07:12:54.518952+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518952+0000","last_undegraded":"2023-11-26T07:12:54.518952+0000","last_fullsized":"2023-11-26T07:12:54.518952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.530096+0000","last_change":"2023-11-26T07:12:54.528493+0000","last_active":"2023-11-26T07:12:54.530096+0000","last_peered":"2023-11-26T07:12:54.530096+0000","last_clean":"2023-11-26T07:12:54.530096+0000","last_became_active":"2023-11-26T07:12:54.527916+0000","last_became_peered":"2023-11-26T07:12:54.527916+0000","last_unstale":"2023-11-26T07:12:54.530096+0000","last_undegraded":"2023-11-26T07:12:54.530096+0000","last_fullsized":"2023-11-26T07:12:54.530096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:23:22.243947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520365+0000","last_change":"2023-11-26T07:12:54.520365+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520365+0000","last_undegraded":"2023-11-26T07:12:54.520365+0000","last_fullsized":"2023-11-26T07:12:54.520365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541667+0000","last_change":"2023-11-26T07:12:54.529990+0000","last_active":"2023-11-26T07:12:54.541667+0000","last_peered":"2023-11-26T07:12:54.541667+0000","last_clean":"2023-11-26T07:12:54.541667+0000","last_became_active":"2023-11-26T07:12:54.529727+0000","last_became_peered":"2023-11-26T07:12:54.529727+0000","last_unstale":"2023-11-26T07:12:54.541667+0000","last_undegraded":"2023-11-26T07:12:54.541667+0000","last_fullsized":"2023-11-26T07:12:54.541667+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:41:21.117692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525687+0000","last_change":"2023-11-26T07:12:54.525687+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525687+0000","last_undegraded":"2023-11-26T07:12:54.525687+0000","last_fullsized":"2023-11-26T07:12:54.525687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540184+0000","last_change":"2023-11-26T07:12:54.534123+0000","last_active":"2023-11-26T07:12:54.540184+0000","last_peered":"2023-11-26T07:12:54.540184+0000","last_clean":"2023-11-26T07:12:54.540184+0000","last_became_active":"2023-11-26T07:12:54.533831+0000","last_became_peered":"2023-11-26T07:12:54.533831+0000","last_unstale":"2023-11-26T07:12:54.540184+0000","last_undegraded":"2023-11-26T07:12:54.540184+0000","last_fullsized":"2023-11-26T07:12:54.540184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:25:35.938501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522255+0000","last_change":"2023-11-26T07:12:54.522255+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522255+0000","last_undegraded":"2023-11-26T07:12:54.522255+0000","last_fullsized":"2023-11-26T07:12:54.522255+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.531202+0000","last_change":"2023-11-26T07:12:54.528248+0000","last_active":"2023-11-26T07:12:54.531202+0000","last_peered":"2023-11-26T07:12:54.531202+0000","last_clean":"2023-11-26T07:12:54.531202+0000","last_became_active":"2023-11-26T07:12:54.527887+0000","last_became_peered":"2023-11-26T07:12:54.527887+0000","last_unstale":"2023-11-26T07:12:54.531202+0000","last_undegraded":"2023-11-26T07:12:54.531202+0000","last_fullsized":"2023-11-26T07:12:54.531202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:52.719388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.527948+0000","last_change":"2023-11-26T07:12:54.527948+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.527948+0000","last_undegraded":"2023-11-26T07:12:54.527948+0000","last_fullsized":"2023-11-26T07:12:54.527948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542608+0000","last_change":"2023-11-26T07:12:54.537145+0000","last_active":"2023-11-26T07:12:54.542608+0000","last_peered":"2023-11-26T07:12:54.542608+0000","last_clean":"2023-11-26T07:12:54.542608+0000","last_became_active":"2023-11-26T07:12:54.536921+0000","last_became_peered":"2023-11-26T07:12:54.536921+0000","last_unstale":"2023-11-26T07:12:54.542608+0000","last_undegraded":"2023-11-26T07:12:54.542608+0000","last_fullsized":"2023-11-26T07:12:54.542608+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:02:54.984506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521236+0000","last_change":"2023-11-26T07:12:54.521236+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521236+0000","last_undegraded":"2023-11-26T07:12:54.521236+0000","last_fullsized":"2023-11-26T07:12:54.521236+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536376+0000","last_change":"2023-11-26T07:12:54.532202+0000","last_active":"2023-11-26T07:12:54.536376+0000","last_peered":"2023-11-26T07:12:54.536376+0000","last_clean":"2023-11-26T07:12:54.536376+0000","last_became_active":"2023-11-26T07:12:54.531841+0000","last_became_peered":"2023-11-26T07:12:54.531841+0000","last_unstale":"2023-11-26T07:12:54.536376+0000","last_undegraded":"2023-11-26T07:12:54.536376+0000","last_fullsized":"2023-11-26T07:12:54.536376+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:20:45.041782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525718+0000","last_change":"2023-11-26T07:12:54.525718+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525718+0000","last_undegraded":"2023-11-26T07:12:54.525718+0000","last_fullsized":"2023-11-26T07:12:54.525718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542408+0000","last_change":"2023-11-26T07:12:54.531548+0000","last_active":"2023-11-26T07:12:54.542408+0000","last_peered":"2023-11-26T07:12:54.542408+0000","last_clean":"2023-11-26T07:12:54.542408+0000","last_became_active":"2023-11-26T07:12:54.531133+0000","last_became_peered":"2023-11-26T07:12:54.531133+0000","last_unstale":"2023-11-26T07:12:54.542408+0000","last_undegraded":"2023-11-26T07:12:54.542408+0000","last_fullsized":"2023-11-26T07:12:54.542408+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:58.991270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519968+0000","last_change":"2023-11-26T07:12:54.519968+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519968+0000","last_undegraded":"2023-11-26T07:12:54.519968+0000","last_fullsized":"2023-11-26T07:12:54.519968+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541930+0000","last_change":"2023-11-26T07:12:54.536843+0000","last_active":"2023-11-26T07:12:54.541930+0000","last_peered":"2023-11-26T07:12:54.541930+0000","last_clean":"2023-11-26T07:12:54.541930+0000","last_became_active":"2023-11-26T07:12:54.536560+0000","last_became_peered":"2023-11-26T07:12:54.536560+0000","last_unstale":"2023-11-26T07:12:54.541930+0000","last_undegraded":"2023-11-26T07:12:54.541930+0000","last_fullsized":"2023-11-26T07:12:54.541930+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:47.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519772+0000","last_change":"2023-11-26T07:12:54.519772+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519772+0000","last_undegraded":"2023-11-26T07:12:54.519772+0000","last_fullsized":"2023-11-26T07:12:54.519772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.529907+0000","last_change":"2023-11-26T07:12:54.527254+0000","last_active":"2023-11-26T07:12:54.529907+0000","last_peered":"2023-11-26T07:12:54.529907+0000","last_clean":"2023-11-26T07:12:54.529907+0000","last_became_active":"2023-11-26T07:12:54.526949+0000","last_became_peered":"2023-11-26T07:12:54.526949+0000","last_unstale":"2023-11-26T07:12:54.529907+0000","last_undegraded":"2023-11-26T07:12:54.529907+0000","last_fullsized":"2023-11-26T07:12:54.529907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:20:37.591566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519987+0000","last_change":"2023-11-26T07:12:54.519987+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519987+0000","last_undegraded":"2023-11-26T07:12:54.519987+0000","last_fullsized":"2023-11-26T07:12:54.519987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536115+0000","last_change":"2023-11-26T07:12:54.535255+0000","last_active":"2023-11-26T07:12:54.536115+0000","last_peered":"2023-11-26T07:12:54.536115+0000","last_clean":"2023-11-26T07:12:54.536115+0000","last_became_active":"2023-11-26T07:12:54.534934+0000","last_became_peered":"2023-11-26T07:12:54.534934+0000","last_unstale":"2023-11-26T07:12:54.536115+0000","last_undegraded":"2023-11-26T07:12:54.536115+0000","last_fullsized":"2023-11-26T07:12:54.536115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:59.222804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526457+0000","last_change":"2023-11-26T07:12:54.526457+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526457+0000","last_undegraded":"2023-11-26T07:12:54.526457+0000","last_fullsized":"2023-11-26T07:12:54.526457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534475+0000","last_change":"2023-11-26T07:12:54.531509+0000","last_active":"2023-11-26T07:12:54.534475+0000","last_peered":"2023-11-26T07:12:54.534475+0000","last_clean":"2023-11-26T07:12:54.534475+0000","last_became_active":"2023-11-26T07:12:54.531168+0000","last_became_peered":"2023-11-26T07:12:54.531168+0000","last_unstale":"2023-11-26T07:12:54.534475+0000","last_undegraded":"2023-11-26T07:12:54.534475+0000","last_fullsized":"2023-11-26T07:12:54.534475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:45.828740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.527721+0000","last_change":"2023-11-26T07:12:54.527721+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.527721+0000","last_undegraded":"2023-11-26T07:12:54.527721+0000","last_fullsized":"2023-11-26T07:12:54.527721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542544+0000","last_change":"2023-11-26T07:12:54.536077+0000","last_active":"2023-11-26T07:12:54.542544+0000","last_peered":"2023-11-26T07:12:54.542544+0000","last_clean":"2023-11-26T07:12:54.542544+0000","last_became_active":"2023-11-26T07:12:54.535793+0000","last_became_peered":"2023-11-26T07:12:54.535793+0000","last_unstale":"2023-11-26T07:12:54.542544+0000","last_undegraded":"2023-11-26T07:12:54.542544+0000","last_fullsized":"2023-11-26T07:12:54.542544+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:27:43.076531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529399+0000","last_change":"2023-11-26T07:12:54.529399+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529399+0000","last_undegraded":"2023-11-26T07:12:54.529399+0000","last_fullsized":"2023-11-26T07:12:54.529399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542631+0000","last_change":"2023-11-26T07:12:54.537136+0000","last_active":"2023-11-26T07:12:54.542631+0000","last_peered":"2023-11-26T07:12:54.542631+0000","last_clean":"2023-11-26T07:12:54.542631+0000","last_became_active":"2023-11-26T07:12:54.536557+0000","last_became_peered":"2023-11-26T07:12:54.536557+0000","last_unstale":"2023-11-26T07:12:54.542631+0000","last_undegraded":"2023-11-26T07:12:54.542631+0000","last_fullsized":"2023-11-26T07:12:54.542631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:58:57.698467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519560+0000","last_change":"2023-11-26T07:12:54.519560+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519560+0000","last_undegraded":"2023-11-26T07:12:54.519560+0000","last_fullsized":"2023-11-26T07:12:54.519560+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536191+0000","last_change":"2023-11-26T07:12:54.533932+0000","last_active":"2023-11-26T07:12:54.536191+0000","last_peered":"2023-11-26T07:12:54.536191+0000","last_clean":"2023-11-26T07:12:54.536191+0000","last_became_active":"2023-11-26T07:12:54.533651+0000","last_became_peered":"2023-11-26T07:12:54.533651+0000","last_unstale":"2023-11-26T07:12:54.536191+0000","last_undegraded":"2023-11-26T07:12:54.536191+0000","last_fullsized":"2023-11-26T07:12:54.536191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:16.433050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520815+0000","last_change":"2023-11-26T07:12:54.520815+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520815+0000","last_undegraded":"2023-11-26T07:12:54.520815+0000","last_fullsized":"2023-11-26T07:12:54.520815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541674+0000","last_change":"2023-11-26T07:12:54.533722+0000","last_active":"2023-11-26T07:12:54.541674+0000","last_peered":"2023-11-26T07:12:54.541674+0000","last_clean":"2023-11-26T07:12:54.541674+0000","last_became_active":"2023-11-26T07:12:54.533207+0000","last_became_peered":"2023-11-26T07:12:54.533207+0000","last_unstale":"2023-11-26T07:12:54.541674+0000","last_undegraded":"2023-11-26T07:12:54.541674+0000","last_fullsized":"2023-11-26T07:12:54.541674+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:03.463480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521433+0000","last_change":"2023-11-26T07:12:54.521433+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521433+0000","last_undegraded":"2023-11-26T07:12:54.521433+0000","last_fullsized":"2023-11-26T07:12:54.521433+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540251+0000","last_change":"2023-11-26T07:12:54.535338+0000","last_active":"2023-11-26T07:12:54.540251+0000","last_peered":"2023-11-26T07:12:54.540251+0000","last_clean":"2023-11-26T07:12:54.540251+0000","last_became_active":"2023-11-26T07:12:54.534988+0000","last_became_peered":"2023-11-26T07:12:54.534988+0000","last_unstale":"2023-11-26T07:12:54.540251+0000","last_undegraded":"2023-11-26T07:12:54.540251+0000","last_fullsized":"2023-11-26T07:12:54.540251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:58:57.698467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526021+0000","last_change":"2023-11-26T07:12:54.526021+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526021+0000","last_undegraded":"2023-11-26T07:12:54.526021+0000","last_fullsized":"2023-11-26T07:12:54.526021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542447+0000","last_change":"2023-11-26T07:12:54.531775+0000","last_active":"2023-11-26T07:12:54.542447+0000","last_peered":"2023-11-26T07:12:54.542447+0000","last_clean":"2023-11-26T07:12:54.542447+0000","last_became_active":"2023-11-26T07:12:54.531491+0000","last_became_peered":"2023-11-26T07:12:54.531491+0000","last_unstale":"2023-11-26T07:12:54.542447+0000","last_undegraded":"2023-11-26T07:12:54.542447+0000","last_fullsized":"2023-11-26T07:12:54.542447+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:45.828740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520515+0000","last_change":"2023-11-26T07:12:54.520515+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520515+0000","last_undegraded":"2023-11-26T07:12:54.520515+0000","last_fullsized":"2023-11-26T07:12:54.520515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540202+0000","last_change":"2023-11-26T07:12:54.534426+0000","last_active":"2023-11-26T07:12:54.540202+0000","last_peered":"2023-11-26T07:12:54.540202+0000","last_clean":"2023-11-26T07:12:54.540202+0000","last_became_active":"2023-11-26T07:12:54.534154+0000","last_became_peered":"2023-11-26T07:12:54.534154+0000","last_unstale":"2023-11-26T07:12:54.540202+0000","last_undegraded":"2023-11-26T07:12:54.540202+0000","last_fullsized":"2023-11-26T07:12:54.540202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:38:16.665106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519918+0000","last_change":"2023-11-26T07:12:54.519918+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519918+0000","last_undegraded":"2023-11-26T07:12:54.519918+0000","last_fullsized":"2023-11-26T07:12:54.519918+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534342+0000","last_change":"2023-11-26T07:12:54.529524+0000","last_active":"2023-11-26T07:12:54.534342+0000","last_peered":"2023-11-26T07:12:54.534342+0000","last_clean":"2023-11-26T07:12:54.534342+0000","last_became_active":"2023-11-26T07:12:54.529161+0000","last_became_peered":"2023-11-26T07:12:54.529161+0000","last_unstale":"2023-11-26T07:12:54.534342+0000","last_undegraded":"2023-11-26T07:12:54.534342+0000","last_fullsized":"2023-11-26T07:12:54.534342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:04.125066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.528822+0000","last_change":"2023-11-26T07:12:54.528822+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.528822+0000","last_undegraded":"2023-11-26T07:12:54.528822+0000","last_fullsized":"2023-11-26T07:12:54.528822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540137+0000","last_change":"2023-11-26T07:12:54.533531+0000","last_active":"2023-11-26T07:12:54.540137+0000","last_peered":"2023-11-26T07:12:54.540137+0000","last_clean":"2023-11-26T07:12:54.540137+0000","last_became_active":"2023-11-26T07:12:54.533142+0000","last_became_peered":"2023-11-26T07:12:54.533142+0000","last_unstale":"2023-11-26T07:12:54.540137+0000","last_undegraded":"2023-11-26T07:12:54.540137+0000","last_fullsized":"2023-11-26T07:12:54.540137+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:01:33.193390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526115+0000","last_change":"2023-11-26T07:12:54.526115+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526115+0000","last_undegraded":"2023-11-26T07:12:54.526115+0000","last_fullsized":"2023-11-26T07:12:54.526115+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.528932+0000","last_change":"2023-11-26T07:12:54.525145+0000","last_active":"2023-11-26T07:12:54.528932+0000","last_peered":"2023-11-26T07:12:54.528932+0000","last_clean":"2023-11-26T07:12:54.528932+0000","last_became_active":"2023-11-26T07:12:54.524871+0000","last_became_peered":"2023-11-26T07:12:54.524871+0000","last_unstale":"2023-11-26T07:12:54.528932+0000","last_undegraded":"2023-11-26T07:12:54.528932+0000","last_fullsized":"2023-11-26T07:12:54.528932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:48:46.369669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521333+0000","last_change":"2023-11-26T07:12:54.521333+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521333+0000","last_undegraded":"2023-11-26T07:12:54.521333+0000","last_fullsized":"2023-11-26T07:12:54.521333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542409+0000","last_change":"2023-11-26T07:12:54.533682+0000","last_active":"2023-11-26T07:12:54.542409+0000","last_peered":"2023-11-26T07:12:54.542409+0000","last_clean":"2023-11-26T07:12:54.542409+0000","last_became_active":"2023-11-26T07:12:54.533409+0000","last_became_peered":"2023-11-26T07:12:54.533409+0000","last_unstale":"2023-11-26T07:12:54.542409+0000","last_undegraded":"2023-11-26T07:12:54.542409+0000","last_fullsized":"2023-11-26T07:12:54.542409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:23:22.243947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518436+0000","last_change":"2023-11-26T07:12:54.518436+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518436+0000","last_undegraded":"2023-11-26T07:12:54.518436+0000","last_fullsized":"2023-11-26T07:12:54.518436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541952+0000","last_change":"2023-11-26T07:12:54.537875+0000","last_active":"2023-11-26T07:12:54.541952+0000","last_peered":"2023-11-26T07:12:54.541952+0000","last_clean":"2023-11-26T07:12:54.541952+0000","last_became_active":"2023-11-26T07:12:54.537498+0000","last_became_peered":"2023-11-26T07:12:54.537498+0000","last_unstale":"2023-11-26T07:12:54.541952+0000","last_undegraded":"2023-11-26T07:12:54.541952+0000","last_fullsized":"2023-11-26T07:12:54.541952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:31.377712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520632+0000","last_change":"2023-11-26T07:12:54.520632+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520632+0000","last_undegraded":"2023-11-26T07:12:54.520632+0000","last_fullsized":"2023-11-26T07:12:54.520632+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.531028+0000","last_change":"2023-11-26T07:12:54.528129+0000","last_active":"2023-11-26T07:12:54.531028+0000","last_peered":"2023-11-26T07:12:54.531028+0000","last_clean":"2023-11-26T07:12:54.531028+0000","last_became_active":"2023-11-26T07:12:54.527835+0000","last_became_peered":"2023-11-26T07:12:54.527835+0000","last_unstale":"2023-11-26T07:12:54.531028+0000","last_undegraded":"2023-11-26T07:12:54.531028+0000","last_fullsized":"2023-11-26T07:12:54.531028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:02:54.984506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-26T07:12:53.513412+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:12:53.513412+0000","last_peered":"2023-11-26T07:12:53.513412+0000","last_clean":"2023-11-26T07:12:53.513412+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:12:53.513412+0000","last_undegraded":"2023-11-26T07:12:53.513412+0000","last_fullsized":"2023-11-26T07:12:53.513412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.497}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55800000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.433},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.39800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.63}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.503},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.40300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.57199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.31}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.24399999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.504}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":791748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.313}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.36699999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39800000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.621}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58399999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.253},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.313}]},{"osd":1,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.40899999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.59299999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.628}]},{"osd":6,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.48599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.34799999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.9109999999999996,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.313}]},{"osd":2,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.247},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.32800000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8129999999999997,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.71799999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.883,"5min":4.883,"15min":4.883},"last":0.60699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.8499999999999996,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.67000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47399999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.61099999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.495},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.747}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.60499999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54300000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.621},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.64200000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.67300000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57399999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:13:00.032 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:13:00.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.186+0000 7f9bf116f700 1 Processor -- start 2023-11-26T07:13:00.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.186+0000 7f9bf116f700 1 -- start start 2023-11-26T07:13:00.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bf116f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9bec158a50 0x7f9bec0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bf116f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9bec162a30 0x7f9bec0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bf116f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bec0cfff0 con 0x7f9bec158670 2023-11-26T07:13:00.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bf116f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bec0629b0 con 0x7f9bec158a50 2023-11-26T07:13:00.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bf116f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bec0cfd10 con 0x7f9bec162a30 2023-11-26T07:13:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bea59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9bec158a50 0x7f9bec0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bea59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9bec158a50 0x7f9bec0c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37188/0 (socket says 172.21.15.83:37188) 2023-11-26T07:13:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bea59c700 1 -- 172.21.15.83:0/1740059526 learned_addr learned my addr 172.21.15.83:0/1740059526 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9beb59e700 1 --2- 172.21.15.83:0/1740059526 >> v2:172.21.15.203:3300/0 conn(0x7f9bec162a30 0x7f9bec0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bea59c700 1 -- 172.21.15.83:0/1740059526 >> v2:172.21.15.203:3300/0 conn(0x7f9bec162a30 msgr2=0x7f9bec0c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bea59c700 1 --2- 172.21.15.83:0/1740059526 >> v2:172.21.15.203:3300/0 conn(0x7f9bec162a30 0x7f9bec0c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bea59c700 1 -- 172.21.15.83:0/1740059526 >> v1:172.21.15.203:6789/0 conn(0x7f9bec158670 legacy=0x7f9bec0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bea59c700 1 -- 172.21.15.83:0/1740059526 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec161f50 con 0x7f9bec158a50 2023-11-26T07:13:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9bea59c700 1 --2- 172.21.15.83:0/1740059526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bec158a50 0x7f9bec0c98b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0b0070 tx=0x7f9bdc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1595da256bf29722 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9be9d9b700 1 -- 172.21.15.83:0/1740059526 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc004230 con 0x7f9bec158a50 2023-11-26T07:13:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9be9d9b700 1 -- 172.21.15.83:0/1740059526 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc004390 con 0x7f9bec158a50 2023-11-26T07:13:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.187+0000 7f9be9d9b700 1 -- 172.21.15.83:0/1740059526 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc0036a0 con 0x7f9bec158a50 2023-11-26T07:13:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.188+0000 7f9bf116f700 1 -- 172.21.15.83:0/1740059526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bec158a50 msgr2=0x7f9bec0c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.188+0000 7f9bf116f700 1 --2- 172.21.15.83:0/1740059526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bec158a50 0x7f9bec0c98b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0b0070 tx=0x7f9bdc009bd0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.188+0000 7f9bf116f700 1 -- 172.21.15.83:0/1740059526 shutdown_connections 2023-11-26T07:13:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.188+0000 7f9bf116f700 1 --2- 172.21.15.83:0/1740059526 >> v2:172.21.15.203:3300/0 conn(0x7f9bec162a30 0x7f9bec0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.188+0000 7f9bf116f700 1 --2- 172.21.15.83:0/1740059526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bec158a50 0x7f9bec0c98b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.188+0000 7f9bf116f700 1 -- 172.21.15.83:0/1740059526 >> 172.21.15.83:0/1740059526 conn(0x7f9bec0b57b0 msgr2=0x7f9bec0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:00.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.188+0000 7f9bf116f700 1 -- 172.21.15.83:0/1740059526 shutdown_connections 2023-11-26T07:13:00.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.188+0000 7f9bf116f700 1 -- 172.21.15.83:0/1740059526 wait complete. 2023-11-26T07:13:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.189+0000 7f9bf116f700 1 Processor -- start 2023-11-26T07:13:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.189+0000 7f9bf116f700 1 -- start start 2023-11-26T07:13:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.190+0000 7f9bf116f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bec158670 0x7f9bec1f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.190+0000 7f9bf116f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bec158a50 0x7f9bec0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.190+0000 7f9bf116f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bec162a30 0x7f9bec15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.190+0000 7f9bf116f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec0b3bf0 con 0x7f9bec158a50 2023-11-26T07:13:00.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.190+0000 7f9bf116f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec1f4ae0 con 0x7f9bec162a30 2023-11-26T07:13:00.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.190+0000 7f9bf116f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec0aee10 con 0x7f9bec158670 2023-11-26T07:13:00.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.190+0000 7f9bea59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bec158a50 0x7f9bec0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.190+0000 7f9bea59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bec158a50 0x7f9bec0adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37194/0 (socket says 172.21.15.83:37194) 2023-11-26T07:13:00.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.190+0000 7f9bea59c700 1 -- 172.21.15.83:0/1009549799 learned_addr learned my addr 172.21.15.83:0/1009549799 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:00.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9beb59e700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bec162a30 0x7f9bec15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bead9d700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bec158670 0x7f9bec1f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bea59c700 1 -- 172.21.15.83:0/1009549799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bec158670 msgr2=0x7f9bec1f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bea59c700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bec158670 0x7f9bec1f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bea59c700 1 -- 172.21.15.83:0/1009549799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bec162a30 msgr2=0x7f9bec15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bea59c700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bec162a30 0x7f9bec15aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bea59c700 1 -- 172.21.15.83:0/1009549799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc009550 con 0x7f9bec158a50 2023-11-26T07:13:00.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bea59c700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bec158a50 0x7f9bec0adc80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009130 tx=0x7f9bdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:00.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bd37fe700 1 -- 172.21.15.83:0/1009549799 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc003da0 con 0x7f9bec158a50 2023-11-26T07:13:00.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bd37fe700 1 -- 172.21.15.83:0/1009549799 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc011630 con 0x7f9bec158a50 2023-11-26T07:13:00.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bf116f700 1 -- 172.21.15.83:0/1009549799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bec0b3550 con 0x7f9bec158a50 2023-11-26T07:13:00.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bd37fe700 1 -- 172.21.15.83:0/1009549799 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc0119e0 con 0x7f9bec158a50 2023-11-26T07:13:00.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.191+0000 7f9bf116f700 1 -- 172.21.15.83:0/1009549799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bec0af340 con 0x7f9bec158a50 2023-11-26T07:13:00.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.192+0000 7f9bf116f700 1 -- 172.21.15.83:0/1009549799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bec0b3d30 con 0x7f9bec158a50 2023-11-26T07:13:00.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.193+0000 7f9bd37fe700 1 -- 172.21.15.83:0/1009549799 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f9bdc021070 con 0x7f9bec158a50 2023-11-26T07:13:00.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.193+0000 7f9bd37fe700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9bd806b080 0x7f9bd806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.193+0000 7f9bd37fe700 1 -- 172.21.15.83:0/1009549799 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bdc08af20 con 0x7f9bec158a50 2023-11-26T07:13:00.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.193+0000 7f9bead9d700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9bd806b080 0x7f9bd806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.193+0000 7f9bead9d700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9bd806b080 0x7f9bd806d540 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9be0007fd0 tx=0x7f9be0007ec0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:00.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.197+0000 7f9bd37fe700 1 -- 172.21.15.83:0/1009549799 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9bec0b3d30 con 0x7f9bec158a50 2023-11-26T07:13:00.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.386+0000 7f9bf116f700 1 -- 172.21.15.83:0/1009549799 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bec162510 con 0x7f9bd806b080 2023-11-26T07:13:00.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.393+0000 7f9bd37fe700 1 -- 172.21.15.83:0/1009549799 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283717 (secure 0 0 0) 0x7f9be0003a90 con 0x7f9bd806b080 2023-11-26T07:13:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:00.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.396+0000 7f9bf116f700 1 -- 172.21.15.83:0/1009549799 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9bd806b080 msgr2=0x7f9bd806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.396+0000 7f9bf116f700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9bd806b080 0x7f9bd806d540 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9be0007fd0 tx=0x7f9be0007ec0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.396+0000 7f9bf116f700 1 -- 172.21.15.83:0/1009549799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bec158a50 msgr2=0x7f9bec0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.396+0000 7f9bf116f700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bec158a50 0x7f9bec0adc80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009130 tx=0x7f9bdc004750 comp rx=0 tx=0).stop 2023-11-26T07:13:00.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.397+0000 7f9bf116f700 1 -- 172.21.15.83:0/1009549799 shutdown_connections 2023-11-26T07:13:00.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.397+0000 7f9bf116f700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bec162a30 0x7f9bec15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.397+0000 7f9bf116f700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9bd806b080 0x7f9bd806d540 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.397+0000 7f9bf116f700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bec158670 0x7f9bec1f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.397+0000 7f9bf116f700 1 --2- 172.21.15.83:0/1009549799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bec158a50 0x7f9bec0adc80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.397+0000 7f9bf116f700 1 -- 172.21.15.83:0/1009549799 >> 172.21.15.83:0/1009549799 conn(0x7f9bec0b57b0 msgr2=0x7f9bec0c98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:00.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.397+0000 7f9bf116f700 1 -- 172.21.15.83:0/1009549799 shutdown_connections 2023-11-26T07:13:00.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.397+0000 7f9bf116f700 1 -- 172.21.15.83:0/1009549799 wait complete. 2023-11-26T07:13:00.401 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:13:00.410 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-11-26T07:13:00.325492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221236,"kb_used_data":5048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753484,"statfs":{"total":773094113280,"available":772867567616,"internally_reserved":0,"allocated":5169152,"data_stored":2766708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001087"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521879+0000","last_change":"2023-11-26T07:12:54.521879+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521879+0000","last_undegraded":"2023-11-26T07:12:54.521879+0000","last_fullsized":"2023-11-26T07:12:54.521879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521487+0000","last_change":"2023-11-26T07:12:54.521487+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521487+0000","last_undegraded":"2023-11-26T07:12:54.521487+0000","last_fullsized":"2023-11-26T07:12:54.521487+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519224+0000","last_change":"2023-11-26T07:12:54.519224+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519224+0000","last_undegraded":"2023-11-26T07:12:54.519224+0000","last_fullsized":"2023-11-26T07:12:54.519224+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523622+0000","last_change":"2023-11-26T07:12:54.523622+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523622+0000","last_undegraded":"2023-11-26T07:12:54.523622+0000","last_fullsized":"2023-11-26T07:12:54.523622+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521234+0000","last_change":"2023-11-26T07:12:54.521234+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521234+0000","last_undegraded":"2023-11-26T07:12:54.521234+0000","last_fullsized":"2023-11-26T07:12:54.521234+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529973+0000","last_change":"2023-11-26T07:12:54.529973+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529973+0000","last_undegraded":"2023-11-26T07:12:54.529973+0000","last_fullsized":"2023-11-26T07:12:54.529973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521276+0000","last_change":"2023-11-26T07:12:54.521276+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521276+0000","last_undegraded":"2023-11-26T07:12:54.521276+0000","last_fullsized":"2023-11-26T07:12:54.521276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526287+0000","last_change":"2023-11-26T07:12:54.526287+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526287+0000","last_undegraded":"2023-11-26T07:12:54.526287+0000","last_fullsized":"2023-11-26T07:12:54.526287+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523760+0000","last_change":"2023-11-26T07:12:54.523760+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523760+0000","last_undegraded":"2023-11-26T07:12:54.523760+0000","last_fullsized":"2023-11-26T07:12:54.523760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.524402+0000","last_change":"2023-11-26T07:12:54.524402+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.524402+0000","last_undegraded":"2023-11-26T07:12:54.524402+0000","last_fullsized":"2023-11-26T07:12:54.524402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523524+0000","last_change":"2023-11-26T07:12:54.523524+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523524+0000","last_undegraded":"2023-11-26T07:12:54.523524+0000","last_fullsized":"2023-11-26T07:12:54.523524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521618+0000","last_change":"2023-11-26T07:12:54.521618+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521618+0000","last_undegraded":"2023-11-26T07:12:54.521618+0000","last_fullsized":"2023-11-26T07:12:54.521618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520013+0000","last_change":"2023-11-26T07:12:54.520013+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520013+0000","last_undegraded":"2023-11-26T07:12:54.520013+0000","last_fullsized":"2023-11-26T07:12:54.520013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523982+0000","last_change":"2023-11-26T07:12:54.523982+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523982+0000","last_undegraded":"2023-11-26T07:12:54.523982+0000","last_fullsized":"2023-11-26T07:12:54.523982+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.530689+0000","last_change":"2023-11-26T07:12:54.530689+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.530689+0000","last_undegraded":"2023-11-26T07:12:54.530689+0000","last_fullsized":"2023-11-26T07:12:54.530689+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526270+0000","last_change":"2023-11-26T07:12:54.526270+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526270+0000","last_undegraded":"2023-11-26T07:12:54.526270+0000","last_fullsized":"2023-11-26T07:12:54.526270+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518831+0000","last_change":"2023-11-26T07:12:54.518831+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518831+0000","last_undegraded":"2023-11-26T07:12:54.518831+0000","last_fullsized":"2023-11-26T07:12:54.518831+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520492+0000","last_change":"2023-11-26T07:12:54.520492+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520492+0000","last_undegraded":"2023-11-26T07:12:54.520492+0000","last_fullsized":"2023-11-26T07:12:54.520492+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.528121+0000","last_change":"2023-11-26T07:12:54.528121+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.528121+0000","last_undegraded":"2023-11-26T07:12:54.528121+0000","last_fullsized":"2023-11-26T07:12:54.528121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520226+0000","last_change":"2023-11-26T07:12:54.520226+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520226+0000","last_undegraded":"2023-11-26T07:12:54.520226+0000","last_fullsized":"2023-11-26T07:12:54.520226+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529224+0000","last_change":"2023-11-26T07:12:54.529224+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529224+0000","last_undegraded":"2023-11-26T07:12:54.529224+0000","last_fullsized":"2023-11-26T07:12:54.529224+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523489+0000","last_change":"2023-11-26T07:12:54.523489+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523489+0000","last_undegraded":"2023-11-26T07:12:54.523489+0000","last_fullsized":"2023-11-26T07:12:54.523489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.530050+0000","last_change":"2023-11-26T07:12:54.530050+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.530050+0000","last_undegraded":"2023-11-26T07:12:54.530050+0000","last_fullsized":"2023-11-26T07:12:54.530050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525496+0000","last_change":"2023-11-26T07:12:54.525496+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525496+0000","last_undegraded":"2023-11-26T07:12:54.525496+0000","last_fullsized":"2023-11-26T07:12:54.525496+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525484+0000","last_change":"2023-11-26T07:12:54.525484+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525484+0000","last_undegraded":"2023-11-26T07:12:54.525484+0000","last_fullsized":"2023-11-26T07:12:54.525484+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520283+0000","last_change":"2023-11-26T07:12:54.520283+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520283+0000","last_undegraded":"2023-11-26T07:12:54.520283+0000","last_fullsized":"2023-11-26T07:12:54.520283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518411+0000","last_change":"2023-11-26T07:12:54.518411+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518411+0000","last_undegraded":"2023-11-26T07:12:54.518411+0000","last_fullsized":"2023-11-26T07:12:54.518411+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522587+0000","last_change":"2023-11-26T07:12:54.522587+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522587+0000","last_undegraded":"2023-11-26T07:12:54.522587+0000","last_fullsized":"2023-11-26T07:12:54.522587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526264+0000","last_change":"2023-11-26T07:12:54.526264+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526264+0000","last_undegraded":"2023-11-26T07:12:54.526264+0000","last_fullsized":"2023-11-26T07:12:54.526264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522172+0000","last_change":"2023-11-26T07:12:54.522172+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522172+0000","last_undegraded":"2023-11-26T07:12:54.522172+0000","last_fullsized":"2023-11-26T07:12:54.522172+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526169+0000","last_change":"2023-11-26T07:12:54.526169+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526169+0000","last_undegraded":"2023-11-26T07:12:54.526169+0000","last_fullsized":"2023-11-26T07:12:54.526169+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521324+0000","last_change":"2023-11-26T07:12:54.521324+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521324+0000","last_undegraded":"2023-11-26T07:12:54.521324+0000","last_fullsized":"2023-11-26T07:12:54.521324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521871+0000","last_change":"2023-11-26T07:12:54.521871+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521871+0000","last_undegraded":"2023-11-26T07:12:54.521871+0000","last_fullsized":"2023-11-26T07:12:54.521871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542574+0000","last_change":"2023-11-26T07:12:54.536815+0000","last_active":"2023-11-26T07:12:54.542574+0000","last_peered":"2023-11-26T07:12:54.542574+0000","last_clean":"2023-11-26T07:12:54.542574+0000","last_became_active":"2023-11-26T07:12:54.536526+0000","last_became_peered":"2023-11-26T07:12:54.536526+0000","last_unstale":"2023-11-26T07:12:54.542574+0000","last_undegraded":"2023-11-26T07:12:54.542574+0000","last_fullsized":"2023-11-26T07:12:54.542574+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:20:45.041782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526811+0000","last_change":"2023-11-26T07:12:54.526811+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526811+0000","last_undegraded":"2023-11-26T07:12:54.526811+0000","last_fullsized":"2023-11-26T07:12:54.526811+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536052+0000","last_change":"2023-11-26T07:12:54.533729+0000","last_active":"2023-11-26T07:12:54.536052+0000","last_peered":"2023-11-26T07:12:54.536052+0000","last_clean":"2023-11-26T07:12:54.536052+0000","last_became_active":"2023-11-26T07:12:54.533440+0000","last_became_peered":"2023-11-26T07:12:54.533440+0000","last_unstale":"2023-11-26T07:12:54.536052+0000","last_undegraded":"2023-11-26T07:12:54.536052+0000","last_fullsized":"2023-11-26T07:12:54.536052+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:04.125066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522115+0000","last_change":"2023-11-26T07:12:54.522115+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522115+0000","last_undegraded":"2023-11-26T07:12:54.522115+0000","last_fullsized":"2023-11-26T07:12:54.522115+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534324+0000","last_change":"2023-11-26T07:12:54.529395+0000","last_active":"2023-11-26T07:12:54.534324+0000","last_peered":"2023-11-26T07:12:54.534324+0000","last_clean":"2023-11-26T07:12:54.534324+0000","last_became_active":"2023-11-26T07:12:54.529093+0000","last_became_peered":"2023-11-26T07:12:54.529093+0000","last_unstale":"2023-11-26T07:12:54.534324+0000","last_undegraded":"2023-11-26T07:12:54.534324+0000","last_fullsized":"2023-11-26T07:12:54.534324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:59.222804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520849+0000","last_change":"2023-11-26T07:12:54.520849+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520849+0000","last_undegraded":"2023-11-26T07:12:54.520849+0000","last_fullsized":"2023-11-26T07:12:54.520849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.532244+0000","last_change":"2023-11-26T07:12:54.529608+0000","last_active":"2023-11-26T07:12:54.532244+0000","last_peered":"2023-11-26T07:12:54.532244+0000","last_clean":"2023-11-26T07:12:54.532244+0000","last_became_active":"2023-11-26T07:12:54.529334+0000","last_became_peered":"2023-11-26T07:12:54.529334+0000","last_unstale":"2023-11-26T07:12:54.532244+0000","last_undegraded":"2023-11-26T07:12:54.532244+0000","last_fullsized":"2023-11-26T07:12:54.532244+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:27:43.076531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519440+0000","last_change":"2023-11-26T07:12:54.519440+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519440+0000","last_undegraded":"2023-11-26T07:12:54.519440+0000","last_fullsized":"2023-11-26T07:12:54.519440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542468+0000","last_change":"2023-11-26T07:12:54.532202+0000","last_active":"2023-11-26T07:12:54.542468+0000","last_peered":"2023-11-26T07:12:54.542468+0000","last_clean":"2023-11-26T07:12:54.542468+0000","last_became_active":"2023-11-26T07:12:54.531692+0000","last_became_peered":"2023-11-26T07:12:54.531692+0000","last_unstale":"2023-11-26T07:12:54.542468+0000","last_undegraded":"2023-11-26T07:12:54.542468+0000","last_fullsized":"2023-11-26T07:12:54.542468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:16.433050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526413+0000","last_change":"2023-11-26T07:12:54.526413+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526413+0000","last_undegraded":"2023-11-26T07:12:54.526413+0000","last_fullsized":"2023-11-26T07:12:54.526413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534545+0000","last_change":"2023-11-26T07:12:54.529445+0000","last_active":"2023-11-26T07:12:54.534545+0000","last_peered":"2023-11-26T07:12:54.534545+0000","last_clean":"2023-11-26T07:12:54.534545+0000","last_became_active":"2023-11-26T07:12:54.529160+0000","last_became_peered":"2023-11-26T07:12:54.529160+0000","last_unstale":"2023-11-26T07:12:54.534545+0000","last_undegraded":"2023-11-26T07:12:54.534545+0000","last_fullsized":"2023-11-26T07:12:54.534545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:45:17.796276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525851+0000","last_change":"2023-11-26T07:12:54.525851+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525851+0000","last_undegraded":"2023-11-26T07:12:54.525851+0000","last_fullsized":"2023-11-26T07:12:54.525851+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534715+0000","last_change":"2023-11-26T07:12:54.531344+0000","last_active":"2023-11-26T07:12:54.534715+0000","last_peered":"2023-11-26T07:12:54.534715+0000","last_clean":"2023-11-26T07:12:54.534715+0000","last_became_active":"2023-11-26T07:12:54.530914+0000","last_became_peered":"2023-11-26T07:12:54.530914+0000","last_unstale":"2023-11-26T07:12:54.534715+0000","last_undegraded":"2023-11-26T07:12:54.534715+0000","last_fullsized":"2023-11-26T07:12:54.534715+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:58.991270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529503+0000","last_change":"2023-11-26T07:12:54.529503+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529503+0000","last_undegraded":"2023-11-26T07:12:54.529503+0000","last_fullsized":"2023-11-26T07:12:54.529503+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541649+0000","last_change":"2023-11-26T07:12:54.533250+0000","last_active":"2023-11-26T07:12:54.541649+0000","last_peered":"2023-11-26T07:12:54.541649+0000","last_clean":"2023-11-26T07:12:54.541649+0000","last_became_active":"2023-11-26T07:12:54.532934+0000","last_became_peered":"2023-11-26T07:12:54.532934+0000","last_unstale":"2023-11-26T07:12:54.541649+0000","last_undegraded":"2023-11-26T07:12:54.541649+0000","last_fullsized":"2023-11-26T07:12:54.541649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:15:29.001269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518952+0000","last_change":"2023-11-26T07:12:54.518952+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518952+0000","last_undegraded":"2023-11-26T07:12:54.518952+0000","last_fullsized":"2023-11-26T07:12:54.518952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.530096+0000","last_change":"2023-11-26T07:12:54.528493+0000","last_active":"2023-11-26T07:12:54.530096+0000","last_peered":"2023-11-26T07:12:54.530096+0000","last_clean":"2023-11-26T07:12:54.530096+0000","last_became_active":"2023-11-26T07:12:54.527916+0000","last_became_peered":"2023-11-26T07:12:54.527916+0000","last_unstale":"2023-11-26T07:12:54.530096+0000","last_undegraded":"2023-11-26T07:12:54.530096+0000","last_fullsized":"2023-11-26T07:12:54.530096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:23:22.243947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520365+0000","last_change":"2023-11-26T07:12:54.520365+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520365+0000","last_undegraded":"2023-11-26T07:12:54.520365+0000","last_fullsized":"2023-11-26T07:12:54.520365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541667+0000","last_change":"2023-11-26T07:12:54.529990+0000","last_active":"2023-11-26T07:12:54.541667+0000","last_peered":"2023-11-26T07:12:54.541667+0000","last_clean":"2023-11-26T07:12:54.541667+0000","last_became_active":"2023-11-26T07:12:54.529727+0000","last_became_peered":"2023-11-26T07:12:54.529727+0000","last_unstale":"2023-11-26T07:12:54.541667+0000","last_undegraded":"2023-11-26T07:12:54.541667+0000","last_fullsized":"2023-11-26T07:12:54.541667+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:41:21.117692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525687+0000","last_change":"2023-11-26T07:12:54.525687+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525687+0000","last_undegraded":"2023-11-26T07:12:54.525687+0000","last_fullsized":"2023-11-26T07:12:54.525687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540184+0000","last_change":"2023-11-26T07:12:54.534123+0000","last_active":"2023-11-26T07:12:54.540184+0000","last_peered":"2023-11-26T07:12:54.540184+0000","last_clean":"2023-11-26T07:12:54.540184+0000","last_became_active":"2023-11-26T07:12:54.533831+0000","last_became_peered":"2023-11-26T07:12:54.533831+0000","last_unstale":"2023-11-26T07:12:54.540184+0000","last_undegraded":"2023-11-26T07:12:54.540184+0000","last_fullsized":"2023-11-26T07:12:54.540184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:25:35.938501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522255+0000","last_change":"2023-11-26T07:12:54.522255+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522255+0000","last_undegraded":"2023-11-26T07:12:54.522255+0000","last_fullsized":"2023-11-26T07:12:54.522255+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.531202+0000","last_change":"2023-11-26T07:12:54.528248+0000","last_active":"2023-11-26T07:12:54.531202+0000","last_peered":"2023-11-26T07:12:54.531202+0000","last_clean":"2023-11-26T07:12:54.531202+0000","last_became_active":"2023-11-26T07:12:54.527887+0000","last_became_peered":"2023-11-26T07:12:54.527887+0000","last_unstale":"2023-11-26T07:12:54.531202+0000","last_undegraded":"2023-11-26T07:12:54.531202+0000","last_fullsized":"2023-11-26T07:12:54.531202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:52.719388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.527948+0000","last_change":"2023-11-26T07:12:54.527948+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.527948+0000","last_undegraded":"2023-11-26T07:12:54.527948+0000","last_fullsized":"2023-11-26T07:12:54.527948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542608+0000","last_change":"2023-11-26T07:12:54.537145+0000","last_active":"2023-11-26T07:12:54.542608+0000","last_peered":"2023-11-26T07:12:54.542608+0000","last_clean":"2023-11-26T07:12:54.542608+0000","last_became_active":"2023-11-26T07:12:54.536921+0000","last_became_peered":"2023-11-26T07:12:54.536921+0000","last_unstale":"2023-11-26T07:12:54.542608+0000","last_undegraded":"2023-11-26T07:12:54.542608+0000","last_fullsized":"2023-11-26T07:12:54.542608+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:02:54.984506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521236+0000","last_change":"2023-11-26T07:12:54.521236+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521236+0000","last_undegraded":"2023-11-26T07:12:54.521236+0000","last_fullsized":"2023-11-26T07:12:54.521236+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536376+0000","last_change":"2023-11-26T07:12:54.532202+0000","last_active":"2023-11-26T07:12:54.536376+0000","last_peered":"2023-11-26T07:12:54.536376+0000","last_clean":"2023-11-26T07:12:54.536376+0000","last_became_active":"2023-11-26T07:12:54.531841+0000","last_became_peered":"2023-11-26T07:12:54.531841+0000","last_unstale":"2023-11-26T07:12:54.536376+0000","last_undegraded":"2023-11-26T07:12:54.536376+0000","last_fullsized":"2023-11-26T07:12:54.536376+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:20:45.041782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525718+0000","last_change":"2023-11-26T07:12:54.525718+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525718+0000","last_undegraded":"2023-11-26T07:12:54.525718+0000","last_fullsized":"2023-11-26T07:12:54.525718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542408+0000","last_change":"2023-11-26T07:12:54.531548+0000","last_active":"2023-11-26T07:12:54.542408+0000","last_peered":"2023-11-26T07:12:54.542408+0000","last_clean":"2023-11-26T07:12:54.542408+0000","last_became_active":"2023-11-26T07:12:54.531133+0000","last_became_peered":"2023-11-26T07:12:54.531133+0000","last_unstale":"2023-11-26T07:12:54.542408+0000","last_undegraded":"2023-11-26T07:12:54.542408+0000","last_fullsized":"2023-11-26T07:12:54.542408+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:58.991270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519968+0000","last_change":"2023-11-26T07:12:54.519968+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519968+0000","last_undegraded":"2023-11-26T07:12:54.519968+0000","last_fullsized":"2023-11-26T07:12:54.519968+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541930+0000","last_change":"2023-11-26T07:12:54.536843+0000","last_active":"2023-11-26T07:12:54.541930+0000","last_peered":"2023-11-26T07:12:54.541930+0000","last_clean":"2023-11-26T07:12:54.541930+0000","last_became_active":"2023-11-26T07:12:54.536560+0000","last_became_peered":"2023-11-26T07:12:54.536560+0000","last_unstale":"2023-11-26T07:12:54.541930+0000","last_undegraded":"2023-11-26T07:12:54.541930+0000","last_fullsized":"2023-11-26T07:12:54.541930+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:47.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519772+0000","last_change":"2023-11-26T07:12:54.519772+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519772+0000","last_undegraded":"2023-11-26T07:12:54.519772+0000","last_fullsized":"2023-11-26T07:12:54.519772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.529907+0000","last_change":"2023-11-26T07:12:54.527254+0000","last_active":"2023-11-26T07:12:54.529907+0000","last_peered":"2023-11-26T07:12:54.529907+0000","last_clean":"2023-11-26T07:12:54.529907+0000","last_became_active":"2023-11-26T07:12:54.526949+0000","last_became_peered":"2023-11-26T07:12:54.526949+0000","last_unstale":"2023-11-26T07:12:54.529907+0000","last_undegraded":"2023-11-26T07:12:54.529907+0000","last_fullsized":"2023-11-26T07:12:54.529907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:20:37.591566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519987+0000","last_change":"2023-11-26T07:12:54.519987+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519987+0000","last_undegraded":"2023-11-26T07:12:54.519987+0000","last_fullsized":"2023-11-26T07:12:54.519987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536115+0000","last_change":"2023-11-26T07:12:54.535255+0000","last_active":"2023-11-26T07:12:54.536115+0000","last_peered":"2023-11-26T07:12:54.536115+0000","last_clean":"2023-11-26T07:12:54.536115+0000","last_became_active":"2023-11-26T07:12:54.534934+0000","last_became_peered":"2023-11-26T07:12:54.534934+0000","last_unstale":"2023-11-26T07:12:54.536115+0000","last_undegraded":"2023-11-26T07:12:54.536115+0000","last_fullsized":"2023-11-26T07:12:54.536115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:59.222804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526457+0000","last_change":"2023-11-26T07:12:54.526457+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526457+0000","last_undegraded":"2023-11-26T07:12:54.526457+0000","last_fullsized":"2023-11-26T07:12:54.526457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534475+0000","last_change":"2023-11-26T07:12:54.531509+0000","last_active":"2023-11-26T07:12:54.534475+0000","last_peered":"2023-11-26T07:12:54.534475+0000","last_clean":"2023-11-26T07:12:54.534475+0000","last_became_active":"2023-11-26T07:12:54.531168+0000","last_became_peered":"2023-11-26T07:12:54.531168+0000","last_unstale":"2023-11-26T07:12:54.534475+0000","last_undegraded":"2023-11-26T07:12:54.534475+0000","last_fullsized":"2023-11-26T07:12:54.534475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:45.828740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.527721+0000","last_change":"2023-11-26T07:12:54.527721+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.527721+0000","last_undegraded":"2023-11-26T07:12:54.527721+0000","last_fullsized":"2023-11-26T07:12:54.527721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542544+0000","last_change":"2023-11-26T07:12:54.536077+0000","last_active":"2023-11-26T07:12:54.542544+0000","last_peered":"2023-11-26T07:12:54.542544+0000","last_clean":"2023-11-26T07:12:54.542544+0000","last_became_active":"2023-11-26T07:12:54.535793+0000","last_became_peered":"2023-11-26T07:12:54.535793+0000","last_unstale":"2023-11-26T07:12:54.542544+0000","last_undegraded":"2023-11-26T07:12:54.542544+0000","last_fullsized":"2023-11-26T07:12:54.542544+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:27:43.076531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529399+0000","last_change":"2023-11-26T07:12:54.529399+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529399+0000","last_undegraded":"2023-11-26T07:12:54.529399+0000","last_fullsized":"2023-11-26T07:12:54.529399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542631+0000","last_change":"2023-11-26T07:12:54.537136+0000","last_active":"2023-11-26T07:12:54.542631+0000","last_peered":"2023-11-26T07:12:54.542631+0000","last_clean":"2023-11-26T07:12:54.542631+0000","last_became_active":"2023-11-26T07:12:54.536557+0000","last_became_peered":"2023-11-26T07:12:54.536557+0000","last_unstale":"2023-11-26T07:12:54.542631+0000","last_undegraded":"2023-11-26T07:12:54.542631+0000","last_fullsized":"2023-11-26T07:12:54.542631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:58:57.698467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519560+0000","last_change":"2023-11-26T07:12:54.519560+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519560+0000","last_undegraded":"2023-11-26T07:12:54.519560+0000","last_fullsized":"2023-11-26T07:12:54.519560+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536191+0000","last_change":"2023-11-26T07:12:54.533932+0000","last_active":"2023-11-26T07:12:54.536191+0000","last_peered":"2023-11-26T07:12:54.536191+0000","last_clean":"2023-11-26T07:12:54.536191+0000","last_became_active":"2023-11-26T07:12:54.533651+0000","last_became_peered":"2023-11-26T07:12:54.533651+0000","last_unstale":"2023-11-26T07:12:54.536191+0000","last_undegraded":"2023-11-26T07:12:54.536191+0000","last_fullsized":"2023-11-26T07:12:54.536191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:16.433050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520815+0000","last_change":"2023-11-26T07:12:54.520815+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520815+0000","last_undegraded":"2023-11-26T07:12:54.520815+0000","last_fullsized":"2023-11-26T07:12:54.520815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541674+0000","last_change":"2023-11-26T07:12:54.533722+0000","last_active":"2023-11-26T07:12:54.541674+0000","last_peered":"2023-11-26T07:12:54.541674+0000","last_clean":"2023-11-26T07:12:54.541674+0000","last_became_active":"2023-11-26T07:12:54.533207+0000","last_became_peered":"2023-11-26T07:12:54.533207+0000","last_unstale":"2023-11-26T07:12:54.541674+0000","last_undegraded":"2023-11-26T07:12:54.541674+0000","last_fullsized":"2023-11-26T07:12:54.541674+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:03.463480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521433+0000","last_change":"2023-11-26T07:12:54.521433+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521433+0000","last_undegraded":"2023-11-26T07:12:54.521433+0000","last_fullsized":"2023-11-26T07:12:54.521433+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540251+0000","last_change":"2023-11-26T07:12:54.535338+0000","last_active":"2023-11-26T07:12:54.540251+0000","last_peered":"2023-11-26T07:12:54.540251+0000","last_clean":"2023-11-26T07:12:54.540251+0000","last_became_active":"2023-11-26T07:12:54.534988+0000","last_became_peered":"2023-11-26T07:12:54.534988+0000","last_unstale":"2023-11-26T07:12:54.540251+0000","last_undegraded":"2023-11-26T07:12:54.540251+0000","last_fullsized":"2023-11-26T07:12:54.540251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:58:57.698467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526021+0000","last_change":"2023-11-26T07:12:54.526021+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526021+0000","last_undegraded":"2023-11-26T07:12:54.526021+0000","last_fullsized":"2023-11-26T07:12:54.526021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542447+0000","last_change":"2023-11-26T07:12:54.531775+0000","last_active":"2023-11-26T07:12:54.542447+0000","last_peered":"2023-11-26T07:12:54.542447+0000","last_clean":"2023-11-26T07:12:54.542447+0000","last_became_active":"2023-11-26T07:12:54.531491+0000","last_became_peered":"2023-11-26T07:12:54.531491+0000","last_unstale":"2023-11-26T07:12:54.542447+0000","last_undegraded":"2023-11-26T07:12:54.542447+0000","last_fullsized":"2023-11-26T07:12:54.542447+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:45.828740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520515+0000","last_change":"2023-11-26T07:12:54.520515+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520515+0000","last_undegraded":"2023-11-26T07:12:54.520515+0000","last_fullsized":"2023-11-26T07:12:54.520515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540202+0000","last_change":"2023-11-26T07:12:54.534426+0000","last_active":"2023-11-26T07:12:54.540202+0000","last_peered":"2023-11-26T07:12:54.540202+0000","last_clean":"2023-11-26T07:12:54.540202+0000","last_became_active":"2023-11-26T07:12:54.534154+0000","last_became_peered":"2023-11-26T07:12:54.534154+0000","last_unstale":"2023-11-26T07:12:54.540202+0000","last_undegraded":"2023-11-26T07:12:54.540202+0000","last_fullsized":"2023-11-26T07:12:54.540202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:38:16.665106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519918+0000","last_change":"2023-11-26T07:12:54.519918+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519918+0000","last_undegraded":"2023-11-26T07:12:54.519918+0000","last_fullsized":"2023-11-26T07:12:54.519918+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534342+0000","last_change":"2023-11-26T07:12:54.529524+0000","last_active":"2023-11-26T07:12:54.534342+0000","last_peered":"2023-11-26T07:12:54.534342+0000","last_clean":"2023-11-26T07:12:54.534342+0000","last_became_active":"2023-11-26T07:12:54.529161+0000","last_became_peered":"2023-11-26T07:12:54.529161+0000","last_unstale":"2023-11-26T07:12:54.534342+0000","last_undegraded":"2023-11-26T07:12:54.534342+0000","last_fullsized":"2023-11-26T07:12:54.534342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:04.125066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.528822+0000","last_change":"2023-11-26T07:12:54.528822+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.528822+0000","last_undegraded":"2023-11-26T07:12:54.528822+0000","last_fullsized":"2023-11-26T07:12:54.528822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540137+0000","last_change":"2023-11-26T07:12:54.533531+0000","last_active":"2023-11-26T07:12:54.540137+0000","last_peered":"2023-11-26T07:12:54.540137+0000","last_clean":"2023-11-26T07:12:54.540137+0000","last_became_active":"2023-11-26T07:12:54.533142+0000","last_became_peered":"2023-11-26T07:12:54.533142+0000","last_unstale":"2023-11-26T07:12:54.540137+0000","last_undegraded":"2023-11-26T07:12:54.540137+0000","last_fullsized":"2023-11-26T07:12:54.540137+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:01:33.193390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526115+0000","last_change":"2023-11-26T07:12:54.526115+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526115+0000","last_undegraded":"2023-11-26T07:12:54.526115+0000","last_fullsized":"2023-11-26T07:12:54.526115+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.528932+0000","last_change":"2023-11-26T07:12:54.525145+0000","last_active":"2023-11-26T07:12:54.528932+0000","last_peered":"2023-11-26T07:12:54.528932+0000","last_clean":"2023-11-26T07:12:54.528932+0000","last_became_active":"2023-11-26T07:12:54.524871+0000","last_became_peered":"2023-11-26T07:12:54.524871+0000","last_unstale":"2023-11-26T07:12:54.528932+0000","last_undegraded":"2023-11-26T07:12:54.528932+0000","last_fullsized":"2023-11-26T07:12:54.528932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:48:46.369669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521333+0000","last_change":"2023-11-26T07:12:54.521333+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521333+0000","last_undegraded":"2023-11-26T07:12:54.521333+0000","last_fullsized":"2023-11-26T07:12:54.521333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542409+0000","last_change":"2023-11-26T07:12:54.533682+0000","last_active":"2023-11-26T07:12:54.542409+0000","last_peered":"2023-11-26T07:12:54.542409+0000","last_clean":"2023-11-26T07:12:54.542409+0000","last_became_active":"2023-11-26T07:12:54.533409+0000","last_became_peered":"2023-11-26T07:12:54.533409+0000","last_unstale":"2023-11-26T07:12:54.542409+0000","last_undegraded":"2023-11-26T07:12:54.542409+0000","last_fullsized":"2023-11-26T07:12:54.542409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:23:22.243947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518436+0000","last_change":"2023-11-26T07:12:54.518436+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518436+0000","last_undegraded":"2023-11-26T07:12:54.518436+0000","last_fullsized":"2023-11-26T07:12:54.518436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541952+0000","last_change":"2023-11-26T07:12:54.537875+0000","last_active":"2023-11-26T07:12:54.541952+0000","last_peered":"2023-11-26T07:12:54.541952+0000","last_clean":"2023-11-26T07:12:54.541952+0000","last_became_active":"2023-11-26T07:12:54.537498+0000","last_became_peered":"2023-11-26T07:12:54.537498+0000","last_unstale":"2023-11-26T07:12:54.541952+0000","last_undegraded":"2023-11-26T07:12:54.541952+0000","last_fullsized":"2023-11-26T07:12:54.541952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:31.377712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520632+0000","last_change":"2023-11-26T07:12:54.520632+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520632+0000","last_undegraded":"2023-11-26T07:12:54.520632+0000","last_fullsized":"2023-11-26T07:12:54.520632+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.531028+0000","last_change":"2023-11-26T07:12:54.528129+0000","last_active":"2023-11-26T07:12:54.531028+0000","last_peered":"2023-11-26T07:12:54.531028+0000","last_clean":"2023-11-26T07:12:54.531028+0000","last_became_active":"2023-11-26T07:12:54.527835+0000","last_became_peered":"2023-11-26T07:12:54.527835+0000","last_unstale":"2023-11-26T07:12:54.531028+0000","last_undegraded":"2023-11-26T07:12:54.531028+0000","last_fullsized":"2023-11-26T07:12:54.531028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:02:54.984506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-26T07:12:53.513412+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:12:53.513412+0000","last_peered":"2023-11-26T07:12:53.513412+0000","last_clean":"2023-11-26T07:12:53.513412+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:12:53.513412+0000","last_undegraded":"2023-11-26T07:12:53.513412+0000","last_fullsized":"2023-11-26T07:12:53.513412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.497}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55800000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.433},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.39800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.63}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.503},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.40300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.57199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.31}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.24399999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.504}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":791748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.313}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.36699999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39800000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.621}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58399999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.253},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.313}]},{"osd":1,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.40899999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.59299999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.628}]},{"osd":6,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.48599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.34799999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.9109999999999996,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.313}]},{"osd":2,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.247},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.32800000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8129999999999997,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.71799999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.883,"5min":4.883,"15min":4.883},"last":0.60699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.8499999999999996,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.67000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47399999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.61099999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.495},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.747}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.60499999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54300000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.621},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.64200000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.67300000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57399999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:13:00.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2023-11-26T07:13:00.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.521879+0000', 'last_change': '2023-11-26T07:12:54.521879+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.521879+0000', 'last_undegraded': '2023-11-26T07:12:54.521879+0000', 'last_fullsized': '2023-11-26T07:12:54.521879+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:13:00.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2023-11-26T07:13:00.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.521487+0000', 'last_change': '2023-11-26T07:12:54.521487+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.521487+0000', 'last_undegraded': '2023-11-26T07:12:54.521487+0000', 'last_fullsized': '2023-11-26T07:12:54.521487+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:13:00.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2023-11-26T07:13:00.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.519224+0000', 'last_change': '2023-11-26T07:12:54.519224+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.519224+0000', 'last_undegraded': '2023-11-26T07:12:54.519224+0000', 'last_fullsized': '2023-11-26T07:12:54.519224+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:13:00.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2023-11-26T07:13:00.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.523622+0000', 'last_change': '2023-11-26T07:12:54.523622+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.523622+0000', 'last_undegraded': '2023-11-26T07:12:54.523622+0000', 'last_fullsized': '2023-11-26T07:12:54.523622+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2023-11-26T07:13:00.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.521234+0000', 'last_change': '2023-11-26T07:12:54.521234+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.521234+0000', 'last_undegraded': '2023-11-26T07:12:54.521234+0000', 'last_fullsized': '2023-11-26T07:12:54.521234+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:13:00.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2023-11-26T07:13:00.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.529973+0000', 'last_change': '2023-11-26T07:12:54.529973+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.529973+0000', 'last_undegraded': '2023-11-26T07:12:54.529973+0000', 'last_fullsized': '2023-11-26T07:12:54.529973+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:13:00.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2023-11-26T07:13:00.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.521276+0000', 'last_change': '2023-11-26T07:12:54.521276+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.521276+0000', 'last_undegraded': '2023-11-26T07:12:54.521276+0000', 'last_fullsized': '2023-11-26T07:12:54.521276+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:13:00.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2023-11-26T07:13:00.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.526287+0000', 'last_change': '2023-11-26T07:12:54.526287+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.526287+0000', 'last_undegraded': '2023-11-26T07:12:54.526287+0000', 'last_fullsized': '2023-11-26T07:12:54.526287+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2023-11-26T07:13:00.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.523760+0000', 'last_change': '2023-11-26T07:12:54.523760+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.523760+0000', 'last_undegraded': '2023-11-26T07:12:54.523760+0000', 'last_fullsized': '2023-11-26T07:12:54.523760+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2023-11-26T07:13:00.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.524402+0000', 'last_change': '2023-11-26T07:12:54.524402+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.524402+0000', 'last_undegraded': '2023-11-26T07:12:54.524402+0000', 'last_fullsized': '2023-11-26T07:12:54.524402+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2023-11-26T07:13:00.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.523524+0000', 'last_change': '2023-11-26T07:12:54.523524+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.523524+0000', 'last_undegraded': '2023-11-26T07:12:54.523524+0000', 'last_fullsized': '2023-11-26T07:12:54.523524+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2023-11-26T07:13:00.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.521618+0000', 'last_change': '2023-11-26T07:12:54.521618+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.521618+0000', 'last_undegraded': '2023-11-26T07:12:54.521618+0000', 'last_fullsized': '2023-11-26T07:12:54.521618+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:13:00.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2023-11-26T07:13:00.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.520013+0000', 'last_change': '2023-11-26T07:12:54.520013+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.520013+0000', 'last_undegraded': '2023-11-26T07:12:54.520013+0000', 'last_fullsized': '2023-11-26T07:12:54.520013+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:13:00.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2023-11-26T07:13:00.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.523982+0000', 'last_change': '2023-11-26T07:12:54.523982+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.523982+0000', 'last_undegraded': '2023-11-26T07:12:54.523982+0000', 'last_fullsized': '2023-11-26T07:12:54.523982+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2023-11-26T07:13:00.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.530689+0000', 'last_change': '2023-11-26T07:12:54.530689+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.530689+0000', 'last_undegraded': '2023-11-26T07:12:54.530689+0000', 'last_fullsized': '2023-11-26T07:12:54.530689+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:13:00.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2023-11-26T07:13:00.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.526270+0000', 'last_change': '2023-11-26T07:12:54.526270+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.526270+0000', 'last_undegraded': '2023-11-26T07:12:54.526270+0000', 'last_fullsized': '2023-11-26T07:12:54.526270+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2023-11-26T07:13:00.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.518831+0000', 'last_change': '2023-11-26T07:12:54.518831+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.518831+0000', 'last_undegraded': '2023-11-26T07:12:54.518831+0000', 'last_fullsized': '2023-11-26T07:12:54.518831+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:13:00.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2023-11-26T07:13:00.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.520492+0000', 'last_change': '2023-11-26T07:12:54.520492+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.520492+0000', 'last_undegraded': '2023-11-26T07:12:54.520492+0000', 'last_fullsized': '2023-11-26T07:12:54.520492+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:13:00.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2023-11-26T07:13:00.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.528121+0000', 'last_change': '2023-11-26T07:12:54.528121+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.528121+0000', 'last_undegraded': '2023-11-26T07:12:54.528121+0000', 'last_fullsized': '2023-11-26T07:12:54.528121+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2023-11-26T07:13:00.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.520226+0000', 'last_change': '2023-11-26T07:12:54.520226+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.520226+0000', 'last_undegraded': '2023-11-26T07:12:54.520226+0000', 'last_fullsized': '2023-11-26T07:12:54.520226+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:13:00.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2023-11-26T07:13:00.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.529224+0000', 'last_change': '2023-11-26T07:12:54.529224+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.529224+0000', 'last_undegraded': '2023-11-26T07:12:54.529224+0000', 'last_fullsized': '2023-11-26T07:12:54.529224+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:13:00.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2023-11-26T07:13:00.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.523489+0000', 'last_change': '2023-11-26T07:12:54.523489+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.523489+0000', 'last_undegraded': '2023-11-26T07:12:54.523489+0000', 'last_fullsized': '2023-11-26T07:12:54.523489+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2023-11-26T07:13:00.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.530050+0000', 'last_change': '2023-11-26T07:12:54.530050+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.530050+0000', 'last_undegraded': '2023-11-26T07:12:54.530050+0000', 'last_fullsized': '2023-11-26T07:12:54.530050+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2023-11-26T07:13:00.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.525496+0000', 'last_change': '2023-11-26T07:12:54.525496+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.525496+0000', 'last_undegraded': '2023-11-26T07:12:54.525496+0000', 'last_fullsized': '2023-11-26T07:12:54.525496+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2023-11-26T07:13:00.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.525484+0000', 'last_change': '2023-11-26T07:12:54.525484+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.525484+0000', 'last_undegraded': '2023-11-26T07:12:54.525484+0000', 'last_fullsized': '2023-11-26T07:12:54.525484+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2023-11-26T07:13:00.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.520283+0000', 'last_change': '2023-11-26T07:12:54.520283+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.520283+0000', 'last_undegraded': '2023-11-26T07:12:54.520283+0000', 'last_fullsized': '2023-11-26T07:12:54.520283+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:13:00.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2023-11-26T07:13:00.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.518411+0000', 'last_change': '2023-11-26T07:12:54.518411+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.518411+0000', 'last_undegraded': '2023-11-26T07:12:54.518411+0000', 'last_fullsized': '2023-11-26T07:12:54.518411+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:13:00.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2023-11-26T07:13:00.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.522587+0000', 'last_change': '2023-11-26T07:12:54.522587+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.522587+0000', 'last_undegraded': '2023-11-26T07:12:54.522587+0000', 'last_fullsized': '2023-11-26T07:12:54.522587+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:13:00.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2023-11-26T07:13:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.526264+0000', 'last_change': '2023-11-26T07:12:54.526264+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.526264+0000', 'last_undegraded': '2023-11-26T07:12:54.526264+0000', 'last_fullsized': '2023-11-26T07:12:54.526264+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2023-11-26T07:13:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.522172+0000', 'last_change': '2023-11-26T07:12:54.522172+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.522172+0000', 'last_undegraded': '2023-11-26T07:12:54.522172+0000', 'last_fullsized': '2023-11-26T07:12:54.522172+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:13:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2023-11-26T07:13:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.526169+0000', 'last_change': '2023-11-26T07:12:54.526169+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.526169+0000', 'last_undegraded': '2023-11-26T07:12:54.526169+0000', 'last_fullsized': '2023-11-26T07:12:54.526169+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:13:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2023-11-26T07:13:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.521324+0000', 'last_change': '2023-11-26T07:12:54.521324+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.521324+0000', 'last_undegraded': '2023-11-26T07:12:54.521324+0000', 'last_fullsized': '2023-11-26T07:12:54.521324+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:13:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-11-26T07:13:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.521871+0000', 'last_change': '2023-11-26T07:12:54.521871+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.521871+0000', 'last_undegraded': '2023-11-26T07:12:54.521871+0000', 'last_fullsized': '2023-11-26T07:12:54.521871+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:13:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-11-26T07:13:00.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.526811+0000', 'last_change': '2023-11-26T07:12:54.526811+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.526811+0000', 'last_undegraded': '2023-11-26T07:12:54.526811+0000', 'last_fullsized': '2023-11-26T07:12:54.526811+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-11-26T07:13:00.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.522115+0000', 'last_change': '2023-11-26T07:12:54.522115+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.522115+0000', 'last_undegraded': '2023-11-26T07:12:54.522115+0000', 'last_fullsized': '2023-11-26T07:12:54.522115+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:13:00.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-11-26T07:13:00.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.520849+0000', 'last_change': '2023-11-26T07:12:54.520849+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.520849+0000', 'last_undegraded': '2023-11-26T07:12:54.520849+0000', 'last_fullsized': '2023-11-26T07:12:54.520849+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:13:00.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-11-26T07:13:00.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.519440+0000', 'last_change': '2023-11-26T07:12:54.519440+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.519440+0000', 'last_undegraded': '2023-11-26T07:12:54.519440+0000', 'last_fullsized': '2023-11-26T07:12:54.519440+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:13:00.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2023-11-26T07:13:00.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.526413+0000', 'last_change': '2023-11-26T07:12:54.526413+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.526413+0000', 'last_undegraded': '2023-11-26T07:12:54.526413+0000', 'last_fullsized': '2023-11-26T07:12:54.526413+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2023-11-26T07:13:00.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.525851+0000', 'last_change': '2023-11-26T07:12:54.525851+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.525851+0000', 'last_undegraded': '2023-11-26T07:12:54.525851+0000', 'last_fullsized': '2023-11-26T07:12:54.525851+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-11-26T07:13:00.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.529503+0000', 'last_change': '2023-11-26T07:12:54.529503+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.529503+0000', 'last_undegraded': '2023-11-26T07:12:54.529503+0000', 'last_fullsized': '2023-11-26T07:12:54.529503+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-11-26T07:13:00.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.518952+0000', 'last_change': '2023-11-26T07:12:54.518952+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.518952+0000', 'last_undegraded': '2023-11-26T07:12:54.518952+0000', 'last_fullsized': '2023-11-26T07:12:54.518952+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:13:00.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-11-26T07:13:00.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.520365+0000', 'last_change': '2023-11-26T07:12:54.520365+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.520365+0000', 'last_undegraded': '2023-11-26T07:12:54.520365+0000', 'last_fullsized': '2023-11-26T07:12:54.520365+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:13:00.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-11-26T07:13:00.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.525687+0000', 'last_change': '2023-11-26T07:12:54.525687+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.525687+0000', 'last_undegraded': '2023-11-26T07:12:54.525687+0000', 'last_fullsized': '2023-11-26T07:12:54.525687+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:13:00.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-11-26T07:13:00.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.522255+0000', 'last_change': '2023-11-26T07:12:54.522255+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.522255+0000', 'last_undegraded': '2023-11-26T07:12:54.522255+0000', 'last_fullsized': '2023-11-26T07:12:54.522255+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:13:00.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-11-26T07:13:00.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.527948+0000', 'last_change': '2023-11-26T07:12:54.527948+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.527948+0000', 'last_undegraded': '2023-11-26T07:12:54.527948+0000', 'last_fullsized': '2023-11-26T07:12:54.527948+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2023-11-26T07:13:00.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.521236+0000', 'last_change': '2023-11-26T07:12:54.521236+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.521236+0000', 'last_undegraded': '2023-11-26T07:12:54.521236+0000', 'last_fullsized': '2023-11-26T07:12:54.521236+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-11-26T07:13:00.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.525718+0000', 'last_change': '2023-11-26T07:12:54.525718+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.525718+0000', 'last_undegraded': '2023-11-26T07:12:54.525718+0000', 'last_fullsized': '2023-11-26T07:12:54.525718+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:13:00.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-11-26T07:13:00.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.519968+0000', 'last_change': '2023-11-26T07:12:54.519968+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.519968+0000', 'last_undegraded': '2023-11-26T07:12:54.519968+0000', 'last_fullsized': '2023-11-26T07:12:54.519968+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:13:00.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-11-26T07:13:00.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.519772+0000', 'last_change': '2023-11-26T07:12:54.519772+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.519772+0000', 'last_undegraded': '2023-11-26T07:12:54.519772+0000', 'last_fullsized': '2023-11-26T07:12:54.519772+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:13:00.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2023-11-26T07:13:00.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.519987+0000', 'last_change': '2023-11-26T07:12:54.519987+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.519987+0000', 'last_undegraded': '2023-11-26T07:12:54.519987+0000', 'last_fullsized': '2023-11-26T07:12:54.519987+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-11-26T07:13:00.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.526457+0000', 'last_change': '2023-11-26T07:12:54.526457+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.526457+0000', 'last_undegraded': '2023-11-26T07:12:54.526457+0000', 'last_fullsized': '2023-11-26T07:12:54.526457+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-11-26T07:13:00.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.527721+0000', 'last_change': '2023-11-26T07:12:54.527721+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.527721+0000', 'last_undegraded': '2023-11-26T07:12:54.527721+0000', 'last_fullsized': '2023-11-26T07:12:54.527721+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-11-26T07:13:00.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.529399+0000', 'last_change': '2023-11-26T07:12:54.529399+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.529399+0000', 'last_undegraded': '2023-11-26T07:12:54.529399+0000', 'last_fullsized': '2023-11-26T07:12:54.529399+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:13:00.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-11-26T07:13:00.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.519560+0000', 'last_change': '2023-11-26T07:12:54.519560+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.519560+0000', 'last_undegraded': '2023-11-26T07:12:54.519560+0000', 'last_fullsized': '2023-11-26T07:12:54.519560+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:13:00.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-11-26T07:13:00.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.520815+0000', 'last_change': '2023-11-26T07:12:54.520815+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.520815+0000', 'last_undegraded': '2023-11-26T07:12:54.520815+0000', 'last_fullsized': '2023-11-26T07:12:54.520815+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:13:00.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-11-26T07:13:00.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.521433+0000', 'last_change': '2023-11-26T07:12:54.521433+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.521433+0000', 'last_undegraded': '2023-11-26T07:12:54.521433+0000', 'last_fullsized': '2023-11-26T07:12:54.521433+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-11-26T07:13:00.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.526021+0000', 'last_change': '2023-11-26T07:12:54.526021+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.526021+0000', 'last_undegraded': '2023-11-26T07:12:54.526021+0000', 'last_fullsized': '2023-11-26T07:12:54.526021+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:13:00.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2023-11-26T07:13:00.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.520515+0000', 'last_change': '2023-11-26T07:12:54.520515+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.520515+0000', 'last_undegraded': '2023-11-26T07:12:54.520515+0000', 'last_fullsized': '2023-11-26T07:12:54.520515+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-11-26T07:13:00.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.519918+0000', 'last_change': '2023-11-26T07:12:54.519918+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.519918+0000', 'last_undegraded': '2023-11-26T07:12:54.519918+0000', 'last_fullsized': '2023-11-26T07:12:54.519918+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:13:00.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-11-26T07:13:00.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.528822+0000', 'last_change': '2023-11-26T07:12:54.528822+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.528822+0000', 'last_undegraded': '2023-11-26T07:12:54.528822+0000', 'last_fullsized': '2023-11-26T07:12:54.528822+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:13:00.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-11-26T07:13:00.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.526115+0000', 'last_change': '2023-11-26T07:12:54.526115+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.526115+0000', 'last_undegraded': '2023-11-26T07:12:54.526115+0000', 'last_fullsized': '2023-11-26T07:12:54.526115+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:13:00.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-11-26T07:13:00.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.521333+0000', 'last_change': '2023-11-26T07:12:54.521333+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.521333+0000', 'last_undegraded': '2023-11-26T07:12:54.521333+0000', 'last_fullsized': '2023-11-26T07:12:54.521333+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:13:00.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-11-26T07:13:00.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.518436+0000', 'last_change': '2023-11-26T07:12:54.518436+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.518436+0000', 'last_undegraded': '2023-11-26T07:12:54.518436+0000', 'last_fullsized': '2023-11-26T07:12:54.518436+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:13:00.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-11-26T07:13:00.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:12:54.520632+0000', 'last_change': '2023-11-26T07:12:54.520632+0000', 'last_active': '2023-11-26T07:12:54.508362+0000', 'last_peered': '2023-11-26T07:12:54.508362+0000', 'last_clean': '2023-11-26T07:12:54.508362+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:12:54.520632+0000', 'last_undegraded': '2023-11-26T07:12:54.520632+0000', 'last_fullsized': '2023-11-26T07:12:54.520632+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'last_clean_scrub_stamp': '2023-11-26T07:12:54.508362+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:13:00.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-26T07:13:00.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.568+0000 7fcc4d990700 1 Processor -- start 2023-11-26T07:13:00.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.568+0000 7fcc4d990700 1 -- start start 2023-11-26T07:13:00.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.568+0000 7fcc4d990700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcc480d3610 0x7fcc480b5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.568+0000 7fcc4d990700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcc480b5e90 0x7fcc480b7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.568+0000 7fcc4d990700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc480d2810 con 0x7fcc4815e480 2023-11-26T07:13:00.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.568+0000 7fcc4d990700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc480b62c0 con 0x7fcc480d3610 2023-11-26T07:13:00.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.568+0000 7fcc4d990700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc480d2a70 con 0x7fcc480b5e90 2023-11-26T07:13:00.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.569+0000 7fcc467fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcc480d3610 0x7fcc480b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.569+0000 7fcc467fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcc480d3610 0x7fcc480b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37202/0 (socket says 172.21.15.83:37202) 2023-11-26T07:13:00.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.569+0000 7fcc467fc700 1 -- 172.21.15.83:0/1544203898 learned_addr learned my addr 172.21.15.83:0/1544203898 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.569+0000 7fcc477fe700 1 --2- 172.21.15.83:0/1544203898 >> v2:172.21.15.203:3300/0 conn(0x7fcc480b5e90 0x7fcc480b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.569+0000 7fcc467fc700 1 -- 172.21.15.83:0/1544203898 >> v2:172.21.15.203:3300/0 conn(0x7fcc480b5e90 msgr2=0x7fcc480b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.569+0000 7fcc467fc700 1 --2- 172.21.15.83:0/1544203898 >> v2:172.21.15.203:3300/0 conn(0x7fcc480b5e90 0x7fcc480b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.569+0000 7fcc467fc700 1 -- 172.21.15.83:0/1544203898 >> v1:172.21.15.203:6789/0 conn(0x7fcc4815e480 legacy=0x7fcc480d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.569+0000 7fcc467fc700 1 -- 172.21.15.83:0/1544203898 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc480d2d50 con 0x7fcc480d3610 2023-11-26T07:13:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.569+0000 7fcc467fc700 1 --2- 172.21.15.83:0/1544203898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc480d3610 0x7fcc480b5950 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcc4806c090 tx=0x7fcc3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f24919557444ae server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.571+0000 7fcc45ffb700 1 -- 172.21.15.83:0/1544203898 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00b870 con 0x7fcc480d3610 2023-11-26T07:13:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.571+0000 7fcc45ffb700 1 -- 172.21.15.83:0/1544203898 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c00b9d0 con 0x7fcc480d3610 2023-11-26T07:13:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.571+0000 7fcc45ffb700 1 -- 172.21.15.83:0/1544203898 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00bd60 con 0x7fcc480d3610 2023-11-26T07:13:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.571+0000 7fcc4d990700 1 -- 172.21.15.83:0/1544203898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc480d3610 msgr2=0x7fcc480b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.571+0000 7fcc4d990700 1 --2- 172.21.15.83:0/1544203898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc480d3610 0x7fcc480b5950 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcc4806c090 tx=0x7fcc3c009bd0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.571+0000 7fcc4d990700 1 -- 172.21.15.83:0/1544203898 shutdown_connections 2023-11-26T07:13:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.571+0000 7fcc4d990700 1 --2- 172.21.15.83:0/1544203898 >> v2:172.21.15.203:3300/0 conn(0x7fcc480b5e90 0x7fcc480b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.571+0000 7fcc4d990700 1 --2- 172.21.15.83:0/1544203898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc480d3610 0x7fcc480b5950 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.571+0000 7fcc4d990700 1 -- 172.21.15.83:0/1544203898 >> 172.21.15.83:0/1544203898 conn(0x7fcc480d0950 msgr2=0x7fcc480d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:00.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.571+0000 7fcc4d990700 1 -- 172.21.15.83:0/1544203898 shutdown_connections 2023-11-26T07:13:00.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.571+0000 7fcc4d990700 1 -- 172.21.15.83:0/1544203898 wait complete. 2023-11-26T07:13:00.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.573+0000 7fcc4d990700 1 Processor -- start 2023-11-26T07:13:00.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.573+0000 7fcc4d990700 1 -- start start 2023-11-26T07:13:00.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc4d990700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc480b5e90 0x7fcc480accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc4d990700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc480d3610 0x7fcc481f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc4d990700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc4815e480 0x7fcc480ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc4d990700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4815e9e0 con 0x7fcc480b5e90 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc4d990700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc481f54e0 con 0x7fcc4815e480 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc4d990700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4815eca0 con 0x7fcc480d3610 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc480b5e90 0x7fcc480accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc480b5e90 0x7fcc480accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37206/0 (socket says 172.21.15.83:37206) 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc46ffd700 1 -- 172.21.15.83:0/1275345391 learned_addr learned my addr 172.21.15.83:0/1275345391 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc477fe700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc4815e480 0x7fcc480ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc467fc700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc480d3610 0x7fcc481f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc46ffd700 1 -- 172.21.15.83:0/1275345391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc480d3610 msgr2=0x7fcc481f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc46ffd700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc480d3610 0x7fcc481f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc46ffd700 1 -- 172.21.15.83:0/1275345391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc4815e480 msgr2=0x7fcc480ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc46ffd700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc4815e480 0x7fcc480ab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc46ffd700 1 -- 172.21.15.83:0/1275345391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c009550 con 0x7fcc480b5e90 2023-11-26T07:13:00.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc46ffd700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc480b5e90 0x7fcc480accf0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcc3800c4e0 tx=0x7fcc3800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:00.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc2ffff700 1 -- 172.21.15.83:0/1275345391 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc380052d0 con 0x7fcc480b5e90 2023-11-26T07:13:00.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc2ffff700 1 -- 172.21.15.83:0/1275345391 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc38005430 con 0x7fcc480b5e90 2023-11-26T07:13:00.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc4d990700 1 -- 172.21.15.83:0/1275345391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4815ef80 con 0x7fcc480b5e90 2023-11-26T07:13:00.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc2ffff700 1 -- 172.21.15.83:0/1275345391 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc380038c0 con 0x7fcc480b5e90 2023-11-26T07:13:00.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.574+0000 7fcc4d990700 1 -- 172.21.15.83:0/1275345391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc480ab7a0 con 0x7fcc480b5e90 2023-11-26T07:13:00.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.575+0000 7fcc4d990700 1 -- 172.21.15.83:0/1275345391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4815f4a0 con 0x7fcc480b5e90 2023-11-26T07:13:00.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.576+0000 7fcc2ffff700 1 -- 172.21.15.83:0/1275345391 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fcc38003a20 con 0x7fcc480b5e90 2023-11-26T07:13:00.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.576+0000 7fcc2ffff700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc3406b080 0x7fcc3406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.576+0000 7fcc467fc700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc3406b080 0x7fcc3406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.576+0000 7fcc2ffff700 1 -- 172.21.15.83:0/1275345391 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc38088c30 con 0x7fcc480b5e90 2023-11-26T07:13:00.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.577+0000 7fcc467fc700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc3406b080 0x7fcc3406d540 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c000c00 tx=0x7fcc3c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:00.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.581+0000 7fcc2ffff700 1 -- 172.21.15.83:0/1275345391 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc38053a60 con 0x7fcc480b5e90 2023-11-26T07:13:00.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.853+0000 7fcc4d990700 1 -- 172.21.15.83:0/1275345391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcc480751e0 con 0x7fcc480b5e90 2023-11-26T07:13:00.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.853+0000 7fcc2ffff700 1 -- 172.21.15.83:0/1275345391 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1581 (secure 0 0 0) 0x7fcc38017640 con 0x7fcc480b5e90 2023-11-26T07:13:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:00.855 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":118,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":45,"num_osds":8,"num_up_osds":8,"osd_up_since":1700982674,"num_in_osds":8,"osd_in_since":1700982665,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":226545664,"bytes_avail":772867567616,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":27,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5544}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-11-26T07:12:16.317493+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-26T07:13:00.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.855+0000 7fcc4d990700 1 -- 172.21.15.83:0/1275345391 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc3406b080 msgr2=0x7fcc3406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.855+0000 7fcc4d990700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc3406b080 0x7fcc3406d540 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c000c00 tx=0x7fcc3c008030 comp rx=0 tx=0).stop 2023-11-26T07:13:00.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.855+0000 7fcc4d990700 1 -- 172.21.15.83:0/1275345391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc480b5e90 msgr2=0x7fcc480accf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.855+0000 7fcc4d990700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc480b5e90 0x7fcc480accf0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcc3800c4e0 tx=0x7fcc3800c7f0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.855+0000 7fcc4d990700 1 -- 172.21.15.83:0/1275345391 shutdown_connections 2023-11-26T07:13:00.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.855+0000 7fcc4d990700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc4815e480 0x7fcc480ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.855+0000 7fcc4d990700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc480b5e90 0x7fcc480accf0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.856+0000 7fcc4d990700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc3406b080 0x7fcc3406d540 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.856+0000 7fcc4d990700 1 --2- 172.21.15.83:0/1275345391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc480d3610 0x7fcc481f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.856+0000 7fcc4d990700 1 -- 172.21.15.83:0/1275345391 >> 172.21.15.83:0/1275345391 conn(0x7fcc480d0950 msgr2=0x7fcc4815c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:00.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.856+0000 7fcc4d990700 1 -- 172.21.15.83:0/1275345391 shutdown_connections 2023-11-26T07:13:00.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.856+0000 7fcc4d990700 1 -- 172.21.15.83:0/1275345391 wait complete. 2023-11-26T07:13:00.869 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-26T07:13:00.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:13:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.989+0000 7fc5a9815700 1 Processor -- start 2023-11-26T07:13:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.989+0000 7fc5a9815700 1 -- start start 2023-11-26T07:13:00.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.989+0000 7fc5a9815700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5a415e1e0 0x7fc5a40d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a9815700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5a40d0f80 0x7fc5a40b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a9815700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a40d20b0 con 0x7fc5a40d3a00 2023-11-26T07:13:00.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a9815700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a40d3150 con 0x7fc5a415e1e0 2023-11-26T07:13:00.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a9815700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a415e5c0 con 0x7fc5a40d0f80 2023-11-26T07:13:00.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a2ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5a415e1e0 0x7fc5a40d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a2ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5a415e1e0 0x7fc5a40d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37214/0 (socket says 172.21.15.83:37214) 2023-11-26T07:13:00.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a2ffd700 1 -- 172.21.15.83:0/1986894622 learned_addr learned my addr 172.21.15.83:0/1986894622 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:00.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a37fe700 1 --2- 172.21.15.83:0/1986894622 >> v2:172.21.15.203:3300/0 conn(0x7fc5a40d0f80 0x7fc5a40b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a2ffd700 1 -- 172.21.15.83:0/1986894622 >> v2:172.21.15.203:3300/0 conn(0x7fc5a40d0f80 msgr2=0x7fc5a40b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:00.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a2ffd700 1 --2- 172.21.15.83:0/1986894622 >> v2:172.21.15.203:3300/0 conn(0x7fc5a40d0f80 0x7fc5a40b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a2ffd700 1 -- 172.21.15.83:0/1986894622 >> v1:172.21.15.203:6789/0 conn(0x7fc5a40d3a00 legacy=0x7fc5a40d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a2ffd700 1 -- 172.21.15.83:0/1986894622 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40d2390 con 0x7fc5a415e1e0 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a2ffd700 1 --2- 172.21.15.83:0/1986894622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a415e1e0 0x7fc5a40d34c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc5a408c640 tx=0x7fc594009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445b841f17115aad server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a1ffb700 1 -- 172.21.15.83:0/1986894622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc594004230 con 0x7fc5a415e1e0 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a1ffb700 1 -- 172.21.15.83:0/1986894622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc594004390 con 0x7fc5a415e1e0 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.990+0000 7fc5a1ffb700 1 -- 172.21.15.83:0/1986894622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5940036a0 con 0x7fc5a415e1e0 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.991+0000 7fc5a9815700 1 -- 172.21.15.83:0/1986894622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a415e1e0 msgr2=0x7fc5a40d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.991+0000 7fc5a9815700 1 --2- 172.21.15.83:0/1986894622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a415e1e0 0x7fc5a40d34c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc5a408c640 tx=0x7fc594009c00 comp rx=0 tx=0).stop 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.991+0000 7fc5a9815700 1 -- 172.21.15.83:0/1986894622 shutdown_connections 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.991+0000 7fc5a9815700 1 --2- 172.21.15.83:0/1986894622 >> v2:172.21.15.203:3300/0 conn(0x7fc5a40d0f80 0x7fc5a40b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.991+0000 7fc5a9815700 1 --2- 172.21.15.83:0/1986894622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a415e1e0 0x7fc5a40d34c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.991+0000 7fc5a9815700 1 -- 172.21.15.83:0/1986894622 >> 172.21.15.83:0/1986894622 conn(0x7fc5a40b5b50 msgr2=0x7fc5a40b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:00.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.991+0000 7fc5a9815700 1 -- 172.21.15.83:0/1986894622 shutdown_connections 2023-11-26T07:13:00.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.991+0000 7fc5a9815700 1 -- 172.21.15.83:0/1986894622 wait complete. 2023-11-26T07:13:00.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.992+0000 7fc5a9815700 1 Processor -- start 2023-11-26T07:13:00.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.992+0000 7fc5a9815700 1 -- start start 2023-11-26T07:13:00.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a9815700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5a40d0f80 0x7fc5a41524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a9815700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a40d3a00 0x7fc5a406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a9815700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5a415e1e0 0x7fc5a406b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a9815700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40ac1f0 con 0x7fc5a40d3a00 2023-11-26T07:13:00.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a9815700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a4152a30 con 0x7fc5a415e1e0 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a9815700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40ab570 con 0x7fc5a40d0f80 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a40d3a00 0x7fc5a406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a40d3a00 0x7fc5a406aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37222/0 (socket says 172.21.15.83:37222) 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a27fc700 1 -- 172.21.15.83:0/3104003453 learned_addr learned my addr 172.21.15.83:0/3104003453 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a37fe700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5a415e1e0 0x7fc5a406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a27fc700 1 -- 172.21.15.83:0/3104003453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5a40d0f80 msgr2=0x7fc5a41524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a27fc700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5a40d0f80 0x7fc5a41524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a27fc700 1 -- 172.21.15.83:0/3104003453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5a415e1e0 msgr2=0x7fc5a406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a27fc700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5a415e1e0 0x7fc5a406b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.993+0000 7fc5a27fc700 1 -- 172.21.15.83:0/3104003453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc594009550 con 0x7fc5a40d3a00 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.994+0000 7fc5a27fc700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a40d3a00 0x7fc5a406aff0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc59800c4d0 tx=0x7fc59800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:00.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.994+0000 7fc58bfff700 1 -- 172.21.15.83:0/3104003453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5980054e0 con 0x7fc5a40d3a00 2023-11-26T07:13:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.994+0000 7fc58bfff700 1 -- 172.21.15.83:0/3104003453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598003680 con 0x7fc5a40d3a00 2023-11-26T07:13:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.994+0000 7fc5a9815700 1 -- 172.21.15.83:0/3104003453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a40ab850 con 0x7fc5a40d3a00 2023-11-26T07:13:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.994+0000 7fc58bfff700 1 -- 172.21.15.83:0/3104003453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598003a10 con 0x7fc5a40d3a00 2023-11-26T07:13:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.994+0000 7fc5a9815700 1 -- 172.21.15.83:0/3104003453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a40abe10 con 0x7fc5a40d3a00 2023-11-26T07:13:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.995+0000 7fc5a9815700 1 -- 172.21.15.83:0/3104003453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a40ac330 con 0x7fc5a40d3a00 2023-11-26T07:13:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.995+0000 7fc58bfff700 1 -- 172.21.15.83:0/3104003453 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc598003b70 con 0x7fc5a40d3a00 2023-11-26T07:13:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.995+0000 7fc58bfff700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc59006b080 0x7fc59006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.996+0000 7fc58bfff700 1 -- 172.21.15.83:0/3104003453 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc598088c20 con 0x7fc5a40d3a00 2023-11-26T07:13:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.996+0000 7fc5a2ffd700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc59006b080 0x7fc59006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:00.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:00.996+0000 7fc5a2ffd700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc59006b080 0x7fc59006d540 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc594000c00 tx=0x7fc594010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:01.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.000+0000 7fc58bfff700 1 -- 172.21.15.83:0/3104003453 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc5a40ac330 con 0x7fc5a40d3a00 2023-11-26T07:13:01.159 INFO:tasks.ceph.mds.c.smithi083.stderr:2023-11-26T07:13:01.157+0000 7f0a57a55700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:13:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.187+0000 7fc5a9815700 1 -- 172.21.15.83:0/3104003453 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5a4069000 con 0x7fc59006b080 2023-11-26T07:13:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.194+0000 7fc58bfff700 1 -- 172.21.15.83:0/3104003453 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283717 (secure 0 0 0) 0x7fc5a4069000 con 0x7fc59006b080 2023-11-26T07:13:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 -- 172.21.15.83:0/3104003453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc59006b080 msgr2=0x7fc59006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc59006b080 0x7fc59006d540 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc594000c00 tx=0x7fc594010000 comp rx=0 tx=0).stop 2023-11-26T07:13:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 -- 172.21.15.83:0/3104003453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a40d3a00 msgr2=0x7fc5a406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a40d3a00 0x7fc5a406aff0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc59800c4d0 tx=0x7fc59800c890 comp rx=0 tx=0).stop 2023-11-26T07:13:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 -- 172.21.15.83:0/3104003453 shutdown_connections 2023-11-26T07:13:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5a415e1e0 0x7fc5a406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc59006b080 0x7fc59006d540 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5a40d0f80 0x7fc5a41524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 --2- 172.21.15.83:0/3104003453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5a40d3a00 0x7fc5a406aff0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:01.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 -- 172.21.15.83:0/3104003453 >> 172.21.15.83:0/3104003453 conn(0x7fc5a40b5b50 msgr2=0x7fc5a40ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:01.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 -- 172.21.15.83:0/3104003453 shutdown_connections 2023-11-26T07:13:01.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:01.198+0000 7fc5a9815700 1 -- 172.21.15.83:0/3104003453 wait complete. 2023-11-26T07:13:01.203 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:13:01.211 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-11-26T07:13:00.325492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221236,"kb_used_data":5048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753484,"statfs":{"total":773094113280,"available":772867567616,"internally_reserved":0,"allocated":5169152,"data_stored":2766708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001087"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521879+0000","last_change":"2023-11-26T07:12:54.521879+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521879+0000","last_undegraded":"2023-11-26T07:12:54.521879+0000","last_fullsized":"2023-11-26T07:12:54.521879+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521487+0000","last_change":"2023-11-26T07:12:54.521487+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521487+0000","last_undegraded":"2023-11-26T07:12:54.521487+0000","last_fullsized":"2023-11-26T07:12:54.521487+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519224+0000","last_change":"2023-11-26T07:12:54.519224+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519224+0000","last_undegraded":"2023-11-26T07:12:54.519224+0000","last_fullsized":"2023-11-26T07:12:54.519224+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523622+0000","last_change":"2023-11-26T07:12:54.523622+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523622+0000","last_undegraded":"2023-11-26T07:12:54.523622+0000","last_fullsized":"2023-11-26T07:12:54.523622+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521234+0000","last_change":"2023-11-26T07:12:54.521234+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521234+0000","last_undegraded":"2023-11-26T07:12:54.521234+0000","last_fullsized":"2023-11-26T07:12:54.521234+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529973+0000","last_change":"2023-11-26T07:12:54.529973+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529973+0000","last_undegraded":"2023-11-26T07:12:54.529973+0000","last_fullsized":"2023-11-26T07:12:54.529973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521276+0000","last_change":"2023-11-26T07:12:54.521276+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521276+0000","last_undegraded":"2023-11-26T07:12:54.521276+0000","last_fullsized":"2023-11-26T07:12:54.521276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526287+0000","last_change":"2023-11-26T07:12:54.526287+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526287+0000","last_undegraded":"2023-11-26T07:12:54.526287+0000","last_fullsized":"2023-11-26T07:12:54.526287+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523760+0000","last_change":"2023-11-26T07:12:54.523760+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523760+0000","last_undegraded":"2023-11-26T07:12:54.523760+0000","last_fullsized":"2023-11-26T07:12:54.523760+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.524402+0000","last_change":"2023-11-26T07:12:54.524402+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.524402+0000","last_undegraded":"2023-11-26T07:12:54.524402+0000","last_fullsized":"2023-11-26T07:12:54.524402+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523524+0000","last_change":"2023-11-26T07:12:54.523524+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523524+0000","last_undegraded":"2023-11-26T07:12:54.523524+0000","last_fullsized":"2023-11-26T07:12:54.523524+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521618+0000","last_change":"2023-11-26T07:12:54.521618+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521618+0000","last_undegraded":"2023-11-26T07:12:54.521618+0000","last_fullsized":"2023-11-26T07:12:54.521618+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520013+0000","last_change":"2023-11-26T07:12:54.520013+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520013+0000","last_undegraded":"2023-11-26T07:12:54.520013+0000","last_fullsized":"2023-11-26T07:12:54.520013+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523982+0000","last_change":"2023-11-26T07:12:54.523982+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523982+0000","last_undegraded":"2023-11-26T07:12:54.523982+0000","last_fullsized":"2023-11-26T07:12:54.523982+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.530689+0000","last_change":"2023-11-26T07:12:54.530689+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.530689+0000","last_undegraded":"2023-11-26T07:12:54.530689+0000","last_fullsized":"2023-11-26T07:12:54.530689+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526270+0000","last_change":"2023-11-26T07:12:54.526270+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526270+0000","last_undegraded":"2023-11-26T07:12:54.526270+0000","last_fullsized":"2023-11-26T07:12:54.526270+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518831+0000","last_change":"2023-11-26T07:12:54.518831+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518831+0000","last_undegraded":"2023-11-26T07:12:54.518831+0000","last_fullsized":"2023-11-26T07:12:54.518831+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520492+0000","last_change":"2023-11-26T07:12:54.520492+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520492+0000","last_undegraded":"2023-11-26T07:12:54.520492+0000","last_fullsized":"2023-11-26T07:12:54.520492+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.528121+0000","last_change":"2023-11-26T07:12:54.528121+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.528121+0000","last_undegraded":"2023-11-26T07:12:54.528121+0000","last_fullsized":"2023-11-26T07:12:54.528121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520226+0000","last_change":"2023-11-26T07:12:54.520226+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520226+0000","last_undegraded":"2023-11-26T07:12:54.520226+0000","last_fullsized":"2023-11-26T07:12:54.520226+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529224+0000","last_change":"2023-11-26T07:12:54.529224+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529224+0000","last_undegraded":"2023-11-26T07:12:54.529224+0000","last_fullsized":"2023-11-26T07:12:54.529224+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.523489+0000","last_change":"2023-11-26T07:12:54.523489+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.523489+0000","last_undegraded":"2023-11-26T07:12:54.523489+0000","last_fullsized":"2023-11-26T07:12:54.523489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.530050+0000","last_change":"2023-11-26T07:12:54.530050+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.530050+0000","last_undegraded":"2023-11-26T07:12:54.530050+0000","last_fullsized":"2023-11-26T07:12:54.530050+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525496+0000","last_change":"2023-11-26T07:12:54.525496+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525496+0000","last_undegraded":"2023-11-26T07:12:54.525496+0000","last_fullsized":"2023-11-26T07:12:54.525496+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525484+0000","last_change":"2023-11-26T07:12:54.525484+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525484+0000","last_undegraded":"2023-11-26T07:12:54.525484+0000","last_fullsized":"2023-11-26T07:12:54.525484+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520283+0000","last_change":"2023-11-26T07:12:54.520283+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520283+0000","last_undegraded":"2023-11-26T07:12:54.520283+0000","last_fullsized":"2023-11-26T07:12:54.520283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518411+0000","last_change":"2023-11-26T07:12:54.518411+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518411+0000","last_undegraded":"2023-11-26T07:12:54.518411+0000","last_fullsized":"2023-11-26T07:12:54.518411+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522587+0000","last_change":"2023-11-26T07:12:54.522587+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522587+0000","last_undegraded":"2023-11-26T07:12:54.522587+0000","last_fullsized":"2023-11-26T07:12:54.522587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526264+0000","last_change":"2023-11-26T07:12:54.526264+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526264+0000","last_undegraded":"2023-11-26T07:12:54.526264+0000","last_fullsized":"2023-11-26T07:12:54.526264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522172+0000","last_change":"2023-11-26T07:12:54.522172+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522172+0000","last_undegraded":"2023-11-26T07:12:54.522172+0000","last_fullsized":"2023-11-26T07:12:54.522172+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526169+0000","last_change":"2023-11-26T07:12:54.526169+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526169+0000","last_undegraded":"2023-11-26T07:12:54.526169+0000","last_fullsized":"2023-11-26T07:12:54.526169+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521324+0000","last_change":"2023-11-26T07:12:54.521324+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521324+0000","last_undegraded":"2023-11-26T07:12:54.521324+0000","last_fullsized":"2023-11-26T07:12:54.521324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521871+0000","last_change":"2023-11-26T07:12:54.521871+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521871+0000","last_undegraded":"2023-11-26T07:12:54.521871+0000","last_fullsized":"2023-11-26T07:12:54.521871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542574+0000","last_change":"2023-11-26T07:12:54.536815+0000","last_active":"2023-11-26T07:12:54.542574+0000","last_peered":"2023-11-26T07:12:54.542574+0000","last_clean":"2023-11-26T07:12:54.542574+0000","last_became_active":"2023-11-26T07:12:54.536526+0000","last_became_peered":"2023-11-26T07:12:54.536526+0000","last_unstale":"2023-11-26T07:12:54.542574+0000","last_undegraded":"2023-11-26T07:12:54.542574+0000","last_fullsized":"2023-11-26T07:12:54.542574+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:20:45.041782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526811+0000","last_change":"2023-11-26T07:12:54.526811+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526811+0000","last_undegraded":"2023-11-26T07:12:54.526811+0000","last_fullsized":"2023-11-26T07:12:54.526811+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536052+0000","last_change":"2023-11-26T07:12:54.533729+0000","last_active":"2023-11-26T07:12:54.536052+0000","last_peered":"2023-11-26T07:12:54.536052+0000","last_clean":"2023-11-26T07:12:54.536052+0000","last_became_active":"2023-11-26T07:12:54.533440+0000","last_became_peered":"2023-11-26T07:12:54.533440+0000","last_unstale":"2023-11-26T07:12:54.536052+0000","last_undegraded":"2023-11-26T07:12:54.536052+0000","last_fullsized":"2023-11-26T07:12:54.536052+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:04.125066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522115+0000","last_change":"2023-11-26T07:12:54.522115+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522115+0000","last_undegraded":"2023-11-26T07:12:54.522115+0000","last_fullsized":"2023-11-26T07:12:54.522115+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534324+0000","last_change":"2023-11-26T07:12:54.529395+0000","last_active":"2023-11-26T07:12:54.534324+0000","last_peered":"2023-11-26T07:12:54.534324+0000","last_clean":"2023-11-26T07:12:54.534324+0000","last_became_active":"2023-11-26T07:12:54.529093+0000","last_became_peered":"2023-11-26T07:12:54.529093+0000","last_unstale":"2023-11-26T07:12:54.534324+0000","last_undegraded":"2023-11-26T07:12:54.534324+0000","last_fullsized":"2023-11-26T07:12:54.534324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:59.222804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520849+0000","last_change":"2023-11-26T07:12:54.520849+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520849+0000","last_undegraded":"2023-11-26T07:12:54.520849+0000","last_fullsized":"2023-11-26T07:12:54.520849+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.532244+0000","last_change":"2023-11-26T07:12:54.529608+0000","last_active":"2023-11-26T07:12:54.532244+0000","last_peered":"2023-11-26T07:12:54.532244+0000","last_clean":"2023-11-26T07:12:54.532244+0000","last_became_active":"2023-11-26T07:12:54.529334+0000","last_became_peered":"2023-11-26T07:12:54.529334+0000","last_unstale":"2023-11-26T07:12:54.532244+0000","last_undegraded":"2023-11-26T07:12:54.532244+0000","last_fullsized":"2023-11-26T07:12:54.532244+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:27:43.076531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519440+0000","last_change":"2023-11-26T07:12:54.519440+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519440+0000","last_undegraded":"2023-11-26T07:12:54.519440+0000","last_fullsized":"2023-11-26T07:12:54.519440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542468+0000","last_change":"2023-11-26T07:12:54.532202+0000","last_active":"2023-11-26T07:12:54.542468+0000","last_peered":"2023-11-26T07:12:54.542468+0000","last_clean":"2023-11-26T07:12:54.542468+0000","last_became_active":"2023-11-26T07:12:54.531692+0000","last_became_peered":"2023-11-26T07:12:54.531692+0000","last_unstale":"2023-11-26T07:12:54.542468+0000","last_undegraded":"2023-11-26T07:12:54.542468+0000","last_fullsized":"2023-11-26T07:12:54.542468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:16.433050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526413+0000","last_change":"2023-11-26T07:12:54.526413+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526413+0000","last_undegraded":"2023-11-26T07:12:54.526413+0000","last_fullsized":"2023-11-26T07:12:54.526413+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534545+0000","last_change":"2023-11-26T07:12:54.529445+0000","last_active":"2023-11-26T07:12:54.534545+0000","last_peered":"2023-11-26T07:12:54.534545+0000","last_clean":"2023-11-26T07:12:54.534545+0000","last_became_active":"2023-11-26T07:12:54.529160+0000","last_became_peered":"2023-11-26T07:12:54.529160+0000","last_unstale":"2023-11-26T07:12:54.534545+0000","last_undegraded":"2023-11-26T07:12:54.534545+0000","last_fullsized":"2023-11-26T07:12:54.534545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:45:17.796276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525851+0000","last_change":"2023-11-26T07:12:54.525851+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525851+0000","last_undegraded":"2023-11-26T07:12:54.525851+0000","last_fullsized":"2023-11-26T07:12:54.525851+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534715+0000","last_change":"2023-11-26T07:12:54.531344+0000","last_active":"2023-11-26T07:12:54.534715+0000","last_peered":"2023-11-26T07:12:54.534715+0000","last_clean":"2023-11-26T07:12:54.534715+0000","last_became_active":"2023-11-26T07:12:54.530914+0000","last_became_peered":"2023-11-26T07:12:54.530914+0000","last_unstale":"2023-11-26T07:12:54.534715+0000","last_undegraded":"2023-11-26T07:12:54.534715+0000","last_fullsized":"2023-11-26T07:12:54.534715+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:58.991270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529503+0000","last_change":"2023-11-26T07:12:54.529503+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529503+0000","last_undegraded":"2023-11-26T07:12:54.529503+0000","last_fullsized":"2023-11-26T07:12:54.529503+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541649+0000","last_change":"2023-11-26T07:12:54.533250+0000","last_active":"2023-11-26T07:12:54.541649+0000","last_peered":"2023-11-26T07:12:54.541649+0000","last_clean":"2023-11-26T07:12:54.541649+0000","last_became_active":"2023-11-26T07:12:54.532934+0000","last_became_peered":"2023-11-26T07:12:54.532934+0000","last_unstale":"2023-11-26T07:12:54.541649+0000","last_undegraded":"2023-11-26T07:12:54.541649+0000","last_fullsized":"2023-11-26T07:12:54.541649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:15:29.001269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518952+0000","last_change":"2023-11-26T07:12:54.518952+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518952+0000","last_undegraded":"2023-11-26T07:12:54.518952+0000","last_fullsized":"2023-11-26T07:12:54.518952+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.530096+0000","last_change":"2023-11-26T07:12:54.528493+0000","last_active":"2023-11-26T07:12:54.530096+0000","last_peered":"2023-11-26T07:12:54.530096+0000","last_clean":"2023-11-26T07:12:54.530096+0000","last_became_active":"2023-11-26T07:12:54.527916+0000","last_became_peered":"2023-11-26T07:12:54.527916+0000","last_unstale":"2023-11-26T07:12:54.530096+0000","last_undegraded":"2023-11-26T07:12:54.530096+0000","last_fullsized":"2023-11-26T07:12:54.530096+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:23:22.243947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520365+0000","last_change":"2023-11-26T07:12:54.520365+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520365+0000","last_undegraded":"2023-11-26T07:12:54.520365+0000","last_fullsized":"2023-11-26T07:12:54.520365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541667+0000","last_change":"2023-11-26T07:12:54.529990+0000","last_active":"2023-11-26T07:12:54.541667+0000","last_peered":"2023-11-26T07:12:54.541667+0000","last_clean":"2023-11-26T07:12:54.541667+0000","last_became_active":"2023-11-26T07:12:54.529727+0000","last_became_peered":"2023-11-26T07:12:54.529727+0000","last_unstale":"2023-11-26T07:12:54.541667+0000","last_undegraded":"2023-11-26T07:12:54.541667+0000","last_fullsized":"2023-11-26T07:12:54.541667+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:41:21.117692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525687+0000","last_change":"2023-11-26T07:12:54.525687+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525687+0000","last_undegraded":"2023-11-26T07:12:54.525687+0000","last_fullsized":"2023-11-26T07:12:54.525687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540184+0000","last_change":"2023-11-26T07:12:54.534123+0000","last_active":"2023-11-26T07:12:54.540184+0000","last_peered":"2023-11-26T07:12:54.540184+0000","last_clean":"2023-11-26T07:12:54.540184+0000","last_became_active":"2023-11-26T07:12:54.533831+0000","last_became_peered":"2023-11-26T07:12:54.533831+0000","last_unstale":"2023-11-26T07:12:54.540184+0000","last_undegraded":"2023-11-26T07:12:54.540184+0000","last_fullsized":"2023-11-26T07:12:54.540184+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:25:35.938501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.522255+0000","last_change":"2023-11-26T07:12:54.522255+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.522255+0000","last_undegraded":"2023-11-26T07:12:54.522255+0000","last_fullsized":"2023-11-26T07:12:54.522255+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.531202+0000","last_change":"2023-11-26T07:12:54.528248+0000","last_active":"2023-11-26T07:12:54.531202+0000","last_peered":"2023-11-26T07:12:54.531202+0000","last_clean":"2023-11-26T07:12:54.531202+0000","last_became_active":"2023-11-26T07:12:54.527887+0000","last_became_peered":"2023-11-26T07:12:54.527887+0000","last_unstale":"2023-11-26T07:12:54.531202+0000","last_undegraded":"2023-11-26T07:12:54.531202+0000","last_fullsized":"2023-11-26T07:12:54.531202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:52.719388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.527948+0000","last_change":"2023-11-26T07:12:54.527948+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.527948+0000","last_undegraded":"2023-11-26T07:12:54.527948+0000","last_fullsized":"2023-11-26T07:12:54.527948+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542608+0000","last_change":"2023-11-26T07:12:54.537145+0000","last_active":"2023-11-26T07:12:54.542608+0000","last_peered":"2023-11-26T07:12:54.542608+0000","last_clean":"2023-11-26T07:12:54.542608+0000","last_became_active":"2023-11-26T07:12:54.536921+0000","last_became_peered":"2023-11-26T07:12:54.536921+0000","last_unstale":"2023-11-26T07:12:54.542608+0000","last_undegraded":"2023-11-26T07:12:54.542608+0000","last_fullsized":"2023-11-26T07:12:54.542608+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:02:54.984506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521236+0000","last_change":"2023-11-26T07:12:54.521236+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521236+0000","last_undegraded":"2023-11-26T07:12:54.521236+0000","last_fullsized":"2023-11-26T07:12:54.521236+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536376+0000","last_change":"2023-11-26T07:12:54.532202+0000","last_active":"2023-11-26T07:12:54.536376+0000","last_peered":"2023-11-26T07:12:54.536376+0000","last_clean":"2023-11-26T07:12:54.536376+0000","last_became_active":"2023-11-26T07:12:54.531841+0000","last_became_peered":"2023-11-26T07:12:54.531841+0000","last_unstale":"2023-11-26T07:12:54.536376+0000","last_undegraded":"2023-11-26T07:12:54.536376+0000","last_fullsized":"2023-11-26T07:12:54.536376+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:20:45.041782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.525718+0000","last_change":"2023-11-26T07:12:54.525718+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.525718+0000","last_undegraded":"2023-11-26T07:12:54.525718+0000","last_fullsized":"2023-11-26T07:12:54.525718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542408+0000","last_change":"2023-11-26T07:12:54.531548+0000","last_active":"2023-11-26T07:12:54.542408+0000","last_peered":"2023-11-26T07:12:54.542408+0000","last_clean":"2023-11-26T07:12:54.542408+0000","last_became_active":"2023-11-26T07:12:54.531133+0000","last_became_peered":"2023-11-26T07:12:54.531133+0000","last_unstale":"2023-11-26T07:12:54.542408+0000","last_undegraded":"2023-11-26T07:12:54.542408+0000","last_fullsized":"2023-11-26T07:12:54.542408+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:58.991270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519968+0000","last_change":"2023-11-26T07:12:54.519968+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519968+0000","last_undegraded":"2023-11-26T07:12:54.519968+0000","last_fullsized":"2023-11-26T07:12:54.519968+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541930+0000","last_change":"2023-11-26T07:12:54.536843+0000","last_active":"2023-11-26T07:12:54.541930+0000","last_peered":"2023-11-26T07:12:54.541930+0000","last_clean":"2023-11-26T07:12:54.541930+0000","last_became_active":"2023-11-26T07:12:54.536560+0000","last_became_peered":"2023-11-26T07:12:54.536560+0000","last_unstale":"2023-11-26T07:12:54.541930+0000","last_undegraded":"2023-11-26T07:12:54.541930+0000","last_fullsized":"2023-11-26T07:12:54.541930+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:47.592063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519772+0000","last_change":"2023-11-26T07:12:54.519772+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519772+0000","last_undegraded":"2023-11-26T07:12:54.519772+0000","last_fullsized":"2023-11-26T07:12:54.519772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.529907+0000","last_change":"2023-11-26T07:12:54.527254+0000","last_active":"2023-11-26T07:12:54.529907+0000","last_peered":"2023-11-26T07:12:54.529907+0000","last_clean":"2023-11-26T07:12:54.529907+0000","last_became_active":"2023-11-26T07:12:54.526949+0000","last_became_peered":"2023-11-26T07:12:54.526949+0000","last_unstale":"2023-11-26T07:12:54.529907+0000","last_undegraded":"2023-11-26T07:12:54.529907+0000","last_fullsized":"2023-11-26T07:12:54.529907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:20:37.591566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519987+0000","last_change":"2023-11-26T07:12:54.519987+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519987+0000","last_undegraded":"2023-11-26T07:12:54.519987+0000","last_fullsized":"2023-11-26T07:12:54.519987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536115+0000","last_change":"2023-11-26T07:12:54.535255+0000","last_active":"2023-11-26T07:12:54.536115+0000","last_peered":"2023-11-26T07:12:54.536115+0000","last_clean":"2023-11-26T07:12:54.536115+0000","last_became_active":"2023-11-26T07:12:54.534934+0000","last_became_peered":"2023-11-26T07:12:54.534934+0000","last_unstale":"2023-11-26T07:12:54.536115+0000","last_undegraded":"2023-11-26T07:12:54.536115+0000","last_fullsized":"2023-11-26T07:12:54.536115+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:59.222804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526457+0000","last_change":"2023-11-26T07:12:54.526457+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526457+0000","last_undegraded":"2023-11-26T07:12:54.526457+0000","last_fullsized":"2023-11-26T07:12:54.526457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534475+0000","last_change":"2023-11-26T07:12:54.531509+0000","last_active":"2023-11-26T07:12:54.534475+0000","last_peered":"2023-11-26T07:12:54.534475+0000","last_clean":"2023-11-26T07:12:54.534475+0000","last_became_active":"2023-11-26T07:12:54.531168+0000","last_became_peered":"2023-11-26T07:12:54.531168+0000","last_unstale":"2023-11-26T07:12:54.534475+0000","last_undegraded":"2023-11-26T07:12:54.534475+0000","last_fullsized":"2023-11-26T07:12:54.534475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:45.828740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.527721+0000","last_change":"2023-11-26T07:12:54.527721+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.527721+0000","last_undegraded":"2023-11-26T07:12:54.527721+0000","last_fullsized":"2023-11-26T07:12:54.527721+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542544+0000","last_change":"2023-11-26T07:12:54.536077+0000","last_active":"2023-11-26T07:12:54.542544+0000","last_peered":"2023-11-26T07:12:54.542544+0000","last_clean":"2023-11-26T07:12:54.542544+0000","last_became_active":"2023-11-26T07:12:54.535793+0000","last_became_peered":"2023-11-26T07:12:54.535793+0000","last_unstale":"2023-11-26T07:12:54.542544+0000","last_undegraded":"2023-11-26T07:12:54.542544+0000","last_fullsized":"2023-11-26T07:12:54.542544+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:27:43.076531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.529399+0000","last_change":"2023-11-26T07:12:54.529399+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.529399+0000","last_undegraded":"2023-11-26T07:12:54.529399+0000","last_fullsized":"2023-11-26T07:12:54.529399+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542631+0000","last_change":"2023-11-26T07:12:54.537136+0000","last_active":"2023-11-26T07:12:54.542631+0000","last_peered":"2023-11-26T07:12:54.542631+0000","last_clean":"2023-11-26T07:12:54.542631+0000","last_became_active":"2023-11-26T07:12:54.536557+0000","last_became_peered":"2023-11-26T07:12:54.536557+0000","last_unstale":"2023-11-26T07:12:54.542631+0000","last_undegraded":"2023-11-26T07:12:54.542631+0000","last_fullsized":"2023-11-26T07:12:54.542631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:58:57.698467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519560+0000","last_change":"2023-11-26T07:12:54.519560+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519560+0000","last_undegraded":"2023-11-26T07:12:54.519560+0000","last_fullsized":"2023-11-26T07:12:54.519560+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.536191+0000","last_change":"2023-11-26T07:12:54.533932+0000","last_active":"2023-11-26T07:12:54.536191+0000","last_peered":"2023-11-26T07:12:54.536191+0000","last_clean":"2023-11-26T07:12:54.536191+0000","last_became_active":"2023-11-26T07:12:54.533651+0000","last_became_peered":"2023-11-26T07:12:54.533651+0000","last_unstale":"2023-11-26T07:12:54.536191+0000","last_undegraded":"2023-11-26T07:12:54.536191+0000","last_fullsized":"2023-11-26T07:12:54.536191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:16.433050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520815+0000","last_change":"2023-11-26T07:12:54.520815+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520815+0000","last_undegraded":"2023-11-26T07:12:54.520815+0000","last_fullsized":"2023-11-26T07:12:54.520815+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541674+0000","last_change":"2023-11-26T07:12:54.533722+0000","last_active":"2023-11-26T07:12:54.541674+0000","last_peered":"2023-11-26T07:12:54.541674+0000","last_clean":"2023-11-26T07:12:54.541674+0000","last_became_active":"2023-11-26T07:12:54.533207+0000","last_became_peered":"2023-11-26T07:12:54.533207+0000","last_unstale":"2023-11-26T07:12:54.541674+0000","last_undegraded":"2023-11-26T07:12:54.541674+0000","last_fullsized":"2023-11-26T07:12:54.541674+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:03.463480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521433+0000","last_change":"2023-11-26T07:12:54.521433+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521433+0000","last_undegraded":"2023-11-26T07:12:54.521433+0000","last_fullsized":"2023-11-26T07:12:54.521433+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540251+0000","last_change":"2023-11-26T07:12:54.535338+0000","last_active":"2023-11-26T07:12:54.540251+0000","last_peered":"2023-11-26T07:12:54.540251+0000","last_clean":"2023-11-26T07:12:54.540251+0000","last_became_active":"2023-11-26T07:12:54.534988+0000","last_became_peered":"2023-11-26T07:12:54.534988+0000","last_unstale":"2023-11-26T07:12:54.540251+0000","last_undegraded":"2023-11-26T07:12:54.540251+0000","last_fullsized":"2023-11-26T07:12:54.540251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:58:57.698467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526021+0000","last_change":"2023-11-26T07:12:54.526021+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526021+0000","last_undegraded":"2023-11-26T07:12:54.526021+0000","last_fullsized":"2023-11-26T07:12:54.526021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542447+0000","last_change":"2023-11-26T07:12:54.531775+0000","last_active":"2023-11-26T07:12:54.542447+0000","last_peered":"2023-11-26T07:12:54.542447+0000","last_clean":"2023-11-26T07:12:54.542447+0000","last_became_active":"2023-11-26T07:12:54.531491+0000","last_became_peered":"2023-11-26T07:12:54.531491+0000","last_unstale":"2023-11-26T07:12:54.542447+0000","last_undegraded":"2023-11-26T07:12:54.542447+0000","last_fullsized":"2023-11-26T07:12:54.542447+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:45.828740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520515+0000","last_change":"2023-11-26T07:12:54.520515+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520515+0000","last_undegraded":"2023-11-26T07:12:54.520515+0000","last_fullsized":"2023-11-26T07:12:54.520515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540202+0000","last_change":"2023-11-26T07:12:54.534426+0000","last_active":"2023-11-26T07:12:54.540202+0000","last_peered":"2023-11-26T07:12:54.540202+0000","last_clean":"2023-11-26T07:12:54.540202+0000","last_became_active":"2023-11-26T07:12:54.534154+0000","last_became_peered":"2023-11-26T07:12:54.534154+0000","last_unstale":"2023-11-26T07:12:54.540202+0000","last_undegraded":"2023-11-26T07:12:54.540202+0000","last_fullsized":"2023-11-26T07:12:54.540202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:38:16.665106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.519918+0000","last_change":"2023-11-26T07:12:54.519918+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.519918+0000","last_undegraded":"2023-11-26T07:12:54.519918+0000","last_fullsized":"2023-11-26T07:12:54.519918+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.534342+0000","last_change":"2023-11-26T07:12:54.529524+0000","last_active":"2023-11-26T07:12:54.534342+0000","last_peered":"2023-11-26T07:12:54.534342+0000","last_clean":"2023-11-26T07:12:54.534342+0000","last_became_active":"2023-11-26T07:12:54.529161+0000","last_became_peered":"2023-11-26T07:12:54.529161+0000","last_unstale":"2023-11-26T07:12:54.534342+0000","last_undegraded":"2023-11-26T07:12:54.534342+0000","last_fullsized":"2023-11-26T07:12:54.534342+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:04.125066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.528822+0000","last_change":"2023-11-26T07:12:54.528822+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.528822+0000","last_undegraded":"2023-11-26T07:12:54.528822+0000","last_fullsized":"2023-11-26T07:12:54.528822+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.540137+0000","last_change":"2023-11-26T07:12:54.533531+0000","last_active":"2023-11-26T07:12:54.540137+0000","last_peered":"2023-11-26T07:12:54.540137+0000","last_clean":"2023-11-26T07:12:54.540137+0000","last_became_active":"2023-11-26T07:12:54.533142+0000","last_became_peered":"2023-11-26T07:12:54.533142+0000","last_unstale":"2023-11-26T07:12:54.540137+0000","last_undegraded":"2023-11-26T07:12:54.540137+0000","last_fullsized":"2023-11-26T07:12:54.540137+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:01:33.193390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.526115+0000","last_change":"2023-11-26T07:12:54.526115+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.526115+0000","last_undegraded":"2023-11-26T07:12:54.526115+0000","last_fullsized":"2023-11-26T07:12:54.526115+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.528932+0000","last_change":"2023-11-26T07:12:54.525145+0000","last_active":"2023-11-26T07:12:54.528932+0000","last_peered":"2023-11-26T07:12:54.528932+0000","last_clean":"2023-11-26T07:12:54.528932+0000","last_became_active":"2023-11-26T07:12:54.524871+0000","last_became_peered":"2023-11-26T07:12:54.524871+0000","last_unstale":"2023-11-26T07:12:54.528932+0000","last_undegraded":"2023-11-26T07:12:54.528932+0000","last_fullsized":"2023-11-26T07:12:54.528932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:48:46.369669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.521333+0000","last_change":"2023-11-26T07:12:54.521333+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.521333+0000","last_undegraded":"2023-11-26T07:12:54.521333+0000","last_fullsized":"2023-11-26T07:12:54.521333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.542409+0000","last_change":"2023-11-26T07:12:54.533682+0000","last_active":"2023-11-26T07:12:54.542409+0000","last_peered":"2023-11-26T07:12:54.542409+0000","last_clean":"2023-11-26T07:12:54.542409+0000","last_became_active":"2023-11-26T07:12:54.533409+0000","last_became_peered":"2023-11-26T07:12:54.533409+0000","last_unstale":"2023-11-26T07:12:54.542409+0000","last_undegraded":"2023-11-26T07:12:54.542409+0000","last_fullsized":"2023-11-26T07:12:54.542409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:23:22.243947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.518436+0000","last_change":"2023-11-26T07:12:54.518436+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.518436+0000","last_undegraded":"2023-11-26T07:12:54.518436+0000","last_fullsized":"2023-11-26T07:12:54.518436+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.541952+0000","last_change":"2023-11-26T07:12:54.537875+0000","last_active":"2023-11-26T07:12:54.541952+0000","last_peered":"2023-11-26T07:12:54.541952+0000","last_clean":"2023-11-26T07:12:54.541952+0000","last_became_active":"2023-11-26T07:12:54.537498+0000","last_became_peered":"2023-11-26T07:12:54.537498+0000","last_unstale":"2023-11-26T07:12:54.541952+0000","last_undegraded":"2023-11-26T07:12:54.541952+0000","last_fullsized":"2023-11-26T07:12:54.541952+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:31.377712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-11-26T07:12:54.520632+0000","last_change":"2023-11-26T07:12:54.520632+0000","last_active":"2023-11-26T07:12:54.508362+0000","last_peered":"2023-11-26T07:12:54.508362+0000","last_clean":"2023-11-26T07:12:54.508362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:12:54.520632+0000","last_undegraded":"2023-11-26T07:12:54.520632+0000","last_fullsized":"2023-11-26T07:12:54.520632+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:12:54.531028+0000","last_change":"2023-11-26T07:12:54.528129+0000","last_active":"2023-11-26T07:12:54.531028+0000","last_peered":"2023-11-26T07:12:54.531028+0000","last_clean":"2023-11-26T07:12:54.531028+0000","last_became_active":"2023-11-26T07:12:54.527835+0000","last_became_peered":"2023-11-26T07:12:54.527835+0000","last_unstale":"2023-11-26T07:12:54.531028+0000","last_undegraded":"2023-11-26T07:12:54.531028+0000","last_fullsized":"2023-11-26T07:12:54.531028+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:02:54.984506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-11-26T07:12:53.513412+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:12:53.513412+0000","last_peered":"2023-11-26T07:12:53.513412+0000","last_clean":"2023-11-26T07:12:53.513412+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:12:53.513412+0000","last_undegraded":"2023-11-26T07:12:53.513412+0000","last_fullsized":"2023-11-26T07:12:53.513412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.497}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55800000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.433},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.39800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.63}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.503},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67900000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.40300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.57199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.31}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.24399999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.504}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":791748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.313}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.36699999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39800000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.621}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58399999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.253},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.313}]},{"osd":1,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.40899999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.59299999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.628}]},{"osd":6,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.48599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.34799999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.9109999999999996,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.313}]},{"osd":2,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.247},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.32800000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8129999999999997,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.71799999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.883,"5min":4.883,"15min":4.883},"last":0.60699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.8499999999999996,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.67000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":201380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.47399999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.61099999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.495},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.747}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.60499999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.54300000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.621},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.64200000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.67300000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57399999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:13:04.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:13:04.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.350+0000 7f19c5413700 1 Processor -- start 2023-11-26T07:13:04.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.350+0000 7f19c5413700 1 -- start start 2023-11-26T07:13:04.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.350+0000 7f19c5413700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f19c015e1e0 0x7f19c00d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:04.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.350+0000 7f19c5413700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f19c00d0f80 0x7f19c00b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:04.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.350+0000 7f19c5413700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c00d20b0 con 0x7f19c00d3a00 2023-11-26T07:13:04.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.350+0000 7f19c5413700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c00d3150 con 0x7f19c00d0f80 2023-11-26T07:13:04.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.350+0000 7f19c5413700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c015e5c0 con 0x7f19c015e1e0 2023-11-26T07:13:04.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19bf7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f19c00d0f80 0x7f19c00b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:04.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19be7fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f19c00d3a00 0x7f19c00d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57270/0 (socket says 172.21.15.83:57270) 2023-11-26T07:13:04.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19be7fc700 1 -- 172.21.15.83:0/2424908647 learned_addr learned my addr 172.21.15.83:0/2424908647 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:04.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19beffd700 1 --2- 172.21.15.83:0/2424908647 >> v2:172.21.15.203:3300/0 conn(0x7f19c015e1e0 0x7f19c00d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:04.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19bf7fe700 1 -- 172.21.15.83:0/2424908647 >> v2:172.21.15.203:3300/0 conn(0x7f19c015e1e0 msgr2=0x7f19c00d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:04.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19bf7fe700 1 --2- 172.21.15.83:0/2424908647 >> v2:172.21.15.203:3300/0 conn(0x7f19c015e1e0 0x7f19c00d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19bf7fe700 1 -- 172.21.15.83:0/2424908647 >> v1:172.21.15.83:6789/0 conn(0x7f19c00d3a00 legacy=0x7f19c00d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:04.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19bf7fe700 1 -- 172.21.15.83:0/2424908647 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00d2390 con 0x7f19c00d0f80 2023-11-26T07:13:04.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19bf7fe700 1 --2- 172.21.15.83:0/2424908647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19c00d0f80 0x7f19c00b6ef0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f19c00d0840 tx=0x7f19b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3e703df17df5a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:04.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19bdffb700 1 -- 172.21.15.83:0/2424908647 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800e070 con 0x7f19c00d0f80 2023-11-26T07:13:04.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19bdffb700 1 -- 172.21.15.83:0/2424908647 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b800bad0 con 0x7f19c00d0f80 2023-11-26T07:13:04.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.351+0000 7f19bdffb700 1 -- 172.21.15.83:0/2424908647 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800be60 con 0x7f19c00d0f80 2023-11-26T07:13:04.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.352+0000 7f19c5413700 1 -- 172.21.15.83:0/2424908647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19c00d0f80 msgr2=0x7f19c00b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:04.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.352+0000 7f19c5413700 1 --2- 172.21.15.83:0/2424908647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19c00d0f80 0x7f19c00b6ef0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f19c00d0840 tx=0x7f19b800b330 comp rx=0 tx=0).stop 2023-11-26T07:13:04.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.352+0000 7f19c5413700 1 -- 172.21.15.83:0/2424908647 shutdown_connections 2023-11-26T07:13:04.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.352+0000 7f19c5413700 1 --2- 172.21.15.83:0/2424908647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19c00d0f80 0x7f19c00b6ef0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.352+0000 7f19c5413700 1 --2- 172.21.15.83:0/2424908647 >> v2:172.21.15.203:3300/0 conn(0x7f19c015e1e0 0x7f19c00d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.352+0000 7f19c5413700 1 -- 172.21.15.83:0/2424908647 >> 172.21.15.83:0/2424908647 conn(0x7f19c00b5b50 msgr2=0x7f19c00b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:04.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.352+0000 7f19c5413700 1 -- 172.21.15.83:0/2424908647 shutdown_connections 2023-11-26T07:13:04.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.352+0000 7f19c5413700 1 -- 172.21.15.83:0/2424908647 wait complete. 2023-11-26T07:13:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.353+0000 7f19c5413700 1 Processor -- start 2023-11-26T07:13:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.353+0000 7f19c5413700 1 -- start start 2023-11-26T07:13:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19c5413700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c00d0f80 0x7f19c015e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19c5413700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19c00d3a00 0x7f19c015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19c5413700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19c015e1e0 0x7f19c015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19c5413700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c014e600 con 0x7f19c00d3a00 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19c5413700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00caf30 con 0x7f19c00d0f80 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19c5413700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c015f7e0 con 0x7f19c015e1e0 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19be7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19c00d3a00 0x7f19c015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19be7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19c00d3a00 0x7f19c015ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37236/0 (socket says 172.21.15.83:37236) 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19be7fc700 1 -- 172.21.15.83:0/2144791677 learned_addr learned my addr 172.21.15.83:0/2144791677 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19beffd700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c00d0f80 0x7f19c015e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19bf7fe700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19c015e1e0 0x7f19c015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19be7fc700 1 -- 172.21.15.83:0/2144791677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19c015e1e0 msgr2=0x7f19c015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19be7fc700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19c015e1e0 0x7f19c015f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19be7fc700 1 -- 172.21.15.83:0/2144791677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c00d0f80 msgr2=0x7f19c015e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19be7fc700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c00d0f80 0x7f19c015e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19be7fc700 1 -- 172.21.15.83:0/2144791677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b8009a70 con 0x7f19c00d3a00 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.354+0000 7f19beffd700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c00d0f80 0x7f19c015e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.355+0000 7f19be7fc700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19c00d3a00 0x7f19c015ec70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f19b400c750 tx=0x7f19b400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.355+0000 7f19a77fe700 1 -- 172.21.15.83:0/2144791677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4004c80 con 0x7f19c00d3a00 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.355+0000 7f19a77fe700 1 -- 172.21.15.83:0/2144791677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b4004de0 con 0x7f19c00d3a00 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.355+0000 7f19a77fe700 1 -- 172.21.15.83:0/2144791677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4007310 con 0x7f19c00d3a00 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.355+0000 7f19c5413700 1 -- 172.21.15.83:0/2144791677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c015fac0 con 0x7f19c00d3a00 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.355+0000 7f19c5413700 1 -- 172.21.15.83:0/2144791677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c015fe50 con 0x7f19c00d3a00 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.356+0000 7f19c5413700 1 -- 172.21.15.83:0/2144791677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c00cebb0 con 0x7f19c00d3a00 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.356+0000 7f19a77fe700 1 -- 172.21.15.83:0/2144791677 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f19b4007470 con 0x7f19c00d3a00 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.357+0000 7f19a77fe700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f19ac06b080 0x7f19ac06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.357+0000 7f19a77fe700 1 -- 172.21.15.83:0/2144791677 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19b4088a90 con 0x7f19c00d3a00 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.357+0000 7f19beffd700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f19ac06b080 0x7f19ac06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.357+0000 7f19beffd700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f19ac06b080 0x7f19ac06d540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f19b0007a50 tx=0x7f19b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.361+0000 7f19a77fe700 1 -- 172.21.15.83:0/2144791677 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19b40538c0 con 0x7f19c00d3a00 2023-11-26T07:13:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.549+0000 7f19c5413700 1 -- 172.21.15.83:0/2144791677 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19c00d2c30 con 0x7f19ac06b080 2023-11-26T07:13:04.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.557+0000 7f19a77fe700 1 -- 172.21.15.83:0/2144791677 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289049 (secure 0 0 0) 0x7f19b0007260 con 0x7f19ac06b080 2023-11-26T07:13:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 -- 172.21.15.83:0/2144791677 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f19ac06b080 msgr2=0x7f19ac06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f19ac06b080 0x7f19ac06d540 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f19b0007a50 tx=0x7f19b0008040 comp rx=0 tx=0).stop 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 -- 172.21.15.83:0/2144791677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19c00d3a00 msgr2=0x7f19c015ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19c00d3a00 0x7f19c015ec70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f19b400c750 tx=0x7f19b400cb10 comp rx=0 tx=0).stop 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 -- 172.21.15.83:0/2144791677 shutdown_connections 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f19ac06b080 0x7f19ac06d540 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19c015e1e0 0x7f19c015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19c00d0f80 0x7f19c015e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 --2- 172.21.15.83:0/2144791677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19c00d3a00 0x7f19c015ec70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 -- 172.21.15.83:0/2144791677 >> 172.21.15.83:0/2144791677 conn(0x7f19c00b5b50 msgr2=0x7f19c00b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 -- 172.21.15.83:0/2144791677 shutdown_connections 2023-11-26T07:13:04.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.560+0000 7f19c5413700 1 -- 172.21.15.83:0/2144791677 wait complete. 2023-11-26T07:13:04.565 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:13:04.573 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-11-26T07:13:04.326716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221540,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753180,"statfs":{"total":773094113280,"available":772867256320,"internally_reserved":0,"allocated":5480448,"data_stored":2901196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.810758"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.523602+0000","last_change":"2023-11-26T07:12:55.536749+0000","last_active":"2023-11-26T07:12:56.523602+0000","last_peered":"2023-11-26T07:12:56.523602+0000","last_clean":"2023-11-26T07:12:56.523602+0000","last_became_active":"2023-11-26T07:12:55.536412+0000","last_became_peered":"2023-11-26T07:12:55.536412+0000","last_unstale":"2023-11-26T07:12:56.523602+0000","last_undegraded":"2023-11-26T07:12:56.523602+0000","last_fullsized":"2023-11-26T07:12:56.523602+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:08:52.459657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.523773+0000","last_change":"2023-11-26T07:12:55.546802+0000","last_active":"2023-11-26T07:12:56.523773+0000","last_peered":"2023-11-26T07:12:56.523773+0000","last_clean":"2023-11-26T07:12:56.523773+0000","last_became_active":"2023-11-26T07:12:55.546480+0000","last_became_peered":"2023-11-26T07:12:55.546480+0000","last_unstale":"2023-11-26T07:12:56.523773+0000","last_undegraded":"2023-11-26T07:12:56.523773+0000","last_fullsized":"2023-11-26T07:12:56.523773+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:00.149141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.542249+0000","last_change":"2023-11-26T07:12:55.530616+0000","last_active":"2023-11-26T07:12:56.542249+0000","last_peered":"2023-11-26T07:12:56.542249+0000","last_clean":"2023-11-26T07:12:56.542249+0000","last_became_active":"2023-11-26T07:12:55.530308+0000","last_became_peered":"2023-11-26T07:12:55.530308+0000","last_unstale":"2023-11-26T07:12:56.542249+0000","last_undegraded":"2023-11-26T07:12:56.542249+0000","last_fullsized":"2023-11-26T07:12:56.542249+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:20:46.043819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762283+0000","last_change":"2023-11-26T07:12:55.545669+0000","last_active":"2023-11-26T07:12:56.762283+0000","last_peered":"2023-11-26T07:12:56.762283+0000","last_clean":"2023-11-26T07:12:56.762283+0000","last_became_active":"2023-11-26T07:12:55.545374+0000","last_became_peered":"2023-11-26T07:12:55.545374+0000","last_unstale":"2023-11-26T07:12:56.762283+0000","last_undegraded":"2023-11-26T07:12:56.762283+0000","last_fullsized":"2023-11-26T07:12:56.762283+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:36:58.637642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.523323+0000","last_change":"2023-11-26T07:12:55.546648+0000","last_active":"2023-11-26T07:12:56.523323+0000","last_peered":"2023-11-26T07:12:56.523323+0000","last_clean":"2023-11-26T07:12:56.523323+0000","last_became_active":"2023-11-26T07:12:55.546355+0000","last_became_peered":"2023-11-26T07:12:55.546355+0000","last_unstale":"2023-11-26T07:12:56.523323+0000","last_undegraded":"2023-11-26T07:12:56.523323+0000","last_fullsized":"2023-11-26T07:12:56.523323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:49.225243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762154+0000","last_change":"2023-11-26T07:12:55.540830+0000","last_active":"2023-11-26T07:12:56.762154+0000","last_peered":"2023-11-26T07:12:56.762154+0000","last_clean":"2023-11-26T07:12:56.762154+0000","last_became_active":"2023-11-26T07:12:55.540396+0000","last_became_peered":"2023-11-26T07:12:55.540396+0000","last_unstale":"2023-11-26T07:12:56.762154+0000","last_undegraded":"2023-11-26T07:12:56.762154+0000","last_fullsized":"2023-11-26T07:12:56.762154+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:02:58.048967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.523683+0000","last_change":"2023-11-26T07:12:55.542059+0000","last_active":"2023-11-26T07:12:56.523683+0000","last_peered":"2023-11-26T07:12:56.523683+0000","last_clean":"2023-11-26T07:12:56.523683+0000","last_became_active":"2023-11-26T07:12:55.541679+0000","last_became_peered":"2023-11-26T07:12:55.541679+0000","last_unstale":"2023-11-26T07:12:56.523683+0000","last_undegraded":"2023-11-26T07:12:56.523683+0000","last_fullsized":"2023-11-26T07:12:56.523683+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:47.695036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762209+0000","last_change":"2023-11-26T07:12:55.547069+0000","last_active":"2023-11-26T07:12:56.762209+0000","last_peered":"2023-11-26T07:12:56.762209+0000","last_clean":"2023-11-26T07:12:56.762209+0000","last_became_active":"2023-11-26T07:12:55.546547+0000","last_became_peered":"2023-11-26T07:12:55.546547+0000","last_unstale":"2023-11-26T07:12:56.762209+0000","last_undegraded":"2023-11-26T07:12:56.762209+0000","last_fullsized":"2023-11-26T07:12:56.762209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:25:36.940538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762452+0000","last_change":"2023-11-26T07:12:55.545353+0000","last_active":"2023-11-26T07:12:56.762452+0000","last_peered":"2023-11-26T07:12:56.762452+0000","last_clean":"2023-11-26T07:12:56.762452+0000","last_became_active":"2023-11-26T07:12:55.545081+0000","last_became_peered":"2023-11-26T07:12:55.545081+0000","last_unstale":"2023-11-26T07:12:56.762452+0000","last_undegraded":"2023-11-26T07:12:56.762452+0000","last_fullsized":"2023-11-26T07:12:56.762452+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:38:17.667144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762098+0000","last_change":"2023-11-26T07:12:55.545926+0000","last_active":"2023-11-26T07:12:56.762098+0000","last_peered":"2023-11-26T07:12:56.762098+0000","last_clean":"2023-11-26T07:12:56.762098+0000","last_became_active":"2023-11-26T07:12:55.545677+0000","last_became_peered":"2023-11-26T07:12:55.545677+0000","last_unstale":"2023-11-26T07:12:56.762098+0000","last_undegraded":"2023-11-26T07:12:56.762098+0000","last_fullsized":"2023-11-26T07:12:56.762098+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:27:44.078568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762634+0000","last_change":"2023-11-26T07:12:55.541068+0000","last_active":"2023-11-26T07:12:56.762634+0000","last_peered":"2023-11-26T07:12:56.762634+0000","last_clean":"2023-11-26T07:12:56.762634+0000","last_became_active":"2023-11-26T07:12:55.540820+0000","last_became_peered":"2023-11-26T07:12:55.540820+0000","last_unstale":"2023-11-26T07:12:56.762634+0000","last_undegraded":"2023-11-26T07:12:56.762634+0000","last_fullsized":"2023-11-26T07:12:56.762634+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:22:20.487642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.523506+0000","last_change":"2023-11-26T07:12:55.533814+0000","last_active":"2023-11-26T07:12:56.523506+0000","last_peered":"2023-11-26T07:12:56.523506+0000","last_clean":"2023-11-26T07:12:56.523506+0000","last_became_active":"2023-11-26T07:12:55.533502+0000","last_became_peered":"2023-11-26T07:12:55.533502+0000","last_unstale":"2023-11-26T07:12:56.523506+0000","last_undegraded":"2023-11-26T07:12:56.523506+0000","last_fullsized":"2023-11-26T07:12:56.523506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:58:42.073243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.520924+0000","last_change":"2023-11-26T07:12:55.536914+0000","last_active":"2023-11-26T07:12:56.520924+0000","last_peered":"2023-11-26T07:12:56.520924+0000","last_clean":"2023-11-26T07:12:56.520924+0000","last_became_active":"2023-11-26T07:12:55.536526+0000","last_became_peered":"2023-11-26T07:12:55.536526+0000","last_unstale":"2023-11-26T07:12:56.520924+0000","last_undegraded":"2023-11-26T07:12:56.520924+0000","last_fullsized":"2023-11-26T07:12:56.520924+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:45:18.798313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762579+0000","last_change":"2023-11-26T07:12:55.539113+0000","last_active":"2023-11-26T07:12:56.762579+0000","last_peered":"2023-11-26T07:12:56.762579+0000","last_clean":"2023-11-26T07:12:56.762579+0000","last_became_active":"2023-11-26T07:12:55.533799+0000","last_became_peered":"2023-11-26T07:12:55.533799+0000","last_unstale":"2023-11-26T07:12:56.762579+0000","last_undegraded":"2023-11-26T07:12:56.762579+0000","last_fullsized":"2023-11-26T07:12:56.762579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:10:12.066814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.761678+0000","last_change":"2023-11-26T07:12:55.534720+0000","last_active":"2023-11-26T07:12:56.761678+0000","last_peered":"2023-11-26T07:12:56.761678+0000","last_clean":"2023-11-26T07:12:56.761678+0000","last_became_active":"2023-11-26T07:12:55.534441+0000","last_became_peered":"2023-11-26T07:12:55.534441+0000","last_unstale":"2023-11-26T07:12:56.761678+0000","last_undegraded":"2023-11-26T07:12:56.761678+0000","last_fullsized":"2023-11-26T07:12:56.761678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:08:52.459657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762228+0000","last_change":"2023-11-26T07:12:55.550151+0000","last_active":"2023-11-26T07:12:56.762228+0000","last_peered":"2023-11-26T07:12:56.762228+0000","last_clean":"2023-11-26T07:12:56.762228+0000","last_became_active":"2023-11-26T07:12:55.549873+0000","last_became_peered":"2023-11-26T07:12:55.549873+0000","last_unstale":"2023-11-26T07:12:56.762228+0000","last_undegraded":"2023-11-26T07:12:56.762228+0000","last_fullsized":"2023-11-26T07:12:56.762228+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:02:48.160400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.542519+0000","last_change":"2023-11-26T07:12:55.534303+0000","last_active":"2023-11-26T07:12:56.542519+0000","last_peered":"2023-11-26T07:12:56.542519+0000","last_clean":"2023-11-26T07:12:56.542519+0000","last_became_active":"2023-11-26T07:12:55.533987+0000","last_became_peered":"2023-11-26T07:12:55.533987+0000","last_unstale":"2023-11-26T07:12:56.542519+0000","last_undegraded":"2023-11-26T07:12:56.542519+0000","last_fullsized":"2023-11-26T07:12:56.542519+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:25:36.940538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.542453+0000","last_change":"2023-11-26T07:12:55.546487+0000","last_active":"2023-11-26T07:12:56.542453+0000","last_peered":"2023-11-26T07:12:56.542453+0000","last_clean":"2023-11-26T07:12:56.542453+0000","last_became_active":"2023-11-26T07:12:55.546158+0000","last_became_peered":"2023-11-26T07:12:55.546158+0000","last_unstale":"2023-11-26T07:12:56.542453+0000","last_undegraded":"2023-11-26T07:12:56.542453+0000","last_fullsized":"2023-11-26T07:12:56.542453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:58:58.700504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762452+0000","last_change":"2023-11-26T07:12:55.547922+0000","last_active":"2023-11-26T07:12:56.762452+0000","last_peered":"2023-11-26T07:12:56.762452+0000","last_clean":"2023-11-26T07:12:56.762452+0000","last_became_active":"2023-11-26T07:12:55.547198+0000","last_became_peered":"2023-11-26T07:12:55.547198+0000","last_unstale":"2023-11-26T07:12:56.762452+0000","last_undegraded":"2023-11-26T07:12:56.762452+0000","last_fullsized":"2023-11-26T07:12:56.762452+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:22:00.643900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.520934+0000","last_change":"2023-11-26T07:12:55.547247+0000","last_active":"2023-11-26T07:12:56.520934+0000","last_peered":"2023-11-26T07:12:56.520934+0000","last_clean":"2023-11-26T07:12:56.520934+0000","last_became_active":"2023-11-26T07:12:55.546943+0000","last_became_peered":"2023-11-26T07:12:55.546943+0000","last_unstale":"2023-11-26T07:12:56.520934+0000","last_undegraded":"2023-11-26T07:12:56.520934+0000","last_fullsized":"2023-11-26T07:12:56.520934+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:05.127103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762257+0000","last_change":"2023-11-26T07:12:55.545815+0000","last_active":"2023-11-26T07:12:56.762257+0000","last_peered":"2023-11-26T07:12:56.762257+0000","last_clean":"2023-11-26T07:12:56.762257+0000","last_became_active":"2023-11-26T07:12:55.545539+0000","last_became_peered":"2023-11-26T07:12:55.545539+0000","last_unstale":"2023-11-26T07:12:56.762257+0000","last_undegraded":"2023-11-26T07:12:56.762257+0000","last_fullsized":"2023-11-26T07:12:56.762257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:44:52.981896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762684+0000","last_change":"2023-11-26T07:12:55.539577+0000","last_active":"2023-11-26T07:12:56.762684+0000","last_peered":"2023-11-26T07:12:56.762684+0000","last_clean":"2023-11-26T07:12:56.762684+0000","last_became_active":"2023-11-26T07:12:55.539192+0000","last_became_peered":"2023-11-26T07:12:55.539192+0000","last_unstale":"2023-11-26T07:12:56.762684+0000","last_undegraded":"2023-11-26T07:12:56.762684+0000","last_fullsized":"2023-11-26T07:12:56.762684+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:36.356765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762087+0000","last_change":"2023-11-26T07:12:55.542743+0000","last_active":"2023-11-26T07:12:56.762087+0000","last_peered":"2023-11-26T07:12:56.762087+0000","last_clean":"2023-11-26T07:12:56.762087+0000","last_became_active":"2023-11-26T07:12:55.542291+0000","last_became_peered":"2023-11-26T07:12:55.542291+0000","last_unstale":"2023-11-26T07:12:56.762087+0000","last_undegraded":"2023-11-26T07:12:56.762087+0000","last_fullsized":"2023-11-26T07:12:56.762087+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:56:55.813612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762356+0000","last_change":"2023-11-26T07:12:55.540177+0000","last_active":"2023-11-26T07:12:56.762356+0000","last_peered":"2023-11-26T07:12:56.762356+0000","last_clean":"2023-11-26T07:12:56.762356+0000","last_became_active":"2023-11-26T07:12:55.539833+0000","last_became_peered":"2023-11-26T07:12:55.539833+0000","last_unstale":"2023-11-26T07:12:56.762356+0000","last_undegraded":"2023-11-26T07:12:56.762356+0000","last_fullsized":"2023-11-26T07:12:56.762356+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:58:46.200072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762742+0000","last_change":"2023-11-26T07:12:55.539046+0000","last_active":"2023-11-26T07:12:56.762742+0000","last_peered":"2023-11-26T07:12:56.762742+0000","last_clean":"2023-11-26T07:12:56.762742+0000","last_became_active":"2023-11-26T07:12:55.538775+0000","last_became_peered":"2023-11-26T07:12:55.538775+0000","last_unstale":"2023-11-26T07:12:56.762742+0000","last_undegraded":"2023-11-26T07:12:56.762742+0000","last_fullsized":"2023-11-26T07:12:56.762742+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:02:55.986543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.521085+0000","last_change":"2023-11-26T07:12:55.536864+0000","last_active":"2023-11-26T07:12:56.521085+0000","last_peered":"2023-11-26T07:12:56.521085+0000","last_clean":"2023-11-26T07:12:56.521085+0000","last_became_active":"2023-11-26T07:12:55.536553+0000","last_became_peered":"2023-11-26T07:12:55.536553+0000","last_unstale":"2023-11-26T07:12:56.521085+0000","last_undegraded":"2023-11-26T07:12:56.521085+0000","last_fullsized":"2023-11-26T07:12:56.521085+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:29:00.224841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.542385+0000","last_change":"2023-11-26T07:12:55.533915+0000","last_active":"2023-11-26T07:12:56.542385+0000","last_peered":"2023-11-26T07:12:56.542385+0000","last_clean":"2023-11-26T07:12:56.542385+0000","last_became_active":"2023-11-26T07:12:55.533610+0000","last_became_peered":"2023-11-26T07:12:55.533610+0000","last_unstale":"2023-11-26T07:12:56.542385+0000","last_undegraded":"2023-11-26T07:12:56.542385+0000","last_fullsized":"2023-11-26T07:12:56.542385+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:01:34.195427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.523117+0000","last_change":"2023-11-26T07:12:55.533623+0000","last_active":"2023-11-26T07:12:56.523117+0000","last_peered":"2023-11-26T07:12:56.523117+0000","last_clean":"2023-11-26T07:12:56.523117+0000","last_became_active":"2023-11-26T07:12:55.533294+0000","last_became_peered":"2023-11-26T07:12:55.533294+0000","last_unstale":"2023-11-26T07:12:56.523117+0000","last_undegraded":"2023-11-26T07:12:56.523117+0000","last_fullsized":"2023-11-26T07:12:56.523117+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:42:36.356765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762347+0000","last_change":"2023-11-26T07:12:55.538927+0000","last_active":"2023-11-26T07:12:56.762347+0000","last_peered":"2023-11-26T07:12:56.762347+0000","last_clean":"2023-11-26T07:12:56.762347+0000","last_became_active":"2023-11-26T07:12:55.538635+0000","last_became_peered":"2023-11-26T07:12:55.538635+0000","last_unstale":"2023-11-26T07:12:56.762347+0000","last_undegraded":"2023-11-26T07:12:56.762347+0000","last_fullsized":"2023-11-26T07:12:56.762347+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:19:28.651226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.542175+0000","last_change":"2023-11-26T07:12:55.530114+0000","last_active":"2023-11-26T07:12:56.542175+0000","last_peered":"2023-11-26T07:12:56.542175+0000","last_clean":"2023-11-26T07:12:56.542175+0000","last_became_active":"2023-11-26T07:12:55.529793+0000","last_became_peered":"2023-11-26T07:12:55.529793+0000","last_unstale":"2023-11-26T07:12:56.542175+0000","last_undegraded":"2023-11-26T07:12:56.542175+0000","last_fullsized":"2023-11-26T07:12:56.542175+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:46.830777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.761730+0000","last_change":"2023-11-26T07:12:55.538880+0000","last_active":"2023-11-26T07:12:56.761730+0000","last_peered":"2023-11-26T07:12:56.761730+0000","last_clean":"2023-11-26T07:12:56.761730+0000","last_became_active":"2023-11-26T07:12:55.538644+0000","last_became_peered":"2023-11-26T07:12:55.538644+0000","last_unstale":"2023-11-26T07:12:56.761730+0000","last_undegraded":"2023-11-26T07:12:56.761730+0000","last_fullsized":"2023-11-26T07:12:56.761730+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:47.695036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.520741+0000","last_change":"2023-11-26T07:12:55.528283+0000","last_active":"2023-11-26T07:12:56.520741+0000","last_peered":"2023-11-26T07:12:56.520741+0000","last_clean":"2023-11-26T07:12:56.520741+0000","last_became_active":"2023-11-26T07:12:55.527740+0000","last_became_peered":"2023-11-26T07:12:55.527740+0000","last_unstale":"2023-11-26T07:12:56.520741+0000","last_undegraded":"2023-11-26T07:12:56.520741+0000","last_fullsized":"2023-11-26T07:12:56.520741+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:48:47.371706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762084+0000","last_change":"2023-11-26T07:12:55.537612+0000","last_active":"2023-11-26T07:12:56.762084+0000","last_peered":"2023-11-26T07:12:56.762084+0000","last_clean":"2023-11-26T07:12:56.762084+0000","last_became_active":"2023-11-26T07:12:55.537339+0000","last_became_peered":"2023-11-26T07:12:55.537339+0000","last_unstale":"2023-11-26T07:12:56.762084+0000","last_undegraded":"2023-11-26T07:12:56.762084+0000","last_fullsized":"2023-11-26T07:12:56.762084+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:07:00.097693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.546215+0000","last_change":"2023-11-26T07:12:54.536815+0000","last_active":"2023-11-26T07:12:55.546215+0000","last_peered":"2023-11-26T07:12:55.546215+0000","last_clean":"2023-11-26T07:12:55.546215+0000","last_became_active":"2023-11-26T07:12:54.536526+0000","last_became_peered":"2023-11-26T07:12:54.536526+0000","last_unstale":"2023-11-26T07:12:55.546215+0000","last_undegraded":"2023-11-26T07:12:55.546215+0000","last_fullsized":"2023-11-26T07:12:55.546215+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:01:33.193390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762588+0000","last_change":"2023-11-26T07:12:55.538418+0000","last_active":"2023-11-26T07:12:56.762588+0000","last_peered":"2023-11-26T07:12:56.762588+0000","last_clean":"2023-11-26T07:12:56.762588+0000","last_became_active":"2023-11-26T07:12:55.538125+0000","last_became_peered":"2023-11-26T07:12:55.538125+0000","last_unstale":"2023-11-26T07:12:56.762588+0000","last_undegraded":"2023-11-26T07:12:56.762588+0000","last_fullsized":"2023-11-26T07:12:56.762588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:59.993307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.540467+0000","last_change":"2023-11-26T07:12:54.533729+0000","last_active":"2023-11-26T07:12:55.540467+0000","last_peered":"2023-11-26T07:12:55.540467+0000","last_clean":"2023-11-26T07:12:55.540467+0000","last_became_active":"2023-11-26T07:12:54.533440+0000","last_became_peered":"2023-11-26T07:12:54.533440+0000","last_unstale":"2023-11-26T07:12:55.540467+0000","last_undegraded":"2023-11-26T07:12:55.540467+0000","last_fullsized":"2023-11-26T07:12:55.540467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:16.433050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.523275+0000","last_change":"2023-11-26T07:12:55.535712+0000","last_active":"2023-11-26T07:12:56.523275+0000","last_peered":"2023-11-26T07:12:56.523275+0000","last_clean":"2023-11-26T07:12:56.523275+0000","last_became_active":"2023-11-26T07:12:55.535406+0000","last_became_peered":"2023-11-26T07:12:55.535406+0000","last_unstale":"2023-11-26T07:12:56.523275+0000","last_undegraded":"2023-11-26T07:12:56.523275+0000","last_fullsized":"2023-11-26T07:12:56.523275+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:51:09.256382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.538008+0000","last_change":"2023-11-26T07:12:54.529395+0000","last_active":"2023-11-26T07:12:55.538008+0000","last_peered":"2023-11-26T07:12:55.538008+0000","last_clean":"2023-11-26T07:12:55.538008+0000","last_became_active":"2023-11-26T07:12:54.529093+0000","last_became_peered":"2023-11-26T07:12:54.529093+0000","last_unstale":"2023-11-26T07:12:55.538008+0000","last_undegraded":"2023-11-26T07:12:55.538008+0000","last_fullsized":"2023-11-26T07:12:55.538008+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:19:27.649189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.761895+0000","last_change":"2023-11-26T07:12:55.538545+0000","last_active":"2023-11-26T07:12:56.761895+0000","last_peered":"2023-11-26T07:12:56.761895+0000","last_clean":"2023-11-26T07:12:56.761895+0000","last_became_active":"2023-11-26T07:12:55.538245+0000","last_became_peered":"2023-11-26T07:12:55.538245+0000","last_unstale":"2023-11-26T07:12:56.761895+0000","last_undegraded":"2023-11-26T07:12:56.761895+0000","last_fullsized":"2023-11-26T07:12:56.761895+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:02:48.160400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.537486+0000","last_change":"2023-11-26T07:12:54.529608+0000","last_active":"2023-11-26T07:12:55.537486+0000","last_peered":"2023-11-26T07:12:55.537486+0000","last_clean":"2023-11-26T07:12:55.537486+0000","last_became_active":"2023-11-26T07:12:54.529334+0000","last_became_peered":"2023-11-26T07:12:54.529334+0000","last_unstale":"2023-11-26T07:12:55.537486+0000","last_undegraded":"2023-11-26T07:12:55.537486+0000","last_fullsized":"2023-11-26T07:12:55.537486+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:51.735731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.542201+0000","last_change":"2023-11-26T07:12:55.537662+0000","last_active":"2023-11-26T07:12:56.542201+0000","last_peered":"2023-11-26T07:12:56.542201+0000","last_clean":"2023-11-26T07:12:56.542201+0000","last_became_active":"2023-11-26T07:12:55.537377+0000","last_became_peered":"2023-11-26T07:12:55.537377+0000","last_unstale":"2023-11-26T07:12:56.542201+0000","last_undegraded":"2023-11-26T07:12:56.542201+0000","last_fullsized":"2023-11-26T07:12:56.542201+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:27:44.078568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.541314+0000","last_change":"2023-11-26T07:12:54.532202+0000","last_active":"2023-11-26T07:12:55.541314+0000","last_peered":"2023-11-26T07:12:55.541314+0000","last_clean":"2023-11-26T07:12:55.541314+0000","last_became_active":"2023-11-26T07:12:54.531692+0000","last_became_peered":"2023-11-26T07:12:54.531692+0000","last_unstale":"2023-11-26T07:12:55.541314+0000","last_undegraded":"2023-11-26T07:12:55.541314+0000","last_fullsized":"2023-11-26T07:12:55.541314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:48.223206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762474+0000","last_change":"2023-11-26T07:12:55.545545+0000","last_active":"2023-11-26T07:12:56.762474+0000","last_peered":"2023-11-26T07:12:56.762474+0000","last_clean":"2023-11-26T07:12:56.762474+0000","last_became_active":"2023-11-26T07:12:55.545263+0000","last_became_peered":"2023-11-26T07:12:55.545263+0000","last_unstale":"2023-11-26T07:12:56.762474+0000","last_undegraded":"2023-11-26T07:12:56.762474+0000","last_fullsized":"2023-11-26T07:12:56.762474+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:01:34.195427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.533467+0000","last_change":"2023-11-26T07:12:54.529445+0000","last_active":"2023-11-26T07:12:55.533467+0000","last_peered":"2023-11-26T07:12:55.533467+0000","last_clean":"2023-11-26T07:12:55.533467+0000","last_became_active":"2023-11-26T07:12:54.529160+0000","last_became_peered":"2023-11-26T07:12:54.529160+0000","last_unstale":"2023-11-26T07:12:55.533467+0000","last_undegraded":"2023-11-26T07:12:55.533467+0000","last_fullsized":"2023-11-26T07:12:55.533467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:19:27.649189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762073+0000","last_change":"2023-11-26T07:12:55.547233+0000","last_active":"2023-11-26T07:12:56.762073+0000","last_peered":"2023-11-26T07:12:56.762073+0000","last_clean":"2023-11-26T07:12:56.762073+0000","last_became_active":"2023-11-26T07:12:55.546822+0000","last_became_peered":"2023-11-26T07:12:55.546822+0000","last_unstale":"2023-11-26T07:12:56.762073+0000","last_undegraded":"2023-11-26T07:12:56.762073+0000","last_fullsized":"2023-11-26T07:12:56.762073+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:02:55.986543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.526796+0000","last_change":"2023-11-26T07:12:54.531344+0000","last_active":"2023-11-26T07:12:55.526796+0000","last_peered":"2023-11-26T07:12:55.526796+0000","last_clean":"2023-11-26T07:12:55.526796+0000","last_became_active":"2023-11-26T07:12:54.530914+0000","last_became_peered":"2023-11-26T07:12:54.530914+0000","last_unstale":"2023-11-26T07:12:55.526796+0000","last_undegraded":"2023-11-26T07:12:55.526796+0000","last_fullsized":"2023-11-26T07:12:55.526796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:01:33.193390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762425+0000","last_change":"2023-11-26T07:12:55.542573+0000","last_active":"2023-11-26T07:12:56.762425+0000","last_peered":"2023-11-26T07:12:56.762425+0000","last_clean":"2023-11-26T07:12:56.762425+0000","last_became_active":"2023-11-26T07:12:55.541912+0000","last_became_peered":"2023-11-26T07:12:55.541912+0000","last_unstale":"2023-11-26T07:12:56.762425+0000","last_undegraded":"2023-11-26T07:12:56.762425+0000","last_fullsized":"2023-11-26T07:12:56.762425+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:07:00.097693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.526320+0000","last_change":"2023-11-26T07:12:54.533250+0000","last_active":"2023-11-26T07:12:55.526320+0000","last_peered":"2023-11-26T07:12:55.526320+0000","last_clean":"2023-11-26T07:12:55.526320+0000","last_became_active":"2023-11-26T07:12:54.532934+0000","last_became_peered":"2023-11-26T07:12:54.532934+0000","last_unstale":"2023-11-26T07:12:55.526320+0000","last_undegraded":"2023-11-26T07:12:55.526320+0000","last_fullsized":"2023-11-26T07:12:55.526320+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:25:59.772453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.542581+0000","last_change":"2023-11-26T07:12:55.546431+0000","last_active":"2023-11-26T07:12:56.542581+0000","last_peered":"2023-11-26T07:12:56.542581+0000","last_clean":"2023-11-26T07:12:56.542581+0000","last_became_active":"2023-11-26T07:12:55.546105+0000","last_became_peered":"2023-11-26T07:12:55.546105+0000","last_unstale":"2023-11-26T07:12:56.542581+0000","last_undegraded":"2023-11-26T07:12:56.542581+0000","last_fullsized":"2023-11-26T07:12:56.542581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:38:17.667144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.519946+0000","last_change":"2023-11-26T07:12:54.528493+0000","last_active":"2023-11-26T07:12:55.519946+0000","last_peered":"2023-11-26T07:12:55.519946+0000","last_clean":"2023-11-26T07:12:55.519946+0000","last_became_active":"2023-11-26T07:12:54.527916+0000","last_became_peered":"2023-11-26T07:12:54.527916+0000","last_unstale":"2023-11-26T07:12:55.519946+0000","last_undegraded":"2023-11-26T07:12:55.519946+0000","last_fullsized":"2023-11-26T07:12:55.519946+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:36:57.817896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.523693+0000","last_change":"2023-11-26T07:12:55.533884+0000","last_active":"2023-11-26T07:12:56.523693+0000","last_peered":"2023-11-26T07:12:56.523693+0000","last_clean":"2023-11-26T07:12:56.523693+0000","last_became_active":"2023-11-26T07:12:55.533631+0000","last_became_peered":"2023-11-26T07:12:55.533631+0000","last_unstale":"2023-11-26T07:12:56.523693+0000","last_undegraded":"2023-11-26T07:12:56.523693+0000","last_fullsized":"2023-11-26T07:12:56.523693+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:31:46.715769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.527065+0000","last_change":"2023-11-26T07:12:54.529990+0000","last_active":"2023-11-26T07:12:55.527065+0000","last_peered":"2023-11-26T07:12:55.527065+0000","last_clean":"2023-11-26T07:12:55.527065+0000","last_became_active":"2023-11-26T07:12:54.529727+0000","last_became_peered":"2023-11-26T07:12:54.529727+0000","last_unstale":"2023-11-26T07:12:55.527065+0000","last_undegraded":"2023-11-26T07:12:55.527065+0000","last_fullsized":"2023-11-26T07:12:55.527065+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:45.828740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762174+0000","last_change":"2023-11-26T07:12:55.539903+0000","last_active":"2023-11-26T07:12:56.762174+0000","last_peered":"2023-11-26T07:12:56.762174+0000","last_clean":"2023-11-26T07:12:56.762174+0000","last_became_active":"2023-11-26T07:12:55.539485+0000","last_became_peered":"2023-11-26T07:12:55.539485+0000","last_unstale":"2023-11-26T07:12:56.762174+0000","last_undegraded":"2023-11-26T07:12:56.762174+0000","last_fullsized":"2023-11-26T07:12:56.762174+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:07:53.830574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.537891+0000","last_change":"2023-11-26T07:12:54.534123+0000","last_active":"2023-11-26T07:12:55.537891+0000","last_peered":"2023-11-26T07:12:55.537891+0000","last_clean":"2023-11-26T07:12:55.537891+0000","last_became_active":"2023-11-26T07:12:54.533831+0000","last_became_peered":"2023-11-26T07:12:54.533831+0000","last_unstale":"2023-11-26T07:12:55.537891+0000","last_undegraded":"2023-11-26T07:12:55.537891+0000","last_fullsized":"2023-11-26T07:12:55.537891+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:48.223206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.523353+0000","last_change":"2023-11-26T07:12:55.540332+0000","last_active":"2023-11-26T07:12:56.523353+0000","last_peered":"2023-11-26T07:12:56.523353+0000","last_clean":"2023-11-26T07:12:56.523353+0000","last_became_active":"2023-11-26T07:12:55.540076+0000","last_became_peered":"2023-11-26T07:12:55.540076+0000","last_unstale":"2023-11-26T07:12:56.523353+0000","last_undegraded":"2023-11-26T07:12:56.523353+0000","last_fullsized":"2023-11-26T07:12:56.523353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:08:54.043018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.547437+0000","last_change":"2023-11-26T07:12:54.528248+0000","last_active":"2023-11-26T07:12:55.547437+0000","last_peered":"2023-11-26T07:12:55.547437+0000","last_clean":"2023-11-26T07:12:55.547437+0000","last_became_active":"2023-11-26T07:12:54.527887+0000","last_became_peered":"2023-11-26T07:12:54.527887+0000","last_unstale":"2023-11-26T07:12:55.547437+0000","last_undegraded":"2023-11-26T07:12:55.547437+0000","last_fullsized":"2023-11-26T07:12:55.547437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:31.377712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762253+0000","last_change":"2023-11-26T07:12:55.547640+0000","last_active":"2023-11-26T07:12:56.762253+0000","last_peered":"2023-11-26T07:12:56.762253+0000","last_clean":"2023-11-26T07:12:56.762253+0000","last_became_active":"2023-11-26T07:12:55.547324+0000","last_became_peered":"2023-11-26T07:12:55.547324+0000","last_unstale":"2023-11-26T07:12:56.762253+0000","last_undegraded":"2023-11-26T07:12:56.762253+0000","last_fullsized":"2023-11-26T07:12:56.762253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:36:58.819933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.549832+0000","last_change":"2023-11-26T07:12:54.537145+0000","last_active":"2023-11-26T07:12:55.549832+0000","last_peered":"2023-11-26T07:12:55.549832+0000","last_clean":"2023-11-26T07:12:55.549832+0000","last_became_active":"2023-11-26T07:12:54.536921+0000","last_became_peered":"2023-11-26T07:12:54.536921+0000","last_unstale":"2023-11-26T07:12:55.549832+0000","last_undegraded":"2023-11-26T07:12:55.549832+0000","last_fullsized":"2023-11-26T07:12:55.549832+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:58:41.071206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762772+0000","last_change":"2023-11-26T07:12:55.539413+0000","last_active":"2023-11-26T07:12:56.762772+0000","last_peered":"2023-11-26T07:12:56.762772+0000","last_clean":"2023-11-26T07:12:56.762772+0000","last_became_active":"2023-11-26T07:12:55.539040+0000","last_became_peered":"2023-11-26T07:12:55.539040+0000","last_unstale":"2023-11-26T07:12:56.762772+0000","last_undegraded":"2023-11-26T07:12:56.762772+0000","last_fullsized":"2023-11-26T07:12:56.762772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:51:09.256382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.522026+0000","last_change":"2023-11-26T07:12:54.532202+0000","last_active":"2023-11-26T07:12:55.522026+0000","last_peered":"2023-11-26T07:12:55.522026+0000","last_clean":"2023-11-26T07:12:55.522026+0000","last_became_active":"2023-11-26T07:12:54.531841+0000","last_became_peered":"2023-11-26T07:12:54.531841+0000","last_unstale":"2023-11-26T07:12:55.522026+0000","last_undegraded":"2023-11-26T07:12:55.522026+0000","last_fullsized":"2023-11-26T07:12:55.522026+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:54:03.549867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.761556+0000","last_change":"2023-11-26T07:12:55.539883+0000","last_active":"2023-11-26T07:12:56.761556+0000","last_peered":"2023-11-26T07:12:56.761556+0000","last_clean":"2023-11-26T07:12:56.761556+0000","last_became_active":"2023-11-26T07:12:55.539611+0000","last_became_peered":"2023-11-26T07:12:55.539611+0000","last_unstale":"2023-11-26T07:12:56.761556+0000","last_undegraded":"2023-11-26T07:12:56.761556+0000","last_fullsized":"2023-11-26T07:12:56.761556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:45:55.024079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.532148+0000","last_change":"2023-11-26T07:12:54.531548+0000","last_active":"2023-11-26T07:12:55.532148+0000","last_peered":"2023-11-26T07:12:55.532148+0000","last_clean":"2023-11-26T07:12:55.532148+0000","last_became_active":"2023-11-26T07:12:54.531133+0000","last_became_peered":"2023-11-26T07:12:54.531133+0000","last_unstale":"2023-11-26T07:12:55.532148+0000","last_undegraded":"2023-11-26T07:12:55.532148+0000","last_fullsized":"2023-11-26T07:12:55.532148+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:49.710471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.523920+0000","last_change":"2023-11-26T07:12:55.545676+0000","last_active":"2023-11-26T07:12:56.523920+0000","last_peered":"2023-11-26T07:12:56.523920+0000","last_clean":"2023-11-26T07:12:56.523920+0000","last_became_active":"2023-11-26T07:12:55.545374+0000","last_became_peered":"2023-11-26T07:12:55.545374+0000","last_unstale":"2023-11-26T07:12:56.523920+0000","last_undegraded":"2023-11-26T07:12:56.523920+0000","last_fullsized":"2023-11-26T07:12:56.523920+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:02:58.048967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.553355+0000","last_change":"2023-11-26T07:12:54.536843+0000","last_active":"2023-11-26T07:12:55.553355+0000","last_peered":"2023-11-26T07:12:55.553355+0000","last_clean":"2023-11-26T07:12:55.553355+0000","last_became_active":"2023-11-26T07:12:54.536560+0000","last_became_peered":"2023-11-26T07:12:54.536560+0000","last_unstale":"2023-11-26T07:12:55.553355+0000","last_undegraded":"2023-11-26T07:12:55.553355+0000","last_fullsized":"2023-11-26T07:12:55.553355+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:02:47.158363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.521203+0000","last_change":"2023-11-26T07:12:55.528907+0000","last_active":"2023-11-26T07:12:56.521203+0000","last_peered":"2023-11-26T07:12:56.521203+0000","last_clean":"2023-11-26T07:12:56.521203+0000","last_became_active":"2023-11-26T07:12:55.528659+0000","last_became_peered":"2023-11-26T07:12:55.528659+0000","last_unstale":"2023-11-26T07:12:56.521203+0000","last_undegraded":"2023-11-26T07:12:56.521203+0000","last_fullsized":"2023-11-26T07:12:56.521203+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:31:45.401141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.534697+0000","last_change":"2023-11-26T07:12:54.527254+0000","last_active":"2023-11-26T07:12:55.534697+0000","last_peered":"2023-11-26T07:12:55.534697+0000","last_clean":"2023-11-26T07:12:55.534697+0000","last_became_active":"2023-11-26T07:12:54.526949+0000","last_became_peered":"2023-11-26T07:12:54.526949+0000","last_unstale":"2023-11-26T07:12:55.534697+0000","last_undegraded":"2023-11-26T07:12:55.534697+0000","last_fullsized":"2023-11-26T07:12:55.534697+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:15:29.001269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762574+0000","last_change":"2023-11-26T07:12:55.541847+0000","last_active":"2023-11-26T07:12:56.762574+0000","last_peered":"2023-11-26T07:12:56.762574+0000","last_clean":"2023-11-26T07:12:56.762574+0000","last_became_active":"2023-11-26T07:12:55.541562+0000","last_became_peered":"2023-11-26T07:12:55.541562+0000","last_unstale":"2023-11-26T07:12:56.762574+0000","last_undegraded":"2023-11-26T07:12:56.762574+0000","last_fullsized":"2023-11-26T07:12:56.762574+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:46:27.624298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.536348+0000","last_change":"2023-11-26T07:12:54.535255+0000","last_active":"2023-11-26T07:12:55.536348+0000","last_peered":"2023-11-26T07:12:55.536348+0000","last_clean":"2023-11-26T07:12:55.536348+0000","last_became_active":"2023-11-26T07:12:54.534934+0000","last_became_peered":"2023-11-26T07:12:54.534934+0000","last_unstale":"2023-11-26T07:12:55.536348+0000","last_undegraded":"2023-11-26T07:12:55.536348+0000","last_fullsized":"2023-11-26T07:12:55.536348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:25:59.772453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762522+0000","last_change":"2023-11-26T07:12:55.538727+0000","last_active":"2023-11-26T07:12:56.762522+0000","last_peered":"2023-11-26T07:12:56.762522+0000","last_clean":"2023-11-26T07:12:56.762522+0000","last_became_active":"2023-11-26T07:12:55.537682+0000","last_became_peered":"2023-11-26T07:12:55.537682+0000","last_unstale":"2023-11-26T07:12:56.762522+0000","last_undegraded":"2023-11-26T07:12:56.762522+0000","last_fullsized":"2023-11-26T07:12:56.762522+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:17.435087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.542961+0000","last_change":"2023-11-26T07:12:54.531509+0000","last_active":"2023-11-26T07:12:55.542961+0000","last_peered":"2023-11-26T07:12:55.542961+0000","last_clean":"2023-11-26T07:12:55.542961+0000","last_became_active":"2023-11-26T07:12:54.531168+0000","last_became_peered":"2023-11-26T07:12:54.531168+0000","last_unstale":"2023-11-26T07:12:55.542961+0000","last_undegraded":"2023-11-26T07:12:55.542961+0000","last_fullsized":"2023-11-26T07:12:55.542961+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:10:11.064777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762502+0000","last_change":"2023-11-26T07:12:55.543640+0000","last_active":"2023-11-26T07:12:56.762502+0000","last_peered":"2023-11-26T07:12:56.762502+0000","last_clean":"2023-11-26T07:12:56.762502+0000","last_became_active":"2023-11-26T07:12:55.543407+0000","last_became_peered":"2023-11-26T07:12:55.543407+0000","last_unstale":"2023-11-26T07:12:56.762502+0000","last_undegraded":"2023-11-26T07:12:56.762502+0000","last_fullsized":"2023-11-26T07:12:56.762502+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:26:00.774490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.533428+0000","last_change":"2023-11-26T07:12:54.536077+0000","last_active":"2023-11-26T07:12:55.533428+0000","last_peered":"2023-11-26T07:12:55.533428+0000","last_clean":"2023-11-26T07:12:55.533428+0000","last_became_active":"2023-11-26T07:12:54.535793+0000","last_became_peered":"2023-11-26T07:12:54.535793+0000","last_unstale":"2023-11-26T07:12:55.533428+0000","last_undegraded":"2023-11-26T07:12:55.533428+0000","last_fullsized":"2023-11-26T07:12:55.533428+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:08:53.040981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762022+0000","last_change":"2023-11-26T07:12:55.540443+0000","last_active":"2023-11-26T07:12:56.762022+0000","last_peered":"2023-11-26T07:12:56.762022+0000","last_clean":"2023-11-26T07:12:56.762022+0000","last_became_active":"2023-11-26T07:12:55.540131+0000","last_became_peered":"2023-11-26T07:12:55.540131+0000","last_unstale":"2023-11-26T07:12:56.762022+0000","last_undegraded":"2023-11-26T07:12:56.762022+0000","last_fullsized":"2023-11-26T07:12:56.762022+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:31:46.715769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.541211+0000","last_change":"2023-11-26T07:12:54.537136+0000","last_active":"2023-11-26T07:12:55.541211+0000","last_peered":"2023-11-26T07:12:55.541211+0000","last_clean":"2023-11-26T07:12:55.541211+0000","last_became_active":"2023-11-26T07:12:54.536557+0000","last_became_peered":"2023-11-26T07:12:54.536557+0000","last_unstale":"2023-11-26T07:12:55.541211+0000","last_undegraded":"2023-11-26T07:12:55.541211+0000","last_fullsized":"2023-11-26T07:12:55.541211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:24:51.735731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.761911+0000","last_change":"2023-11-26T07:12:55.531425+0000","last_active":"2023-11-26T07:12:56.761911+0000","last_peered":"2023-11-26T07:12:56.761911+0000","last_clean":"2023-11-26T07:12:56.761911+0000","last_became_active":"2023-11-26T07:12:55.531174+0000","last_became_peered":"2023-11-26T07:12:55.531174+0000","last_unstale":"2023-11-26T07:12:56.761911+0000","last_undegraded":"2023-11-26T07:12:56.761911+0000","last_fullsized":"2023-11-26T07:12:56.761911+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:58:46.200072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.527273+0000","last_change":"2023-11-26T07:12:54.533932+0000","last_active":"2023-11-26T07:12:55.527273+0000","last_peered":"2023-11-26T07:12:55.527273+0000","last_clean":"2023-11-26T07:12:55.527273+0000","last_became_active":"2023-11-26T07:12:54.533651+0000","last_became_peered":"2023-11-26T07:12:54.533651+0000","last_unstale":"2023-11-26T07:12:55.527273+0000","last_undegraded":"2023-11-26T07:12:55.527273+0000","last_fullsized":"2023-11-26T07:12:55.527273+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:36:57.635605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.520761+0000","last_change":"2023-11-26T07:12:55.529722+0000","last_active":"2023-11-26T07:12:56.520761+0000","last_peered":"2023-11-26T07:12:56.520761+0000","last_clean":"2023-11-26T07:12:56.520761+0000","last_became_active":"2023-11-26T07:12:55.529438+0000","last_became_peered":"2023-11-26T07:12:55.529438+0000","last_unstale":"2023-11-26T07:12:56.520761+0000","last_undegraded":"2023-11-26T07:12:56.520761+0000","last_fullsized":"2023-11-26T07:12:56.520761+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:23:23.245984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.524438+0000","last_change":"2023-11-26T07:12:54.533722+0000","last_active":"2023-11-26T07:12:55.524438+0000","last_peered":"2023-11-26T07:12:55.524438+0000","last_clean":"2023-11-26T07:12:55.524438+0000","last_became_active":"2023-11-26T07:12:54.533207+0000","last_became_peered":"2023-11-26T07:12:54.533207+0000","last_unstale":"2023-11-26T07:12:55.524438+0000","last_undegraded":"2023-11-26T07:12:55.524438+0000","last_fullsized":"2023-11-26T07:12:55.524438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:56:54.811575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762563+0000","last_change":"2023-11-26T07:12:55.533945+0000","last_active":"2023-11-26T07:12:56.762563+0000","last_peered":"2023-11-26T07:12:56.762563+0000","last_clean":"2023-11-26T07:12:56.762563+0000","last_became_active":"2023-11-26T07:12:55.533677+0000","last_became_peered":"2023-11-26T07:12:55.533677+0000","last_unstale":"2023-11-26T07:12:56.762563+0000","last_undegraded":"2023-11-26T07:12:56.762563+0000","last_fullsized":"2023-11-26T07:12:56.762563+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:54:04.551904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.522173+0000","last_change":"2023-11-26T07:12:54.535338+0000","last_active":"2023-11-26T07:12:55.522173+0000","last_peered":"2023-11-26T07:12:55.522173+0000","last_clean":"2023-11-26T07:12:55.522173+0000","last_became_active":"2023-11-26T07:12:54.534988+0000","last_became_peered":"2023-11-26T07:12:54.534988+0000","last_unstale":"2023-11-26T07:12:55.522173+0000","last_undegraded":"2023-11-26T07:12:55.522173+0000","last_fullsized":"2023-11-26T07:12:55.522173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:46:26.622261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762630+0000","last_change":"2023-11-26T07:12:55.547412+0000","last_active":"2023-11-26T07:12:56.762630+0000","last_peered":"2023-11-26T07:12:56.762630+0000","last_clean":"2023-11-26T07:12:56.762630+0000","last_became_active":"2023-11-26T07:12:55.547122+0000","last_became_peered":"2023-11-26T07:12:55.547122+0000","last_unstale":"2023-11-26T07:12:56.762630+0000","last_undegraded":"2023-11-26T07:12:56.762630+0000","last_fullsized":"2023-11-26T07:12:56.762630+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:40:46.830777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762102+0000","last_change":"2023-11-26T07:12:54.531775+0000","last_active":"2023-11-26T07:12:56.762102+0000","last_peered":"2023-11-26T07:12:56.762102+0000","last_clean":"2023-11-26T07:12:56.762102+0000","last_became_active":"2023-11-26T07:12:54.531491+0000","last_became_peered":"2023-11-26T07:12:54.531491+0000","last_unstale":"2023-11-26T07:12:56.762102+0000","last_undegraded":"2023-11-26T07:12:56.762102+0000","last_fullsized":"2023-11-26T07:12:56.762102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:02:17.314756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762148+0000","last_change":"2023-11-26T07:12:55.533378+0000","last_active":"2023-11-26T07:12:56.762148+0000","last_peered":"2023-11-26T07:12:56.762148+0000","last_clean":"2023-11-26T07:12:56.762148+0000","last_became_active":"2023-11-26T07:12:55.533117+0000","last_became_peered":"2023-11-26T07:12:55.533117+0000","last_unstale":"2023-11-26T07:12:56.762148+0000","last_undegraded":"2023-11-26T07:12:56.762148+0000","last_fullsized":"2023-11-26T07:12:56.762148+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:20:46.043819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.523812+0000","last_change":"2023-11-26T07:12:54.534426+0000","last_active":"2023-11-26T07:12:55.523812+0000","last_peered":"2023-11-26T07:12:55.523812+0000","last_clean":"2023-11-26T07:12:55.523812+0000","last_became_active":"2023-11-26T07:12:54.534154+0000","last_became_peered":"2023-11-26T07:12:54.534154+0000","last_unstale":"2023-11-26T07:12:55.523812+0000","last_undegraded":"2023-11-26T07:12:55.523812+0000","last_fullsized":"2023-11-26T07:12:55.523812+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:49.710471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.523527+0000","last_change":"2023-11-26T07:12:55.532724+0000","last_active":"2023-11-26T07:12:56.523527+0000","last_peered":"2023-11-26T07:12:56.523527+0000","last_clean":"2023-11-26T07:12:56.523527+0000","last_became_active":"2023-11-26T07:12:55.532393+0000","last_became_peered":"2023-11-26T07:12:55.532393+0000","last_unstale":"2023-11-26T07:12:56.523527+0000","last_undegraded":"2023-11-26T07:12:56.523527+0000","last_fullsized":"2023-11-26T07:12:56.523527+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:45:55.024079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.520830+0000","last_change":"2023-11-26T07:12:54.529524+0000","last_active":"2023-11-26T07:12:55.520830+0000","last_peered":"2023-11-26T07:12:55.520830+0000","last_clean":"2023-11-26T07:12:55.520830+0000","last_became_active":"2023-11-26T07:12:54.529161+0000","last_became_peered":"2023-11-26T07:12:54.529161+0000","last_unstale":"2023-11-26T07:12:55.520830+0000","last_undegraded":"2023-11-26T07:12:55.520830+0000","last_fullsized":"2023-11-26T07:12:55.520830+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:04:57.018851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762282+0000","last_change":"2023-11-26T07:12:55.538834+0000","last_active":"2023-11-26T07:12:56.762282+0000","last_peered":"2023-11-26T07:12:56.762282+0000","last_clean":"2023-11-26T07:12:56.762282+0000","last_became_active":"2023-11-26T07:12:55.538553+0000","last_became_peered":"2023-11-26T07:12:55.538553+0000","last_unstale":"2023-11-26T07:12:56.762282+0000","last_undegraded":"2023-11-26T07:12:56.762282+0000","last_fullsized":"2023-11-26T07:12:56.762282+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:20.975692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.522260+0000","last_change":"2023-11-26T07:12:54.533531+0000","last_active":"2023-11-26T07:12:55.522260+0000","last_peered":"2023-11-26T07:12:55.522260+0000","last_clean":"2023-11-26T07:12:55.522260+0000","last_became_active":"2023-11-26T07:12:54.533142+0000","last_became_peered":"2023-11-26T07:12:54.533142+0000","last_unstale":"2023-11-26T07:12:55.522260+0000","last_undegraded":"2023-11-26T07:12:55.522260+0000","last_fullsized":"2023-11-26T07:12:55.522260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:10:11.064777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762366+0000","last_change":"2023-11-26T07:12:55.540953+0000","last_active":"2023-11-26T07:12:56.762366+0000","last_peered":"2023-11-26T07:12:56.762366+0000","last_clean":"2023-11-26T07:12:56.762366+0000","last_became_active":"2023-11-26T07:12:55.540716+0000","last_became_peered":"2023-11-26T07:12:55.540716+0000","last_unstale":"2023-11-26T07:12:56.762366+0000","last_undegraded":"2023-11-26T07:12:56.762366+0000","last_fullsized":"2023-11-26T07:12:56.762366+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:14:59.315519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.520327+0000","last_change":"2023-11-26T07:12:54.525145+0000","last_active":"2023-11-26T07:12:55.520327+0000","last_peered":"2023-11-26T07:12:55.520327+0000","last_clean":"2023-11-26T07:12:55.520327+0000","last_became_active":"2023-11-26T07:12:54.524871+0000","last_became_peered":"2023-11-26T07:12:54.524871+0000","last_unstale":"2023-11-26T07:12:55.520327+0000","last_undegraded":"2023-11-26T07:12:55.520327+0000","last_fullsized":"2023-11-26T07:12:55.520327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:58:45.198035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.522974+0000","last_change":"2023-11-26T07:12:55.534270+0000","last_active":"2023-11-26T07:12:56.522974+0000","last_peered":"2023-11-26T07:12:56.522974+0000","last_clean":"2023-11-26T07:12:56.522974+0000","last_became_active":"2023-11-26T07:12:55.533892+0000","last_became_peered":"2023-11-26T07:12:55.533892+0000","last_unstale":"2023-11-26T07:12:56.522974+0000","last_undegraded":"2023-11-26T07:12:56.522974+0000","last_fullsized":"2023-11-26T07:12:56.522974+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:22:20.487642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.521239+0000","last_change":"2023-11-26T07:12:54.533682+0000","last_active":"2023-11-26T07:12:55.521239+0000","last_peered":"2023-11-26T07:12:55.521239+0000","last_clean":"2023-11-26T07:12:55.521239+0000","last_became_active":"2023-11-26T07:12:54.533409+0000","last_became_peered":"2023-11-26T07:12:54.533409+0000","last_unstale":"2023-11-26T07:12:55.521239+0000","last_undegraded":"2023-11-26T07:12:55.521239+0000","last_fullsized":"2023-11-26T07:12:55.521239+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:05:11.673958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.542173+0000","last_change":"2023-11-26T07:12:55.535621+0000","last_active":"2023-11-26T07:12:56.542173+0000","last_peered":"2023-11-26T07:12:56.542173+0000","last_clean":"2023-11-26T07:12:56.542173+0000","last_became_active":"2023-11-26T07:12:55.535316+0000","last_became_peered":"2023-11-26T07:12:55.535316+0000","last_unstale":"2023-11-26T07:12:56.542173+0000","last_undegraded":"2023-11-26T07:12:56.542173+0000","last_fullsized":"2023-11-26T07:12:56.542173+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:02:55.986543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.526201+0000","last_change":"2023-11-26T07:12:54.537875+0000","last_active":"2023-11-26T07:12:55.526201+0000","last_peered":"2023-11-26T07:12:55.526201+0000","last_clean":"2023-11-26T07:12:55.526201+0000","last_became_active":"2023-11-26T07:12:54.537498+0000","last_became_peered":"2023-11-26T07:12:54.537498+0000","last_unstale":"2023-11-26T07:12:55.526201+0000","last_undegraded":"2023-11-26T07:12:55.526201+0000","last_fullsized":"2023-11-26T07:12:55.526201+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:21:59.641863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-26T07:12:56.762237+0000","last_change":"2023-11-26T07:12:55.545331+0000","last_active":"2023-11-26T07:12:56.762237+0000","last_peered":"2023-11-26T07:12:56.762237+0000","last_clean":"2023-11-26T07:12:56.762237+0000","last_became_active":"2023-11-26T07:12:55.545049+0000","last_became_peered":"2023-11-26T07:12:55.545049+0000","last_unstale":"2023-11-26T07:12:56.762237+0000","last_undegraded":"2023-11-26T07:12:56.762237+0000","last_fullsized":"2023-11-26T07:12:56.762237+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:54.508362+0000","last_clean_scrub_stamp":"2023-11-26T07:12:54.508362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:58:58.700504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.537231+0000","last_change":"2023-11-26T07:12:54.528129+0000","last_active":"2023-11-26T07:12:55.537231+0000","last_peered":"2023-11-26T07:12:55.537231+0000","last_clean":"2023-11-26T07:12:55.537231+0000","last_became_active":"2023-11-26T07:12:54.527835+0000","last_became_peered":"2023-11-26T07:12:54.527835+0000","last_unstale":"2023-11-26T07:12:55.537231+0000","last_undegraded":"2023-11-26T07:12:55.537231+0000","last_fullsized":"2023-11-26T07:12:55.537231+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:12:53.506325+0000","last_clean_scrub_stamp":"2023-11-26T07:12:53.506325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:54:03.549867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:12:55.522281+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:12:55.522281+0000","last_peered":"2023-11-26T07:12:55.522281+0000","last_clean":"2023-11-26T07:12:55.522281+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:12:55.522281+0000","last_undegraded":"2023-11-26T07:12:55.522281+0000","last_fullsized":"2023-11-26T07:12:55.522281+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343716,"statfs":{"total":96636764160,"available":96607965184,"internally_reserved":0,"allocated":1126400,"data_stored":804913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.41499999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69199999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.442},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55300000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.58499999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.49399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":536576,"data_stored":215127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.502}]},{"osd":1,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.49199999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.376},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57599999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.47999999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.55800000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.46300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.45000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":214525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.31}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.32200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.24399999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.504}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.52300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":1126400,"data_stored":804857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.313}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.36699999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39800000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.621}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58399999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":544768,"data_stored":216089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.35399999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.40500000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.371}]},{"osd":4,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56499999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.5},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.59299999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.57899999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.60499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":215093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.34799999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.9109999999999996,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.313}]},{"osd":2,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.247},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.32800000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.9500000000000002,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":4.8129999999999997,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.71799999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.883,"5min":4.883,"15min":4.883},"last":0.60699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.8499999999999996,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.67000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":216067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.29599999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.9820000000000002,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.33000000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.53500000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.504}]},{"osd":5,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.52000000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.627}]},{"osd":7,"last update":"Sun Nov 26 07:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":214525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.629}]},{"osd":1,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.51000000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.61299999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.441},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.53900000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59599999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:13:04.578 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:13:04.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:13:04.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.740+0000 7f14df28a700 1 Processor -- start 2023-11-26T07:13:04.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.741+0000 7f14df28a700 1 -- start start 2023-11-26T07:13:04.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14df28a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14d815b370 0x7f14d815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:04.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14df28a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d80ae5e0 con 0x7f14d80b3c60 2023-11-26T07:13:04.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14df28a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d815be30 con 0x7f14d81587f0 2023-11-26T07:13:04.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14df28a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d815c110 con 0x7f14d815b370 2023-11-26T07:13:04.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14dc825700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14d815b370 0x7f14d815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:04.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14dd827700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f14d80b3c60 0x7f14d80af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57286/0 (socket says 172.21.15.83:57286) 2023-11-26T07:13:04.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14dd827700 1 -- 172.21.15.83:0/2257771223 learned_addr learned my addr 172.21.15.83:0/2257771223 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:04.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14cffff700 1 -- 172.21.15.83:0/2257771223 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210906057 0 0) 0x7f14d80ae5e0 con 0x7f14d80b3c60 2023-11-26T07:13:04.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14cffff700 1 -- 172.21.15.83:0/2257771223 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14c40035b0 con 0x7f14d80b3c60 2023-11-26T07:13:04.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14dc825700 1 -- 172.21.15.83:0/2257771223 >> v1:172.21.15.203:6789/0 conn(0x7f14d81587f0 legacy=0x7f14d806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14dc825700 1 -- 172.21.15.83:0/2257771223 >> v1:172.21.15.83:6789/0 conn(0x7f14d80b3c60 legacy=0x7f14d80af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14dc825700 1 -- 172.21.15.83:0/2257771223 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d80ae860 con 0x7f14d815b370 2023-11-26T07:13:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14dc825700 1 --2- 172.21.15.83:0/2257771223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d815b370 0x7f14d815b800 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009870 tx=0x7f14d4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cf0619c9fa8bae server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14cffff700 1 -- 172.21.15.83:0/2257771223 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4011830 con 0x7f14d815b370 2023-11-26T07:13:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14cffff700 1 -- 172.21.15.83:0/2257771223 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d4011990 con 0x7f14d815b370 2023-11-26T07:13:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.742+0000 7f14cffff700 1 -- 172.21.15.83:0/2257771223 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4011d20 con 0x7f14d815b370 2023-11-26T07:13:04.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.743+0000 7f14df28a700 1 -- 172.21.15.83:0/2257771223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d815b370 msgr2=0x7f14d815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:04.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.743+0000 7f14df28a700 1 --2- 172.21.15.83:0/2257771223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d815b370 0x7f14d815b800 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009870 tx=0x7f14d4009b80 comp rx=0 tx=0).stop 2023-11-26T07:13:04.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.743+0000 7f14df28a700 1 -- 172.21.15.83:0/2257771223 shutdown_connections 2023-11-26T07:13:04.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.743+0000 7f14df28a700 1 --2- 172.21.15.83:0/2257771223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d815b370 0x7f14d815b800 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.743+0000 7f14df28a700 1 -- 172.21.15.83:0/2257771223 >> 172.21.15.83:0/2257771223 conn(0x7f14d80d0bf0 msgr2=0x7f14d80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:04.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.743+0000 7f14df28a700 1 -- 172.21.15.83:0/2257771223 shutdown_connections 2023-11-26T07:13:04.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.743+0000 7f14df28a700 1 -- 172.21.15.83:0/2257771223 wait complete. 2023-11-26T07:13:04.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.744+0000 7f14df28a700 1 Processor -- start 2023-11-26T07:13:04.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.744+0000 7f14df28a700 1 -- start start 2023-11-26T07:13:04.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14df28a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d80b3c60 0x7f14d8147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:04.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14df28a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14d81587f0 0x7f14d809e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14df28a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14d815b370 0x7f14d80af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14df28a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d809af70 con 0x7f14d80b3c60 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14df28a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d809b250 con 0x7f14d81587f0 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14df28a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d809b530 con 0x7f14d815b370 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14dd026700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d80b3c60 0x7f14d8147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14dd026700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d80b3c60 0x7f14d8147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37248/0 (socket says 172.21.15.83:37248) 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14dd026700 1 -- 172.21.15.83:0/1855334139 learned_addr learned my addr 172.21.15.83:0/1855334139 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14dc825700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14d81587f0 0x7f14d809e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14dd827700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14d815b370 0x7f14d80af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14dd026700 1 -- 172.21.15.83:0/1855334139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14d815b370 msgr2=0x7f14d80af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14dd026700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14d815b370 0x7f14d80af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14dd026700 1 -- 172.21.15.83:0/1855334139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14d81587f0 msgr2=0x7f14d809e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14dd026700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14d81587f0 0x7f14d809e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14dd026700 1 -- 172.21.15.83:0/1855334139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d40094a0 con 0x7f14d80b3c60 2023-11-26T07:13:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.745+0000 7f14dd827700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14d815b370 0x7f14d80af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.746+0000 7f14dd026700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d80b3c60 0x7f14d8147e90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f14c800c4e0 tx=0x7f14c800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.746+0000 7f14cdffb700 1 -- 172.21.15.83:0/1855334139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c80052d0 con 0x7f14d80b3c60 2023-11-26T07:13:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.746+0000 7f14cdffb700 1 -- 172.21.15.83:0/1855334139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c8005430 con 0x7f14d80b3c60 2023-11-26T07:13:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.746+0000 7f14df28a700 1 -- 172.21.15.83:0/1855334139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d809b810 con 0x7f14d80b3c60 2023-11-26T07:13:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.746+0000 7f14cdffb700 1 -- 172.21.15.83:0/1855334139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c80038c0 con 0x7f14d80b3c60 2023-11-26T07:13:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.746+0000 7f14df28a700 1 -- 172.21.15.83:0/1855334139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d809ea00 con 0x7f14d80b3c60 2023-11-26T07:13:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.747+0000 7f14df28a700 1 -- 172.21.15.83:0/1855334139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d809eb90 con 0x7f14d80b3c60 2023-11-26T07:13:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.747+0000 7f14cdffb700 1 -- 172.21.15.83:0/1855334139 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f14c8003a20 con 0x7f14d80b3c60 2023-11-26T07:13:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.748+0000 7f14cdffb700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f14bc06b080 0x7f14bc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.748+0000 7f14cdffb700 1 -- 172.21.15.83:0/1855334139 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14c8088d40 con 0x7f14d80b3c60 2023-11-26T07:13:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.748+0000 7f14dc825700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f14bc06b080 0x7f14bc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.748+0000 7f14dc825700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f14bc06b080 0x7f14bc06d540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f14d40097c0 tx=0x7f14d401c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:04.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:04.752+0000 7f14cdffb700 1 -- 172.21.15.83:0/1855334139 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14c8053c00 con 0x7f14d80b3c60 2023-11-26T07:13:05.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.026+0000 7f14df28a700 1 -- 172.21.15.83:0/1855334139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f14d8068fb0 con 0x7f14d80b3c60 2023-11-26T07:13:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.028+0000 7f14cdffb700 1 -- 172.21.15.83:0/1855334139 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f14c80170e0 con 0x7f14d80b3c60 2023-11-26T07:13:05.030 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2023-11-26T07:13:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ== 2023-11-26T07:13:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2023-11-26T07:13:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2023-11-26T07:13:05.030 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:13:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 -- 172.21.15.83:0/1855334139 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f14bc06b080 msgr2=0x7f14bc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f14bc06b080 0x7f14bc06d540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f14d40097c0 tx=0x7f14d401c040 comp rx=0 tx=0).stop 2023-11-26T07:13:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 -- 172.21.15.83:0/1855334139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d80b3c60 msgr2=0x7f14d8147e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d80b3c60 0x7f14d8147e90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f14c800c4e0 tx=0x7f14c800c8a0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 -- 172.21.15.83:0/1855334139 shutdown_connections 2023-11-26T07:13:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f14bc06b080 0x7f14bc06d540 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14d815b370 0x7f14d80af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d80b3c60 0x7f14d8147e90 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 --2- 172.21.15.83:0/1855334139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14d81587f0 0x7f14d809e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 -- 172.21.15.83:0/1855334139 >> 172.21.15.83:0/1855334139 conn(0x7f14d80d0bf0 msgr2=0x7f14d806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 -- 172.21.15.83:0/1855334139 shutdown_connections 2023-11-26T07:13:05.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.030+0000 7f14df28a700 1 -- 172.21.15.83:0/1855334139 wait complete. 2023-11-26T07:13:05.034 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2023-11-26T07:13:05.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:05.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.162+0000 7fe217140700 1 Processor -- start 2023-11-26T07:13:05.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.162+0000 7fe217140700 1 -- start start 2023-11-26T07:13:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe217140700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe21006c710 0x7fe210165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe217140700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2100af950 con 0x7fe2100adf00 2023-11-26T07:13:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe217140700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2100afc30 con 0x7fe2101446a0 2023-11-26T07:13:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe217140700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe210144a80 con 0x7fe21006c710 2023-11-26T07:13:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20ffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe2100adf00 0x7fe21006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57296/0 (socket says 172.21.15.83:57296) 2023-11-26T07:13:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20ffff700 1 -- 172.21.15.83:0/4153893512 learned_addr learned my addr 172.21.15.83:0/4153893512 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe2156dd700 1 --2- 172.21.15.83:0/4153893512 >> v2:172.21.15.203:3300/0 conn(0x7fe21006c710 0x7fe210165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340300503 0 0) 0x7fe2100af950 con 0x7fe2100adf00 2023-11-26T07:13:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2000035b0 con 0x7fe2100adf00 2023-11-26T07:13:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 974044669 0 0) 0x7fe2000035b0 con 0x7fe2100adf00 2023-11-26T07:13:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 >> v2:172.21.15.203:3300/0 conn(0x7fe21006c710 msgr2=0x7fe210165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 --2- 172.21.15.83:0/4153893512 >> v2:172.21.15.203:3300/0 conn(0x7fe21006c710 0x7fe210165b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 >> v1:172.21.15.203:6789/0 conn(0x7fe2101446a0 legacy=0x7fe2100b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100aff10 con 0x7fe2100adf00 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fe1fc001670 con 0x7fe2100adf00 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 >> v1:172.21.15.83:6789/0 conn(0x7fe2100adf00 legacy=0x7fe21006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 --2- 172.21.15.83:0/4153893512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe200003a00 0x7fe200003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 --2- 172.21.15.83:0/4153893512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe200004850 0x7fe200004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.163+0000 7fe20f7fe700 1 --2- 172.21.15.83:0/4153893512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe200005210 0x7fe20000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe210144a80 con 0x7fe200003a00 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20000e3c0 con 0x7fe200005210 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20000e580 con 0x7fe200004850 2023-11-26T07:13:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe214edc700 1 --2- 172.21.15.83:0/4153893512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe200003a00 0x7fe200003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe2156dd700 1 --2- 172.21.15.83:0/4153893512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe200005210 0x7fe20000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe20ffff700 1 --2- 172.21.15.83:0/4153893512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe200004850 0x7fe200004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe214edc700 1 -- 172.21.15.83:0/4153893512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe200004850 msgr2=0x7fe200004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe214edc700 1 --2- 172.21.15.83:0/4153893512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe200004850 0x7fe200004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe214edc700 1 -- 172.21.15.83:0/4153893512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe200005210 msgr2=0x7fe20000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe214edc700 1 --2- 172.21.15.83:0/4153893512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe200005210 0x7fe20000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe2156dd700 1 -- 172.21.15.83:0/4153893512 reap_dead start 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe214edc700 1 -- 172.21.15.83:0/4153893512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000035b0 con 0x7fe200003a00 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe214edc700 1 --2- 172.21.15.83:0/4153893512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe200003a00 0x7fe200003e20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe2040017c0 tx=0x7fe200005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5266489828fbd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040052d0 con 0x7fe200003a00 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe204005450 con 0x7fe200003a00 2023-11-26T07:13:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.164+0000 7fe20f7fe700 1 -- 172.21.15.83:0/4153893512 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040038c0 con 0x7fe200003a00 2023-11-26T07:13:05.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.165+0000 7fe217140700 1 -- 172.21.15.83:0/4153893512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe200003a00 msgr2=0x7fe200003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.165+0000 7fe217140700 1 --2- 172.21.15.83:0/4153893512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe200003a00 0x7fe200003e20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe2040017c0 tx=0x7fe200005880 comp rx=0 tx=0).stop 2023-11-26T07:13:05.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.165+0000 7fe217140700 1 -- 172.21.15.83:0/4153893512 shutdown_connections 2023-11-26T07:13:05.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.165+0000 7fe217140700 1 --2- 172.21.15.83:0/4153893512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe200003a00 0x7fe200003e20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.165+0000 7fe217140700 1 -- 172.21.15.83:0/4153893512 >> 172.21.15.83:0/4153893512 conn(0x7fe210071b90 msgr2=0x7fe210071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:05.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.165+0000 7fe217140700 1 -- 172.21.15.83:0/4153893512 shutdown_connections 2023-11-26T07:13:05.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.165+0000 7fe217140700 1 -- 172.21.15.83:0/4153893512 wait complete. 2023-11-26T07:13:05.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.166+0000 7fe217140700 1 Processor -- start 2023-11-26T07:13:05.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.166+0000 7fe217140700 1 -- start start 2023-11-26T07:13:05.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe217140700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe200003a00 0x7fe2100ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe217140700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2100c6910 0x7fe2100cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe217140700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2100ccf30 0x7fe2101f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe217140700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100cd3c0 con 0x7fe2100ccf30 2023-11-26T07:13:05.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe217140700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe210141610 con 0x7fe2100c6910 2023-11-26T07:13:05.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe217140700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe210142340 con 0x7fe200003a00 2023-11-26T07:13:05.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe2156dd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2100ccf30 0x7fe2101f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe2156dd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2100ccf30 0x7fe2101f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37272/0 (socket says 172.21.15.83:37272) 2023-11-26T07:13:05.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe2156dd700 1 -- 172.21.15.83:0/449641996 learned_addr learned my addr 172.21.15.83:0/449641996 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:05.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe20ffff700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2100c6910 0x7fe2100cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe2156dd700 1 -- 172.21.15.83:0/449641996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe200003a00 msgr2=0x7fe2100ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe214edc700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe200003a00 0x7fe2100ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe2156dd700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe200003a00 0x7fe2100ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe2156dd700 1 -- 172.21.15.83:0/449641996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2100c6910 msgr2=0x7fe2100cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe2156dd700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2100c6910 0x7fe2100cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe2156dd700 1 -- 172.21.15.83:0/449641996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100c6d90 con 0x7fe2100ccf30 2023-11-26T07:13:05.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe20ffff700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2100c6910 0x7fe2100cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:05.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.167+0000 7fe2156dd700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2100ccf30 0x7fe2101f3e20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe2100ae860 tx=0x7fe208011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.168+0000 7fe20d7fa700 1 -- 172.21.15.83:0/449641996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20800e040 con 0x7fe2100ccf30 2023-11-26T07:13:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.168+0000 7fe20d7fa700 1 -- 172.21.15.83:0/449641996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20800f040 con 0x7fe2100ccf30 2023-11-26T07:13:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.168+0000 7fe217140700 1 -- 172.21.15.83:0/449641996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2101f4a70 con 0x7fe2100ccf30 2023-11-26T07:13:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.168+0000 7fe20d7fa700 1 -- 172.21.15.83:0/449641996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20800fd50 con 0x7fe2100ccf30 2023-11-26T07:13:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.168+0000 7fe217140700 1 -- 172.21.15.83:0/449641996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe210141cf0 con 0x7fe2100ccf30 2023-11-26T07:13:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.169+0000 7fe217140700 1 -- 172.21.15.83:0/449641996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2100753f0 con 0x7fe2100ccf30 2023-11-26T07:13:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.169+0000 7fe20d7fa700 1 -- 172.21.15.83:0/449641996 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe208015430 con 0x7fe2100ccf30 2023-11-26T07:13:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.169+0000 7fe20d7fa700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe1f406b080 0x7fe1f406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.170+0000 7fe214edc700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe1f406b080 0x7fe1f406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.170+0000 7fe20d7fa700 1 -- 172.21.15.83:0/449641996 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe208089f90 con 0x7fe2100ccf30 2023-11-26T07:13:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.170+0000 7fe214edc700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe1f406b080 0x7fe1f406d540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe21006fc50 tx=0x7fe204009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:05.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.174+0000 7fe20d7fa700 1 -- 172.21.15.83:0/449641996 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe208054d40 con 0x7fe2100ccf30 2023-11-26T07:13:05.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.407+0000 7fe217140700 1 -- 172.21.15.83:0/449641996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2101424b0 con 0x7fe2100ccf30 2023-11-26T07:13:05.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.407+0000 7fe20d7fa700 1 -- 172.21.15.83:0/449641996 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fe208058360 con 0x7fe2100ccf30 2023-11-26T07:13:05.410 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:05.410 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:13:05.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 -- 172.21.15.83:0/449641996 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe1f406b080 msgr2=0x7fe1f406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe1f406b080 0x7fe1f406d540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe21006fc50 tx=0x7fe204009030 comp rx=0 tx=0).stop 2023-11-26T07:13:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 -- 172.21.15.83:0/449641996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2100ccf30 msgr2=0x7fe2101f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2100ccf30 0x7fe2101f3e20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe2100ae860 tx=0x7fe208011f80 comp rx=0 tx=0).stop 2023-11-26T07:13:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 -- 172.21.15.83:0/449641996 shutdown_connections 2023-11-26T07:13:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2100ccf30 0x7fe2101f3e20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe1f406b080 0x7fe1f406d540 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe200003a00 0x7fe2100ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 --2- 172.21.15.83:0/449641996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2100c6910 0x7fe2100cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 -- 172.21.15.83:0/449641996 >> 172.21.15.83:0/449641996 conn(0x7fe210071b90 msgr2=0x7fe2100b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 -- 172.21.15.83:0/449641996 shutdown_connections 2023-11-26T07:13:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.410+0000 7fe217140700 1 -- 172.21.15.83:0/449641996 wait complete. 2023-11-26T07:13:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2023-11-26T07:13:05.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:12:54.920948+0000', 'modified': '2023-11-26T07:12:57.942955+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5544}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5544': {'gid': 5544, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/2643742427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 2643742427}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 2643742427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:13:05.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:13:05.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-26T07:13:05.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.947+0000 7f2bd6b69700 1 Processor -- start 2023-11-26T07:13:05.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.947+0000 7f2bd6b69700 1 -- start start 2023-11-26T07:13:05.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.947+0000 7f2bd6b69700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bd00d2240 0x7f2bd00b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.947+0000 7f2bd6b69700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd00b1d40 con 0x7f2bd015ddd0 2023-11-26T07:13:05.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.947+0000 7f2bd6b69700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd00b28c0 con 0x7f2bd015e1b0 2023-11-26T07:13:05.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.947+0000 7f2bd6b69700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00b1f90 con 0x7f2bd00d2240 2023-11-26T07:13:05.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bd4905700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2bd015ddd0 0x7f2bd00d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57316/0 (socket says 172.21.15.83:57316) 2023-11-26T07:13:05.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bd4905700 1 -- 172.21.15.83:0/4281470882 learned_addr learned my addr 172.21.15.83:0/4281470882 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:05.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084259412 0 0) 0x7f2bd00b1d40 con 0x7f2bd015ddd0 2023-11-26T07:13:05.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bc00035b0 con 0x7f2bd015ddd0 2023-11-26T07:13:05.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcffff700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bd00d2240 0x7f2bd00b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2969155669 0 0) 0x7f2bc00035b0 con 0x7f2bd015ddd0 2023-11-26T07:13:05.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bd00d2240 msgr2=0x7f2bd00b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bd00d2240 0x7f2bd00b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 >> v1:172.21.15.203:6789/0 conn(0x7f2bd015e1b0 legacy=0x7f2bd006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd00b2270 con 0x7f2bd015ddd0 2023-11-26T07:13:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f2bc4001670 con 0x7f2bd015ddd0 2023-11-26T07:13:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 >> v1:172.21.15.83:6789/0 conn(0x7f2bd015ddd0 legacy=0x7f2bd00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bc0003a30 0x7f2bc0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bd00d2240 0x7f2bc0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bc0004d90 0x7f2bc0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00b1f90 con 0x7f2bc0003a30 2023-11-26T07:13:05.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc000bbd0 con 0x7f2bc0004d90 2023-11-26T07:13:05.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc000bd90 con 0x7f2bd00d2240 2023-11-26T07:13:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bd4905700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bc0003a30 0x7f2bc0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bd5106700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bc0004d90 0x7f2bc0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.948+0000 7f2bcffff700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bd00d2240 0x7f2bc0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd4905700 1 -- 172.21.15.83:0/4281470882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bd00d2240 msgr2=0x7f2bc0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd4905700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bd00d2240 0x7f2bc0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd4905700 1 -- 172.21.15.83:0/4281470882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bc0004d90 msgr2=0x7f2bc0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd4905700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bc0004d90 0x7f2bc0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd4905700 1 -- 172.21.15.83:0/4281470882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc00035b0 con 0x7f2bc0003a30 2023-11-26T07:13:05.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd4905700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bc0003a30 0x7f2bc0003e50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400c240 tx=0x7f2bc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76738704db7e5831 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:05.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4007c10 con 0x7f2bc0003a30 2023-11-26T07:13:05.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc400cde0 con 0x7f2bc0003a30 2023-11-26T07:13:05.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bcf7fe700 1 -- 172.21.15.83:0/4281470882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4005ba0 con 0x7f2bc0003a30 2023-11-26T07:13:05.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd6b69700 1 -- 172.21.15.83:0/4281470882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bc0003a30 msgr2=0x7f2bc0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd6b69700 1 --2- 172.21.15.83:0/4281470882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bc0003a30 0x7f2bc0003e50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400c240 tx=0x7f2bc0005880 comp rx=0 tx=0).stop 2023-11-26T07:13:05.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd5106700 1 -- 172.21.15.83:0/4281470882 reap_dead start 2023-11-26T07:13:05.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd6b69700 1 -- 172.21.15.83:0/4281470882 shutdown_connections 2023-11-26T07:13:05.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd6b69700 1 -- 172.21.15.83:0/4281470882 >> 172.21.15.83:0/4281470882 conn(0x7f2bd00731a0 msgr2=0x7f2bd009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:05.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd6b69700 1 -- 172.21.15.83:0/4281470882 shutdown_connections 2023-11-26T07:13:05.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.949+0000 7f2bd6b69700 1 -- 172.21.15.83:0/4281470882 wait complete. 2023-11-26T07:13:05.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.951+0000 7f2bd6b69700 1 Processor -- start 2023-11-26T07:13:05.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.951+0000 7f2bd6b69700 1 -- start start 2023-11-26T07:13:05.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd6b69700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bc0003a30 0x7f2bd014f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd6b69700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bd006c040 0x7f2bd00b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd6b69700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd00b5720 0x7f2bd00b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd6b69700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd00b6f90 con 0x7f2bd00b5720 2023-11-26T07:13:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd6b69700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd01f47e0 con 0x7f2bd006c040 2023-11-26T07:13:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd6b69700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd01f4db0 con 0x7f2bc0003a30 2023-11-26T07:13:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd5106700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd00b5720 0x7f2bd00b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd5106700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd00b5720 0x7f2bd00b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37292/0 (socket says 172.21.15.83:37292) 2023-11-26T07:13:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd5106700 1 -- 172.21.15.83:0/2137706428 learned_addr learned my addr 172.21.15.83:0/2137706428 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd4905700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bc0003a30 0x7f2bd014f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bcffff700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bd006c040 0x7f2bd00b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd5106700 1 -- 172.21.15.83:0/2137706428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bc0003a30 msgr2=0x7f2bd014f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd5106700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bc0003a30 0x7f2bd014f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd5106700 1 -- 172.21.15.83:0/2137706428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bd006c040 msgr2=0x7f2bd00b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd5106700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bd006c040 0x7f2bd00b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd5106700 1 -- 172.21.15.83:0/2137706428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd00b2ed0 con 0x7f2bd00b5720 2023-11-26T07:13:05.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd5106700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd00b5720 0x7f2bd00b5ba0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2bd015e590 tx=0x7f2bc800d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:05.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bcd7fa700 1 -- 172.21.15.83:0/2137706428 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8003c10 con 0x7f2bd00b5720 2023-11-26T07:13:05.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bcd7fa700 1 -- 172.21.15.83:0/2137706428 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc8003d90 con 0x7f2bd00b5720 2023-11-26T07:13:05.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd6b69700 1 -- 172.21.15.83:0/2137706428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd01f5090 con 0x7f2bd00b5720 2023-11-26T07:13:05.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bcd7fa700 1 -- 172.21.15.83:0/2137706428 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800d950 con 0x7f2bd00b5720 2023-11-26T07:13:05.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.952+0000 7f2bd6b69700 1 -- 172.21.15.83:0/2137706428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd01f4ac0 con 0x7f2bd00b5720 2023-11-26T07:13:05.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.954+0000 7f2bcd7fa700 1 -- 172.21.15.83:0/2137706428 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f2bc800dab0 con 0x7f2bd00b5720 2023-11-26T07:13:05.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.954+0000 7f2bcd7fa700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2bb406b080 0x7f2bb406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.955+0000 7f2bcd7fa700 1 -- 172.21.15.83:0/2137706428 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bc8089970 con 0x7f2bd00b5720 2023-11-26T07:13:05.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.955+0000 7f2bd4905700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2bb406b080 0x7f2bb406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.955+0000 7f2bd4905700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2bb406b080 0x7f2bb406d540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4001930 tx=0x7f2bc4007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:05.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.956+0000 7f2be2d62b80 1 Processor -- start 2023-11-26T07:13:05.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.956+0000 7f2be2d62b80 1 -- start start 2023-11-26T07:13:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d8f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d8fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d72e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b583cbc770 con 0x55b583d49080 2023-11-26T07:13:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b583ca3750 con 0x55b583d46490 2023-11-26T07:13:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b583ca1070 con 0x55b583d5eed0 2023-11-26T07:13:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bd4905700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d8f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bd4905700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d8f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37296/0 (socket says 172.21.15.83:37296) 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bd4905700 1 -- 172.21.15.83:0/601606685 learned_addr learned my addr 172.21.15.83:0/601606685 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bcffff700 1 --2- 172.21.15.83:0/601606685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d8fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bd4905700 1 -- 172.21.15.83:0/601606685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 msgr2=0x55b583d8fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bd5106700 1 --2- 172.21.15.83:0/601606685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d72e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bd4905700 1 --2- 172.21.15.83:0/601606685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d8fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bd4905700 1 -- 172.21.15.83:0/601606685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 msgr2=0x55b583d72e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bd4905700 1 --2- 172.21.15.83:0/601606685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d72e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bd4905700 1 -- 172.21.15.83:0/601606685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b583b195e0 con 0x55b583d49080 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bcffff700 1 --2- 172.21.15.83:0/601606685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d8fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.957+0000 7f2bd5106700 1 --2- 172.21.15.83:0/601606685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d72e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2bd4905700 1 --2- 172.21.15.83:0/601606685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d8f840 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4015040 tx=0x7f2bc400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a456c33166f934ec server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2bba7fc700 1 -- 172.21.15.83:0/601606685 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4004af0 con 0x55b583d49080 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2bba7fc700 1 -- 172.21.15.83:0/601606685 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc4023070 con 0x55b583d49080 2023-11-26T07:13:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2bba7fc700 1 -- 172.21.15.83:0/601606685 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4004e70 con 0x55b583d49080 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2be2d62b80 1 -- 172.21.15.83:0/601606685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 msgr2=0x55b583d8f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/601606685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d8f840 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4015040 tx=0x7f2bc400a000 comp rx=0 tx=0).stop 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2be2d62b80 1 -- 172.21.15.83:0/601606685 shutdown_connections 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/601606685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d72e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/601606685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d8f840 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/601606685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d8fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2be2d62b80 1 -- 172.21.15.83:0/601606685 >> 172.21.15.83:0/601606685 conn(0x55b583c77310 msgr2=0x55b583d8de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2be2d62b80 1 -- 172.21.15.83:0/601606685 shutdown_connections 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.958+0000 7f2be2d62b80 1 -- 172.21.15.83:0/601606685 wait complete. 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.959+0000 7f2be2d62b80 1 -- start start 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.959+0000 7f2be2d62b80 1 Processor -- start 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d46490 0x55b583d85a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d49080 0x55b583d85f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d76030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b583c9c960 con 0x55b583d46490 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b583c56510 con 0x55b583d49080 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b583c561b0 con 0x55b583d5eed0 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2be2d62b80 10 client.0 _ll_register_callbacks cb 0x55b583b81a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2bd4905700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d46490 0x55b583d85a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2bd4905700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d46490 0x55b583d85a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37302/0 (socket says 172.21.15.83:37302) 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2bd4905700 1 -- 172.21.15.83:0/2788519430 learned_addr learned my addr 172.21.15.83:0/2788519430 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2bd5106700 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d76030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2bcffff700 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d49080 0x55b583d85f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2bd4905700 1 -- 172.21.15.83:0/2788519430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 msgr2=0x55b583d76030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2bd4905700 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d76030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2bd4905700 1 -- 172.21.15.83:0/2788519430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d49080 msgr2=0x55b583d85f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2bd4905700 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d49080 0x55b583d85f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2bd4905700 1 -- 172.21.15.83:0/2788519430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc00035b0 con 0x55b583d46490 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.960+0000 7f2bcffff700 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d49080 0x55b583d85f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2bd4905700 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d46490 0x55b583d85a10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4004d10 tx=0x7f2bc4009050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d6b11198af10b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2baf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2788519430 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4039070 con 0x55b583d46490 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2788519430 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bc40075a0 con 0x55b583d46490 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2788519430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b583d3ba40 con 0x55b583d46490 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2788519430 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bc4007760 con 0x55b583d46490 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2be2d62b80 10 client.5721 fetch_fsmap learned FSMap version 27 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2788519430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55b583ac5d90 con 0x55b583d46490 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2788519430 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b583d3ba40 con 0x55b583d46490 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2788519430 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2bc4033470 con 0x55b583d46490 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2be2d62b80 10 client.5721 fetch_fsmap finished waiting for FSMap version 27 2023-11-26T07:13:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2be2d62b80 10 client.5721 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2be2d62b80 20 client.5721 populate_metadata read hostname 'smithi083' 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.83:6834/2643742427,v1:172.21.15.83:6837/2643742427] conn(0x55b583d76760 0x55b583da09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2be2d62b80 4 client.5721 mds_command: new command op to 5544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2788519430 --> [v2:172.21.15.83:6834/2643742427,v1:172.21.15.83:6837/2643742427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b583d17840 con 0x55b583d76760 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.961+0000 7f2bcffff700 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.83:6834/2643742427,v1:172.21.15.83:6837/2643742427] conn(0x55b583d76760 0x55b583da09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2baf7fe700 10 client.5721 ms_handle_connect on v2:172.21.15.83:6834/2643742427 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2bcffff700 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.83:6834/2643742427,v1:172.21.15.83:6837/2643742427] conn(0x55b583d76760 0x55b583da09f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b33bb5e95d6867d server_cookie=70b72c0e507fe38e in_seq=0 out_seq=0 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2788519430 <== mds.0 v2:172.21.15.83:6834/2643742427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b583d17840 con 0x55b583d76760 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2baf7fe700 10 client.5721 handle_command_reply: tid=0 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2be2d62b80 1 client.5721 shutdown 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2788519430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d46490 msgr2=0x55b583d85a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d46490 0x55b583d85a10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4004d10 tx=0x7f2bc4009050 comp rx=0 tx=0).stop 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2788519430 shutdown_connections 2023-11-26T07:13:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.83:6834/2643742427,v1:172.21.15.83:6837/2643742427] conn(0x55b583d76760 0x55b583da09f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d76030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2baf7fe700 0 client.5721 ms_handle_reset on v2:172.21.15.83:6834/2643742427 2023-11-26T07:13:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d46490 0x55b583d85a10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2788519430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d49080 0x55b583d85f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2788519430 >> 172.21.15.83:0/2788519430 conn(0x55b583c77310 msgr2=0x55b583d73800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2788519430 shutdown_connections 2023-11-26T07:13:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.962+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2788519430 wait complete. 2023-11-26T07:13:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.963+0000 7f2be2d62b80 20 client.5721 trim_cache size 0 max 16384 2023-11-26T07:13:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.980+0000 7f2be2d62b80 1 Processor -- start 2023-11-26T07:13:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.980+0000 7f2be2d62b80 1 -- start start 2023-11-26T07:13:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d458e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d49080 0x55b583d8ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d5eed0 0x55b583d906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b583b1b8d0 con 0x55b583d5eed0 2023-11-26T07:13:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b583d3f820 con 0x55b583d46490 2023-11-26T07:13:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b583d86550 con 0x55b583d49080 2023-11-26T07:13:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2bd5106700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d5eed0 0x55b583d906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2bd5106700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d5eed0 0x55b583d906a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37314/0 (socket says 172.21.15.83:37314) 2023-11-26T07:13:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2bd5106700 1 -- 172.21.15.83:0/39042492 learned_addr learned my addr 172.21.15.83:0/39042492 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2bd5106700 1 -- 172.21.15.83:0/39042492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d49080 msgr2=0x55b583d8ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2bd5106700 1 --2- 172.21.15.83:0/39042492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d49080 0x55b583d8ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2bd5106700 1 -- 172.21.15.83:0/39042492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 msgr2=0x55b583d458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2bd5106700 1 --2- 172.21.15.83:0/39042492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2bd5106700 1 -- 172.21.15.83:0/39042492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b583aaec10 con 0x55b583d5eed0 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2bd5106700 1 --2- 172.21.15.83:0/39042492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d5eed0 0x55b583d906a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8058480 tx=0x7f2bc8058100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1fb2a8e396e1d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2bbaffd700 1 -- 172.21.15.83:0/39042492 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc805c640 con 0x55b583d5eed0 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.981+0000 7f2bbaffd700 1 -- 172.21.15.83:0/39042492 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc80607f0 con 0x55b583d5eed0 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.982+0000 7f2bbaffd700 1 -- 172.21.15.83:0/39042492 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc801a310 con 0x55b583d5eed0 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.982+0000 7f2be2d62b80 1 -- 172.21.15.83:0/39042492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d5eed0 msgr2=0x55b583d906a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.982+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/39042492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d5eed0 0x55b583d906a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8058480 tx=0x7f2bc8058100 comp rx=0 tx=0).stop 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.982+0000 7f2be2d62b80 1 -- 172.21.15.83:0/39042492 shutdown_connections 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.982+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/39042492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d5eed0 0x55b583d906a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.982+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/39042492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.982+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/39042492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d49080 0x55b583d8ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.982+0000 7f2be2d62b80 1 -- 172.21.15.83:0/39042492 >> 172.21.15.83:0/39042492 conn(0x55b583c77310 msgr2=0x55b583db00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.982+0000 7f2be2d62b80 1 -- 172.21.15.83:0/39042492 shutdown_connections 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.982+0000 7f2be2d62b80 1 -- 172.21.15.83:0/39042492 wait complete. 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.983+0000 7f2be2d62b80 1 -- start start 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.983+0000 7f2be2d62b80 1 Processor -- start 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d83800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d83d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2be2d62b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d73970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b583da2080 con 0x55b583d49080 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b583d85b40 con 0x55b583d46490 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2be2d62b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b583d85e20 con 0x55b583d5eed0 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2be2d62b80 10 client.0 _ll_register_callbacks cb 0x55b583b81a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:13:05.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2bcffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d83d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2bcffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d83d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37318/0 (socket says 172.21.15.83:37318) 2023-11-26T07:13:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2bcffff700 1 -- 172.21.15.83:0/2460555402 learned_addr learned my addr 172.21.15.83:0/2460555402 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2bd4905700 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d83800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2bd5106700 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d73970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2bcffff700 1 -- 172.21.15.83:0/2460555402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 msgr2=0x55b583d73970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2bcffff700 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d73970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2bcffff700 1 -- 172.21.15.83:0/2460555402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 msgr2=0x55b583d83800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2bcffff700 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d83800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2bcffff700 1 -- 172.21.15.83:0/2460555402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd01f4ac0 con 0x55b583d49080 2023-11-26T07:13:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2bcffff700 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d83d40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55b583d92490 tx=0x7f2bbc01fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f309f98a5ba3158 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2baf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:13:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2460555402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc01d040 con 0x55b583d49080 2023-11-26T07:13:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2460555402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bbc024030 con 0x55b583d49080 2023-11-26T07:13:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2460555402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bc4023070 con 0x55b583d49080 2023-11-26T07:13:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2460555402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bbc005e00 con 0x55b583d49080 2023-11-26T07:13:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2be2d62b80 10 client.5727 fetch_fsmap learned FSMap version 27 2023-11-26T07:13:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.984+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2460555402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55b583aec200 con 0x55b583d49080 2023-11-26T07:13:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2460555402 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc4023070 con 0x55b583d49080 2023-11-26T07:13:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2460555402 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2bbc004240 con 0x55b583d49080 2023-11-26T07:13:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2be2d62b80 10 client.5727 fetch_fsmap finished waiting for FSMap version 27 2023-11-26T07:13:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2be2d62b80 10 client.5727 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:13:05.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2be2d62b80 20 client.5727 populate_metadata read hostname 'smithi083' 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.83:6834/2643742427,v1:172.21.15.83:6837/2643742427] conn(0x55b583d73eb0 0x55b583d8bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2be2d62b80 4 client.5727 mds_command: new command op to 5544 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2460555402 --> [v2:172.21.15.83:6834/2643742427,v1:172.21.15.83:6837/2643742427] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b583c62f80 con 0x55b583d73eb0 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2bd4905700 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.83:6834/2643742427,v1:172.21.15.83:6837/2643742427] conn(0x55b583d73eb0 0x55b583d8bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2baf7fe700 10 client.5727 ms_handle_connect on v2:172.21.15.83:6834/2643742427 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2bd4905700 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.83:6834/2643742427,v1:172.21.15.83:6837/2643742427] conn(0x55b583d73eb0 0x55b583d8bce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a12562eeb8867b server_cookie=5231e589cae352cd in_seq=0 out_seq=0 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2baf7fe700 1 -- 172.21.15.83:0/2460555402 <== mds.0 v2:172.21.15.83:6834/2643742427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b583c62f80 con 0x55b583d73eb0 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2baf7fe700 10 client.5727 handle_command_reply: tid=0 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.985+0000 7f2be2d62b80 1 client.5727 shutdown 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2460555402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 msgr2=0x55b583d83d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d83d40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55b583d92490 tx=0x7f2bbc01fda0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2460555402 shutdown_connections 2023-11-26T07:13:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.83:6834/2643742427,v1:172.21.15.83:6837/2643742427] conn(0x55b583d73eb0 0x55b583d8bce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b583d5eed0 0x55b583d73970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2baf7fe700 0 client.5727 ms_handle_reset on v2:172.21.15.83:6834/2643742427 2023-11-26T07:13:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b583d46490 0x55b583d83800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2be2d62b80 1 --2- 172.21.15.83:0/2460555402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b583d49080 0x55b583d83d40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2460555402 >> 172.21.15.83:0/2460555402 conn(0x55b583c77310 msgr2=0x55b583da0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2460555402 shutdown_connections 2023-11-26T07:13:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2be2d62b80 1 -- 172.21.15.83:0/2460555402 wait complete. 2023-11-26T07:13:05.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2be2d62b80 20 client.5727 trim_cache size 0 max 16384 2023-11-26T07:13:05.998 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:13:05.998 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "0aae1049-cc8c-4011-a01f-378673e57a96", 2023-11-26T07:13:05.999 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-26T07:13:05.999 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 5544, 2023-11-26T07:13:05.999 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-26T07:13:05.999 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-26T07:13:05.999 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-26T07:13:05.999 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 10.471335488999999, 2023-11-26T07:13:06.000 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2023-11-26T07:13:06.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 45, 2023-11-26T07:13:06.000 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 45, 2023-11-26T07:13:06.000 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 15.746385289999999 2023-11-26T07:13:06.000 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 -- 172.21.15.83:0/2137706428 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2bb406b080 msgr2=0x7f2bb406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2bb406b080 0x7f2bb406d540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4001930 tx=0x7f2bc4007000 comp rx=0 tx=0).stop 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 -- 172.21.15.83:0/2137706428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd00b5720 msgr2=0x7f2bd00b5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd00b5720 0x7f2bd00b5ba0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2bd015e590 tx=0x7f2bc800d150 comp rx=0 tx=0).stop 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 -- 172.21.15.83:0/2137706428 shutdown_connections 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bd00b5720 0x7f2bd00b5ba0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2bb406b080 0x7f2bb406d540 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bc0003a30 0x7f2bd014f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 --2- 172.21.15.83:0/2137706428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bd006c040 0x7f2bd00b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 -- 172.21.15.83:0/2137706428 >> 172.21.15.83:0/2137706428 conn(0x7f2bd00731a0 msgr2=0x7f2bd00701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 -- 172.21.15.83:0/2137706428 shutdown_connections 2023-11-26T07:13:06.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:05.986+0000 7f2bd6b69700 1 -- 172.21.15.83:0/2137706428 wait complete. 2023-11-26T07:13:06.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:13:06.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.121+0000 7fbf30518700 1 Processor -- start 2023-11-26T07:13:06.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.121+0000 7fbf30518700 1 -- start start 2023-11-26T07:13:06.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.122+0000 7fbf30518700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf2806c710 0x7fbf280b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:06.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.122+0000 7fbf30518700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf280af4d0 0x7fbf280afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.122+0000 7fbf30518700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf280ae430 con 0x7fbf2815e080 2023-11-26T07:13:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.122+0000 7fbf30518700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf280af0b0 con 0x7fbf2806c710 2023-11-26T07:13:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.122+0000 7fbf30518700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf280b05a0 con 0x7fbf280af4d0 2023-11-26T07:13:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.122+0000 7fbf2dab3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf2806c710 0x7fbf280b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.122+0000 7fbf2dab3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf2806c710 0x7fbf280b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37324/0 (socket says 172.21.15.83:37324) 2023-11-26T07:13:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.122+0000 7fbf2dab3700 1 -- 172.21.15.83:0/3965492827 learned_addr learned my addr 172.21.15.83:0/3965492827 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.123+0000 7fbf2eab5700 1 --2- 172.21.15.83:0/3965492827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf280af4d0 0x7fbf280afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.123+0000 7fbf2dab3700 1 -- 172.21.15.83:0/3965492827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf280af4d0 msgr2=0x7fbf280afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.123+0000 7fbf2dab3700 1 --2- 172.21.15.83:0/3965492827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf280af4d0 0x7fbf280afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.123+0000 7fbf2dab3700 1 -- 172.21.15.83:0/3965492827 >> v1:172.21.15.203:6789/0 conn(0x7fbf2815e080 legacy=0x7fbf2806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.123+0000 7fbf2dab3700 1 -- 172.21.15.83:0/3965492827 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf280ae710 con 0x7fbf2806c710 2023-11-26T07:13:06.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.123+0000 7fbf2dab3700 1 --2- 172.21.15.83:0/3965492827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2806c710 0x7fbf280b5050 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbf240098f0 tx=0x7fbf24009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51a021c289cbda4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:06.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.123+0000 7fbf2d2b2700 1 -- 172.21.15.83:0/3965492827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24004230 con 0x7fbf2806c710 2023-11-26T07:13:06.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.123+0000 7fbf2d2b2700 1 -- 172.21.15.83:0/3965492827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf24004390 con 0x7fbf2806c710 2023-11-26T07:13:06.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.123+0000 7fbf2d2b2700 1 -- 172.21.15.83:0/3965492827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf240036a0 con 0x7fbf2806c710 2023-11-26T07:13:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.123+0000 7fbf30518700 1 -- 172.21.15.83:0/3965492827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2806c710 msgr2=0x7fbf280b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.123+0000 7fbf30518700 1 --2- 172.21.15.83:0/3965492827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2806c710 0x7fbf280b5050 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbf240098f0 tx=0x7fbf24009c00 comp rx=0 tx=0).stop 2023-11-26T07:13:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.124+0000 7fbf30518700 1 -- 172.21.15.83:0/3965492827 shutdown_connections 2023-11-26T07:13:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.124+0000 7fbf30518700 1 --2- 172.21.15.83:0/3965492827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf280af4d0 0x7fbf280afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.124+0000 7fbf30518700 1 --2- 172.21.15.83:0/3965492827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2806c710 0x7fbf280b5050 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.124+0000 7fbf30518700 1 -- 172.21.15.83:0/3965492827 >> 172.21.15.83:0/3965492827 conn(0x7fbf28071b90 msgr2=0x7fbf28071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.124+0000 7fbf30518700 1 -- 172.21.15.83:0/3965492827 shutdown_connections 2023-11-26T07:13:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.124+0000 7fbf30518700 1 -- 172.21.15.83:0/3965492827 wait complete. 2023-11-26T07:13:06.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.125+0000 7fbf30518700 1 Processor -- start 2023-11-26T07:13:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.125+0000 7fbf30518700 1 -- start start 2023-11-26T07:13:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.125+0000 7fbf30518700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf2806c710 0x7fbf28156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.125+0000 7fbf30518700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf280af4d0 0x7fbf28160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.125+0000 7fbf30518700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815e080 0x7fbf2815e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.125+0000 7fbf30518700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf28157180 con 0x7fbf2815e080 2023-11-26T07:13:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.125+0000 7fbf30518700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf28155ba0 con 0x7fbf2806c710 2023-11-26T07:13:06.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.125+0000 7fbf30518700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf28155e80 con 0x7fbf280af4d0 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf2eab5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815e080 0x7fbf2815e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf2eab5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815e080 0x7fbf2815e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37332/0 (socket says 172.21.15.83:37332) 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf2eab5700 1 -- 172.21.15.83:0/4071134251 learned_addr learned my addr 172.21.15.83:0/4071134251 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf2dab3700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf280af4d0 0x7fbf28160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf2e2b4700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf2806c710 0x7fbf28156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf2eab5700 1 -- 172.21.15.83:0/4071134251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf280af4d0 msgr2=0x7fbf28160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf2eab5700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf280af4d0 0x7fbf28160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf2eab5700 1 -- 172.21.15.83:0/4071134251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf2806c710 msgr2=0x7fbf28156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf2eab5700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf2806c710 0x7fbf28156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf2eab5700 1 -- 172.21.15.83:0/4071134251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf24009550 con 0x7fbf2815e080 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf2eab5700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815e080 0x7fbf2815e4a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbf20007e40 tx=0x7fbf2000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf1effd700 1 -- 172.21.15.83:0/4071134251 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2000ed30 con 0x7fbf2815e080 2023-11-26T07:13:06.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf1effd700 1 -- 172.21.15.83:0/4071134251 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf20014420 con 0x7fbf2815e080 2023-11-26T07:13:06.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf30518700 1 -- 172.21.15.83:0/4071134251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf28156160 con 0x7fbf2815e080 2023-11-26T07:13:06.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf1effd700 1 -- 172.21.15.83:0/4071134251 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf200147d0 con 0x7fbf2815e080 2023-11-26T07:13:06.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.126+0000 7fbf30518700 1 -- 172.21.15.83:0/4071134251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf28156720 con 0x7fbf2815e080 2023-11-26T07:13:06.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.127+0000 7fbf30518700 1 -- 172.21.15.83:0/4071134251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf280753f0 con 0x7fbf2815e080 2023-11-26T07:13:06.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.127+0000 7fbf1effd700 1 -- 172.21.15.83:0/4071134251 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fbf20014930 con 0x7fbf2815e080 2023-11-26T07:13:06.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.128+0000 7fbf1effd700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf0c06b080 0x7fbf0c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:06.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.128+0000 7fbf1effd700 1 -- 172.21.15.83:0/4071134251 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf20089f20 con 0x7fbf2815e080 2023-11-26T07:13:06.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.128+0000 7fbf2e2b4700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf0c06b080 0x7fbf0c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:06.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.128+0000 7fbf2e2b4700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf0c06b080 0x7fbf0c06d540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009960 tx=0x7fbf180090e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:06.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.132+0000 7fbf1effd700 1 -- 172.21.15.83:0/4071134251 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf20054de0 con 0x7fbf2815e080 2023-11-26T07:13:06.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.367+0000 7fbf30518700 1 -- 172.21.15.83:0/4071134251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbf281568b0 con 0x7fbf2815e080 2023-11-26T07:13:06.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.367+0000 7fbf1effd700 1 -- 172.21.15.83:0/4071134251 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbf20058400 con 0x7fbf2815e080 2023-11-26T07:13:06.370 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:13:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.369+0000 7fbf30518700 1 -- 172.21.15.83:0/4071134251 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf0c06b080 msgr2=0x7fbf0c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.369+0000 7fbf30518700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf0c06b080 0x7fbf0c06d540 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009960 tx=0x7fbf180090e0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.369+0000 7fbf30518700 1 -- 172.21.15.83:0/4071134251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815e080 msgr2=0x7fbf2815e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.369+0000 7fbf30518700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815e080 0x7fbf2815e4a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbf20007e40 tx=0x7fbf2000e3c0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.369+0000 7fbf30518700 1 -- 172.21.15.83:0/4071134251 shutdown_connections 2023-11-26T07:13:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.369+0000 7fbf30518700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf2815e080 0x7fbf2815e4a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.369+0000 7fbf30518700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf2806c710 0x7fbf28156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.369+0000 7fbf30518700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf0c06b080 0x7fbf0c06d540 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.369+0000 7fbf30518700 1 --2- 172.21.15.83:0/4071134251 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf280af4d0 0x7fbf28160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.369+0000 7fbf30518700 1 -- 172.21.15.83:0/4071134251 >> 172.21.15.83:0/4071134251 conn(0x7fbf28071b90 msgr2=0x7fbf2806bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.370+0000 7fbf30518700 1 -- 172.21.15.83:0/4071134251 shutdown_connections 2023-11-26T07:13:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:06.370+0000 7fbf30518700 1 -- 172.21.15.83:0/4071134251 wait complete. 2023-11-26T07:13:06.384 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:13:06.384 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:13:06.384 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-26T07:13:06.384 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:13:06.384 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:13:06.384 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:13:06.384 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:13:06.384 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:13:06.385 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:13:06.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:06.385 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-26T07:13:06.402 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:13:06.402 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:13:06.403 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:13:06.403 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:06.403 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-26T07:13:06.403 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:06.403 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:13:06.403 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:13:06.403 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:13:06.403 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:13:06.403 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:13:06.403 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:13:06.403 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:13:06.404 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:13:06.404 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:13:06.404 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42300sec preferred_lft 42300sec 2023-11-26T07:13:06.404 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::3a41:fab:adc5:5d6c/64 scope link noprefixroute 2023-11-26T07:13:06.404 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:06.404 INFO:teuthology.orchestra.run.smithi083.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:13:06.404 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff 2023-11-26T07:13:06.404 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:13:06.404 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:06.404 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::54b0:6eff:fe33:bb15/64 scope link 2023-11-26T07:13:06.405 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:06.405 INFO:teuthology.orchestra.run.smithi083.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:13:06.405 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether fe:46:b0:72:7b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:13:06.405 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::fc46:b0ff:fe72:7b3f/64 scope link 2023-11-26T07:13:06.405 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:06.405 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:13:06.405 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:13:06.405 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::7808:fdff:fe1b:770c/64 scope link 2023-11-26T07:13:06.405 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:06.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:06.406 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:13:06.468 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-26T07:13:06.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:06.469 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-26T07:13:06.528 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-26T07:13:06.528 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:13:06.528 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:13:06.528 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:13:06.528 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:13:06.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:06.529 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:13:06.529 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:13:06.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:13:06.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:13:06.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:13:06.529 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:13:06.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:06.632 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:13:06.648 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:13:06.649 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:13:06.649 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:13:06.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:06.649 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-26T07:13:06.715 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:13:06.716 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:13:06.772 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:13:06.772 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:13:06.772 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:13:06.828 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:13:06.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:06.829 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:13:06.914 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:13:06.914 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:13:06.943 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:13:06.943 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:13:06.943 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:13:06.943 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:13:06.943 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:13:06.944 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:13:06.944 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:13:06.944 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:13:07.019 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:13:07.019 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:13:07.019 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:13:07.019 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:13:07.019 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:13:07.019 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:13:07.019 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:13:07.019 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:13:07.019 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:13:07.020 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:13:07.020 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:13:07.021 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:13:07.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:07.021 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:13:07.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:07.048 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:13:07.130 INFO:teuthology.orchestra.run.smithi083.stdout:client5012 2023-11-26T07:13:07.130 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:13:07.130 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:13:07.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:13:07.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.346+0000 7f9843e63700 1 Processor -- start 2023-11-26T07:13:07.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.346+0000 7f9843e63700 1 -- start start 2023-11-26T07:13:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.347+0000 7f9843e63700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f983c163230 0x7f983c165620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.347+0000 7f9843e63700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983c15a790 con 0x7f983c0cf360 2023-11-26T07:13:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.347+0000 7f9843e63700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983c0b4c40 con 0x7f983c15a3b0 2023-11-26T07:13:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.347+0000 7f9843e63700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983c15eec0 con 0x7f983c163230 2023-11-26T07:13:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.347+0000 7f98413fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f983c0cf360 0x7f983c0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57382/0 (socket says 172.21.15.83:57382) 2023-11-26T07:13:07.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.347+0000 7f98413fe700 1 -- 172.21.15.83:0/3806523 learned_addr learned my addr 172.21.15.83:0/3806523 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515456919 0 0) 0x7f983c15a790 con 0x7f983c0cf360 2023-11-26T07:13:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98280046c0 con 0x7f983c0cf360 2023-11-26T07:13:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9842400700 1 --2- 172.21.15.83:0/3806523 >> v2:172.21.15.203:3300/0 conn(0x7f983c163230 0x7f983c165620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3098939223 0 0) 0x7f98280046c0 con 0x7f983c0cf360 2023-11-26T07:13:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 >> v2:172.21.15.203:3300/0 conn(0x7f983c163230 msgr2=0x7f983c165620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:07.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 --2- 172.21.15.83:0/3806523 >> v2:172.21.15.203:3300/0 conn(0x7f983c163230 0x7f983c165620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:07.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 >> v1:172.21.15.203:6789/0 conn(0x7f983c15a3b0 legacy=0x7f983c0b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:07.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c0b4140 con 0x7f983c0cf360 2023-11-26T07:13:07.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f9838001670 con 0x7f983c0cf360 2023-11-26T07:13:07.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 >> v1:172.21.15.83:6789/0 conn(0x7f983c0cf360 legacy=0x7f983c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:07.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 0x7f9828004f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:07.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9828005960 0x7f9828005de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:07.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9828006320 0x7f982800cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:07.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c15eec0 con 0x7f9828004b10 2023-11-26T07:13:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982800d1c0 con 0x7f9828005960 2023-11-26T07:13:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982800d380 con 0x7f9828006320 2023-11-26T07:13:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9841bff700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 0x7f9828004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f98413fe700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9828005960 0x7f9828005de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9842400700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9828006320 0x7f982800cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:07.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9841bff700 1 -- 172.21.15.83:0/3806523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9828006320 msgr2=0x7f982800cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9841bff700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9828006320 0x7f982800cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9841bff700 1 -- 172.21.15.83:0/3806523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9828005960 msgr2=0x7f9828005de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9841bff700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9828005960 0x7f9828005de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9841bff700 1 -- 172.21.15.83:0/3806523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98280046c0 con 0x7f9828004b10 2023-11-26T07:13:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9842400700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9828006320 0x7f982800cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:13:07.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.348+0000 7f9842400700 1 -- 172.21.15.83:0/3806523 reap_dead start 2023-11-26T07:13:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.349+0000 7f9841bff700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 0x7f9828004f30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f982c0017c0 tx=0x7f9828006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcdae6511b2516f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.349+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0052d0 con 0x7f9828004b10 2023-11-26T07:13:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.349+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c005430 con 0x7f9828004b10 2023-11-26T07:13:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.349+0000 7f9840bfd700 1 -- 172.21.15.83:0/3806523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0038c0 con 0x7f9828004b10 2023-11-26T07:13:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.349+0000 7f9843e63700 1 -- 172.21.15.83:0/3806523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 msgr2=0x7f9828004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.349+0000 7f9843e63700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 0x7f9828004f30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f982c0017c0 tx=0x7f9828006990 comp rx=0 tx=0).stop 2023-11-26T07:13:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.349+0000 7f9843e63700 1 -- 172.21.15.83:0/3806523 shutdown_connections 2023-11-26T07:13:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.349+0000 7f9843e63700 1 --2- 172.21.15.83:0/3806523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 0x7f9828004f30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.349+0000 7f9843e63700 1 -- 172.21.15.83:0/3806523 >> 172.21.15.83:0/3806523 conn(0x7f983c0b1740 msgr2=0x7f983c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.349+0000 7f9843e63700 1 -- 172.21.15.83:0/3806523 shutdown_connections 2023-11-26T07:13:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.349+0000 7f9843e63700 1 -- 172.21.15.83:0/3806523 wait complete. 2023-11-26T07:13:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.351+0000 7f9843e63700 1 Processor -- start 2023-11-26T07:13:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.351+0000 7f9843e63700 1 -- start start 2023-11-26T07:13:07.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9843e63700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 0x7f983c146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:07.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9843e63700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f983c15aad0 0x7f983c137340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:07.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9843e63700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f983c1454e0 0x7f983c145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:07.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9843e63700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0cca50 con 0x7f9828004b10 2023-11-26T07:13:07.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9843e63700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c145e80 con 0x7f983c15aad0 2023-11-26T07:13:07.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9843e63700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c149b50 con 0x7f983c1454e0 2023-11-26T07:13:07.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9841bff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 0x7f983c146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9841bff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 0x7f983c146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37352/0 (socket says 172.21.15.83:37352) 2023-11-26T07:13:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9841bff700 1 -- 172.21.15.83:0/891599801 learned_addr learned my addr 172.21.15.83:0/891599801 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f98413fe700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f983c15aad0 0x7f983c137340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9842400700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f983c1454e0 0x7f983c145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9841bff700 1 -- 172.21.15.83:0/891599801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f983c1454e0 msgr2=0x7f983c145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:07.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9841bff700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f983c1454e0 0x7f983c145940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9841bff700 1 -- 172.21.15.83:0/891599801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f983c15aad0 msgr2=0x7f983c137340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9841bff700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f983c15aad0 0x7f983c137340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9841bff700 1 -- 172.21.15.83:0/891599801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c149e30 con 0x7f9828004b10 2023-11-26T07:13:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f98413fe700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f983c15aad0 0x7f983c137340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:07.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.352+0000 7f9841bff700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 0x7f983c146f30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f982c005040 tx=0x7f982c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.353+0000 7f98327fc700 1 -- 172.21.15.83:0/891599801 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c001290 con 0x7f9828004b10 2023-11-26T07:13:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.353+0000 7f98327fc700 1 -- 172.21.15.83:0/891599801 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c01d480 con 0x7f9828004b10 2023-11-26T07:13:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.353+0000 7f9843e63700 1 -- 172.21.15.83:0/891599801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983c137880 con 0x7f9828004b10 2023-11-26T07:13:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.353+0000 7f98327fc700 1 -- 172.21.15.83:0/891599801 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c01d810 con 0x7f9828004b10 2023-11-26T07:13:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.353+0000 7f9843e63700 1 -- 172.21.15.83:0/891599801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983c1487f0 con 0x7f9828004b10 2023-11-26T07:13:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.354+0000 7f9843e63700 1 -- 172.21.15.83:0/891599801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983c0cccc0 con 0x7f9828004b10 2023-11-26T07:13:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.354+0000 7f98327fc700 1 -- 172.21.15.83:0/891599801 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f982c019040 con 0x7f9828004b10 2023-11-26T07:13:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.354+0000 7f98327fc700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f982006b080 0x7f982006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.355+0000 7f98327fc700 1 -- 172.21.15.83:0/891599801 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f982c08a520 con 0x7f9828004b10 2023-11-26T07:13:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.355+0000 7f98413fe700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f982006b080 0x7f982006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.355+0000 7f98413fe700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f982006b080 0x7f982006d540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f983c0b0a80 tx=0x7f9838009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.359+0000 7f98327fc700 1 -- 172.21.15.83:0/891599801 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f982c055460 con 0x7f9828004b10 2023-11-26T07:13:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.595+0000 7f9843e63700 1 -- 172.21.15.83:0/891599801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f983c14a070 con 0x7f9828004b10 2023-11-26T07:13:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.595+0000 7f98327fc700 1 -- 172.21.15.83:0/891599801 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f983c14a070 con 0x7f9828004b10 2023-11-26T07:13:07.598 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:13:07.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.597+0000 7f9843e63700 1 -- 172.21.15.83:0/891599801 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f982006b080 msgr2=0x7f982006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:07.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.597+0000 7f9843e63700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f982006b080 0x7f982006d540 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f983c0b0a80 tx=0x7f9838009040 comp rx=0 tx=0).stop 2023-11-26T07:13:07.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.597+0000 7f9843e63700 1 -- 172.21.15.83:0/891599801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 msgr2=0x7f983c146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:07.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.597+0000 7f9843e63700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 0x7f983c146f30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f982c005040 tx=0x7f982c001120 comp rx=0 tx=0).stop 2023-11-26T07:13:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.597+0000 7f9843e63700 1 -- 172.21.15.83:0/891599801 shutdown_connections 2023-11-26T07:13:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.597+0000 7f9843e63700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f982006b080 0x7f982006d540 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.597+0000 7f9843e63700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f983c1454e0 0x7f983c145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.597+0000 7f9843e63700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828004b10 0x7f983c146f30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.597+0000 7f9843e63700 1 --2- 172.21.15.83:0/891599801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f983c15aad0 0x7f983c137340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.597+0000 7f9843e63700 1 -- 172.21.15.83:0/891599801 >> 172.21.15.83:0/891599801 conn(0x7f983c0b1740 msgr2=0x7f983c0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.598+0000 7f9843e63700 1 -- 172.21.15.83:0/891599801 shutdown_connections 2023-11-26T07:13:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:07.598+0000 7f9843e63700 1 -- 172.21.15.83:0/891599801 wait complete. 2023-11-26T07:13:07.612 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:13:07.612 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:13:07.612 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2023-11-26T07:13:07.612 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:13:07.612 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:13:07.612 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:13:07.612 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:13:07.612 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:13:07.612 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:13:07.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:07.613 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2023-11-26T07:13:07.634 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:13:07.634 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:13:07.635 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42302sec preferred_lft 42302sec 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::64b0:e04:3aa8:a78c/64 scope link noprefixroute 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1092:a6ff:fe63:3da7/64 scope link 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:13:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::b88a:cff:fe60:2b6f/64 scope link 2023-11-26T07:13:07.637 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:07.637 INFO:teuthology.orchestra.run.smithi203.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:13:07.637 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether de:b3:89:ce:09:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:13:07.637 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::dcb3:89ff:fece:9cd/64 scope link 2023-11-26T07:13:07.637 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:13:07.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:07.637 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:13:07.706 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-11-26T07:13:07.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:07.708 DEBUG:teuthology.orchestra.run.smithi203:> route 2023-11-26T07:13:07.770 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2023-11-26T07:13:07.770 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:13:07.770 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:13:07.770 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:13:07.770 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:13:07.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:07.770 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:13:07.771 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:13:07.771 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:13:07.771 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:13:07.771 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:13:07.771 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:13:07.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:07.880 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:13:07.897 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:13:07.897 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:13:07.897 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:13:07.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:07.897 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2023-11-26T07:13:07.964 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:13:07.964 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:13:08.021 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:13:08.021 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:13:08.021 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:13:08.078 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:13:08.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:08.078 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:13:08.164 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:13:08.164 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:13:08.191 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:13:08.191 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:13:08.191 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:13:08.191 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:13:08.191 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:13:08.191 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:13:08.191 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:13:08.191 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:13:08.220 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:13:08.220 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:13:08.220 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:13:08.220 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:13:08.220 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:13:08.221 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:13:08.221 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:13:08.221 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:13:08.221 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:13:08.221 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:13:08.221 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:13:08.224 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:13:08.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:08.224 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:13:08.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:08.254 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:13:08.326 INFO:teuthology.orchestra.run.smithi203.stdout:client5015 2023-11-26T07:13:08.327 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:13:08.327 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:13:08.384 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:13:08.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.518+0000 7f68ba1c9700 1 Processor -- start 2023-11-26T07:13:08.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.518+0000 7f68ba1c9700 1 -- start start 2023-11-26T07:13:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68ba1c9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68b40cf040 0x7f68b406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68ba1c9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f68b40d21c0 0x7f68b40af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68ba1c9700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b40d01c0 con 0x7f68b40702e0 2023-11-26T07:13:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68ba1c9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b4070910 con 0x7f68b40cf040 2023-11-26T07:13:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68ba1c9700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b4070bf0 con 0x7f68b40d21c0 2023-11-26T07:13:08.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68b37fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68b40cf040 0x7f68b406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68b37fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68b40cf040 0x7f68b406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37362/0 (socket says 172.21.15.83:37362) 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68b37fe700 1 -- 172.21.15.83:0/3921445145 learned_addr learned my addr 172.21.15.83:0/3921445145 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68b3fff700 1 --2- 172.21.15.83:0/3921445145 >> v2:172.21.15.203:3300/0 conn(0x7f68b40d21c0 0x7f68b40af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68b37fe700 1 -- 172.21.15.83:0/3921445145 >> v2:172.21.15.203:3300/0 conn(0x7f68b40d21c0 msgr2=0x7f68b40af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68b37fe700 1 --2- 172.21.15.83:0/3921445145 >> v2:172.21.15.203:3300/0 conn(0x7f68b40d21c0 0x7f68b40af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68b37fe700 1 -- 172.21.15.83:0/3921445145 >> v1:172.21.15.203:6789/0 conn(0x7f68b40702e0 legacy=0x7f68b40d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68b37fe700 1 -- 172.21.15.83:0/3921445145 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b40aebb0 con 0x7f68b40cf040 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.519+0000 7f68b37fe700 1 --2- 172.21.15.83:0/3921445145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b40cf040 0x7f68b406fda0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f68b40cc1e0 tx=0x7f68a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2cdd044e13f0a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.520+0000 7f68b27fc700 1 -- 172.21.15.83:0/3921445145 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a800b870 con 0x7f68b40cf040 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.520+0000 7f68b27fc700 1 -- 172.21.15.83:0/3921445145 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a800b9d0 con 0x7f68b40cf040 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.520+0000 7f68b27fc700 1 -- 172.21.15.83:0/3921445145 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a800bd60 con 0x7f68b40cf040 2023-11-26T07:13:08.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.520+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3921445145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b40cf040 msgr2=0x7f68b406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.520+0000 7f68ba1c9700 1 --2- 172.21.15.83:0/3921445145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b40cf040 0x7f68b406fda0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f68b40cc1e0 tx=0x7f68a8009bd0 comp rx=0 tx=0).stop 2023-11-26T07:13:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.520+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3921445145 shutdown_connections 2023-11-26T07:13:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.520+0000 7f68ba1c9700 1 --2- 172.21.15.83:0/3921445145 >> v2:172.21.15.203:3300/0 conn(0x7f68b40d21c0 0x7f68b40af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.520+0000 7f68ba1c9700 1 --2- 172.21.15.83:0/3921445145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b40cf040 0x7f68b406fda0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.520+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3921445145 >> 172.21.15.83:0/3921445145 conn(0x7f68b409aa00 msgr2=0x7f68b409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.520+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3921445145 shutdown_connections 2023-11-26T07:13:08.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.520+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3921445145 wait complete. 2023-11-26T07:13:08.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.521+0000 7f68ba1c9700 1 Processor -- start 2023-11-26T07:13:08.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.522+0000 7f68ba1c9700 1 -- start start 2023-11-26T07:13:08.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.522+0000 7f68ba1c9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68b40702e0 0x7f68b414d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:08.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.522+0000 7f68ba1c9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68b40cf040 0x7f68b40cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:08.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.522+0000 7f68ba1c9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b40d21c0 0x7f68b40b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:08.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.522+0000 7f68ba1c9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40c9090 con 0x7f68b40d21c0 2023-11-26T07:13:08.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.522+0000 7f68ba1c9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40b6980 con 0x7f68b40702e0 2023-11-26T07:13:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.522+0000 7f68ba1c9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40b5d70 con 0x7f68b40cf040 2023-11-26T07:13:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b40d21c0 0x7f68b40b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b40d21c0 0x7f68b40b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37370/0 (socket says 172.21.15.83:37370) 2023-11-26T07:13:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b3fff700 1 -- 172.21.15.83:0/3654631855 learned_addr learned my addr 172.21.15.83:0/3654631855 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b37fe700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68b40702e0 0x7f68b414d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b2ffd700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68b40cf040 0x7f68b40cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b3fff700 1 -- 172.21.15.83:0/3654631855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68b40cf040 msgr2=0x7f68b40cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b3fff700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68b40cf040 0x7f68b40cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:08.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b3fff700 1 -- 172.21.15.83:0/3654631855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68b40702e0 msgr2=0x7f68b414d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b3fff700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68b40702e0 0x7f68b414d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b3fff700 1 -- 172.21.15.83:0/3654631855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a8009550 con 0x7f68b40d21c0 2023-11-26T07:13:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b2ffd700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68b40cf040 0x7f68b40cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68b3fff700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b40d21c0 0x7f68b40b6e10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f68ac007e60 tx=0x7f68ac00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f689bfff700 1 -- 172.21.15.83:0/3654631855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00ede0 con 0x7f68b40d21c0 2023-11-26T07:13:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f689bfff700 1 -- 172.21.15.83:0/3654631855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac014420 con 0x7f68b40d21c0 2023-11-26T07:13:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3654631855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b40b6050 con 0x7f68b40d21c0 2023-11-26T07:13:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f689bfff700 1 -- 172.21.15.83:0/3654631855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0147d0 con 0x7f68b40d21c0 2023-11-26T07:13:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.523+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3654631855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b40b6610 con 0x7f68b40d21c0 2023-11-26T07:13:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.524+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3654631855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b40b67d0 con 0x7f68b40d21c0 2023-11-26T07:13:08.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.525+0000 7f689bfff700 1 -- 172.21.15.83:0/3654631855 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f68ac014930 con 0x7f68b40d21c0 2023-11-26T07:13:08.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.525+0000 7f689bfff700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68a006b080 0x7f68a006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:08.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.525+0000 7f68b37fe700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68a006b080 0x7f68a006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:08.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.525+0000 7f689bfff700 1 -- 172.21.15.83:0/3654631855 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68ac08a050 con 0x7f68b40d21c0 2023-11-26T07:13:08.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.526+0000 7f68b37fe700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68a006b080 0x7f68a006d540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f68a8000c00 tx=0x7f68a8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:08.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.530+0000 7f689bfff700 1 -- 172.21.15.83:0/3654631855 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68ac054e80 con 0x7f68b40d21c0 2023-11-26T07:13:08.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:08.766+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3654631855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f68b40b6af0 con 0x7f68b40d21c0 2023-11-26T07:13:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.601+0000 7f689bfff700 1 -- 172.21.15.83:0/3654631855 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 136+0+0 (secure 0 0 0) 0x7f68ac0584a0 con 0x7f68b40d21c0 2023-11-26T07:13:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.604+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3654631855 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68a006b080 msgr2=0x7f68a006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.604+0000 7f68ba1c9700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68a006b080 0x7f68a006d540 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f68a8000c00 tx=0x7f68a8008030 comp rx=0 tx=0).stop 2023-11-26T07:13:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.604+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3654631855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b40d21c0 msgr2=0x7f68b40b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.604+0000 7f68ba1c9700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b40d21c0 0x7f68b40b6e10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f68ac007e60 tx=0x7f68ac00e3c0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.605+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3654631855 shutdown_connections 2023-11-26T07:13:09.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.605+0000 7f68ba1c9700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68b40d21c0 0x7f68b40b6e10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.605+0000 7f68ba1c9700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68b40702e0 0x7f68b414d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.605+0000 7f68ba1c9700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68a006b080 0x7f68a006d540 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.605+0000 7f68ba1c9700 1 --2- 172.21.15.83:0/3654631855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68b40cf040 0x7f68b40cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.605+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3654631855 >> 172.21.15.83:0/3654631855 conn(0x7f68b409aa00 msgr2=0x7f68b40cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.605+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3654631855 shutdown_connections 2023-11-26T07:13:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.605+0000 7f68ba1c9700 1 -- 172.21.15.83:0/3654631855 wait complete. 2023-11-26T07:13:09.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:13:09.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.750+0000 7f8ed71ea700 1 Processor -- start 2023-11-26T07:13:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.750+0000 7f8ed71ea700 1 -- start start 2023-11-26T07:13:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed71ea700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ed015c7d0 0x7f8ed015cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed71ea700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ed00d0790 0x7f8ed015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed71ea700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed015fa50 con 0x7f8ed0062310 2023-11-26T07:13:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed71ea700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed015fca0 con 0x7f8ed00d0790 2023-11-26T07:13:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed71ea700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed015ff80 con 0x7f8ed015c7d0 2023-11-26T07:13:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed5787700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ed00d0790 0x7f8ed015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:09.755 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ecffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8ed0062310 0x7f8ed00d0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57414/0 (socket says 172.21.15.83:57414) 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ecffff700 1 -- 172.21.15.83:0/156067963 learned_addr learned my addr 172.21.15.83:0/156067963 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed5787700 1 -- 172.21.15.83:0/156067963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ed015c7d0 msgr2=0x7f8ed015cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed4f86700 1 --2- 172.21.15.83:0/156067963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ed015c7d0 0x7f8ed015cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ecf7fe700 1 -- 172.21.15.83:0/156067963 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126493288 0 0) 0x7f8ed015fa50 con 0x7f8ed0062310 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed5787700 1 --2- 172.21.15.83:0/156067963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ed015c7d0 0x7f8ed015cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed5787700 1 -- 172.21.15.83:0/156067963 >> v1:172.21.15.83:6789/0 conn(0x7f8ed0062310 legacy=0x7f8ed00d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed5787700 1 -- 172.21.15.83:0/156067963 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed0160260 con 0x7f8ed00d0790 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed4f86700 1 --2- 172.21.15.83:0/156067963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ed015c7d0 0x7f8ed015cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ed5787700 1 --2- 172.21.15.83:0/156067963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ed00d0790 0x7f8ed015d7d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009d90 tx=0x7f8ec800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1cdd6d25a7cb1d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ecf7fe700 1 -- 172.21.15.83:0/156067963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800e070 con 0x7f8ed00d0790 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ecf7fe700 1 -- 172.21.15.83:0/156067963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec800bbd0 con 0x7f8ed00d0790 2023-11-26T07:13:09.756 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.751+0000 7f8ecf7fe700 1 -- 172.21.15.83:0/156067963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8012430 con 0x7f8ed00d0790 2023-11-26T07:13:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.752+0000 7f8ed71ea700 1 -- 172.21.15.83:0/156067963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ed00d0790 msgr2=0x7f8ed015d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.752+0000 7f8ed71ea700 1 --2- 172.21.15.83:0/156067963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ed00d0790 0x7f8ed015d7d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009d90 tx=0x7f8ec800b330 comp rx=0 tx=0).stop 2023-11-26T07:13:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.752+0000 7f8ed71ea700 1 -- 172.21.15.83:0/156067963 shutdown_connections 2023-11-26T07:13:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.752+0000 7f8ed71ea700 1 --2- 172.21.15.83:0/156067963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ed00d0790 0x7f8ed015d7d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.752+0000 7f8ed71ea700 1 --2- 172.21.15.83:0/156067963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ed015c7d0 0x7f8ed015cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.752+0000 7f8ed71ea700 1 -- 172.21.15.83:0/156067963 >> 172.21.15.83:0/156067963 conn(0x7f8ed00b5640 msgr2=0x7f8ed00b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.752+0000 7f8ed71ea700 1 -- 172.21.15.83:0/156067963 shutdown_connections 2023-11-26T07:13:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.752+0000 7f8ed71ea700 1 -- 172.21.15.83:0/156067963 wait complete. 2023-11-26T07:13:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.753+0000 7f8ed71ea700 1 Processor -- start 2023-11-26T07:13:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.753+0000 7f8ed71ea700 1 -- start start 2023-11-26T07:13:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.754+0000 7f8ed71ea700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ed0062310 0x7f8ed0144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.754+0000 7f8ed71ea700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ed00d0790 0x7f8ed00cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.754+0000 7f8ed71ea700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ed015c7d0 0x7f8ed00d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.754+0000 7f8ed71ea700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed015c090 con 0x7f8ed0062310 2023-11-26T07:13:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.754+0000 7f8ed71ea700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00cf350 con 0x7f8ed015c7d0 2023-11-26T07:13:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.754+0000 7f8ed71ea700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00afd20 con 0x7f8ed00d0790 2023-11-26T07:13:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.754+0000 7f8ed4f86700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ed0062310 0x7f8ed0144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.754+0000 7f8ed4f86700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ed0062310 0x7f8ed0144550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37378/0 (socket says 172.21.15.83:37378) 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.754+0000 7f8ed4f86700 1 -- 172.21.15.83:0/1741891875 learned_addr learned my addr 172.21.15.83:0/1741891875 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ed5787700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ed015c7d0 0x7f8ed00d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ecffff700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ed00d0790 0x7f8ed00cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ed4f86700 1 -- 172.21.15.83:0/1741891875 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ed00d0790 msgr2=0x7f8ed00cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ed4f86700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ed00d0790 0x7f8ed00cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ed4f86700 1 -- 172.21.15.83:0/1741891875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ed015c7d0 msgr2=0x7f8ed00d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ed4f86700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ed015c7d0 0x7f8ed00d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ed4f86700 1 -- 172.21.15.83:0/1741891875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec8009a70 con 0x7f8ed0062310 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ed4f86700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ed0062310 0x7f8ed0144550 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400c7f0 tx=0x7f8ec400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ecd7fa700 1 -- 172.21.15.83:0/1741891875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4011800 con 0x7f8ed0062310 2023-11-26T07:13:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ecd7fa700 1 -- 172.21.15.83:0/1741891875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec4011960 con 0x7f8ed0062310 2023-11-26T07:13:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ed71ea700 1 -- 172.21.15.83:0/1741891875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed00b2300 con 0x7f8ed0062310 2023-11-26T07:13:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ecd7fa700 1 -- 172.21.15.83:0/1741891875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4011d10 con 0x7f8ed0062310 2023-11-26T07:13:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.755+0000 7f8ed71ea700 1 -- 172.21.15.83:0/1741891875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed00cf5a0 con 0x7f8ed0062310 2023-11-26T07:13:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.756+0000 7f8ed71ea700 1 -- 172.21.15.83:0/1741891875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed00b2820 con 0x7f8ed0062310 2023-11-26T07:13:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.756+0000 7f8ecd7fa700 1 -- 172.21.15.83:0/1741891875 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f8ec4018360 con 0x7f8ed0062310 2023-11-26T07:13:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.757+0000 7f8ecd7fa700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8eb406b100 0x7f8eb406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.757+0000 7f8ecd7fa700 1 -- 172.21.15.83:0/1741891875 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8ec40898c0 con 0x7f8ed0062310 2023-11-26T07:13:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.757+0000 7f8ecffff700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8eb406b100 0x7f8eb406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.757+0000 7f8ecffff700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8eb406b100 0x7f8eb406d5c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0155660 tx=0x7f8ebc007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.761+0000 7f8ecd7fa700 1 -- 172.21.15.83:0/1741891875 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ec4054680 con 0x7f8ed0062310 2023-11-26T07:13:09.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.979+0000 7f8ed71ea700 1 -- 172.21.15.83:0/1741891875 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ed0160820 con 0x7f8eb406b100 2023-11-26T07:13:09.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.980+0000 7f8ecd7fa700 1 -- 172.21.15.83:0/1741891875 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f8ebc013420 con 0x7f8eb406b100 2023-11-26T07:13:09.983 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:13:09.983 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:13:09.983 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephfs" 2023-11-26T07:13:09.983 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:13:09.983 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:13:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.982+0000 7f8ed71ea700 1 -- 172.21.15.83:0/1741891875 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8eb406b100 msgr2=0x7f8eb406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:09.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.982+0000 7f8ed71ea700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8eb406b100 0x7f8eb406d5c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0155660 tx=0x7f8ebc007c00 comp rx=0 tx=0).stop 2023-11-26T07:13:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.982+0000 7f8ed71ea700 1 -- 172.21.15.83:0/1741891875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ed0062310 msgr2=0x7f8ed0144550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.982+0000 7f8ed71ea700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ed0062310 0x7f8ed0144550 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400c7f0 tx=0x7f8ec400cbb0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.983+0000 7f8ed71ea700 1 -- 172.21.15.83:0/1741891875 shutdown_connections 2023-11-26T07:13:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.983+0000 7f8ed71ea700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ed015c7d0 0x7f8ed00d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.983+0000 7f8ed71ea700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ed0062310 0x7f8ed0144550 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.983+0000 7f8ed71ea700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8eb406b100 0x7f8eb406d5c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.983+0000 7f8ed71ea700 1 --2- 172.21.15.83:0/1741891875 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ed00d0790 0x7f8ed00cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.983+0000 7f8ed71ea700 1 -- 172.21.15.83:0/1741891875 >> 172.21.15.83:0/1741891875 conn(0x7f8ed00b5640 msgr2=0x7f8ed01557d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.983+0000 7f8ed71ea700 1 -- 172.21.15.83:0/1741891875 shutdown_connections 2023-11-26T07:13:09.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:09.983+0000 7f8ed71ea700 1 -- 172.21.15.83:0/1741891875 wait complete. 2023-11-26T07:13:09.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:13:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.118+0000 7f17fb052700 1 Processor -- start 2023-11-26T07:13:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.118+0000 7f17fb052700 1 -- start start 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.118+0000 7f17fb052700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f406dec0 0x7f17f4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.118+0000 7f17fb052700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f17f4071080 0x7f17f415e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.118+0000 7f17fb052700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f40b2b00 con 0x7f17f4151660 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.118+0000 7f17fb052700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f406ee40 con 0x7f17f4071080 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.118+0000 7f17fb052700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f406e3a0 con 0x7f17f406dec0 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.119+0000 7f17f3fff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f406dec0 0x7f17f4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.119+0000 7f17f3fff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f406dec0 0x7f17f4070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:47846/0 (socket says 172.21.15.83:47846) 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.119+0000 7f17f95ef700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f17f4071080 0x7f17f415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.119+0000 7f17f3fff700 1 -- 172.21.15.83:0/4248265169 learned_addr learned my addr 172.21.15.83:0/4248265169 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.119+0000 7f17f95ef700 1 -- 172.21.15.83:0/4248265169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f406dec0 msgr2=0x7f17f4070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.119+0000 7f17f95ef700 1 --2- 172.21.15.83:0/4248265169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f406dec0 0x7f17f4070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.119+0000 7f17f95ef700 1 -- 172.21.15.83:0/4248265169 >> v1:172.21.15.203:6789/0 conn(0x7f17f4151660 legacy=0x7f17f40cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.119+0000 7f17f95ef700 1 -- 172.21.15.83:0/4248265169 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f40b2da0 con 0x7f17f4071080 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.119+0000 7f17f3fff700 1 --2- 172.21.15.83:0/4248265169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f406dec0 0x7f17f4070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.119+0000 7f17f95ef700 1 --2- 172.21.15.83:0/4248265169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4071080 0x7f17f415e040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00aea0 tx=0x7f17ec00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76727c3cd2ace260 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.120+0000 7f17f37fe700 1 -- 172.21.15.83:0/4248265169 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00f070 con 0x7f17f4071080 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.120+0000 7f17f37fe700 1 -- 172.21.15.83:0/4248265169 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec00a2e0 con 0x7f17f4071080 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.120+0000 7f17f37fe700 1 -- 172.21.15.83:0/4248265169 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00cb50 con 0x7f17f4071080 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.120+0000 7f17fb052700 1 -- 172.21.15.83:0/4248265169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4071080 msgr2=0x7f17f415e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.120+0000 7f17fb052700 1 --2- 172.21.15.83:0/4248265169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4071080 0x7f17f415e040 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00aea0 tx=0x7f17ec00c330 comp rx=0 tx=0).stop 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.120+0000 7f17fb052700 1 -- 172.21.15.83:0/4248265169 shutdown_connections 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.120+0000 7f17fb052700 1 --2- 172.21.15.83:0/4248265169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4071080 0x7f17f415e040 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.120+0000 7f17fb052700 1 --2- 172.21.15.83:0/4248265169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f406dec0 0x7f17f4070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.120+0000 7f17fb052700 1 -- 172.21.15.83:0/4248265169 >> 172.21.15.83:0/4248265169 conn(0x7f17f406b410 msgr2=0x7f17f406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.120+0000 7f17fb052700 1 -- 172.21.15.83:0/4248265169 shutdown_connections 2023-11-26T07:13:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.120+0000 7f17fb052700 1 -- 172.21.15.83:0/4248265169 wait complete. 2023-11-26T07:13:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.121+0000 7f17fb052700 1 Processor -- start 2023-11-26T07:13:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.121+0000 7f17fb052700 1 -- start start 2023-11-26T07:13:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.122+0000 7f17fb052700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f406dec0 0x7f17f41458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.122+0000 7f17fb052700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4071080 0x7f17f40cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.122+0000 7f17fb052700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f4151660 0x7f17f40cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.122+0000 7f17fb052700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40c8de0 con 0x7f17f406dec0 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.122+0000 7f17fb052700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40c90c0 con 0x7f17f4071080 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.122+0000 7f17fb052700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40c93a0 con 0x7f17f4151660 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.122+0000 7f17f8dee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f406dec0 0x7f17f41458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.122+0000 7f17f8dee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f406dec0 0x7f17f41458e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37394/0 (socket says 172.21.15.83:37394) 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.122+0000 7f17f8dee700 1 -- 172.21.15.83:0/3179400531 learned_addr learned my addr 172.21.15.83:0/3179400531 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17f95ef700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f4151660 0x7f17f40cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17f3fff700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4071080 0x7f17f40cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17f8dee700 1 -- 172.21.15.83:0/3179400531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f4151660 msgr2=0x7f17f40cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17f8dee700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f4151660 0x7f17f40cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17f8dee700 1 -- 172.21.15.83:0/3179400531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4071080 msgr2=0x7f17f40cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17f8dee700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4071080 0x7f17f40cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17f8dee700 1 -- 172.21.15.83:0/3179400531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e0009550 con 0x7f17f406dec0 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17f8dee700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f406dec0 0x7f17f41458e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f17e800c450 tx=0x7f17e800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17f17fa700 1 -- 172.21.15.83:0/3179400531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e80052d0 con 0x7f17f406dec0 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17f17fa700 1 -- 172.21.15.83:0/3179400531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e8005430 con 0x7f17f406dec0 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17fb052700 1 -- 172.21.15.83:0/3179400531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ec00ab80 con 0x7f17f406dec0 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17f17fa700 1 -- 172.21.15.83:0/3179400531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e80038c0 con 0x7f17f406dec0 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.123+0000 7f17fb052700 1 -- 172.21.15.83:0/3179400531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f40c9960 con 0x7f17f406dec0 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.124+0000 7f17fb052700 1 -- 172.21.15.83:0/3179400531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f4068480 con 0x7f17f406dec0 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.125+0000 7f17f17fa700 1 -- 172.21.15.83:0/3179400531 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f17e8003a20 con 0x7f17f406dec0 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.125+0000 7f17f17fa700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f17d806b080 0x7f17d806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.125+0000 7f17f17fa700 1 -- 172.21.15.83:0/3179400531 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f17e8088bc0 con 0x7f17f406dec0 2023-11-26T07:13:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.125+0000 7f17f3fff700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f17d806b080 0x7f17d806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.125+0000 7f17f3fff700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f17d806b080 0x7f17d806d540 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f17f4149660 tx=0x7f17e00092c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.129+0000 7f17f17fa700 1 -- 172.21.15.83:0/3179400531 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17e8053a40 con 0x7f17f406dec0 2023-11-26T07:13:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.322+0000 7f17fb052700 1 -- 172.21.15.83:0/3179400531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f17f4068fb0 con 0x7f17f406dec0 2023-11-26T07:13:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.322+0000 7f17f17fa700 1 -- 172.21.15.83:0/3179400531 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f17e80170c0 con 0x7f17f406dec0 2023-11-26T07:13:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 -- 172.21.15.83:0/3179400531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f17d806b080 msgr2=0x7f17d806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f17d806b080 0x7f17d806d540 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f17f4149660 tx=0x7f17e00092c0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 -- 172.21.15.83:0/3179400531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f406dec0 msgr2=0x7f17f41458e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f406dec0 0x7f17f41458e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f17e800c450 tx=0x7f17e800c760 comp rx=0 tx=0).stop 2023-11-26T07:13:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 -- 172.21.15.83:0/3179400531 shutdown_connections 2023-11-26T07:13:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f17d806b080 0x7f17d806d540 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17f4151660 0x7f17f40cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f406dec0 0x7f17f41458e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 --2- 172.21.15.83:0/3179400531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17f4071080 0x7f17f40cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 -- 172.21.15.83:0/3179400531 >> 172.21.15.83:0/3179400531 conn(0x7f17f406b410 msgr2=0x7f17f4150700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:10.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 -- 172.21.15.83:0/3179400531 shutdown_connections 2023-11-26T07:13:10.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.324+0000 7f17fb052700 1 -- 172.21.15.83:0/3179400531 wait complete. 2023-11-26T07:13:10.339 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:13:10.339 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:13:10.356 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:13:10.357 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:13:10.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:13:10.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.573+0000 7f92e1083700 1 Processor -- start 2023-11-26T07:13:10.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.573+0000 7f92e1083700 1 -- start start 2023-11-26T07:13:10.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92e1083700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92dc15bdd0 0x7f92dc1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92e1083700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92dc0b22d0 con 0x7f92dc0d1c70 2023-11-26T07:13:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92e1083700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92dc0b25b0 con 0x7f92dc0b3370 2023-11-26T07:13:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92e1083700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92dc0b2f50 con 0x7f92dc15bdd0 2023-11-26T07:13:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92dad9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92dc15bdd0 0x7f92dc1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92db59e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f92dc0d1c70 0x7f92dc165ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57442/0 (socket says 172.21.15.83:57442) 2023-11-26T07:13:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92db59e700 1 -- 172.21.15.83:0/2027568804 learned_addr learned my addr 172.21.15.83:0/2027568804 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92dad9d700 1 -- 172.21.15.83:0/2027568804 >> v1:172.21.15.203:6789/0 conn(0x7f92dc0b3370 legacy=0x7f92dc0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92dad9d700 1 -- 172.21.15.83:0/2027568804 >> v1:172.21.15.83:6789/0 conn(0x7f92dc0d1c70 legacy=0x7f92dc165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92dad9d700 1 -- 172.21.15.83:0/2027568804 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc0b2890 con 0x7f92dc15bdd0 2023-11-26T07:13:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92dad9d700 1 --2- 172.21.15.83:0/2027568804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc15bdd0 0x7f92dc1604e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009840 tx=0x7f92cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5644ec4db4d997a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92d9d9b700 1 -- 172.21.15.83:0/2027568804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc011830 con 0x7f92dc15bdd0 2023-11-26T07:13:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92d9d9b700 1 -- 172.21.15.83:0/2027568804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92cc011990 con 0x7f92dc15bdd0 2023-11-26T07:13:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.574+0000 7f92d9d9b700 1 -- 172.21.15.83:0/2027568804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc011d20 con 0x7f92dc15bdd0 2023-11-26T07:13:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.575+0000 7f92e1083700 1 -- 172.21.15.83:0/2027568804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc15bdd0 msgr2=0x7f92dc1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.575+0000 7f92e1083700 1 --2- 172.21.15.83:0/2027568804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc15bdd0 0x7f92dc1604e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009840 tx=0x7f92cc009b50 comp rx=0 tx=0).stop 2023-11-26T07:13:10.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.575+0000 7f92e1083700 1 -- 172.21.15.83:0/2027568804 shutdown_connections 2023-11-26T07:13:10.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.575+0000 7f92e1083700 1 --2- 172.21.15.83:0/2027568804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc15bdd0 0x7f92dc1604e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.575+0000 7f92e1083700 1 -- 172.21.15.83:0/2027568804 >> 172.21.15.83:0/2027568804 conn(0x7f92dc0731a0 msgr2=0x7f92dc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:10.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.575+0000 7f92e1083700 1 -- 172.21.15.83:0/2027568804 shutdown_connections 2023-11-26T07:13:10.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.575+0000 7f92e1083700 1 -- 172.21.15.83:0/2027568804 wait complete. 2023-11-26T07:13:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.576+0000 7f92e1083700 1 Processor -- start 2023-11-26T07:13:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.576+0000 7f92e1083700 1 -- start start 2023-11-26T07:13:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.577+0000 7f92e1083700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92dc0b3370 0x7f92dc156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.577+0000 7f92e1083700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92dc0d1c70 0x7f92dc06cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.577+0000 7f92e1083700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc15bdd0 0x7f92dc0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.577+0000 7f92e1083700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc0ca180 con 0x7f92dc15bdd0 2023-11-26T07:13:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.577+0000 7f92e1083700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc0c9cb0 con 0x7f92dc0d1c70 2023-11-26T07:13:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.577+0000 7f92e1083700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc15efb0 con 0x7f92dc0b3370 2023-11-26T07:13:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.577+0000 7f92db59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc15bdd0 0x7f92dc0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.577+0000 7f92db59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc15bdd0 0x7f92dc0c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37408/0 (socket says 172.21.15.83:37408) 2023-11-26T07:13:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.577+0000 7f92db59e700 1 -- 172.21.15.83:0/2329887355 learned_addr learned my addr 172.21.15.83:0/2329887355 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92dad9d700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92dc0b3370 0x7f92dc156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92da59c700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92dc0d1c70 0x7f92dc06cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92db59e700 1 -- 172.21.15.83:0/2329887355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92dc0b3370 msgr2=0x7f92dc156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92db59e700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92dc0b3370 0x7f92dc156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92db59e700 1 -- 172.21.15.83:0/2329887355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92dc0d1c70 msgr2=0x7f92dc06cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92db59e700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92dc0d1c70 0x7f92dc06cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92db59e700 1 -- 172.21.15.83:0/2329887355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc0094a0 con 0x7f92dc15bdd0 2023-11-26T07:13:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92dad9d700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92dc0b3370 0x7f92dc156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92da59c700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92dc0d1c70 0x7f92dc06cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92db59e700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc15bdd0 0x7f92dc0c9770 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f92d400cbc0 tx=0x7f92d400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92c37fe700 1 -- 172.21.15.83:0/2329887355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d400ab50 con 0x7f92dc15bdd0 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92c37fe700 1 -- 172.21.15.83:0/2329887355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d400acb0 con 0x7f92dc15bdd0 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92e1083700 1 -- 172.21.15.83:0/2329887355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92dc0c9ea0 con 0x7f92dc15bdd0 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92c37fe700 1 -- 172.21.15.83:0/2329887355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4012600 con 0x7f92dc15bdd0 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.578+0000 7f92e1083700 1 -- 172.21.15.83:0/2329887355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92dc0cabf0 con 0x7f92dc15bdd0 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.579+0000 7f92e1083700 1 -- 172.21.15.83:0/2329887355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92dc0ca770 con 0x7f92dc15bdd0 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.579+0000 7f92c37fe700 1 -- 172.21.15.83:0/2329887355 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f92d401b3b0 con 0x7f92dc15bdd0 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.580+0000 7f92c37fe700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f92c806b080 0x7f92c806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.580+0000 7f92c37fe700 1 -- 172.21.15.83:0/2329887355 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f92d4089530 con 0x7f92dc15bdd0 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.580+0000 7f92dad9d700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f92c806b080 0x7f92c806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.580+0000 7f92dad9d700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f92c806b080 0x7f92c806d540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0090b0 tx=0x7f92cc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.584+0000 7f92c37fe700 1 -- 172.21.15.83:0/2329887355 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92dc0ca770 con 0x7f92dc15bdd0 2023-11-26T07:13:10.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.780+0000 7f92e1083700 1 -- 172.21.15.83:0/2329887355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92dc0ca770 con 0x7f92dc15bdd0 2023-11-26T07:13:10.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.781+0000 7f92c37fe700 1 -- 172.21.15.83:0/2329887355 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f92dc0ca770 con 0x7f92dc15bdd0 2023-11-26T07:13:10.784 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:10.784 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:13:10.603141+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:52.958067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:53.875459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":5}]},{"pgid":"7.2c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:13:10.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.783+0000 7f92e1083700 1 -- 172.21.15.83:0/2329887355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f92c806b080 msgr2=0x7f92c806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.783+0000 7f92e1083700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f92c806b080 0x7f92c806d540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0090b0 tx=0x7f92cc008030 comp rx=0 tx=0).stop 2023-11-26T07:13:10.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.783+0000 7f92e1083700 1 -- 172.21.15.83:0/2329887355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc15bdd0 msgr2=0x7f92dc0c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.783+0000 7f92e1083700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc15bdd0 0x7f92dc0c9770 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f92d400cbc0 tx=0x7f92d400cf80 comp rx=0 tx=0).stop 2023-11-26T07:13:10.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.783+0000 7f92e1083700 1 -- 172.21.15.83:0/2329887355 shutdown_connections 2023-11-26T07:13:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.783+0000 7f92e1083700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92dc15bdd0 0x7f92dc0c9770 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.783+0000 7f92e1083700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f92c806b080 0x7f92c806d540 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.783+0000 7f92e1083700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92dc0b3370 0x7f92dc156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.783+0000 7f92e1083700 1 --2- 172.21.15.83:0/2329887355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92dc0d1c70 0x7f92dc06cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.783+0000 7f92e1083700 1 -- 172.21.15.83:0/2329887355 >> 172.21.15.83:0/2329887355 conn(0x7f92dc0731a0 msgr2=0x7f92dc0aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.784+0000 7f92e1083700 1 -- 172.21.15.83:0/2329887355 shutdown_connections 2023-11-26T07:13:10.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.784+0000 7f92e1083700 1 -- 172.21.15.83:0/2329887355 wait complete. 2023-11-26T07:13:10.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:12:52.958067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:12:53.875459+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:13:10.799 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:13:10.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.920+0000 7f95b2a2a700 1 Processor -- start 2023-11-26T07:13:10.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.920+0000 7f95b2a2a700 1 -- start start 2023-11-26T07:13:10.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.920+0000 7f95b2a2a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95ac15e160 0x7f95ac0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.920+0000 7f95b2a2a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ac0afc90 0x7f95ac073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.920+0000 7f95b2a2a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0b3c60 con 0x7f95ac0744d0 2023-11-26T07:13:10.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.920+0000 7f95b2a2a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac0b4500 con 0x7f95ac15e160 2023-11-26T07:13:10.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.920+0000 7f95b2a2a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac074960 con 0x7f95ac0afc90 2023-11-26T07:13:10.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95abfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95ac15e160 0x7f95ac0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95abfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f95ac15e160 0x7f95ac0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37412/0 (socket says 172.21.15.83:37412) 2023-11-26T07:13:10.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95abfff700 1 -- 172.21.15.83:0/441969672 learned_addr learned my addr 172.21.15.83:0/441969672 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:10.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95ab7fe700 1 --2- 172.21.15.83:0/441969672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ac0afc90 0x7f95ac073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95abfff700 1 -- 172.21.15.83:0/441969672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ac0afc90 msgr2=0x7f95ac073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:10.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95abfff700 1 --2- 172.21.15.83:0/441969672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ac0afc90 0x7f95ac073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95abfff700 1 -- 172.21.15.83:0/441969672 >> v1:172.21.15.203:6789/0 conn(0x7f95ac0744d0 legacy=0x7f95ac06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:10.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95abfff700 1 -- 172.21.15.83:0/441969672 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0b58c0 con 0x7f95ac15e160 2023-11-26T07:13:10.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95ab7fe700 1 --2- 172.21.15.83:0/441969672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ac0afc90 0x7f95ac073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:10.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95abfff700 1 --2- 172.21.15.83:0/441969672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95ac15e160 0x7f95ac0b5da0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f959c0098f0 tx=0x7f959c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed79c32ff430a18 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:10.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95aaffd700 1 -- 172.21.15.83:0/441969672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c004230 con 0x7f95ac15e160 2023-11-26T07:13:10.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95aaffd700 1 -- 172.21.15.83:0/441969672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c004390 con 0x7f95ac15e160 2023-11-26T07:13:10.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.921+0000 7f95aaffd700 1 -- 172.21.15.83:0/441969672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c0036a0 con 0x7f95ac15e160 2023-11-26T07:13:10.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.922+0000 7f95b2a2a700 1 -- 172.21.15.83:0/441969672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95ac15e160 msgr2=0x7f95ac0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.922+0000 7f95b2a2a700 1 --2- 172.21.15.83:0/441969672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95ac15e160 0x7f95ac0b5da0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f959c0098f0 tx=0x7f959c009c00 comp rx=0 tx=0).stop 2023-11-26T07:13:10.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.922+0000 7f95b2a2a700 1 -- 172.21.15.83:0/441969672 shutdown_connections 2023-11-26T07:13:10.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.922+0000 7f95b2a2a700 1 --2- 172.21.15.83:0/441969672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95ac15e160 0x7f95ac0b5da0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.922+0000 7f95b2a2a700 1 --2- 172.21.15.83:0/441969672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ac0afc90 0x7f95ac073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.922+0000 7f95b2a2a700 1 -- 172.21.15.83:0/441969672 >> 172.21.15.83:0/441969672 conn(0x7f95ac0d0bf0 msgr2=0x7f95ac0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:10.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.922+0000 7f95b2a2a700 1 -- 172.21.15.83:0/441969672 shutdown_connections 2023-11-26T07:13:10.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.922+0000 7f95b2a2a700 1 -- 172.21.15.83:0/441969672 wait complete. 2023-11-26T07:13:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.923+0000 7f95b2a2a700 1 Processor -- start 2023-11-26T07:13:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.923+0000 7f95b2a2a700 1 -- start start 2023-11-26T07:13:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95b2a2a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95ac0744d0 0x7f95ac1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95b2a2a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95ac0afc90 0x7f95ac15e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95b2a2a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ac15e160 0x7f95ac157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95b2a2a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac161640 con 0x7f95ac0744d0 2023-11-26T07:13:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95b2a2a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac1560a0 con 0x7f95ac0afc90 2023-11-26T07:13:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95b2a2a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac161360 con 0x7f95ac15e160 2023-11-26T07:13:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95abfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95ac0744d0 0x7f95ac1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95abfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95ac0744d0 0x7f95ac1556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37416/0 (socket says 172.21.15.83:37416) 2023-11-26T07:13:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95abfff700 1 -- 172.21.15.83:0/3412617743 learned_addr learned my addr 172.21.15.83:0/3412617743 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95abfff700 1 -- 172.21.15.83:0/3412617743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ac15e160 msgr2=0x7f95ac157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95abfff700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ac15e160 0x7f95ac157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95abfff700 1 -- 172.21.15.83:0/3412617743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95ac0afc90 msgr2=0x7f95ac15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95ab7fe700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95ac0afc90 0x7f95ac15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95abfff700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95ac0afc90 0x7f95ac15e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95abfff700 1 -- 172.21.15.83:0/3412617743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c009550 con 0x7f95ac0744d0 2023-11-26T07:13:10.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.924+0000 7f95abfff700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95ac0744d0 0x7f95ac1556c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f95ac072ed0 tx=0x7f959c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.925+0000 7f95a8ff9700 1 -- 172.21.15.83:0/3412617743 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00b860 con 0x7f95ac0744d0 2023-11-26T07:13:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.925+0000 7f95a8ff9700 1 -- 172.21.15.83:0/3412617743 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00b9e0 con 0x7f95ac0744d0 2023-11-26T07:13:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.925+0000 7f95a8ff9700 1 -- 172.21.15.83:0/3412617743 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00bd90 con 0x7f95ac0744d0 2023-11-26T07:13:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.925+0000 7f95b2a2a700 1 -- 172.21.15.83:0/3412617743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac156340 con 0x7f95ac0744d0 2023-11-26T07:13:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.925+0000 7f95b2a2a700 1 -- 172.21.15.83:0/3412617743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac155c00 con 0x7f95ac0744d0 2023-11-26T07:13:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.926+0000 7f95b2a2a700 1 -- 172.21.15.83:0/3412617743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac155f00 con 0x7f95ac0744d0 2023-11-26T07:13:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.926+0000 7f95a8ff9700 1 -- 172.21.15.83:0/3412617743 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f959c01b440 con 0x7f95ac0744d0 2023-11-26T07:13:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.926+0000 7f95a8ff9700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f959806b080 0x7f959806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.927+0000 7f95a8ff9700 1 -- 172.21.15.83:0/3412617743 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f959c08b290 con 0x7f95ac0744d0 2023-11-26T07:13:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.927+0000 7f95ab7fe700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f959806b080 0x7f959806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.927+0000 7f95ab7fe700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f959806b080 0x7f959806d540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f95a0006fd0 tx=0x7f95a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:10.931+0000 7f95a8ff9700 1 -- 172.21.15.83:0/3412617743 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95ac155f00 con 0x7f95ac0744d0 2023-11-26T07:13:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.120+0000 7f95b2a2a700 1 -- 172.21.15.83:0/3412617743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95ac155f00 con 0x7f95ac0744d0 2023-11-26T07:13:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.120+0000 7f95a8ff9700 1 -- 172.21.15.83:0/3412617743 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95ac155f00 con 0x7f95ac0744d0 2023-11-26T07:13:11.123 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:13:11.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.122+0000 7f95b2a2a700 1 -- 172.21.15.83:0/3412617743 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f959806b080 msgr2=0x7f959806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.122+0000 7f95b2a2a700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f959806b080 0x7f959806d540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f95a0006fd0 tx=0x7f95a0008040 comp rx=0 tx=0).stop 2023-11-26T07:13:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.122+0000 7f95b2a2a700 1 -- 172.21.15.83:0/3412617743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95ac0744d0 msgr2=0x7f95ac1556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.122+0000 7f95b2a2a700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95ac0744d0 0x7f95ac1556c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f95ac072ed0 tx=0x7f959c004750 comp rx=0 tx=0).stop 2023-11-26T07:13:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.122+0000 7f95b2a2a700 1 -- 172.21.15.83:0/3412617743 shutdown_connections 2023-11-26T07:13:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.122+0000 7f95b2a2a700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f959806b080 0x7f959806d540 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.122+0000 7f95b2a2a700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95ac15e160 0x7f95ac157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.122+0000 7f95b2a2a700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95ac0744d0 0x7f95ac1556c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.122+0000 7f95b2a2a700 1 --2- 172.21.15.83:0/3412617743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95ac0afc90 0x7f95ac15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.122+0000 7f95b2a2a700 1 -- 172.21.15.83:0/3412617743 >> 172.21.15.83:0/3412617743 conn(0x7f95ac0d0bf0 msgr2=0x7f95ac06e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.123+0000 7f95b2a2a700 1 -- 172.21.15.83:0/3412617743 shutdown_connections 2023-11-26T07:13:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.123+0000 7f95b2a2a700 1 -- 172.21.15.83:0/3412617743 wait complete. 2023-11-26T07:13:11.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:13:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.258+0000 7f388efb8700 1 Processor -- start 2023-11-26T07:13:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.258+0000 7f388efb8700 1 -- start start 2023-11-26T07:13:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f388efb8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f388815bf50 0x7f38880ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f388efb8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38880ce830 0x7f38880cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f388efb8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38880d0140 con 0x7f38880cf1f0 2023-11-26T07:13:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f388efb8700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388815c330 con 0x7f38880ce830 2023-11-26T07:13:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f388efb8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38880d03e0 con 0x7f388815bf50 2023-11-26T07:13:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f3887fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38880ce830 0x7f38880cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f388d555700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f38880cf1f0 0x7f38880c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57468/0 (socket says 172.21.15.83:57468) 2023-11-26T07:13:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f388d555700 1 -- 172.21.15.83:0/473059252 learned_addr learned my addr 172.21.15.83:0/473059252 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f388cd54700 1 --2- 172.21.15.83:0/473059252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f388815bf50 0x7f38880ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f3887fff700 1 -- 172.21.15.83:0/473059252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f388815bf50 msgr2=0x7f38880ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f3887fff700 1 --2- 172.21.15.83:0/473059252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f388815bf50 0x7f38880ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f3887fff700 1 -- 172.21.15.83:0/473059252 >> v1:172.21.15.83:6789/0 conn(0x7f38880cf1f0 legacy=0x7f38880c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f3887fff700 1 -- 172.21.15.83:0/473059252 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38880d06c0 con 0x7f38880ce830 2023-11-26T07:13:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f3887fff700 1 --2- 172.21.15.83:0/473059252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38880ce830 0x7f38880cecb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3874009840 tx=0x7f3874009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaf24f1408bc48d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f38877fe700 1 -- 172.21.15.83:0/473059252 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874011830 con 0x7f38880ce830 2023-11-26T07:13:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f38877fe700 1 -- 172.21.15.83:0/473059252 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3874011990 con 0x7f38880ce830 2023-11-26T07:13:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.259+0000 7f38877fe700 1 -- 172.21.15.83:0/473059252 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874011d20 con 0x7f38880ce830 2023-11-26T07:13:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.260+0000 7f388efb8700 1 -- 172.21.15.83:0/473059252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38880ce830 msgr2=0x7f38880cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.260+0000 7f388efb8700 1 --2- 172.21.15.83:0/473059252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38880ce830 0x7f38880cecb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3874009840 tx=0x7f3874009b50 comp rx=0 tx=0).stop 2023-11-26T07:13:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.260+0000 7f388efb8700 1 -- 172.21.15.83:0/473059252 shutdown_connections 2023-11-26T07:13:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.260+0000 7f388efb8700 1 --2- 172.21.15.83:0/473059252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f388815bf50 0x7f38880ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.260+0000 7f388efb8700 1 --2- 172.21.15.83:0/473059252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38880ce830 0x7f38880cecb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.260+0000 7f388efb8700 1 -- 172.21.15.83:0/473059252 >> 172.21.15.83:0/473059252 conn(0x7f38880b3780 msgr2=0x7f38880b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:11.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.260+0000 7f388efb8700 1 -- 172.21.15.83:0/473059252 shutdown_connections 2023-11-26T07:13:11.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.260+0000 7f388efb8700 1 -- 172.21.15.83:0/473059252 wait complete. 2023-11-26T07:13:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.261+0000 7f388efb8700 1 Processor -- start 2023-11-26T07:13:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.261+0000 7f388efb8700 1 -- start start 2023-11-26T07:13:11.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f388efb8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38880ce830 0x7f3888154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f388efb8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38880cf1f0 0x7f38881e5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f388efb8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f388815bf50 0x7f38881e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f388efb8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38881547d0 con 0x7f38880cf1f0 2023-11-26T07:13:11.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f388efb8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38881e63a0 con 0x7f38880ce830 2023-11-26T07:13:11.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f388efb8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38881e6680 con 0x7f388815bf50 2023-11-26T07:13:11.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f3887fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38880cf1f0 0x7f38881e5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f3887fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38880cf1f0 0x7f38881e5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37432/0 (socket says 172.21.15.83:37432) 2023-11-26T07:13:11.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f3887fff700 1 -- 172.21.15.83:0/1651836039 learned_addr learned my addr 172.21.15.83:0/1651836039 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:11.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f388cd54700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38880ce830 0x7f3888154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f388d555700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f388815bf50 0x7f38881e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f3887fff700 1 -- 172.21.15.83:0/1651836039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f388815bf50 msgr2=0x7f38881e5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f3887fff700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f388815bf50 0x7f38881e5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f3887fff700 1 -- 172.21.15.83:0/1651836039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38880ce830 msgr2=0x7f3888154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f3887fff700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38880ce830 0x7f3888154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.262+0000 7f3887fff700 1 -- 172.21.15.83:0/1651836039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740094a0 con 0x7f38880cf1f0 2023-11-26T07:13:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.263+0000 7f3887fff700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38880cf1f0 0x7f38881e5620 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f38740090b0 tx=0x7f3874004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.263+0000 7f38857fa700 1 -- 172.21.15.83:0/1651836039 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38740038a0 con 0x7f38880cf1f0 2023-11-26T07:13:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.263+0000 7f38857fa700 1 -- 172.21.15.83:0/1651836039 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387401f070 con 0x7f38880cf1f0 2023-11-26T07:13:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.263+0000 7f388efb8700 1 -- 172.21.15.83:0/1651836039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38881e6960 con 0x7f38880cf1f0 2023-11-26T07:13:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.263+0000 7f38857fa700 1 -- 172.21.15.83:0/1651836039 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3874003c60 con 0x7f38880cf1f0 2023-11-26T07:13:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.263+0000 7f388efb8700 1 -- 172.21.15.83:0/1651836039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38881e6f20 con 0x7f38880cf1f0 2023-11-26T07:13:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.264+0000 7f388efb8700 1 -- 172.21.15.83:0/1651836039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38881e70b0 con 0x7f38880cf1f0 2023-11-26T07:13:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.264+0000 7f38857fa700 1 -- 172.21.15.83:0/1651836039 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f387401a430 con 0x7f38880cf1f0 2023-11-26T07:13:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.265+0000 7f38857fa700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f386c06b080 0x7f386c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.265+0000 7f38857fa700 1 -- 172.21.15.83:0/1651836039 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3874021070 con 0x7f38880cf1f0 2023-11-26T07:13:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.265+0000 7f388cd54700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f386c06b080 0x7f386c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.265+0000 7f388cd54700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f386c06b080 0x7f386c06d540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f387c00a950 tx=0x7f387c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.268+0000 7f38857fa700 1 -- 172.21.15.83:0/1651836039 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38740566e0 con 0x7f38880cf1f0 2023-11-26T07:13:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.456+0000 7f388efb8700 1 -- 172.21.15.83:0/1651836039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3888068fb0 con 0x7f38880cf1f0 2023-11-26T07:13:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.457+0000 7f38857fa700 1 -- 172.21.15.83:0/1651836039 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3874059d00 con 0x7f38880cf1f0 2023-11-26T07:13:11.459 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:13:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.458+0000 7f388efb8700 1 -- 172.21.15.83:0/1651836039 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f386c06b080 msgr2=0x7f386c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.458+0000 7f388efb8700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f386c06b080 0x7f386c06d540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f387c00a950 tx=0x7f387c008040 comp rx=0 tx=0).stop 2023-11-26T07:13:11.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.458+0000 7f388efb8700 1 -- 172.21.15.83:0/1651836039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38880cf1f0 msgr2=0x7f38881e5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.459+0000 7f388efb8700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38880cf1f0 0x7f38881e5620 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f38740090b0 tx=0x7f3874004750 comp rx=0 tx=0).stop 2023-11-26T07:13:11.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.459+0000 7f388efb8700 1 -- 172.21.15.83:0/1651836039 shutdown_connections 2023-11-26T07:13:11.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.459+0000 7f388efb8700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f386c06b080 0x7f386c06d540 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.459+0000 7f388efb8700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f388815bf50 0x7f38881e5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.459+0000 7f388efb8700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38880ce830 0x7f3888154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.459+0000 7f388efb8700 1 --2- 172.21.15.83:0/1651836039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38880cf1f0 0x7f38881e5620 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.459+0000 7f388efb8700 1 -- 172.21.15.83:0/1651836039 >> 172.21.15.83:0/1651836039 conn(0x7f38880b3780 msgr2=0x7f38880c6240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:11.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.459+0000 7f388efb8700 1 -- 172.21.15.83:0/1651836039 shutdown_connections 2023-11-26T07:13:11.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.459+0000 7f388efb8700 1 -- 172.21.15.83:0/1651836039 wait complete. 2023-11-26T07:13:11.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:13:11.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.597+0000 7f35aafa1700 1 Processor -- start 2023-11-26T07:13:11.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.597+0000 7f35aafa1700 1 -- start start 2023-11-26T07:13:11.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35aafa1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f35a415e160 0x7f35a40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35aafa1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35a40746a0 0x7f35a406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35aafa1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a40b3c60 con 0x7f35a40afc90 2023-11-26T07:13:11.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35aafa1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a40b4500 con 0x7f35a40746a0 2023-11-26T07:13:11.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35aafa1700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a40b58c0 con 0x7f35a415e160 2023-11-26T07:13:11.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35a953e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35a40746a0 0x7f35a406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35a3fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f35a40afc90 0x7f35a4073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57478/0 (socket says 172.21.15.83:57478) 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35a3fff700 1 -- 172.21.15.83:0/188476823 learned_addr learned my addr 172.21.15.83:0/188476823 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35a8d3d700 1 --2- 172.21.15.83:0/188476823 >> v2:172.21.15.203:3300/0 conn(0x7f35a415e160 0x7f35a40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35a953e700 1 -- 172.21.15.83:0/188476823 >> v2:172.21.15.203:3300/0 conn(0x7f35a415e160 msgr2=0x7f35a40b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35a953e700 1 --2- 172.21.15.83:0/188476823 >> v2:172.21.15.203:3300/0 conn(0x7f35a415e160 0x7f35a40b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35a953e700 1 -- 172.21.15.83:0/188476823 >> v1:172.21.15.83:6789/0 conn(0x7f35a40afc90 legacy=0x7f35a4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35a953e700 1 -- 172.21.15.83:0/188476823 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40b0140 con 0x7f35a40746a0 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35a37fe700 1 -- 172.21.15.83:0/188476823 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002170106 0 0) 0x7f35a40b3c60 con 0x7f35a40afc90 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35a953e700 1 --2- 172.21.15.83:0/188476823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a40746a0 0x7f35a406d930 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f35a409efa0 tx=0x7f359c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4922c9eff42947 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.598+0000 7f35a37fe700 1 -- 172.21.15.83:0/188476823 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c00e070 con 0x7f35a40746a0 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.599+0000 7f35a37fe700 1 -- 172.21.15.83:0/188476823 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c00bb20 con 0x7f35a40746a0 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.599+0000 7f35a37fe700 1 -- 172.21.15.83:0/188476823 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c012430 con 0x7f35a40746a0 2023-11-26T07:13:11.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.599+0000 7f35aafa1700 1 -- 172.21.15.83:0/188476823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a40746a0 msgr2=0x7f35a406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.599+0000 7f35aafa1700 1 --2- 172.21.15.83:0/188476823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a40746a0 0x7f35a406d930 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f35a409efa0 tx=0x7f359c00b330 comp rx=0 tx=0).stop 2023-11-26T07:13:11.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.599+0000 7f35aafa1700 1 -- 172.21.15.83:0/188476823 shutdown_connections 2023-11-26T07:13:11.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.599+0000 7f35aafa1700 1 --2- 172.21.15.83:0/188476823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a40746a0 0x7f35a406d930 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.599+0000 7f35aafa1700 1 --2- 172.21.15.83:0/188476823 >> v2:172.21.15.203:3300/0 conn(0x7f35a415e160 0x7f35a40b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.599+0000 7f35aafa1700 1 -- 172.21.15.83:0/188476823 >> 172.21.15.83:0/188476823 conn(0x7f35a40d0bf0 msgr2=0x7f35a40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:11.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.599+0000 7f35aafa1700 1 -- 172.21.15.83:0/188476823 shutdown_connections 2023-11-26T07:13:11.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.599+0000 7f35aafa1700 1 -- 172.21.15.83:0/188476823 wait complete. 2023-11-26T07:13:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.600+0000 7f35aafa1700 1 Processor -- start 2023-11-26T07:13:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.600+0000 7f35aafa1700 1 -- start start 2023-11-26T07:13:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.601+0000 7f35aafa1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40746a0 0x7f35a41f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.601+0000 7f35aafa1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a40afc90 0x7f35a41f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.601+0000 7f35aafa1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a415e160 0x7f35a41f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.601+0000 7f35aafa1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a414b210 con 0x7f35a40afc90 2023-11-26T07:13:11.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.601+0000 7f35aafa1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a41f3990 con 0x7f35a40746a0 2023-11-26T07:13:11.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.601+0000 7f35aafa1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40c75f0 con 0x7f35a415e160 2023-11-26T07:13:11.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.601+0000 7f35a3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a40afc90 0x7f35a41f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.601+0000 7f35a3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a40afc90 0x7f35a41f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37446/0 (socket says 172.21.15.83:37446) 2023-11-26T07:13:11.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.601+0000 7f35a3fff700 1 -- 172.21.15.83:0/76406001 learned_addr learned my addr 172.21.15.83:0/76406001 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:11.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.601+0000 7f35a953e700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a415e160 0x7f35a41f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a8d3d700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40746a0 0x7f35a41f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a3fff700 1 -- 172.21.15.83:0/76406001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a415e160 msgr2=0x7f35a41f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a3fff700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a415e160 0x7f35a41f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a3fff700 1 -- 172.21.15.83:0/76406001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40746a0 msgr2=0x7f35a41f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a3fff700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40746a0 0x7f35a41f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a3fff700 1 -- 172.21.15.83:0/76406001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c009a70 con 0x7f35a40afc90 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a953e700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a415e160 0x7f35a41f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a8d3d700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40746a0 0x7f35a41f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a3fff700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a40afc90 0x7f35a41f7110 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f35900011e0 tx=0x7f359000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a17fa700 1 -- 172.21.15.83:0/76406001 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590004c80 con 0x7f35a40afc90 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a17fa700 1 -- 172.21.15.83:0/76406001 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3590004e00 con 0x7f35a40afc90 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35aafa1700 1 -- 172.21.15.83:0/76406001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a414b540 con 0x7f35a40afc90 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35a17fa700 1 -- 172.21.15.83:0/76406001 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590007310 con 0x7f35a40afc90 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.602+0000 7f35aafa1700 1 -- 172.21.15.83:0/76406001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a414bb00 con 0x7f35a40afc90 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.603+0000 7f35aafa1700 1 -- 172.21.15.83:0/76406001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a414bc90 con 0x7f35a40afc90 2023-11-26T07:13:11.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.603+0000 7f35a17fa700 1 -- 172.21.15.83:0/76406001 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3590007470 con 0x7f35a40afc90 2023-11-26T07:13:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.604+0000 7f35a17fa700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f358806b080 0x7f358806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.604+0000 7f35a17fa700 1 -- 172.21.15.83:0/76406001 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f35900896e0 con 0x7f35a40afc90 2023-11-26T07:13:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.604+0000 7f35a8d3d700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f358806b080 0x7f358806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.604+0000 7f35a8d3d700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f358806b080 0x7f358806d540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3598009ac0 tx=0x7f3598008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:11.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.608+0000 7f35a17fa700 1 -- 172.21.15.83:0/76406001 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35900544d0 con 0x7f35a40afc90 2023-11-26T07:13:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.800+0000 7f35aafa1700 1 -- 172.21.15.83:0/76406001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f35a4068fb0 con 0x7f35a40afc90 2023-11-26T07:13:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.800+0000 7f35a17fa700 1 -- 172.21.15.83:0/76406001 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3590057af0 con 0x7f35a40afc90 2023-11-26T07:13:11.803 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:13:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.802+0000 7f35aafa1700 1 -- 172.21.15.83:0/76406001 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f358806b080 msgr2=0x7f358806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.802+0000 7f35aafa1700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f358806b080 0x7f358806d540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3598009ac0 tx=0x7f3598008040 comp rx=0 tx=0).stop 2023-11-26T07:13:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.802+0000 7f35aafa1700 1 -- 172.21.15.83:0/76406001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a40afc90 msgr2=0x7f35a41f7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.802+0000 7f35aafa1700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a40afc90 0x7f35a41f7110 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f35900011e0 tx=0x7f359000cae0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.802+0000 7f35aafa1700 1 -- 172.21.15.83:0/76406001 shutdown_connections 2023-11-26T07:13:11.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.802+0000 7f35aafa1700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f358806b080 0x7f358806d540 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.802+0000 7f35aafa1700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a415e160 0x7f35a41f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.802+0000 7f35aafa1700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40746a0 0x7f35a41f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.802+0000 7f35aafa1700 1 --2- 172.21.15.83:0/76406001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35a40afc90 0x7f35a41f7110 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.802+0000 7f35aafa1700 1 -- 172.21.15.83:0/76406001 >> 172.21.15.83:0/76406001 conn(0x7f35a40d0bf0 msgr2=0x7f35a406e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:11.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.803+0000 7f35aafa1700 1 -- 172.21.15.83:0/76406001 shutdown_connections 2023-11-26T07:13:11.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.803+0000 7f35aafa1700 1 -- 172.21.15.83:0/76406001 wait complete. 2023-11-26T07:13:11.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:13:11.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.936+0000 7f56896f1700 1 Processor -- start 2023-11-26T07:13:11.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.936+0000 7f56896f1700 1 -- start start 2023-11-26T07:13:11.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.937+0000 7f56896f1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f568415af10 0x7f568406ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.937+0000 7f56896f1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568406f720 0x7f5684071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.937+0000 7f56896f1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5684072070 0x7f568415e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.937+0000 7f56896f1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5684072520 con 0x7f568415af10 2023-11-26T07:13:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.937+0000 7f56896f1700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568406d8b0 con 0x7f5684072070 2023-11-26T07:13:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.937+0000 7f56896f1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568406db00 con 0x7f568406f720 2023-11-26T07:13:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.937+0000 7f5682ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f568415af10 0x7f568406ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.937+0000 7f5682ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f568415af10 0x7f568406ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37452/0 (socket says 172.21.15.83:37452) 2023-11-26T07:13:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.937+0000 7f5682ffd700 1 -- 172.21.15.83:0/182132932 learned_addr learned my addr 172.21.15.83:0/182132932 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f56827fc700 1 --2- 172.21.15.83:0/182132932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568406f720 0x7f5684071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f56837fe700 1 --2- 172.21.15.83:0/182132932 >> v2:172.21.15.203:3300/0 conn(0x7f5684072070 0x7f568415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f5682ffd700 1 -- 172.21.15.83:0/182132932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568406f720 msgr2=0x7f5684071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f5682ffd700 1 --2- 172.21.15.83:0/182132932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568406f720 0x7f5684071b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f5682ffd700 1 -- 172.21.15.83:0/182132932 >> v2:172.21.15.203:3300/0 conn(0x7f5684072070 msgr2=0x7f568415e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:11.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f5682ffd700 1 --2- 172.21.15.83:0/182132932 >> v2:172.21.15.203:3300/0 conn(0x7f5684072070 0x7f568415e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f5682ffd700 1 -- 172.21.15.83:0/182132932 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568406dde0 con 0x7f568415af10 2023-11-26T07:13:11.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f5682ffd700 1 --2- 172.21.15.83:0/182132932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568415af10 0x7f568406ba00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5684072fd0 tx=0x7f5678009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e51058d5944bcce server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:11.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f5681ffb700 1 -- 172.21.15.83:0/182132932 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567800b870 con 0x7f568415af10 2023-11-26T07:13:11.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f5681ffb700 1 -- 172.21.15.83:0/182132932 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567800b9d0 con 0x7f568415af10 2023-11-26T07:13:11.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f5681ffb700 1 -- 172.21.15.83:0/182132932 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567800bd60 con 0x7f568415af10 2023-11-26T07:13:11.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f56896f1700 1 -- 172.21.15.83:0/182132932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568415af10 msgr2=0x7f568406ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.938+0000 7f56896f1700 1 --2- 172.21.15.83:0/182132932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568415af10 0x7f568406ba00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5684072fd0 tx=0x7f5678009bd0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.939+0000 7f56896f1700 1 -- 172.21.15.83:0/182132932 shutdown_connections 2023-11-26T07:13:11.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.939+0000 7f56896f1700 1 --2- 172.21.15.83:0/182132932 >> v2:172.21.15.203:3300/0 conn(0x7f5684072070 0x7f568415e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.939+0000 7f56896f1700 1 --2- 172.21.15.83:0/182132932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568415af10 0x7f568406ba00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.939+0000 7f56896f1700 1 --2- 172.21.15.83:0/182132932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568406f720 0x7f5684071b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.939+0000 7f56896f1700 1 -- 172.21.15.83:0/182132932 >> 172.21.15.83:0/182132932 conn(0x7f56840b3190 msgr2=0x7f56840d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:11.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.939+0000 7f56896f1700 1 -- 172.21.15.83:0/182132932 shutdown_connections 2023-11-26T07:13:11.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.939+0000 7f56896f1700 1 -- 172.21.15.83:0/182132932 wait complete. 2023-11-26T07:13:11.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.940+0000 7f56896f1700 1 Processor -- start 2023-11-26T07:13:11.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.940+0000 7f56896f1700 1 -- start start 2023-11-26T07:13:11.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56896f1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568406f720 0x7f5684157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56896f1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5684072070 0x7f568414d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56896f1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684154870 0x7f5684157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56896f1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5684157820 con 0x7f5684154870 2023-11-26T07:13:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56896f1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56840aeee0 con 0x7f5684072070 2023-11-26T07:13:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56896f1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56840af1c0 con 0x7f568406f720 2023-11-26T07:13:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56837fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684154870 0x7f5684157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56837fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684154870 0x7f5684157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37462/0 (socket says 172.21.15.83:37462) 2023-11-26T07:13:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56837fe700 1 -- 172.21.15.83:0/4269539958 learned_addr learned my addr 172.21.15.83:0/4269539958 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:11.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56827fc700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5684072070 0x7f568414d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f5682ffd700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568406f720 0x7f5684157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56837fe700 1 -- 172.21.15.83:0/4269539958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568406f720 msgr2=0x7f5684157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56837fe700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568406f720 0x7f5684157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56837fe700 1 -- 172.21.15.83:0/4269539958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5684072070 msgr2=0x7f568414d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56837fe700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5684072070 0x7f568414d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56837fe700 1 -- 172.21.15.83:0/4269539958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5678009550 con 0x7f5684154870 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56827fc700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5684072070 0x7f568414d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56837fe700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684154870 0x7f5684157f90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f567c007e40 tx=0x7f567c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f566b7fe700 1 -- 172.21.15.83:0/4269539958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c00eea0 con 0x7f5684154870 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f566b7fe700 1 -- 172.21.15.83:0/4269539958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c014420 con 0x7f5684154870 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56896f1700 1 -- 172.21.15.83:0/4269539958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56840af4a0 con 0x7f5684154870 2023-11-26T07:13:11.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f566b7fe700 1 -- 172.21.15.83:0/4269539958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0147b0 con 0x7f5684154870 2023-11-26T07:13:11.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.941+0000 7f56896f1700 1 -- 172.21.15.83:0/4269539958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568415a310 con 0x7f5684154870 2023-11-26T07:13:11.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.942+0000 7f56896f1700 1 -- 172.21.15.83:0/4269539958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56840af020 con 0x7f5684154870 2023-11-26T07:13:11.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.943+0000 7f566b7fe700 1 -- 172.21.15.83:0/4269539958 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f567c014910 con 0x7f5684154870 2023-11-26T07:13:11.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.943+0000 7f566b7fe700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f567006b080 0x7f567006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:11.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.943+0000 7f566b7fe700 1 -- 172.21.15.83:0/4269539958 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f567c08a1c0 con 0x7f5684154870 2023-11-26T07:13:11.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.943+0000 7f5682ffd700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f567006b080 0x7f567006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:11.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.944+0000 7f5682ffd700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f567006b080 0x7f567006d540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5678009c10 tx=0x7f5678008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:11.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:11.948+0000 7f566b7fe700 1 -- 172.21.15.83:0/4269539958 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56840af020 con 0x7f5684154870 2023-11-26T07:13:12.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.152+0000 7f56896f1700 1 -- 172.21.15.83:0/4269539958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f56840af020 con 0x7f5684154870 2023-11-26T07:13:12.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.153+0000 7f566b7fe700 1 -- 172.21.15.83:0/4269539958 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7f56840af020 con 0x7f5684154870 2023-11-26T07:13:12.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.154+0000 7f56896f1700 1 -- 172.21.15.83:0/4269539958 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f567006b080 msgr2=0x7f567006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.154+0000 7f56896f1700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f567006b080 0x7f567006d540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5678009c10 tx=0x7f5678008030 comp rx=0 tx=0).stop 2023-11-26T07:13:12.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.154+0000 7f56896f1700 1 -- 172.21.15.83:0/4269539958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684154870 msgr2=0x7f5684157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.154+0000 7f56896f1700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684154870 0x7f5684157f90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f567c007e40 tx=0x7f567c00e3c0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.155+0000 7f56896f1700 1 -- 172.21.15.83:0/4269539958 shutdown_connections 2023-11-26T07:13:12.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.155+0000 7f56896f1700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5684154870 0x7f5684157f90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.155+0000 7f56896f1700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f567006b080 0x7f567006d540 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.155+0000 7f56896f1700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568406f720 0x7f5684157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.155+0000 7f56896f1700 1 --2- 172.21.15.83:0/4269539958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5684072070 0x7f568414d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.155+0000 7f56896f1700 1 -- 172.21.15.83:0/4269539958 >> 172.21.15.83:0/4269539958 conn(0x7f56840b3190 msgr2=0x7f56840b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:12.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.155+0000 7f56896f1700 1 -- 172.21.15.83:0/4269539958 shutdown_connections 2023-11-26T07:13:12.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.155+0000 7f56896f1700 1 -- 172.21.15.83:0/4269539958 wait complete. 2023-11-26T07:13:12.159 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:13:12.169 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:13:12.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:12.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:13:12.214 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:13:12.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:12.214 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:13:12.231 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:13:12.231 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:13:12.253 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:13:12.253 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:13:12.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:13:12.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.430+0000 7f4b9ebe0700 1 Processor -- start 2023-11-26T07:13:12.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.430+0000 7f4b9ebe0700 1 -- start start 2023-11-26T07:13:12.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b9ebe0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b9815e080 0x7f4b980b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b9ebe0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4b9806c710 0x7f4b980afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b9ebe0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b980ae430 con 0x7f4b980af4d0 2023-11-26T07:13:12.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b9ebe0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b980af0b0 con 0x7f4b9815e080 2023-11-26T07:13:12.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b9ebe0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b980b05a0 con 0x7f4b9806c710 2023-11-26T07:13:12.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b9c97c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4b9815e080 0x7f4b980b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:12.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b97fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4b980af4d0 0x7f4b9806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57506/0 (socket says 172.21.15.83:57506) 2023-11-26T07:13:12.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b97fff700 1 -- 172.21.15.83:0/701605598 learned_addr learned my addr 172.21.15.83:0/701605598 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:12.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b9c97c700 1 -- 172.21.15.83:0/701605598 >> v2:172.21.15.203:3300/0 conn(0x7f4b9806c710 msgr2=0x7f4b980afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:12.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b9c97c700 1 --2- 172.21.15.83:0/701605598 >> v2:172.21.15.203:3300/0 conn(0x7f4b9806c710 0x7f4b980afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b9c97c700 1 -- 172.21.15.83:0/701605598 >> v1:172.21.15.83:6789/0 conn(0x7f4b980af4d0 legacy=0x7f4b9806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b9c97c700 1 -- 172.21.15.83:0/701605598 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980ae710 con 0x7f4b9815e080 2023-11-26T07:13:12.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b977fe700 1 -- 172.21.15.83:0/701605598 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996668890 0 0) 0x7f4b980ae430 con 0x7f4b980af4d0 2023-11-26T07:13:12.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.431+0000 7f4b9c97c700 1 --2- 172.21.15.83:0/701605598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b9815e080 0x7f4b980b5050 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4b9806e060 tx=0x7f4b8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff1ea8486f34fa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:12.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.432+0000 7f4b977fe700 1 -- 172.21.15.83:0/701605598 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c011830 con 0x7f4b9815e080 2023-11-26T07:13:12.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.432+0000 7f4b977fe700 1 -- 172.21.15.83:0/701605598 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8c011990 con 0x7f4b9815e080 2023-11-26T07:13:12.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.432+0000 7f4b977fe700 1 -- 172.21.15.83:0/701605598 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c011d20 con 0x7f4b9815e080 2023-11-26T07:13:12.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.432+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/701605598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b9815e080 msgr2=0x7f4b980b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.432+0000 7f4b9ebe0700 1 --2- 172.21.15.83:0/701605598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b9815e080 0x7f4b980b5050 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4b9806e060 tx=0x7f4b8c009b50 comp rx=0 tx=0).stop 2023-11-26T07:13:12.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.432+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/701605598 shutdown_connections 2023-11-26T07:13:12.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.432+0000 7f4b9ebe0700 1 --2- 172.21.15.83:0/701605598 >> v2:172.21.15.203:3300/0 conn(0x7f4b9806c710 0x7f4b980afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.432+0000 7f4b9ebe0700 1 --2- 172.21.15.83:0/701605598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b9815e080 0x7f4b980b5050 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.432+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/701605598 >> 172.21.15.83:0/701605598 conn(0x7f4b98071b90 msgr2=0x7f4b98071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:12.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.432+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/701605598 shutdown_connections 2023-11-26T07:13:12.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.432+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/701605598 wait complete. 2023-11-26T07:13:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.433+0000 7f4b9ebe0700 1 Processor -- start 2023-11-26T07:13:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.433+0000 7f4b9ebe0700 1 -- start start 2023-11-26T07:13:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.434+0000 7f4b9ebe0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b9806c710 0x7f4b981429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.434+0000 7f4b9ebe0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b980af4d0 0x7f4b9815a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.434+0000 7f4b9ebe0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b9815e080 0x7f4b98155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.434+0000 7f4b9ebe0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b98157ff0 con 0x7f4b9815e080 2023-11-26T07:13:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.434+0000 7f4b9ebe0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b98146120 con 0x7f4b9806c710 2023-11-26T07:13:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.434+0000 7f4b9ebe0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b981593b0 con 0x7f4b980af4d0 2023-11-26T07:13:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.434+0000 7f4b9d17d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b9815e080 0x7f4b98155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.434+0000 7f4b9d17d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b9815e080 0x7f4b98155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37472/0 (socket says 172.21.15.83:37472) 2023-11-26T07:13:12.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.434+0000 7f4b9d17d700 1 -- 172.21.15.83:0/3297540108 learned_addr learned my addr 172.21.15.83:0/3297540108 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:12.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b9c97c700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b9806c710 0x7f4b981429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:12.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b9d17d700 1 -- 172.21.15.83:0/3297540108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b980af4d0 msgr2=0x7f4b9815a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b9d17d700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b980af4d0 0x7f4b9815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b9d17d700 1 -- 172.21.15.83:0/3297540108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b9806c710 msgr2=0x7f4b981429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b9d17d700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b9806c710 0x7f4b981429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b9d17d700 1 -- 172.21.15.83:0/3297540108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c0094a0 con 0x7f4b9815e080 2023-11-26T07:13:12.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b9c97c700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b9806c710 0x7f4b981429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b9d17d700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b9815e080 0x7f4b98155e10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000b9e0 tx=0x7f4b9000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b957fa700 1 -- 172.21.15.83:0/3297540108 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000c9b0 con 0x7f4b9815e080 2023-11-26T07:13:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b957fa700 1 -- 172.21.15.83:0/3297540108 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9000cb30 con 0x7f4b9815e080 2023-11-26T07:13:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/3297540108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b98145e40 con 0x7f4b9815e080 2023-11-26T07:13:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b957fa700 1 -- 172.21.15.83:0/3297540108 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90014440 con 0x7f4b9815e080 2023-11-26T07:13:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.435+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/3297540108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b98159930 con 0x7f4b9815e080 2023-11-26T07:13:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.436+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/3297540108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b980753f0 con 0x7f4b9815e080 2023-11-26T07:13:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.436+0000 7f4b957fa700 1 -- 172.21.15.83:0/3297540108 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4b9001d3b0 con 0x7f4b9815e080 2023-11-26T07:13:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.437+0000 7f4b957fa700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b7c06b080 0x7f4b7c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.437+0000 7f4b957fa700 1 -- 172.21.15.83:0/3297540108 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4b90006e40 con 0x7f4b9815e080 2023-11-26T07:13:12.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.437+0000 7f4b9c97c700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b7c06b080 0x7f4b7c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:12.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.437+0000 7f4b9c97c700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b7c06b080 0x7f4b7c06d540 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4b9806c000 tx=0x7f4b8c01c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:12.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.441+0000 7f4b957fa700 1 -- 172.21.15.83:0/3297540108 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b90054480 con 0x7f4b9815e080 2023-11-26T07:13:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.633+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/3297540108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b9806e590 con 0x7f4b9815e080 2023-11-26T07:13:12.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.634+0000 7f4b957fa700 1 -- 172.21.15.83:0/3297540108 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f4b90057aa0 con 0x7f4b9815e080 2023-11-26T07:13:12.637 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:12.637 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:13:10.603141+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:52.958067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:53.875459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":5}]},{"pgid":"7.2c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:13:12.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/3297540108 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b7c06b080 msgr2=0x7f4b7c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b7c06b080 0x7f4b7c06d540 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4b9806c000 tx=0x7f4b8c01c040 comp rx=0 tx=0).stop 2023-11-26T07:13:12.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/3297540108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b9815e080 msgr2=0x7f4b98155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b9815e080 0x7f4b98155e10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000b9e0 tx=0x7f4b9000bda0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/3297540108 shutdown_connections 2023-11-26T07:13:12.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b9815e080 0x7f4b98155e10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b9806c710 0x7f4b981429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4b7c06b080 0x7f4b7c06d540 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 --2- 172.21.15.83:0/3297540108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b980af4d0 0x7f4b9815a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/3297540108 >> 172.21.15.83:0/3297540108 conn(0x7f4b98071b90 msgr2=0x7f4b980b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/3297540108 shutdown_connections 2023-11-26T07:13:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.636+0000 7f4b9ebe0700 1 -- 172.21.15.83:0/3297540108 wait complete. 2023-11-26T07:13:12.651 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:12:52.958067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:12:53.875459+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:13:12.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:13:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.772+0000 7fcc5d513700 1 Processor -- start 2023-11-26T07:13:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.772+0000 7fcc5d513700 1 -- start start 2023-11-26T07:13:12.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc5d513700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcc5815e090 0x7fcc580d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc5d513700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc580d24e0 0x7fcc58071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc5d513700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc58071f70 con 0x7fcc58071b50 2023-11-26T07:13:12.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc5d513700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc580d2a60 con 0x7fcc5815e090 2023-11-26T07:13:12.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc5d513700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc580d2d40 con 0x7fcc580d24e0 2023-11-26T07:13:12.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc56ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcc5815e090 0x7fcc580d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:12.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc567fc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc580d24e0 0x7fcc58071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc577fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fcc58071b50 0x7fcc580b5920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:42136/0 (socket says 172.21.15.83:42136) 2023-11-26T07:13:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc56ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcc5815e090 0x7fcc580d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:36930/0 (socket says 172.21.15.83:36930) 2023-11-26T07:13:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc567fc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc580d24e0 0x7fcc58071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:47936/0 (socket says 172.21.15.83:47936) 2023-11-26T07:13:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc577fe700 1 -- 172.21.15.83:0/2218952454 learned_addr learned my addr 172.21.15.83:0/2218952454 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc56ffd700 1 -- 172.21.15.83:0/2218952454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc580d24e0 msgr2=0x7fcc58071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc56ffd700 1 --2- 172.21.15.83:0/2218952454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc580d24e0 0x7fcc58071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc56ffd700 1 -- 172.21.15.83:0/2218952454 >> v1:172.21.15.203:6789/0 conn(0x7fcc58071b50 legacy=0x7fcc580b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc56ffd700 1 -- 172.21.15.83:0/2218952454 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc580d3020 con 0x7fcc5815e090 2023-11-26T07:13:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.773+0000 7fcc567fc700 1 --2- 172.21.15.83:0/2218952454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc580d24e0 0x7fcc58071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:13:12.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc56ffd700 1 --2- 172.21.15.83:0/2218952454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc5815e090 0x7fcc580d1fa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009840 tx=0x7fcc48009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e1991ed09117a27 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:12.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc55ffb700 1 -- 172.21.15.83:0/2218952454 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48011830 con 0x7fcc5815e090 2023-11-26T07:13:12.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc55ffb700 1 -- 172.21.15.83:0/2218952454 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48011990 con 0x7fcc5815e090 2023-11-26T07:13:12.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc55ffb700 1 -- 172.21.15.83:0/2218952454 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48011d20 con 0x7fcc5815e090 2023-11-26T07:13:12.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc5d513700 1 -- 172.21.15.83:0/2218952454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc5815e090 msgr2=0x7fcc580d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc5d513700 1 --2- 172.21.15.83:0/2218952454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc5815e090 0x7fcc580d1fa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009840 tx=0x7fcc48009b50 comp rx=0 tx=0).stop 2023-11-26T07:13:12.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc5d513700 1 -- 172.21.15.83:0/2218952454 shutdown_connections 2023-11-26T07:13:12.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc5d513700 1 --2- 172.21.15.83:0/2218952454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc5815e090 0x7fcc580d1fa0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc5d513700 1 --2- 172.21.15.83:0/2218952454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc580d24e0 0x7fcc58071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc5d513700 1 -- 172.21.15.83:0/2218952454 >> 172.21.15.83:0/2218952454 conn(0x7fcc5806b410 msgr2=0x7fcc5806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc5d513700 1 -- 172.21.15.83:0/2218952454 shutdown_connections 2023-11-26T07:13:12.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.774+0000 7fcc5d513700 1 -- 172.21.15.83:0/2218952454 wait complete. 2023-11-26T07:13:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.776+0000 7fcc5d513700 1 Processor -- start 2023-11-26T07:13:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.776+0000 7fcc5d513700 1 -- start start 2023-11-26T07:13:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.776+0000 7fcc5d513700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc58071b50 0x7fcc58151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.776+0000 7fcc5d513700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc580d24e0 0x7fcc580cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.776+0000 7fcc5d513700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5815e090 0x7fcc581604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.776+0000 7fcc5d513700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc581614f0 con 0x7fcc5815e090 2023-11-26T07:13:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.776+0000 7fcc5d513700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc58160f30 con 0x7fcc580d24e0 2023-11-26T07:13:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.776+0000 7fcc5d513700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc58161210 con 0x7fcc58071b50 2023-11-26T07:13:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.776+0000 7fcc577fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5815e090 0x7fcc581604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.776+0000 7fcc577fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5815e090 0x7fcc581604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37484/0 (socket says 172.21.15.83:37484) 2023-11-26T07:13:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.776+0000 7fcc577fe700 1 -- 172.21.15.83:0/872947192 learned_addr learned my addr 172.21.15.83:0/872947192 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc56ffd700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc58071b50 0x7fcc58151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc567fc700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc580d24e0 0x7fcc580cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc577fe700 1 -- 172.21.15.83:0/872947192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc58071b50 msgr2=0x7fcc58151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc577fe700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc58071b50 0x7fcc58151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc577fe700 1 -- 172.21.15.83:0/872947192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc580d24e0 msgr2=0x7fcc580cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc577fe700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc580d24e0 0x7fcc580cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc577fe700 1 -- 172.21.15.83:0/872947192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0094a0 con 0x7fcc5815e090 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc56ffd700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc58071b50 0x7fcc58151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc577fe700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5815e090 0x7fcc581604e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000cbc0 tx=0x7fcc5000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc3f7fe700 1 -- 172.21.15.83:0/872947192 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000ab50 con 0x7fcc5815e090 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc3f7fe700 1 -- 172.21.15.83:0/872947192 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5000acb0 con 0x7fcc5815e090 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc5d513700 1 -- 172.21.15.83:0/872947192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc480094a0 con 0x7fcc5815e090 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc3f7fe700 1 -- 172.21.15.83:0/872947192 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50012600 con 0x7fcc5815e090 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.777+0000 7fcc5d513700 1 -- 172.21.15.83:0/872947192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc580cb360 con 0x7fcc5815e090 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.778+0000 7fcc5d513700 1 -- 172.21.15.83:0/872947192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc580caee0 con 0x7fcc5815e090 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.778+0000 7fcc3f7fe700 1 -- 172.21.15.83:0/872947192 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fcc5001b3b0 con 0x7fcc5815e090 2023-11-26T07:13:12.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.779+0000 7fcc3f7fe700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc4406b080 0x7fcc4406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:12.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.779+0000 7fcc3f7fe700 1 -- 172.21.15.83:0/872947192 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcc50089490 con 0x7fcc5815e090 2023-11-26T07:13:12.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.779+0000 7fcc56ffd700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc4406b080 0x7fcc4406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:12.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.779+0000 7fcc56ffd700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc4406b080 0x7fcc4406d540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc4801c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:12.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.783+0000 7fcc3f7fe700 1 -- 172.21.15.83:0/872947192 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc580caee0 con 0x7fcc5815e090 2023-11-26T07:13:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.970+0000 7fcc5d513700 1 -- 172.21.15.83:0/872947192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc580caee0 con 0x7fcc5815e090 2023-11-26T07:13:12.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.970+0000 7fcc3f7fe700 1 -- 172.21.15.83:0/872947192 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc580caee0 con 0x7fcc5815e090 2023-11-26T07:13:12.973 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:13:12.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 -- 172.21.15.83:0/872947192 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc4406b080 msgr2=0x7fcc4406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc4406b080 0x7fcc4406d540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc4801c040 comp rx=0 tx=0).stop 2023-11-26T07:13:12.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 -- 172.21.15.83:0/872947192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5815e090 msgr2=0x7fcc581604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:12.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5815e090 0x7fcc581604e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000cbc0 tx=0x7fcc5000cf80 comp rx=0 tx=0).stop 2023-11-26T07:13:12.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 -- 172.21.15.83:0/872947192 shutdown_connections 2023-11-26T07:13:12.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc5815e090 0x7fcc581604e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc4406b080 0x7fcc4406d540 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc58071b50 0x7fcc58151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 --2- 172.21.15.83:0/872947192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc580d24e0 0x7fcc580cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:12.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 -- 172.21.15.83:0/872947192 >> 172.21.15.83:0/872947192 conn(0x7fcc5806b410 msgr2=0x7fcc5806b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:12.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 -- 172.21.15.83:0/872947192 shutdown_connections 2023-11-26T07:13:12.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:12.972+0000 7fcc5d513700 1 -- 172.21.15.83:0/872947192 wait complete. 2023-11-26T07:13:12.987 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:13:13.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.107+0000 7f900072c700 1 Processor -- start 2023-11-26T07:13:13.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.107+0000 7f900072c700 1 -- start start 2023-11-26T07:13:13.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.108+0000 7f900072c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ff815e160 0x7f8ff80b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.108+0000 7f900072c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ff80afc90 0x7f8ff8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.108+0000 7f900072c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff80b3c60 con 0x7f8ff80744d0 2023-11-26T07:13:13.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.108+0000 7f900072c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff80b4500 con 0x7f8ff80afc90 2023-11-26T07:13:13.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.108+0000 7f900072c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff8074960 con 0x7f8ff815e160 2023-11-26T07:13:13.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.108+0000 7f8ffdcc7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ff80afc90 0x7f8ff8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.108+0000 7f8ffe4c8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ff815e160 0x7f8ff80b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.108+0000 7f8ffecc9700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f8ff80744d0 0x7f8ff806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:42150/0 (socket says 172.21.15.83:42150) 2023-11-26T07:13:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.108+0000 7f8ffdcc7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ff80afc90 0x7f8ff8073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:36942/0 (socket says 172.21.15.83:36942) 2023-11-26T07:13:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.108+0000 7f8ffecc9700 1 -- 172.21.15.83:0/2585387092 learned_addr learned my addr 172.21.15.83:0/2585387092 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.109+0000 7f8ffe4c8700 1 -- 172.21.15.83:0/2585387092 >> v2:172.21.15.203:3300/0 conn(0x7f8ff80afc90 msgr2=0x7f8ff8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.109+0000 7f8ffe4c8700 1 --2- 172.21.15.83:0/2585387092 >> v2:172.21.15.203:3300/0 conn(0x7f8ff80afc90 0x7f8ff8073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.109+0000 7f8ffe4c8700 1 -- 172.21.15.83:0/2585387092 >> v1:172.21.15.203:6789/0 conn(0x7f8ff80744d0 legacy=0x7f8ff806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.109+0000 7f8ffd4c6700 1 -- 172.21.15.83:0/2585387092 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443791604 0 0) 0x7f8ff80b3c60 con 0x7f8ff80744d0 2023-11-26T07:13:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.109+0000 7f8ffe4c8700 1 -- 172.21.15.83:0/2585387092 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff80b58c0 con 0x7f8ff815e160 2023-11-26T07:13:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.109+0000 7f8ffe4c8700 1 --2- 172.21.15.83:0/2585387092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ff815e160 0x7f8ff80b5da0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009840 tx=0x7f8fe8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cbfea7564753bca server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.109+0000 7f8ffd4c6700 1 -- 172.21.15.83:0/2585387092 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8011830 con 0x7f8ff815e160 2023-11-26T07:13:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.109+0000 7f8ffd4c6700 1 -- 172.21.15.83:0/2585387092 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe8011990 con 0x7f8ff815e160 2023-11-26T07:13:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.109+0000 7f8ffd4c6700 1 -- 172.21.15.83:0/2585387092 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8011d20 con 0x7f8ff815e160 2023-11-26T07:13:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.110+0000 7f900072c700 1 -- 172.21.15.83:0/2585387092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ff815e160 msgr2=0x7f8ff80b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.110+0000 7f900072c700 1 --2- 172.21.15.83:0/2585387092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ff815e160 0x7f8ff80b5da0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009840 tx=0x7f8fe8009b50 comp rx=0 tx=0).stop 2023-11-26T07:13:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.110+0000 7f900072c700 1 -- 172.21.15.83:0/2585387092 shutdown_connections 2023-11-26T07:13:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.110+0000 7f900072c700 1 --2- 172.21.15.83:0/2585387092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ff815e160 0x7f8ff80b5da0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.110+0000 7f900072c700 1 --2- 172.21.15.83:0/2585387092 >> v2:172.21.15.203:3300/0 conn(0x7f8ff80afc90 0x7f8ff8073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.110+0000 7f900072c700 1 -- 172.21.15.83:0/2585387092 >> 172.21.15.83:0/2585387092 conn(0x7f8ff80d0bf0 msgr2=0x7f8ff80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.110+0000 7f900072c700 1 -- 172.21.15.83:0/2585387092 shutdown_connections 2023-11-26T07:13:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.110+0000 7f900072c700 1 -- 172.21.15.83:0/2585387092 wait complete. 2023-11-26T07:13:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.111+0000 7f900072c700 1 Processor -- start 2023-11-26T07:13:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.111+0000 7f900072c700 1 -- start start 2023-11-26T07:13:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f900072c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff80744d0 0x7f8ff814c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f900072c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff80afc90 0x7f8ff80c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f900072c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ff815e160 0x7f8ff814a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f900072c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80c5c70 con 0x7f8ff80afc90 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f900072c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff80c77d0 con 0x7f8ff80744d0 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f900072c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff814cfe0 con 0x7f8ff815e160 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8ffdcc7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff80afc90 0x7f8ff80c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8ffdcc7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff80afc90 0x7f8ff80c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37496/0 (socket says 172.21.15.83:37496) 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8ffdcc7700 1 -- 172.21.15.83:0/2873078581 learned_addr learned my addr 172.21.15.83:0/2873078581 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8ffe4c8700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff80744d0 0x7f8ff814c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8ffecc9700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ff815e160 0x7f8ff814a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8ffdcc7700 1 -- 172.21.15.83:0/2873078581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ff815e160 msgr2=0x7f8ff814a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8ffdcc7700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ff815e160 0x7f8ff814a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8ffdcc7700 1 -- 172.21.15.83:0/2873078581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff80744d0 msgr2=0x7f8ff814c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8ffdcc7700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff80744d0 0x7f8ff814c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8ffdcc7700 1 -- 172.21.15.83:0/2873078581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe80094a0 con 0x7f8ff80afc90 2023-11-26T07:13:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8ffdcc7700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff80afc90 0x7f8ff80c7290 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400e9b0 tx=0x7f8ff400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8feeffd700 1 -- 172.21.15.83:0/2873078581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4009710 con 0x7f8ff80afc90 2023-11-26T07:13:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8feeffd700 1 -- 172.21.15.83:0/2873078581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff400ee50 con 0x7f8ff80afc90 2023-11-26T07:13:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f8feeffd700 1 -- 172.21.15.83:0/2873078581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff400ca50 con 0x7f8ff80afc90 2023-11-26T07:13:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f900072c700 1 -- 172.21.15.83:0/2873078581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff814d2c0 con 0x7f8ff80afc90 2023-11-26T07:13:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.112+0000 7f900072c700 1 -- 172.21.15.83:0/2873078581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff814d880 con 0x7f8ff80afc90 2023-11-26T07:13:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.113+0000 7f900072c700 1 -- 172.21.15.83:0/2873078581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff8068fb0 con 0x7f8ff80afc90 2023-11-26T07:13:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.114+0000 7f8feeffd700 1 -- 172.21.15.83:0/2873078581 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f8ff400cbb0 con 0x7f8ff80afc90 2023-11-26T07:13:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.114+0000 7f8feeffd700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8fdc06b080 0x7f8fdc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.114+0000 7f8ffe4c8700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8fdc06b080 0x7f8fdc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.114+0000 7f8feeffd700 1 -- 172.21.15.83:0/2873078581 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8ff4089910 con 0x7f8ff80afc90 2023-11-26T07:13:13.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.115+0000 7f8ffe4c8700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8fdc06b080 0x7f8fdc06d540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009fd0 tx=0x7f8fe8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.119+0000 7f8feeffd700 1 -- 172.21.15.83:0/2873078581 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ff4054790 con 0x7f8ff80afc90 2023-11-26T07:13:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.308+0000 7f900072c700 1 -- 172.21.15.83:0/2873078581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ff80c5db0 con 0x7f8ff80afc90 2023-11-26T07:13:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.309+0000 7f8feeffd700 1 -- 172.21.15.83:0/2873078581 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8ff4021020 con 0x7f8ff80afc90 2023-11-26T07:13:13.312 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:13:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 -- 172.21.15.83:0/2873078581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8fdc06b080 msgr2=0x7f8fdc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8fdc06b080 0x7f8fdc06d540 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009fd0 tx=0x7f8fe8008030 comp rx=0 tx=0).stop 2023-11-26T07:13:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 -- 172.21.15.83:0/2873078581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff80afc90 msgr2=0x7f8ff80c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff80afc90 0x7f8ff80c7290 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400e9b0 tx=0x7f8ff400ecc0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 -- 172.21.15.83:0/2873078581 shutdown_connections 2023-11-26T07:13:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8fdc06b080 0x7f8fdc06d540 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ff815e160 0x7f8ff814a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ff80744d0 0x7f8ff814c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 --2- 172.21.15.83:0/2873078581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ff80afc90 0x7f8ff80c7290 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 -- 172.21.15.83:0/2873078581 >> 172.21.15.83:0/2873078581 conn(0x7f8ff80d0bf0 msgr2=0x7f8ff806ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 -- 172.21.15.83:0/2873078581 shutdown_connections 2023-11-26T07:13:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.311+0000 7f900072c700 1 -- 172.21.15.83:0/2873078581 wait complete. 2023-11-26T07:13:13.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:13:13.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.446+0000 7f2982712700 1 Processor -- start 2023-11-26T07:13:13.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.446+0000 7f2982712700 1 -- start start 2023-11-26T07:13:13.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.447+0000 7f2982712700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0af520 0x7f297c0afb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.447+0000 7f2982712700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c15bf70 con 0x7f297c0b0070 2023-11-26T07:13:13.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.447+0000 7f2982712700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297c15ee00 con 0x7f297c0746a0 2023-11-26T07:13:13.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.447+0000 7f2982712700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c15e4f0 con 0x7f297c0af520 2023-11-26T07:13:13.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.447+0000 7f297b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f297c0b0070 0x7f297c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57542/0 (socket says 172.21.15.83:57542) 2023-11-26T07:13:13.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.447+0000 7f297b7fe700 1 -- 172.21.15.83:0/917424889 learned_addr learned my addr 172.21.15.83:0/917424889 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.447+0000 7f297bfff700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0af520 0x7f297c0afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.447+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689435655 0 0) 0x7f296c001110 con 0x7f297c0b0070 2023-11-26T07:13:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.447+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29640035b0 con 0x7f297c0b0070 2023-11-26T07:13:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2265738481 0 0) 0x7f29640035b0 con 0x7f297c0b0070 2023-11-26T07:13:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0af520 msgr2=0x7f297c0afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0af520 0x7f297c0afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 >> v1:172.21.15.203:6789/0 conn(0x7f297c0746a0 legacy=0x7f297c06d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0b04f0 con 0x7f297c0b0070 2023-11-26T07:13:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f296c001800 con 0x7f297c0b0070 2023-11-26T07:13:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 >> v1:172.21.15.83:6789/0 conn(0x7f297c0b0070 legacy=0x7f297c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2964003a30 0x7f2964003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0af520 0x7f2964004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2964004d90 0x7f2964005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c15e4f0 con 0x7f2964004d90 2023-11-26T07:13:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296400bac0 con 0x7f2964003a30 2023-11-26T07:13:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296400bc80 con 0x7f297c0af520 2023-11-26T07:13:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f2980caf700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2964004d90 0x7f2964005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297bfff700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2964003a30 0x7f2964003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297b7fe700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0af520 0x7f2964004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f2980caf700 1 -- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0af520 msgr2=0x7f2964004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f2980caf700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0af520 0x7f2964004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f2980caf700 1 -- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2964003a30 msgr2=0x7f2964003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f2980caf700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2964003a30 0x7f2964003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f2980caf700 1 -- 172.21.15.83:0/917424889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29640035b0 con 0x7f2964004d90 2023-11-26T07:13:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297bfff700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2964003a30 0x7f2964003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:13:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297b7fe700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0af520 0x7f2964004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:13:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f2980caf700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2964004d90 0x7f2964005210 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f297c06f9a0 tx=0x7f2964005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49fa397c6de8658 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974001f20 con 0x7f2964004d90 2023-11-26T07:13:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297400ce50 con 0x7f2964004d90 2023-11-26T07:13:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.448+0000 7f297affd700 1 -- 172.21.15.83:0/917424889 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974012660 con 0x7f2964004d90 2023-11-26T07:13:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.449+0000 7f2982712700 1 -- 172.21.15.83:0/917424889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2964004d90 msgr2=0x7f2964005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.449+0000 7f2982712700 1 --2- 172.21.15.83:0/917424889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2964004d90 0x7f2964005210 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f297c06f9a0 tx=0x7f2964005880 comp rx=0 tx=0).stop 2023-11-26T07:13:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.449+0000 7f2980caf700 1 -- 172.21.15.83:0/917424889 reap_dead start 2023-11-26T07:13:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.449+0000 7f2982712700 1 -- 172.21.15.83:0/917424889 shutdown_connections 2023-11-26T07:13:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.449+0000 7f2982712700 1 -- 172.21.15.83:0/917424889 >> 172.21.15.83:0/917424889 conn(0x7f297c0b28f0 msgr2=0x7f297c0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.449+0000 7f2982712700 1 -- 172.21.15.83:0/917424889 shutdown_connections 2023-11-26T07:13:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.449+0000 7f2982712700 1 -- 172.21.15.83:0/917424889 wait complete. 2023-11-26T07:13:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.450+0000 7f2982712700 1 Processor -- start 2023-11-26T07:13:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.450+0000 7f2982712700 1 -- start start 2023-11-26T07:13:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.451+0000 7f2982712700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c1434b0 0x7f297c1625c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.451+0000 7f2982712700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c143890 0x7f297c15d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.451+0000 7f2982712700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15ffc0 0x7f297c15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.451+0000 7f2982712700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c14e8e0 con 0x7f297c15ffc0 2023-11-26T07:13:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.451+0000 7f2982712700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c1604a0 con 0x7f297c1434b0 2023-11-26T07:13:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.451+0000 7f2982712700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c14dfb0 con 0x7f297c143890 2023-11-26T07:13:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.451+0000 7f2980caf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15ffc0 0x7f297c15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.451+0000 7f2980caf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15ffc0 0x7f297c15d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37518/0 (socket says 172.21.15.83:37518) 2023-11-26T07:13:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.451+0000 7f2980caf700 1 -- 172.21.15.83:0/2830434183 learned_addr learned my addr 172.21.15.83:0/2830434183 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f297bfff700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c1434b0 0x7f297c1625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f2980caf700 1 -- 172.21.15.83:0/2830434183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c143890 msgr2=0x7f297c15d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f297b7fe700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c143890 0x7f297c15d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f2980caf700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c143890 0x7f297c15d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f2980caf700 1 -- 172.21.15.83:0/2830434183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c1434b0 msgr2=0x7f297c1625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f2980caf700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c1434b0 0x7f297c1625c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f2980caf700 1 -- 172.21.15.83:0/2830434183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c14e290 con 0x7f297c15ffc0 2023-11-26T07:13:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f297bfff700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c1434b0 0x7f297c1625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f2980caf700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15ffc0 0x7f297c15d8e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2974013000 tx=0x7f2974001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f2978ff9700 1 -- 172.21.15.83:0/2830434183 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974003c00 con 0x7f297c15ffc0 2023-11-26T07:13:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f2978ff9700 1 -- 172.21.15.83:0/2830434183 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2974020070 con 0x7f297c15ffc0 2023-11-26T07:13:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f2982712700 1 -- 172.21.15.83:0/2830434183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c14e570 con 0x7f297c15ffc0 2023-11-26T07:13:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f2978ff9700 1 -- 172.21.15.83:0/2830434183 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29740050c0 con 0x7f297c15ffc0 2023-11-26T07:13:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.452+0000 7f2982712700 1 -- 172.21.15.83:0/2830434183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c14db40 con 0x7f297c15ffc0 2023-11-26T07:13:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.453+0000 7f2982712700 1 -- 172.21.15.83:0/2830434183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c068fb0 con 0x7f297c15ffc0 2023-11-26T07:13:13.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.453+0000 7f2978ff9700 1 -- 172.21.15.83:0/2830434183 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f2974022030 con 0x7f297c15ffc0 2023-11-26T07:13:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.454+0000 7f2978ff9700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806b080 0x7f296806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.454+0000 7f2978ff9700 1 -- 172.21.15.83:0/2830434183 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f297408add0 con 0x7f297c15ffc0 2023-11-26T07:13:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.454+0000 7f297bfff700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806b080 0x7f296806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.454+0000 7f297bfff700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806b080 0x7f296806d540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2970009b00 tx=0x7f2970008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.458+0000 7f2978ff9700 1 -- 172.21.15.83:0/2830434183 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2974055cf0 con 0x7f297c15ffc0 2023-11-26T07:13:13.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.649+0000 7f2982712700 1 -- 172.21.15.83:0/2830434183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f297c14ea20 con 0x7f297c15ffc0 2023-11-26T07:13:13.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.649+0000 7f2978ff9700 1 -- 172.21.15.83:0/2830434183 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f297c14ea20 con 0x7f297c15ffc0 2023-11-26T07:13:13.652 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:13:13.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 -- 172.21.15.83:0/2830434183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806b080 msgr2=0x7f296806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806b080 0x7f296806d540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2970009b00 tx=0x7f2970008040 comp rx=0 tx=0).stop 2023-11-26T07:13:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 -- 172.21.15.83:0/2830434183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15ffc0 msgr2=0x7f297c15d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15ffc0 0x7f297c15d8e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2974013000 tx=0x7f2974001630 comp rx=0 tx=0).stop 2023-11-26T07:13:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 -- 172.21.15.83:0/2830434183 shutdown_connections 2023-11-26T07:13:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15ffc0 0x7f297c15d8e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c1434b0 0x7f297c1625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f296806b080 0x7f296806d540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 --2- 172.21.15.83:0/2830434183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c143890 0x7f297c15d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 -- 172.21.15.83:0/2830434183 >> 172.21.15.83:0/2830434183 conn(0x7f297c0b28f0 msgr2=0x7f297c073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 -- 172.21.15.83:0/2830434183 shutdown_connections 2023-11-26T07:13:13.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.651+0000 7f2982712700 1 -- 172.21.15.83:0/2830434183 wait complete. 2023-11-26T07:13:13.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:13:13.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.784+0000 7f4ebeb9f700 1 Processor -- start 2023-11-26T07:13:13.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.784+0000 7f4ebeb9f700 1 -- start start 2023-11-26T07:13:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.785+0000 7f4ebeb9f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb806c710 0x7f4eb80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.785+0000 7f4ebeb9f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4eb80af4d0 0x7f4eb80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.785+0000 7f4ebeb9f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb80ae430 con 0x7f4eb815e080 2023-11-26T07:13:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.785+0000 7f4ebeb9f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb80af0b0 con 0x7f4eb80af4d0 2023-11-26T07:13:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.785+0000 7f4ebeb9f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80b05a0 con 0x7f4eb806c710 2023-11-26T07:13:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.785+0000 7f4eb7fff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb806c710 0x7f4eb80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.785+0000 7f4ebd13c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4eb80af4d0 0x7f4eb80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.785+0000 7f4ebc93b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4eb815e080 0x7f4eb806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:42180/0 (socket says 172.21.15.83:42180) 2023-11-26T07:13:13.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4ebc93b700 1 -- 172.21.15.83:0/3096883381 learned_addr learned my addr 172.21.15.83:0/3096883381 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4eb77fe700 1 -- 172.21.15.83:0/3096883381 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115959476 0 0) 0x7f4eb80ae430 con 0x7f4eb815e080 2023-11-26T07:13:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4eb77fe700 1 -- 172.21.15.83:0/3096883381 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea80035b0 con 0x7f4eb815e080 2023-11-26T07:13:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4ebd13c700 1 -- 172.21.15.83:0/3096883381 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb806c710 msgr2=0x7f4eb80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4ebd13c700 1 --2- 172.21.15.83:0/3096883381 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb806c710 0x7f4eb80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4ebd13c700 1 -- 172.21.15.83:0/3096883381 >> v1:172.21.15.203:6789/0 conn(0x7f4eb815e080 legacy=0x7f4eb806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4ebd13c700 1 -- 172.21.15.83:0/3096883381 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb80ae710 con 0x7f4eb80af4d0 2023-11-26T07:13:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4eb7fff700 1 --2- 172.21.15.83:0/3096883381 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb806c710 0x7f4eb80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:13:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4ebd13c700 1 --2- 172.21.15.83:0/3096883381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb80af4d0 0x7f4eb80afb90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009dc0 tx=0x7f4eb000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86eed43437a248fb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4eb77fe700 1 -- 172.21.15.83:0/3096883381 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000e070 con 0x7f4eb80af4d0 2023-11-26T07:13:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4eb77fe700 1 -- 172.21.15.83:0/3096883381 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb000bae0 con 0x7f4eb80af4d0 2023-11-26T07:13:13.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.786+0000 7f4eb77fe700 1 -- 172.21.15.83:0/3096883381 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000be90 con 0x7f4eb80af4d0 2023-11-26T07:13:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.787+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/3096883381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb80af4d0 msgr2=0x7f4eb80afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.787+0000 7f4ebeb9f700 1 --2- 172.21.15.83:0/3096883381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb80af4d0 0x7f4eb80afb90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009dc0 tx=0x7f4eb000b330 comp rx=0 tx=0).stop 2023-11-26T07:13:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.787+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/3096883381 shutdown_connections 2023-11-26T07:13:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.787+0000 7f4ebeb9f700 1 --2- 172.21.15.83:0/3096883381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb80af4d0 0x7f4eb80afb90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.787+0000 7f4ebeb9f700 1 --2- 172.21.15.83:0/3096883381 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb806c710 0x7f4eb80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.787+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/3096883381 >> 172.21.15.83:0/3096883381 conn(0x7f4eb8071b90 msgr2=0x7f4eb8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.787+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/3096883381 shutdown_connections 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.787+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/3096883381 wait complete. 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.788+0000 7f4ebeb9f700 1 Processor -- start 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.788+0000 7f4ebeb9f700 1 -- start start 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebeb9f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb806c710 0x7f4eb8151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebeb9f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb80af4d0 0x7f4eb8153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebeb9f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4eb815e080 0x7f4eb81544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebeb9f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb8161350 con 0x7f4eb815e080 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebeb9f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb8160ab0 con 0x7f4eb80af4d0 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebeb9f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb8160d90 con 0x7f4eb806c710 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebd13c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4eb815e080 0x7f4eb81544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebd13c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4eb815e080 0x7f4eb81544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37532/0 (socket says 172.21.15.83:37532) 2023-11-26T07:13:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebd13c700 1 -- 172.21.15.83:0/836956420 learned_addr learned my addr 172.21.15.83:0/836956420 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebc93b700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb806c710 0x7f4eb8151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4eb7fff700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb80af4d0 0x7f4eb8153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebd13c700 1 -- 172.21.15.83:0/836956420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb806c710 msgr2=0x7f4eb8151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebd13c700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb806c710 0x7f4eb8151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebd13c700 1 -- 172.21.15.83:0/836956420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb80af4d0 msgr2=0x7f4eb8153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebd13c700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb80af4d0 0x7f4eb8153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.789+0000 7f4ebd13c700 1 -- 172.21.15.83:0/836956420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40094a0 con 0x7f4eb815e080 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.790+0000 7f4ebd13c700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4eb815e080 0x7f4eb81544d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b370 tx=0x7f4eb0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.790+0000 7f4eb57fa700 1 -- 172.21.15.83:0/836956420 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0004680 con 0x7f4eb815e080 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.790+0000 7f4eb57fa700 1 -- 172.21.15.83:0/836956420 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb0020070 con 0x7f4eb815e080 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.790+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/836956420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb0009a70 con 0x7f4eb815e080 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.790+0000 7f4eb57fa700 1 -- 172.21.15.83:0/836956420 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0027480 con 0x7f4eb815e080 2023-11-26T07:13:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.790+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/836956420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb80ce7c0 con 0x7f4eb815e080 2023-11-26T07:13:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.791+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/836956420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb80753f0 con 0x7f4eb815e080 2023-11-26T07:13:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.791+0000 7f4eb57fa700 1 -- 172.21.15.83:0/836956420 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4eb0012430 con 0x7f4eb815e080 2023-11-26T07:13:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.791+0000 7f4eb57fa700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4e9c06b0b0 0x7f4e9c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.792+0000 7f4eb57fa700 1 -- 172.21.15.83:0/836956420 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4eb0022070 con 0x7f4eb815e080 2023-11-26T07:13:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.792+0000 7f4ebc93b700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4e9c06b0b0 0x7f4e9c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.792+0000 7f4ebc93b700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4e9c06b0b0 0x7f4e9c06d570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4eac0011e0 tx=0x7f4eac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:13.796+0000 7f4eb57fa700 1 -- 172.21.15.83:0/836956420 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4eb00565c0 con 0x7f4eb815e080 2023-11-26T07:13:14.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.001+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/836956420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4eb8068670 con 0x7f4eb815e080 2023-11-26T07:13:14.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.001+0000 7f4eb57fa700 1 -- 172.21.15.83:0/836956420 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4eb0059be0 con 0x7f4eb815e080 2023-11-26T07:13:14.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/836956420 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4e9c06b0b0 msgr2=0x7f4e9c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4e9c06b0b0 0x7f4e9c06d570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4eac0011e0 tx=0x7f4eac009040 comp rx=0 tx=0).stop 2023-11-26T07:13:14.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/836956420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4eb815e080 msgr2=0x7f4eb81544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4eb815e080 0x7f4eb81544d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b370 tx=0x7f4eb0004ad0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/836956420 shutdown_connections 2023-11-26T07:13:14.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4eb815e080 0x7f4eb81544d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4e9c06b0b0 0x7f4e9c06d570 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eb806c710 0x7f4eb8151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 --2- 172.21.15.83:0/836956420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eb80af4d0 0x7f4eb8153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/836956420 >> 172.21.15.83:0/836956420 conn(0x7f4eb8071b90 msgr2=0x7f4eb80b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:14.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/836956420 shutdown_connections 2023-11-26T07:13:14.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.004+0000 7f4ebeb9f700 1 -- 172.21.15.83:0/836956420 wait complete. 2023-11-26T07:13:14.008 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:13:14.018 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:13:14.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:14.019 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:13:14.069 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:13:14.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:13:14.069 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:13:14.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:14.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.206+0000 7fefa3d9a700 1 Processor -- start 2023-11-26T07:13:14.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.206+0000 7fefa3d9a700 1 -- start start 2023-11-26T07:13:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa3d9a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef9c152cc0 0x7fef9c06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa3d9a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fef9c0b49a0 0x7fef9c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa3d9a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9c15c4b0 con 0x7fef9c0cff20 2023-11-26T07:13:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa3d9a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c15c790 con 0x7fef9c152cc0 2023-11-26T07:13:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa3d9a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c15ca70 con 0x7fef9c0b49a0 2023-11-26T07:13:14.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa1b36700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef9c152cc0 0x7fef9c06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa2337700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fef9c0cff20 0x7fef9c0d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57580/0 (socket says 172.21.15.83:57580) 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa2337700 1 -- 172.21.15.83:0/508425502 learned_addr learned my addr 172.21.15.83:0/508425502 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa1335700 1 --2- 172.21.15.83:0/508425502 >> v2:172.21.15.203:3300/0 conn(0x7fef9c0b49a0 0x7fef9c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa1b36700 1 -- 172.21.15.83:0/508425502 >> v2:172.21.15.203:3300/0 conn(0x7fef9c0b49a0 msgr2=0x7fef9c06c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa1b36700 1 --2- 172.21.15.83:0/508425502 >> v2:172.21.15.203:3300/0 conn(0x7fef9c0b49a0 0x7fef9c06c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa1b36700 1 -- 172.21.15.83:0/508425502 >> v1:172.21.15.83:6789/0 conn(0x7fef9c0cff20 legacy=0x7fef9c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa1b36700 1 -- 172.21.15.83:0/508425502 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c15cd50 con 0x7fef9c152cc0 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.207+0000 7fefa0b34700 1 -- 172.21.15.83:0/508425502 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183013250 0 0) 0x7fef9c15c4b0 con 0x7fef9c0cff20 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa1b36700 1 --2- 172.21.15.83:0/508425502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c152cc0 0x7fef9c06c000 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0b4e50 tx=0x7fef8c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e6dafa04ca93d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa0b34700 1 -- 172.21.15.83:0/508425502 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c011830 con 0x7fef9c152cc0 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa0b34700 1 -- 172.21.15.83:0/508425502 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c011990 con 0x7fef9c152cc0 2023-11-26T07:13:14.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa0b34700 1 -- 172.21.15.83:0/508425502 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c011d20 con 0x7fef9c152cc0 2023-11-26T07:13:14.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa3d9a700 1 -- 172.21.15.83:0/508425502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c152cc0 msgr2=0x7fef9c06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa3d9a700 1 --2- 172.21.15.83:0/508425502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c152cc0 0x7fef9c06c000 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0b4e50 tx=0x7fef8c009b20 comp rx=0 tx=0).stop 2023-11-26T07:13:14.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa3d9a700 1 -- 172.21.15.83:0/508425502 shutdown_connections 2023-11-26T07:13:14.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa3d9a700 1 --2- 172.21.15.83:0/508425502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c152cc0 0x7fef9c06c000 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa3d9a700 1 --2- 172.21.15.83:0/508425502 >> v2:172.21.15.203:3300/0 conn(0x7fef9c0b49a0 0x7fef9c06c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa3d9a700 1 -- 172.21.15.83:0/508425502 >> 172.21.15.83:0/508425502 conn(0x7fef9c071b90 msgr2=0x7fef9c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:14.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa3d9a700 1 -- 172.21.15.83:0/508425502 shutdown_connections 2023-11-26T07:13:14.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.208+0000 7fefa3d9a700 1 -- 172.21.15.83:0/508425502 wait complete. 2023-11-26T07:13:14.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.210+0000 7fefa3d9a700 1 Processor -- start 2023-11-26T07:13:14.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.210+0000 7fefa3d9a700 1 -- start start 2023-11-26T07:13:14.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.210+0000 7fefa3d9a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c0b49a0 0x7fef9c15e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.210+0000 7fefa3d9a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef9c0cff20 0x7fef9c155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.210+0000 7fefa3d9a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c152cc0 0x7fef9c0b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.210+0000 7fefa3d9a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c15d400 con 0x7fef9c152cc0 2023-11-26T07:13:14.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.210+0000 7fefa3d9a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c1f25b0 con 0x7fef9c0b49a0 2023-11-26T07:13:14.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.210+0000 7fefa3d9a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c15d610 con 0x7fef9c0cff20 2023-11-26T07:13:14.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.210+0000 7fefa2337700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c152cc0 0x7fef9c0b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:14.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa2337700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c152cc0 0x7fef9c0b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37546/0 (socket says 172.21.15.83:37546) 2023-11-26T07:13:14.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa2337700 1 -- 172.21.15.83:0/2054331872 learned_addr learned my addr 172.21.15.83:0/2054331872 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:14.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa1b36700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c0b49a0 0x7fef9c15e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:14.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa1335700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef9c0cff20 0x7fef9c155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:14.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa2337700 1 -- 172.21.15.83:0/2054331872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef9c0cff20 msgr2=0x7fef9c155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa2337700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef9c0cff20 0x7fef9c155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa2337700 1 -- 172.21.15.83:0/2054331872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c0b49a0 msgr2=0x7fef9c15e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa2337700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c0b49a0 0x7fef9c15e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa2337700 1 -- 172.21.15.83:0/2054331872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0094a0 con 0x7fef9c152cc0 2023-11-26T07:13:14.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa1b36700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c0b49a0 0x7fef9c15e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:13:14.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa2337700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c152cc0 0x7fef9c0b16c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fef94002230 tx=0x7fef9400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:14.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fef927fc700 1 -- 172.21.15.83:0/2054331872 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9400ac20 con 0x7fef9c152cc0 2023-11-26T07:13:14.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fef927fc700 1 -- 172.21.15.83:0/2054331872 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9400ada0 con 0x7fef9c152cc0 2023-11-26T07:13:14.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa3d9a700 1 -- 172.21.15.83:0/2054331872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c15d8f0 con 0x7fef9c152cc0 2023-11-26T07:13:14.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fef927fc700 1 -- 172.21.15.83:0/2054331872 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94007310 con 0x7fef9c152cc0 2023-11-26T07:13:14.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.211+0000 7fefa3d9a700 1 -- 172.21.15.83:0/2054331872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c1560d0 con 0x7fef9c152cc0 2023-11-26T07:13:14.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.212+0000 7fefa3d9a700 1 -- 172.21.15.83:0/2054331872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c0753f0 con 0x7fef9c152cc0 2023-11-26T07:13:14.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.212+0000 7fef927fc700 1 -- 172.21.15.83:0/2054331872 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fef94007470 con 0x7fef9c152cc0 2023-11-26T07:13:14.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.213+0000 7fef927fc700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef8006b080 0x7fef8006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.213+0000 7fef927fc700 1 -- 172.21.15.83:0/2054331872 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fef94089790 con 0x7fef9c152cc0 2023-11-26T07:13:14.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.213+0000 7fefa1b36700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef8006b080 0x7fef8006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:14.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.213+0000 7fefa1b36700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef8006b080 0x7fef8006d540 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0090b0 tx=0x7fef8c01c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:14.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.217+0000 7fef927fc700 1 -- 172.21.15.83:0/2054331872 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef94054610 con 0x7fef9c152cc0 2023-11-26T07:13:14.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.452+0000 7fefa3d9a700 1 -- 172.21.15.83:0/2054331872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef9c067f60 con 0x7fef9c152cc0 2023-11-26T07:13:14.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.453+0000 7fef927fc700 1 -- 172.21.15.83:0/2054331872 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fef94018090 con 0x7fef9c152cc0 2023-11-26T07:13:14.456 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:14.456 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:13:14.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 -- 172.21.15.83:0/2054331872 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef8006b080 msgr2=0x7fef8006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef8006b080 0x7fef8006d540 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0090b0 tx=0x7fef8c01c040 comp rx=0 tx=0).stop 2023-11-26T07:13:14.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 -- 172.21.15.83:0/2054331872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c152cc0 msgr2=0x7fef9c0b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c152cc0 0x7fef9c0b16c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fef94002230 tx=0x7fef9400a060 comp rx=0 tx=0).stop 2023-11-26T07:13:14.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 -- 172.21.15.83:0/2054331872 shutdown_connections 2023-11-26T07:13:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c152cc0 0x7fef9c0b16c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c0b49a0 0x7fef9c15e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fef8006b080 0x7fef8006d540 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 --2- 172.21.15.83:0/2054331872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef9c0cff20 0x7fef9c155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 -- 172.21.15.83:0/2054331872 >> 172.21.15.83:0/2054331872 conn(0x7fef9c071b90 msgr2=0x7fef9c150e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 -- 172.21.15.83:0/2054331872 shutdown_connections 2023-11-26T07:13:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.455+0000 7fefa3d9a700 1 -- 172.21.15.83:0/2054331872 wait complete. 2023-11-26T07:13:14.460 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:13:14.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:13:14.612 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2023-11-26T07:13:14.612 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-11-26T07:13:14.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol33240 2023-11-26T07:13:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.733+0000 7f1835b5c700 1 Processor -- start 2023-11-26T07:13:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.734+0000 7f1835b5c700 1 -- start start 2023-11-26T07:13:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.734+0000 7f1835b5c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18300cf610 0x7f18300add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.734+0000 7f1835b5c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f183015f9e0 0x7f18300b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.734+0000 7f1835b5c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183006b080 con 0x7f1830073f90 2023-11-26T07:13:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.734+0000 7f1835b5c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183015c290 con 0x7f18300cf610 2023-11-26T07:13:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.734+0000 7f1835b5c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18300cf9f0 con 0x7f183015f9e0 2023-11-26T07:13:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.734+0000 7f182f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18300cf610 0x7f18300add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.734+0000 7f182ffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1830073f90 0x7f183006b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57590/0 (socket says 172.21.15.83:57590) 2023-11-26T07:13:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.735+0000 7f182ffff700 1 -- 172.21.15.83:0/3004483356 learned_addr learned my addr 172.21.15.83:0/3004483356 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.735+0000 7f182effd700 1 --2- 172.21.15.83:0/3004483356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f183015f9e0 0x7f18300b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.735+0000 7f182f7fe700 1 -- 172.21.15.83:0/3004483356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f183015f9e0 msgr2=0x7f18300b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.735+0000 7f182f7fe700 1 --2- 172.21.15.83:0/3004483356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f183015f9e0 0x7f18300b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.735+0000 7f182f7fe700 1 -- 172.21.15.83:0/3004483356 >> v1:172.21.15.83:6789/0 conn(0x7f1830073f90 legacy=0x7f183006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.735+0000 7f182f7fe700 1 -- 172.21.15.83:0/3004483356 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183015ff80 con 0x7f18300cf610 2023-11-26T07:13:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.735+0000 7f182f7fe700 1 --2- 172.21.15.83:0/3004483356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18300cf610 0x7f18300add40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1824009840 tx=0x7f1824009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d23812ff7614431 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.735+0000 7f182e7fc700 1 -- 172.21.15.83:0/3004483356 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824011830 con 0x7f18300cf610 2023-11-26T07:13:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.735+0000 7f182e7fc700 1 -- 172.21.15.83:0/3004483356 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1824011990 con 0x7f18300cf610 2023-11-26T07:13:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.735+0000 7f182e7fc700 1 -- 172.21.15.83:0/3004483356 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824011d20 con 0x7f18300cf610 2023-11-26T07:13:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.736+0000 7f1835b5c700 1 -- 172.21.15.83:0/3004483356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18300cf610 msgr2=0x7f18300add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.736+0000 7f1835b5c700 1 --2- 172.21.15.83:0/3004483356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18300cf610 0x7f18300add40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1824009840 tx=0x7f1824009b50 comp rx=0 tx=0).stop 2023-11-26T07:13:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.736+0000 7f1835b5c700 1 -- 172.21.15.83:0/3004483356 shutdown_connections 2023-11-26T07:13:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.736+0000 7f1835b5c700 1 --2- 172.21.15.83:0/3004483356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18300cf610 0x7f18300add40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.736+0000 7f1835b5c700 1 --2- 172.21.15.83:0/3004483356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f183015f9e0 0x7f18300b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.736+0000 7f1835b5c700 1 -- 172.21.15.83:0/3004483356 >> 172.21.15.83:0/3004483356 conn(0x7f18300d0bf0 msgr2=0x7f18300d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.736+0000 7f1835b5c700 1 -- 172.21.15.83:0/3004483356 shutdown_connections 2023-11-26T07:13:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.736+0000 7f1835b5c700 1 -- 172.21.15.83:0/3004483356 wait complete. 2023-11-26T07:13:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.737+0000 7f1835b5c700 1 Processor -- start 2023-11-26T07:13:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.737+0000 7f1835b5c700 1 -- start start 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f1835b5c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1830073f90 0x7f1830140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f1835b5c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18300cf610 0x7f183015e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f1835b5c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f183015f9e0 0x7f183015e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f1835b5c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183015edd0 con 0x7f1830073f90 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f1835b5c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830154920 con 0x7f18300cf610 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f1835b5c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1830154b90 con 0x7f183015f9e0 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f182f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1830073f90 0x7f1830140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f182f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1830073f90 0x7f1830140490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37556/0 (socket says 172.21.15.83:37556) 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f182f7fe700 1 -- 172.21.15.83:0/4141070404 learned_addr learned my addr 172.21.15.83:0/4141070404 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f182f7fe700 1 -- 172.21.15.83:0/4141070404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f183015f9e0 msgr2=0x7f183015e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f182f7fe700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f183015f9e0 0x7f183015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f182f7fe700 1 -- 172.21.15.83:0/4141070404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18300cf610 msgr2=0x7f183015e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f182f7fe700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18300cf610 0x7f183015e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f182f7fe700 1 -- 172.21.15.83:0/4141070404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18240094a0 con 0x7f1830073f90 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f182f7fe700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1830073f90 0x7f1830140490 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1824009b90 tx=0x7f1824004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f1817fff700 1 -- 172.21.15.83:0/4141070404 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240036a0 con 0x7f1830073f90 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f1817fff700 1 -- 172.21.15.83:0/4141070404 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182401f070 con 0x7f1830073f90 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f1835b5c700 1 -- 172.21.15.83:0/4141070404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1830154e70 con 0x7f1830073f90 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.738+0000 7f1817fff700 1 -- 172.21.15.83:0/4141070404 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824003a20 con 0x7f1830073f90 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.739+0000 7f1835b5c700 1 -- 172.21.15.83:0/4141070404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1830151610 con 0x7f1830073f90 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.739+0000 7f1835b5c700 1 -- 172.21.15.83:0/4141070404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1830068fb0 con 0x7f1830073f90 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.740+0000 7f1817fff700 1 -- 172.21.15.83:0/4141070404 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f1824003b80 con 0x7f1830073f90 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.740+0000 7f1817fff700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f181c06b080 0x7f181c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.740+0000 7f1817fff700 1 -- 172.21.15.83:0/4141070404 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1824021070 con 0x7f1830073f90 2023-11-26T07:13:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.740+0000 7f182effd700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f181c06b080 0x7f181c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.741+0000 7f182effd700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f181c06b080 0x7f181c06d540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f18301562f0 tx=0x7f1820008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.745+0000 7f1817fff700 1 -- 172.21.15.83:0/4141070404 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1824056170 con 0x7f1830073f90 2023-11-26T07:13:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:14.967+0000 7f1835b5c700 1 -- 172.21.15.83:0/4141070404 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol33240", "target": ["mon-mgr", ""]}) v1 -- 0x7f1830160820 con 0x7f181c06b080 2023-11-26T07:13:14.974 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:13:14.972+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:13:14.974 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:13:14.972+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:13:14.974 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:13:14.972+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:13:14.974 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:13:14.972+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:13:14.974 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:13:14.972+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:13:15.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.000+0000 7f1817fff700 1 -- 172.21.15.83:0/4141070404 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1820009480 con 0x7f181c06b080 2023-11-26T07:13:15.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.002+0000 7f1835b5c700 1 -- 172.21.15.83:0/4141070404 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f181c06b080 msgr2=0x7f181c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.002+0000 7f1835b5c700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f181c06b080 0x7f181c06d540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f18301562f0 tx=0x7f1820008040 comp rx=0 tx=0).stop 2023-11-26T07:13:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.002+0000 7f1835b5c700 1 -- 172.21.15.83:0/4141070404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1830073f90 msgr2=0x7f1830140490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.002+0000 7f1835b5c700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1830073f90 0x7f1830140490 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1824009b90 tx=0x7f1824004750 comp rx=0 tx=0).stop 2023-11-26T07:13:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.002+0000 7f1835b5c700 1 -- 172.21.15.83:0/4141070404 shutdown_connections 2023-11-26T07:13:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.002+0000 7f1835b5c700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f181c06b080 0x7f181c06d540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.002+0000 7f1835b5c700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f183015f9e0 0x7f183015e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.002+0000 7f1835b5c700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1830073f90 0x7f1830140490 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.002+0000 7f1835b5c700 1 --2- 172.21.15.83:0/4141070404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18300cf610 0x7f183015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.002+0000 7f1835b5c700 1 -- 172.21.15.83:0/4141070404 >> 172.21.15.83:0/4141070404 conn(0x7f18300d0bf0 msgr2=0x7f18300af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.003+0000 7f1835b5c700 1 -- 172.21.15.83:0/4141070404 shutdown_connections 2023-11-26T07:13:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.003+0000 7f1835b5c700 1 -- 172.21.15.83:0/4141070404 wait complete. 2023-11-26T07:13:15.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:15.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.134+0000 7f370fafc700 1 Processor -- start 2023-11-26T07:13:15.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.134+0000 7f370fafc700 1 -- start start 2023-11-26T07:13:15.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370fafc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f370815e080 0x7f37080b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370fafc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37080af4d0 0x7f37080afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370fafc700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37080ae430 con 0x7f370806c000 2023-11-26T07:13:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370fafc700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37080af0b0 con 0x7f370815e080 2023-11-26T07:13:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370fafc700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37080b05a0 con 0x7f37080af4d0 2023-11-26T07:13:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370d898700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f370815e080 0x7f37080b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370e099700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f370806c000 0x7f3708073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57622/0 (socket says 172.21.15.83:57622) 2023-11-26T07:13:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370e099700 1 -- 172.21.15.83:0/1920793406 learned_addr learned my addr 172.21.15.83:0/1920793406 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370d898700 1 -- 172.21.15.83:0/1920793406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37080af4d0 msgr2=0x7f37080afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370d898700 1 --2- 172.21.15.83:0/1920793406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37080af4d0 0x7f37080afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370d898700 1 -- 172.21.15.83:0/1920793406 >> v1:172.21.15.83:6789/0 conn(0x7f370806c000 legacy=0x7f3708073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370d898700 1 -- 172.21.15.83:0/1920793406 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37080ae710 con 0x7f370815e080 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370c896700 1 -- 172.21.15.83:0/1920793406 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901710615 0 0) 0x7f37080ae430 con 0x7f370806c000 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370d898700 1 --2- 172.21.15.83:0/1920793406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f370815e080 0x7f37080b5050 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3704009870 tx=0x7f3704009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e91e5385e798306 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.135+0000 7f370c896700 1 -- 172.21.15.83:0/1920793406 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704011830 con 0x7f370815e080 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.136+0000 7f370c896700 1 -- 172.21.15.83:0/1920793406 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3704011990 con 0x7f370815e080 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.136+0000 7f370c896700 1 -- 172.21.15.83:0/1920793406 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704011d20 con 0x7f370815e080 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.136+0000 7f370fafc700 1 -- 172.21.15.83:0/1920793406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f370815e080 msgr2=0x7f37080b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.136+0000 7f370fafc700 1 --2- 172.21.15.83:0/1920793406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f370815e080 0x7f37080b5050 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3704009870 tx=0x7f3704009b80 comp rx=0 tx=0).stop 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.136+0000 7f370fafc700 1 -- 172.21.15.83:0/1920793406 shutdown_connections 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.136+0000 7f370fafc700 1 --2- 172.21.15.83:0/1920793406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f370815e080 0x7f37080b5050 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.136+0000 7f370fafc700 1 --2- 172.21.15.83:0/1920793406 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37080af4d0 0x7f37080afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.136+0000 7f370fafc700 1 -- 172.21.15.83:0/1920793406 >> 172.21.15.83:0/1920793406 conn(0x7f3708071b90 msgr2=0x7f3708071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.136+0000 7f370fafc700 1 -- 172.21.15.83:0/1920793406 shutdown_connections 2023-11-26T07:13:15.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.136+0000 7f370fafc700 1 -- 172.21.15.83:0/1920793406 wait complete. 2023-11-26T07:13:15.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.137+0000 7f370fafc700 1 Processor -- start 2023-11-26T07:13:15.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.137+0000 7f370fafc700 1 -- start start 2023-11-26T07:13:15.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370fafc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f370806c000 0x7f3708147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370fafc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37080af4d0 0x7f3708159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370fafc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f370815e080 0x7f3708159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370fafc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37080d02f0 con 0x7f37080af4d0 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370fafc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3708158f50 con 0x7f370815e080 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370fafc700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37081593e0 con 0x7f370806c000 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370d097700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37080af4d0 0x7f3708159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370d097700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37080af4d0 0x7f3708159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37586/0 (socket says 172.21.15.83:37586) 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370d097700 1 -- 172.21.15.83:0/260893531 learned_addr learned my addr 172.21.15.83:0/260893531 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370d898700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f370806c000 0x7f3708147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370e099700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f370815e080 0x7f3708159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370d097700 1 -- 172.21.15.83:0/260893531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f370806c000 msgr2=0x7f3708147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370d097700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f370806c000 0x7f3708147480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370d097700 1 -- 172.21.15.83:0/260893531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f370815e080 msgr2=0x7f3708159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370d097700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f370815e080 0x7f3708159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370d097700 1 -- 172.21.15.83:0/260893531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37040094a0 con 0x7f37080af4d0 2023-11-26T07:13:15.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370d898700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f370806c000 0x7f3708147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:15.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.138+0000 7f370e099700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f370815e080 0x7f3708159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:15.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.139+0000 7f370d097700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37080af4d0 0x7f3708159870 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f36f800b500 tx=0x7f36f800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:15.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.139+0000 7f36fe7fc700 1 -- 172.21.15.83:0/260893531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8011830 con 0x7f37080af4d0 2023-11-26T07:13:15.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.139+0000 7f36fe7fc700 1 -- 172.21.15.83:0/260893531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f80119b0 con 0x7f37080af4d0 2023-11-26T07:13:15.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.139+0000 7f370fafc700 1 -- 172.21.15.83:0/260893531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3708158340 con 0x7f37080af4d0 2023-11-26T07:13:15.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.139+0000 7f36fe7fc700 1 -- 172.21.15.83:0/260893531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8011d60 con 0x7f37080af4d0 2023-11-26T07:13:15.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.139+0000 7f370fafc700 1 -- 172.21.15.83:0/260893531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3708158900 con 0x7f37080af4d0 2023-11-26T07:13:15.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.140+0000 7f370fafc700 1 -- 172.21.15.83:0/260893531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37080753f0 con 0x7f37080af4d0 2023-11-26T07:13:15.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.140+0000 7f36fe7fc700 1 -- 172.21.15.83:0/260893531 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f36f801a400 con 0x7f37080af4d0 2023-11-26T07:13:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.141+0000 7f36fe7fc700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36ec06b080 0x7f36ec06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.141+0000 7f370d898700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36ec06b080 0x7f36ec06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.141+0000 7f36fe7fc700 1 -- 172.21.15.83:0/260893531 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f36f80899a0 con 0x7f37080af4d0 2023-11-26T07:13:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.141+0000 7f370d898700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36ec06b080 0x7f36ec06d540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f37040090b0 tx=0x7f3704008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:15.147 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.145+0000 7f36fe7fc700 1 -- 172.21.15.83:0/260893531 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36f8054820 con 0x7f37080af4d0 2023-11-26T07:13:15.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.380+0000 7f370fafc700 1 -- 172.21.15.83:0/260893531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37080b00d0 con 0x7f37080af4d0 2023-11-26T07:13:15.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.380+0000 7f36fe7fc700 1 -- 172.21.15.83:0/260893531 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f36f8015020 con 0x7f37080af4d0 2023-11-26T07:13:15.383 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:15.383 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:13:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 -- 172.21.15.83:0/260893531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36ec06b080 msgr2=0x7f36ec06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36ec06b080 0x7f36ec06d540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f37040090b0 tx=0x7f3704008030 comp rx=0 tx=0).stop 2023-11-26T07:13:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 -- 172.21.15.83:0/260893531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37080af4d0 msgr2=0x7f3708159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37080af4d0 0x7f3708159870 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f36f800b500 tx=0x7f36f800b8c0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 -- 172.21.15.83:0/260893531 shutdown_connections 2023-11-26T07:13:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f370815e080 0x7f3708159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36ec06b080 0x7f36ec06d540 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f370806c000 0x7f3708147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 --2- 172.21.15.83:0/260893531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37080af4d0 0x7f3708159870 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 -- 172.21.15.83:0/260893531 >> 172.21.15.83:0/260893531 conn(0x7f3708071b90 msgr2=0x7f37080b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 -- 172.21.15.83:0/260893531 shutdown_connections 2023-11-26T07:13:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.382+0000 7f370fafc700 1 -- 172.21.15.83:0/260893531 wait complete. 2023-11-26T07:13:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:13:15.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:13:15.537 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.55098684499999995,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.972518378952053,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:13:15.538 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.972518378952053 }, "state": "open", "uptime": 0.550986845 } ] 2023-11-26T07:13:15.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.661+0000 7f36f34fd700 1 Processor -- start 2023-11-26T07:13:15.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.662+0000 7f36f34fd700 1 -- start start 2023-11-26T07:13:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.662+0000 7f36f34fd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec06c710 0x7f36ec0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36f34fd700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ec0ae430 con 0x7f36ec15e080 2023-11-26T07:13:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36f34fd700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ec0ae710 con 0x7f36ec0af4d0 2023-11-26T07:13:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36f34fd700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0af0b0 con 0x7f36ec06c710 2023-11-26T07:13:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36f1299700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f36ec15e080 0x7f36ec06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57632/0 (socket says 172.21.15.83:57632) 2023-11-26T07:13:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36f1299700 1 -- 172.21.15.83:0/4036343612 learned_addr learned my addr 172.21.15.83:0/4036343612 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867444221 0 0) 0x7f36ec0ae430 con 0x7f36ec15e080 2023-11-26T07:13:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36f0a98700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec06c710 0x7f36ec0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d80035b0 con 0x7f36ec15e080 2023-11-26T07:13:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1520166357 0 0) 0x7f36d80035b0 con 0x7f36ec15e080 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec06c710 msgr2=0x7f36ec0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec06c710 0x7f36ec0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 >> v1:172.21.15.203:6789/0 conn(0x7f36ec0af4d0 legacy=0x7f36ec06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec0ae9f0 con 0x7f36ec15e080 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f36e0001670 con 0x7f36ec15e080 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 >> v1:172.21.15.83:6789/0 conn(0x7f36ec15e080 legacy=0x7f36ec06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36d8003a30 0x7f36d8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec06c710 0x7f36d8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d8004d90 0x7f36d8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0af0b0 con 0x7f36d8004d90 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d800ba10 con 0x7f36d8003a30 2023-11-26T07:13:15.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d800bbd0 con 0x7f36ec06c710 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36f1a9a700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d8004d90 0x7f36d8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.663+0000 7f36f1299700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36d8003a30 0x7f36d8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36f0a98700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec06c710 0x7f36d8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36f1a9a700 1 -- 172.21.15.83:0/4036343612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec06c710 msgr2=0x7f36d8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36f1a9a700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec06c710 0x7f36d8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36f1a9a700 1 -- 172.21.15.83:0/4036343612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36d8003a30 msgr2=0x7f36d8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36f1a9a700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36d8003a30 0x7f36d8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36f1a9a700 1 -- 172.21.15.83:0/4036343612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d80035b0 con 0x7f36d8004d90 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36f1a9a700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d8004d90 0x7f36d8005210 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f36e400c290 tx=0x7f36d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617c93406d5dfcbd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4001e70 con 0x7f36d8004d90 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e400ce50 con 0x7f36d8004d90 2023-11-26T07:13:15.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36dffff700 1 -- 172.21.15.83:0/4036343612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4012660 con 0x7f36d8004d90 2023-11-26T07:13:15.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36f34fd700 1 -- 172.21.15.83:0/4036343612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d8004d90 msgr2=0x7f36d8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36f34fd700 1 --2- 172.21.15.83:0/4036343612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36d8004d90 0x7f36d8005210 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f36e400c290 tx=0x7f36d8005880 comp rx=0 tx=0).stop 2023-11-26T07:13:15.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.664+0000 7f36f1a9a700 1 -- 172.21.15.83:0/4036343612 reap_dead start 2023-11-26T07:13:15.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.665+0000 7f36f34fd700 1 -- 172.21.15.83:0/4036343612 shutdown_connections 2023-11-26T07:13:15.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.665+0000 7f36f34fd700 1 -- 172.21.15.83:0/4036343612 >> 172.21.15.83:0/4036343612 conn(0x7f36ec071b90 msgr2=0x7f36ec071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:15.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.665+0000 7f36f34fd700 1 -- 172.21.15.83:0/4036343612 shutdown_connections 2023-11-26T07:13:15.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.665+0000 7f36f34fd700 1 -- 172.21.15.83:0/4036343612 wait complete. 2023-11-26T07:13:15.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.666+0000 7f36f34fd700 1 Processor -- start 2023-11-26T07:13:15.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.666+0000 7f36f34fd700 1 -- start start 2023-11-26T07:13:15.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.667+0000 7f36f34fd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36ec1f69d0 0x7f36ec1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.667+0000 7f36f34fd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec1f6db0 0x7f36ec1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.667+0000 7f36f34fd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36ec158df0 0x7f36ec159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.667+0000 7f36f34fd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec1597b0 con 0x7f36ec158df0 2023-11-26T07:13:15.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.667+0000 7f36f34fd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec154d10 con 0x7f36ec1f69d0 2023-11-26T07:13:15.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.667+0000 7f36f34fd700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec155af0 con 0x7f36ec1f6db0 2023-11-26T07:13:15.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.667+0000 7f36f1a9a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36ec158df0 0x7f36ec159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.667+0000 7f36f1a9a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36ec158df0 0x7f36ec159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37608/0 (socket says 172.21.15.83:37608) 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.667+0000 7f36f1a9a700 1 -- 172.21.15.83:0/838123375 learned_addr learned my addr 172.21.15.83:0/838123375 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.667+0000 7f36f1299700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36ec1f69d0 0x7f36ec1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.667+0000 7f36f0a98700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec1f6db0 0x7f36ec1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.668+0000 7f36f1a9a700 1 -- 172.21.15.83:0/838123375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec1f6db0 msgr2=0x7f36ec1547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.668+0000 7f36f1a9a700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec1f6db0 0x7f36ec1547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.668+0000 7f36f1a9a700 1 -- 172.21.15.83:0/838123375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36ec1f69d0 msgr2=0x7f36ec1568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.668+0000 7f36f1a9a700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36ec1f69d0 0x7f36ec1568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.668+0000 7f36f1a9a700 1 -- 172.21.15.83:0/838123375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec154f70 con 0x7f36ec158df0 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.668+0000 7f36f1a9a700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36ec158df0 0x7f36ec159270 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f36e4013000 tx=0x7f36e40015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.668+0000 7f36ddffb700 1 -- 172.21.15.83:0/838123375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e4003c00 con 0x7f36ec158df0 2023-11-26T07:13:15.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.668+0000 7f36ddffb700 1 -- 172.21.15.83:0/838123375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e4020070 con 0x7f36ec158df0 2023-11-26T07:13:15.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.668+0000 7f36f34fd700 1 -- 172.21.15.83:0/838123375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec155250 con 0x7f36ec158df0 2023-11-26T07:13:15.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.668+0000 7f36ddffb700 1 -- 172.21.15.83:0/838123375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e40050c0 con 0x7f36ec158df0 2023-11-26T07:13:15.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.668+0000 7f36f34fd700 1 -- 172.21.15.83:0/838123375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec155810 con 0x7f36ec158df0 2023-11-26T07:13:15.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.669+0000 7f36f34fd700 1 -- 172.21.15.83:0/838123375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ec1f3ea0 con 0x7f36ec158df0 2023-11-26T07:13:15.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.670+0000 7f36ddffb700 1 -- 172.21.15.83:0/838123375 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f36e4022030 con 0x7f36ec158df0 2023-11-26T07:13:15.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.670+0000 7f36ddffb700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36d006b110 0x7f36d006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:15.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.670+0000 7f36ddffb700 1 -- 172.21.15.83:0/838123375 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f36e408a590 con 0x7f36ec158df0 2023-11-26T07:13:15.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.670+0000 7f36f1299700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36d006b110 0x7f36d006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:15.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.670+0000 7f36f1299700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36d006b110 0x7f36d006d5d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f36e0000c00 tx=0x7f36e0001c40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:15.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.675+0000 7f36ddffb700 1 -- 172.21.15.83:0/838123375 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36e40554c0 con 0x7f36ec158df0 2023-11-26T07:13:15.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.905+0000 7f36f34fd700 1 -- 172.21.15.83:0/838123375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36ec0753f0 con 0x7f36ec158df0 2023-11-26T07:13:15.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.906+0000 7f36ddffb700 1 -- 172.21.15.83:0/838123375 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f36e4058ae0 con 0x7f36ec158df0 2023-11-26T07:13:15.909 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:15.909 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:13:15.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 -- 172.21.15.83:0/838123375 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36d006b110 msgr2=0x7f36d006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36d006b110 0x7f36d006d5d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f36e0000c00 tx=0x7f36e0001c40 comp rx=0 tx=0).stop 2023-11-26T07:13:15.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 -- 172.21.15.83:0/838123375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36ec158df0 msgr2=0x7f36ec159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:15.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36ec158df0 0x7f36ec159270 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f36e4013000 tx=0x7f36e40015b0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 -- 172.21.15.83:0/838123375 shutdown_connections 2023-11-26T07:13:15.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36ec158df0 0x7f36ec159270 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36ec1f69d0 0x7f36ec1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f36d006b110 0x7f36d006d5d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 --2- 172.21.15.83:0/838123375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36ec1f6db0 0x7f36ec1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:15.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 -- 172.21.15.83:0/838123375 >> 172.21.15.83:0/838123375 conn(0x7f36ec071b90 msgr2=0x7f36ec15d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:15.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 -- 172.21.15.83:0/838123375 shutdown_connections 2023-11-26T07:13:15.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:15.908+0000 7f36f34fd700 1 -- 172.21.15.83:0/838123375 wait complete. 2023-11-26T07:13:15.912 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:13:15.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:13:16.063 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.075974311,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.945882341969401,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:13:16.064 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.9458823419694 }, "state": "open", "uptime": 1.075974311 } ] 2023-11-26T07:13:16.064 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:13:21.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:21.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.196+0000 7f88839e4700 1 Processor -- start 2023-11-26T07:13:21.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.196+0000 7f88839e4700 1 -- start start 2023-11-26T07:13:21.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.197+0000 7f88839e4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f887c06c710 0x7f887c165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:21.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.197+0000 7f88839e4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0af950 con 0x7f887c1446a0 2023-11-26T07:13:21.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.197+0000 7f88839e4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0afc30 con 0x7f887c0adf00 2023-11-26T07:13:21.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.197+0000 7f88839e4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c144a80 con 0x7f887c06c710 2023-11-26T07:13:21.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.197+0000 7f8881f81700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f887c06c710 0x7f887c165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:21.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.197+0000 7f8881780700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f887c1446a0 0x7f887c0b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57652/0 (socket says 172.21.15.83:57652) 2023-11-26T07:13:21.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.197+0000 7f8881780700 1 -- 172.21.15.83:0/1792467520 learned_addr learned my addr 172.21.15.83:0/1792467520 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:21.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.197+0000 7f8881f81700 1 -- 172.21.15.83:0/1792467520 >> v1:172.21.15.203:6789/0 conn(0x7f887c0adf00 legacy=0x7f887c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:21.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.197+0000 7f8881f81700 1 -- 172.21.15.83:0/1792467520 >> v1:172.21.15.83:6789/0 conn(0x7f887c1446a0 legacy=0x7f887c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:21.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.197+0000 7f8881f81700 1 -- 172.21.15.83:0/1792467520 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0aff10 con 0x7f887c06c710 2023-11-26T07:13:21.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.197+0000 7f8873fff700 1 -- 172.21.15.83:0/1792467520 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276386046 0 0) 0x7f887c0af950 con 0x7f887c1446a0 2023-11-26T07:13:21.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.198+0000 7f8881f81700 1 --2- 172.21.15.83:0/1792467520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c06c710 0x7f887c165b70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8874009d90 tx=0x7f887400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8df927118c5d9c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:21.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.198+0000 7f8873fff700 1 -- 172.21.15.83:0/1792467520 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400e070 con 0x7f887c06c710 2023-11-26T07:13:21.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.198+0000 7f8873fff700 1 -- 172.21.15.83:0/1792467520 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887400bb90 con 0x7f887c06c710 2023-11-26T07:13:21.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.198+0000 7f8873fff700 1 -- 172.21.15.83:0/1792467520 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874003a20 con 0x7f887c06c710 2023-11-26T07:13:21.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.198+0000 7f88839e4700 1 -- 172.21.15.83:0/1792467520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c06c710 msgr2=0x7f887c165b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:21.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.198+0000 7f88839e4700 1 --2- 172.21.15.83:0/1792467520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c06c710 0x7f887c165b70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8874009d90 tx=0x7f887400b330 comp rx=0 tx=0).stop 2023-11-26T07:13:21.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.198+0000 7f88839e4700 1 -- 172.21.15.83:0/1792467520 shutdown_connections 2023-11-26T07:13:21.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.198+0000 7f88839e4700 1 --2- 172.21.15.83:0/1792467520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c06c710 0x7f887c165b70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:21.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.198+0000 7f88839e4700 1 -- 172.21.15.83:0/1792467520 >> 172.21.15.83:0/1792467520 conn(0x7f887c071b90 msgr2=0x7f887c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:21.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.198+0000 7f88839e4700 1 -- 172.21.15.83:0/1792467520 shutdown_connections 2023-11-26T07:13:21.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.198+0000 7f88839e4700 1 -- 172.21.15.83:0/1792467520 wait complete. 2023-11-26T07:13:21.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.200+0000 7f88839e4700 1 Processor -- start 2023-11-26T07:13:21.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.200+0000 7f88839e4700 1 -- start start 2023-11-26T07:13:21.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.200+0000 7f88839e4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f887c06c710 0x7f887c138910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:21.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.200+0000 7f88839e4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c0adf00 0x7f887c159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:21.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.200+0000 7f88839e4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f887c1446a0 0x7f887c13a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:21.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.200+0000 7f88839e4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c15ab60 con 0x7f887c0adf00 2023-11-26T07:13:21.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.200+0000 7f88839e4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c138e50 con 0x7f887c06c710 2023-11-26T07:13:21.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.200+0000 7f88839e4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c13a120 con 0x7f887c1446a0 2023-11-26T07:13:21.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.200+0000 7f8880f7f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c0adf00 0x7f887c159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:21.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.200+0000 7f8880f7f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c0adf00 0x7f887c159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37618/0 (socket says 172.21.15.83:37618) 2023-11-26T07:13:21.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.200+0000 7f8880f7f700 1 -- 172.21.15.83:0/896958713 learned_addr learned my addr 172.21.15.83:0/896958713 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:21.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f8881780700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f887c06c710 0x7f887c138910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:21.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f8881f81700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f887c1446a0 0x7f887c13a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:21.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f8880f7f700 1 -- 172.21.15.83:0/896958713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f887c1446a0 msgr2=0x7f887c13a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:21.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f8880f7f700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f887c1446a0 0x7f887c13a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:21.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f8880f7f700 1 -- 172.21.15.83:0/896958713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f887c06c710 msgr2=0x7f887c138910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f8880f7f700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f887c06c710 0x7f887c138910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f8880f7f700 1 -- 172.21.15.83:0/896958713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8874009a70 con 0x7f887c0adf00 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f8880f7f700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c0adf00 0x7f887c159f40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f887800c450 tx=0x7f887800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f8871ffb700 1 -- 172.21.15.83:0/896958713 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8878016070 con 0x7f887c0adf00 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f88839e4700 1 -- 172.21.15.83:0/896958713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c139080 con 0x7f887c0adf00 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f8871ffb700 1 -- 172.21.15.83:0/896958713 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8878006860 con 0x7f887c0adf00 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f8871ffb700 1 -- 172.21.15.83:0/896958713 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8878006bd0 con 0x7f887c0adf00 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.201+0000 7f88839e4700 1 -- 172.21.15.83:0/896958713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c139640 con 0x7f887c0adf00 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.202+0000 7f88839e4700 1 -- 172.21.15.83:0/896958713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c139f70 con 0x7f887c0adf00 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.202+0000 7f8871ffb700 1 -- 172.21.15.83:0/896958713 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8878011590 con 0x7f887c0adf00 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.203+0000 7f8871ffb700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f886006b110 0x7f886006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.203+0000 7f8871ffb700 1 -- 172.21.15.83:0/896958713 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8878088ed0 con 0x7f887c0adf00 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.203+0000 7f8881780700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f886006b110 0x7f886006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:21.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.203+0000 7f8881780700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f886006b110 0x7f886006d5d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f886c000f60 tx=0x7f886c007c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:21.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.207+0000 7f8871ffb700 1 -- 172.21.15.83:0/896958713 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8878053cc0 con 0x7f887c0adf00 2023-11-26T07:13:21.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.440+0000 7f88839e4700 1 -- 172.21.15.83:0/896958713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f887c0753f0 con 0x7f887c0adf00 2023-11-26T07:13:21.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.441+0000 7f8871ffb700 1 -- 172.21.15.83:0/896958713 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f88780572e0 con 0x7f887c0adf00 2023-11-26T07:13:21.444 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:21.444 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:13:21.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 -- 172.21.15.83:0/896958713 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f886006b110 msgr2=0x7f886006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:21.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f886006b110 0x7f886006d5d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f886c000f60 tx=0x7f886c007c90 comp rx=0 tx=0).stop 2023-11-26T07:13:21.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 -- 172.21.15.83:0/896958713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c0adf00 msgr2=0x7f887c159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:21.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c0adf00 0x7f887c159f40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f887800c450 tx=0x7f887800c810 comp rx=0 tx=0).stop 2023-11-26T07:13:21.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 -- 172.21.15.83:0/896958713 shutdown_connections 2023-11-26T07:13:21.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f886006b110 0x7f886006d5d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:21.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f887c1446a0 0x7f887c13a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:21.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f887c06c710 0x7f887c138910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 --2- 172.21.15.83:0/896958713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f887c0adf00 0x7f887c159f40 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 -- 172.21.15.83:0/896958713 >> 172.21.15.83:0/896958713 conn(0x7f887c071b90 msgr2=0x7f887c0b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 -- 172.21.15.83:0/896958713 shutdown_connections 2023-11-26T07:13:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:21.443+0000 7f88839e4700 1 -- 172.21.15.83:0/896958713 wait complete. 2023-11-26T07:13:21.447 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:13:21.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:13:21.598 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":6.6108421689999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.667019574181349,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:13:21.599 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.66701957418135 }, "state": "open", "uptime": 6.610842169 } ] 2023-11-26T07:13:21.599 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:13:26.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.731+0000 7f05c467d700 1 Processor -- start 2023-11-26T07:13:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.731+0000 7f05c467d700 1 -- start start 2023-11-26T07:13:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c467d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05bc152cc0 0x7f05bc06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c467d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc15c4b0 con 0x7f05bc0b49a0 2023-11-26T07:13:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c467d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc15c790 con 0x7f05bc0d0630 2023-11-26T07:13:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c467d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05bc15ca70 con 0x7f05bc152cc0 2023-11-26T07:13:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c1c18700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f05bc0b49a0 0x7f05bc0cff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57666/0 (socket says 172.21.15.83:57666) 2023-11-26T07:13:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c1c18700 1 -- 172.21.15.83:0/2494687970 learned_addr learned my addr 172.21.15.83:0/2494687970 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398868581 0 0) 0x7f05bc15c4b0 con 0x7f05bc0b49a0 2023-11-26T07:13:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a80035b0 con 0x7f05bc0b49a0 2023-11-26T07:13:26.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c2419700 1 --2- 172.21.15.83:0/2494687970 >> v2:172.21.15.203:3300/0 conn(0x7f05bc152cc0 0x7f05bc06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2277825656 0 0) 0x7f05a80035b0 con 0x7f05bc0b49a0 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 >> v2:172.21.15.203:3300/0 conn(0x7f05bc152cc0 msgr2=0x7f05bc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c1417700 1 --2- 172.21.15.83:0/2494687970 >> v2:172.21.15.203:3300/0 conn(0x7f05bc152cc0 0x7f05bc06c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 >> v1:172.21.15.203:6789/0 conn(0x7f05bc0d0630 legacy=0x7f05bc15fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc15cd50 con 0x7f05bc0b49a0 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.732+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f05ac001670 con 0x7f05bc0b49a0 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 >> v1:172.21.15.83:6789/0 conn(0x7f05bc0b49a0 legacy=0x7f05bc0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c1417700 1 --2- 172.21.15.83:0/2494687970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a8003a00 0x7f05a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c1417700 1 --2- 172.21.15.83:0/2494687970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a8004850 0x7f05a8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c1417700 1 --2- 172.21.15.83:0/2494687970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 0x7f05a800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc15ca70 con 0x7f05a8005210 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a800c1b0 con 0x7f05a8003a00 2023-11-26T07:13:26.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a800c370 con 0x7f05a8004850 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c2c1a700 1 --2- 172.21.15.83:0/2494687970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 0x7f05a800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c2419700 1 --2- 172.21.15.83:0/2494687970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a8003a00 0x7f05a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c1c18700 1 --2- 172.21.15.83:0/2494687970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a8004850 0x7f05a8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c2c1a700 1 -- 172.21.15.83:0/2494687970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a8004850 msgr2=0x7f05a8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c2c1a700 1 --2- 172.21.15.83:0/2494687970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05a8004850 0x7f05a8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c2c1a700 1 -- 172.21.15.83:0/2494687970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a8003a00 msgr2=0x7f05a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c2c1a700 1 --2- 172.21.15.83:0/2494687970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05a8003a00 0x7f05a8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c2c1a700 1 -- 172.21.15.83:0/2494687970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80035b0 con 0x7f05a8005210 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c2c1a700 1 -- 172.21.15.83:0/2494687970 reap_dead start 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c2c1a700 1 --2- 172.21.15.83:0/2494687970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 0x7f05a800ba10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f05b400c190 tx=0x7f05a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8e54674c57c786 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4001db0 con 0x7f05a8005210 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b400ccf0 con 0x7f05a8005210 2023-11-26T07:13:26.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.733+0000 7f05c1417700 1 -- 172.21.15.83:0/2494687970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4003c40 con 0x7f05a8005210 2023-11-26T07:13:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.734+0000 7f05c467d700 1 -- 172.21.15.83:0/2494687970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 msgr2=0x7f05a800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.734+0000 7f05c467d700 1 --2- 172.21.15.83:0/2494687970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 0x7f05a800ba10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f05b400c190 tx=0x7f05a8005880 comp rx=0 tx=0).stop 2023-11-26T07:13:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.734+0000 7f05c467d700 1 -- 172.21.15.83:0/2494687970 shutdown_connections 2023-11-26T07:13:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.734+0000 7f05c467d700 1 --2- 172.21.15.83:0/2494687970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 0x7f05a800ba10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.734+0000 7f05c467d700 1 -- 172.21.15.83:0/2494687970 >> 172.21.15.83:0/2494687970 conn(0x7f05bc071b90 msgr2=0x7f05bc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.734+0000 7f05c467d700 1 -- 172.21.15.83:0/2494687970 shutdown_connections 2023-11-26T07:13:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.734+0000 7f05c467d700 1 -- 172.21.15.83:0/2494687970 wait complete. 2023-11-26T07:13:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.735+0000 7f05c467d700 1 Processor -- start 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.735+0000 7f05c467d700 1 -- start start 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c467d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 0x7f05bc15afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c467d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05bc1f37d0 0x7f05bc146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c467d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05bc147520 0x7f05bc1f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c467d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc1479d0 con 0x7f05a8005210 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c467d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc15f270 con 0x7f05bc1f37d0 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c467d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc15e660 con 0x7f05bc147520 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c2419700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 0x7f05bc15afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c2419700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 0x7f05bc15afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37636/0 (socket says 172.21.15.83:37636) 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c2419700 1 -- 172.21.15.83:0/1687211468 learned_addr learned my addr 172.21.15.83:0/1687211468 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c2419700 1 -- 172.21.15.83:0/1687211468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05bc147520 msgr2=0x7f05bc1f6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c2419700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05bc147520 0x7f05bc1f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c2419700 1 -- 172.21.15.83:0/1687211468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05bc1f37d0 msgr2=0x7f05bc146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c2419700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05bc1f37d0 0x7f05bc146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.736+0000 7f05c2419700 1 -- 172.21.15.83:0/1687211468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc15e940 con 0x7f05a8005210 2023-11-26T07:13:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.737+0000 7f05c2419700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 0x7f05bc15afc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f05b80119d0 tx=0x7f05b8011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.737+0000 7f05b2ffd700 1 -- 172.21.15.83:0/1687211468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b800f940 con 0x7f05a8005210 2023-11-26T07:13:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.737+0000 7f05b2ffd700 1 -- 172.21.15.83:0/1687211468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b800faa0 con 0x7f05a8005210 2023-11-26T07:13:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.737+0000 7f05c467d700 1 -- 172.21.15.83:0/1687211468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc15ec20 con 0x7f05a8005210 2023-11-26T07:13:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.737+0000 7f05b2ffd700 1 -- 172.21.15.83:0/1687211468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b800fe30 con 0x7f05a8005210 2023-11-26T07:13:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.737+0000 7f05c467d700 1 -- 172.21.15.83:0/1687211468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc15b500 con 0x7f05a8005210 2023-11-26T07:13:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.738+0000 7f05c467d700 1 -- 172.21.15.83:0/1687211468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc0753f0 con 0x7f05a8005210 2023-11-26T07:13:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.738+0000 7f05b2ffd700 1 -- 172.21.15.83:0/1687211468 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f05b801d400 con 0x7f05a8005210 2023-11-26T07:13:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.739+0000 7f05b2ffd700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05a006b110 0x7f05a006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.739+0000 7f05b2ffd700 1 -- 172.21.15.83:0/1687211468 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f05b808a4c0 con 0x7f05a8005210 2023-11-26T07:13:26.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.739+0000 7f05c1c18700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05a006b110 0x7f05a006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:26.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.739+0000 7f05c1c18700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05a006b110 0x7f05a006d5d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000f60 tx=0x7f05ac001110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:26.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.743+0000 7f05b2ffd700 1 -- 172.21.15.83:0/1687211468 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05b80553f0 con 0x7f05a8005210 2023-11-26T07:13:26.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:13:26.888 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:13:26.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.985+0000 7f05c467d700 1 -- 172.21.15.83:0/1687211468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05bc067f60 con 0x7f05a8005210 2023-11-26T07:13:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.985+0000 7f05b2ffd700 1 -- 172.21.15.83:0/1687211468 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f05b8058a10 con 0x7f05a8005210 2023-11-26T07:13:26.988 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:26.988 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:13:26.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.987+0000 7f05c467d700 1 -- 172.21.15.83:0/1687211468 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05a006b110 msgr2=0x7f05a006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:26.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.987+0000 7f05c467d700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05a006b110 0x7f05a006d5d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000f60 tx=0x7f05ac001110 comp rx=0 tx=0).stop 2023-11-26T07:13:26.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.987+0000 7f05c467d700 1 -- 172.21.15.83:0/1687211468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 msgr2=0x7f05bc15afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.987+0000 7f05c467d700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 0x7f05bc15afc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f05b80119d0 tx=0x7f05b8011d90 comp rx=0 tx=0).stop 2023-11-26T07:13:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.987+0000 7f05c467d700 1 -- 172.21.15.83:0/1687211468 shutdown_connections 2023-11-26T07:13:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.987+0000 7f05c467d700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05a006b110 0x7f05a006d5d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.987+0000 7f05c467d700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05bc147520 0x7f05bc1f6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.987+0000 7f05c467d700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a8005210 0x7f05bc15afc0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.987+0000 7f05c467d700 1 --2- 172.21.15.83:0/1687211468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05bc1f37d0 0x7f05bc146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.987+0000 7f05c467d700 1 -- 172.21.15.83:0/1687211468 >> 172.21.15.83:0/1687211468 conn(0x7f05bc071b90 msgr2=0x7f05bc074880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.988+0000 7f05c467d700 1 -- 172.21.15.83:0/1687211468 shutdown_connections 2023-11-26T07:13:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:26.988+0000 7f05c467d700 1 -- 172.21.15.83:0/1687211468 wait complete. 2023-11-26T07:13:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:13:27.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:13:27.142 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":12.155709791,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.391206034507849,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:13:27.143 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.39120603450785 }, "state": "open", "uptime": 12.155709791 } ] 2023-11-26T07:13:27.143 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:13:32.144 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:32.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.274+0000 7fb2cf97c700 1 Processor -- start 2023-11-26T07:13:32.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.274+0000 7fb2cf97c700 1 -- start start 2023-11-26T07:13:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.275+0000 7fb2cf97c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb2c8162920 0x7fb2c815e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.275+0000 7fb2cf97c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb2c80cfbd0 0x7fb2c815ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.275+0000 7fb2cf97c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c8161bf0 con 0x7fb2c815bd30 2023-11-26T07:13:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.275+0000 7fb2cf97c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c815f450 con 0x7fb2c8162920 2023-11-26T07:13:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.275+0000 7fb2cf97c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c8161e40 con 0x7fb2c80cfbd0 2023-11-26T07:13:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.275+0000 7fb2cd718700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb2c815bd30 0x7fb2c80cf4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57684/0 (socket says 172.21.15.83:57684) 2023-11-26T07:13:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.275+0000 7fb2ccf17700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb2c8162920 0x7fb2c815e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.275+0000 7fb2cd718700 1 -- 172.21.15.83:0/3935390992 learned_addr learned my addr 172.21.15.83:0/3935390992 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.275+0000 7fb2cdf19700 1 --2- 172.21.15.83:0/3935390992 >> v2:172.21.15.203:3300/0 conn(0x7fb2c80cfbd0 0x7fb2c815ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.275+0000 7fb2ccf17700 1 -- 172.21.15.83:0/3935390992 >> v2:172.21.15.203:3300/0 conn(0x7fb2c80cfbd0 msgr2=0x7fb2c815ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.275+0000 7fb2ccf17700 1 --2- 172.21.15.83:0/3935390992 >> v2:172.21.15.203:3300/0 conn(0x7fb2c80cfbd0 0x7fb2c815ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2ccf17700 1 -- 172.21.15.83:0/3935390992 >> v1:172.21.15.83:6789/0 conn(0x7fb2c815bd30 legacy=0x7fb2c80cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2bffff700 1 -- 172.21.15.83:0/3935390992 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582506744 0 0) 0x7fb2c8161bf0 con 0x7fb2c815bd30 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2ccf17700 1 -- 172.21.15.83:0/3935390992 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8162120 con 0x7fb2c8162920 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2ccf17700 1 --2- 172.21.15.83:0/3935390992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c8162920 0x7fb2c815e6e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009840 tx=0x7fb2b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4280e9f4b0fd08fd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2bffff700 1 -- 172.21.15.83:0/3935390992 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8011830 con 0x7fb2c8162920 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2bffff700 1 -- 172.21.15.83:0/3935390992 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b8011990 con 0x7fb2c8162920 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2bffff700 1 -- 172.21.15.83:0/3935390992 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8011d20 con 0x7fb2c8162920 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2cf97c700 1 -- 172.21.15.83:0/3935390992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c8162920 msgr2=0x7fb2c815e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2cf97c700 1 --2- 172.21.15.83:0/3935390992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c8162920 0x7fb2c815e6e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009840 tx=0x7fb2b8009b50 comp rx=0 tx=0).stop 2023-11-26T07:13:32.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2cf97c700 1 -- 172.21.15.83:0/3935390992 shutdown_connections 2023-11-26T07:13:32.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2cf97c700 1 --2- 172.21.15.83:0/3935390992 >> v2:172.21.15.203:3300/0 conn(0x7fb2c80cfbd0 0x7fb2c815ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:32.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2cf97c700 1 --2- 172.21.15.83:0/3935390992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c8162920 0x7fb2c815e6e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:32.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.276+0000 7fb2cf97c700 1 -- 172.21.15.83:0/3935390992 >> 172.21.15.83:0/3935390992 conn(0x7fb2c8072710 msgr2=0x7fb2c8072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:32.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.277+0000 7fb2cf97c700 1 -- 172.21.15.83:0/3935390992 shutdown_connections 2023-11-26T07:13:32.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.277+0000 7fb2cf97c700 1 -- 172.21.15.83:0/3935390992 wait complete. 2023-11-26T07:13:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.278+0000 7fb2cf97c700 1 Processor -- start 2023-11-26T07:13:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.278+0000 7fb2cf97c700 1 -- start start 2023-11-26T07:13:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2cf97c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cfbd0 0x7fb2c80c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2cf97c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c815bd30 0x7fb2c81510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2cf97c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c8162920 0x7fb2c81f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:32.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2cf97c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8151610 con 0x7fb2c815bd30 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2cf97c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81f5410 con 0x7fb2c80cfbd0 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2cf97c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81f56f0 con 0x7fb2c8162920 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2ccf17700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c815bd30 0x7fb2c81510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2ccf17700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c815bd30 0x7fb2c81510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37652/0 (socket says 172.21.15.83:37652) 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2ccf17700 1 -- 172.21.15.83:0/2971916104 learned_addr learned my addr 172.21.15.83:0/2971916104 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2cd718700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cfbd0 0x7fb2c80c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2cdf19700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c8162920 0x7fb2c81f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2ccf17700 1 -- 172.21.15.83:0/2971916104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c8162920 msgr2=0x7fb2c81f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2ccf17700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c8162920 0x7fb2c81f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2ccf17700 1 -- 172.21.15.83:0/2971916104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cfbd0 msgr2=0x7fb2c80c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2ccf17700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cfbd0 0x7fb2c80c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2ccf17700 1 -- 172.21.15.83:0/2971916104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b80094a0 con 0x7fb2c815bd30 2023-11-26T07:13:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2cd718700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cfbd0 0x7fb2c80c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.279+0000 7fb2cdf19700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c8162920 0x7fb2c81f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.280+0000 7fb2ccf17700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c815bd30 0x7fb2c81510d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8014040 tx=0x7fb2b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.280+0000 7fb2bdffb700 1 -- 172.21.15.83:0/2971916104 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80038a0 con 0x7fb2c815bd30 2023-11-26T07:13:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.280+0000 7fb2bdffb700 1 -- 172.21.15.83:0/2971916104 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b801e070 con 0x7fb2c815bd30 2023-11-26T07:13:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.280+0000 7fb2cf97c700 1 -- 172.21.15.83:0/2971916104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c81f59d0 con 0x7fb2c815bd30 2023-11-26T07:13:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.280+0000 7fb2bdffb700 1 -- 172.21.15.83:0/2971916104 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8003c40 con 0x7fb2c815bd30 2023-11-26T07:13:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.280+0000 7fb2cf97c700 1 -- 172.21.15.83:0/2971916104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c81f5f90 con 0x7fb2c815bd30 2023-11-26T07:13:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.281+0000 7fb2cf97c700 1 -- 172.21.15.83:0/2971916104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c80ccb20 con 0x7fb2c815bd30 2023-11-26T07:13:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.282+0000 7fb2bdffb700 1 -- 172.21.15.83:0/2971916104 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fb2b802d450 con 0x7fb2c815bd30 2023-11-26T07:13:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.282+0000 7fb2bdffb700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb2ac06b110 0x7fb2ac06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.282+0000 7fb2bdffb700 1 -- 172.21.15.83:0/2971916104 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb2b808abc0 con 0x7fb2c815bd30 2023-11-26T07:13:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.282+0000 7fb2cd718700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb2ac06b110 0x7fb2ac06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.282+0000 7fb2cd718700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb2ac06b110 0x7fb2ac06d5d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4000f60 tx=0x7fb2c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:32.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.287+0000 7fb2bdffb700 1 -- 172.21.15.83:0/2971916104 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb2b80558e0 con 0x7fb2c815bd30 2023-11-26T07:13:32.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.523+0000 7fb2cf97c700 1 -- 172.21.15.83:0/2971916104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2c80751e0 con 0x7fb2c815bd30 2023-11-26T07:13:32.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.523+0000 7fb2bdffb700 1 -- 172.21.15.83:0/2971916104 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fb2b8058f00 con 0x7fb2c815bd30 2023-11-26T07:13:32.526 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:32.526 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:13:32.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.525+0000 7fb2cf97c700 1 -- 172.21.15.83:0/2971916104 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb2ac06b110 msgr2=0x7fb2ac06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.525+0000 7fb2cf97c700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb2ac06b110 0x7fb2ac06d5d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4000f60 tx=0x7fb2c4009040 comp rx=0 tx=0).stop 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.525+0000 7fb2cf97c700 1 -- 172.21.15.83:0/2971916104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c815bd30 msgr2=0x7fb2c81510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.525+0000 7fb2cf97c700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c815bd30 0x7fb2c81510d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8014040 tx=0x7fb2b8004750 comp rx=0 tx=0).stop 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.526+0000 7fb2cf97c700 1 -- 172.21.15.83:0/2971916104 shutdown_connections 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.526+0000 7fb2cf97c700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb2ac06b110 0x7fb2ac06d5d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.526+0000 7fb2cf97c700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c8162920 0x7fb2c81f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.526+0000 7fb2cf97c700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cfbd0 0x7fb2c80c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.526+0000 7fb2cf97c700 1 --2- 172.21.15.83:0/2971916104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2c815bd30 0x7fb2c81510d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.526+0000 7fb2cf97c700 1 -- 172.21.15.83:0/2971916104 >> 172.21.15.83:0/2971916104 conn(0x7fb2c8072710 msgr2=0x7fb2c815c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.526+0000 7fb2cf97c700 1 -- 172.21.15.83:0/2971916104 shutdown_connections 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:32.526+0000 7fb2cf97c700 1 -- 172.21.15.83:0/2971916104 wait complete. 2023-11-26T07:13:32.529 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:13:32.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:13:32.681 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":17.693577695999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.119245058350181,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:13:32.682 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.11924505835018 }, "state": "open", "uptime": 17.693577696 } ] 2023-11-26T07:13:32.682 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:13:37.684 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.816+0000 7f1418669700 1 Processor -- start 2023-11-26T07:13:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.816+0000 7f1418669700 1 -- start start 2023-11-26T07:13:37.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.817+0000 7f1418669700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f14101446a0 0x7f14100b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:37.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.817+0000 7f1418669700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14100adf00 0x7f141006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:37.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.817+0000 7f1418669700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f141006c540 0x7f141006da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:37.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.817+0000 7f1418669700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1410144a80 con 0x7f14100adf00 2023-11-26T07:13:37.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.817+0000 7f1418669700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141006c9c0 con 0x7f14101446a0 2023-11-26T07:13:37.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.817+0000 7f1418669700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14100af950 con 0x7f141006c540 2023-11-26T07:13:37.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415c04700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14100adf00 0x7f141006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:37.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415c04700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14100adf00 0x7f141006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37660/0 (socket says 172.21.15.83:37660) 2023-11-26T07:13:37.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415c04700 1 -- 172.21.15.83:0/1056054117 learned_addr learned my addr 172.21.15.83:0/1056054117 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:37.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1416405700 1 --2- 172.21.15.83:0/1056054117 >> v2:172.21.15.203:3300/0 conn(0x7f14101446a0 0x7f14100b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:37.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1416c06700 1 --2- 172.21.15.83:0/1056054117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f141006c540 0x7f141006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:37.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415c04700 1 -- 172.21.15.83:0/1056054117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f141006c540 msgr2=0x7f141006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:37.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415c04700 1 --2- 172.21.15.83:0/1056054117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f141006c540 0x7f141006da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:37.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415c04700 1 -- 172.21.15.83:0/1056054117 >> v2:172.21.15.203:3300/0 conn(0x7f14101446a0 msgr2=0x7f14100b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:37.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415c04700 1 --2- 172.21.15.83:0/1056054117 >> v2:172.21.15.203:3300/0 conn(0x7f14101446a0 0x7f14100b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:37.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415c04700 1 -- 172.21.15.83:0/1056054117 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14100afc30 con 0x7f14100adf00 2023-11-26T07:13:37.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415c04700 1 --2- 172.21.15.83:0/1056054117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14100adf00 0x7f141006c000 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f14100729a0 tx=0x7f140c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3690434b9856d9b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:37.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415403700 1 -- 172.21.15.83:0/1056054117 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c011830 con 0x7f14100adf00 2023-11-26T07:13:37.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415403700 1 -- 172.21.15.83:0/1056054117 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c011990 con 0x7f14100adf00 2023-11-26T07:13:37.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.818+0000 7f1415403700 1 -- 172.21.15.83:0/1056054117 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c011d20 con 0x7f14100adf00 2023-11-26T07:13:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.819+0000 7f1418669700 1 -- 172.21.15.83:0/1056054117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14100adf00 msgr2=0x7f141006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.819+0000 7f1418669700 1 --2- 172.21.15.83:0/1056054117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14100adf00 0x7f141006c000 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f14100729a0 tx=0x7f140c009bd0 comp rx=0 tx=0).stop 2023-11-26T07:13:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.819+0000 7f1418669700 1 -- 172.21.15.83:0/1056054117 shutdown_connections 2023-11-26T07:13:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.819+0000 7f1418669700 1 --2- 172.21.15.83:0/1056054117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f141006c540 0x7f141006da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.819+0000 7f1418669700 1 --2- 172.21.15.83:0/1056054117 >> v2:172.21.15.203:3300/0 conn(0x7f14101446a0 0x7f14100b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.819+0000 7f1418669700 1 --2- 172.21.15.83:0/1056054117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14100adf00 0x7f141006c000 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.819+0000 7f1418669700 1 -- 172.21.15.83:0/1056054117 >> 172.21.15.83:0/1056054117 conn(0x7f1410071b90 msgr2=0x7f1410071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.819+0000 7f1418669700 1 -- 172.21.15.83:0/1056054117 shutdown_connections 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.819+0000 7f1418669700 1 -- 172.21.15.83:0/1056054117 wait complete. 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.820+0000 7f1418669700 1 Processor -- start 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.820+0000 7f1418669700 1 -- start start 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1418669700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141006c540 0x7f1410157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1418669700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14100adf00 0x7f14101583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1418669700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14101446a0 0x7f141015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1418669700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14101426a0 con 0x7f141006c540 2023-11-26T07:13:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1418669700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1410143200 con 0x7f14100adf00 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1418669700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14101428d0 con 0x7f14101446a0 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1416405700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141006c540 0x7f1410157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1416405700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141006c540 0x7f1410157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37664/0 (socket says 172.21.15.83:37664) 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1416405700 1 -- 172.21.15.83:0/1616414825 learned_addr learned my addr 172.21.15.83:0/1616414825 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1415c04700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14100adf00 0x7f14101583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1416c06700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14101446a0 0x7f141015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1416405700 1 -- 172.21.15.83:0/1616414825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14101446a0 msgr2=0x7f141015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1416405700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14101446a0 0x7f141015bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1416405700 1 -- 172.21.15.83:0/1616414825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14100adf00 msgr2=0x7f14101583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1416405700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14100adf00 0x7f14101583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1416405700 1 -- 172.21.15.83:0/1616414825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c009550 con 0x7f141006c540 2023-11-26T07:13:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1416c06700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14101446a0 0x7f141015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:13:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.821+0000 7f1416405700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141006c540 0x7f1410157880 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f140000e870 tx=0x7f140000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.822+0000 7f1406ffd700 1 -- 172.21.15.83:0/1616414825 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1400009590 con 0x7f141006c540 2023-11-26T07:13:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.822+0000 7f1406ffd700 1 -- 172.21.15.83:0/1616414825 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140000ee70 con 0x7f141006c540 2023-11-26T07:13:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.822+0000 7f1418669700 1 -- 172.21.15.83:0/1616414825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1410142bb0 con 0x7f141006c540 2023-11-26T07:13:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.822+0000 7f1406ffd700 1 -- 172.21.15.83:0/1616414825 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140000ca50 con 0x7f141006c540 2023-11-26T07:13:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.822+0000 7f1418669700 1 -- 172.21.15.83:0/1616414825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141015c100 con 0x7f141006c540 2023-11-26T07:13:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.823+0000 7f1418669700 1 -- 172.21.15.83:0/1616414825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14100753f0 con 0x7f141006c540 2023-11-26T07:13:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.823+0000 7f1406ffd700 1 -- 172.21.15.83:0/1616414825 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f140000cbb0 con 0x7f141006c540 2023-11-26T07:13:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.823+0000 7f1406ffd700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13f406b110 0x7f13f406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.824+0000 7f1406ffd700 1 -- 172.21.15.83:0/1616414825 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1400089940 con 0x7f141006c540 2023-11-26T07:13:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.824+0000 7f1415c04700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13f406b110 0x7f13f406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.824+0000 7f1415c04700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13f406b110 0x7f13f406d5d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f14100729a0 tx=0x7f140c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:37.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:37.828+0000 7f1406ffd700 1 -- 172.21.15.83:0/1616414825 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14000546e0 con 0x7f141006c540 2023-11-26T07:13:38.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.060+0000 7f1418669700 1 -- 172.21.15.83:0/1616414825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1410067f60 con 0x7f141006c540 2023-11-26T07:13:38.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.060+0000 7f1406ffd700 1 -- 172.21.15.83:0/1616414825 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f1400021090 con 0x7f141006c540 2023-11-26T07:13:38.062 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:38.062 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:13:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 -- 172.21.15.83:0/1616414825 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13f406b110 msgr2=0x7f13f406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13f406b110 0x7f13f406d5d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f14100729a0 tx=0x7f140c008030 comp rx=0 tx=0).stop 2023-11-26T07:13:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 -- 172.21.15.83:0/1616414825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141006c540 msgr2=0x7f1410157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141006c540 0x7f1410157880 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f140000e870 tx=0x7f140000ec30 comp rx=0 tx=0).stop 2023-11-26T07:13:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 -- 172.21.15.83:0/1616414825 shutdown_connections 2023-11-26T07:13:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f13f406b110 0x7f13f406d5d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14101446a0 0x7f141015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f141006c540 0x7f1410157880 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 --2- 172.21.15.83:0/1616414825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14100adf00 0x7f14101583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 -- 172.21.15.83:0/1616414825 >> 172.21.15.83:0/1616414825 conn(0x7f1410071b90 msgr2=0x7f141006b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 -- 172.21.15.83:0/1616414825 shutdown_connections 2023-11-26T07:13:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:38.062+0000 7f1418669700 1 -- 172.21.15.83:0/1616414825 wait complete. 2023-11-26T07:13:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:13:38.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:13:38.215 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":23.229445711,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.850838053178382,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:13:38.216 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.850838053178382 }, "state": "open", "uptime": 23.229445711 } ] 2023-11-26T07:13:38.216 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:13:43.217 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:43.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.348+0000 7f502d36e700 1 Processor -- start 2023-11-26T07:13:43.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.348+0000 7f502d36e700 1 -- start start 2023-11-26T07:13:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.349+0000 7f502d36e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f50280b4de0 0x7f50280b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.349+0000 7f502d36e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50280b3a60 con 0x7f502815b310 2023-11-26T07:13:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.349+0000 7f502d36e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5028155670 con 0x7f5028155290 2023-11-26T07:13:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.349+0000 7f502d36e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50280ce820 con 0x7f50280b4de0 2023-11-26T07:13:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.349+0000 7f50267fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f502815b310 0x7f502806d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57714/0 (socket says 172.21.15.83:57714) 2023-11-26T07:13:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.349+0000 7f50267fc700 1 -- 172.21.15.83:0/1846003794 learned_addr learned my addr 172.21.15.83:0/1846003794 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:13:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.349+0000 7f50277fe700 1 --2- 172.21.15.83:0/1846003794 >> v2:172.21.15.203:3300/0 conn(0x7f50280b4de0 0x7f50280b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656659299 0 0) 0x7f501c001110 con 0x7f502815b310 2023-11-26T07:13:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50100035b0 con 0x7f502815b310 2023-11-26T07:13:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2373558816 0 0) 0x7f50100035b0 con 0x7f502815b310 2023-11-26T07:13:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 >> v2:172.21.15.203:3300/0 conn(0x7f50280b4de0 msgr2=0x7f50280b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 --2- 172.21.15.83:0/1846003794 >> v2:172.21.15.203:3300/0 conn(0x7f50280b4de0 0x7f50280b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 >> v1:172.21.15.203:6789/0 conn(0x7f5028155290 legacy=0x7f50280b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280b4260 con 0x7f502815b310 2023-11-26T07:13:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f501c001800 con 0x7f502815b310 2023-11-26T07:13:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 >> v1:172.21.15.83:6789/0 conn(0x7f502815b310 legacy=0x7f502806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 --2- 172.21.15.83:0/1846003794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5010003a00 0x7f5010003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 --2- 172.21.15.83:0/1846003794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010004850 0x7f5010004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 --2- 172.21.15.83:0/1846003794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5010005210 0x7f501000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50280ce820 con 0x7f5010004850 2023-11-26T07:13:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501000c0b0 con 0x7f5010005210 2023-11-26T07:13:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501000c270 con 0x7f5010003a00 2023-11-26T07:13:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f50267fc700 1 --2- 172.21.15.83:0/1846003794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010004850 0x7f5010004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f5026ffd700 1 --2- 172.21.15.83:0/1846003794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5010003a00 0x7f5010003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f50277fe700 1 --2- 172.21.15.83:0/1846003794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5010005210 0x7f501000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f50267fc700 1 -- 172.21.15.83:0/1846003794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5010003a00 msgr2=0x7f5010003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f50267fc700 1 --2- 172.21.15.83:0/1846003794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5010003a00 0x7f5010003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f50267fc700 1 -- 172.21.15.83:0/1846003794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5010005210 msgr2=0x7f501000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f50267fc700 1 --2- 172.21.15.83:0/1846003794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5010005210 0x7f501000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f50277fe700 1 -- 172.21.15.83:0/1846003794 reap_dead start 2023-11-26T07:13:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.350+0000 7f50267fc700 1 -- 172.21.15.83:0/1846003794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50100035b0 con 0x7f5010004850 2023-11-26T07:13:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.351+0000 7f50267fc700 1 --2- 172.21.15.83:0/1846003794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010004850 0x7f5010004cd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f501c00c210 tx=0x7f5010005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c058aeacfefcc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.351+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c007d90 con 0x7f5010004850 2023-11-26T07:13:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.351+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c00cd30 con 0x7f5010004850 2023-11-26T07:13:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.351+0000 7f5025ffb700 1 -- 172.21.15.83:0/1846003794 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00cea0 con 0x7f5010004850 2023-11-26T07:13:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.351+0000 7f502d36e700 1 -- 172.21.15.83:0/1846003794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010004850 msgr2=0x7f5010004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.351+0000 7f502d36e700 1 --2- 172.21.15.83:0/1846003794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010004850 0x7f5010004cd0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f501c00c210 tx=0x7f5010005880 comp rx=0 tx=0).stop 2023-11-26T07:13:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.351+0000 7f502d36e700 1 -- 172.21.15.83:0/1846003794 shutdown_connections 2023-11-26T07:13:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.351+0000 7f502d36e700 1 --2- 172.21.15.83:0/1846003794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010004850 0x7f5010004cd0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.351+0000 7f502d36e700 1 -- 172.21.15.83:0/1846003794 >> 172.21.15.83:0/1846003794 conn(0x7f50280743b0 msgr2=0x7f502806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.351+0000 7f502d36e700 1 -- 172.21.15.83:0/1846003794 shutdown_connections 2023-11-26T07:13:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.351+0000 7f502d36e700 1 -- 172.21.15.83:0/1846003794 wait complete. 2023-11-26T07:13:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.353+0000 7f502d36e700 1 Processor -- start 2023-11-26T07:13:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.353+0000 7f502d36e700 1 -- start start 2023-11-26T07:13:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.353+0000 7f502d36e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5010004850 0x7f5028141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.353+0000 7f502d36e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5028153d30 0x7f502806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:43.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.353+0000 7f502d36e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50281541b0 0x7f502806e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:43.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.353+0000 7f502d36e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50281420c0 con 0x7f50281541b0 2023-11-26T07:13:43.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.353+0000 7f502d36e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5028157840 con 0x7f5010004850 2023-11-26T07:13:43.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.353+0000 7f502d36e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5028157b20 con 0x7f5028153d30 2023-11-26T07:13:43.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f50277fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50281541b0 0x7f502806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:43.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f50277fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50281541b0 0x7f502806e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37688/0 (socket says 172.21.15.83:37688) 2023-11-26T07:13:43.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f50277fe700 1 -- 172.21.15.83:0/4257537781 learned_addr learned my addr 172.21.15.83:0/4257537781 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:43.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f50267fc700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5028153d30 0x7f502806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:43.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f5026ffd700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5010004850 0x7f5028141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:43.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f50277fe700 1 -- 172.21.15.83:0/4257537781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5028153d30 msgr2=0x7f502806dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:43.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f50277fe700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5028153d30 0x7f502806dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:43.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f50277fe700 1 -- 172.21.15.83:0/4257537781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5010004850 msgr2=0x7f5028141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:43.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f50277fe700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5010004850 0x7f5028141b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:43.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f50277fe700 1 -- 172.21.15.83:0/4257537781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5028157e00 con 0x7f50281541b0 2023-11-26T07:13:43.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f50277fe700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50281541b0 0x7f502806e240 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f502806f2c0 tx=0x7f5020011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:43.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f500f7fe700 1 -- 172.21.15.83:0/4257537781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502000e040 con 0x7f50281541b0 2023-11-26T07:13:43.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f500f7fe700 1 -- 172.21.15.83:0/4257537781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502000f040 con 0x7f50281541b0 2023-11-26T07:13:43.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f502d36e700 1 -- 172.21.15.83:0/4257537781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50281580e0 con 0x7f50281541b0 2023-11-26T07:13:43.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f500f7fe700 1 -- 172.21.15.83:0/4257537781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502000fc40 con 0x7f50281541b0 2023-11-26T07:13:43.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.354+0000 7f502d36e700 1 -- 172.21.15.83:0/4257537781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502814d290 con 0x7f50281541b0 2023-11-26T07:13:43.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.355+0000 7f502d36e700 1 -- 172.21.15.83:0/4257537781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502814d420 con 0x7f50281541b0 2023-11-26T07:13:43.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.356+0000 7f500f7fe700 1 -- 172.21.15.83:0/4257537781 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f5020022460 con 0x7f50281541b0 2023-11-26T07:13:43.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.356+0000 7f500f7fe700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f501406b110 0x7f501406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:43.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.356+0000 7f500f7fe700 1 -- 172.21.15.83:0/4257537781 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f502008a080 con 0x7f50281541b0 2023-11-26T07:13:43.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.356+0000 7f5026ffd700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f501406b110 0x7f501406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:43.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.357+0000 7f5026ffd700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f501406b110 0x7f501406d5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5018007fd0 tx=0x7f5018009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:43.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.361+0000 7f500f7fe700 1 -- 172.21.15.83:0/4257537781 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5020054bf0 con 0x7f50281541b0 2023-11-26T07:13:43.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.600+0000 7f502d36e700 1 -- 172.21.15.83:0/4257537781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50280753f0 con 0x7f50281541b0 2023-11-26T07:13:43.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.600+0000 7f500f7fe700 1 -- 172.21.15.83:0/4257537781 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f5020058210 con 0x7f50281541b0 2023-11-26T07:13:43.602 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:43.603 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:13:43.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 -- 172.21.15.83:0/4257537781 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f501406b110 msgr2=0x7f501406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f501406b110 0x7f501406d5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5018007fd0 tx=0x7f5018009040 comp rx=0 tx=0).stop 2023-11-26T07:13:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 -- 172.21.15.83:0/4257537781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50281541b0 msgr2=0x7f502806e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50281541b0 0x7f502806e240 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f502806f2c0 tx=0x7f5020011e20 comp rx=0 tx=0).stop 2023-11-26T07:13:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 -- 172.21.15.83:0/4257537781 shutdown_connections 2023-11-26T07:13:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50281541b0 0x7f502806e240 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:43.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5010004850 0x7f5028141b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:43.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f501406b110 0x7f501406d5d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:43.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 --2- 172.21.15.83:0/4257537781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5028153d30 0x7f502806dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:43.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 -- 172.21.15.83:0/4257537781 >> 172.21.15.83:0/4257537781 conn(0x7f50280743b0 msgr2=0x7f502815c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:43.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 -- 172.21.15.83:0/4257537781 shutdown_connections 2023-11-26T07:13:43.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:43.603+0000 7f502d36e700 1 -- 172.21.15.83:0/4257537781 wait complete. 2023-11-26T07:13:43.607 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:13:43.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:13:43.756 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":28.769313634,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.585652016333412,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:13:43.756 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.585652016333412 }, "state": "open", "uptime": 28.769313634 } ] 2023-11-26T07:13:43.757 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:13:48.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:48.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.889+0000 7fe2ad03b700 1 Processor -- start 2023-11-26T07:13:48.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.889+0000 7fe2ad03b700 1 -- start start 2023-11-26T07:13:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2ad03b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe2a815af10 0x7fe2a815e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2ad03b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2a80ce140 0x7fe2a815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2ad03b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe2a815f460 0x7fe2a815b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2ad03b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a815f8c0 con 0x7fe2a80ce140 2023-11-26T07:13:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2ad03b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a81414c0 con 0x7fe2a815f460 2023-11-26T07:13:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2ad03b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a80ce9f0 con 0x7fe2a815af10 2023-11-26T07:13:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2a659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2a80ce140 0x7fe2a815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:48.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2a659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2a80ce140 0x7fe2a815ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37692/0 (socket says 172.21.15.83:37692) 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2a659c700 1 -- 172.21.15.83:0/2458652924 learned_addr learned my addr 172.21.15.83:0/2458652924 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2a659c700 1 -- 172.21.15.83:0/2458652924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe2a815af10 msgr2=0x7fe2a815e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2a659c700 1 --2- 172.21.15.83:0/2458652924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe2a815af10 0x7fe2a815e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2a659c700 1 -- 172.21.15.83:0/2458652924 >> v2:172.21.15.203:3300/0 conn(0x7fe2a815f460 msgr2=0x7fe2a815b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2a659c700 1 --2- 172.21.15.83:0/2458652924 >> v2:172.21.15.203:3300/0 conn(0x7fe2a815f460 0x7fe2a815b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2a659c700 1 -- 172.21.15.83:0/2458652924 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a80cecd0 con 0x7fe2a80ce140 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2a659c700 1 --2- 172.21.15.83:0/2458652924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a80ce140 0x7fe2a815ef20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8072dd0 tx=0x7fe29c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1a419f6f3040c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.890+0000 7fe2a5d9b700 1 -- 172.21.15.83:0/2458652924 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00b870 con 0x7fe2a80ce140 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.891+0000 7fe2a5d9b700 1 -- 172.21.15.83:0/2458652924 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c00b9d0 con 0x7fe2a80ce140 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.891+0000 7fe2a5d9b700 1 -- 172.21.15.83:0/2458652924 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00bd60 con 0x7fe2a80ce140 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.891+0000 7fe2ad03b700 1 -- 172.21.15.83:0/2458652924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a80ce140 msgr2=0x7fe2a815ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:48.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.891+0000 7fe2ad03b700 1 --2- 172.21.15.83:0/2458652924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a80ce140 0x7fe2a815ef20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8072dd0 tx=0x7fe29c009bd0 comp rx=0 tx=0).stop 2023-11-26T07:13:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.891+0000 7fe2ad03b700 1 -- 172.21.15.83:0/2458652924 shutdown_connections 2023-11-26T07:13:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.891+0000 7fe2ad03b700 1 --2- 172.21.15.83:0/2458652924 >> v2:172.21.15.203:3300/0 conn(0x7fe2a815f460 0x7fe2a815b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.891+0000 7fe2ad03b700 1 --2- 172.21.15.83:0/2458652924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe2a815af10 0x7fe2a815e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.891+0000 7fe2ad03b700 1 --2- 172.21.15.83:0/2458652924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a80ce140 0x7fe2a815ef20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.891+0000 7fe2ad03b700 1 -- 172.21.15.83:0/2458652924 >> 172.21.15.83:0/2458652924 conn(0x7fe2a80cd570 msgr2=0x7fe2a80cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.891+0000 7fe2ad03b700 1 -- 172.21.15.83:0/2458652924 shutdown_connections 2023-11-26T07:13:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.891+0000 7fe2ad03b700 1 -- 172.21.15.83:0/2458652924 wait complete. 2023-11-26T07:13:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.893+0000 7fe2ad03b700 1 Processor -- start 2023-11-26T07:13:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.893+0000 7fe2ad03b700 1 -- start start 2023-11-26T07:13:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.893+0000 7fe2ad03b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a80ce140 0x7fe2a80b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.893+0000 7fe2ad03b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2a815af10 0x7fe2a813d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2ad03b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe2a815f460 0x7fe2a813f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2ad03b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a80b5830 con 0x7fe2a80ce140 2023-11-26T07:13:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2ad03b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a815e450 con 0x7fe2a815af10 2023-11-26T07:13:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2ad03b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a815e170 con 0x7fe2a815f460 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2a6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a80ce140 0x7fe2a80b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2a6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a80ce140 0x7fe2a80b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37698/0 (socket says 172.21.15.83:37698) 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2a6d9d700 1 -- 172.21.15.83:0/1323044757 learned_addr learned my addr 172.21.15.83:0/1323044757 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2a759e700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe2a815f460 0x7fe2a813f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2a659c700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2a815af10 0x7fe2a813d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2a6d9d700 1 -- 172.21.15.83:0/1323044757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe2a815f460 msgr2=0x7fe2a813f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2a6d9d700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe2a815f460 0x7fe2a813f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2a6d9d700 1 -- 172.21.15.83:0/1323044757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2a815af10 msgr2=0x7fe2a813d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2a6d9d700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2a815af10 0x7fe2a813d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2a6d9d700 1 -- 172.21.15.83:0/1323044757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c009550 con 0x7fe2a80ce140 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2a6d9d700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a80ce140 0x7fe2a80b52f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe29800b530 tx=0x7fe29800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe28f7fe700 1 -- 172.21.15.83:0/1323044757 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298011830 con 0x7fe2a80ce140 2023-11-26T07:13:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe28f7fe700 1 -- 172.21.15.83:0/1323044757 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe298011990 con 0x7fe2a80ce140 2023-11-26T07:13:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2ad03b700 1 -- 172.21.15.83:0/1323044757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a81f6f60 con 0x7fe2a80ce140 2023-11-26T07:13:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe28f7fe700 1 -- 172.21.15.83:0/1323044757 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298011d40 con 0x7fe2a80ce140 2023-11-26T07:13:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.894+0000 7fe2ad03b700 1 -- 172.21.15.83:0/1323044757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a81f7520 con 0x7fe2a80ce140 2023-11-26T07:13:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.895+0000 7fe2ad03b700 1 -- 172.21.15.83:0/1323044757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a81f7890 con 0x7fe2a80ce140 2023-11-26T07:13:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.896+0000 7fe28f7fe700 1 -- 172.21.15.83:0/1323044757 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fe29801a400 con 0x7fe2a80ce140 2023-11-26T07:13:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.896+0000 7fe28f7fe700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe29406b110 0x7fe29406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.896+0000 7fe28f7fe700 1 -- 172.21.15.83:0/1323044757 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe298089920 con 0x7fe2a80ce140 2023-11-26T07:13:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.896+0000 7fe2a659c700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe29406b110 0x7fe29406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.897+0000 7fe2a659c700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe29406b110 0x7fe29406d5d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80ce540 tx=0x7fe29c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:48.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:48.901+0000 7fe28f7fe700 1 -- 172.21.15.83:0/1323044757 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe298054780 con 0x7fe2a80ce140 2023-11-26T07:13:49.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.133+0000 7fe2ad03b700 1 -- 172.21.15.83:0/1323044757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2a8067f60 con 0x7fe2a80ce140 2023-11-26T07:13:49.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.134+0000 7fe28f7fe700 1 -- 172.21.15.83:0/1323044757 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fe298015020 con 0x7fe2a80ce140 2023-11-26T07:13:49.136 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:49.137 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:13:49.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 -- 172.21.15.83:0/1323044757 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe29406b110 msgr2=0x7fe29406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:49.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe29406b110 0x7fe29406d5d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80ce540 tx=0x7fe29c008030 comp rx=0 tx=0).stop 2023-11-26T07:13:49.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 -- 172.21.15.83:0/1323044757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a80ce140 msgr2=0x7fe2a80b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:49.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a80ce140 0x7fe2a80b52f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe29800b530 tx=0x7fe29800b840 comp rx=0 tx=0).stop 2023-11-26T07:13:49.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 -- 172.21.15.83:0/1323044757 shutdown_connections 2023-11-26T07:13:49.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe29406b110 0x7fe29406d5d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:49.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe2a815f460 0x7fe2a813f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:49.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2a80ce140 0x7fe2a80b52f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:49.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 --2- 172.21.15.83:0/1323044757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2a815af10 0x7fe2a813d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:49.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 -- 172.21.15.83:0/1323044757 >> 172.21.15.83:0/1323044757 conn(0x7fe2a80cd570 msgr2=0x7fe2a815b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:49.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 -- 172.21.15.83:0/1323044757 shutdown_connections 2023-11-26T07:13:49.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:49.136+0000 7fe2ad03b700 1 -- 172.21.15.83:0/1323044757 wait complete. 2023-11-26T07:13:49.140 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:13:49.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:13:49.290 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":34.303181704000004,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.32412042337404,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:13:49.290 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.32412042337404 }, "state": "open", "uptime": 34.303181704 } ] 2023-11-26T07:13:49.291 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:13:54.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:54.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.423+0000 7f23cc37e700 1 Processor -- start 2023-11-26T07:13:54.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.423+0000 7f23cc37e700 1 -- start start 2023-11-26T07:13:54.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.424+0000 7f23cc37e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f23c415e080 0x7f23c40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.424+0000 7f23cc37e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23c40af4d0 0x7f23c40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.424+0000 7f23cc37e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f23c406c000 0x7f23c406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.424+0000 7f23cc37e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c40af0b0 con 0x7f23c415e080 2023-11-26T07:13:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.424+0000 7f23cc37e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c406c9f0 con 0x7f23c406c000 2023-11-26T07:13:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.424+0000 7f23cc37e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c40b05a0 con 0x7f23c40af4d0 2023-11-26T07:13:54.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.424+0000 7f23ca11a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f23c415e080 0x7f23c40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:54.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.424+0000 7f23ca11a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f23c415e080 0x7f23c40b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37706/0 (socket says 172.21.15.83:37706) 2023-11-26T07:13:54.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.424+0000 7f23ca11a700 1 -- 172.21.15.83:0/3195648343 learned_addr learned my addr 172.21.15.83:0/3195648343 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:54.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.424+0000 7f23c9919700 1 --2- 172.21.15.83:0/3195648343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23c40af4d0 0x7f23c40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:54.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.424+0000 7f23ca91b700 1 --2- 172.21.15.83:0/3195648343 >> v2:172.21.15.203:3300/0 conn(0x7f23c406c000 0x7f23c406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:54.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.425+0000 7f23ca11a700 1 -- 172.21.15.83:0/3195648343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23c40af4d0 msgr2=0x7f23c40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:54.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.425+0000 7f23ca11a700 1 --2- 172.21.15.83:0/3195648343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23c40af4d0 0x7f23c40afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.425+0000 7f23ca11a700 1 -- 172.21.15.83:0/3195648343 >> v2:172.21.15.203:3300/0 conn(0x7f23c406c000 msgr2=0x7f23c406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.425+0000 7f23ca11a700 1 --2- 172.21.15.83:0/3195648343 >> v2:172.21.15.203:3300/0 conn(0x7f23c406c000 0x7f23c406c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.425+0000 7f23ca11a700 1 -- 172.21.15.83:0/3195648343 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c40ae430 con 0x7f23c415e080 2023-11-26T07:13:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.425+0000 7f23ca11a700 1 --2- 172.21.15.83:0/3195648343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c415e080 0x7f23c40b5050 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f23c409eea0 tx=0x7f23b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b28f13acd485029 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.425+0000 7f23c9118700 1 -- 172.21.15.83:0/3195648343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b400b860 con 0x7f23c415e080 2023-11-26T07:13:54.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.425+0000 7f23c9118700 1 -- 172.21.15.83:0/3195648343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b400b9c0 con 0x7f23c415e080 2023-11-26T07:13:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.425+0000 7f23c9118700 1 -- 172.21.15.83:0/3195648343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b400bd50 con 0x7f23c415e080 2023-11-26T07:13:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.425+0000 7f23cc37e700 1 -- 172.21.15.83:0/3195648343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c415e080 msgr2=0x7f23c40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.425+0000 7f23cc37e700 1 --2- 172.21.15.83:0/3195648343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c415e080 0x7f23c40b5050 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f23c409eea0 tx=0x7f23b4009c00 comp rx=0 tx=0).stop 2023-11-26T07:13:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.426+0000 7f23cc37e700 1 -- 172.21.15.83:0/3195648343 shutdown_connections 2023-11-26T07:13:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.426+0000 7f23cc37e700 1 --2- 172.21.15.83:0/3195648343 >> v2:172.21.15.203:3300/0 conn(0x7f23c406c000 0x7f23c406c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:54.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.426+0000 7f23cc37e700 1 --2- 172.21.15.83:0/3195648343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c415e080 0x7f23c40b5050 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:54.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.426+0000 7f23cc37e700 1 --2- 172.21.15.83:0/3195648343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23c40af4d0 0x7f23c40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:54.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.426+0000 7f23cc37e700 1 -- 172.21.15.83:0/3195648343 >> 172.21.15.83:0/3195648343 conn(0x7f23c4071b90 msgr2=0x7f23c4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:54.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.426+0000 7f23cc37e700 1 -- 172.21.15.83:0/3195648343 shutdown_connections 2023-11-26T07:13:54.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.426+0000 7f23cc37e700 1 -- 172.21.15.83:0/3195648343 wait complete. 2023-11-26T07:13:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.427+0000 7f23cc37e700 1 Processor -- start 2023-11-26T07:13:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.427+0000 7f23cc37e700 1 -- start start 2023-11-26T07:13:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23cc37e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c406c000 0x7f23c4146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23cc37e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23c40af4d0 0x7f23c41551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:54.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23cc37e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23c415e080 0x7f23c4155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23cc37e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c4153190 con 0x7f23c406c000 2023-11-26T07:13:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23cc37e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c415f7c0 con 0x7f23c40af4d0 2023-11-26T07:13:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23cc37e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c4146a90 con 0x7f23c415e080 2023-11-26T07:13:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23ca11a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c406c000 0x7f23c4146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23ca11a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c406c000 0x7f23c4146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37712/0 (socket says 172.21.15.83:37712) 2023-11-26T07:13:54.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23ca11a700 1 -- 172.21.15.83:0/3843343930 learned_addr learned my addr 172.21.15.83:0/3843343930 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:54.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23ca11a700 1 -- 172.21.15.83:0/3843343930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23c415e080 msgr2=0x7f23c4155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:54.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23ca11a700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23c415e080 0x7f23c4155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:54.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23ca11a700 1 -- 172.21.15.83:0/3843343930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23c40af4d0 msgr2=0x7f23c41551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:54.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.428+0000 7f23ca11a700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23c40af4d0 0x7f23c41551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:54.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.429+0000 7f23ca11a700 1 -- 172.21.15.83:0/3843343930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b4009550 con 0x7f23c406c000 2023-11-26T07:13:54.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.429+0000 7f23ca11a700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c406c000 0x7f23c4146550 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009130 tx=0x7f23b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.429+0000 7f23baffd700 1 -- 172.21.15.83:0/3843343930 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b40044d0 con 0x7f23c406c000 2023-11-26T07:13:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.429+0000 7f23baffd700 1 -- 172.21.15.83:0/3843343930 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b401e070 con 0x7f23c406c000 2023-11-26T07:13:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.429+0000 7f23cc37e700 1 -- 172.21.15.83:0/3843343930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c4157f30 con 0x7f23c406c000 2023-11-26T07:13:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.429+0000 7f23baffd700 1 -- 172.21.15.83:0/3843343930 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b40038e0 con 0x7f23c406c000 2023-11-26T07:13:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.429+0000 7f23cc37e700 1 -- 172.21.15.83:0/3843343930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c41584f0 con 0x7f23c406c000 2023-11-26T07:13:54.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.430+0000 7f23cc37e700 1 -- 172.21.15.83:0/3843343930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c40753f0 con 0x7f23c406c000 2023-11-26T07:13:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.430+0000 7f23baffd700 1 -- 172.21.15.83:0/3843343930 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f23b4003a40 con 0x7f23c406c000 2023-11-26T07:13:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.431+0000 7f23baffd700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f23a806b110 0x7f23a806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.431+0000 7f23baffd700 1 -- 172.21.15.83:0/3843343930 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f23b4020070 con 0x7f23c406c000 2023-11-26T07:13:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.431+0000 7f23c9919700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f23a806b110 0x7f23a806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:54.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.431+0000 7f23c9919700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f23a806b110 0x7f23a806d5d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23c41598e0 tx=0x7f23c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:54.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.435+0000 7f23baffd700 1 -- 172.21.15.83:0/3843343930 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23b4055c40 con 0x7f23c406c000 2023-11-26T07:13:54.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.668+0000 7f23cc37e700 1 -- 172.21.15.83:0/3843343930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23c4068670 con 0x7f23c406c000 2023-11-26T07:13:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.668+0000 7f23baffd700 1 -- 172.21.15.83:0/3843343930 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f23b4015070 con 0x7f23c406c000 2023-11-26T07:13:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:13:54.671 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:13:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 -- 172.21.15.83:0/3843343930 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f23a806b110 msgr2=0x7f23a806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f23a806b110 0x7f23a806d5d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23c41598e0 tx=0x7f23c0008040 comp rx=0 tx=0).stop 2023-11-26T07:13:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 -- 172.21.15.83:0/3843343930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c406c000 msgr2=0x7f23c4146550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c406c000 0x7f23c4146550 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009130 tx=0x7f23b4004750 comp rx=0 tx=0).stop 2023-11-26T07:13:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 -- 172.21.15.83:0/3843343930 shutdown_connections 2023-11-26T07:13:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f23a806b110 0x7f23a806d5d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23c415e080 0x7f23c4155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c406c000 0x7f23c4146550 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 --2- 172.21.15.83:0/3843343930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23c40af4d0 0x7f23c41551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 -- 172.21.15.83:0/3843343930 >> 172.21.15.83:0/3843343930 conn(0x7f23c4071b90 msgr2=0x7f23c40b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 -- 172.21.15.83:0/3843343930 shutdown_connections 2023-11-26T07:13:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:54.671+0000 7f23cc37e700 1 -- 172.21.15.83:0/3843343930 wait complete. 2023-11-26T07:13:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:13:54.686 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:13:54.826 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":39.840049696999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.065772391241904,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:13:54.827 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.065772391241904 }, "state": "open", "uptime": 39.840049697 } ] 2023-11-26T07:13:54.827 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:13:56.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:13:56.957 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:13:59.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:13:59.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.959+0000 7f6ccde85700 1 Processor -- start 2023-11-26T07:13:59.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.959+0000 7f6ccde85700 1 -- start start 2023-11-26T07:13:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.959+0000 7f6ccde85700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc815bdd0 0x7f6cc81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.959+0000 7f6ccde85700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6cc80d1c70 0x7f6cc80d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.959+0000 7f6ccde85700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc80b22d0 con 0x7f6cc80b3370 2023-11-26T07:13:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.959+0000 7f6ccde85700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc80b2f50 con 0x7f6cc80d1c70 2023-11-26T07:13:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.959+0000 7f6ccde85700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80d2640 con 0x7f6cc815bdd0 2023-11-26T07:13:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.960+0000 7f6cc77fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc815bdd0 0x7f6cc81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:59.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.960+0000 7f6cc77fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc815bdd0 0x7f6cc81604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:48178/0 (socket says 172.21.15.83:48178) 2023-11-26T07:13:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.960+0000 7f6cc77fe700 1 -- 172.21.15.83:0/1061963890 learned_addr learned my addr 172.21.15.83:0/1061963890 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.960+0000 7f6cc7fff700 1 --2- 172.21.15.83:0/1061963890 >> v2:172.21.15.203:3300/0 conn(0x7f6cc80d1c70 0x7f6cc80d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.960+0000 7f6cc77fe700 1 -- 172.21.15.83:0/1061963890 >> v2:172.21.15.203:3300/0 conn(0x7f6cc80d1c70 msgr2=0x7f6cc80d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:13:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.960+0000 7f6cc77fe700 1 --2- 172.21.15.83:0/1061963890 >> v2:172.21.15.203:3300/0 conn(0x7f6cc80d1c70 0x7f6cc80d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.960+0000 7f6cc77fe700 1 -- 172.21.15.83:0/1061963890 >> v1:172.21.15.203:6789/0 conn(0x7f6cc80b3370 legacy=0x7f6cc80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.960+0000 7f6cc77fe700 1 -- 172.21.15.83:0/1061963890 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80b25b0 con 0x7f6cc815bdd0 2023-11-26T07:13:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.960+0000 7f6cc7fff700 1 --2- 172.21.15.83:0/1061963890 >> v2:172.21.15.203:3300/0 conn(0x7f6cc80d1c70 0x7f6cc80d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:13:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.960+0000 7f6cc77fe700 1 --2- 172.21.15.83:0/1061963890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc815bdd0 0x7f6cc81604e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80098f0 tx=0x7f6cb8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81f5cc8b58ba9207 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.961+0000 7f6cc67fc700 1 -- 172.21.15.83:0/1061963890 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8011830 con 0x7f6cc815bdd0 2023-11-26T07:13:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.961+0000 7f6cc67fc700 1 -- 172.21.15.83:0/1061963890 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb8011990 con 0x7f6cc815bdd0 2023-11-26T07:13:59.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.961+0000 7f6cc67fc700 1 -- 172.21.15.83:0/1061963890 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8011d20 con 0x7f6cc815bdd0 2023-11-26T07:13:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.961+0000 7f6ccde85700 1 -- 172.21.15.83:0/1061963890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc815bdd0 msgr2=0x7f6cc81604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.961+0000 7f6ccde85700 1 --2- 172.21.15.83:0/1061963890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc815bdd0 0x7f6cc81604e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80098f0 tx=0x7f6cb8009c00 comp rx=0 tx=0).stop 2023-11-26T07:13:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.961+0000 7f6ccde85700 1 -- 172.21.15.83:0/1061963890 shutdown_connections 2023-11-26T07:13:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.961+0000 7f6ccde85700 1 --2- 172.21.15.83:0/1061963890 >> v2:172.21.15.203:3300/0 conn(0x7f6cc80d1c70 0x7f6cc80d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:59.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.961+0000 7f6ccde85700 1 --2- 172.21.15.83:0/1061963890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc815bdd0 0x7f6cc81604e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.961+0000 7f6ccde85700 1 -- 172.21.15.83:0/1061963890 >> 172.21.15.83:0/1061963890 conn(0x7f6cc80731a0 msgr2=0x7f6cc809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.961+0000 7f6ccde85700 1 -- 172.21.15.83:0/1061963890 shutdown_connections 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.961+0000 7f6ccde85700 1 -- 172.21.15.83:0/1061963890 wait complete. 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.963+0000 7f6ccde85700 1 Processor -- start 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.963+0000 7f6ccde85700 1 -- start start 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.963+0000 7f6ccde85700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cc80b3370 0x7f6cc8146960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.963+0000 7f6ccde85700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc80d1c70 0x7f6cc80c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.963+0000 7f6ccde85700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815bdd0 0x7f6cc81f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6ccde85700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80cb760 con 0x7f6cc815bdd0 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6ccde85700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc8146ea0 con 0x7f6cc80b3370 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6ccde85700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80cbb80 con 0x7f6cc80d1c70 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6cc7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815bdd0 0x7f6cc81f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6cc7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815bdd0 0x7f6cc81f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37730/0 (socket says 172.21.15.83:37730) 2023-11-26T07:13:59.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6cc7fff700 1 -- 172.21.15.83:0/1965532080 learned_addr learned my addr 172.21.15.83:0/1965532080 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6cc6ffd700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc80d1c70 0x7f6cc80c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6cc77fe700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cc80b3370 0x7f6cc8146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6cc7fff700 1 -- 172.21.15.83:0/1965532080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc80d1c70 msgr2=0x7f6cc80c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6cc7fff700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc80d1c70 0x7f6cc80c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6cc7fff700 1 -- 172.21.15.83:0/1965532080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cc80b3370 msgr2=0x7f6cc8146960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6cc7fff700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cc80b3370 0x7f6cc8146960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6cc7fff700 1 -- 172.21.15.83:0/1965532080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc0009a70 con 0x7f6cc815bdd0 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6cc7fff700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815bdd0 0x7f6cc81f68a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000efc0 tx=0x7f6cc000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6caffff700 1 -- 172.21.15.83:0/1965532080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000ca90 con 0x7f6cc815bdd0 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6caffff700 1 -- 172.21.15.83:0/1965532080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc000f040 con 0x7f6cc815bdd0 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6ccde85700 1 -- 172.21.15.83:0/1965532080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb8009550 con 0x7f6cc815bdd0 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6caffff700 1 -- 172.21.15.83:0/1965532080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000ce10 con 0x7f6cc815bdd0 2023-11-26T07:13:59.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.964+0000 7f6ccde85700 1 -- 172.21.15.83:0/1965532080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc80caae0 con 0x7f6cc815bdd0 2023-11-26T07:13:59.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.965+0000 7f6ccde85700 1 -- 172.21.15.83:0/1965532080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc80cb9d0 con 0x7f6cc815bdd0 2023-11-26T07:13:59.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.966+0000 7f6caffff700 1 -- 172.21.15.83:0/1965532080 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6cc0014480 con 0x7f6cc815bdd0 2023-11-26T07:13:59.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.966+0000 7f6caffff700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6cb406b110 0x7f6cb406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:13:59.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.966+0000 7f6caffff700 1 -- 172.21.15.83:0/1965532080 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6cc008a180 con 0x7f6cc815bdd0 2023-11-26T07:13:59.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.966+0000 7f6cc77fe700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6cb406b110 0x7f6cb406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:13:59.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.967+0000 7f6cc77fe700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6cb406b110 0x7f6cb406d5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8014040 tx=0x7f6cb8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:13:59.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:13:59.971+0000 7f6caffff700 1 -- 172.21.15.83:0/1965532080 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6cc0054e10 con 0x7f6cc815bdd0 2023-11-26T07:14:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.202+0000 7f6ccde85700 1 -- 172.21.15.83:0/1965532080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cc8067f60 con 0x7f6cc815bdd0 2023-11-26T07:14:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.203+0000 7f6caffff700 1 -- 172.21.15.83:0/1965532080 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f6cc0009d70 con 0x7f6cc815bdd0 2023-11-26T07:14:00.206 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:00.206 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 -- 172.21.15.83:0/1965532080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6cb406b110 msgr2=0x7f6cb406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6cb406b110 0x7f6cb406d5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8014040 tx=0x7f6cb8008030 comp rx=0 tx=0).stop 2023-11-26T07:14:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 -- 172.21.15.83:0/1965532080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815bdd0 msgr2=0x7f6cc81f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815bdd0 0x7f6cc81f68a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000efc0 tx=0x7f6cc000c3a0 comp rx=0 tx=0).stop 2023-11-26T07:14:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 -- 172.21.15.83:0/1965532080 shutdown_connections 2023-11-26T07:14:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cc815bdd0 0x7f6cc81f68a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cc80b3370 0x7f6cc8146960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6cb406b110 0x7f6cb406d5d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 --2- 172.21.15.83:0/1965532080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6cc80d1c70 0x7f6cc80c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 -- 172.21.15.83:0/1965532080 >> 172.21.15.83:0/1965532080 conn(0x7f6cc80731a0 msgr2=0x7f6cc80b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 -- 172.21.15.83:0/1965532080 shutdown_connections 2023-11-26T07:14:00.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:00.205+0000 7f6ccde85700 1 -- 172.21.15.83:0/1965532080 wait complete. 2023-11-26T07:14:00.210 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:00.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:14:00.360 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":45.372917788000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.810891410566246,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:14:00.361 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.810891410566246 }, "state": "open", "uptime": 45.372917788 } ] 2023-11-26T07:14:00.361 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:14:05.362 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:05.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.493+0000 7f6f41165700 1 Processor -- start 2023-11-26T07:14:05.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.493+0000 7f6f41165700 1 -- start start 2023-11-26T07:14:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.493+0000 7f6f41165700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c15bdd0 0x7f6f3c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.493+0000 7f6f41165700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6f3c0d1c70 0x7f6f3c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.493+0000 7f6f41165700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3c0b22d0 con 0x7f6f3c0b3370 2023-11-26T07:14:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.493+0000 7f6f41165700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3c0b2f50 con 0x7f6f3c0d1c70 2023-11-26T07:14:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f41165700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0d2640 con 0x7f6f3c15bdd0 2023-11-26T07:14:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3ad9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c15bdd0 0x7f6f3c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3b59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6f3c0d1c70 0x7f6f3c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3ad9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c15bdd0 0x7f6f3c1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:48192/0 (socket says 172.21.15.83:48192) 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3a59c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f6f3c0b3370 0x7f6f3c0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:42392/0 (socket says 172.21.15.83:42392) 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3b59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6f3c0d1c70 0x7f6f3c0d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:37190/0 (socket says 172.21.15.83:37190) 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3ad9d700 1 -- 172.21.15.83:0/1293477972 learned_addr learned my addr 172.21.15.83:0/1293477972 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3b59e700 1 -- 172.21.15.83:0/1293477972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c15bdd0 msgr2=0x7f6f3c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3b59e700 1 --2- 172.21.15.83:0/1293477972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c15bdd0 0x7f6f3c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3b59e700 1 -- 172.21.15.83:0/1293477972 >> v1:172.21.15.203:6789/0 conn(0x7f6f3c0b3370 legacy=0x7f6f3c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3b59e700 1 -- 172.21.15.83:0/1293477972 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c0b25b0 con 0x7f6f3c0d1c70 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3ad9d700 1 --2- 172.21.15.83:0/1293477972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c15bdd0 0x7f6f3c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.494+0000 7f6f3b59e700 1 --2- 172.21.15.83:0/1293477972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f3c0d1c70 0x7f6f3c0d2100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009d90 tx=0x7f6f3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7655ec0a2ca15c2a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.495+0000 7f6f39d9b700 1 -- 172.21.15.83:0/1293477972 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3400e070 con 0x7f6f3c0d1c70 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.495+0000 7f6f39d9b700 1 -- 172.21.15.83:0/1293477972 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3400bb20 con 0x7f6f3c0d1c70 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.495+0000 7f6f39d9b700 1 -- 172.21.15.83:0/1293477972 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34012430 con 0x7f6f3c0d1c70 2023-11-26T07:14:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.495+0000 7f6f41165700 1 -- 172.21.15.83:0/1293477972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f3c0d1c70 msgr2=0x7f6f3c0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.495+0000 7f6f41165700 1 --2- 172.21.15.83:0/1293477972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f3c0d1c70 0x7f6f3c0d2100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009d90 tx=0x7f6f3400b330 comp rx=0 tx=0).stop 2023-11-26T07:14:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.495+0000 7f6f41165700 1 -- 172.21.15.83:0/1293477972 shutdown_connections 2023-11-26T07:14:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.495+0000 7f6f41165700 1 --2- 172.21.15.83:0/1293477972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f3c0d1c70 0x7f6f3c0d2100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.495+0000 7f6f41165700 1 --2- 172.21.15.83:0/1293477972 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c15bdd0 0x7f6f3c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.495+0000 7f6f41165700 1 -- 172.21.15.83:0/1293477972 >> 172.21.15.83:0/1293477972 conn(0x7f6f3c0731a0 msgr2=0x7f6f3c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.495+0000 7f6f41165700 1 -- 172.21.15.83:0/1293477972 shutdown_connections 2023-11-26T07:14:05.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.495+0000 7f6f41165700 1 -- 172.21.15.83:0/1293477972 wait complete. 2023-11-26T07:14:05.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.497+0000 7f6f41165700 1 Processor -- start 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.497+0000 7f6f41165700 1 -- start start 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.497+0000 7f6f41165700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c0b3370 0x7f6f3c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.497+0000 7f6f41165700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c0d1c70 0x7f6f3c0c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.497+0000 7f6f41165700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f3c15bdd0 0x7f6f3c0c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f41165700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0c66b0 con 0x7f6f3c0b3370 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f41165700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c15cf10 con 0x7f6f3c15bdd0 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f41165700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0c60f0 con 0x7f6f3c0d1c70 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f3ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c0b3370 0x7f6f3c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f3ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c0b3370 0x7f6f3c158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37740/0 (socket says 172.21.15.83:37740) 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f3ad9d700 1 -- 172.21.15.83:0/2571898272 learned_addr learned my addr 172.21.15.83:0/2571898272 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f3b59e700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f3c15bdd0 0x7f6f3c0c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f3a59c700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c0d1c70 0x7f6f3c0c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f3ad9d700 1 -- 172.21.15.83:0/2571898272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c0d1c70 msgr2=0x7f6f3c0c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:05.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f3ad9d700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c0d1c70 0x7f6f3c0c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f3ad9d700 1 -- 172.21.15.83:0/2571898272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f3c15bdd0 msgr2=0x7f6f3c0c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f3ad9d700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f3c15bdd0 0x7f6f3c0c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f3ad9d700 1 -- 172.21.15.83:0/2571898272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c009550 con 0x7f6f3c0b3370 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f3ad9d700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c0b3370 0x7f6f3c158150 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00e9b0 tx=0x7f6f2c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f237fe700 1 -- 172.21.15.83:0/2571898272 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00ee50 con 0x7f6f3c0b3370 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f237fe700 1 -- 172.21.15.83:0/2571898272 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c00c850 con 0x7f6f3c0b3370 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f41165700 1 -- 172.21.15.83:0/2571898272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f34009a70 con 0x7f6f3c0b3370 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f237fe700 1 -- 172.21.15.83:0/2571898272 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00cbc0 con 0x7f6f3c0b3370 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.498+0000 7f6f41165700 1 -- 172.21.15.83:0/2571898272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3c15d420 con 0x7f6f3c0b3370 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.499+0000 7f6f41165700 1 -- 172.21.15.83:0/2571898272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3c0b1260 con 0x7f6f3c0b3370 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.500+0000 7f6f237fe700 1 -- 172.21.15.83:0/2571898272 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6f2c00cd20 con 0x7f6f3c0b3370 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.500+0000 7f6f237fe700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6f2806b110 0x7f6f2806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:05.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.500+0000 7f6f237fe700 1 -- 172.21.15.83:0/2571898272 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6f2c0899e0 con 0x7f6f3c0b3370 2023-11-26T07:14:05.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.500+0000 7f6f3a59c700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6f2806b110 0x7f6f2806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:05.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.501+0000 7f6f3a59c700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6f2806b110 0x7f6f2806d5d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0cb520 tx=0x7f6f30007ed0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:05.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.505+0000 7f6f237fe700 1 -- 172.21.15.83:0/2571898272 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f2c0546f0 con 0x7f6f3c0b3370 2023-11-26T07:14:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.738+0000 7f6f41165700 1 -- 172.21.15.83:0/2571898272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f3c067f60 con 0x7f6f3c0b3370 2023-11-26T07:14:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.738+0000 7f6f237fe700 1 -- 172.21.15.83:0/2571898272 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f6f2c021090 con 0x7f6f3c0b3370 2023-11-26T07:14:05.741 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:05.741 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 -- 172.21.15.83:0/2571898272 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6f2806b110 msgr2=0x7f6f2806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6f2806b110 0x7f6f2806d5d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0cb520 tx=0x7f6f30007ed0 comp rx=0 tx=0).stop 2023-11-26T07:14:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 -- 172.21.15.83:0/2571898272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c0b3370 msgr2=0x7f6f3c158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c0b3370 0x7f6f3c158150 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00e9b0 tx=0x7f6f2c00ecc0 comp rx=0 tx=0).stop 2023-11-26T07:14:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 -- 172.21.15.83:0/2571898272 shutdown_connections 2023-11-26T07:14:05.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6f3c15bdd0 0x7f6f3c0c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:05.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c0b3370 0x7f6f3c158150 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:05.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6f2806b110 0x7f6f2806d5d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:05.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 --2- 172.21.15.83:0/2571898272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6f3c0d1c70 0x7f6f3c0c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:05.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 -- 172.21.15.83:0/2571898272 >> 172.21.15.83:0/2571898272 conn(0x7f6f3c0731a0 msgr2=0x7f6f3c0cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:05.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 -- 172.21.15.83:0/2571898272 shutdown_connections 2023-11-26T07:14:05.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:05.740+0000 7f6f41165700 1 -- 172.21.15.83:0/2571898272 wait complete. 2023-11-26T07:14:05.744 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:05.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:14:05.895 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":50.907785830999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.559157617935373,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:14:05.895 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.559157617935373 }, "state": "open", "uptime": 50.907785831 } ] 2023-11-26T07:14:05.896 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:14:10.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.027+0000 7fcdf97c6700 1 Processor -- start 2023-11-26T07:14:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.027+0000 7fcdf97c6700 1 -- start start 2023-11-26T07:14:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.028+0000 7fcdf97c6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcdf406ed40 0x7fcdf40afa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.028+0000 7fcdf97c6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf4073f90 0x7fcdf40743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.028+0000 7fcdf97c6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf4074900 con 0x7fcdf415deb0 2023-11-26T07:14:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.028+0000 7fcdf97c6700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf40affa0 con 0x7fcdf406ed40 2023-11-26T07:14:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.028+0000 7fcdf97c6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf406dca0 con 0x7fcdf4073f90 2023-11-26T07:14:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.028+0000 7fcdf27fc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcdf406ed40 0x7fcdf40afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.028+0000 7fcdf37fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf4073f90 0x7fcdf40743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.028+0000 7fcdf2ffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fcdf415deb0 0x7fcdf40b5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:42404/0 (socket says 172.21.15.83:42404) 2023-11-26T07:14:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.028+0000 7fcdf2ffd700 1 -- 172.21.15.83:0/3419563536 learned_addr learned my addr 172.21.15.83:0/3419563536 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.029+0000 7fcdf37fe700 1 -- 172.21.15.83:0/3419563536 >> v2:172.21.15.203:3300/0 conn(0x7fcdf406ed40 msgr2=0x7fcdf40afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.029+0000 7fcdf37fe700 1 --2- 172.21.15.83:0/3419563536 >> v2:172.21.15.203:3300/0 conn(0x7fcdf406ed40 0x7fcdf40afa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.029+0000 7fcdf37fe700 1 -- 172.21.15.83:0/3419563536 >> v1:172.21.15.203:6789/0 conn(0x7fcdf415deb0 legacy=0x7fcdf40b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.029+0000 7fcdf37fe700 1 -- 172.21.15.83:0/3419563536 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf406df80 con 0x7fcdf4073f90 2023-11-26T07:14:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.029+0000 7fcdf37fe700 1 --2- 172.21.15.83:0/3419563536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf4073f90 0x7fcdf40743c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009d90 tx=0x7fcdec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f54d129758d1449 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.029+0000 7fcdf1ffb700 1 -- 172.21.15.83:0/3419563536 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00e070 con 0x7fcdf4073f90 2023-11-26T07:14:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.029+0000 7fcdf1ffb700 1 -- 172.21.15.83:0/3419563536 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec00bb20 con 0x7fcdf4073f90 2023-11-26T07:14:11.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.029+0000 7fcdf1ffb700 1 -- 172.21.15.83:0/3419563536 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec012430 con 0x7fcdf4073f90 2023-11-26T07:14:11.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.029+0000 7fcdf97c6700 1 -- 172.21.15.83:0/3419563536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf4073f90 msgr2=0x7fcdf40743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:11.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.029+0000 7fcdf97c6700 1 --2- 172.21.15.83:0/3419563536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf4073f90 0x7fcdf40743c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009d90 tx=0x7fcdec00b330 comp rx=0 tx=0).stop 2023-11-26T07:14:11.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.030+0000 7fcdf97c6700 1 -- 172.21.15.83:0/3419563536 shutdown_connections 2023-11-26T07:14:11.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.030+0000 7fcdf97c6700 1 --2- 172.21.15.83:0/3419563536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf4073f90 0x7fcdf40743c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:11.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.030+0000 7fcdf97c6700 1 --2- 172.21.15.83:0/3419563536 >> v2:172.21.15.203:3300/0 conn(0x7fcdf406ed40 0x7fcdf40afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:11.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.030+0000 7fcdf97c6700 1 -- 172.21.15.83:0/3419563536 >> 172.21.15.83:0/3419563536 conn(0x7fcdf40d0ba0 msgr2=0x7fcdf40d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:11.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.030+0000 7fcdf97c6700 1 -- 172.21.15.83:0/3419563536 shutdown_connections 2023-11-26T07:14:11.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.030+0000 7fcdf97c6700 1 -- 172.21.15.83:0/3419563536 wait complete. 2023-11-26T07:14:11.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.031+0000 7fcdf97c6700 1 Processor -- start 2023-11-26T07:14:11.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.031+0000 7fcdf97c6700 1 -- start start 2023-11-26T07:14:11.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf97c6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf406ed40 0x7fcdf41f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:11.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf97c6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf4073f90 0x7fcdf414c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:11.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf97c6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf415deb0 0x7fcdf41f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:11.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf97c6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf414ceb0 con 0x7fcdf4073f90 2023-11-26T07:14:11.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf97c6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf41f7550 con 0x7fcdf406ed40 2023-11-26T07:14:11.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf97c6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf41f7830 con 0x7fcdf415deb0 2023-11-26T07:14:11.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf4073f90 0x7fcdf414c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:11.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf4073f90 0x7fcdf414c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37754/0 (socket says 172.21.15.83:37754) 2023-11-26T07:14:11.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf27fc700 1 -- 172.21.15.83:0/2879398751 learned_addr learned my addr 172.21.15.83:0/2879398751 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:11.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf2ffd700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf406ed40 0x7fcdf41f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:11.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf37fe700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf415deb0 0x7fcdf41f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:11.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf27fc700 1 -- 172.21.15.83:0/2879398751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf415deb0 msgr2=0x7fcdf41f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:11.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf27fc700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf415deb0 0x7fcdf41f4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:11.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf27fc700 1 -- 172.21.15.83:0/2879398751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf406ed40 msgr2=0x7fcdf41f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:11.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf27fc700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf406ed40 0x7fcdf41f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:11.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf27fc700 1 -- 172.21.15.83:0/2879398751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec009a70 con 0x7fcdf4073f90 2023-11-26T07:14:11.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf27fc700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf4073f90 0x7fcdf414c810 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcde400e840 tx=0x7fcde400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:11.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcddb7fe700 1 -- 172.21.15.83:0/2879398751 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4009590 con 0x7fcdf4073f90 2023-11-26T07:14:11.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcddb7fe700 1 -- 172.21.15.83:0/2879398751 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde4013680 con 0x7fcdf4073f90 2023-11-26T07:14:11.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcdf97c6700 1 -- 172.21.15.83:0/2879398751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf41f7b10 con 0x7fcdf4073f90 2023-11-26T07:14:11.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.032+0000 7fcddb7fe700 1 -- 172.21.15.83:0/2879398751 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4013a30 con 0x7fcdf4073f90 2023-11-26T07:14:11.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.033+0000 7fcdf97c6700 1 -- 172.21.15.83:0/2879398751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf41f7df0 con 0x7fcdf4073f90 2023-11-26T07:14:11.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.034+0000 7fcdf97c6700 1 -- 172.21.15.83:0/2879398751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf4068480 con 0x7fcdf4073f90 2023-11-26T07:14:11.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.034+0000 7fcddb7fe700 1 -- 172.21.15.83:0/2879398751 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fcde4013b90 con 0x7fcdf4073f90 2023-11-26T07:14:11.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.034+0000 7fcddb7fe700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcde006b110 0x7fcde006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:11.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.034+0000 7fcddb7fe700 1 -- 172.21.15.83:0/2879398751 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcde40899c0 con 0x7fcdf4073f90 2023-11-26T07:14:11.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.034+0000 7fcdf2ffd700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcde006b110 0x7fcde006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:11.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.035+0000 7fcdf2ffd700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcde006b110 0x7fcde006d5d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcde8005910 tx=0x7fcde8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:11.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.039+0000 7fcddb7fe700 1 -- 172.21.15.83:0/2879398751 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcde40546d0 con 0x7fcdf4073f90 2023-11-26T07:14:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.274+0000 7fcdf97c6700 1 -- 172.21.15.83:0/2879398751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdf40751e0 con 0x7fcdf4073f90 2023-11-26T07:14:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.274+0000 7fcddb7fe700 1 -- 172.21.15.83:0/2879398751 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fcde4021090 con 0x7fcdf4073f90 2023-11-26T07:14:11.277 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:11.277 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:11.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.276+0000 7fcdf97c6700 1 -- 172.21.15.83:0/2879398751 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcde006b110 msgr2=0x7fcde006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:11.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.276+0000 7fcdf97c6700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcde006b110 0x7fcde006d5d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcde8005910 tx=0x7fcde8009040 comp rx=0 tx=0).stop 2023-11-26T07:14:11.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.276+0000 7fcdf97c6700 1 -- 172.21.15.83:0/2879398751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf4073f90 msgr2=0x7fcdf414c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:11.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.276+0000 7fcdf97c6700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf4073f90 0x7fcdf414c810 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcde400e840 tx=0x7fcde400ec00 comp rx=0 tx=0).stop 2023-11-26T07:14:11.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.276+0000 7fcdf97c6700 1 -- 172.21.15.83:0/2879398751 shutdown_connections 2023-11-26T07:14:11.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.276+0000 7fcdf97c6700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcde006b110 0x7fcde006d5d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:11.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.276+0000 7fcdf97c6700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf415deb0 0x7fcdf41f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:11.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.276+0000 7fcdf97c6700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf406ed40 0x7fcdf41f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:11.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.276+0000 7fcdf97c6700 1 --2- 172.21.15.83:0/2879398751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf4073f90 0x7fcdf414c810 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:11.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.276+0000 7fcdf97c6700 1 -- 172.21.15.83:0/2879398751 >> 172.21.15.83:0/2879398751 conn(0x7fcdf40d0ba0 msgr2=0x7fcdf406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:11.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.276+0000 7fcdf97c6700 1 -- 172.21.15.83:0/2879398751 shutdown_connections 2023-11-26T07:14:11.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:11.277+0000 7fcdf97c6700 1 -- 172.21.15.83:0/2879398751 wait complete. 2023-11-26T07:14:11.280 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:11.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:14:11.430 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":56.443653849,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.310577950043108,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:14:11.431 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.310577950043108 }, "state": "open", "uptime": 56.443653849 } ] 2023-11-26T07:14:11.431 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:14:16.432 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:16.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.564+0000 7f8e4a975700 1 Processor -- start 2023-11-26T07:14:16.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.564+0000 7f8e4a975700 1 -- start start 2023-11-26T07:14:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e4a975700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8e4415bdd0 0x7f8e441604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e4a975700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8e440b3370 0x7f8e440b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e4a975700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e440b2030 con 0x7f8e440d1c70 2023-11-26T07:14:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e4a975700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440b2f50 con 0x7f8e440b3370 2023-11-26T07:14:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e4a975700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440b22d0 con 0x7f8e4415bdd0 2023-11-26T07:14:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e437fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8e440b3370 0x7f8e440b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e48f12700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8e440d1c70 0x7f8e440ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57802/0 (socket says 172.21.15.83:57802) 2023-11-26T07:14:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e48f12700 1 -- 172.21.15.83:0/3180915203 learned_addr learned my addr 172.21.15.83:0/3180915203 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e43fff700 1 --2- 172.21.15.83:0/3180915203 >> v2:172.21.15.203:3300/0 conn(0x7f8e4415bdd0 0x7f8e441604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e437fe700 1 -- 172.21.15.83:0/3180915203 >> v2:172.21.15.203:3300/0 conn(0x7f8e4415bdd0 msgr2=0x7f8e441604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e437fe700 1 --2- 172.21.15.83:0/3180915203 >> v2:172.21.15.203:3300/0 conn(0x7f8e4415bdd0 0x7f8e441604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e437fe700 1 -- 172.21.15.83:0/3180915203 >> v1:172.21.15.83:6789/0 conn(0x7f8e440d1c70 legacy=0x7f8e440ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e437fe700 1 -- 172.21.15.83:0/3180915203 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440b25b0 con 0x7f8e440b3370 2023-11-26T07:14:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.565+0000 7f8e42ffd700 1 -- 172.21.15.83:0/3180915203 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692152837 0 0) 0x7f8e440b2030 con 0x7f8e440d1c70 2023-11-26T07:14:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e437fe700 1 --2- 172.21.15.83:0/3180915203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e440b3370 0x7f8e440b1af0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8e4406ec10 tx=0x7f8e34009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72c8be29e79163c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e42ffd700 1 -- 172.21.15.83:0/3180915203 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34011830 con 0x7f8e440b3370 2023-11-26T07:14:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e42ffd700 1 -- 172.21.15.83:0/3180915203 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e34011990 con 0x7f8e440b3370 2023-11-26T07:14:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e42ffd700 1 -- 172.21.15.83:0/3180915203 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34011d20 con 0x7f8e440b3370 2023-11-26T07:14:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e4a975700 1 -- 172.21.15.83:0/3180915203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e440b3370 msgr2=0x7f8e440b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e4a975700 1 --2- 172.21.15.83:0/3180915203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e440b3370 0x7f8e440b1af0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8e4406ec10 tx=0x7f8e34009b20 comp rx=0 tx=0).stop 2023-11-26T07:14:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e4a975700 1 -- 172.21.15.83:0/3180915203 shutdown_connections 2023-11-26T07:14:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e4a975700 1 --2- 172.21.15.83:0/3180915203 >> v2:172.21.15.203:3300/0 conn(0x7f8e4415bdd0 0x7f8e441604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e4a975700 1 --2- 172.21.15.83:0/3180915203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e440b3370 0x7f8e440b1af0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e4a975700 1 -- 172.21.15.83:0/3180915203 >> 172.21.15.83:0/3180915203 conn(0x7f8e440731a0 msgr2=0x7f8e4409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e4a975700 1 -- 172.21.15.83:0/3180915203 shutdown_connections 2023-11-26T07:14:16.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.566+0000 7f8e4a975700 1 -- 172.21.15.83:0/3180915203 wait complete. 2023-11-26T07:14:16.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.567+0000 7f8e4a975700 1 Processor -- start 2023-11-26T07:14:16.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.568+0000 7f8e4a975700 1 -- start start 2023-11-26T07:14:16.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.568+0000 7f8e4a975700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e440b3370 0x7f8e44158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:16.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.568+0000 7f8e4a975700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e440d1c70 0x7f8e440c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:16.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.568+0000 7f8e4a975700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e4415bdd0 0x7f8e440cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:16.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.568+0000 7f8e4a975700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440c5a90 con 0x7f8e440d1c70 2023-11-26T07:14:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.568+0000 7f8e4a975700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e441593e0 con 0x7f8e440b3370 2023-11-26T07:14:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.568+0000 7f8e4a975700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440b7020 con 0x7f8e4415bdd0 2023-11-26T07:14:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.568+0000 7f8e437fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e440d1c70 0x7f8e440c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e437fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e440d1c70 0x7f8e440c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37766/0 (socket says 172.21.15.83:37766) 2023-11-26T07:14:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e437fe700 1 -- 172.21.15.83:0/328931986 learned_addr learned my addr 172.21.15.83:0/328931986 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e43fff700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e440b3370 0x7f8e44158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e437fe700 1 -- 172.21.15.83:0/328931986 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e4415bdd0 msgr2=0x7f8e440cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e437fe700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e4415bdd0 0x7f8e440cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e437fe700 1 -- 172.21.15.83:0/328931986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e440b3370 msgr2=0x7f8e44158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e437fe700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e440b3370 0x7f8e44158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e437fe700 1 -- 172.21.15.83:0/328931986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e340094a0 con 0x7f8e440d1c70 2023-11-26T07:14:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e437fe700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e440d1c70 0x7f8e440c8060 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8e340090b0 tx=0x7f8e34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e40ff9700 1 -- 172.21.15.83:0/328931986 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e340036a0 con 0x7f8e440d1c70 2023-11-26T07:14:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e40ff9700 1 -- 172.21.15.83:0/328931986 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3401f070 con 0x7f8e440d1c70 2023-11-26T07:14:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e4a975700 1 -- 172.21.15.83:0/328931986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e440b7300 con 0x7f8e440d1c70 2023-11-26T07:14:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e40ff9700 1 -- 172.21.15.83:0/328931986 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34003a60 con 0x7f8e440d1c70 2023-11-26T07:14:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.569+0000 7f8e4a975700 1 -- 172.21.15.83:0/328931986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e440c85a0 con 0x7f8e440d1c70 2023-11-26T07:14:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.570+0000 7f8e4a975700 1 -- 172.21.15.83:0/328931986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e440c57f0 con 0x7f8e440d1c70 2023-11-26T07:14:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.571+0000 7f8e40ff9700 1 -- 172.21.15.83:0/328931986 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8e34003bc0 con 0x7f8e440d1c70 2023-11-26T07:14:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.571+0000 7f8e40ff9700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8e3006b110 0x7f8e3006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.571+0000 7f8e40ff9700 1 -- 172.21.15.83:0/328931986 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8e34021070 con 0x7f8e440d1c70 2023-11-26T07:14:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.571+0000 7f8e43fff700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8e3006b110 0x7f8e3006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.571+0000 7f8e43fff700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8e3006b110 0x7f8e3006d5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8e441f5180 tx=0x7f8e38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.575+0000 7f8e40ff9700 1 -- 172.21.15.83:0/328931986 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e440c57f0 con 0x7f8e440d1c70 2023-11-26T07:14:16.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.806+0000 7f8e4a975700 1 -- 172.21.15.83:0/328931986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e440c57f0 con 0x7f8e440d1c70 2023-11-26T07:14:16.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.807+0000 7f8e40ff9700 1 -- 172.21.15.83:0/328931986 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f8e440c57f0 con 0x7f8e440d1c70 2023-11-26T07:14:16.808 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:16.809 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 -- 172.21.15.83:0/328931986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8e3006b110 msgr2=0x7f8e3006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8e3006b110 0x7f8e3006d5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8e441f5180 tx=0x7f8e38008040 comp rx=0 tx=0).stop 2023-11-26T07:14:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 -- 172.21.15.83:0/328931986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e440d1c70 msgr2=0x7f8e440c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e440d1c70 0x7f8e440c8060 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8e340090b0 tx=0x7f8e34004750 comp rx=0 tx=0).stop 2023-11-26T07:14:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 -- 172.21.15.83:0/328931986 shutdown_connections 2023-11-26T07:14:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8e3006b110 0x7f8e3006d5d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e4415bdd0 0x7f8e440cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e440b3370 0x7f8e44158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 --2- 172.21.15.83:0/328931986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e440d1c70 0x7f8e440c8060 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 -- 172.21.15.83:0/328931986 >> 172.21.15.83:0/328931986 conn(0x7f8e440731a0 msgr2=0x7f8e440cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 -- 172.21.15.83:0/328931986 shutdown_connections 2023-11-26T07:14:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:16.809+0000 7f8e4a975700 1 -- 172.21.15.83:0/328931986 wait complete. 2023-11-26T07:14:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:16.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:14:16.961 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":61.975521966000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.065333706731941,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:14:16.962 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.06533370673194 }, "state": "open", "uptime": 61.975521966 } ] 2023-11-26T07:14:16.962 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:14:21.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.095+0000 7fbded139700 1 Processor -- start 2023-11-26T07:14:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.095+0000 7fbded139700 1 -- start start 2023-11-26T07:14:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.096+0000 7fbded139700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde8158670 0x7fbde80622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.096+0000 7fbded139700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbde80c9fc0 0x7fbde80ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.096+0000 7fbded139700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde80cfff0 con 0x7fbde8158a50 2023-11-26T07:14:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.096+0000 7fbded139700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde80cfd10 con 0x7fbde80c9fc0 2023-11-26T07:14:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.096+0000 7fbded139700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8161f50 con 0x7fbde8158670 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.096+0000 7fbde659c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbde8158a50 0x7fbde80c98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57814/0 (socket says 172.21.15.83:57814) 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.096+0000 7fbde659c700 1 -- 172.21.15.83:0/3904951612 learned_addr learned my addr 172.21.15.83:0/3904951612 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde759e700 1 --2- 172.21.15.83:0/3904951612 >> v2:172.21.15.203:3300/0 conn(0x7fbde80c9fc0 0x7fbde80ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde6d9d700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde8158670 0x7fbde80622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565309195 0 0) 0x7fbde80cfff0 con 0x7fbde8158a50 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd00035b0 con 0x7fbde8158a50 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1378797152 0 0) 0x7fbdd00035b0 con 0x7fbde8158a50 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde8158670 msgr2=0x7fbde80622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde8158670 0x7fbde80622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 >> v2:172.21.15.203:3300/0 conn(0x7fbde80c9fc0 msgr2=0x7fbde80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 --2- 172.21.15.83:0/3904951612 >> v2:172.21.15.203:3300/0 conn(0x7fbde80c9fc0 0x7fbde80ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde8162230 con 0x7fbde8158a50 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fbddc001690 con 0x7fbde8158a50 2023-11-26T07:14:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 >> v1:172.21.15.83:6789/0 conn(0x7fbde8158a50 legacy=0x7fbde80c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdd0003a00 0x7fbdd0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde8158670 0x7fbdd0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdd0004d90 0x7fbdd0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80cfd10 con 0x7fbdd0004d90 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8161f50 con 0x7fbdd0003a00 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd000ba10 con 0x7fbde8158670 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde759e700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdd0004d90 0x7fbdd0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde6d9d700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdd0003a00 0x7fbdd0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde659c700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde8158670 0x7fbdd0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde759e700 1 -- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde8158670 msgr2=0x7fbdd0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde759e700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde8158670 0x7fbdd0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde759e700 1 -- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdd0003a00 msgr2=0x7fbdd0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde759e700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdd0003a00 0x7fbdd0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde759e700 1 -- 172.21.15.83:0/3904951612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00035b0 con 0x7fbdd0004d90 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde6d9d700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbdd0003a00 0x7fbdd0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.097+0000 7fbde759e700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdd0004d90 0x7fbdd0005210 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009fc0 tx=0x7fbdd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a3a19424fc0e61 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.098+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde000ec30 con 0x7fbdd0004d90 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.098+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde000f040 con 0x7fbdd0004d90 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.098+0000 7fbde5d9b700 1 -- 172.21.15.83:0/3904951612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0014480 con 0x7fbdd0004d90 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.098+0000 7fbded139700 1 -- 172.21.15.83:0/3904951612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdd0004d90 msgr2=0x7fbdd0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.098+0000 7fbded139700 1 --2- 172.21.15.83:0/3904951612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdd0004d90 0x7fbdd0005210 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009fc0 tx=0x7fbdd0005880 comp rx=0 tx=0).stop 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.098+0000 7fbde759e700 1 -- 172.21.15.83:0/3904951612 reap_dead start 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.098+0000 7fbded139700 1 -- 172.21.15.83:0/3904951612 shutdown_connections 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.098+0000 7fbded139700 1 -- 172.21.15.83:0/3904951612 >> 172.21.15.83:0/3904951612 conn(0x7fbde80b57b0 msgr2=0x7fbde80b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.098+0000 7fbded139700 1 -- 172.21.15.83:0/3904951612 shutdown_connections 2023-11-26T07:14:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.098+0000 7fbded139700 1 -- 172.21.15.83:0/3904951612 wait complete. 2023-11-26T07:14:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.100+0000 7fbded139700 1 Processor -- start 2023-11-26T07:14:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.100+0000 7fbded139700 1 -- start start 2023-11-26T07:14:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.101+0000 7fbded139700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde80cac20 0x7fbde80ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.101+0000 7fbded139700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80ad7e0 0x7fbde80cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.101+0000 7fbded139700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbde80cb540 0x7fbde80b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.101+0000 7fbded139700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80add60 con 0x7fbde80ad7e0 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.101+0000 7fbded139700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80ae040 con 0x7fbde80cb540 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.101+0000 7fbded139700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80ae320 con 0x7fbde80cac20 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.101+0000 7fbde659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80ad7e0 0x7fbde80cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.101+0000 7fbde659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80ad7e0 0x7fbde80cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37786/0 (socket says 172.21.15.83:37786) 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.101+0000 7fbde659c700 1 -- 172.21.15.83:0/625559863 learned_addr learned my addr 172.21.15.83:0/625559863 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.101+0000 7fbde6d9d700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde80cac20 0x7fbde80ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.101+0000 7fbde759e700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbde80cb540 0x7fbde80b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.102+0000 7fbde659c700 1 -- 172.21.15.83:0/625559863 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde80cac20 msgr2=0x7fbde80ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.102+0000 7fbde659c700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde80cac20 0x7fbde80ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.102+0000 7fbde659c700 1 -- 172.21.15.83:0/625559863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbde80cb540 msgr2=0x7fbde80b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.102+0000 7fbde659c700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbde80cb540 0x7fbde80b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.102+0000 7fbde659c700 1 -- 172.21.15.83:0/625559863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80ae600 con 0x7fbde80ad7e0 2023-11-26T07:14:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.102+0000 7fbde659c700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80ad7e0 0x7fbde80cb000 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbddc001220 tx=0x7fbddc00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.102+0000 7fbdcf7fe700 1 -- 172.21.15.83:0/625559863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc012590 con 0x7fbde80ad7e0 2023-11-26T07:14:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.102+0000 7fbdcf7fe700 1 -- 172.21.15.83:0/625559863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc012710 con 0x7fbde80ad7e0 2023-11-26T07:14:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.102+0000 7fbded139700 1 -- 172.21.15.83:0/625559863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde80ae8e0 con 0x7fbde80ad7e0 2023-11-26T07:14:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.102+0000 7fbdcf7fe700 1 -- 172.21.15.83:0/625559863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc012ac0 con 0x7fbde80ad7e0 2023-11-26T07:14:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.102+0000 7fbded139700 1 -- 172.21.15.83:0/625559863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde8155890 con 0x7fbde80ad7e0 2023-11-26T07:14:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.103+0000 7fbded139700 1 -- 172.21.15.83:0/625559863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde8068670 con 0x7fbde80ad7e0 2023-11-26T07:14:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.103+0000 7fbdcf7fe700 1 -- 172.21.15.83:0/625559863 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fbddc012c20 con 0x7fbde80ad7e0 2023-11-26T07:14:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.104+0000 7fbdcf7fe700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdd406b110 0x7fbdd406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.104+0000 7fbdcf7fe700 1 -- 172.21.15.83:0/625559863 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbddc0896c0 con 0x7fbde80ad7e0 2023-11-26T07:14:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.104+0000 7fbde6d9d700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdd406b110 0x7fbdd406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.104+0000 7fbde6d9d700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdd406b110 0x7fbdd406d5d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009b40 tx=0x7fbdd8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:22.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.108+0000 7fbdcf7fe700 1 -- 172.21.15.83:0/625559863 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbddc054490 con 0x7fbde80ad7e0 2023-11-26T07:14:22.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.339+0000 7fbded139700 1 -- 172.21.15.83:0/625559863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbde80689c0 con 0x7fbde80ad7e0 2023-11-26T07:14:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.340+0000 7fbdcf7fe700 1 -- 172.21.15.83:0/625559863 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fbddc012ed0 con 0x7fbde80ad7e0 2023-11-26T07:14:22.341 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:22.341 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 -- 172.21.15.83:0/625559863 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdd406b110 msgr2=0x7fbdd406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdd406b110 0x7fbdd406d5d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009b40 tx=0x7fbdd8008040 comp rx=0 tx=0).stop 2023-11-26T07:14:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 -- 172.21.15.83:0/625559863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80ad7e0 msgr2=0x7fbde80cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80ad7e0 0x7fbde80cb000 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbddc001220 tx=0x7fbddc00ebc0 comp rx=0 tx=0).stop 2023-11-26T07:14:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 -- 172.21.15.83:0/625559863 shutdown_connections 2023-11-26T07:14:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbde80cb540 0x7fbde80b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdd406b110 0x7fbdd406d5d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde80cac20 0x7fbde80ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 --2- 172.21.15.83:0/625559863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80ad7e0 0x7fbde80cb000 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 -- 172.21.15.83:0/625559863 >> 172.21.15.83:0/625559863 conn(0x7fbde80b57b0 msgr2=0x7fbde806ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 -- 172.21.15.83:0/625559863 shutdown_connections 2023-11-26T07:14:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:22.342+0000 7fbded139700 1 -- 172.21.15.83:0/625559863 wait complete. 2023-11-26T07:14:22.345 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:22.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:14:22.495 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":67.508390058000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.823160574384733,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:14:22.495 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.823160574384733 }, "state": "open", "uptime": 67.508390058 } ] 2023-11-26T07:14:22.496 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:14:27.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:14:27.028 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:14:27.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.629+0000 7f856ba87700 1 Processor -- start 2023-11-26T07:14:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.629+0000 7f856ba87700 1 -- start start 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f856ba87700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f856415e080 0x7f85640b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f856ba87700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f856406c710 0x7f85640afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f856ba87700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85640ae430 con 0x7f85640af4d0 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f856ba87700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85640af0b0 con 0x7f856415e080 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f856ba87700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85640b05a0 con 0x7f856406c710 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f8569823700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f856415e080 0x7f85640b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f8569823700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f856415e080 0x7f85640b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37792/0 (socket says 172.21.15.83:37792) 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f8569823700 1 -- 172.21.15.83:0/2781913550 learned_addr learned my addr 172.21.15.83:0/2781913550 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f8569823700 1 -- 172.21.15.83:0/2781913550 >> v2:172.21.15.203:3300/0 conn(0x7f856406c710 msgr2=0x7f85640afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f856a024700 1 --2- 172.21.15.83:0/2781913550 >> v2:172.21.15.203:3300/0 conn(0x7f856406c710 0x7f85640afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f8569823700 1 --2- 172.21.15.83:0/2781913550 >> v2:172.21.15.203:3300/0 conn(0x7f856406c710 0x7f85640afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f8569823700 1 -- 172.21.15.83:0/2781913550 >> v1:172.21.15.203:6789/0 conn(0x7f85640af4d0 legacy=0x7f856406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f8569823700 1 -- 172.21.15.83:0/2781913550 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85640ae710 con 0x7f856415e080 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f856a024700 1 --2- 172.21.15.83:0/2781913550 >> v2:172.21.15.203:3300/0 conn(0x7f856406c710 0x7f85640afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f8569823700 1 --2- 172.21.15.83:0/2781913550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f856415e080 0x7f85640b5050 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f856406e060 tx=0x7f8560009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26755e932d533a3d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.630+0000 7f8568821700 1 -- 172.21.15.83:0/2781913550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560004230 con 0x7f856415e080 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.631+0000 7f8568821700 1 -- 172.21.15.83:0/2781913550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8560004390 con 0x7f856415e080 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.631+0000 7f8568821700 1 -- 172.21.15.83:0/2781913550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85600036a0 con 0x7f856415e080 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.631+0000 7f856ba87700 1 -- 172.21.15.83:0/2781913550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f856415e080 msgr2=0x7f85640b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.631+0000 7f856ba87700 1 --2- 172.21.15.83:0/2781913550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f856415e080 0x7f85640b5050 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f856406e060 tx=0x7f8560009c00 comp rx=0 tx=0).stop 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.631+0000 7f856ba87700 1 -- 172.21.15.83:0/2781913550 shutdown_connections 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.631+0000 7f856ba87700 1 --2- 172.21.15.83:0/2781913550 >> v2:172.21.15.203:3300/0 conn(0x7f856406c710 0x7f85640afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.631+0000 7f856ba87700 1 --2- 172.21.15.83:0/2781913550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f856415e080 0x7f85640b5050 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.631+0000 7f856ba87700 1 -- 172.21.15.83:0/2781913550 >> 172.21.15.83:0/2781913550 conn(0x7f8564071b90 msgr2=0x7f8564071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.631+0000 7f856ba87700 1 -- 172.21.15.83:0/2781913550 shutdown_connections 2023-11-26T07:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.631+0000 7f856ba87700 1 -- 172.21.15.83:0/2781913550 wait complete. 2023-11-26T07:14:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.632+0000 7f856ba87700 1 Processor -- start 2023-11-26T07:14:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.632+0000 7f856ba87700 1 -- start start 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856ba87700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f856406c710 0x7f85641f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856ba87700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85640af4d0 0x7f85641f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856ba87700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8564159b20 0x7f8564159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856ba87700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85641f72e0 con 0x7f8564159b20 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856ba87700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85641583c0 con 0x7f856406c710 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856ba87700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85641586a0 con 0x7f85640af4d0 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856a024700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8564159b20 0x7f8564159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856a024700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8564159b20 0x7f8564159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37802/0 (socket says 172.21.15.83:37802) 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856a024700 1 -- 172.21.15.83:0/4294242983 learned_addr learned my addr 172.21.15.83:0/4294242983 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f8569823700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f856406c710 0x7f85641f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f8569022700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85640af4d0 0x7f85641f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856a024700 1 -- 172.21.15.83:0/4294242983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85640af4d0 msgr2=0x7f85641f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856a024700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85640af4d0 0x7f85641f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856a024700 1 -- 172.21.15.83:0/4294242983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f856406c710 msgr2=0x7f85641f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856a024700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f856406c710 0x7f85641f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f856a024700 1 -- 172.21.15.83:0/4294242983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560009550 con 0x7f8564159b20 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.633+0000 7f8569022700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85640af4d0 0x7f85641f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.634+0000 7f856a024700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8564159b20 0x7f8564159fb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f855c00ce80 tx=0x7f855c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.634+0000 7f855a7fc700 1 -- 172.21.15.83:0/4294242983 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c00ae20 con 0x7f8564159b20 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.634+0000 7f855a7fc700 1 -- 172.21.15.83:0/4294242983 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855c015440 con 0x7f8564159b20 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.634+0000 7f856ba87700 1 -- 172.21.15.83:0/4294242983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8564158980 con 0x7f8564159b20 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.634+0000 7f855a7fc700 1 -- 172.21.15.83:0/4294242983 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c0157f0 con 0x7f8564159b20 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.634+0000 7f856ba87700 1 -- 172.21.15.83:0/4294242983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8564158f40 con 0x7f8564159b20 2023-11-26T07:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.635+0000 7f856ba87700 1 -- 172.21.15.83:0/4294242983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85640753f0 con 0x7f8564159b20 2023-11-26T07:14:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.635+0000 7f855a7fc700 1 -- 172.21.15.83:0/4294242983 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f855c00d430 con 0x7f8564159b20 2023-11-26T07:14:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.635+0000 7f855a7fc700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f854806b110 0x7f854806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.636+0000 7f855a7fc700 1 -- 172.21.15.83:0/4294242983 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f855c08a010 con 0x7f8564159b20 2023-11-26T07:14:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.636+0000 7f8569823700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f854806b110 0x7f854806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.636+0000 7f8569823700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f854806b110 0x7f854806d5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f856406d350 tx=0x7f8560008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:27.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.640+0000 7f855a7fc700 1 -- 172.21.15.83:0/4294242983 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f855c054d20 con 0x7f8564159b20 2023-11-26T07:14:27.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.871+0000 7f856ba87700 1 -- 172.21.15.83:0/4294242983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8564067f60 con 0x7f8564159b20 2023-11-26T07:14:27.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.871+0000 7f855a7fc700 1 -- 172.21.15.83:0/4294242983 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f855c01a090 con 0x7f8564159b20 2023-11-26T07:14:27.873 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:27.873 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 -- 172.21.15.83:0/4294242983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f854806b110 msgr2=0x7f854806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f854806b110 0x7f854806d5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f856406d350 tx=0x7f8560008030 comp rx=0 tx=0).stop 2023-11-26T07:14:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 -- 172.21.15.83:0/4294242983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8564159b20 msgr2=0x7f8564159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8564159b20 0x7f8564159fb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f855c00ce80 tx=0x7f855c00cf60 comp rx=0 tx=0).stop 2023-11-26T07:14:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 -- 172.21.15.83:0/4294242983 shutdown_connections 2023-11-26T07:14:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8564159b20 0x7f8564159fb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f856406c710 0x7f85641f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:27.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f854806b110 0x7f854806d5d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 --2- 172.21.15.83:0/4294242983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85640af4d0 0x7f85641f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 -- 172.21.15.83:0/4294242983 >> 172.21.15.83:0/4294242983 conn(0x7f8564071b90 msgr2=0x7f856406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 -- 172.21.15.83:0/4294242983 shutdown_connections 2023-11-26T07:14:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:27.873+0000 7f856ba87700 1 -- 172.21.15.83:0/4294242983 wait complete. 2023-11-26T07:14:27.876 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:27.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:14:28.025 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":73.039258195000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.584149466899085,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:14:28.026 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.584149466899085 }, "state": "open", "uptime": 73.039258195 } ] 2023-11-26T07:14:28.026 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:14:33.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:33.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.158+0000 7fbaf1884700 1 Processor -- start 2023-11-26T07:14:33.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.158+0000 7fbaf1884700 1 -- start start 2023-11-26T07:14:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaf1884700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbaec15e1e0 0x7fbaec0d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaf1884700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbaec0d3a00 0x7fbaec0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaf1884700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaec0d1200 con 0x7fbaec0d0db0 2023-11-26T07:14:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaf1884700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaec0d3150 con 0x7fbaec0d3a00 2023-11-26T07:14:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaf1884700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaec15e5c0 con 0x7fbaec15e1e0 2023-11-26T07:14:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaea7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbaec0d3a00 0x7fbaec0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaea7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbaec0d3a00 0x7fbaec0d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37810/0 (socket says 172.21.15.83:37810) 2023-11-26T07:14:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaea7fc700 1 -- 172.21.15.83:0/1559521676 learned_addr learned my addr 172.21.15.83:0/1559521676 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaea7fc700 1 -- 172.21.15.83:0/1559521676 >> v2:172.21.15.203:3300/0 conn(0x7fbaec15e1e0 msgr2=0x7fbaec0d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaeaffd700 1 --2- 172.21.15.83:0/1559521676 >> v2:172.21.15.203:3300/0 conn(0x7fbaec15e1e0 0x7fbaec0d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaea7fc700 1 --2- 172.21.15.83:0/1559521676 >> v2:172.21.15.203:3300/0 conn(0x7fbaec15e1e0 0x7fbaec0d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaea7fc700 1 -- 172.21.15.83:0/1559521676 >> v1:172.21.15.203:6789/0 conn(0x7fbaec0d0db0 legacy=0x7fbaec06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaea7fc700 1 -- 172.21.15.83:0/1559521676 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec0d20b0 con 0x7fbaec0d3a00 2023-11-26T07:14:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbaea7fc700 1 --2- 172.21.15.83:0/1559521676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaec0d3a00 0x7fbaec0d0870 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0d0840 tx=0x7fbae0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b9f87c85e9e37a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbae9ffb700 1 -- 172.21.15.83:0/1559521676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0004230 con 0x7fbaec0d3a00 2023-11-26T07:14:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbae9ffb700 1 -- 172.21.15.83:0/1559521676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae0004390 con 0x7fbaec0d3a00 2023-11-26T07:14:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.159+0000 7fbae9ffb700 1 -- 172.21.15.83:0/1559521676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae00036a0 con 0x7fbaec0d3a00 2023-11-26T07:14:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.160+0000 7fbaf1884700 1 -- 172.21.15.83:0/1559521676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaec0d3a00 msgr2=0x7fbaec0d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.160+0000 7fbaf1884700 1 --2- 172.21.15.83:0/1559521676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaec0d3a00 0x7fbaec0d0870 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0d0840 tx=0x7fbae0009bd0 comp rx=0 tx=0).stop 2023-11-26T07:14:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.160+0000 7fbaf1884700 1 -- 172.21.15.83:0/1559521676 shutdown_connections 2023-11-26T07:14:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.160+0000 7fbaf1884700 1 --2- 172.21.15.83:0/1559521676 >> v2:172.21.15.203:3300/0 conn(0x7fbaec15e1e0 0x7fbaec0d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.160+0000 7fbaf1884700 1 --2- 172.21.15.83:0/1559521676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaec0d3a00 0x7fbaec0d0870 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.160+0000 7fbaf1884700 1 -- 172.21.15.83:0/1559521676 >> 172.21.15.83:0/1559521676 conn(0x7fbaec0b5b50 msgr2=0x7fbaec0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.160+0000 7fbaf1884700 1 -- 172.21.15.83:0/1559521676 shutdown_connections 2023-11-26T07:14:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.160+0000 7fbaf1884700 1 -- 172.21.15.83:0/1559521676 wait complete. 2023-11-26T07:14:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.161+0000 7fbaf1884700 1 Processor -- start 2023-11-26T07:14:33.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.162+0000 7fbaf1884700 1 -- start start 2023-11-26T07:14:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.162+0000 7fbaf1884700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaec0d0db0 0x7fbaec144920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.162+0000 7fbaf1884700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaec0d3a00 0x7fbaec144e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.162+0000 7fbaf1884700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaec15e1e0 0x7fbaec1453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.162+0000 7fbaf1884700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec14e7b0 con 0x7fbaec0d3a00 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.162+0000 7fbaf1884700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec14eb90 con 0x7fbaec0d0db0 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.162+0000 7fbaf1884700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec0cafd0 con 0x7fbaec15e1e0 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.162+0000 7fbaea7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaec0d3a00 0x7fbaec144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.162+0000 7fbaea7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaec0d3a00 0x7fbaec144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37814/0 (socket says 172.21.15.83:37814) 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.162+0000 7fbaea7fc700 1 -- 172.21.15.83:0/2571187015 learned_addr learned my addr 172.21.15.83:0/2571187015 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbaeaffd700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaec0d0db0 0x7fbaec144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbaeb7fe700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaec15e1e0 0x7fbaec1453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbaea7fc700 1 -- 172.21.15.83:0/2571187015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaec15e1e0 msgr2=0x7fbaec1453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbaea7fc700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaec15e1e0 0x7fbaec1453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbaea7fc700 1 -- 172.21.15.83:0/2571187015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaec0d0db0 msgr2=0x7fbaec144920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbaea7fc700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaec0d0db0 0x7fbaec144920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbaea7fc700 1 -- 172.21.15.83:0/2571187015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae0009550 con 0x7fbaec0d3a00 2023-11-26T07:14:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbaea7fc700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaec0d3a00 0x7fbaec144e60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbae0004000 tx=0x7fbae0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbad3fff700 1 -- 172.21.15.83:0/2571187015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0003d80 con 0x7fbaec0d3a00 2023-11-26T07:14:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbad3fff700 1 -- 172.21.15.83:0/2571187015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae001f070 con 0x7fbaec0d3a00 2023-11-26T07:14:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbaf1884700 1 -- 172.21.15.83:0/2571187015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaec0cb2b0 con 0x7fbaec0d3a00 2023-11-26T07:14:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbad3fff700 1 -- 172.21.15.83:0/2571187015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0011870 con 0x7fbaec0d3a00 2023-11-26T07:14:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.163+0000 7fbaf1884700 1 -- 172.21.15.83:0/2571187015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaec0cb870 con 0x7fbaec0d3a00 2023-11-26T07:14:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.164+0000 7fbaf1884700 1 -- 172.21.15.83:0/2571187015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaec0751e0 con 0x7fbaec0d3a00 2023-11-26T07:14:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.164+0000 7fbad3fff700 1 -- 172.21.15.83:0/2571187015 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fbae0021030 con 0x7fbaec0d3a00 2023-11-26T07:14:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.165+0000 7fbad3fff700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbad806b110 0x7fbad806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.165+0000 7fbad3fff700 1 -- 172.21.15.83:0/2571187015 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbae008ae90 con 0x7fbaec0d3a00 2023-11-26T07:14:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.165+0000 7fbaeaffd700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbad806b110 0x7fbad806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.165+0000 7fbaeaffd700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbad806b110 0x7fbad806d5d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00a950 tx=0x7fbadc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.169+0000 7fbad3fff700 1 -- 172.21.15.83:0/2571187015 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbae0055c70 con 0x7fbaec0d3a00 2023-11-26T07:14:33.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.403+0000 7fbaf1884700 1 -- 172.21.15.83:0/2571187015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbaec0d1200 con 0x7fbaec0d3a00 2023-11-26T07:14:33.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.404+0000 7fbad3fff700 1 -- 172.21.15.83:0/2571187015 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fbaec0d1200 con 0x7fbaec0d3a00 2023-11-26T07:14:33.406 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:33.407 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:33.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 -- 172.21.15.83:0/2571187015 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbad806b110 msgr2=0x7fbad806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:33.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbad806b110 0x7fbad806d5d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00a950 tx=0x7fbadc008040 comp rx=0 tx=0).stop 2023-11-26T07:14:33.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 -- 172.21.15.83:0/2571187015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaec0d3a00 msgr2=0x7fbaec144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:33.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaec0d3a00 0x7fbaec144e60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbae0004000 tx=0x7fbae0004750 comp rx=0 tx=0).stop 2023-11-26T07:14:33.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 -- 172.21.15.83:0/2571187015 shutdown_connections 2023-11-26T07:14:33.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbad806b110 0x7fbad806d5d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:33.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbaec15e1e0 0x7fbaec1453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:33.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbaec0d0db0 0x7fbaec144920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:33.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 --2- 172.21.15.83:0/2571187015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaec0d3a00 0x7fbaec144e60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:33.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 -- 172.21.15.83:0/2571187015 >> 172.21.15.83:0/2571187015 conn(0x7fbaec0b5b50 msgr2=0x7fbaec15a910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:33.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 -- 172.21.15.83:0/2571187015 shutdown_connections 2023-11-26T07:14:33.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:33.406+0000 7fbaf1884700 1 -- 172.21.15.83:0/2571187015 wait complete. 2023-11-26T07:14:33.410 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:33.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:14:33.561 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":5844,"entity":{"name":{"type":"client","num":5844},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2509513349}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":78.574126234999994,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.348003685841146,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5844 172.21.15.83:0/2509513349","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:14:33.561 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2509513349, "type": "any" }, "name": { "num": 5844, "type": "client" } }, "id": 5844, "inst": "client.5844 172.21.15.83:0/2509513349", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.348003685841146 }, "state": "open", "uptime": 78.574126235 } ] 2023-11-26T07:14:33.561 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:14:38.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:38.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.693+0000 7fc8f2841700 1 Processor -- start 2023-11-26T07:14:38.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.693+0000 7fc8f2841700 1 -- start start 2023-11-26T07:14:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8f2841700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8ec0b0fa0 0x7fc8ec0afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8f2841700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ec0ca190 0x7fc8ec0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8f2841700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ec0ca620 con 0x7fc8ec0d0c40 2023-11-26T07:14:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8f2841700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ec0d10a0 con 0x7fc8ec0b0fa0 2023-11-26T07:14:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8f2841700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec0622a0 con 0x7fc8ec0ca190 2023-11-26T07:14:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8ebfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8ec0b0fa0 0x7fc8ec0afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8ebfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8ec0b0fa0 0x7fc8ec0afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37822/0 (socket says 172.21.15.83:37822) 2023-11-26T07:14:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8ebfff700 1 -- 172.21.15.83:0/2522931676 learned_addr learned my addr 172.21.15.83:0/2522931676 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8ebfff700 1 -- 172.21.15.83:0/2522931676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ec0ca190 msgr2=0x7fc8ec0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8f0dde700 1 --2- 172.21.15.83:0/2522931676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ec0ca190 0x7fc8ec0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:38.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8ebfff700 1 --2- 172.21.15.83:0/2522931676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ec0ca190 0x7fc8ec0ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8ebfff700 1 -- 172.21.15.83:0/2522931676 >> v1:172.21.15.203:6789/0 conn(0x7fc8ec0d0c40 legacy=0x7fc8ec0b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8ebfff700 1 -- 172.21.15.83:0/2522931676 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec062580 con 0x7fc8ec0b0fa0 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8f0dde700 1 --2- 172.21.15.83:0/2522931676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ec0ca190 0x7fc8ec0ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.694+0000 7fc8ebfff700 1 --2- 172.21.15.83:0/2522931676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec0b0fa0 0x7fc8ec0afc80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00098f0 tx=0x7fc8e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6561f9ea716fdb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.695+0000 7fc8eaffd700 1 -- 172.21.15.83:0/2522931676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0004230 con 0x7fc8ec0b0fa0 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.695+0000 7fc8eaffd700 1 -- 172.21.15.83:0/2522931676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e0004390 con 0x7fc8ec0b0fa0 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.695+0000 7fc8eaffd700 1 -- 172.21.15.83:0/2522931676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00036a0 con 0x7fc8ec0b0fa0 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.695+0000 7fc8f2841700 1 -- 172.21.15.83:0/2522931676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec0b0fa0 msgr2=0x7fc8ec0afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.695+0000 7fc8f2841700 1 --2- 172.21.15.83:0/2522931676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec0b0fa0 0x7fc8ec0afc80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00098f0 tx=0x7fc8e0009c00 comp rx=0 tx=0).stop 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.695+0000 7fc8f2841700 1 -- 172.21.15.83:0/2522931676 shutdown_connections 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.695+0000 7fc8f2841700 1 --2- 172.21.15.83:0/2522931676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ec0ca190 0x7fc8ec0ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:38.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.695+0000 7fc8f2841700 1 --2- 172.21.15.83:0/2522931676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec0b0fa0 0x7fc8ec0afc80 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:38.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.695+0000 7fc8f2841700 1 -- 172.21.15.83:0/2522931676 >> 172.21.15.83:0/2522931676 conn(0x7fc8ec0b57b0 msgr2=0x7fc8ec0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:38.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.695+0000 7fc8f2841700 1 -- 172.21.15.83:0/2522931676 shutdown_connections 2023-11-26T07:14:38.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.695+0000 7fc8f2841700 1 -- 172.21.15.83:0/2522931676 wait complete. 2023-11-26T07:14:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.696+0000 7fc8f2841700 1 Processor -- start 2023-11-26T07:14:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.697+0000 7fc8f2841700 1 -- start start 2023-11-26T07:14:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.697+0000 7fc8f2841700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ec0b0fa0 0x7fc8ec144450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.697+0000 7fc8f2841700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec0ca190 0x7fc8ec1606c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.697+0000 7fc8f2841700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ec0d0c40 0x7fc8ec160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.697+0000 7fc8f2841700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec144cf0 con 0x7fc8ec0ca190 2023-11-26T07:14:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.697+0000 7fc8f2841700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec161280 con 0x7fc8ec0d0c40 2023-11-26T07:14:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.697+0000 7fc8f2841700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec161560 con 0x7fc8ec0b0fa0 2023-11-26T07:14:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.697+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec0ca190 0x7fc8ec1606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.697+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec0ca190 0x7fc8ec1606c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37830/0 (socket says 172.21.15.83:37830) 2023-11-26T07:14:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.697+0000 7fc8eb7fe700 1 -- 172.21.15.83:0/2881352660 learned_addr learned my addr 172.21.15.83:0/2881352660 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8f0dde700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ec0d0c40 0x7fc8ec160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8ebfff700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ec0b0fa0 0x7fc8ec144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8eb7fe700 1 -- 172.21.15.83:0/2881352660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ec0b0fa0 msgr2=0x7fc8ec144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8eb7fe700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ec0b0fa0 0x7fc8ec144450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8eb7fe700 1 -- 172.21.15.83:0/2881352660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ec0d0c40 msgr2=0x7fc8ec160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8eb7fe700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ec0d0c40 0x7fc8ec160c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8eb7fe700 1 -- 172.21.15.83:0/2881352660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e0009550 con 0x7fc8ec0ca190 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8eb7fe700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec0ca190 0x7fc8ec1606c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00b4a0 tx=0x7fc8dc00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8e8ff9700 1 -- 172.21.15.83:0/2881352660 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc011830 con 0x7fc8ec0ca190 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8e8ff9700 1 -- 172.21.15.83:0/2881352660 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc011990 con 0x7fc8ec0ca190 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8f2841700 1 -- 172.21.15.83:0/2881352660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec161840 con 0x7fc8ec0ca190 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8e8ff9700 1 -- 172.21.15.83:0/2881352660 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc011d40 con 0x7fc8ec0ca190 2023-11-26T07:14:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.698+0000 7fc8f2841700 1 -- 172.21.15.83:0/2881352660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec0b3300 con 0x7fc8ec0ca190 2023-11-26T07:14:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.699+0000 7fc8f2841700 1 -- 172.21.15.83:0/2881352660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec067f60 con 0x7fc8ec0ca190 2023-11-26T07:14:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.700+0000 7fc8e8ff9700 1 -- 172.21.15.83:0/2881352660 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fc8dc01a400 con 0x7fc8ec0ca190 2023-11-26T07:14:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.700+0000 7fc8e8ff9700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8d806b110 0x7fc8d806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.700+0000 7fc8e8ff9700 1 -- 172.21.15.83:0/2881352660 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc8dc089680 con 0x7fc8ec0ca190 2023-11-26T07:14:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.700+0000 7fc8ebfff700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8d806b110 0x7fc8d806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.700+0000 7fc8ebfff700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8d806b110 0x7fc8d806d5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009130 tx=0x7fc8e0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.704+0000 7fc8e8ff9700 1 -- 172.21.15.83:0/2881352660 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8dc054390 con 0x7fc8ec0ca190 2023-11-26T07:14:38.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.937+0000 7fc8f2841700 1 -- 172.21.15.83:0/2881352660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8ec06e380 con 0x7fc8ec0ca190 2023-11-26T07:14:38.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.937+0000 7fc8e8ff9700 1 -- 172.21.15.83:0/2881352660 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fc8dc0579b0 con 0x7fc8ec0ca190 2023-11-26T07:14:38.940 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:38.940 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:38.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.939+0000 7fc8f2841700 1 -- 172.21.15.83:0/2881352660 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8d806b110 msgr2=0x7fc8d806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.939+0000 7fc8f2841700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8d806b110 0x7fc8d806d5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009130 tx=0x7fc8e0008030 comp rx=0 tx=0).stop 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.939+0000 7fc8f2841700 1 -- 172.21.15.83:0/2881352660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec0ca190 msgr2=0x7fc8ec1606c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.939+0000 7fc8f2841700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec0ca190 0x7fc8ec1606c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00b4a0 tx=0x7fc8dc00b7b0 comp rx=0 tx=0).stop 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.939+0000 7fc8f2841700 1 -- 172.21.15.83:0/2881352660 shutdown_connections 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.939+0000 7fc8f2841700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ec0d0c40 0x7fc8ec160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.939+0000 7fc8f2841700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8d806b110 0x7fc8d806d5d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.939+0000 7fc8f2841700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ec0b0fa0 0x7fc8ec144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.939+0000 7fc8f2841700 1 --2- 172.21.15.83:0/2881352660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8ec0ca190 0x7fc8ec1606c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.940+0000 7fc8f2841700 1 -- 172.21.15.83:0/2881352660 >> 172.21.15.83:0/2881352660 conn(0x7fc8ec0b57b0 msgr2=0x7fc8ec0ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.940+0000 7fc8f2841700 1 -- 172.21.15.83:0/2881352660 shutdown_connections 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:38.940+0000 7fc8f2841700 1 -- 172.21.15.83:0/2881352660 wait complete. 2023-11-26T07:14:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:38.954 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:14:39.092 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2023-11-26T07:14:39.093 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-11-26T07:14:39.093 DEBUG:tasks.ceph_test_case:wait_until_true: success in 75s and 0 retries 2023-11-26T07:14:39.093 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:14:39.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:14:39.132 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:14:39.177 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:14:39.178 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:14:39.178 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:14:39.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:14:39.195 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:14:39.195 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:14:39.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:14:39.215 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:14:39.215 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:14:39.252 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:14:39.252 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:14:39.252 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:14:39.273 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:14:39.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:39.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.410+0000 7f84aaab0700 1 Processor -- start 2023-11-26T07:14:39.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.410+0000 7f84aaab0700 1 -- start start 2023-11-26T07:14:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84aaab0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84a40b59f0 0x7f84a406db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84aaab0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f84a406e0b0 0x7f84a40b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84aaab0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a40b4820 con 0x7f84a40b5dd0 2023-11-26T07:14:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84aaab0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a40b4b00 con 0x7f84a40b59f0 2023-11-26T07:14:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84aaab0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a40ae9b0 con 0x7f84a406e0b0 2023-11-26T07:14:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84a884c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84a40b59f0 0x7f84a406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84a884c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f84a40b59f0 0x7f84a406db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37836/0 (socket says 172.21.15.83:37836) 2023-11-26T07:14:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84a884c700 1 -- 172.21.15.83:0/2025013009 learned_addr learned my addr 172.21.15.83:0/2025013009 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:39.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84a3fff700 1 --2- 172.21.15.83:0/2025013009 >> v2:172.21.15.203:3300/0 conn(0x7f84a406e0b0 0x7f84a40b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84a884c700 1 -- 172.21.15.83:0/2025013009 >> v2:172.21.15.203:3300/0 conn(0x7f84a406e0b0 msgr2=0x7f84a40b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84a884c700 1 --2- 172.21.15.83:0/2025013009 >> v2:172.21.15.203:3300/0 conn(0x7f84a406e0b0 0x7f84a40b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84a884c700 1 -- 172.21.15.83:0/2025013009 >> v1:172.21.15.203:6789/0 conn(0x7f84a40b5dd0 legacy=0x7f84a40af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84a884c700 1 -- 172.21.15.83:0/2025013009 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a40aec90 con 0x7f84a40b59f0 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84a884c700 1 --2- 172.21.15.83:0/2025013009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a40b59f0 0x7f84a406db70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f84980098f0 tx=0x7f8498009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83855bb1c27aa842 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.411+0000 7f84a37fe700 1 -- 172.21.15.83:0/2025013009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498004230 con 0x7f84a40b59f0 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.412+0000 7f84a37fe700 1 -- 172.21.15.83:0/2025013009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8498004390 con 0x7f84a40b59f0 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.412+0000 7f84a37fe700 1 -- 172.21.15.83:0/2025013009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84980036a0 con 0x7f84a40b59f0 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.412+0000 7f84aaab0700 1 -- 172.21.15.83:0/2025013009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a40b59f0 msgr2=0x7f84a406db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.412+0000 7f84aaab0700 1 --2- 172.21.15.83:0/2025013009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a40b59f0 0x7f84a406db70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f84980098f0 tx=0x7f8498009c00 comp rx=0 tx=0).stop 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.412+0000 7f84aaab0700 1 -- 172.21.15.83:0/2025013009 shutdown_connections 2023-11-26T07:14:39.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.412+0000 7f84aaab0700 1 --2- 172.21.15.83:0/2025013009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a40b59f0 0x7f84a406db70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.412+0000 7f84aaab0700 1 --2- 172.21.15.83:0/2025013009 >> v2:172.21.15.203:3300/0 conn(0x7f84a406e0b0 0x7f84a40b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.412+0000 7f84aaab0700 1 -- 172.21.15.83:0/2025013009 >> 172.21.15.83:0/2025013009 conn(0x7f84a4072990 msgr2=0x7f84a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:39.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.412+0000 7f84aaab0700 1 -- 172.21.15.83:0/2025013009 shutdown_connections 2023-11-26T07:14:39.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.412+0000 7f84aaab0700 1 -- 172.21.15.83:0/2025013009 wait complete. 2023-11-26T07:14:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.413+0000 7f84aaab0700 1 Processor -- start 2023-11-26T07:14:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.413+0000 7f84aaab0700 1 -- start start 2023-11-26T07:14:39.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.414+0000 7f84aaab0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a406e0b0 0x7f84a414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.414+0000 7f84aaab0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84a40b59f0 0x7f84a40ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.414+0000 7f84aaab0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a40b5dd0 0x7f84a4162230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.414+0000 7f84aaab0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a41f4c50 con 0x7f84a406e0b0 2023-11-26T07:14:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.414+0000 7f84aaab0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a414ea90 con 0x7f84a40b5dd0 2023-11-26T07:14:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.414+0000 7f84aaab0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a41f3fd0 con 0x7f84a40b59f0 2023-11-26T07:14:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.414+0000 7f84a884c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a406e0b0 0x7f84a414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.414+0000 7f84a884c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a406e0b0 0x7f84a414e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37844/0 (socket says 172.21.15.83:37844) 2023-11-26T07:14:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.414+0000 7f84a884c700 1 -- 172.21.15.83:0/3443206733 learned_addr learned my addr 172.21.15.83:0/3443206733 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a904d700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a40b5dd0 0x7f84a4162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a3fff700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84a40b59f0 0x7f84a40ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a884c700 1 -- 172.21.15.83:0/3443206733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84a40b59f0 msgr2=0x7f84a40ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a884c700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84a40b59f0 0x7f84a40ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a884c700 1 -- 172.21.15.83:0/3443206733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a40b5dd0 msgr2=0x7f84a4162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a884c700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a40b5dd0 0x7f84a4162230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a884c700 1 -- 172.21.15.83:0/3443206733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8498009550 con 0x7f84a406e0b0 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a904d700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a40b5dd0 0x7f84a4162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a3fff700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84a40b59f0 0x7f84a40ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a884c700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a406e0b0 0x7f84a414e550 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8498009130 tx=0x7f8498004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a17fa700 1 -- 172.21.15.83:0/3443206733 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498003e70 con 0x7f84a406e0b0 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a17fa700 1 -- 172.21.15.83:0/3443206733 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849801f070 con 0x7f84a406e0b0 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84a17fa700 1 -- 172.21.15.83:0/3443206733 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8498011830 con 0x7f84a406e0b0 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84aaab0700 1 -- 172.21.15.83:0/3443206733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a41f42b0 con 0x7f84a406e0b0 2023-11-26T07:14:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.415+0000 7f84aaab0700 1 -- 172.21.15.83:0/3443206733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a41f4870 con 0x7f84a406e0b0 2023-11-26T07:14:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.416+0000 7f84aaab0700 1 -- 172.21.15.83:0/3443206733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a4067f60 con 0x7f84a406e0b0 2023-11-26T07:14:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.416+0000 7f84a17fa700 1 -- 172.21.15.83:0/3443206733 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8498021030 con 0x7f84a406e0b0 2023-11-26T07:14:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.417+0000 7f84a17fa700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f848806b110 0x7f848806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.417+0000 7f84a17fa700 1 -- 172.21.15.83:0/3443206733 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f849808b080 con 0x7f84a406e0b0 2023-11-26T07:14:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.417+0000 7f84a3fff700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f848806b110 0x7f848806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.417+0000 7f84a3fff700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f848806b110 0x7f848806d5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8490006fd0 tx=0x7f8490008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.421+0000 7f84a17fa700 1 -- 172.21.15.83:0/3443206733 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8498055ef0 con 0x7f84a406e0b0 2023-11-26T07:14:39.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.636+0000 7f84a17fa700 1 -- 172.21.15.83:0/3443206733 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f84980562c0 con 0x7f84a406e0b0 2023-11-26T07:14:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.662+0000 7f84aaab0700 1 -- 172.21.15.83:0/3443206733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84a4067f60 con 0x7f84a406e0b0 2023-11-26T07:14:39.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.663+0000 7f84a17fa700 1 -- 172.21.15.83:0/3443206733 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f849806f580 con 0x7f84a406e0b0 2023-11-26T07:14:39.665 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:39.665 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 -- 172.21.15.83:0/3443206733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f848806b110 msgr2=0x7f848806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f848806b110 0x7f848806d5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8490006fd0 tx=0x7f8490008040 comp rx=0 tx=0).stop 2023-11-26T07:14:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 -- 172.21.15.83:0/3443206733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a406e0b0 msgr2=0x7f84a414e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a406e0b0 0x7f84a414e550 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8498009130 tx=0x7f8498004750 comp rx=0 tx=0).stop 2023-11-26T07:14:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 -- 172.21.15.83:0/3443206733 shutdown_connections 2023-11-26T07:14:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84a40b5dd0 0x7f84a4162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84a406e0b0 0x7f84a414e550 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f848806b110 0x7f848806d5d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 --2- 172.21.15.83:0/3443206733 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84a40b59f0 0x7f84a40ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 -- 172.21.15.83:0/3443206733 >> 172.21.15.83:0/3443206733 conn(0x7f84a4072990 msgr2=0x7f84a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 -- 172.21.15.83:0/3443206733 shutdown_connections 2023-11-26T07:14:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.665+0000 7f84aaab0700 1 -- 172.21.15.83:0/3443206733 wait complete. 2023-11-26T07:14:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:39.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:14:39.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.797+0000 7fe12c1df700 1 Processor -- start 2023-11-26T07:14:39.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.797+0000 7fe12c1df700 1 -- start start 2023-11-26T07:14:39.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe12c1df700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe124152cc0 0x7fe12406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe12c1df700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe1240d0630 0x7fe12406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe12c1df700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12415c4b0 con 0x7fe1240b49a0 2023-11-26T07:14:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe12c1df700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12415c790 con 0x7fe124152cc0 2023-11-26T07:14:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe12c1df700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12415ca70 con 0x7fe1240d0630 2023-11-26T07:14:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe129f7b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe124152cc0 0x7fe12406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe129f7b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe124152cc0 0x7fe12406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37850/0 (socket says 172.21.15.83:37850) 2023-11-26T07:14:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe129f7b700 1 -- 172.21.15.83:0/6533675 learned_addr learned my addr 172.21.15.83:0/6533675 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe129f7b700 1 -- 172.21.15.83:0/6533675 >> v2:172.21.15.203:3300/0 conn(0x7fe1240d0630 msgr2=0x7fe12406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe129f7b700 1 --2- 172.21.15.83:0/6533675 >> v2:172.21.15.203:3300/0 conn(0x7fe1240d0630 0x7fe12406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe129f7b700 1 -- 172.21.15.83:0/6533675 >> v1:172.21.15.203:6789/0 conn(0x7fe1240b49a0 legacy=0x7fe1240cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe129f7b700 1 -- 172.21.15.83:0/6533675 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12415cd50 con 0x7fe124152cc0 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe129f7b700 1 --2- 172.21.15.83:0/6533675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe124152cc0 0x7fe12406c000 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe1240b4e80 tx=0x7fe120009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c0d68e838a9f42 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.798+0000 7fe128f79700 1 -- 172.21.15.83:0/6533675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120004230 con 0x7fe124152cc0 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.799+0000 7fe128f79700 1 -- 172.21.15.83:0/6533675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe120004390 con 0x7fe124152cc0 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.799+0000 7fe128f79700 1 -- 172.21.15.83:0/6533675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1200036a0 con 0x7fe124152cc0 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.799+0000 7fe12c1df700 1 -- 172.21.15.83:0/6533675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe124152cc0 msgr2=0x7fe12406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.799+0000 7fe12c1df700 1 --2- 172.21.15.83:0/6533675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe124152cc0 0x7fe12406c000 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe1240b4e80 tx=0x7fe120009bd0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.799+0000 7fe12c1df700 1 -- 172.21.15.83:0/6533675 shutdown_connections 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.799+0000 7fe12c1df700 1 --2- 172.21.15.83:0/6533675 >> v2:172.21.15.203:3300/0 conn(0x7fe1240d0630 0x7fe12406c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.799+0000 7fe12c1df700 1 --2- 172.21.15.83:0/6533675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe124152cc0 0x7fe12406c000 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.799+0000 7fe12c1df700 1 -- 172.21.15.83:0/6533675 >> 172.21.15.83:0/6533675 conn(0x7fe124071b90 msgr2=0x7fe124071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:39.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.799+0000 7fe12c1df700 1 -- 172.21.15.83:0/6533675 shutdown_connections 2023-11-26T07:14:39.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.799+0000 7fe12c1df700 1 -- 172.21.15.83:0/6533675 wait complete. 2023-11-26T07:14:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.800+0000 7fe12c1df700 1 Processor -- start 2023-11-26T07:14:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.800+0000 7fe12c1df700 1 -- start start 2023-11-26T07:14:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12c1df700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1240b49a0 0x7fe1241472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12c1df700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240d0630 0x7fe12415eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12c1df700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe124152cc0 0x7fe12415f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12c1df700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1241532b0 con 0x7fe1240d0630 2023-11-26T07:14:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12c1df700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1241575b0 con 0x7fe1240b49a0 2023-11-26T07:14:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12c1df700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1241569e0 con 0x7fe124152cc0 2023-11-26T07:14:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12977a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240d0630 0x7fe12415eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12977a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240d0630 0x7fe12415eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37856/0 (socket says 172.21.15.83:37856) 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12977a700 1 -- 172.21.15.83:0/3872009096 learned_addr learned my addr 172.21.15.83:0/3872009096 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12977a700 1 -- 172.21.15.83:0/3872009096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe124152cc0 msgr2=0x7fe12415f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12977a700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe124152cc0 0x7fe12415f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12977a700 1 -- 172.21.15.83:0/3872009096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1240b49a0 msgr2=0x7fe1241472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12977a700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1240b49a0 0x7fe1241472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12977a700 1 -- 172.21.15.83:0/3872009096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe120009550 con 0x7fe1240d0630 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.801+0000 7fe12977a700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240d0630 0x7fe12415eb20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe11400c4d0 tx=0x7fe11400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.802+0000 7fe11a7fc700 1 -- 172.21.15.83:0/3872009096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140054e0 con 0x7fe1240d0630 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.802+0000 7fe11a7fc700 1 -- 172.21.15.83:0/3872009096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe114003680 con 0x7fe1240d0630 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.802+0000 7fe11a7fc700 1 -- 172.21.15.83:0/3872009096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114003a30 con 0x7fe1240d0630 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.802+0000 7fe12c1df700 1 -- 172.21.15.83:0/3872009096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe124156c80 con 0x7fe1240d0630 2023-11-26T07:14:39.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.802+0000 7fe12c1df700 1 -- 172.21.15.83:0/3872009096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe124157240 con 0x7fe1240d0630 2023-11-26T07:14:39.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.803+0000 7fe12c1df700 1 -- 172.21.15.83:0/3872009096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1240753f0 con 0x7fe1240d0630 2023-11-26T07:14:39.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.803+0000 7fe11a7fc700 1 -- 172.21.15.83:0/3872009096 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe114003b90 con 0x7fe1240d0630 2023-11-26T07:14:39.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.803+0000 7fe11a7fc700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe10806b080 0x7fe10806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:39.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.803+0000 7fe11a7fc700 1 -- 172.21.15.83:0/3872009096 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe1140889f0 con 0x7fe1240d0630 2023-11-26T07:14:39.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.803+0000 7fe129f7b700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe10806b080 0x7fe10806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:39.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.804+0000 7fe129f7b700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe10806b080 0x7fe10806d540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe120000c00 tx=0x7fe120010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:39.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.808+0000 7fe11a7fc700 1 -- 172.21.15.83:0/3872009096 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe1140537e0 con 0x7fe1240d0630 2023-11-26T07:14:40.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.998+0000 7fe12c1df700 1 -- 172.21.15.83:0/3872009096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1241576f0 con 0x7fe1240d0630 2023-11-26T07:14:40.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:39.999+0000 7fe11a7fc700 1 -- 172.21.15.83:0/3872009096 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16653 (secure 0 0 0) 0x7fe114056e00 con 0x7fe1240d0630 2023-11-26T07:14:40.002 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:40.002 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:13:10.603141+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:52.958067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:53.875459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":5}]},{"pgid":"7.2c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:14:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 -- 172.21.15.83:0/3872009096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe10806b080 msgr2=0x7fe10806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe10806b080 0x7fe10806d540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe120000c00 tx=0x7fe120010000 comp rx=0 tx=0).stop 2023-11-26T07:14:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 -- 172.21.15.83:0/3872009096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240d0630 msgr2=0x7fe12415eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240d0630 0x7fe12415eb20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe11400c4d0 tx=0x7fe11400c890 comp rx=0 tx=0).stop 2023-11-26T07:14:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 -- 172.21.15.83:0/3872009096 shutdown_connections 2023-11-26T07:14:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe10806b080 0x7fe10806d540 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe124152cc0 0x7fe12415f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1240b49a0 0x7fe1241472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 --2- 172.21.15.83:0/3872009096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1240d0630 0x7fe12415eb20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 -- 172.21.15.83:0/3872009096 >> 172.21.15.83:0/3872009096 conn(0x7fe124071b90 msgr2=0x7fe12415ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 -- 172.21.15.83:0/3872009096 shutdown_connections 2023-11-26T07:14:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.001+0000 7fe12c1df700 1 -- 172.21.15.83:0/3872009096 wait complete. 2023-11-26T07:14:40.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:12:52.958067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:12:53.875459+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:14:40.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:14:40.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.136+0000 7fa8250cc700 1 Processor -- start 2023-11-26T07:14:40.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.136+0000 7fa8250cc700 1 -- start start 2023-11-26T07:14:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa8250cc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8200995e0 0x7fa820099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa8250cc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa82006e3a0 0x7fa82006e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa8250cc700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8200ce2e0 con 0x7fa820161d20 2023-11-26T07:14:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa8250cc700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8200d0700 con 0x7fa82006e3a0 2023-11-26T07:14:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa8250cc700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82006f2d0 con 0x7fa8200995e0 2023-11-26T07:14:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81ed9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa820161d20 0x7fa820162100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57904/0 (socket says 172.21.15.83:57904) 2023-11-26T07:14:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81ed9d700 1 -- 172.21.15.83:0/2497466088 learned_addr learned my addr 172.21.15.83:0/2497466088 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81e59c700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8200995e0 0x7fa820099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81f59e700 1 --2- 172.21.15.83:0/2497466088 >> v2:172.21.15.203:3300/0 conn(0x7fa82006e3a0 0x7fa82006e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705101865 0 0) 0x7fa8200ce2e0 con 0x7fa820161d20 2023-11-26T07:14:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8080035b0 con 0x7fa820161d20 2023-11-26T07:14:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3715912419 0 0) 0x7fa8080035b0 con 0x7fa820161d20 2023-11-26T07:14:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8200995e0 msgr2=0x7fa820099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8200995e0 0x7fa820099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 >> v2:172.21.15.203:3300/0 conn(0x7fa82006e3a0 msgr2=0x7fa82006e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 --2- 172.21.15.83:0/2497466088 >> v2:172.21.15.203:3300/0 conn(0x7fa82006e3a0 0x7fa82006e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8200ce5a0 con 0x7fa820161d20 2023-11-26T07:14:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fa810001670 con 0x7fa820161d20 2023-11-26T07:14:40.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 >> v1:172.21.15.83:6789/0 conn(0x7fa820161d20 legacy=0x7fa820162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa808003a00 0x7fa808003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa808004850 0x7fa808004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8200995e0 0x7fa808005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8200d0700 con 0x7fa808004850 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82006f2d0 con 0x7fa808003a00 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80800ba10 con 0x7fa8200995e0 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.137+0000 7fa81e59c700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa808004850 0x7fa808004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81ed9d700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa808003a00 0x7fa808003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81f59e700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8200995e0 0x7fa808005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81e59c700 1 -- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8200995e0 msgr2=0x7fa808005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81e59c700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8200995e0 0x7fa808005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81e59c700 1 -- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa808003a00 msgr2=0x7fa808003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:40.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81e59c700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa808003a00 0x7fa808003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81e59c700 1 -- 172.21.15.83:0/2497466088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8080035b0 con 0x7fa808004850 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81f59e700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8200995e0 0x7fa808005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81e59c700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa808004850 0x7fa808004cd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa814009b60 tx=0x7fa808005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f67cf2af306ae5c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81400e6c0 con 0x7fa808004850 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81400e820 con 0x7fa808004850 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81dd9b700 1 -- 172.21.15.83:0/2497466088 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81400eb90 con 0x7fa808004850 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa8250cc700 1 -- 172.21.15.83:0/2497466088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa808004850 msgr2=0x7fa808004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa8250cc700 1 --2- 172.21.15.83:0/2497466088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa808004850 0x7fa808004cd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa814009b60 tx=0x7fa808005880 comp rx=0 tx=0).stop 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa81f59e700 1 -- 172.21.15.83:0/2497466088 reap_dead start 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa8250cc700 1 -- 172.21.15.83:0/2497466088 shutdown_connections 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.138+0000 7fa8250cc700 1 -- 172.21.15.83:0/2497466088 >> 172.21.15.83:0/2497466088 conn(0x7fa8200726e0 msgr2=0x7fa820072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.139+0000 7fa8250cc700 1 -- 172.21.15.83:0/2497466088 shutdown_connections 2023-11-26T07:14:40.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.139+0000 7fa8250cc700 1 -- 172.21.15.83:0/2497466088 wait complete. 2023-11-26T07:14:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.140+0000 7fa8250cc700 1 Processor -- start 2023-11-26T07:14:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.140+0000 7fa8250cc700 1 -- start start 2023-11-26T07:14:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa8250cc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa808004850 0x7fa8201567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa8250cc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8200b0e20 0x7fa8200b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa8250cc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8200ae7e0 0x7fa8200aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa8250cc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8200b17c0 con 0x7fa8200ae7e0 2023-11-26T07:14:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa8250cc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8201f39e0 con 0x7fa8200b0e20 2023-11-26T07:14:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa8250cc700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa820156d10 con 0x7fa808004850 2023-11-26T07:14:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81f59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8200ae7e0 0x7fa8200aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81f59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8200ae7e0 0x7fa8200aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37880/0 (socket says 172.21.15.83:37880) 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81f59e700 1 -- 172.21.15.83:0/1729093602 learned_addr learned my addr 172.21.15.83:0/1729093602 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81e59c700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8200b0e20 0x7fa8200b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81ed9d700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa808004850 0x7fa8201567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81f59e700 1 -- 172.21.15.83:0/1729093602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa808004850 msgr2=0x7fa8201567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81f59e700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa808004850 0x7fa8201567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81f59e700 1 -- 172.21.15.83:0/1729093602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8200b0e20 msgr2=0x7fa8200b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81f59e700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8200b0e20 0x7fa8200b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81f59e700 1 -- 172.21.15.83:0/1729093602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8201f3c30 con 0x7fa8200ae7e0 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81ed9d700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa808004850 0x7fa8201567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa81f59e700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8200ae7e0 0x7fa8200aec60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa8200c64a0 tx=0x7fa8180104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa8077fe700 1 -- 172.21.15.83:0/1729093602 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81801b070 con 0x7fa8200ae7e0 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.141+0000 7fa8077fe700 1 -- 172.21.15.83:0/1729093602 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81800f040 con 0x7fa8200ae7e0 2023-11-26T07:14:40.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.142+0000 7fa8250cc700 1 -- 172.21.15.83:0/1729093602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8201f3f10 con 0x7fa8200ae7e0 2023-11-26T07:14:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.142+0000 7fa8077fe700 1 -- 172.21.15.83:0/1729093602 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81800e050 con 0x7fa8200ae7e0 2023-11-26T07:14:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.142+0000 7fa8250cc700 1 -- 172.21.15.83:0/1729093602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8201f44d0 con 0x7fa8200ae7e0 2023-11-26T07:14:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.142+0000 7fa8250cc700 1 -- 172.21.15.83:0/1729093602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa820068fb0 con 0x7fa8200ae7e0 2023-11-26T07:14:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.143+0000 7fa8077fe700 1 -- 172.21.15.83:0/1729093602 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fa818015440 con 0x7fa8200ae7e0 2023-11-26T07:14:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.143+0000 7fa8077fe700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa80c06b080 0x7fa80c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.143+0000 7fa8077fe700 1 -- 172.21.15.83:0/1729093602 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa818089b90 con 0x7fa8200ae7e0 2023-11-26T07:14:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.143+0000 7fa81ed9d700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa80c06b080 0x7fa80c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.144+0000 7fa81ed9d700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa80c06b080 0x7fa80c06d540 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa810001510 tx=0x7fa810009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.148+0000 7fa8077fe700 1 -- 172.21.15.83:0/1729093602 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa818054870 con 0x7fa8200ae7e0 2023-11-26T07:14:40.340 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.338+0000 7fa8250cc700 1 -- 172.21.15.83:0/1729093602 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa8201f46f0 con 0x7fa8200ae7e0 2023-11-26T07:14:40.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.339+0000 7fa8077fe700 1 -- 172.21.15.83:0/1729093602 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa818057e90 con 0x7fa8200ae7e0 2023-11-26T07:14:40.341 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:14:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 -- 172.21.15.83:0/1729093602 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa80c06b080 msgr2=0x7fa80c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa80c06b080 0x7fa80c06d540 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa810001510 tx=0x7fa810009040 comp rx=0 tx=0).stop 2023-11-26T07:14:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 -- 172.21.15.83:0/1729093602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8200ae7e0 msgr2=0x7fa8200aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8200ae7e0 0x7fa8200aec60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa8200c64a0 tx=0x7fa8180104a0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 -- 172.21.15.83:0/1729093602 shutdown_connections 2023-11-26T07:14:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8200ae7e0 0x7fa8200aec60 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa80c06b080 0x7fa80c06d540 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa808004850 0x7fa8201567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 --2- 172.21.15.83:0/1729093602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8200b0e20 0x7fa8200b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 -- 172.21.15.83:0/1729093602 >> 172.21.15.83:0/1729093602 conn(0x7fa8200726e0 msgr2=0x7fa820061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 -- 172.21.15.83:0/1729093602 shutdown_connections 2023-11-26T07:14:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.341+0000 7fa8250cc700 1 -- 172.21.15.83:0/1729093602 wait complete. 2023-11-26T07:14:40.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:14:40.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.476+0000 7fe05f555700 1 Processor -- start 2023-11-26T07:14:40.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.476+0000 7fe05f555700 1 -- start start 2023-11-26T07:14:40.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05f555700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe05815b370 0x7fe05815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05f555700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0580b3c60 0x7fe0580b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05f555700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0580ae5e0 con 0x7fe0581587f0 2023-11-26T07:14:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05f555700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0580b4630 con 0x7fe0580b3c60 2023-11-26T07:14:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05f555700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05815be30 con 0x7fe05815b370 2023-11-26T07:14:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05daf2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0580b3c60 0x7fe0580b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05daf2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe0580b3c60 0x7fe0580b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37888/0 (socket says 172.21.15.83:37888) 2023-11-26T07:14:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05daf2700 1 -- 172.21.15.83:0/2451314737 learned_addr learned my addr 172.21.15.83:0/2451314737 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05caf0700 1 --2- 172.21.15.83:0/2451314737 >> v2:172.21.15.203:3300/0 conn(0x7fe05815b370 0x7fe05815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05daf2700 1 -- 172.21.15.83:0/2451314737 >> v2:172.21.15.203:3300/0 conn(0x7fe05815b370 msgr2=0x7fe05815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05daf2700 1 --2- 172.21.15.83:0/2451314737 >> v2:172.21.15.203:3300/0 conn(0x7fe05815b370 0x7fe05815b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05daf2700 1 -- 172.21.15.83:0/2451314737 >> v1:172.21.15.203:6789/0 conn(0x7fe0581587f0 legacy=0x7fe05806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05daf2700 1 -- 172.21.15.83:0/2451314737 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05815c110 con 0x7fe0580b3c60 2023-11-26T07:14:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe05daf2700 1 --2- 172.21.15.83:0/2451314737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0580b3c60 0x7fe0580b40f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe050009e50 tx=0x7fe05000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8fd143d9a9af33 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.477+0000 7fe04ffff700 1 -- 172.21.15.83:0/2451314737 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05000e070 con 0x7fe0580b3c60 2023-11-26T07:14:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.478+0000 7fe04ffff700 1 -- 172.21.15.83:0/2451314737 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe050003a20 con 0x7fe0580b3c60 2023-11-26T07:14:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.478+0000 7fe04ffff700 1 -- 172.21.15.83:0/2451314737 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050003dd0 con 0x7fe0580b3c60 2023-11-26T07:14:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.478+0000 7fe05f555700 1 -- 172.21.15.83:0/2451314737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0580b3c60 msgr2=0x7fe0580b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.478+0000 7fe05f555700 1 --2- 172.21.15.83:0/2451314737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0580b3c60 0x7fe0580b40f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe050009e50 tx=0x7fe05000b330 comp rx=0 tx=0).stop 2023-11-26T07:14:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.478+0000 7fe05f555700 1 -- 172.21.15.83:0/2451314737 shutdown_connections 2023-11-26T07:14:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.478+0000 7fe05f555700 1 --2- 172.21.15.83:0/2451314737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0580b3c60 0x7fe0580b40f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.478+0000 7fe05f555700 1 --2- 172.21.15.83:0/2451314737 >> v2:172.21.15.203:3300/0 conn(0x7fe05815b370 0x7fe05815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.478+0000 7fe05f555700 1 -- 172.21.15.83:0/2451314737 >> 172.21.15.83:0/2451314737 conn(0x7fe0580d0bf0 msgr2=0x7fe0580d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.478+0000 7fe05f555700 1 -- 172.21.15.83:0/2451314737 shutdown_connections 2023-11-26T07:14:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.478+0000 7fe05f555700 1 -- 172.21.15.83:0/2451314737 wait complete. 2023-11-26T07:14:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.479+0000 7fe05f555700 1 Processor -- start 2023-11-26T07:14:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.479+0000 7fe05f555700 1 -- start start 2023-11-26T07:14:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05f555700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0580b3c60 0x7fe058151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05f555700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0581587f0 0x7fe058154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05f555700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05815b370 0x7fe0581f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05f555700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0581456b0 con 0x7fe0580b3c60 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05f555700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0581595e0 con 0x7fe05815b370 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05f555700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0581517c0 con 0x7fe0581587f0 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05d2f1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0580b3c60 0x7fe058151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05d2f1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0580b3c60 0x7fe058151170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37890/0 (socket says 172.21.15.83:37890) 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05d2f1700 1 -- 172.21.15.83:0/2450509692 learned_addr learned my addr 172.21.15.83:0/2450509692 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05d2f1700 1 -- 172.21.15.83:0/2450509692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0581587f0 msgr2=0x7fe058154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05d2f1700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0581587f0 0x7fe058154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05d2f1700 1 -- 172.21.15.83:0/2450509692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05815b370 msgr2=0x7fe0581f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05d2f1700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05815b370 0x7fe0581f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.480+0000 7fe05d2f1700 1 -- 172.21.15.83:0/2450509692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe050009b30 con 0x7fe0580b3c60 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.481+0000 7fe05d2f1700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0580b3c60 0x7fe058151170 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe05400c4d0 tx=0x7fe05400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.481+0000 7fe04dffb700 1 -- 172.21.15.83:0/2450509692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe054011790 con 0x7fe0580b3c60 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.481+0000 7fe04dffb700 1 -- 172.21.15.83:0/2450509692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0540118f0 con 0x7fe0580b3c60 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.481+0000 7fe05f555700 1 -- 172.21.15.83:0/2450509692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0581598a0 con 0x7fe0580b3c60 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.481+0000 7fe04dffb700 1 -- 172.21.15.83:0/2450509692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe054011ca0 con 0x7fe0580b3c60 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.481+0000 7fe05f555700 1 -- 172.21.15.83:0/2450509692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe058159e60 con 0x7fe0580b3c60 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.482+0000 7fe05f555700 1 -- 172.21.15.83:0/2450509692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe058068fb0 con 0x7fe0580b3c60 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.482+0000 7fe04dffb700 1 -- 172.21.15.83:0/2450509692 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe05401c460 con 0x7fe0580b3c60 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.483+0000 7fe04dffb700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe03c06b080 0x7fe03c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.483+0000 7fe04dffb700 1 -- 172.21.15.83:0/2450509692 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe0540897b0 con 0x7fe0580b3c60 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.483+0000 7fe05caf0700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe03c06b080 0x7fe03c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.483+0000 7fe05caf0700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe03c06b080 0x7fe03c06d540 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe05815bb40 tx=0x7fe048008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:40.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.487+0000 7fe04dffb700 1 -- 172.21.15.83:0/2450509692 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe054054500 con 0x7fe0580b3c60 2023-11-26T07:14:40.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.676+0000 7fe05f555700 1 -- 172.21.15.83:0/2450509692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe05815a080 con 0x7fe0580b3c60 2023-11-26T07:14:40.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.676+0000 7fe04dffb700 1 -- 172.21.15.83:0/2450509692 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe054057b20 con 0x7fe0580b3c60 2023-11-26T07:14:40.679 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:14:40.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 -- 172.21.15.83:0/2450509692 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe03c06b080 msgr2=0x7fe03c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe03c06b080 0x7fe03c06d540 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe05815bb40 tx=0x7fe048008040 comp rx=0 tx=0).stop 2023-11-26T07:14:40.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 -- 172.21.15.83:0/2450509692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0580b3c60 msgr2=0x7fe058151170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0580b3c60 0x7fe058151170 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe05400c4d0 tx=0x7fe05400c890 comp rx=0 tx=0).stop 2023-11-26T07:14:40.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 -- 172.21.15.83:0/2450509692 shutdown_connections 2023-11-26T07:14:40.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe05815b370 0x7fe0581f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0580b3c60 0x7fe058151170 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe03c06b080 0x7fe03c06d540 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 --2- 172.21.15.83:0/2450509692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0581587f0 0x7fe058154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 -- 172.21.15.83:0/2450509692 >> 172.21.15.83:0/2450509692 conn(0x7fe0580d0bf0 msgr2=0x7fe058073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:40.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 -- 172.21.15.83:0/2450509692 shutdown_connections 2023-11-26T07:14:40.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.678+0000 7fe05f555700 1 -- 172.21.15.83:0/2450509692 wait complete. 2023-11-26T07:14:40.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:14:40.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.813+0000 7f735ee95700 1 Processor -- start 2023-11-26T07:14:40.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.813+0000 7f735ee95700 1 -- start start 2023-11-26T07:14:40.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.813+0000 7f735ee95700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73580b4310 0x7f735806bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.813+0000 7f735ee95700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735806ba50 con 0x7f73580ce430 2023-11-26T07:14:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.813+0000 7f735ee95700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735806b120 con 0x7f735815e080 2023-11-26T07:14:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.813+0000 7f735ee95700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73580d1200 con 0x7f73580b4310 2023-11-26T07:14:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f735d432700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f73580ce430 0x7f73580cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57942/0 (socket says 172.21.15.83:57942) 2023-11-26T07:14:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f735d432700 1 -- 172.21.15.83:0/447405763 learned_addr learned my addr 172.21.15.83:0/447405763 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666018494 0 0) 0x7f735806ba50 con 0x7f73580ce430 2023-11-26T07:14:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f7357fff700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73580b4310 0x7f735806bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73480035b0 con 0x7f73580ce430 2023-11-26T07:14:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1025845994 0 0) 0x7f73480035b0 con 0x7f73580ce430 2023-11-26T07:14:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73580b4310 msgr2=0x7f735806bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73580b4310 0x7f735806bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 >> v1:172.21.15.203:6789/0 conn(0x7f735815e080 legacy=0x7f73580d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735806b400 con 0x7f73580ce430 2023-11-26T07:14:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f73500019f0 con 0x7f73580ce430 2023-11-26T07:14:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 >> v1:172.21.15.83:6789/0 conn(0x7f73580ce430 legacy=0x7f73580cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73580b4310 0x7f7348003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7348003fb0 0x7f7348004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7348004d90 0x7f7348005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73580d1200 con 0x7f7348004d90 2023-11-26T07:14:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734800bba0 con 0x7f7348003fb0 2023-11-26T07:14:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734800bd60 con 0x7f73580b4310 2023-11-26T07:14:40.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f735d432700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7348004d90 0x7f7348005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f735cc31700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73580b4310 0x7f7348003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.814+0000 7f7357fff700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7348003fb0 0x7f7348004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f735cc31700 1 -- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7348003fb0 msgr2=0x7f7348004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f735cc31700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7348003fb0 0x7f7348004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f735cc31700 1 -- 172.21.15.83:0/447405763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7348004d90 msgr2=0x7f7348005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f735cc31700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7348004d90 0x7f7348005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f735cc31700 1 -- 172.21.15.83:0/447405763 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480035b0 con 0x7f73580b4310 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f735d432700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7348004d90 0x7f7348005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f735cc31700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73580b4310 0x7f7348003a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f734c0017f0 tx=0x7f7348005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb59b1eeadfddb84 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c0052d0 con 0x7f73580b4310 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c005430 con 0x7f73580b4310 2023-11-26T07:14:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f73577fe700 1 -- 172.21.15.83:0/447405763 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c0038c0 con 0x7f73580b4310 2023-11-26T07:14:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f735ee95700 1 -- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73580b4310 msgr2=0x7f7348003a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f735ee95700 1 --2- 172.21.15.83:0/447405763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73580b4310 0x7f7348003a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f734c0017f0 tx=0x7f7348005880 comp rx=0 tx=0).stop 2023-11-26T07:14:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.815+0000 7f735d432700 1 -- 172.21.15.83:0/447405763 reap_dead start 2023-11-26T07:14:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.816+0000 7f735ee95700 1 -- 172.21.15.83:0/447405763 shutdown_connections 2023-11-26T07:14:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.816+0000 7f735ee95700 1 -- 172.21.15.83:0/447405763 >> 172.21.15.83:0/447405763 conn(0x7f73580b3780 msgr2=0x7f73580b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.816+0000 7f735ee95700 1 -- 172.21.15.83:0/447405763 shutdown_connections 2023-11-26T07:14:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.816+0000 7f735ee95700 1 -- 172.21.15.83:0/447405763 wait complete. 2023-11-26T07:14:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.817+0000 7f735ee95700 1 Processor -- start 2023-11-26T07:14:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.817+0000 7f735ee95700 1 -- start start 2023-11-26T07:14:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735ee95700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73580b4310 0x7f73580b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735ee95700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f735815a690 0x7f73580b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735ee95700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73581f3730 0x7f73581f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735ee95700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73581f40d0 con 0x7f73580b4310 2023-11-26T07:14:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735ee95700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73580c6a60 con 0x7f73581f3730 2023-11-26T07:14:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735ee95700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73581f4310 con 0x7f735815a690 2023-11-26T07:14:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735cc31700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73580b4310 0x7f73580b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735cc31700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73580b4310 0x7f73580b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37910/0 (socket says 172.21.15.83:37910) 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735cc31700 1 -- 172.21.15.83:0/4134986352 learned_addr learned my addr 172.21.15.83:0/4134986352 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735d432700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73581f3730 0x7f73581f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f7357fff700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f735815a690 0x7f73580b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735cc31700 1 -- 172.21.15.83:0/4134986352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f735815a690 msgr2=0x7f73580b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735cc31700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f735815a690 0x7f73580b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735cc31700 1 -- 172.21.15.83:0/4134986352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73581f3730 msgr2=0x7f73581f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735cc31700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73581f3730 0x7f73581f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735cc31700 1 -- 172.21.15.83:0/4134986352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73581f45f0 con 0x7f73580b4310 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f7357fff700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f735815a690 0x7f73580b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735cc31700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73580b4310 0x7f73580b29d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f734800bef0 tx=0x7f734c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f73557fa700 1 -- 172.21.15.83:0/4134986352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c001290 con 0x7f73580b4310 2023-11-26T07:14:40.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f73557fa700 1 -- 172.21.15.83:0/4134986352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c003680 con 0x7f73580b4310 2023-11-26T07:14:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735ee95700 1 -- 172.21.15.83:0/4134986352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73580c6410 con 0x7f73580b4310 2023-11-26T07:14:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f73557fa700 1 -- 172.21.15.83:0/4134986352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c01d660 con 0x7f73580b4310 2023-11-26T07:14:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.818+0000 7f735ee95700 1 -- 172.21.15.83:0/4134986352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73580c9790 con 0x7f73580b4310 2023-11-26T07:14:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.819+0000 7f735ee95700 1 -- 172.21.15.83:0/4134986352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73580c6d40 con 0x7f73580b4310 2023-11-26T07:14:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.820+0000 7f73557fa700 1 -- 172.21.15.83:0/4134986352 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f734c01d7c0 con 0x7f73580b4310 2023-11-26T07:14:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.820+0000 7f73557fa700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f733c06b080 0x7f733c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.820+0000 7f73557fa700 1 -- 172.21.15.83:0/4134986352 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f734c08a2b0 con 0x7f73580b4310 2023-11-26T07:14:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.820+0000 7f7357fff700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f733c06b080 0x7f733c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:40.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.821+0000 7f7357fff700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f733c06b080 0x7f733c06d540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f73580b0e80 tx=0x7f7344008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:40.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:40.825+0000 7f73557fa700 1 -- 172.21.15.83:0/4134986352 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f734c055130 con 0x7f73580b4310 2023-11-26T07:14:41.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.013+0000 7f735ee95700 1 -- 172.21.15.83:0/4134986352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7358068fb0 con 0x7f73580b4310 2023-11-26T07:14:41.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.014+0000 7f73557fa700 1 -- 172.21.15.83:0/4134986352 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f734c058750 con 0x7f73580b4310 2023-11-26T07:14:41.017 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:14:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 -- 172.21.15.83:0/4134986352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f733c06b080 msgr2=0x7f733c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f733c06b080 0x7f733c06d540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f73580b0e80 tx=0x7f7344008040 comp rx=0 tx=0).stop 2023-11-26T07:14:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 -- 172.21.15.83:0/4134986352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73580b4310 msgr2=0x7f73580b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73580b4310 0x7f73580b29d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f734800bef0 tx=0x7f734c001120 comp rx=0 tx=0).stop 2023-11-26T07:14:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 -- 172.21.15.83:0/4134986352 shutdown_connections 2023-11-26T07:14:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73581f3730 0x7f73581f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73580b4310 0x7f73580b29d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f733c06b080 0x7f733c06d540 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 --2- 172.21.15.83:0/4134986352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f735815a690 0x7f73580b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 -- 172.21.15.83:0/4134986352 >> 172.21.15.83:0/4134986352 conn(0x7f73580b3780 msgr2=0x7f73580b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 -- 172.21.15.83:0/4134986352 shutdown_connections 2023-11-26T07:14:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.016+0000 7f735ee95700 1 -- 172.21.15.83:0/4134986352 wait complete. 2023-11-26T07:14:41.030 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.147+0000 7f233994f700 1 Processor -- start 2023-11-26T07:14:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.147+0000 7f233994f700 1 -- start start 2023-11-26T07:14:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.148+0000 7f233994f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415bdd0 0x7f23341604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.148+0000 7f233994f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23340b22d0 con 0x7f23340b3370 2023-11-26T07:14:41.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.148+0000 7f233994f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23340b25b0 con 0x7f23340d1c70 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.148+0000 7f233994f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23340b2f50 con 0x7f233415bdd0 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.148+0000 7f23327fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f23340b3370 0x7f23340b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:57958/0 (socket says 172.21.15.83:57958) 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.148+0000 7f23327fc700 1 -- 172.21.15.83:0/2458509254 learned_addr learned my addr 172.21.15.83:0/2458509254 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.148+0000 7f2332ffd700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415bdd0 0x7f23341604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.148+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672050415 0 0) 0x7f23340b22d0 con 0x7f23340b3370 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231c0035b0 con 0x7f23340b3370 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1012310193 0 0) 0x7f231c0035b0 con 0x7f23340b3370 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415bdd0 msgr2=0x7f23341604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415bdd0 0x7f23341604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 >> v1:172.21.15.203:6789/0 conn(0x7f23340d1c70 legacy=0x7f2334165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340b2890 con 0x7f23340b3370 2023-11-26T07:14:41.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f2324001670 con 0x7f23340b3370 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 >> v1:172.21.15.83:6789/0 conn(0x7f23340b3370 legacy=0x7f23340b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f231c003a30 0x7f231c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415bdd0 0x7f231c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f231c004d90 0x7f231c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340b2f50 con 0x7f231c003a30 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c00bac0 con 0x7f231c004d90 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c00bc80 con 0x7f233415bdd0 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2332ffd700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f231c003a30 0x7f231c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f23327fc700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415bdd0 0x7f231c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f23337fe700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f231c004d90 0x7f231c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2332ffd700 1 -- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415bdd0 msgr2=0x7f231c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2332ffd700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415bdd0 0x7f231c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2332ffd700 1 -- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f231c004d90 msgr2=0x7f231c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2332ffd700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f231c004d90 0x7f231c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2332ffd700 1 -- 172.21.15.83:0/2458509254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c0035b0 con 0x7f231c003a30 2023-11-26T07:14:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f23337fe700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f231c004d90 0x7f231c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f23327fc700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415bdd0 0x7f231c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.149+0000 7f2332ffd700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f231c003a30 0x7f231c003e50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2328009b30 tx=0x7f231c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb056dc476fc692 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.150+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232800e6c0 con 0x7f231c003a30 2023-11-26T07:14:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.150+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232800e820 con 0x7f231c003a30 2023-11-26T07:14:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.150+0000 7f2331ffb700 1 -- 172.21.15.83:0/2458509254 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232800eb90 con 0x7f231c003a30 2023-11-26T07:14:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.150+0000 7f233994f700 1 -- 172.21.15.83:0/2458509254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f231c003a30 msgr2=0x7f231c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.150+0000 7f233994f700 1 --2- 172.21.15.83:0/2458509254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f231c003a30 0x7f231c003e50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2328009b30 tx=0x7f231c005880 comp rx=0 tx=0).stop 2023-11-26T07:14:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.150+0000 7f23337fe700 1 -- 172.21.15.83:0/2458509254 reap_dead start 2023-11-26T07:14:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.150+0000 7f233994f700 1 -- 172.21.15.83:0/2458509254 shutdown_connections 2023-11-26T07:14:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.150+0000 7f233994f700 1 -- 172.21.15.83:0/2458509254 >> 172.21.15.83:0/2458509254 conn(0x7f23340731a0 msgr2=0x7f233409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.150+0000 7f233994f700 1 -- 172.21.15.83:0/2458509254 shutdown_connections 2023-11-26T07:14:41.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.150+0000 7f233994f700 1 -- 172.21.15.83:0/2458509254 wait complete. 2023-11-26T07:14:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.152+0000 7f233994f700 1 Processor -- start 2023-11-26T07:14:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.152+0000 7f233994f700 1 -- start start 2023-11-26T07:14:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f233994f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f231c003a30 0x7f23341f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f233994f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233415d440 0x7f23340cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f233994f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415d8c0 0x7f23340cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f233994f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23341f2360 con 0x7f233415d440 2023-11-26T07:14:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f233994f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23341f1a70 con 0x7f231c003a30 2023-11-26T07:14:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f233994f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23341f27f0 con 0x7f233415d8c0 2023-11-26T07:14:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f23327fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233415d440 0x7f23340cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f23327fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233415d440 0x7f23340cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37932/0 (socket says 172.21.15.83:37932) 2023-11-26T07:14:41.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f23327fc700 1 -- 172.21.15.83:0/1404585207 learned_addr learned my addr 172.21.15.83:0/1404585207 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:41.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f2332ffd700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f231c003a30 0x7f23341f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f23337fe700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415d8c0 0x7f23340cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f23327fc700 1 -- 172.21.15.83:0/1404585207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415d8c0 msgr2=0x7f23340cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f23327fc700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415d8c0 0x7f23340cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f23327fc700 1 -- 172.21.15.83:0/1404585207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f231c003a30 msgr2=0x7f23341f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f23327fc700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f231c003a30 0x7f23341f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f23327fc700 1 -- 172.21.15.83:0/1404585207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23341f1d10 con 0x7f233415d440 2023-11-26T07:14:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f2332ffd700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f231c003a30 0x7f23341f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f23327fc700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233415d440 0x7f23340cd970 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2324001c90 tx=0x7f232400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f231bfff700 1 -- 172.21.15.83:0/1404585207 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232400c750 con 0x7f233415d440 2023-11-26T07:14:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f231bfff700 1 -- 172.21.15.83:0/1404585207 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232400c8b0 con 0x7f233415d440 2023-11-26T07:14:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.153+0000 7f233994f700 1 -- 172.21.15.83:0/1404585207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23341f1ff0 con 0x7f233415d440 2023-11-26T07:14:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.154+0000 7f231bfff700 1 -- 172.21.15.83:0/1404585207 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232400cc40 con 0x7f233415d440 2023-11-26T07:14:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.154+0000 7f233994f700 1 -- 172.21.15.83:0/1404585207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233415de90 con 0x7f233415d440 2023-11-26T07:14:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.155+0000 7f233994f700 1 -- 172.21.15.83:0/1404585207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2334067f60 con 0x7f233415d440 2023-11-26T07:14:41.162 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.155+0000 7f231bfff700 1 -- 172.21.15.83:0/1404585207 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f232401d460 con 0x7f233415d440 2023-11-26T07:14:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.155+0000 7f231bfff700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f232006b080 0x7f232006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.155+0000 7f231bfff700 1 -- 172.21.15.83:0/1404585207 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2324089820 con 0x7f233415d440 2023-11-26T07:14:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.155+0000 7f2332ffd700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f232006b080 0x7f232006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.156+0000 7f2332ffd700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f232006b080 0x7f232006d540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f23280090b0 tx=0x7f2328008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:41.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.160+0000 7f231bfff700 1 -- 172.21.15.83:0/1404585207 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2324054610 con 0x7f233415d440 2023-11-26T07:14:41.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.392+0000 7f233994f700 1 -- 172.21.15.83:0/1404585207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23340afca0 con 0x7f233415d440 2023-11-26T07:14:41.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.393+0000 7f231bfff700 1 -- 172.21.15.83:0/1404585207 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f2324057c30 con 0x7f233415d440 2023-11-26T07:14:41.395 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:41.396 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 -- 172.21.15.83:0/1404585207 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f232006b080 msgr2=0x7f232006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f232006b080 0x7f232006d540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f23280090b0 tx=0x7f2328008030 comp rx=0 tx=0).stop 2023-11-26T07:14:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 -- 172.21.15.83:0/1404585207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233415d440 msgr2=0x7f23340cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233415d440 0x7f23340cd970 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2324001c90 tx=0x7f232400ee80 comp rx=0 tx=0).stop 2023-11-26T07:14:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 -- 172.21.15.83:0/1404585207 shutdown_connections 2023-11-26T07:14:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f232006b080 0x7f232006d540 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233415d8c0 0x7f23340cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f231c003a30 0x7f23341f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 --2- 172.21.15.83:0/1404585207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233415d440 0x7f23340cd970 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 -- 172.21.15.83:0/1404585207 >> 172.21.15.83:0/1404585207 conn(0x7f23340731a0 msgr2=0x7f23340b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:41.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 -- 172.21.15.83:0/1404585207 shutdown_connections 2023-11-26T07:14:41.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.395+0000 7f233994f700 1 -- 172.21.15.83:0/1404585207 wait complete. 2023-11-26T07:14:41.399 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:41.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:14:41.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.527+0000 7ff15488d700 1 Processor -- start 2023-11-26T07:14:41.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.527+0000 7ff15488d700 1 -- start start 2023-11-26T07:14:41.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff15488d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff1501446a0 0x7ff1500b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff15488d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff15006c710 0x7ff1500b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff15488d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1500af950 con 0x7ff1500adf00 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff15488d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff150144a80 con 0x7ff15006c710 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff15488d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1500afc30 con 0x7ff1501446a0 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14ed9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff15006c710 0x7ff1500b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14ed9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff15006c710 0x7ff1500b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37942/0 (socket says 172.21.15.83:37942) 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14ed9d700 1 -- 172.21.15.83:0/3667880109 learned_addr learned my addr 172.21.15.83:0/3667880109 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14e59c700 1 --2- 172.21.15.83:0/3667880109 >> v2:172.21.15.203:3300/0 conn(0x7ff1501446a0 0x7ff1500b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14ed9d700 1 -- 172.21.15.83:0/3667880109 >> v2:172.21.15.203:3300/0 conn(0x7ff1501446a0 msgr2=0x7ff1500b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14ed9d700 1 --2- 172.21.15.83:0/3667880109 >> v2:172.21.15.203:3300/0 conn(0x7ff1501446a0 0x7ff1500b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14ed9d700 1 -- 172.21.15.83:0/3667880109 >> v1:172.21.15.203:6789/0 conn(0x7ff1500adf00 legacy=0x7ff15006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14ed9d700 1 -- 172.21.15.83:0/3667880109 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1500aff10 con 0x7ff15006c710 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14ed9d700 1 --2- 172.21.15.83:0/3667880109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15006c710 0x7ff1500b3d40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff1500729a0 tx=0x7ff14800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e570b5f2ffd4402 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:41.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14d59a700 1 -- 172.21.15.83:0/3667880109 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14800dcf0 con 0x7ff15006c710 2023-11-26T07:14:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14d59a700 1 -- 172.21.15.83:0/3667880109 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14800de50 con 0x7ff15006c710 2023-11-26T07:14:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.528+0000 7ff14d59a700 1 -- 172.21.15.83:0/3667880109 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148004250 con 0x7ff15006c710 2023-11-26T07:14:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.529+0000 7ff15488d700 1 -- 172.21.15.83:0/3667880109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15006c710 msgr2=0x7ff1500b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.529+0000 7ff15488d700 1 --2- 172.21.15.83:0/3667880109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15006c710 0x7ff1500b3d40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff1500729a0 tx=0x7ff14800be70 comp rx=0 tx=0).stop 2023-11-26T07:14:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.529+0000 7ff15488d700 1 -- 172.21.15.83:0/3667880109 shutdown_connections 2023-11-26T07:14:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.529+0000 7ff15488d700 1 --2- 172.21.15.83:0/3667880109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff15006c710 0x7ff1500b3d40 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.529+0000 7ff15488d700 1 --2- 172.21.15.83:0/3667880109 >> v2:172.21.15.203:3300/0 conn(0x7ff1501446a0 0x7ff1500b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.529+0000 7ff15488d700 1 -- 172.21.15.83:0/3667880109 >> 172.21.15.83:0/3667880109 conn(0x7ff150071b90 msgr2=0x7ff150071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.529+0000 7ff15488d700 1 -- 172.21.15.83:0/3667880109 shutdown_connections 2023-11-26T07:14:41.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.529+0000 7ff15488d700 1 -- 172.21.15.83:0/3667880109 wait complete. 2023-11-26T07:14:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.530+0000 7ff15488d700 1 Processor -- start 2023-11-26T07:14:41.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.530+0000 7ff15488d700 1 -- start start 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff15488d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15006c710 0x7ff150159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff15488d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1500adf00 0x7ff1500b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff15488d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1501446a0 0x7ff1500b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff15488d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500cf020 con 0x7ff1501446a0 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff15488d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500ce4a0 con 0x7ff1500adf00 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff15488d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500ce780 con 0x7ff15006c710 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff14ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1501446a0 0x7ff1500b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff14ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1501446a0 0x7ff1500b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37948/0 (socket says 172.21.15.83:37948) 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff14ed9d700 1 -- 172.21.15.83:0/1097959650 learned_addr learned my addr 172.21.15.83:0/1097959650 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff14e59c700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15006c710 0x7ff150159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff14dd9b700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1500adf00 0x7ff1500b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff14ed9d700 1 -- 172.21.15.83:0/1097959650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15006c710 msgr2=0x7ff150159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff14ed9d700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15006c710 0x7ff150159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff14ed9d700 1 -- 172.21.15.83:0/1097959650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1500adf00 msgr2=0x7ff1500b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff14ed9d700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1500adf00 0x7ff1500b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff14ed9d700 1 -- 172.21.15.83:0/1097959650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14800b7f0 con 0x7ff1501446a0 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.531+0000 7ff14dd9b700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1500adf00 0x7ff1500b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.532+0000 7ff14e59c700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15006c710 0x7ff150159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.532+0000 7ff14ed9d700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1501446a0 0x7ff1500b4210 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff14800b340 tx=0x7ff148004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.532+0000 7ff146ffd700 1 -- 172.21.15.83:0/1097959650 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148020070 con 0x7ff1501446a0 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.532+0000 7ff146ffd700 1 -- 172.21.15.83:0/1097959650 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff148004030 con 0x7ff1501446a0 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.532+0000 7ff15488d700 1 -- 172.21.15.83:0/1097959650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1500cea60 con 0x7ff1501446a0 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.532+0000 7ff146ffd700 1 -- 172.21.15.83:0/1097959650 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148010470 con 0x7ff1501446a0 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.532+0000 7ff15488d700 1 -- 172.21.15.83:0/1097959650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1500cdfe0 con 0x7ff1501446a0 2023-11-26T07:14:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.533+0000 7ff15488d700 1 -- 172.21.15.83:0/1097959650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1500753f0 con 0x7ff1501446a0 2023-11-26T07:14:41.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.533+0000 7ff146ffd700 1 -- 172.21.15.83:0/1097959650 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7ff1480193f0 con 0x7ff1501446a0 2023-11-26T07:14:41.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.534+0000 7ff146ffd700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff13006b080 0x7ff13006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.534+0000 7ff14e59c700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff13006b080 0x7ff13006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.534+0000 7ff146ffd700 1 -- 172.21.15.83:0/1097959650 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff14808d250 con 0x7ff1501446a0 2023-11-26T07:14:41.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.534+0000 7ff14e59c700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff13006b080 0x7ff13006d540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff140007eb0 tx=0x7ff140007400 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:41.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.538+0000 7ff146ffd700 1 -- 172.21.15.83:0/1097959650 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff148057fc0 con 0x7ff1501446a0 2023-11-26T07:14:41.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.730+0000 7ff15488d700 1 -- 172.21.15.83:0/1097959650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff150067f60 con 0x7ff1501446a0 2023-11-26T07:14:41.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.732+0000 7ff146ffd700 1 -- 172.21.15.83:0/1097959650 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16653 (secure 0 0 0) 0x7ff14805b5e0 con 0x7ff1501446a0 2023-11-26T07:14:41.734 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:41.735 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:13:10.603141+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:52.958067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:53.875459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":5}]},{"pgid":"7.2c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:14:41.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 -- 172.21.15.83:0/1097959650 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff13006b080 msgr2=0x7ff13006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff13006b080 0x7ff13006d540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff140007eb0 tx=0x7ff140007400 comp rx=0 tx=0).stop 2023-11-26T07:14:41.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 -- 172.21.15.83:0/1097959650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1501446a0 msgr2=0x7ff1500b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1501446a0 0x7ff1500b4210 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff14800b340 tx=0x7ff148004750 comp rx=0 tx=0).stop 2023-11-26T07:14:41.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 -- 172.21.15.83:0/1097959650 shutdown_connections 2023-11-26T07:14:41.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1501446a0 0x7ff1500b4210 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff13006b080 0x7ff13006d540 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff15006c710 0x7ff150159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 --2- 172.21.15.83:0/1097959650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1500adf00 0x7ff1500b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 -- 172.21.15.83:0/1097959650 >> 172.21.15.83:0/1097959650 conn(0x7ff150071b90 msgr2=0x7ff15006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:41.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 -- 172.21.15.83:0/1097959650 shutdown_connections 2023-11-26T07:14:41.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.734+0000 7ff15488d700 1 -- 172.21.15.83:0/1097959650 wait complete. 2023-11-26T07:14:41.748 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:14:41.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.865+0000 7f6a99345700 1 Processor -- start 2023-11-26T07:14:41.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.865+0000 7f6a99345700 1 -- start start 2023-11-26T07:14:41.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.866+0000 7f6a99345700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a9406c710 0x7f6a940b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.866+0000 7f6a99345700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6a940af4d0 0x7f6a940afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.866+0000 7f6a99345700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a940ae430 con 0x7f6a9415e080 2023-11-26T07:14:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.866+0000 7f6a99345700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a940af0b0 con 0x7f6a940af4d0 2023-11-26T07:14:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.866+0000 7f6a99345700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a940b05a0 con 0x7f6a9406c710 2023-11-26T07:14:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.866+0000 7f6a937fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6a940af4d0 0x7f6a940afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.866+0000 7f6a937fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6a940af4d0 0x7f6a940afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37956/0 (socket says 172.21.15.83:37956) 2023-11-26T07:14:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.866+0000 7f6a937fe700 1 -- 172.21.15.83:0/1739394092 learned_addr learned my addr 172.21.15.83:0/1739394092 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.867+0000 7f6a927fc700 1 --2- 172.21.15.83:0/1739394092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a9406c710 0x7f6a940b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.867+0000 7f6a937fe700 1 -- 172.21.15.83:0/1739394092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a9406c710 msgr2=0x7f6a940b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.867+0000 7f6a937fe700 1 --2- 172.21.15.83:0/1739394092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a9406c710 0x7f6a940b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.867+0000 7f6a937fe700 1 -- 172.21.15.83:0/1739394092 >> v1:172.21.15.203:6789/0 conn(0x7f6a9415e080 legacy=0x7f6a9406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.867+0000 7f6a937fe700 1 -- 172.21.15.83:0/1739394092 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a940ae710 con 0x7f6a940af4d0 2023-11-26T07:14:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.867+0000 7f6a927fc700 1 --2- 172.21.15.83:0/1739394092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a9406c710 0x7f6a940b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.867+0000 7f6a937fe700 1 --2- 172.21.15.83:0/1739394092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a940af4d0 0x7f6a940afb90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009e80 tx=0x7f6a8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b474e20c79788368 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.867+0000 7f6a91ffb700 1 -- 172.21.15.83:0/1739394092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00e070 con 0x7f6a940af4d0 2023-11-26T07:14:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.867+0000 7f6a91ffb700 1 -- 172.21.15.83:0/1739394092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c00b980 con 0x7f6a940af4d0 2023-11-26T07:14:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.867+0000 7f6a91ffb700 1 -- 172.21.15.83:0/1739394092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00bd30 con 0x7f6a940af4d0 2023-11-26T07:14:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.868+0000 7f6a99345700 1 -- 172.21.15.83:0/1739394092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a940af4d0 msgr2=0x7f6a940afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.868+0000 7f6a99345700 1 --2- 172.21.15.83:0/1739394092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a940af4d0 0x7f6a940afb90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009e80 tx=0x7f6a8c00b330 comp rx=0 tx=0).stop 2023-11-26T07:14:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.868+0000 7f6a99345700 1 -- 172.21.15.83:0/1739394092 shutdown_connections 2023-11-26T07:14:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.868+0000 7f6a99345700 1 --2- 172.21.15.83:0/1739394092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a940af4d0 0x7f6a940afb90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.868+0000 7f6a99345700 1 --2- 172.21.15.83:0/1739394092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a9406c710 0x7f6a940b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.868+0000 7f6a99345700 1 -- 172.21.15.83:0/1739394092 >> 172.21.15.83:0/1739394092 conn(0x7f6a94071b90 msgr2=0x7f6a94071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.868+0000 7f6a99345700 1 -- 172.21.15.83:0/1739394092 shutdown_connections 2023-11-26T07:14:41.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.868+0000 7f6a99345700 1 -- 172.21.15.83:0/1739394092 wait complete. 2023-11-26T07:14:41.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.869+0000 7f6a99345700 1 Processor -- start 2023-11-26T07:14:41.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.869+0000 7f6a99345700 1 -- start start 2023-11-26T07:14:41.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a99345700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9406c710 0x7f6a940d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a99345700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a940af4d0 0x7f6a941607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a99345700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a9415e080 0x7f6a941f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a99345700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a94153fe0 con 0x7f6a9406c710 2023-11-26T07:14:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a99345700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a941542c0 con 0x7f6a940af4d0 2023-11-26T07:14:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a99345700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a94156420 con 0x7f6a9415e080 2023-11-26T07:14:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a92ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9406c710 0x7f6a940d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a92ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9406c710 0x7f6a940d0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37958/0 (socket says 172.21.15.83:37958) 2023-11-26T07:14:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a92ffd700 1 -- 172.21.15.83:0/92301649 learned_addr learned my addr 172.21.15.83:0/92301649 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a927fc700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a940af4d0 0x7f6a941607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a937fe700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a9415e080 0x7f6a941f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a92ffd700 1 -- 172.21.15.83:0/92301649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a9415e080 msgr2=0x7f6a941f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a92ffd700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a9415e080 0x7f6a941f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a92ffd700 1 -- 172.21.15.83:0/92301649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a940af4d0 msgr2=0x7f6a941607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a92ffd700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a940af4d0 0x7f6a941607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a92ffd700 1 -- 172.21.15.83:0/92301649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c009b30 con 0x7f6a9406c710 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a92ffd700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9406c710 0x7f6a940d0260 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800c450 tx=0x7f6a8800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a7b7fe700 1 -- 172.21.15.83:0/92301649 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a880052d0 con 0x7f6a9406c710 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a7b7fe700 1 -- 172.21.15.83:0/92301649 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a88005430 con 0x7f6a9406c710 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a99345700 1 -- 172.21.15.83:0/92301649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a94156700 con 0x7f6a9406c710 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.870+0000 7f6a7b7fe700 1 -- 172.21.15.83:0/92301649 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a880038c0 con 0x7f6a9406c710 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.871+0000 7f6a99345700 1 -- 172.21.15.83:0/92301649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a94160d10 con 0x7f6a9406c710 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.872+0000 7f6a99345700 1 -- 172.21.15.83:0/92301649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a94156d50 con 0x7f6a9406c710 2023-11-26T07:14:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.872+0000 7f6a7b7fe700 1 -- 172.21.15.83:0/92301649 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6a88003a20 con 0x7f6a9406c710 2023-11-26T07:14:41.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.872+0000 7f6a7b7fe700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6a8006b080 0x7f6a8006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:41.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.872+0000 7f6a7b7fe700 1 -- 172.21.15.83:0/92301649 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6a88088c00 con 0x7f6a9406c710 2023-11-26T07:14:41.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.872+0000 7f6a927fc700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6a8006b080 0x7f6a8006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:41.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.873+0000 7f6a927fc700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6a8006b080 0x7f6a8006d540 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6a84006fd0 tx=0x7f6a84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:41.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:41.877+0000 7f6a7b7fe700 1 -- 172.21.15.83:0/92301649 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a8808f080 con 0x7f6a9406c710 2023-11-26T07:14:42.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.107+0000 7f6a99345700 1 -- 172.21.15.83:0/92301649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a940753f0 con 0x7f6a9406c710 2023-11-26T07:14:42.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.107+0000 7f6a7b7fe700 1 -- 172.21.15.83:0/92301649 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f6a88053a80 con 0x7f6a9406c710 2023-11-26T07:14:42.110 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:42.110 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 -- 172.21.15.83:0/92301649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6a8006b080 msgr2=0x7f6a8006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6a8006b080 0x7f6a8006d540 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6a84006fd0 tx=0x7f6a84008040 comp rx=0 tx=0).stop 2023-11-26T07:14:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 -- 172.21.15.83:0/92301649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9406c710 msgr2=0x7f6a940d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9406c710 0x7f6a940d0260 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800c450 tx=0x7f6a8800c810 comp rx=0 tx=0).stop 2023-11-26T07:14:42.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 -- 172.21.15.83:0/92301649 shutdown_connections 2023-11-26T07:14:42.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6a8006b080 0x7f6a8006d540 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a9415e080 0x7f6a941f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9406c710 0x7f6a940d0260 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 --2- 172.21.15.83:0/92301649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a940af4d0 0x7f6a941607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 -- 172.21.15.83:0/92301649 >> 172.21.15.83:0/92301649 conn(0x7f6a94071b90 msgr2=0x7f6a9406bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:42.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 -- 172.21.15.83:0/92301649 shutdown_connections 2023-11-26T07:14:42.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.109+0000 7f6a99345700 1 -- 172.21.15.83:0/92301649 wait complete. 2023-11-26T07:14:42.113 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:42.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:42.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.241+0000 7f3bd531a700 1 Processor -- start 2023-11-26T07:14:42.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.241+0000 7f3bd531a700 1 -- start start 2023-11-26T07:14:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bd531a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd015e480 0x7f3bd00d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bd531a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bd00b6060 0x7f3bd00b7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bd531a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd00d2a70 con 0x7f3bd00d3610 2023-11-26T07:14:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bd531a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00d2d50 con 0x7f3bd00b6060 2023-11-26T07:14:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bd531a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00d3030 con 0x7f3bd015e480 2023-11-26T07:14:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bcf7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bd00b6060 0x7f3bd00b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bce7fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3bd00d3610 0x7f3bd00b5950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58006/0 (socket says 172.21.15.83:58006) 2023-11-26T07:14:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bce7fc700 1 -- 172.21.15.83:0/504231449 learned_addr learned my addr 172.21.15.83:0/504231449 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:42.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bceffd700 1 --2- 172.21.15.83:0/504231449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd015e480 0x7f3bd00d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bcf7fe700 1 -- 172.21.15.83:0/504231449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd015e480 msgr2=0x7f3bd00d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bcf7fe700 1 --2- 172.21.15.83:0/504231449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd015e480 0x7f3bd00d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bcf7fe700 1 -- 172.21.15.83:0/504231449 >> v1:172.21.15.83:6789/0 conn(0x7f3bd00d3610 legacy=0x7f3bd00b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bcdffb700 1 -- 172.21.15.83:0/504231449 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173967122 0 0) 0x7f3bd00d2a70 con 0x7f3bd00d3610 2023-11-26T07:14:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bcf7fe700 1 -- 172.21.15.83:0/504231449 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00b7340 con 0x7f3bd00b6060 2023-11-26T07:14:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bcf7fe700 1 --2- 172.21.15.83:0/504231449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bd00b6060 0x7f3bd00b7860 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009d90 tx=0x7f3bc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793252b9c8e2dc54 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bcdffb700 1 -- 172.21.15.83:0/504231449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800e070 con 0x7f3bd00b6060 2023-11-26T07:14:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bcdffb700 1 -- 172.21.15.83:0/504231449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc800bbd0 con 0x7f3bd00b6060 2023-11-26T07:14:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.242+0000 7f3bcdffb700 1 -- 172.21.15.83:0/504231449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8012430 con 0x7f3bd00b6060 2023-11-26T07:14:42.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.243+0000 7f3bd531a700 1 -- 172.21.15.83:0/504231449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bd00b6060 msgr2=0x7f3bd00b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.243+0000 7f3bd531a700 1 --2- 172.21.15.83:0/504231449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bd00b6060 0x7f3bd00b7860 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009d90 tx=0x7f3bc800b330 comp rx=0 tx=0).stop 2023-11-26T07:14:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.243+0000 7f3bd531a700 1 -- 172.21.15.83:0/504231449 shutdown_connections 2023-11-26T07:14:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.243+0000 7f3bd531a700 1 --2- 172.21.15.83:0/504231449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bd00b6060 0x7f3bd00b7860 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.243+0000 7f3bd531a700 1 --2- 172.21.15.83:0/504231449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd015e480 0x7f3bd00d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.243+0000 7f3bd531a700 1 -- 172.21.15.83:0/504231449 >> 172.21.15.83:0/504231449 conn(0x7f3bd00d0950 msgr2=0x7f3bd00d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.243+0000 7f3bd531a700 1 -- 172.21.15.83:0/504231449 shutdown_connections 2023-11-26T07:14:42.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.243+0000 7f3bd531a700 1 -- 172.21.15.83:0/504231449 wait complete. 2023-11-26T07:14:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.244+0000 7f3bd531a700 1 Processor -- start 2023-11-26T07:14:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.244+0000 7f3bd531a700 1 -- start start 2023-11-26T07:14:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.245+0000 7f3bd531a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd00b6060 0x7f3bd015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.245+0000 7f3bd531a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bd00d3610 0x7f3bd00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.245+0000 7f3bd531a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd015e480 0x7f3bd00cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.245+0000 7f3bd531a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00c8430 con 0x7f3bd00d3610 2023-11-26T07:14:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.245+0000 7f3bd531a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00c8680 con 0x7f3bd015e480 2023-11-26T07:14:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.245+0000 7f3bd531a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00c8960 con 0x7f3bd00b6060 2023-11-26T07:14:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.245+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bd00d3610 0x7f3bd00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.245+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bd00d3610 0x7f3bd00cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37974/0 (socket says 172.21.15.83:37974) 2023-11-26T07:14:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.245+0000 7f3bce7fc700 1 -- 172.21.15.83:0/2037707575 learned_addr learned my addr 172.21.15.83:0/2037707575 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.245+0000 7f3bceffd700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd00b6060 0x7f3bd015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.245+0000 7f3bcf7fe700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd015e480 0x7f3bd00cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bce7fc700 1 -- 172.21.15.83:0/2037707575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd00b6060 msgr2=0x7f3bd015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bce7fc700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd00b6060 0x7f3bd015aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bce7fc700 1 -- 172.21.15.83:0/2037707575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd015e480 msgr2=0x7f3bd00cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bce7fc700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd015e480 0x7f3bd00cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bce7fc700 1 -- 172.21.15.83:0/2037707575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8009a70 con 0x7f3bd00d3610 2023-11-26T07:14:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bcf7fe700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd015e480 0x7f3bd00cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bce7fc700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bd00d3610 0x7f3bd00cf750 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400c540 tx=0x7f3bc400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bb77fe700 1 -- 172.21.15.83:0/2037707575 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4004c80 con 0x7f3bd00d3610 2023-11-26T07:14:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bb77fe700 1 -- 172.21.15.83:0/2037707575 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc4004de0 con 0x7f3bd00d3610 2023-11-26T07:14:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bd531a700 1 -- 172.21.15.83:0/2037707575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd00c8c40 con 0x7f3bd00d3610 2023-11-26T07:14:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bb77fe700 1 -- 172.21.15.83:0/2037707575 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4007310 con 0x7f3bd00d3610 2023-11-26T07:14:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.246+0000 7f3bd531a700 1 -- 172.21.15.83:0/2037707575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd015af40 con 0x7f3bd00d3610 2023-11-26T07:14:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.247+0000 7f3bd531a700 1 -- 172.21.15.83:0/2037707575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd00751e0 con 0x7f3bd00d3610 2023-11-26T07:14:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.247+0000 7f3bb77fe700 1 -- 172.21.15.83:0/2037707575 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f3bc4007470 con 0x7f3bd00d3610 2023-11-26T07:14:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.248+0000 7f3bb77fe700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3bbc06b080 0x7f3bbc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.248+0000 7f3bb77fe700 1 -- 172.21.15.83:0/2037707575 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3bc4089570 con 0x7f3bd00d3610 2023-11-26T07:14:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.248+0000 7f3bceffd700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3bbc06b080 0x7f3bbc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.248+0000 7f3bceffd700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3bbc06b080 0x7f3bbc06d540 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0006b20 tx=0x7f3bc0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:42.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.252+0000 7f3bb77fe700 1 -- 172.21.15.83:0/2037707575 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3bc40543f0 con 0x7f3bd00d3610 2023-11-26T07:14:42.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.487+0000 7f3bd531a700 1 -- 172.21.15.83:0/2037707575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bd00c7780 con 0x7f3bd00d3610 2023-11-26T07:14:42.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.488+0000 7f3bb77fe700 1 -- 172.21.15.83:0/2037707575 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f3bc4057a10 con 0x7f3bd00d3610 2023-11-26T07:14:42.490 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:42.490 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:12:54.920948+0000","modified":"2023-11-26T07:12:57.942955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5544":{"gid":5544,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/2643742427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2643742427},{"type":"v1","addr":"172.21.15.83:6837","nonce":2643742427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:14:42.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 -- 172.21.15.83:0/2037707575 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3bbc06b080 msgr2=0x7f3bbc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3bbc06b080 0x7f3bbc06d540 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0006b20 tx=0x7f3bc0008040 comp rx=0 tx=0).stop 2023-11-26T07:14:42.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 -- 172.21.15.83:0/2037707575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bd00d3610 msgr2=0x7f3bd00cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bd00d3610 0x7f3bd00cf750 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400c540 tx=0x7f3bc400c850 comp rx=0 tx=0).stop 2023-11-26T07:14:42.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 -- 172.21.15.83:0/2037707575 shutdown_connections 2023-11-26T07:14:42.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd015e480 0x7f3bd00cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3bbc06b080 0x7f3bbc06d540 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd00b6060 0x7f3bd015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 --2- 172.21.15.83:0/2037707575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bd00d3610 0x7f3bd00cf750 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 -- 172.21.15.83:0/2037707575 >> 172.21.15.83:0/2037707575 conn(0x7f3bd00d0950 msgr2=0x7f3bd00b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:42.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 -- 172.21.15.83:0/2037707575 shutdown_connections 2023-11-26T07:14:42.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.490+0000 7f3bd531a700 1 -- 172.21.15.83:0/2037707575 wait complete. 2023-11-26T07:14:42.494 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2023-11-26T07:14:42.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:14:42.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.622+0000 7f8f119c8700 1 Processor -- start 2023-11-26T07:14:42.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.622+0000 7f8f119c8700 1 -- start start 2023-11-26T07:14:42.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.623+0000 7f8f119c8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8f0c15aee0 0x7f8f0c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.623+0000 7f8f119c8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8f0c0b4340 0x7f8f0c0ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.623+0000 7f8f119c8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0c073520 con 0x7f8f0c0aff00 2023-11-26T07:14:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.623+0000 7f8f119c8700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c0ae280 con 0x7f8f0c0b4340 2023-11-26T07:14:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.623+0000 7f8f119c8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c0adbc0 con 0x7f8f0c15aee0 2023-11-26T07:14:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.623+0000 7f8f0b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8f0c0b4340 0x7f8f0c0ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.623+0000 7f8f0a7fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8f0c0aff00 0x7f8f0c0b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58022/0 (socket says 172.21.15.83:58022) 2023-11-26T07:14:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.623+0000 7f8f0a7fc700 1 -- 172.21.15.83:0/1962183074 learned_addr learned my addr 172.21.15.83:0/1962183074 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.623+0000 7f8f0affd700 1 --2- 172.21.15.83:0/1962183074 >> v2:172.21.15.203:3300/0 conn(0x7f8f0c15aee0 0x7f8f0c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f0b7fe700 1 -- 172.21.15.83:0/1962183074 >> v2:172.21.15.203:3300/0 conn(0x7f8f0c15aee0 msgr2=0x7f8f0c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f0b7fe700 1 --2- 172.21.15.83:0/1962183074 >> v2:172.21.15.203:3300/0 conn(0x7f8f0c15aee0 0x7f8f0c0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f0b7fe700 1 -- 172.21.15.83:0/1962183074 >> v1:172.21.15.83:6789/0 conn(0x7f8f0c0aff00 legacy=0x7f8f0c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f0b7fe700 1 -- 172.21.15.83:0/1962183074 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0adea0 con 0x7f8f0c0b4340 2023-11-26T07:14:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f09ffb700 1 -- 172.21.15.83:0/1962183074 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612732975 0 0) 0x7f8f0c073520 con 0x7f8f0c0aff00 2023-11-26T07:14:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f0b7fe700 1 --2- 172.21.15.83:0/1962183074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f0c0b4340 0x7f8f0c0ae6a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c070670 tx=0x7f8f0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f43bfbbf4ca78f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f09ffb700 1 -- 172.21.15.83:0/1962183074 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0400e070 con 0x7f8f0c0b4340 2023-11-26T07:14:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f09ffb700 1 -- 172.21.15.83:0/1962183074 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0400bbd0 con 0x7f8f0c0b4340 2023-11-26T07:14:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f09ffb700 1 -- 172.21.15.83:0/1962183074 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04012430 con 0x7f8f0c0b4340 2023-11-26T07:14:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f119c8700 1 -- 172.21.15.83:0/1962183074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f0c0b4340 msgr2=0x7f8f0c0ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f119c8700 1 --2- 172.21.15.83:0/1962183074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f0c0b4340 0x7f8f0c0ae6a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c070670 tx=0x7f8f0400b330 comp rx=0 tx=0).stop 2023-11-26T07:14:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f119c8700 1 -- 172.21.15.83:0/1962183074 shutdown_connections 2023-11-26T07:14:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f119c8700 1 --2- 172.21.15.83:0/1962183074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f0c0b4340 0x7f8f0c0ae6a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f119c8700 1 --2- 172.21.15.83:0/1962183074 >> v2:172.21.15.203:3300/0 conn(0x7f8f0c15aee0 0x7f8f0c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f119c8700 1 -- 172.21.15.83:0/1962183074 >> 172.21.15.83:0/1962183074 conn(0x7f8f0c0b5640 msgr2=0x7f8f0c0b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f119c8700 1 -- 172.21.15.83:0/1962183074 shutdown_connections 2023-11-26T07:14:42.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.624+0000 7f8f119c8700 1 -- 172.21.15.83:0/1962183074 wait complete. 2023-11-26T07:14:42.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.626+0000 7f8f119c8700 1 Processor -- start 2023-11-26T07:14:42.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.626+0000 7f8f119c8700 1 -- start start 2023-11-26T07:14:42.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.626+0000 7f8f119c8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f0c0aff00 0x7f8f0c1efd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.626+0000 7f8f119c8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f0c0b4340 0x7f8f0c156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f119c8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f0c15aee0 0x7f8f0c156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f119c8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0cef10 con 0x7f8f0c0aff00 2023-11-26T07:14:42.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f119c8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0ce300 con 0x7f8f0c0b4340 2023-11-26T07:14:42.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f119c8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0ce5e0 con 0x7f8f0c15aee0 2023-11-26T07:14:42.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f0c0aff00 0x7f8f0c1efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f0c0aff00 0x7f8f0c1efd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37986/0 (socket says 172.21.15.83:37986) 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0affd700 1 -- 172.21.15.83:0/2843698029 learned_addr learned my addr 172.21.15.83:0/2843698029 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0a7fc700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f0c0b4340 0x7f8f0c156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0b7fe700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f0c15aee0 0x7f8f0c156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0affd700 1 -- 172.21.15.83:0/2843698029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f0c15aee0 msgr2=0x7f8f0c156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0affd700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f0c15aee0 0x7f8f0c156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0affd700 1 -- 172.21.15.83:0/2843698029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f0c0b4340 msgr2=0x7f8f0c156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0affd700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f0c0b4340 0x7f8f0c156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0affd700 1 -- 172.21.15.83:0/2843698029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f04009a70 con 0x7f8f0c0aff00 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0b7fe700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f0c15aee0 0x7f8f0c156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f0affd700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f0c0aff00 0x7f8f0c1efd40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8f00010700 tx=0x7f8f00010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8ef3fff700 1 -- 172.21.15.83:0/2843698029 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00011920 con 0x7f8f0c0aff00 2023-11-26T07:14:42.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8ef3fff700 1 -- 172.21.15.83:0/2843698029 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f00011a80 con 0x7f8f0c0aff00 2023-11-26T07:14:42.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f119c8700 1 -- 172.21.15.83:0/2843698029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c0ce8c0 con 0x7f8f0c0aff00 2023-11-26T07:14:42.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8ef3fff700 1 -- 172.21.15.83:0/2843698029 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00011e30 con 0x7f8f0c0aff00 2023-11-26T07:14:42.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.627+0000 7f8f119c8700 1 -- 172.21.15.83:0/2843698029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c1f0280 con 0x7f8f0c0aff00 2023-11-26T07:14:42.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.628+0000 7f8f119c8700 1 -- 172.21.15.83:0/2843698029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c067f60 con 0x7f8f0c0aff00 2023-11-26T07:14:42.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.629+0000 7f8ef3fff700 1 -- 172.21.15.83:0/2843698029 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f8f00019430 con 0x7f8f0c0aff00 2023-11-26T07:14:42.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.629+0000 7f8ef3fff700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ef806b080 0x7f8ef806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.629+0000 7f8ef3fff700 1 -- 172.21.15.83:0/2843698029 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8f0008b9e0 con 0x7f8f0c0aff00 2023-11-26T07:14:42.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.629+0000 7f8f0a7fc700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ef806b080 0x7f8ef806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.629+0000 7f8f0a7fc700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ef806b080 0x7f8ef806d540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c157520 tx=0x7f8efc007e70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:42.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.633+0000 7f8ef3fff700 1 -- 172.21.15.83:0/2843698029 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f00056740 con 0x7f8f0c0aff00 2023-11-26T07:14:42.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.825+0000 7f8f119c8700 1 -- 172.21.15.83:0/2843698029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f0c0625e0 con 0x7f8f0c0aff00 2023-11-26T07:14:42.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.826+0000 7f8ef3fff700 1 -- 172.21.15.83:0/2843698029 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f8f00059d60 con 0x7f8f0c0aff00 2023-11-26T07:14:42.829 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:42.829 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:13:10.603141+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:12:52.958067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:12:53.875459+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.1b","mappings":[{"from":0,"to":5}]},{"pgid":"7.2c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:14:42.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.828+0000 7f8f119c8700 1 -- 172.21.15.83:0/2843698029 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ef806b080 msgr2=0x7f8ef806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.828+0000 7f8f119c8700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ef806b080 0x7f8ef806d540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c157520 tx=0x7f8efc007e70 comp rx=0 tx=0).stop 2023-11-26T07:14:42.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.828+0000 7f8f119c8700 1 -- 172.21.15.83:0/2843698029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f0c0aff00 msgr2=0x7f8f0c1efd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.828+0000 7f8f119c8700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f0c0aff00 0x7f8f0c1efd40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8f00010700 tx=0x7f8f00010a10 comp rx=0 tx=0).stop 2023-11-26T07:14:42.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.828+0000 7f8f119c8700 1 -- 172.21.15.83:0/2843698029 shutdown_connections 2023-11-26T07:14:42.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.828+0000 7f8f119c8700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8ef806b080 0x7f8ef806d540 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.828+0000 7f8f119c8700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f0c15aee0 0x7f8f0c156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.828+0000 7f8f119c8700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f0c0aff00 0x7f8f0c1efd40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.828+0000 7f8f119c8700 1 --2- 172.21.15.83:0/2843698029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f0c0b4340 0x7f8f0c156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.828+0000 7f8f119c8700 1 -- 172.21.15.83:0/2843698029 >> 172.21.15.83:0/2843698029 conn(0x7f8f0c0b5640 msgr2=0x7f8f0c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:42.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.829+0000 7f8f119c8700 1 -- 172.21.15.83:0/2843698029 shutdown_connections 2023-11-26T07:14:42.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.829+0000 7f8f119c8700 1 -- 172.21.15.83:0/2843698029 wait complete. 2023-11-26T07:14:42.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:14:42.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.960+0000 7f83c5d3e700 1 Processor -- start 2023-11-26T07:14:42.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.960+0000 7f83c5d3e700 1 -- start start 2023-11-26T07:14:42.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.961+0000 7f83c5d3e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 0x7f83c00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.961+0000 7f83c5d3e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83c006c710 0x7f83c00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.961+0000 7f83c5d3e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c00ae430 con 0x7f83c00af4d0 2023-11-26T07:14:42.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.961+0000 7f83c5d3e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c00af0b0 con 0x7f83c006c710 2023-11-26T07:14:42.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.961+0000 7f83c5d3e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c00b05a0 con 0x7f83c015e080 2023-11-26T07:14:42.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83bf7fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 0x7f83c00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83bffff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83c006c710 0x7f83c00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83beffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f83c00af4d0 0x7f83c006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:42652/0 (socket says 172.21.15.83:42652) 2023-11-26T07:14:42.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83bf7fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 0x7f83c00b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:48452/0 (socket says 172.21.15.83:48452) 2023-11-26T07:14:42.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83beffd700 1 -- 172.21.15.83:0/608400384 learned_addr learned my addr 172.21.15.83:0/608400384 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:42.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83bffff700 1 -- 172.21.15.83:0/608400384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 msgr2=0x7f83c00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:42.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83bffff700 1 --2- 172.21.15.83:0/608400384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 0x7f83c00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83bffff700 1 -- 172.21.15.83:0/608400384 >> v1:172.21.15.203:6789/0 conn(0x7f83c00af4d0 legacy=0x7f83c006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83bffff700 1 -- 172.21.15.83:0/608400384 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c00ae710 con 0x7f83c006c710 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83bf7fe700 1 --2- 172.21.15.83:0/608400384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 0x7f83c00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83bffff700 1 --2- 172.21.15.83:0/608400384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c006c710 0x7f83c00afb90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009dc0 tx=0x7f83b800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b721c00f350c3c05 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83be7fc700 1 -- 172.21.15.83:0/608400384 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800e070 con 0x7f83c006c710 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83be7fc700 1 -- 172.21.15.83:0/608400384 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b800bae0 con 0x7f83c006c710 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.962+0000 7f83be7fc700 1 -- 172.21.15.83:0/608400384 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800be90 con 0x7f83c006c710 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.963+0000 7f83c5d3e700 1 -- 172.21.15.83:0/608400384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c006c710 msgr2=0x7f83c00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.963+0000 7f83c5d3e700 1 --2- 172.21.15.83:0/608400384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c006c710 0x7f83c00afb90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009dc0 tx=0x7f83b800b330 comp rx=0 tx=0).stop 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.963+0000 7f83c5d3e700 1 -- 172.21.15.83:0/608400384 shutdown_connections 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.963+0000 7f83c5d3e700 1 --2- 172.21.15.83:0/608400384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c006c710 0x7f83c00afb90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.963+0000 7f83c5d3e700 1 --2- 172.21.15.83:0/608400384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 0x7f83c00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.963+0000 7f83c5d3e700 1 -- 172.21.15.83:0/608400384 >> 172.21.15.83:0/608400384 conn(0x7f83c0071b90 msgr2=0x7f83c0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.963+0000 7f83c5d3e700 1 -- 172.21.15.83:0/608400384 shutdown_connections 2023-11-26T07:14:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.963+0000 7f83c5d3e700 1 -- 172.21.15.83:0/608400384 wait complete. 2023-11-26T07:14:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.964+0000 7f83c5d3e700 1 Processor -- start 2023-11-26T07:14:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.964+0000 7f83c5d3e700 1 -- start start 2023-11-26T07:14:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83c5d3e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83c006c710 0x7f83c0153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83c5d3e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c00af4d0 0x7f83c0159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83c5d3e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 0x7f83c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83c5d3e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c0153c80 con 0x7f83c006c710 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83c5d3e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c01f4750 con 0x7f83c00af4d0 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83c5d3e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c01598c0 con 0x7f83c015e080 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83bf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83c006c710 0x7f83c0153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83bf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83c006c710 0x7f83c0153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38000/0 (socket says 172.21.15.83:38000) 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83bf7fe700 1 -- 172.21.15.83:0/806281410 learned_addr learned my addr 172.21.15.83:0/806281410 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83beffd700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c00af4d0 0x7f83c0159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83bffff700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 0x7f83c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83bf7fe700 1 -- 172.21.15.83:0/806281410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 msgr2=0x7f83c006e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83bf7fe700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 0x7f83c006e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83bf7fe700 1 -- 172.21.15.83:0/806281410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c00af4d0 msgr2=0x7f83c0159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83bf7fe700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c00af4d0 0x7f83c0159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83bf7fe700 1 -- 172.21.15.83:0/806281410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a80094a0 con 0x7f83c006c710 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.965+0000 7f83bf7fe700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83c006c710 0x7f83c0153740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f83a800ebe0 tx=0x7f83a8009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.966+0000 7f83b7fff700 1 -- 172.21.15.83:0/806281410 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a800c8b0 con 0x7f83c006c710 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.966+0000 7f83b7fff700 1 -- 172.21.15.83:0/806281410 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a800ca10 con 0x7f83c006c710 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.966+0000 7f83c5d3e700 1 -- 172.21.15.83:0/806281410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b8009a70 con 0x7f83c006c710 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.966+0000 7f83b7fff700 1 -- 172.21.15.83:0/806281410 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a800cda0 con 0x7f83c006c710 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.966+0000 7f83c5d3e700 1 -- 172.21.15.83:0/806281410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c01f3e20 con 0x7f83c006c710 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.966+0000 7f83c5d3e700 1 -- 172.21.15.83:0/806281410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c00753f0 con 0x7f83c006c710 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.967+0000 7f83b7fff700 1 -- 172.21.15.83:0/806281410 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f83a8013470 con 0x7f83c006c710 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.968+0000 7f83b7fff700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f83a006b080 0x7f83a006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.968+0000 7f83beffd700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f83a006b080 0x7f83a006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:42.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.968+0000 7f83b7fff700 1 -- 172.21.15.83:0/806281410 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f83a80899c0 con 0x7f83c006c710 2023-11-26T07:14:42.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.968+0000 7f83beffd700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f83a006b080 0x7f83a006d540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f83c01f1350 tx=0x7f83b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:42.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:42.972+0000 7f83b7fff700 1 -- 172.21.15.83:0/806281410 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83a8054840 con 0x7f83c006c710 2023-11-26T07:14:43.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.202+0000 7f83c5d3e700 1 -- 172.21.15.83:0/806281410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f83c0067f60 con 0x7f83c006c710 2023-11-26T07:14:43.205 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-26T07:14:43.203+0000 7fcd25823700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:14:43.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.204+0000 7f83b7fff700 1 -- 172.21.15.83:0/806281410 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7f83a8017090 con 0x7f83c006c710 2023-11-26T07:14:43.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.206+0000 7f83c5d3e700 1 -- 172.21.15.83:0/806281410 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f83a006b080 msgr2=0x7f83a006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:43.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.206+0000 7f83c5d3e700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f83a006b080 0x7f83a006d540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f83c01f1350 tx=0x7f83b0009040 comp rx=0 tx=0).stop 2023-11-26T07:14:43.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.206+0000 7f83c5d3e700 1 -- 172.21.15.83:0/806281410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83c006c710 msgr2=0x7f83c0153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:43.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.206+0000 7f83c5d3e700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83c006c710 0x7f83c0153740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f83a800ebe0 tx=0x7f83a8009e20 comp rx=0 tx=0).stop 2023-11-26T07:14:43.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.206+0000 7f83c5d3e700 1 -- 172.21.15.83:0/806281410 shutdown_connections 2023-11-26T07:14:43.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.206+0000 7f83c5d3e700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f83a006b080 0x7f83a006d540 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:43.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.207+0000 7f83c5d3e700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83c015e080 0x7f83c006e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:43.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.207+0000 7f83c5d3e700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83c006c710 0x7f83c0153740 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:43.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.207+0000 7f83c5d3e700 1 --2- 172.21.15.83:0/806281410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83c00af4d0 0x7f83c0159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:43.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.207+0000 7f83c5d3e700 1 -- 172.21.15.83:0/806281410 >> 172.21.15.83:0/806281410 conn(0x7f83c0071b90 msgr2=0x7f83c006bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:43.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.207+0000 7f83c5d3e700 1 -- 172.21.15.83:0/806281410 shutdown_connections 2023-11-26T07:14:43.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.207+0000 7f83c5d3e700 1 -- 172.21.15.83:0/806281410 wait complete. 2023-11-26T07:14:43.211 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:14:43.223 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:14:43.246 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-26T07:14:43.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.344+0000 7fa3e6e6f700 1 Processor -- start 2023-11-26T07:14:43.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.344+0000 7fa3e6e6f700 1 -- start start 2023-11-26T07:14:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.344+0000 7fa3e6e6f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa3e015bf50 0x7fa3e00ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.344+0000 7fa3e6e6f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3e00ce830 0x7fa3e00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.344+0000 7fa3e6e6f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e00d0140 con 0x7fa3e00cf1f0 2023-11-26T07:14:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.344+0000 7fa3e6e6f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e015c330 con 0x7fa3e00ce830 2023-11-26T07:14:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.344+0000 7fa3e6e6f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e00d03e0 con 0x7fa3e015bf50 2023-11-26T07:14:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3dffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3e00ce830 0x7fa3e00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3e540c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa3e00cf1f0 0x7fa3e00c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58060/0 (socket says 172.21.15.83:58060) 2023-11-26T07:14:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3e540c700 1 -- 172.21.15.83:0/1035426041 learned_addr learned my addr 172.21.15.83:0/1035426041 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3dffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa3e00ce830 0x7fa3e00cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38020/0 (socket says 172.21.15.83:38020) 2023-11-26T07:14:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3e4c0b700 1 --2- 172.21.15.83:0/1035426041 >> v2:172.21.15.203:3300/0 conn(0x7fa3e015bf50 0x7fa3e00ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3dffff700 1 -- 172.21.15.83:0/1035426041 >> v2:172.21.15.203:3300/0 conn(0x7fa3e015bf50 msgr2=0x7fa3e00ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3dffff700 1 --2- 172.21.15.83:0/1035426041 >> v2:172.21.15.203:3300/0 conn(0x7fa3e015bf50 0x7fa3e00ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3dffff700 1 -- 172.21.15.83:0/1035426041 >> v1:172.21.15.83:6789/0 conn(0x7fa3e00cf1f0 legacy=0x7fa3e00c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3dffff700 1 -- 172.21.15.83:0/1035426041 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00d06c0 con 0x7fa3e00ce830 2023-11-26T07:14:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3df7fe700 1 -- 172.21.15.83:0/1035426041 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160145979 0 0) 0x7fa3e00d0140 con 0x7fa3e00cf1f0 2023-11-26T07:14:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3dffff700 1 --2- 172.21.15.83:0/1035426041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e00ce830 0x7fa3e00cecb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00715e0 tx=0x7fa3cc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370224a77f610aa4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3df7fe700 1 -- 172.21.15.83:0/1035426041 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc011830 con 0x7fa3e00ce830 2023-11-26T07:14:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3df7fe700 1 -- 172.21.15.83:0/1035426041 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc011990 con 0x7fa3e00ce830 2023-11-26T07:14:43.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.345+0000 7fa3df7fe700 1 -- 172.21.15.83:0/1035426041 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc011d20 con 0x7fa3e00ce830 2023-11-26T07:14:43.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.346+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/1035426041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e00ce830 msgr2=0x7fa3e00cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:43.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.346+0000 7fa3e6e6f700 1 --2- 172.21.15.83:0/1035426041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e00ce830 0x7fa3e00cecb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00715e0 tx=0x7fa3cc009b20 comp rx=0 tx=0).stop 2023-11-26T07:14:43.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.346+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/1035426041 shutdown_connections 2023-11-26T07:14:43.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.346+0000 7fa3e6e6f700 1 --2- 172.21.15.83:0/1035426041 >> v2:172.21.15.203:3300/0 conn(0x7fa3e015bf50 0x7fa3e00ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:43.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.346+0000 7fa3e6e6f700 1 --2- 172.21.15.83:0/1035426041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e00ce830 0x7fa3e00cecb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.346+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/1035426041 >> 172.21.15.83:0/1035426041 conn(0x7fa3e00b3780 msgr2=0x7fa3e00b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.346+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/1035426041 shutdown_connections 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.346+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/1035426041 wait complete. 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.347+0000 7fa3e6e6f700 1 Processor -- start 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.347+0000 7fa3e6e6f700 1 -- start start 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e6e6f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3e00ce830 0x7fa3e00c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e6e6f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3e00cf1f0 0x7fa3e01f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e6e6f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e015bf50 0x7fa3e00c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e6e6f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00b5fc0 con 0x7fa3e015bf50 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e6e6f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00ca500 con 0x7fa3e00ce830 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e6e6f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00ca7e0 con 0x7fa3e00cf1f0 2023-11-26T07:14:43.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e540c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e015bf50 0x7fa3e00c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e540c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e015bf50 0x7fa3e00c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38030/0 (socket says 172.21.15.83:38030) 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e540c700 1 -- 172.21.15.83:0/2038924351 learned_addr learned my addr 172.21.15.83:0/2038924351 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3dffff700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3e00cf1f0 0x7fa3e01f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e4c0b700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3e00ce830 0x7fa3e00c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e540c700 1 -- 172.21.15.83:0/2038924351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3e00cf1f0 msgr2=0x7fa3e01f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e540c700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3e00cf1f0 0x7fa3e01f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e540c700 1 -- 172.21.15.83:0/2038924351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3e00ce830 msgr2=0x7fa3e00c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e540c700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3e00ce830 0x7fa3e00c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e540c700 1 -- 172.21.15.83:0/2038924351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc0094a0 con 0x7fa3e015bf50 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3dffff700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3e00cf1f0 0x7fa3e01f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e4c0b700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3e00ce830 0x7fa3e00c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.348+0000 7fa3e540c700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e015bf50 0x7fa3e00c6010 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800cbe0 tx=0x7fa3d800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.349+0000 7fa3dd7fa700 1 -- 172.21.15.83:0/2038924351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d800abc0 con 0x7fa3e015bf50 2023-11-26T07:14:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.349+0000 7fa3dd7fa700 1 -- 172.21.15.83:0/2038924351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d800ad40 con 0x7fa3e015bf50 2023-11-26T07:14:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.349+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/2038924351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e00caac0 con 0x7fa3e015bf50 2023-11-26T07:14:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.349+0000 7fa3dd7fa700 1 -- 172.21.15.83:0/2038924351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8007310 con 0x7fa3e015bf50 2023-11-26T07:14:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.349+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/2038924351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e00ca210 con 0x7fa3e015bf50 2023-11-26T07:14:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.350+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/2038924351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e00b6100 con 0x7fa3e015bf50 2023-11-26T07:14:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.350+0000 7fa3dd7fa700 1 -- 172.21.15.83:0/2038924351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fa3d8007470 con 0x7fa3e015bf50 2023-11-26T07:14:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.351+0000 7fa3dd7fa700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c406b080 0x7fa3c406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.351+0000 7fa3dd7fa700 1 -- 172.21.15.83:0/2038924351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa3d8089830 con 0x7fa3e015bf50 2023-11-26T07:14:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.351+0000 7fa3e4c0b700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c406b080 0x7fa3c406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.351+0000 7fa3e4c0b700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c406b080 0x7fa3c406d540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009ac0 tx=0x7fa3d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.355+0000 7fa3dd7fa700 1 -- 172.21.15.83:0/2038924351 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3e00b6100 con 0x7fa3e015bf50 2023-11-26T07:14:43.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:43.592+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/2038924351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa3e00b6100 con 0x7fa3e015bf50 2023-11-26T07:14:44.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.205+0000 7fa3dd7fa700 1 -- 172.21.15.83:0/2038924351 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa3e00b6100 con 0x7fa3e015bf50 2023-11-26T07:14:44.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/2038924351 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c406b080 msgr2=0x7fa3c406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:44.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c406b080 0x7fa3c406d540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009ac0 tx=0x7fa3d4008040 comp rx=0 tx=0).stop 2023-11-26T07:14:44.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/2038924351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e015bf50 msgr2=0x7fa3e00c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:44.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e015bf50 0x7fa3e00c6010 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800cbe0 tx=0x7fa3d800a060 comp rx=0 tx=0).stop 2023-11-26T07:14:44.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/2038924351 shutdown_connections 2023-11-26T07:14:44.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e015bf50 0x7fa3e00c6010 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:44.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3e00ce830 0x7fa3e00c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:44.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3c406b080 0x7fa3c406d540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:44.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 --2- 172.21.15.83:0/2038924351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3e00cf1f0 0x7fa3e01f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:44.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/2038924351 >> 172.21.15.83:0/2038924351 conn(0x7fa3e00b3780 msgr2=0x7fa3e00b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:44.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/2038924351 shutdown_connections 2023-11-26T07:14:44.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.209+0000 7fa3e6e6f700 1 -- 172.21.15.83:0/2038924351 wait complete. 2023-11-26T07:14:44.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:14:44.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.393+0000 7fae46572700 1 Processor -- start 2023-11-26T07:14:44.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.393+0000 7fae46572700 1 -- start start 2023-11-26T07:14:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae46572700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae400746a0 0x7fae4006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae46572700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae400b59c0 0x7fae400b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae46572700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae400b5e50 con 0x7fae400b01e0 2023-11-26T07:14:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae46572700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae401470a0 con 0x7fae400746a0 2023-11-26T07:14:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae46572700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae400b05c0 con 0x7fae400b59c0 2023-11-26T07:14:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae3f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae400746a0 0x7fae4006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae3f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae400746a0 0x7fae4006e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38038/0 (socket says 172.21.15.83:38038) 2023-11-26T07:14:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae3ffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fae400b01e0 0x7fae40073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58076/0 (socket says 172.21.15.83:58076) 2023-11-26T07:14:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae3f7fe700 1 -- 172.21.15.83:0/3823982174 learned_addr learned my addr 172.21.15.83:0/3823982174 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae44b0f700 1 --2- 172.21.15.83:0/3823982174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae400b59c0 0x7fae400b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae3f7fe700 1 -- 172.21.15.83:0/3823982174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae400b59c0 msgr2=0x7fae400b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae3f7fe700 1 --2- 172.21.15.83:0/3823982174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae400b59c0 0x7fae400b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae3f7fe700 1 -- 172.21.15.83:0/3823982174 >> v1:172.21.15.83:6789/0 conn(0x7fae400b01e0 legacy=0x7fae40073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae3effd700 1 -- 172.21.15.83:0/3823982174 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68309608 0 0) 0x7fae28001110 con 0x7fae400b01e0 2023-11-26T07:14:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae3f7fe700 1 -- 172.21.15.83:0/3823982174 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae400b4250 con 0x7fae400746a0 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae44b0f700 1 --2- 172.21.15.83:0/3823982174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae400b59c0 0x7fae400b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae3f7fe700 1 --2- 172.21.15.83:0/3823982174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae400746a0 0x7fae4006e9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae300098f0 tx=0x7fae30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdc6b7dc2ba02af server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.394+0000 7fae3effd700 1 -- 172.21.15.83:0/3823982174 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3000b8a0 con 0x7fae400746a0 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.395+0000 7fae3effd700 1 -- 172.21.15.83:0/3823982174 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3000ba00 con 0x7fae400746a0 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.395+0000 7fae3effd700 1 -- 172.21.15.83:0/3823982174 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3000bd90 con 0x7fae400746a0 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.395+0000 7fae46572700 1 -- 172.21.15.83:0/3823982174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae400746a0 msgr2=0x7fae4006e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.395+0000 7fae46572700 1 --2- 172.21.15.83:0/3823982174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae400746a0 0x7fae4006e9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae300098f0 tx=0x7fae30009c00 comp rx=0 tx=0).stop 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.395+0000 7fae46572700 1 -- 172.21.15.83:0/3823982174 shutdown_connections 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.395+0000 7fae46572700 1 --2- 172.21.15.83:0/3823982174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae400b59c0 0x7fae400b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.395+0000 7fae46572700 1 --2- 172.21.15.83:0/3823982174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae400746a0 0x7fae4006e9c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.395+0000 7fae46572700 1 -- 172.21.15.83:0/3823982174 >> 172.21.15.83:0/3823982174 conn(0x7fae400d0b40 msgr2=0x7fae400d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.395+0000 7fae46572700 1 -- 172.21.15.83:0/3823982174 shutdown_connections 2023-11-26T07:14:44.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.395+0000 7fae46572700 1 -- 172.21.15.83:0/3823982174 wait complete. 2023-11-26T07:14:44.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.396+0000 7fae46572700 1 Processor -- start 2023-11-26T07:14:44.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.396+0000 7fae46572700 1 -- start start 2023-11-26T07:14:44.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.397+0000 7fae46572700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae400746a0 0x7fae4015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:44.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.397+0000 7fae46572700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae400b01e0 0x7fae40161440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:44.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.397+0000 7fae46572700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae400b59c0 0x7fae40144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:44.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.397+0000 7fae46572700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae40161980 con 0x7fae400b01e0 2023-11-26T07:14:44.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.397+0000 7fae46572700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4015dd50 con 0x7fae400b59c0 2023-11-26T07:14:44.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.397+0000 7fae46572700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4015dfd0 con 0x7fae400746a0 2023-11-26T07:14:44.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.397+0000 7fae3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae400b01e0 0x7fae40161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:44.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.397+0000 7fae3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae400b01e0 0x7fae40161440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38044/0 (socket says 172.21.15.83:38044) 2023-11-26T07:14:44.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.397+0000 7fae3f7fe700 1 -- 172.21.15.83:0/685592283 learned_addr learned my addr 172.21.15.83:0/685592283 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:44.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.397+0000 7fae44b0f700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae400b59c0 0x7fae40144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:44.404 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.397+0000 7fae3ffff700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae400746a0 0x7fae4015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.398+0000 7fae3f7fe700 1 -- 172.21.15.83:0/685592283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae400746a0 msgr2=0x7fae4015c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.398+0000 7fae3f7fe700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae400746a0 0x7fae4015c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.398+0000 7fae3f7fe700 1 -- 172.21.15.83:0/685592283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae400b59c0 msgr2=0x7fae40144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.398+0000 7fae3f7fe700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae400b59c0 0x7fae40144240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.398+0000 7fae3f7fe700 1 -- 172.21.15.83:0/685592283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30009550 con 0x7fae400b01e0 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.398+0000 7fae3f7fe700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae400b01e0 0x7fae40161440 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fae30000c00 tx=0x7fae30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.398+0000 7fae3cff9700 1 -- 172.21.15.83:0/685592283 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300036a0 con 0x7fae400b01e0 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.398+0000 7fae3cff9700 1 -- 172.21.15.83:0/685592283 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3001e070 con 0x7fae400b01e0 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.398+0000 7fae46572700 1 -- 172.21.15.83:0/685592283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4015e2b0 con 0x7fae400b01e0 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.398+0000 7fae3cff9700 1 -- 172.21.15.83:0/685592283 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30003a20 con 0x7fae400b01e0 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.398+0000 7fae46572700 1 -- 172.21.15.83:0/685592283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4015e870 con 0x7fae400b01e0 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.399+0000 7fae46572700 1 -- 172.21.15.83:0/685592283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4015ea00 con 0x7fae400b01e0 2023-11-26T07:14:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.399+0000 7fae3cff9700 1 -- 172.21.15.83:0/685592283 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fae30003b80 con 0x7fae400b01e0 2023-11-26T07:14:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.400+0000 7fae3cff9700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2006b080 0x7fae2006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.400+0000 7fae3ffff700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2006b080 0x7fae2006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.400+0000 7fae3cff9700 1 -- 172.21.15.83:0/685592283 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fae30020070 con 0x7fae400b01e0 2023-11-26T07:14:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.400+0000 7fae3ffff700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2006b080 0x7fae2006d540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fae28000f60 tx=0x7fae28007c20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:44.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.404+0000 7fae3cff9700 1 -- 172.21.15.83:0/685592283 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae30055be0 con 0x7fae400b01e0 2023-11-26T07:14:44.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:44.594+0000 7fae46572700 1 -- 172.21.15.83:0/685592283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fae40067f60 con 0x7fae400b01e0 2023-11-26T07:14:45.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.211+0000 7fae3cff9700 1 -- 172.21.15.83:0/685592283 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) v1 ==== 181+0+0 (secure 0 0 0) 0x7fae3002a0d0 con 0x7fae400b01e0 2023-11-26T07:14:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.214+0000 7fae46572700 1 -- 172.21.15.83:0/685592283 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2006b080 msgr2=0x7fae2006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.214+0000 7fae46572700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2006b080 0x7fae2006d540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fae28000f60 tx=0x7fae28007c20 comp rx=0 tx=0).stop 2023-11-26T07:14:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.214+0000 7fae46572700 1 -- 172.21.15.83:0/685592283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae400b01e0 msgr2=0x7fae40161440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.214+0000 7fae46572700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae400b01e0 0x7fae40161440 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fae30000c00 tx=0x7fae30004750 comp rx=0 tx=0).stop 2023-11-26T07:14:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.215+0000 7fae46572700 1 -- 172.21.15.83:0/685592283 shutdown_connections 2023-11-26T07:14:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.215+0000 7fae46572700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae400b59c0 0x7fae40144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.215+0000 7fae46572700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2006b080 0x7fae2006d540 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.215+0000 7fae46572700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae400746a0 0x7fae4015c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.215+0000 7fae46572700 1 --2- 172.21.15.83:0/685592283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae400b01e0 0x7fae40161440 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.215+0000 7fae46572700 1 -- 172.21.15.83:0/685592283 >> 172.21.15.83:0/685592283 conn(0x7fae400d0b40 msgr2=0x7fae40072450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.215+0000 7fae46572700 1 -- 172.21.15.83:0/685592283 shutdown_connections 2023-11-26T07:14:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.215+0000 7fae46572700 1 -- 172.21.15.83:0/685592283 wait complete. 2023-11-26T07:14:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:14:45.231 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:14:45.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.357+0000 7f687d259700 1 Processor -- start 2023-11-26T07:14:45.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.357+0000 7f687d259700 1 -- start start 2023-11-26T07:14:45.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.358+0000 7f687d259700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687806e830 0x7f687806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:45.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.358+0000 7f687d259700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687806f1c0 con 0x7f687806f480 2023-11-26T07:14:45.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.358+0000 7f687d259700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687806c9b0 con 0x7f6878144600 2023-11-26T07:14:45.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.358+0000 7f687d259700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687806aff0 con 0x7f687806e830 2023-11-26T07:14:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.358+0000 7f687659c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f687806f480 0x7f687806c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58092/0 (socket says 172.21.15.83:58092) 2023-11-26T07:14:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.358+0000 7f687659c700 1 -- 172.21.15.83:0/1035603474 learned_addr learned my addr 172.21.15.83:0/1035603474 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286424212 0 0) 0x7f686c001110 con 0x7f687806f480 2023-11-26T07:14:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f687759e700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687806e830 0x7f687806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68600035b0 con 0x7f687806f480 2023-11-26T07:14:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3477087669 0 0) 0x7f68600035b0 con 0x7f687806f480 2023-11-26T07:14:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687806e830 msgr2=0x7f687806ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687806e830 0x7f687806ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 >> v1:172.21.15.203:6789/0 conn(0x7f6878144600 legacy=0x7f68780b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687806b200 con 0x7f687806f480 2023-11-26T07:14:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f686c001800 con 0x7f687806f480 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 >> v1:172.21.15.83:6789/0 conn(0x7f687806f480 legacy=0x7f687806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6860003a30 0x7f6860003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6860004850 0x7f6860004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687806e830 0x7f6860005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687806aff0 con 0x7f6860003a30 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686000bac0 con 0x7f6860004850 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686000de90 con 0x7f687806e830 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6876d9d700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6860003a30 0x7f6860003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f687659c700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6860004850 0x7f6860004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f687759e700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687806e830 0x7f6860005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6876d9d700 1 -- 172.21.15.83:0/1035603474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687806e830 msgr2=0x7f6860005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6876d9d700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f687806e830 0x7f6860005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6876d9d700 1 -- 172.21.15.83:0/1035603474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6860004850 msgr2=0x7f6860004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6876d9d700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6860004850 0x7f6860004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.359+0000 7f6876d9d700 1 -- 172.21.15.83:0/1035603474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68600035b0 con 0x7f6860003a30 2023-11-26T07:14:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.360+0000 7f6876d9d700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6860003a30 0x7f6860003e50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6868007fd0 tx=0x7f6860005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896bff657ceb1c58 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.360+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68680052d0 con 0x7f6860003a30 2023-11-26T07:14:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.360+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6868005470 con 0x7f6860003a30 2023-11-26T07:14:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.360+0000 7f6875d9b700 1 -- 172.21.15.83:0/1035603474 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68680038c0 con 0x7f6860003a30 2023-11-26T07:14:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.360+0000 7f687d259700 1 -- 172.21.15.83:0/1035603474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6860003a30 msgr2=0x7f6860003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.360+0000 7f687d259700 1 --2- 172.21.15.83:0/1035603474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6860003a30 0x7f6860003e50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6868007fd0 tx=0x7f6860005880 comp rx=0 tx=0).stop 2023-11-26T07:14:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.360+0000 7f687759e700 1 -- 172.21.15.83:0/1035603474 reap_dead start 2023-11-26T07:14:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.360+0000 7f687d259700 1 -- 172.21.15.83:0/1035603474 shutdown_connections 2023-11-26T07:14:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.360+0000 7f687d259700 1 -- 172.21.15.83:0/1035603474 >> 172.21.15.83:0/1035603474 conn(0x7f68780d0b40 msgr2=0x7f68780d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.360+0000 7f687d259700 1 -- 172.21.15.83:0/1035603474 shutdown_connections 2023-11-26T07:14:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.360+0000 7f687d259700 1 -- 172.21.15.83:0/1035603474 wait complete. 2023-11-26T07:14:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.362+0000 7f687d259700 1 Processor -- start 2023-11-26T07:14:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.362+0000 7f687d259700 1 -- start start 2023-11-26T07:14:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.362+0000 7f687d259700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6860003a30 0x7f68781f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.362+0000 7f687d259700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6878159d10 0x7f68780afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.362+0000 7f687d259700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68780b0050 0x7f687815b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.362+0000 7f687d259700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68780b04e0 con 0x7f6878159d10 2023-11-26T07:14:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.362+0000 7f687d259700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687815dc80 con 0x7f68780b0050 2023-11-26T07:14:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687d259700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68781f8720 con 0x7f6860003a30 2023-11-26T07:14:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6878159d10 0x7f68780afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6878159d10 0x7f68780afb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38064/0 (socket says 172.21.15.83:38064) 2023-11-26T07:14:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687659c700 1 -- 172.21.15.83:0/2848437712 learned_addr learned my addr 172.21.15.83:0/2848437712 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f6876d9d700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6860003a30 0x7f68781f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687759e700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68780b0050 0x7f687815b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687659c700 1 -- 172.21.15.83:0/2848437712 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6860003a30 msgr2=0x7f68781f81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687659c700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6860003a30 0x7f68781f81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687659c700 1 -- 172.21.15.83:0/2848437712 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68780b0050 msgr2=0x7f687815b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687659c700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68780b0050 0x7f687815b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687659c700 1 -- 172.21.15.83:0/2848437712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687815c940 con 0x7f6878159d10 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687659c700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6878159d10 0x7f68780afb10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f686c007290 tx=0x7f686c0075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f685f7fe700 1 -- 172.21.15.83:0/2848437712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c011580 con 0x7f6878159d10 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f685f7fe700 1 -- 172.21.15.83:0/2848437712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c011700 con 0x7f6878159d10 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687d259700 1 -- 172.21.15.83:0/2848437712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687815cbe0 con 0x7f6878159d10 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f685f7fe700 1 -- 172.21.15.83:0/2848437712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c011af0 con 0x7f6878159d10 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.363+0000 7f687d259700 1 -- 172.21.15.83:0/2848437712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687815d1a0 con 0x7f6878159d10 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.364+0000 7f687d259700 1 -- 172.21.15.83:0/2848437712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6878067f60 con 0x7f6878159d10 2023-11-26T07:14:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.365+0000 7f685f7fe700 1 -- 172.21.15.83:0/2848437712 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f686c011c50 con 0x7f6878159d10 2023-11-26T07:14:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.365+0000 7f685f7fe700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686406b080 0x7f686406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.365+0000 7f685f7fe700 1 -- 172.21.15.83:0/2848437712 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f686c0896a0 con 0x7f6878159d10 2023-11-26T07:14:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.365+0000 7f6876d9d700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686406b080 0x7f686406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.365+0000 7f6876d9d700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686406b080 0x7f686406d540 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6868007fd0 tx=0x7f6868009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.369+0000 7f685f7fe700 1 -- 172.21.15.83:0/2848437712 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f686c0546b0 con 0x7f6878159d10 2023-11-26T07:14:45.563 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:45.561+0000 7f687d259700 1 -- 172.21.15.83:0/2848437712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f687806f1c0 con 0x7f6878159d10 2023-11-26T07:14:46.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.213+0000 7f685f7fe700 1 -- 172.21.15.83:0/2848437712 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 169+0+0 (secure 0 0 0) 0x7f687806f1c0 con 0x7f6878159d10 2023-11-26T07:14:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.216+0000 7f687d259700 1 -- 172.21.15.83:0/2848437712 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686406b080 msgr2=0x7f686406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.216+0000 7f687d259700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686406b080 0x7f686406d540 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6868007fd0 tx=0x7f6868009030 comp rx=0 tx=0).stop 2023-11-26T07:14:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.216+0000 7f687d259700 1 -- 172.21.15.83:0/2848437712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6878159d10 msgr2=0x7f68780afb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.216+0000 7f687d259700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6878159d10 0x7f68780afb10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f686c007290 tx=0x7f686c0075a0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.216+0000 7f687d259700 1 -- 172.21.15.83:0/2848437712 shutdown_connections 2023-11-26T07:14:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.216+0000 7f687d259700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68780b0050 0x7f687815b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.216+0000 7f687d259700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f686406b080 0x7f686406d540 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.216+0000 7f687d259700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6860003a30 0x7f68781f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.216+0000 7f687d259700 1 --2- 172.21.15.83:0/2848437712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6878159d10 0x7f68780afb10 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.216+0000 7f687d259700 1 -- 172.21.15.83:0/2848437712 >> 172.21.15.83:0/2848437712 conn(0x7f68780d0b40 msgr2=0x7f68780d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:46.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.217+0000 7f687d259700 1 -- 172.21.15.83:0/2848437712 shutdown_connections 2023-11-26T07:14:46.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.217+0000 7f687d259700 1 -- 172.21.15.83:0/2848437712 wait complete. 2023-11-26T07:14:46.220 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2023-11-26T07:14:46.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:14:46.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.358+0000 7fc8d37fc700 1 Processor -- start 2023-11-26T07:14:46.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.359+0000 7fc8d37fc700 1 -- start start 2023-11-26T07:14:46.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.359+0000 7fc8d37fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8cc0c6c30 0x7fc8cc0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.359+0000 7fc8d37fc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8cc0cef40 0x7fc8cc0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.359+0000 7fc8d37fc700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8cc0d0140 con 0x7fc8cc15bf50 2023-11-26T07:14:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.359+0000 7fc8d37fc700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8cc15c330 con 0x7fc8cc0c6c30 2023-11-26T07:14:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.359+0000 7fc8d37fc700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc0cf3d0 con 0x7fc8cc0cef40 2023-11-26T07:14:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.359+0000 7fc8d0d97700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8cc0c6c30 0x7fc8cc0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.359+0000 7fc8d1598700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc8cc15bf50 0x7fc8cc0ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58110/0 (socket says 172.21.15.83:58110) 2023-11-26T07:14:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.359+0000 7fc8d1598700 1 -- 172.21.15.83:0/2802875359 learned_addr learned my addr 172.21.15.83:0/2802875359 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.359+0000 7fc8d0d97700 1 --2- 172.21.15.83:0/2802875359 >> v2:172.21.15.83:3300/0 conn(0x7fc8cc0c6c30 0x7fc8cc0cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38072/0 (socket says 172.21.15.83:38072) 2023-11-26T07:14:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.360+0000 7fc8d0d97700 1 -- 172.21.15.83:0/2802875359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8cc0cef40 msgr2=0x7fc8cc0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.360+0000 7fc8d0d97700 1 --2- 172.21.15.83:0/2802875359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8cc0cef40 0x7fc8cc0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.360+0000 7fc8d0d97700 1 -- 172.21.15.83:0/2802875359 >> v1:172.21.15.83:6789/0 conn(0x7fc8cc15bf50 legacy=0x7fc8cc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.360+0000 7fc8c3fff700 1 -- 172.21.15.83:0/2802875359 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359375558 0 0) 0x7fc8cc0d0140 con 0x7fc8cc15bf50 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.360+0000 7fc8d0d97700 1 -- 172.21.15.83:0/2802875359 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc0d03e0 con 0x7fc8cc0c6c30 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.360+0000 7fc8d0d97700 1 --2- 172.21.15.83:0/2802875359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8cc0c6c30 0x7fc8cc0cea00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009840 tx=0x7fc8c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9b822904926e0c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.360+0000 7fc8c3fff700 1 -- 172.21.15.83:0/2802875359 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011830 con 0x7fc8cc0c6c30 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.360+0000 7fc8c3fff700 1 -- 172.21.15.83:0/2802875359 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c8011990 con 0x7fc8cc0c6c30 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.360+0000 7fc8c3fff700 1 -- 172.21.15.83:0/2802875359 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011d20 con 0x7fc8cc0c6c30 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.361+0000 7fc8d37fc700 1 -- 172.21.15.83:0/2802875359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8cc0c6c30 msgr2=0x7fc8cc0cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.361+0000 7fc8d37fc700 1 --2- 172.21.15.83:0/2802875359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8cc0c6c30 0x7fc8cc0cea00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009840 tx=0x7fc8c8009b50 comp rx=0 tx=0).stop 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.361+0000 7fc8d37fc700 1 -- 172.21.15.83:0/2802875359 shutdown_connections 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.361+0000 7fc8d37fc700 1 --2- 172.21.15.83:0/2802875359 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8cc0cef40 0x7fc8cc0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.361+0000 7fc8d37fc700 1 --2- 172.21.15.83:0/2802875359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8cc0c6c30 0x7fc8cc0cea00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.361+0000 7fc8d37fc700 1 -- 172.21.15.83:0/2802875359 >> 172.21.15.83:0/2802875359 conn(0x7fc8cc0b3780 msgr2=0x7fc8cc0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:46.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.361+0000 7fc8d37fc700 1 -- 172.21.15.83:0/2802875359 shutdown_connections 2023-11-26T07:14:46.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.361+0000 7fc8d37fc700 1 -- 172.21.15.83:0/2802875359 wait complete. 2023-11-26T07:14:46.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.362+0000 7fc8d37fc700 1 Processor -- start 2023-11-26T07:14:46.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.362+0000 7fc8d37fc700 1 -- start start 2023-11-26T07:14:46.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d37fc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8cc0c6c30 0x7fc8cc159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d37fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8cc0cef40 0x7fc8cc09e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d37fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8cc15bf50 0x7fc8cc09e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d37fc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc1473f0 con 0x7fc8cc15bf50 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d37fc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc0ca850 con 0x7fc8cc0cef40 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d37fc700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc0cab30 con 0x7fc8cc0c6c30 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d1d99700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8cc15bf50 0x7fc8cc09e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d1d99700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8cc15bf50 0x7fc8cc09e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38080/0 (socket says 172.21.15.83:38080) 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d1d99700 1 -- 172.21.15.83:0/854818504 learned_addr learned my addr 172.21.15.83:0/854818504 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d1598700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8cc0c6c30 0x7fc8cc159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d0d97700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8cc0cef40 0x7fc8cc09e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d1d99700 1 -- 172.21.15.83:0/854818504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8cc0c6c30 msgr2=0x7fc8cc159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d1d99700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8cc0c6c30 0x7fc8cc159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d1d99700 1 -- 172.21.15.83:0/854818504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8cc0cef40 msgr2=0x7fc8cc09e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d1d99700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8cc0cef40 0x7fc8cc09e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d1d99700 1 -- 172.21.15.83:0/854818504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c80094a0 con 0x7fc8cc15bf50 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d1d99700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8cc15bf50 0x7fc8cc09e690 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400b970 tx=0x7fc8c400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8c1ffb700 1 -- 172.21.15.83:0/854818504 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c400c9b0 con 0x7fc8cc15bf50 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8c1ffb700 1 -- 172.21.15.83:0/854818504 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c400cb30 con 0x7fc8cc15bf50 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8d37fc700 1 -- 172.21.15.83:0/854818504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8cc146d30 con 0x7fc8cc15bf50 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.363+0000 7fc8c1ffb700 1 -- 172.21.15.83:0/854818504 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4014440 con 0x7fc8cc15bf50 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.364+0000 7fc8d37fc700 1 -- 172.21.15.83:0/854818504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc0c9bd0 con 0x7fc8cc15bf50 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.365+0000 7fc8d37fc700 1 -- 172.21.15.83:0/854818504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8cc068fb0 con 0x7fc8cc15bf50 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.365+0000 7fc8c1ffb700 1 -- 172.21.15.83:0/854818504 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc8c401d3b0 con 0x7fc8cc15bf50 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.365+0000 7fc8c1ffb700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8b006b080 0x7fc8b006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.365+0000 7fc8c1ffb700 1 -- 172.21.15.83:0/854818504 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc8c4006e40 con 0x7fc8cc15bf50 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.365+0000 7fc8d1598700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8b006b080 0x7fc8b006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.366+0000 7fc8d1598700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8b006b080 0x7fc8b006d540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc000f60 tx=0x7fc8bc007c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:46.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.370+0000 7fc8c1ffb700 1 -- 172.21.15.83:0/854818504 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8c4054510 con 0x7fc8cc15bf50 2023-11-26T07:14:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.559+0000 7fc8d37fc700 1 -- 172.21.15.83:0/854818504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc8cc147530 con 0x7fc8cc15bf50 2023-11-26T07:14:46.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.562+0000 7fc8c1ffb700 1 -- 172.21.15.83:0/854818504 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc8cc147530 con 0x7fc8cc15bf50 2023-11-26T07:14:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 -- 172.21.15.83:0/854818504 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8b006b080 msgr2=0x7fc8b006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8b006b080 0x7fc8b006d540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc000f60 tx=0x7fc8bc007c90 comp rx=0 tx=0).stop 2023-11-26T07:14:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 -- 172.21.15.83:0/854818504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8cc15bf50 msgr2=0x7fc8cc09e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8cc15bf50 0x7fc8cc09e690 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400b970 tx=0x7fc8c400bd30 comp rx=0 tx=0).stop 2023-11-26T07:14:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 -- 172.21.15.83:0/854818504 shutdown_connections 2023-11-26T07:14:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8cc15bf50 0x7fc8cc09e690 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc8b006b080 0x7fc8b006d540 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8cc0c6c30 0x7fc8cc159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 --2- 172.21.15.83:0/854818504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8cc0cef40 0x7fc8cc09e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 -- 172.21.15.83:0/854818504 >> 172.21.15.83:0/854818504 conn(0x7fc8cc0b3780 msgr2=0x7fc8cc0ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 -- 172.21.15.83:0/854818504 shutdown_connections 2023-11-26T07:14:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.565+0000 7fc8d37fc700 1 -- 172.21.15.83:0/854818504 wait complete. 2023-11-26T07:14:46.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-11-26T07:14:46.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.697+0000 7f2281f2e700 1 Processor -- start 2023-11-26T07:14:46.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.697+0000 7f2281f2e700 1 -- start start 2023-11-26T07:14:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.698+0000 7f2281f2e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f227c15e080 0x7f227c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.698+0000 7f2281f2e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227c0ae430 con 0x7f227c0af4d0 2023-11-26T07:14:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.698+0000 7f2281f2e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227c0ae710 con 0x7f227c06c710 2023-11-26T07:14:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.698+0000 7f2281f2e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227c0af0b0 con 0x7f227c15e080 2023-11-26T07:14:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.698+0000 7f227affd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f227c0af4d0 0x7f227c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58126/0 (socket says 172.21.15.83:58126) 2023-11-26T07:14:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.698+0000 7f227affd700 1 -- 172.21.15.83:0/1610846468 learned_addr learned my addr 172.21.15.83:0/1610846468 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.698+0000 7f227b7fe700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f227c15e080 0x7f227c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.698+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272863965 0 0) 0x7f227c0ae430 con 0x7f227c0af4d0 2023-11-26T07:14:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.698+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22680035b0 con 0x7f227c0af4d0 2023-11-26T07:14:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2869747780 0 0) 0x7f22680035b0 con 0x7f227c0af4d0 2023-11-26T07:14:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f227c15e080 msgr2=0x7f227c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f227c15e080 0x7f227c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 >> v1:172.21.15.203:6789/0 conn(0x7f227c06c710 legacy=0x7f227c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c0ae9f0 con 0x7f227c0af4d0 2023-11-26T07:14:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f226c001670 con 0x7f227c0af4d0 2023-11-26T07:14:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 >> v1:172.21.15.83:6789/0 conn(0x7f227c0af4d0 legacy=0x7f227c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2268003a30 0x7f2268003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f227c15e080 0x7f2268004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2268004d90 0x7f2268005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c0af0b0 con 0x7f2268003a30 2023-11-26T07:14:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226800bac0 con 0x7f2268004d90 2023-11-26T07:14:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226800bc80 con 0x7f227c15e080 2023-11-26T07:14:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227b7fe700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2268003a30 0x7f2268003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227affd700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f227c15e080 0x7f2268004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227bfff700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2268004d90 0x7f2268005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227b7fe700 1 -- 172.21.15.83:0/1610846468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f227c15e080 msgr2=0x7f2268004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:46.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227b7fe700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f227c15e080 0x7f2268004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227b7fe700 1 -- 172.21.15.83:0/1610846468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2268004d90 msgr2=0x7f2268005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227b7fe700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2268004d90 0x7f2268005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227b7fe700 1 -- 172.21.15.83:0/1610846468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22680035b0 con 0x7f2268003a30 2023-11-26T07:14:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.699+0000 7f227b7fe700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2268003a30 0x7f2268003e50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2264009b30 tx=0x7f2268005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc610ad6bcc697a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.700+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400e6c0 con 0x7f2268003a30 2023-11-26T07:14:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.700+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400e820 con 0x7f2268003a30 2023-11-26T07:14:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.700+0000 7f227a7fc700 1 -- 172.21.15.83:0/1610846468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400eb90 con 0x7f2268003a30 2023-11-26T07:14:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.700+0000 7f2281f2e700 1 -- 172.21.15.83:0/1610846468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2268003a30 msgr2=0x7f2268003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.700+0000 7f2281f2e700 1 --2- 172.21.15.83:0/1610846468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2268003a30 0x7f2268003e50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2264009b30 tx=0x7f2268005880 comp rx=0 tx=0).stop 2023-11-26T07:14:46.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.700+0000 7f227bfff700 1 -- 172.21.15.83:0/1610846468 reap_dead start 2023-11-26T07:14:46.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.700+0000 7f2281f2e700 1 -- 172.21.15.83:0/1610846468 shutdown_connections 2023-11-26T07:14:46.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.700+0000 7f2281f2e700 1 -- 172.21.15.83:0/1610846468 >> 172.21.15.83:0/1610846468 conn(0x7f227c071b90 msgr2=0x7f227c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:46.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.700+0000 7f2281f2e700 1 -- 172.21.15.83:0/1610846468 shutdown_connections 2023-11-26T07:14:46.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.700+0000 7f2281f2e700 1 -- 172.21.15.83:0/1610846468 wait complete. 2023-11-26T07:14:46.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.701+0000 7f2281f2e700 1 Processor -- start 2023-11-26T07:14:46.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.701+0000 7f2281f2e700 1 -- start start 2023-11-26T07:14:46.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.702+0000 7f2281f2e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2268003a30 0x7f227c151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.702+0000 7f2281f2e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227c153180 0x7f227c1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.702+0000 7f2281f2e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f227c153600 0x7f227c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.702+0000 7f2281f2e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c151a60 con 0x7f227c153180 2023-11-26T07:14:46.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.702+0000 7f2281f2e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c161220 con 0x7f227c153600 2023-11-26T07:14:46.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.702+0000 7f2281f2e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227c158840 con 0x7f2268003a30 2023-11-26T07:14:46.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.702+0000 7f227affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227c153180 0x7f227c1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.702+0000 7f227affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227c153180 0x7f227c1595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38098/0 (socket says 172.21.15.83:38098) 2023-11-26T07:14:46.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.702+0000 7f227affd700 1 -- 172.21.15.83:0/4040268813 learned_addr learned my addr 172.21.15.83:0/4040268813 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:46.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.702+0000 7f227b7fe700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2268003a30 0x7f227c151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.702+0000 7f227bfff700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f227c153600 0x7f227c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.703+0000 7f227affd700 1 -- 172.21.15.83:0/4040268813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2268003a30 msgr2=0x7f227c151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.703+0000 7f227affd700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2268003a30 0x7f227c151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.703+0000 7f227affd700 1 -- 172.21.15.83:0/4040268813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f227c153600 msgr2=0x7f227c159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:46.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.703+0000 7f227affd700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f227c153600 0x7f227c159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:46.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.703+0000 7f227affd700 1 -- 172.21.15.83:0/4040268813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c15e590 con 0x7f227c153180 2023-11-26T07:14:46.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.703+0000 7f227affd700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227c153180 0x7f227c1595f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f226c0011e0 tx=0x7f226c007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:46.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.703+0000 7f2273fff700 1 -- 172.21.15.83:0/4040268813 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00f6c0 con 0x7f227c153180 2023-11-26T07:14:46.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.703+0000 7f2273fff700 1 -- 172.21.15.83:0/4040268813 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c00f840 con 0x7f227c153180 2023-11-26T07:14:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.703+0000 7f2281f2e700 1 -- 172.21.15.83:0/4040268813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227c15e830 con 0x7f227c153180 2023-11-26T07:14:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.703+0000 7f2273fff700 1 -- 172.21.15.83:0/4040268813 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00fbf0 con 0x7f227c153180 2023-11-26T07:14:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.703+0000 7f2281f2e700 1 -- 172.21.15.83:0/4040268813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c15edf0 con 0x7f227c153180 2023-11-26T07:14:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.704+0000 7f2281f2e700 1 -- 172.21.15.83:0/4040268813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227c15efb0 con 0x7f227c153180 2023-11-26T07:14:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.704+0000 7f2273fff700 1 -- 172.21.15.83:0/4040268813 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f226c00fd50 con 0x7f227c153180 2023-11-26T07:14:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.705+0000 7f2273fff700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225c06b080 0x7f225c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.705+0000 7f2273fff700 1 -- 172.21.15.83:0/4040268813 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f226c089580 con 0x7f227c153180 2023-11-26T07:14:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.705+0000 7f227b7fe700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225c06b080 0x7f225c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:46.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.705+0000 7f227b7fe700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225c06b080 0x7f225c06d540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f22640090b0 tx=0x7f2264008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:46.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.709+0000 7f2273fff700 1 -- 172.21.15.83:0/4040268813 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f226c054740 con 0x7f227c153180 2023-11-26T07:14:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:46.977+0000 7f2281f2e700 1 -- 172.21.15.83:0/4040268813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f227c0753f0 con 0x7f227c153180 2023-11-26T07:14:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.218+0000 7f2273fff700 1 -- 172.21.15.83:0/4040268813 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v182) v1 ==== 140+0+0 (secure 0 0 0) 0x7f226c057d60 con 0x7f227c153180 2023-11-26T07:14:47.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.220+0000 7f2281f2e700 1 -- 172.21.15.83:0/4040268813 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225c06b080 msgr2=0x7f225c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.220+0000 7f2281f2e700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225c06b080 0x7f225c06d540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f22640090b0 tx=0x7f2264008030 comp rx=0 tx=0).stop 2023-11-26T07:14:47.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.220+0000 7f2281f2e700 1 -- 172.21.15.83:0/4040268813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227c153180 msgr2=0x7f227c1595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.220+0000 7f2281f2e700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227c153180 0x7f227c1595f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f226c0011e0 tx=0x7f226c007620 comp rx=0 tx=0).stop 2023-11-26T07:14:47.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.220+0000 7f2281f2e700 1 -- 172.21.15.83:0/4040268813 shutdown_connections 2023-11-26T07:14:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.220+0000 7f2281f2e700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f227c153600 0x7f227c159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.220+0000 7f2281f2e700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225c06b080 0x7f225c06d540 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.220+0000 7f2281f2e700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2268003a30 0x7f227c151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.220+0000 7f2281f2e700 1 --2- 172.21.15.83:0/4040268813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f227c153180 0x7f227c1595f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.220+0000 7f2281f2e700 1 -- 172.21.15.83:0/4040268813 >> 172.21.15.83:0/4040268813 conn(0x7f227c071b90 msgr2=0x7f227c0b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.221+0000 7f2281f2e700 1 -- 172.21.15.83:0/4040268813 shutdown_connections 2023-11-26T07:14:47.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.221+0000 7f2281f2e700 1 -- 172.21.15.83:0/4040268813 wait complete. 2023-11-26T07:14:47.235 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-11-26T07:14:47.236 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2023-11-26T07:14:47.236 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:14:47.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:14:47.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.353+0000 7fd92d1fa700 1 Processor -- start 2023-11-26T07:14:47.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.353+0000 7fd92d1fa700 1 -- start start 2023-11-26T07:14:47.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.354+0000 7fd92d1fa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9281446a0 0x7fd9280b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.354+0000 7fd92d1fa700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd9280adf00 0x7fd92806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.354+0000 7fd92d1fa700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9280af950 con 0x7fd92806c540 2023-11-26T07:14:47.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.354+0000 7fd92d1fa700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd928144a80 con 0x7fd9280adf00 2023-11-26T07:14:47.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.354+0000 7fd92d1fa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92806c9b0 con 0x7fd9281446a0 2023-11-26T07:14:47.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.354+0000 7fd92659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd9280adf00 0x7fd92806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.354+0000 7fd92659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd9280adf00 0x7fd92806c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38104/0 (socket says 172.21.15.83:38104) 2023-11-26T07:14:47.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.354+0000 7fd92659c700 1 -- 172.21.15.83:0/3177067405 learned_addr learned my addr 172.21.15.83:0/3177067405 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:47.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.354+0000 7fd926d9d700 1 --2- 172.21.15.83:0/3177067405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9281446a0 0x7fd9280b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.354+0000 7fd92659c700 1 -- 172.21.15.83:0/3177067405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9281446a0 msgr2=0x7fd9280b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:47.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92659c700 1 --2- 172.21.15.83:0/3177067405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9281446a0 0x7fd9280b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92659c700 1 -- 172.21.15.83:0/3177067405 >> v1:172.21.15.203:6789/0 conn(0x7fd92806c540 legacy=0x7fd92806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:47.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92659c700 1 -- 172.21.15.83:0/3177067405 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280afc30 con 0x7fd9280adf00 2023-11-26T07:14:47.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd926d9d700 1 --2- 172.21.15.83:0/3177067405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9281446a0 0x7fd9280b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:47.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92659c700 1 --2- 172.21.15.83:0/3177067405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9280adf00 0x7fd92806c000 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd918009920 tx=0x7fd918009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2391690b30c9a5f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd925d9b700 1 -- 172.21.15.83:0/3177067405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918004230 con 0x7fd9280adf00 2023-11-26T07:14:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd925d9b700 1 -- 172.21.15.83:0/3177067405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd918004390 con 0x7fd9280adf00 2023-11-26T07:14:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd925d9b700 1 -- 172.21.15.83:0/3177067405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918009e70 con 0x7fd9280adf00 2023-11-26T07:14:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92d1fa700 1 -- 172.21.15.83:0/3177067405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9280adf00 msgr2=0x7fd92806c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92d1fa700 1 --2- 172.21.15.83:0/3177067405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9280adf00 0x7fd92806c000 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd918009920 tx=0x7fd918009c30 comp rx=0 tx=0).stop 2023-11-26T07:14:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92d1fa700 1 -- 172.21.15.83:0/3177067405 shutdown_connections 2023-11-26T07:14:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92d1fa700 1 --2- 172.21.15.83:0/3177067405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9281446a0 0x7fd9280b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92d1fa700 1 --2- 172.21.15.83:0/3177067405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9280adf00 0x7fd92806c000 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92d1fa700 1 -- 172.21.15.83:0/3177067405 >> 172.21.15.83:0/3177067405 conn(0x7fd928071b90 msgr2=0x7fd928071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92d1fa700 1 -- 172.21.15.83:0/3177067405 shutdown_connections 2023-11-26T07:14:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.355+0000 7fd92d1fa700 1 -- 172.21.15.83:0/3177067405 wait complete. 2023-11-26T07:14:47.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.357+0000 7fd92d1fa700 1 Processor -- start 2023-11-26T07:14:47.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.357+0000 7fd92d1fa700 1 -- start start 2023-11-26T07:14:47.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92d1fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92806c540 0x7fd9281503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92d1fa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9280adf00 0x7fd92813b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92d1fa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9281446a0 0x7fd9280b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92d1fa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92813dd30 con 0x7fd9281446a0 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92d1fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd928138720 con 0x7fd92806c540 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92d1fa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92813c410 con 0x7fd9280adf00 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92759e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9281446a0 0x7fd9280b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92759e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9281446a0 0x7fd9280b0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38114/0 (socket says 172.21.15.83:38114) 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92759e700 1 -- 172.21.15.83:0/1838854357 learned_addr learned my addr 172.21.15.83:0/1838854357 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd926d9d700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92806c540 0x7fd9281503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92659c700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9280adf00 0x7fd92813b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92759e700 1 -- 172.21.15.83:0/1838854357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9280adf00 msgr2=0x7fd92813b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92759e700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9280adf00 0x7fd92813b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92759e700 1 -- 172.21.15.83:0/1838854357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92806c540 msgr2=0x7fd9281503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92759e700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92806c540 0x7fd9281503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92759e700 1 -- 172.21.15.83:0/1838854357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918009550 con 0x7fd9281446a0 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd926d9d700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92806c540 0x7fd9281503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd92759e700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9281446a0 0x7fd9280b0960 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd92000cc40 tx=0x7fd92000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd90f7fe700 1 -- 172.21.15.83:0/1838854357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92000aba0 con 0x7fd9281446a0 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.358+0000 7fd90f7fe700 1 -- 172.21.15.83:0/1838854357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92000ad00 con 0x7fd9281446a0 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.359+0000 7fd92d1fa700 1 -- 172.21.15.83:0/1838854357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd928150910 con 0x7fd9281446a0 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.359+0000 7fd90f7fe700 1 -- 172.21.15.83:0/1838854357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920012660 con 0x7fd9281446a0 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.359+0000 7fd92d1fa700 1 -- 172.21.15.83:0/1838854357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92813be50 con 0x7fd9281446a0 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.359+0000 7fd92d1fa700 1 -- 172.21.15.83:0/1838854357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9280753f0 con 0x7fd9281446a0 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.360+0000 7fd90f7fe700 1 -- 172.21.15.83:0/1838854357 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fd9200127c0 con 0x7fd9281446a0 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.360+0000 7fd90f7fe700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd91406b080 0x7fd91406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.360+0000 7fd90f7fe700 1 -- 172.21.15.83:0/1838854357 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd920089e00 con 0x7fd9281446a0 2023-11-26T07:14:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.360+0000 7fd926d9d700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd91406b080 0x7fd91406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.361+0000 7fd926d9d700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd91406b080 0x7fd91406d540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd91c008910 tx=0x7fd91c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.365+0000 7fd90f7fe700 1 -- 172.21.15.83:0/1838854357 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd920054ea0 con 0x7fd9281446a0 2023-11-26T07:14:47.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.558+0000 7fd92d1fa700 1 -- 172.21.15.83:0/1838854357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9280d0630 con 0x7fd9281446a0 2023-11-26T07:14:47.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.559+0000 7fd90f7fe700 1 -- 172.21.15.83:0/1838854357 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+12974 (secure 0 0 0) 0x7fd920016030 con 0x7fd9281446a0 2023-11-26T07:14:47.562 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:47.562 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":50,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:14:46.212837+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/2643742427":"2023-11-27T07:14:43.204141+0000","172.21.15.83:6834/2643742427":"2023-11-27T07:14:43.204141+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:14:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 -- 172.21.15.83:0/1838854357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd91406b080 msgr2=0x7fd91406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd91406b080 0x7fd91406d540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd91c008910 tx=0x7fd91c008040 comp rx=0 tx=0).stop 2023-11-26T07:14:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 -- 172.21.15.83:0/1838854357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9281446a0 msgr2=0x7fd9280b0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9281446a0 0x7fd9280b0960 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd92000cc40 tx=0x7fd92000a060 comp rx=0 tx=0).stop 2023-11-26T07:14:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 -- 172.21.15.83:0/1838854357 shutdown_connections 2023-11-26T07:14:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9281446a0 0x7fd9280b0960 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd92806c540 0x7fd9281503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd91406b080 0x7fd91406d540 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 --2- 172.21.15.83:0/1838854357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9280adf00 0x7fd92813b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 -- 172.21.15.83:0/1838854357 >> 172.21.15.83:0/1838854357 conn(0x7fd928071b90 msgr2=0x7fd92806b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 -- 172.21.15.83:0/1838854357 shutdown_connections 2023-11-26T07:14:47.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.561+0000 7fd92d1fa700 1 -- 172.21.15.83:0/1838854357 wait complete. 2023-11-26T07:14:47.576 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:14:47.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:14:47.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.697+0000 7fc28e068700 1 Processor -- start 2023-11-26T07:14:47.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.697+0000 7fc28e068700 1 -- start start 2023-11-26T07:14:47.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc28e068700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc2880b4310 0x7fc28806bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc28e068700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28806ba50 con 0x7fc2880ce430 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc28e068700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28806b120 con 0x7fc28815e080 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc28e068700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2880d1200 con 0x7fc2880b4310 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc287fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc2880ce430 0x7fc2880cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58162/0 (socket says 172.21.15.83:58162) 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc287fff700 1 -- 172.21.15.83:0/3987420907 learned_addr learned my addr 172.21.15.83:0/3987420907 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696736646 0 0) 0x7fc28806ba50 con 0x7fc2880ce430 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2740035b0 con 0x7fc2880ce430 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc286ffd700 1 --2- 172.21.15.83:0/3987420907 >> v2:172.21.15.203:3300/0 conn(0x7fc2880b4310 0x7fc28806bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4293810669 0 0) 0x7fc2740035b0 con 0x7fc2880ce430 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 >> v2:172.21.15.203:3300/0 conn(0x7fc2880b4310 msgr2=0x7fc28806bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc2867fc700 1 --2- 172.21.15.83:0/3987420907 >> v2:172.21.15.203:3300/0 conn(0x7fc2880b4310 0x7fc28806bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 >> v1:172.21.15.203:6789/0 conn(0x7fc28815e080 legacy=0x7fc2880d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28806b400 con 0x7fc2880ce430 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fc2800019f0 con 0x7fc2880ce430 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 >> v1:172.21.15.83:6789/0 conn(0x7fc2880ce430 legacy=0x7fc2880cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.698+0000 7fc2867fc700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc274003a50 0x7fc274003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc2867fc700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc274004850 0x7fc274004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc2867fc700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc274005260 0x7fc27400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880d1200 con 0x7fc274004850 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27400c1b0 con 0x7fc274003a50 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27400c370 con 0x7fc274005260 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc286ffd700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc274004850 0x7fc274004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc2877fe700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc274003a50 0x7fc274003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc287fff700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc274005260 0x7fc27400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc286ffd700 1 -- 172.21.15.83:0/3987420907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc274005260 msgr2=0x7fc27400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:47.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc286ffd700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc274005260 0x7fc27400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc286ffd700 1 -- 172.21.15.83:0/3987420907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc274003a50 msgr2=0x7fc274003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc286ffd700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc274003a50 0x7fc274003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc287fff700 1 -- 172.21.15.83:0/3987420907 reap_dead start 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc286ffd700 1 -- 172.21.15.83:0/3987420907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2740035b0 con 0x7fc274004850 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc2877fe700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc274003a50 0x7fc274003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc286ffd700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc274004850 0x7fc274004cd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc2880b4750 tx=0x7fc274005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49f90291ce81170 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27800e6c0 con 0x7fc274004850 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27800e820 con 0x7fc274004850 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.699+0000 7fc2867fc700 1 -- 172.21.15.83:0/3987420907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27800ebd0 con 0x7fc274004850 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.700+0000 7fc28e068700 1 -- 172.21.15.83:0/3987420907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc274004850 msgr2=0x7fc274004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.700+0000 7fc28e068700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc274004850 0x7fc274004cd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc2880b4750 tx=0x7fc274005880 comp rx=0 tx=0).stop 2023-11-26T07:14:47.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.700+0000 7fc28e068700 1 -- 172.21.15.83:0/3987420907 shutdown_connections 2023-11-26T07:14:47.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.700+0000 7fc28e068700 1 --2- 172.21.15.83:0/3987420907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc274004850 0x7fc274004cd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.700+0000 7fc28e068700 1 -- 172.21.15.83:0/3987420907 >> 172.21.15.83:0/3987420907 conn(0x7fc2880b3780 msgr2=0x7fc2880b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:47.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.700+0000 7fc28e068700 1 -- 172.21.15.83:0/3987420907 shutdown_connections 2023-11-26T07:14:47.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.700+0000 7fc28e068700 1 -- 172.21.15.83:0/3987420907 wait complete. 2023-11-26T07:14:47.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.701+0000 7fc28e068700 1 Processor -- start 2023-11-26T07:14:47.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.701+0000 7fc28e068700 1 -- start start 2023-11-26T07:14:47.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc28e068700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc274004850 0x7fc28815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc28e068700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2880b7330 0x7fc2880b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc28e068700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc28814ceb0 0x7fc2880afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc28e068700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880b77b0 con 0x7fc28814ceb0 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc28e068700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880c8f80 con 0x7fc2880b7330 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc28e068700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880b6290 con 0x7fc274004850 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc287fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc28814ceb0 0x7fc2880afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc287fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc28814ceb0 0x7fc2880afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38134/0 (socket says 172.21.15.83:38134) 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc287fff700 1 -- 172.21.15.83:0/1920130975 learned_addr learned my addr 172.21.15.83:0/1920130975 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc286ffd700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2880b7330 0x7fc2880b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc2877fe700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc274004850 0x7fc28815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc287fff700 1 -- 172.21.15.83:0/1920130975 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc274004850 msgr2=0x7fc28815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc287fff700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc274004850 0x7fc28815a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc287fff700 1 -- 172.21.15.83:0/1920130975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2880b7330 msgr2=0x7fc2880b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc287fff700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2880b7330 0x7fc2880b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc287fff700 1 -- 172.21.15.83:0/1920130975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880b6570 con 0x7fc28814ceb0 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc287fff700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc28814ceb0 0x7fc2880afab0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc28806eba0 tx=0x7fc280007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.702+0000 7fc27ffff700 1 -- 172.21.15.83:0/1920130975 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28000ebd0 con 0x7fc28814ceb0 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.703+0000 7fc27ffff700 1 -- 172.21.15.83:0/1920130975 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28000ed50 con 0x7fc28814ceb0 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.703+0000 7fc28e068700 1 -- 172.21.15.83:0/1920130975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2880b6850 con 0x7fc28814ceb0 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.703+0000 7fc27ffff700 1 -- 172.21.15.83:0/1920130975 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280013660 con 0x7fc28814ceb0 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.703+0000 7fc28e068700 1 -- 172.21.15.83:0/1920130975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2880b6e10 con 0x7fc28814ceb0 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.703+0000 7fc28e068700 1 -- 172.21.15.83:0/1920130975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2880b6fa0 con 0x7fc28814ceb0 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.704+0000 7fc27ffff700 1 -- 172.21.15.83:0/1920130975 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc2800137c0 con 0x7fc28814ceb0 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.704+0000 7fc27ffff700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26806b080 0x7fc26806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.704+0000 7fc27ffff700 1 -- 172.21.15.83:0/1920130975 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc280089b20 con 0x7fc28814ceb0 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.704+0000 7fc2877fe700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26806b080 0x7fc26806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:47.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.705+0000 7fc2877fe700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26806b080 0x7fc26806d540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc270007fd0 tx=0x7fc270009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:47.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.709+0000 7fc27ffff700 1 -- 172.21.15.83:0/1920130975 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc280054c50 con 0x7fc28814ceb0 2023-11-26T07:14:47.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.909+0000 7fc28e068700 1 -- 172.21.15.83:0/1920130975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc288068fb0 con 0x7fc28814ceb0 2023-11-26T07:14:47.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.909+0000 7fc27ffff700 1 -- 172.21.15.83:0/1920130975 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc280058270 con 0x7fc28814ceb0 2023-11-26T07:14:47.912 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:14:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.911+0000 7fc28e068700 1 -- 172.21.15.83:0/1920130975 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26806b080 msgr2=0x7fc26806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.911+0000 7fc28e068700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26806b080 0x7fc26806d540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc270007fd0 tx=0x7fc270009040 comp rx=0 tx=0).stop 2023-11-26T07:14:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.911+0000 7fc28e068700 1 -- 172.21.15.83:0/1920130975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc28814ceb0 msgr2=0x7fc2880afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.911+0000 7fc28e068700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc28814ceb0 0x7fc2880afab0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc28806eba0 tx=0x7fc280007c90 comp rx=0 tx=0).stop 2023-11-26T07:14:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.911+0000 7fc28e068700 1 -- 172.21.15.83:0/1920130975 shutdown_connections 2023-11-26T07:14:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.911+0000 7fc28e068700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc28814ceb0 0x7fc2880afab0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.911+0000 7fc28e068700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc26806b080 0x7fc26806d540 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.912+0000 7fc28e068700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc274004850 0x7fc28815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.912+0000 7fc28e068700 1 --2- 172.21.15.83:0/1920130975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2880b7330 0x7fc2880b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.912+0000 7fc28e068700 1 -- 172.21.15.83:0/1920130975 >> 172.21.15.83:0/1920130975 conn(0x7fc2880b3780 msgr2=0x7fc2880b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.912+0000 7fc28e068700 1 -- 172.21.15.83:0/1920130975 shutdown_connections 2023-11-26T07:14:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:47.912+0000 7fc28e068700 1 -- 172.21.15.83:0/1920130975 wait complete. 2023-11-26T07:14:47.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:14:48.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.043+0000 7fa3cd205700 1 Processor -- start 2023-11-26T07:14:48.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.043+0000 7fa3cd205700 1 -- start start 2023-11-26T07:14:48.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.044+0000 7fa3cd205700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa3c80b0f80 0x7fa3c815a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:48.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.044+0000 7fa3cd205700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80cd770 0x7fa3c815a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:48.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.044+0000 7fa3cd205700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c80b2bd0 con 0x7fa3c80b0ba0 2023-11-26T07:14:48.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.044+0000 7fa3cd205700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c80b1fc0 con 0x7fa3c80b0f80 2023-11-26T07:14:48.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.044+0000 7fa3cd205700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80b22a0 con 0x7fa3c80cd770 2023-11-26T07:14:48.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c659c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa3c80b0f80 0x7fa3c815a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c759e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80cd770 0x7fa3c815a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c6d9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa3c80b0ba0 0x7fa3c80af5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:42796/0 (socket says 172.21.15.83:42796) 2023-11-26T07:14:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c6d9d700 1 -- 172.21.15.83:0/3676896186 learned_addr learned my addr 172.21.15.83:0/3676896186 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c759e700 1 -- 172.21.15.83:0/3676896186 >> v2:172.21.15.203:3300/0 conn(0x7fa3c80b0f80 msgr2=0x7fa3c815a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c759e700 1 --2- 172.21.15.83:0/3676896186 >> v2:172.21.15.203:3300/0 conn(0x7fa3c80b0f80 0x7fa3c815a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c759e700 1 -- 172.21.15.83:0/3676896186 >> v1:172.21.15.203:6789/0 conn(0x7fa3c80b0ba0 legacy=0x7fa3c80af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c759e700 1 -- 172.21.15.83:0/3676896186 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80b2580 con 0x7fa3c80cd770 2023-11-26T07:14:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c659c700 1 --2- 172.21.15.83:0/3676896186 >> v2:172.21.15.203:3300/0 conn(0x7fa3c80b0f80 0x7fa3c815a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:14:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c759e700 1 --2- 172.21.15.83:0/3676896186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80cd770 0x7fa3c815a980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009d90 tx=0x7fa3c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cfb6d22898692da server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c5d9b700 1 -- 172.21.15.83:0/3676896186 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000e070 con 0x7fa3c80cd770 2023-11-26T07:14:48.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c5d9b700 1 -- 172.21.15.83:0/3676896186 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c000bb20 con 0x7fa3c80cd770 2023-11-26T07:14:48.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.045+0000 7fa3c5d9b700 1 -- 172.21.15.83:0/3676896186 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0012430 con 0x7fa3c80cd770 2023-11-26T07:14:48.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.046+0000 7fa3cd205700 1 -- 172.21.15.83:0/3676896186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80cd770 msgr2=0x7fa3c815a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:48.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.046+0000 7fa3cd205700 1 --2- 172.21.15.83:0/3676896186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80cd770 0x7fa3c815a980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009d90 tx=0x7fa3c000b330 comp rx=0 tx=0).stop 2023-11-26T07:14:48.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.046+0000 7fa3cd205700 1 -- 172.21.15.83:0/3676896186 shutdown_connections 2023-11-26T07:14:48.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.046+0000 7fa3cd205700 1 --2- 172.21.15.83:0/3676896186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80cd770 0x7fa3c815a980 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:48.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.046+0000 7fa3cd205700 1 --2- 172.21.15.83:0/3676896186 >> v2:172.21.15.203:3300/0 conn(0x7fa3c80b0f80 0x7fa3c815a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:48.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.046+0000 7fa3cd205700 1 -- 172.21.15.83:0/3676896186 >> 172.21.15.83:0/3676896186 conn(0x7fa3c80b57b0 msgr2=0x7fa3c80b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:48.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.046+0000 7fa3cd205700 1 -- 172.21.15.83:0/3676896186 shutdown_connections 2023-11-26T07:14:48.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.046+0000 7fa3cd205700 1 -- 172.21.15.83:0/3676896186 wait complete. 2023-11-26T07:14:48.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.047+0000 7fa3cd205700 1 Processor -- start 2023-11-26T07:14:48.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.047+0000 7fa3cd205700 1 -- start start 2023-11-26T07:14:48.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.048+0000 7fa3cd205700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80b0ba0 0x7fa3c8155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:48.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.048+0000 7fa3cd205700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3c80b0f80 0x7fa3c80adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:48.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.048+0000 7fa3cd205700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c80cd770 0x7fa3c80ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:48.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.048+0000 7fa3cd205700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80af910 con 0x7fa3c80cd770 2023-11-26T07:14:48.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.048+0000 7fa3cd205700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c80ada60 con 0x7fa3c80b0f80 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.048+0000 7fa3cd205700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c815d500 con 0x7fa3c80b0ba0 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.048+0000 7fa3c759e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c80cd770 0x7fa3c80ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.048+0000 7fa3c759e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c80cd770 0x7fa3c80ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38148/0 (socket says 172.21.15.83:38148) 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.048+0000 7fa3c759e700 1 -- 172.21.15.83:0/2647750605 learned_addr learned my addr 172.21.15.83:0/2647750605 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.048+0000 7fa3c6d9d700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80b0ba0 0x7fa3c8155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3c659c700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3c80b0f80 0x7fa3c80adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3c759e700 1 -- 172.21.15.83:0/2647750605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80b0ba0 msgr2=0x7fa3c8155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3c759e700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80b0ba0 0x7fa3c8155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3c759e700 1 -- 172.21.15.83:0/2647750605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3c80b0f80 msgr2=0x7fa3c80adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3c759e700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3c80b0f80 0x7fa3c80adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3c759e700 1 -- 172.21.15.83:0/2647750605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b80094a0 con 0x7fa3c80cd770 2023-11-26T07:14:48.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3c6d9d700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80b0ba0 0x7fa3c8155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3c659c700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3c80b0f80 0x7fa3c80adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3c759e700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c80cd770 0x7fa3c80ae430 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0014040 tx=0x7fa3c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3af7fe700 1 -- 172.21.15.83:0/2647750605 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0012430 con 0x7fa3c80cd770 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3af7fe700 1 -- 172.21.15.83:0/2647750605 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c0020070 con 0x7fa3c80cd770 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3cd205700 1 -- 172.21.15.83:0/2647750605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c0009a70 con 0x7fa3c80cd770 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3af7fe700 1 -- 172.21.15.83:0/2647750605 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0004280 con 0x7fa3c80cd770 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.049+0000 7fa3cd205700 1 -- 172.21.15.83:0/2647750605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c80af630 con 0x7fa3c80cd770 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.050+0000 7fa3cd205700 1 -- 172.21.15.83:0/2647750605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c80adbd0 con 0x7fa3c80cd770 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.050+0000 7fa3af7fe700 1 -- 172.21.15.83:0/2647750605 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fa3c00043e0 con 0x7fa3c80cd770 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.051+0000 7fa3af7fe700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3b406b080 0x7fa3b406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.051+0000 7fa3af7fe700 1 -- 172.21.15.83:0/2647750605 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa3c0022070 con 0x7fa3c80cd770 2023-11-26T07:14:48.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.051+0000 7fa3c6d9d700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3b406b080 0x7fa3b406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:48.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.051+0000 7fa3c6d9d700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3b406b080 0x7fa3b406d540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa3c815b290 tx=0x7fa3bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:48.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.055+0000 7fa3af7fe700 1 -- 172.21.15.83:0/2647750605 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3c0056910 con 0x7fa3c80cd770 2023-11-26T07:14:48.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:48.327+0000 7fa3cd205700 1 -- 172.21.15.83:0/2647750605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7fa3c8067f60 con 0x7fa3c80cd770 2023-11-26T07:14:49.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.223+0000 7fa3af7fe700 1 -- 172.21.15.83:0/2647750605 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v184) v1 ==== 164+0+0 (secure 0 0 0) 0x7fa3c0059f30 con 0x7fa3c80cd770 2023-11-26T07:14:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 -- 172.21.15.83:0/2647750605 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3b406b080 msgr2=0x7fa3b406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3b406b080 0x7fa3b406d540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa3c815b290 tx=0x7fa3bc009040 comp rx=0 tx=0).stop 2023-11-26T07:14:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 -- 172.21.15.83:0/2647750605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c80cd770 msgr2=0x7fa3c80ae430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c80cd770 0x7fa3c80ae430 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0014040 tx=0x7fa3c0004ad0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 -- 172.21.15.83:0/2647750605 shutdown_connections 2023-11-26T07:14:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3c80cd770 0x7fa3c80ae430 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa3b406b080 0x7fa3b406d540 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3c80b0ba0 0x7fa3c8155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 --2- 172.21.15.83:0/2647750605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3c80b0f80 0x7fa3c80adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 -- 172.21.15.83:0/2647750605 >> 172.21.15.83:0/2647750605 conn(0x7fa3c80b57b0 msgr2=0x7fa3c8159870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 -- 172.21.15.83:0/2647750605 shutdown_connections 2023-11-26T07:14:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.227+0000 7fa3cd205700 1 -- 172.21.15.83:0/2647750605 wait complete. 2023-11-26T07:14:49.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:14:49.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.370+0000 7f0f4067f700 1 Processor -- start 2023-11-26T07:14:49.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.370+0000 7f0f4067f700 1 -- start start 2023-11-26T07:14:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.371+0000 7f0f4067f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f380d21d0 0x7f0f38166ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.371+0000 7f0f4067f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f380cc4a0 con 0x7f0f3815bec0 2023-11-26T07:14:49.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.371+0000 7f0f4067f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f380d03e0 con 0x7f0f380704b0 2023-11-26T07:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.371+0000 7f0f4067f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3815c2a0 con 0x7f0f380d21d0 2023-11-26T07:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3ec1c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f380d21d0 0x7f0f38166ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3ec1c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f380d21d0 0x7f0f38166ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38152/0 (socket says 172.21.15.83:38152) 2023-11-26T07:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3ec1c700 1 -- 172.21.15.83:0/2230052186 learned_addr learned my addr 172.21.15.83:0/2230052186 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3e41b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0f3815bec0 0x7f0f3806fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58192/0 (socket says 172.21.15.83:58192) 2023-11-26T07:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3ec1c700 1 -- 172.21.15.83:0/2230052186 >> v1:172.21.15.203:6789/0 conn(0x7f0f380704b0 legacy=0x7f0f380d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3ec1c700 1 -- 172.21.15.83:0/2230052186 >> v1:172.21.15.83:6789/0 conn(0x7f0f3815bec0 legacy=0x7f0f3806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3ec1c700 1 -- 172.21.15.83:0/2230052186 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f38070c60 con 0x7f0f380d21d0 2023-11-26T07:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3d419700 1 -- 172.21.15.83:0/2230052186 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113492811 0 0) 0x7f0f380cc4a0 con 0x7f0f3815bec0 2023-11-26T07:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3ec1c700 1 --2- 172.21.15.83:0/2230052186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f380d21d0 0x7f0f38166ce0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0f38070e50 tx=0x7f0f3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebc4c0c1d5b8404 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3d419700 1 -- 172.21.15.83:0/2230052186 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000e070 con 0x7f0f380d21d0 2023-11-26T07:14:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3d419700 1 -- 172.21.15.83:0/2230052186 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3000b980 con 0x7f0f380d21d0 2023-11-26T07:14:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.372+0000 7f0f3d419700 1 -- 172.21.15.83:0/2230052186 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000bd30 con 0x7f0f380d21d0 2023-11-26T07:14:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.373+0000 7f0f4067f700 1 -- 172.21.15.83:0/2230052186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f380d21d0 msgr2=0x7f0f38166ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.373+0000 7f0f4067f700 1 --2- 172.21.15.83:0/2230052186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f380d21d0 0x7f0f38166ce0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0f38070e50 tx=0x7f0f3000b330 comp rx=0 tx=0).stop 2023-11-26T07:14:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.373+0000 7f0f4067f700 1 -- 172.21.15.83:0/2230052186 shutdown_connections 2023-11-26T07:14:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.373+0000 7f0f4067f700 1 --2- 172.21.15.83:0/2230052186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f380d21d0 0x7f0f38166ce0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.373+0000 7f0f4067f700 1 -- 172.21.15.83:0/2230052186 >> 172.21.15.83:0/2230052186 conn(0x7f0f380730a0 msgr2=0x7f0f3809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.373+0000 7f0f4067f700 1 -- 172.21.15.83:0/2230052186 shutdown_connections 2023-11-26T07:14:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.373+0000 7f0f4067f700 1 -- 172.21.15.83:0/2230052186 wait complete. 2023-11-26T07:14:49.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.374+0000 7f0f4067f700 1 Processor -- start 2023-11-26T07:14:49.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.374+0000 7f0f4067f700 1 -- start start 2023-11-26T07:14:49.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.375+0000 7f0f4067f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f380704b0 0x7f0f381555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:49.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.375+0000 7f0f4067f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f380d21d0 0x7f0f380b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.375+0000 7f0f4067f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f3815bec0 0x7f0f380b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.375+0000 7f0f4067f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f380c91b0 con 0x7f0f380d21d0 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.375+0000 7f0f4067f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f380cea70 con 0x7f0f380704b0 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.375+0000 7f0f4067f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f380ce4b0 con 0x7f0f3815bec0 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.375+0000 7f0f3dc1a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f380d21d0 0x7f0f380b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.375+0000 7f0f3dc1a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f380d21d0 0x7f0f380b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38158/0 (socket says 172.21.15.83:38158) 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.375+0000 7f0f3dc1a700 1 -- 172.21.15.83:0/4253383925 learned_addr learned my addr 172.21.15.83:0/4253383925 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f3e41b700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f380704b0 0x7f0f381555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f3ec1c700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f3815bec0 0x7f0f380b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f3dc1a700 1 -- 172.21.15.83:0/4253383925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f3815bec0 msgr2=0x7f0f380b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f3dc1a700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f3815bec0 0x7f0f380b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f3dc1a700 1 -- 172.21.15.83:0/4253383925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f380704b0 msgr2=0x7f0f381555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f3dc1a700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f380704b0 0x7f0f381555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f3dc1a700 1 -- 172.21.15.83:0/4253383925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f30009b30 con 0x7f0f380d21d0 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f3e41b700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f380704b0 0x7f0f381555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f3ec1c700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f3815bec0 0x7f0f380b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f3dc1a700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f380d21d0 0x7f0f380b68b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400c4d0 tx=0x7f0f3400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f2effd700 1 -- 172.21.15.83:0/4253383925 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34011790 con 0x7f0f380d21d0 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f2effd700 1 -- 172.21.15.83:0/4253383925 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f340118f0 con 0x7f0f380d21d0 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f4067f700 1 -- 172.21.15.83:0/4253383925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f380ce790 con 0x7f0f380d21d0 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f2effd700 1 -- 172.21.15.83:0/4253383925 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f34011ca0 con 0x7f0f380d21d0 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.376+0000 7f0f4067f700 1 -- 172.21.15.83:0/4253383925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f38156230 con 0x7f0f380d21d0 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.377+0000 7f0f4067f700 1 -- 172.21.15.83:0/4253383925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f381563c0 con 0x7f0f380d21d0 2023-11-26T07:14:49.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.378+0000 7f0f2effd700 1 -- 172.21.15.83:0/4253383925 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0f3401c460 con 0x7f0f380d21d0 2023-11-26T07:14:49.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.378+0000 7f0f2effd700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0f1c06b080 0x7f0f1c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:49.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.378+0000 7f0f2effd700 1 -- 172.21.15.83:0/4253383925 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0f340894d0 con 0x7f0f380d21d0 2023-11-26T07:14:49.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.378+0000 7f0f3e41b700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0f1c06b080 0x7f0f1c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:49.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.378+0000 7f0f3e41b700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0f1c06b080 0x7f0f1c06d540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0f28007fd0 tx=0x7f0f28009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:49.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.382+0000 7f0f2effd700 1 -- 172.21.15.83:0/4253383925 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f34054600 con 0x7f0f380d21d0 2023-11-26T07:14:49.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.574+0000 7f0f4067f700 1 -- 172.21.15.83:0/4253383925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0f38068fb0 con 0x7f0f380d21d0 2023-11-26T07:14:49.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.579+0000 7f0f2effd700 1 -- 172.21.15.83:0/4253383925 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0f34057c20 con 0x7f0f380d21d0 2023-11-26T07:14:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 -- 172.21.15.83:0/4253383925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0f1c06b080 msgr2=0x7f0f1c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0f1c06b080 0x7f0f1c06d540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0f28007fd0 tx=0x7f0f28009040 comp rx=0 tx=0).stop 2023-11-26T07:14:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 -- 172.21.15.83:0/4253383925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f380d21d0 msgr2=0x7f0f380b68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f380d21d0 0x7f0f380b68b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400c4d0 tx=0x7f0f3400c890 comp rx=0 tx=0).stop 2023-11-26T07:14:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 -- 172.21.15.83:0/4253383925 shutdown_connections 2023-11-26T07:14:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0f1c06b080 0x7f0f1c06d540 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f3815bec0 0x7f0f380b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f380704b0 0x7f0f381555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 --2- 172.21.15.83:0/4253383925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f380d21d0 0x7f0f380b68b0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 -- 172.21.15.83:0/4253383925 >> 172.21.15.83:0/4253383925 conn(0x7f0f380730a0 msgr2=0x7f0f380b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 -- 172.21.15.83:0/4253383925 shutdown_connections 2023-11-26T07:14:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.581+0000 7f0f4067f700 1 -- 172.21.15.83:0/4253383925 wait complete. 2023-11-26T07:14:49.596 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:14:49.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:14:49.641 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:14:49.687 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:14:49.687 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:14:49.688 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:14:49.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:14:49.705 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:14:49.705 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:14:49.724 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:14:49.724 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:14:49.724 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:14:49.762 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:14:49.762 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:14:49.763 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:14:49.782 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:14:49.783 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:14:49.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.920+0000 7f05f8b4a700 1 Processor -- start 2023-11-26T07:14:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.920+0000 7f05f8b4a700 1 -- start start 2023-11-26T07:14:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.921+0000 7f05f8b4a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 0x7f05f41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.921+0000 7f05f8b4a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05f40d1c70 0x7f05f40d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.921+0000 7f05f8b4a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f40b22d0 con 0x7f05f415bdd0 2023-11-26T07:14:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.921+0000 7f05f8b4a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f40b2f50 con 0x7f05f40d1c70 2023-11-26T07:14:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.921+0000 7f05f8b4a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40d2640 con 0x7f05f40b3370 2023-11-26T07:14:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f1d9b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 0x7f05f41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f2d9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05f40d1c70 0x7f05f40d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f259c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f05f415bdd0 0x7f05f40b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:42822/0 (socket says 172.21.15.83:42822) 2023-11-26T07:14:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f1d9b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 0x7f05f41604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:48624/0 (socket says 172.21.15.83:48624) 2023-11-26T07:14:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f2d9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05f40d1c70 0x7f05f40d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:37620/0 (socket says 172.21.15.83:37620) 2023-11-26T07:14:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f259c700 1 -- 172.21.15.83:0/3550287323 learned_addr learned my addr 172.21.15.83:0/3550287323 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f1d9b700 1 -- 172.21.15.83:0/3550287323 >> v2:172.21.15.203:3300/0 conn(0x7f05f40d1c70 msgr2=0x7f05f40d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f1d9b700 1 --2- 172.21.15.83:0/3550287323 >> v2:172.21.15.203:3300/0 conn(0x7f05f40d1c70 0x7f05f40d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f1d9b700 1 -- 172.21.15.83:0/3550287323 >> v1:172.21.15.203:6789/0 conn(0x7f05f415bdd0 legacy=0x7f05f40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f1d9b700 1 -- 172.21.15.83:0/3550287323 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f40b25b0 con 0x7f05f40b3370 2023-11-26T07:14:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f2d9d700 1 --2- 172.21.15.83:0/3550287323 >> v2:172.21.15.203:3300/0 conn(0x7f05f40d1c70 0x7f05f40d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:14:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f1d9b700 1 --2- 172.21.15.83:0/3550287323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 0x7f05f41604e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009840 tx=0x7f05e4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a27ff58073c411b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:49.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.922+0000 7f05f159a700 1 -- 172.21.15.83:0/3550287323 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4011830 con 0x7f05f40b3370 2023-11-26T07:14:49.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.923+0000 7f05f159a700 1 -- 172.21.15.83:0/3550287323 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e4011990 con 0x7f05f40b3370 2023-11-26T07:14:49.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.923+0000 7f05f159a700 1 -- 172.21.15.83:0/3550287323 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4011d20 con 0x7f05f40b3370 2023-11-26T07:14:49.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.923+0000 7f05f8b4a700 1 -- 172.21.15.83:0/3550287323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 msgr2=0x7f05f41604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.923+0000 7f05f8b4a700 1 --2- 172.21.15.83:0/3550287323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 0x7f05f41604e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009840 tx=0x7f05e4009b50 comp rx=0 tx=0).stop 2023-11-26T07:14:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.923+0000 7f05f8b4a700 1 -- 172.21.15.83:0/3550287323 shutdown_connections 2023-11-26T07:14:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.923+0000 7f05f8b4a700 1 --2- 172.21.15.83:0/3550287323 >> v2:172.21.15.203:3300/0 conn(0x7f05f40d1c70 0x7f05f40d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.923+0000 7f05f8b4a700 1 --2- 172.21.15.83:0/3550287323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 0x7f05f41604e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.923+0000 7f05f8b4a700 1 -- 172.21.15.83:0/3550287323 >> 172.21.15.83:0/3550287323 conn(0x7f05f40731a0 msgr2=0x7f05f409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.923+0000 7f05f8b4a700 1 -- 172.21.15.83:0/3550287323 shutdown_connections 2023-11-26T07:14:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.923+0000 7f05f8b4a700 1 -- 172.21.15.83:0/3550287323 wait complete. 2023-11-26T07:14:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.925+0000 7f05f8b4a700 1 Processor -- start 2023-11-26T07:14:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.925+0000 7f05f8b4a700 1 -- start start 2023-11-26T07:14:49.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.925+0000 7f05f8b4a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 0x7f05f406c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.925+0000 7f05f8b4a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f40d1c70 0x7f05f415b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.925+0000 7f05f8b4a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f415bdd0 0x7f05f40cee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.925+0000 7f05f8b4a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f406cf60 con 0x7f05f40d1c70 2023-11-26T07:14:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.925+0000 7f05f8b4a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f40cf370 con 0x7f05f415bdd0 2023-11-26T07:14:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.925+0000 7f05f8b4a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f41f83f0 con 0x7f05f40b3370 2023-11-26T07:14:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.925+0000 7f05f1d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f40d1c70 0x7f05f415b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.925+0000 7f05f1d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f40d1c70 0x7f05f415b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38170/0 (socket says 172.21.15.83:38170) 2023-11-26T07:14:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05f1d9b700 1 -- 172.21.15.83:0/2199675620 learned_addr learned my addr 172.21.15.83:0/2199675620 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05f2d9d700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f415bdd0 0x7f05f40cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05f259c700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 0x7f05f406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:49.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05f1d9b700 1 -- 172.21.15.83:0/2199675620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 msgr2=0x7f05f406c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05f1d9b700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 0x7f05f406c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05f1d9b700 1 -- 172.21.15.83:0/2199675620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f415bdd0 msgr2=0x7f05f40cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05f1d9b700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f415bdd0 0x7f05f40cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05f1d9b700 1 -- 172.21.15.83:0/2199675620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec009a70 con 0x7f05f40d1c70 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05f1d9b700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f40d1c70 0x7f05f415b180 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009fd0 tx=0x7f05e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05e2ffd700 1 -- 172.21.15.83:0/2199675620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40036a0 con 0x7f05f40d1c70 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05e2ffd700 1 -- 172.21.15.83:0/2199675620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e401f070 con 0x7f05f40d1c70 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05f8b4a700 1 -- 172.21.15.83:0/2199675620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e40094a0 con 0x7f05f40d1c70 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05e2ffd700 1 -- 172.21.15.83:0/2199675620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4003a60 con 0x7f05f40d1c70 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.926+0000 7f05f8b4a700 1 -- 172.21.15.83:0/2199675620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f41f7910 con 0x7f05f40d1c70 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.927+0000 7f05f8b4a700 1 -- 172.21.15.83:0/2199675620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f40cd800 con 0x7f05f40d1c70 2023-11-26T07:14:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.928+0000 7f05e2ffd700 1 -- 172.21.15.83:0/2199675620 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f05e4003bc0 con 0x7f05f40d1c70 2023-11-26T07:14:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.928+0000 7f05e2ffd700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05d406b080 0x7f05d406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.928+0000 7f05e2ffd700 1 -- 172.21.15.83:0/2199675620 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f05e4021070 con 0x7f05f40d1c70 2023-11-26T07:14:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.928+0000 7f05f259c700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05d406b080 0x7f05d406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.928+0000 7f05f259c700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05d406b080 0x7f05d406d540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005910 tx=0x7f05e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:49.932+0000 7f05e2ffd700 1 -- 172.21.15.83:0/2199675620 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05e4056120 con 0x7f05f40d1c70 2023-11-26T07:14:50.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.166+0000 7f05f8b4a700 1 -- 172.21.15.83:0/2199675620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05f4067f60 con 0x7f05f40d1c70 2023-11-26T07:14:50.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.166+0000 7f05e2ffd700 1 -- 172.21.15.83:0/2199675620 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f05e4015090 con 0x7f05f40d1c70 2023-11-26T07:14:50.169 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:50.169 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1603782122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1603782122},{"type":"v1","addr":"172.21.15.203:6834","nonce":1603782122}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/822351975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":822351975},{"type":"v1","addr":"172.21.15.203:6835","nonce":822351975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/110903762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":110903762},{"type":"v1","addr":"172.21.15.83:6836","nonce":110903762}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3702462303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3702462303},{"type":"v1","addr":"172.21.15.83:6837","nonce":3702462303}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[]} 2023-11-26T07:14:50.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 -- 172.21.15.83:0/2199675620 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05d406b080 msgr2=0x7f05d406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:50.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05d406b080 0x7f05d406d540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005910 tx=0x7f05e8009040 comp rx=0 tx=0).stop 2023-11-26T07:14:50.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 -- 172.21.15.83:0/2199675620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f40d1c70 msgr2=0x7f05f415b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:50.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f40d1c70 0x7f05f415b180 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009fd0 tx=0x7f05e4004750 comp rx=0 tx=0).stop 2023-11-26T07:14:50.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 -- 172.21.15.83:0/2199675620 shutdown_connections 2023-11-26T07:14:50.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05f415bdd0 0x7f05f40cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:50.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f05d406b080 0x7f05d406d540 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:50.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f05f40b3370 0x7f05f406c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:50.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 --2- 172.21.15.83:0/2199675620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f40d1c70 0x7f05f415b180 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:50.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 -- 172.21.15.83:0/2199675620 >> 172.21.15.83:0/2199675620 conn(0x7f05f40731a0 msgr2=0x7f05f40aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:50.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 -- 172.21.15.83:0/2199675620 shutdown_connections 2023-11-26T07:14:50.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.168+0000 7f05f8b4a700 1 -- 172.21.15.83:0/2199675620 wait complete. 2023-11-26T07:14:50.172 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 30 2023-11-26T07:14:50.183 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:14:50.183 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:14:50.184 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:14:50.184 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:14:50.184 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:14:50.184 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:14:50.184 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:14:50.184 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:14:50.184 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:14:50.185 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:14:50.185 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:14:50.185 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:14:50.185 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:14:50.185 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:14:50.185 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:14:50.185 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:14:50.193 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:14:50.193 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:14:50.194 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:14:50.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:14:50.197 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:14:50.197 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:14:50.198 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:14:50.198 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:14:50.235 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:14:50.236 DEBUG:teuthology.parallel:result is None 2023-11-26T07:14:50.236 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:14:50.236 DEBUG:teuthology.parallel:result is None 2023-11-26T07:14:50.240 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:14:50.240 DEBUG:teuthology.parallel:result is None 2023-11-26T07:14:50.240 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:14:50.240 DEBUG:teuthology.parallel:result is None 2023-11-26T07:14:50.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:14:50.315 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-26T07:14:50.315 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-26T07:14:50.328 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-11-26T07:14:50.328 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-11-26T07:14:50.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.367+0000 7ff6b7896700 1 Processor -- start 2023-11-26T07:14:50.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.367+0000 7ff6b7896700 1 -- start start 2023-11-26T07:14:50.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b7896700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6b015f180 0x7ff6b00cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:50.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b7896700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff6b00cfd40 0x7ff6b00b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:50.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b7896700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b00b4290 con 0x7ff6b013e000 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b7896700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b013e570 con 0x7ff6b00cfd40 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b7896700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b00d05d0 con 0x7ff6b015f180 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b5e33700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff6b00cfd40 0x7ff6b00b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b5632700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff6b013e000 0x7ff6b015ea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58240/0 (socket says 172.21.15.83:58240) 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b5632700 1 -- 172.21.15.83:0/1883723098 learned_addr learned my addr 172.21.15.83:0/1883723098 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b4e31700 1 --2- 172.21.15.83:0/1883723098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6b015f180 0x7ff6b00cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b5e33700 1 -- 172.21.15.83:0/1883723098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6b015f180 msgr2=0x7ff6b00cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b5e33700 1 --2- 172.21.15.83:0/1883723098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6b015f180 0x7ff6b00cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b5e33700 1 -- 172.21.15.83:0/1883723098 >> v1:172.21.15.83:6789/0 conn(0x7ff6b013e000 legacy=0x7ff6b015ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b5e33700 1 -- 172.21.15.83:0/1883723098 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b00d08b0 con 0x7ff6b00cfd40 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6a7fff700 1 -- 172.21.15.83:0/1883723098 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474272018 0 0) 0x7ff6b00b4290 con 0x7ff6b013e000 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6b5e33700 1 --2- 172.21.15.83:0/1883723098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6b00cfd40 0x7ff6b00b4720 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800bfa0 tx=0x7ff6a800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5264b54959e4adc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:50.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6a7fff700 1 -- 172.21.15.83:0/1883723098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8010070 con 0x7ff6b00cfd40 2023-11-26T07:14:50.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6a7fff700 1 -- 172.21.15.83:0/1883723098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a800b2e0 con 0x7ff6b00cfd40 2023-11-26T07:14:50.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.368+0000 7ff6a7fff700 1 -- 172.21.15.83:0/1883723098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800dcf0 con 0x7ff6b00cfd40 2023-11-26T07:14:50.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.369+0000 7ff6b7896700 1 -- 172.21.15.83:0/1883723098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6b00cfd40 msgr2=0x7ff6b00b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:50.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.369+0000 7ff6b7896700 1 --2- 172.21.15.83:0/1883723098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6b00cfd40 0x7ff6b00b4720 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800bfa0 tx=0x7ff6a800d330 comp rx=0 tx=0).stop 2023-11-26T07:14:50.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.369+0000 7ff6b7896700 1 -- 172.21.15.83:0/1883723098 shutdown_connections 2023-11-26T07:14:50.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.369+0000 7ff6b7896700 1 --2- 172.21.15.83:0/1883723098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6b00cfd40 0x7ff6b00b4720 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:50.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.369+0000 7ff6b7896700 1 --2- 172.21.15.83:0/1883723098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6b015f180 0x7ff6b00cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:50.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.369+0000 7ff6b7896700 1 -- 172.21.15.83:0/1883723098 >> 172.21.15.83:0/1883723098 conn(0x7ff6b0072990 msgr2=0x7ff6b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:50.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.369+0000 7ff6b7896700 1 -- 172.21.15.83:0/1883723098 shutdown_connections 2023-11-26T07:14:50.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.369+0000 7ff6b7896700 1 -- 172.21.15.83:0/1883723098 wait complete. 2023-11-26T07:14:50.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.370+0000 7ff6b7896700 1 Processor -- start 2023-11-26T07:14:50.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.370+0000 7ff6b7896700 1 -- start start 2023-11-26T07:14:50.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b7896700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6b00cfd40 0x7ff6b013f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:50.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b7896700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6b013e000 0x7ff6b013b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:50.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b7896700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6b015f180 0x7ff6b00b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:50.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b7896700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b013fa40 con 0x7ff6b013e000 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b7896700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0157ba0 con 0x7ff6b00cfd40 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b7896700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0157e80 con 0x7ff6b015f180 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b4e31700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6b013e000 0x7ff6b013b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b4e31700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6b013e000 0x7ff6b013b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38208/0 (socket says 172.21.15.83:38208) 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b4e31700 1 -- 172.21.15.83:0/1381434426 learned_addr learned my addr 172.21.15.83:0/1381434426 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b5632700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6b00cfd40 0x7ff6b013f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b4e31700 1 -- 172.21.15.83:0/1381434426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6b015f180 msgr2=0x7ff6b00b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b4e31700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6b015f180 0x7ff6b00b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b4e31700 1 -- 172.21.15.83:0/1381434426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6b00cfd40 msgr2=0x7ff6b013f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b4e31700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6b00cfd40 0x7ff6b013f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b4e31700 1 -- 172.21.15.83:0/1381434426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a800bc80 con 0x7ff6b013e000 2023-11-26T07:14:50.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b5632700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6b00cfd40 0x7ff6b013f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.371+0000 7ff6b4e31700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6b013e000 0x7ff6b013b2f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00d800 tx=0x7ff6ac00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.372+0000 7ff6a5ffb700 1 -- 172.21.15.83:0/1381434426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0125f0 con 0x7ff6b013e000 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.372+0000 7ff6a5ffb700 1 -- 172.21.15.83:0/1381434426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac012750 con 0x7ff6b013e000 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.372+0000 7ff6b7896700 1 -- 172.21.15.83:0/1381434426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b013a810 con 0x7ff6b013e000 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.372+0000 7ff6a5ffb700 1 -- 172.21.15.83:0/1381434426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac012b00 con 0x7ff6b013e000 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.372+0000 7ff6b7896700 1 -- 172.21.15.83:0/1381434426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b013add0 con 0x7ff6b013e000 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.373+0000 7ff6b7896700 1 -- 172.21.15.83:0/1381434426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b013af90 con 0x7ff6b013e000 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.373+0000 7ff6a5ffb700 1 -- 172.21.15.83:0/1381434426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7ff6ac012c60 con 0x7ff6b013e000 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.373+0000 7ff6a5ffb700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff69406b080 0x7ff69406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.374+0000 7ff6a5ffb700 1 -- 172.21.15.83:0/1381434426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff6ac089950 con 0x7ff6b013e000 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.374+0000 7ff6b5632700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff69406b080 0x7ff69406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:50.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.374+0000 7ff6b5632700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff69406b080 0x7ff69406d540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0000f60 tx=0x7ff6a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:50.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.378+0000 7ff6a5ffb700 1 -- 172.21.15.83:0/1381434426 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6ac054a80 con 0x7ff6b013e000 2023-11-26T07:14:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:50.584+0000 7ff6b7896700 1 -- 172.21.15.83:0/1381434426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff6b0067f60 con 0x7ff6b013e000 2023-11-26T07:14:51.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.238+0000 7ff6a5ffb700 1 -- 172.21.15.83:0/1381434426 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v52) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff6ac0580a0 con 0x7ff6b013e000 2023-11-26T07:14:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 -- 172.21.15.83:0/1381434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff69406b080 msgr2=0x7ff69406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff69406b080 0x7ff69406d540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0000f60 tx=0x7ff6a0009040 comp rx=0 tx=0).stop 2023-11-26T07:14:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 -- 172.21.15.83:0/1381434426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6b013e000 msgr2=0x7ff6b013b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6b013e000 0x7ff6b013b2f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00d800 tx=0x7ff6ac00dbc0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 -- 172.21.15.83:0/1381434426 shutdown_connections 2023-11-26T07:14:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff69406b080 0x7ff69406d540 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff6b015f180 0x7ff6b00b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6b00cfd40 0x7ff6b013f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 --2- 172.21.15.83:0/1381434426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6b013e000 0x7ff6b013b2f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 -- 172.21.15.83:0/1381434426 >> 172.21.15.83:0/1381434426 conn(0x7ff6b0072990 msgr2=0x7ff6b013d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:51.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 -- 172.21.15.83:0/1381434426 shutdown_connections 2023-11-26T07:14:51.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.241+0000 7ff6b7896700 1 -- 172.21.15.83:0/1381434426 wait complete. 2023-11-26T07:14:51.245 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2023-11-26T07:14:51.257 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:14:51.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.374+0000 7fa2dd2b5700 1 Processor -- start 2023-11-26T07:14:51.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.374+0000 7fa2dd2b5700 1 -- start start 2023-11-26T07:14:51.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.375+0000 7fa2dd2b5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa2d815e080 0x7fa2d80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.375+0000 7fa2dd2b5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d80ae430 con 0x7fa2d806c710 2023-11-26T07:14:51.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.375+0000 7fa2dd2b5700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d80ae710 con 0x7fa2d80af4d0 2023-11-26T07:14:51.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.375+0000 7fa2dd2b5700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d80af0b0 con 0x7fa2d815e080 2023-11-26T07:14:51.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.375+0000 7fa2d77fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa2d806c710 0x7fa2d806e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58258/0 (socket says 172.21.15.83:58258) 2023-11-26T07:14:51.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.375+0000 7fa2d77fe700 1 -- 172.21.15.83:0/3447399607 learned_addr learned my addr 172.21.15.83:0/3447399607 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:51.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d6ffd700 1 --2- 172.21.15.83:0/3447399607 >> v2:172.21.15.203:3300/0 conn(0x7fa2d815e080 0x7fa2d80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906598061 0 0) 0x7fa2d80ae430 con 0x7fa2d806c710 2023-11-26T07:14:51.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c00035b0 con 0x7fa2d806c710 2023-11-26T07:14:51.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4266425037 0 0) 0x7fa2c00035b0 con 0x7fa2d806c710 2023-11-26T07:14:51.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 >> v2:172.21.15.203:3300/0 conn(0x7fa2d815e080 msgr2=0x7fa2d80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:51.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 --2- 172.21.15.83:0/3447399607 >> v2:172.21.15.203:3300/0 conn(0x7fa2d815e080 0x7fa2d80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 >> v1:172.21.15.203:6789/0 conn(0x7fa2d80af4d0 legacy=0x7fa2d806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d80ae9f0 con 0x7fa2d806c710 2023-11-26T07:14:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fa2d00019f0 con 0x7fa2d806c710 2023-11-26T07:14:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 >> v1:172.21.15.83:6789/0 conn(0x7fa2d806c710 legacy=0x7fa2d806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 --2- 172.21.15.83:0/3447399607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c0003a00 0x7fa2c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 --2- 172.21.15.83:0/3447399607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2c0004850 0x7fa2c0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 --2- 172.21.15.83:0/3447399607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2c0005210 0x7fa2c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80af0b0 con 0x7fa2c0005210 2023-11-26T07:14:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c000c0b0 con 0x7fa2c0003a00 2023-11-26T07:14:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c000c270 con 0x7fa2c0004850 2023-11-26T07:14:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d77fe700 1 --2- 172.21.15.83:0/3447399607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2c0005210 0x7fa2c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d6ffd700 1 --2- 172.21.15.83:0/3447399607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c0003a00 0x7fa2c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d67fc700 1 --2- 172.21.15.83:0/3447399607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2c0004850 0x7fa2c0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d77fe700 1 -- 172.21.15.83:0/3447399607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2c0004850 msgr2=0x7fa2c0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d77fe700 1 --2- 172.21.15.83:0/3447399607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2c0004850 0x7fa2c0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d77fe700 1 -- 172.21.15.83:0/3447399607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c0003a00 msgr2=0x7fa2c0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d77fe700 1 --2- 172.21.15.83:0/3447399607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c0003a00 0x7fa2c0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d77fe700 1 -- 172.21.15.83:0/3447399607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c00035b0 con 0x7fa2c0005210 2023-11-26T07:14:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.376+0000 7fa2d77fe700 1 -- 172.21.15.83:0/3447399607 reap_dead start 2023-11-26T07:14:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.377+0000 7fa2d77fe700 1 --2- 172.21.15.83:0/3447399607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2c0005210 0x7fa2c000ba10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000a800 tx=0x7fa2c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dedf348ea491ba4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.377+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000a2f0 con 0x7fa2c0005210 2023-11-26T07:14:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.377+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d0012600 con 0x7fa2c0005210 2023-11-26T07:14:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.377+0000 7fa2d5ffb700 1 -- 172.21.15.83:0/3447399607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00129b0 con 0x7fa2c0005210 2023-11-26T07:14:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.377+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/3447399607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2c0005210 msgr2=0x7fa2c000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.377+0000 7fa2dd2b5700 1 --2- 172.21.15.83:0/3447399607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2c0005210 0x7fa2c000ba10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000a800 tx=0x7fa2c0005880 comp rx=0 tx=0).stop 2023-11-26T07:14:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.377+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/3447399607 shutdown_connections 2023-11-26T07:14:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.377+0000 7fa2dd2b5700 1 --2- 172.21.15.83:0/3447399607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2c0005210 0x7fa2c000ba10 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.377+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/3447399607 >> 172.21.15.83:0/3447399607 conn(0x7fa2d8071b90 msgr2=0x7fa2d8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.377+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/3447399607 shutdown_connections 2023-11-26T07:14:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.377+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/3447399607 wait complete. 2023-11-26T07:14:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.379+0000 7fa2dd2b5700 1 Processor -- start 2023-11-26T07:14:51.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.379+0000 7fa2dd2b5700 1 -- start start 2023-11-26T07:14:51.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.379+0000 7fa2dd2b5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c0005210 0x7fa2d8144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.379+0000 7fa2dd2b5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8156340 0x7fa2d8160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.379+0000 7fa2dd2b5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d8141bb0 0x7fa2d8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.379+0000 7fa2dd2b5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81608c0 con 0x7fa2d8141bb0 2023-11-26T07:14:51.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.379+0000 7fa2dd2b5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d815ff90 con 0x7fa2c0005210 2023-11-26T07:14:51.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.379+0000 7fa2dd2b5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8160270 con 0x7fa2d8156340 2023-11-26T07:14:51.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2d77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d8141bb0 0x7fa2d8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2d77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d8141bb0 0x7fa2d8073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38232/0 (socket says 172.21.15.83:38232) 2023-11-26T07:14:51.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2d77fe700 1 -- 172.21.15.83:0/1631753933 learned_addr learned my addr 172.21.15.83:0/1631753933 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:51.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2d6ffd700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c0005210 0x7fa2d8144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2d67fc700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8156340 0x7fa2d8160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2d77fe700 1 -- 172.21.15.83:0/1631753933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8156340 msgr2=0x7fa2d8160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2d77fe700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8156340 0x7fa2d8160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2d77fe700 1 -- 172.21.15.83:0/1631753933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c0005210 msgr2=0x7fa2d8144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2d77fe700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c0005210 0x7fa2d8144fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2d77fe700 1 -- 172.21.15.83:0/1631753933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d8160550 con 0x7fa2d8141bb0 2023-11-26T07:14:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2d77fe700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d8141bb0 0x7fa2d8073f90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8161a70 tx=0x7fa2d0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2bf7fe700 1 -- 172.21.15.83:0/1631753933 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0001250 con 0x7fa2d8141bb0 2023-11-26T07:14:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2bf7fe700 1 -- 172.21.15.83:0/1631753933 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d0013dc0 con 0x7fa2d8141bb0 2023-11-26T07:14:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/1631753933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d815fc40 con 0x7fa2d8141bb0 2023-11-26T07:14:51.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2bf7fe700 1 -- 172.21.15.83:0/1631753933 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d001d660 con 0x7fa2d8141bb0 2023-11-26T07:14:51.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.380+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/1631753933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d8161290 con 0x7fa2d8141bb0 2023-11-26T07:14:51.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.381+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/1631753933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d80753f0 con 0x7fa2d8141bb0 2023-11-26T07:14:51.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.382+0000 7fa2bf7fe700 1 -- 172.21.15.83:0/1631753933 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fa2d001d7c0 con 0x7fa2d8141bb0 2023-11-26T07:14:51.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.382+0000 7fa2bf7fe700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa2c406b0b0 0x7fa2c406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.382+0000 7fa2bf7fe700 1 -- 172.21.15.83:0/1631753933 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa2d008b4b0 con 0x7fa2d8141bb0 2023-11-26T07:14:51.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.382+0000 7fa2d6ffd700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa2c406b0b0 0x7fa2c406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.383+0000 7fa2d6ffd700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa2c406b0b0 0x7fa2c406d570 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009be0 tx=0x7fa2cc009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:51.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.387+0000 7fa2bf7fe700 1 -- 172.21.15.83:0/1631753933 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa2d00566c0 con 0x7fa2d8141bb0 2023-11-26T07:14:51.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.655+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/1631753933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa2d8067f60 con 0x7fa2d8141bb0 2023-11-26T07:14:51.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.655+0000 7fa2bf7fe700 1 -- 172.21.15.83:0/1631753933 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa2d0059ce0 con 0x7fa2d8141bb0 2023-11-26T07:14:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:51.658 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:14:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2023-11-26T07:14:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2023-11-26T07:14:51.658 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlA8pxNxAAt5NuewAjm7yPThCguJ6sEA==", 2023-11-26T07:14:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:14:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:14:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:14:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2023-11-26T07:14:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlHBgDCBAAk6egig5krY7kiMiHkMKrWg==", 2023-11-26T07:14:51.659 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:14:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:14:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:14:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2023-11-26T07:14:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlbKEQAxAAM2tBBQwM+tt2d8JU37YURg==", 2023-11-26T07:14:51.660 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:14:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:14:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:14:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2023-11-26T07:14:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlI3ZHDxAArtLmBVGr/vAB/YGwEFfhGw==", 2023-11-26T07:14:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.661 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:14:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:14:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:14:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2023-11-26T07:14:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBo72JlhuSTKRAA6LRDOHBQVl5TL5tIHzyfzA==", 2023-11-26T07:14:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.662 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:14:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:14:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2023-11-26T07:14:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBr72JlteU6KRAAuuHTYosYpl+KptywVmaeHA==", 2023-11-26T07:14:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:14:51.663 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:14:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2023-11-26T07:14:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBu72JlEb6dKBAAebp4pnu9XZsSRJpojTZcvQ==", 2023-11-26T07:14:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:14:51.664 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:14:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2023-11-26T07:14:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBx72JlbTJ3KBAA1JUxxAoyCzTtfgubUxSJEA==", 2023-11-26T07:14:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:14:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:14:51.665 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2023-11-26T07:14:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB372JlufqqLhAAm6ZUt9juWj5yAIDlnCXqWg==", 2023-11-26T07:14:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:14:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:14:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2023-11-26T07:14:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB672Jl1L3vLRAAX06GcsntcmB+7L1CsfnJzw==", 2023-11-26T07:14:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:14:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:14:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2023-11-26T07:14:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB972Jlv41HLRAALtuyuREvGmTuDZt/RT0UcA==", 2023-11-26T07:14:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:14:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:14:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.668 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2023-11-26T07:14:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCA72JlKCHsLBAAAXDzA7AN36wcLjhK97nlkg==", 2023-11-26T07:14:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:14:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:14:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2023-11-26T07:14:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ==", 2023-11-26T07:14:51.669 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:14:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2023-11-26T07:14:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:14:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2023-11-26T07:14:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JleNpIGhAAKwoQj4zB9YOxpFMD28juCA==", 2023-11-26T07:14:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.670 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:14:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:14:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:14:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:14:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2023-11-26T07:14:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JldAHlFhAAb3u9KljU9vssDl54zgno2Q==", 2023-11-26T07:14:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.671 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:14:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:14:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:14:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:14:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2023-11-26T07:14:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JluiQUIBAA06mqpeCzgpbSLo2/fulfmw==", 2023-11-26T07:14:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:14:51.672 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:14:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:14:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:14:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2023-11-26T07:14:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlD4jbGhAA05xtSS5EzQTuhBjVqS2JpA==", 2023-11-26T07:14:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:14:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:14:51.673 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:14:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2023-11-26T07:14:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlsA0VNBAAfSlnVquhI+vpLFwmgLGGVA==", 2023-11-26T07:14:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:14:51.674 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:14:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:14:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2023-11-26T07:14:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72Jl3qkjMBAA+jcxYuDG/UHhB7W7HDwmaw==", 2023-11-26T07:14:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:14:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:14:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:14:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:14:51.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:51.676 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:14:51.676 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:14:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.657+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/1631753933 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa2c406b0b0 msgr2=0x7fa2c406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.657+0000 7fa2dd2b5700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa2c406b0b0 0x7fa2c406d570 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009be0 tx=0x7fa2cc009210 comp rx=0 tx=0).stop 2023-11-26T07:14:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.657+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/1631753933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d8141bb0 msgr2=0x7fa2d8073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.657+0000 7fa2dd2b5700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d8141bb0 0x7fa2d8073f90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8161a70 tx=0x7fa2d0001d80 comp rx=0 tx=0).stop 2023-11-26T07:14:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.657+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/1631753933 shutdown_connections 2023-11-26T07:14:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.657+0000 7fa2dd2b5700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d8141bb0 0x7fa2d8073f90 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.657+0000 7fa2dd2b5700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2c0005210 0x7fa2d8144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.657+0000 7fa2dd2b5700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa2c406b0b0 0x7fa2c406d570 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.657+0000 7fa2dd2b5700 1 --2- 172.21.15.83:0/1631753933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2d8156340 0x7fa2d8160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.657+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/1631753933 >> 172.21.15.83:0/1631753933 conn(0x7fa2d8071b90 msgr2=0x7fa2d815d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.657+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/1631753933 shutdown_connections 2023-11-26T07:14:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.658+0000 7fa2dd2b5700 1 -- 172.21.15.83:0/1631753933 wait complete. 2023-11-26T07:14:51.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:14:51.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.796+0000 7f0bdfadc700 1 Processor -- start 2023-11-26T07:14:51.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.796+0000 7f0bdfadc700 1 -- start start 2023-11-26T07:14:51.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.797+0000 7f0bdfadc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0bd815ae20 0x7f0bd80b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.797+0000 7f0bdfadc700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd80cfd40 con 0x7f0bd815b200 2023-11-26T07:14:51.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.797+0000 7f0bdfadc700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd80affa0 con 0x7f0bd80b1a70 2023-11-26T07:14:51.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.797+0000 7f0bdfadc700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd80d0020 con 0x7f0bd815ae20 2023-11-26T07:14:51.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.797+0000 7f0bdd077700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0bd815b200 0x7f0bd80b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58276/0 (socket says 172.21.15.83:58276) 2023-11-26T07:14:51.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.797+0000 7f0bdd077700 1 -- 172.21.15.83:0/1346168077 learned_addr learned my addr 172.21.15.83:0/1346168077 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:51.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.797+0000 7f0bdd878700 1 --2- 172.21.15.83:0/1346168077 >> v2:172.21.15.203:3300/0 conn(0x7f0bd815ae20 0x7f0bd80b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.797+0000 7f0bdc876700 1 -- 172.21.15.83:0/1346168077 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024276716 0 0) 0x7f0bd80cfd40 con 0x7f0bd815b200 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.797+0000 7f0bdc876700 1 -- 172.21.15.83:0/1346168077 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bc40035b0 con 0x7f0bd815b200 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdc876700 1 -- 172.21.15.83:0/1346168077 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230535493 0 0) 0x7f0bd80affa0 con 0x7f0bd80b1a70 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdc876700 1 -- 172.21.15.83:0/1346168077 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bd80cfd40 con 0x7f0bd80b1a70 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdc876700 1 -- 172.21.15.83:0/1346168077 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 239510573 0 0) 0x7f0bc40035b0 con 0x7f0bd815b200 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdd878700 1 -- 172.21.15.83:0/1346168077 >> v1:172.21.15.203:6789/0 conn(0x7f0bd80b1a70 legacy=0x7f0bd8164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdd878700 1 -- 172.21.15.83:0/1346168077 >> v1:172.21.15.83:6789/0 conn(0x7f0bd815b200 legacy=0x7f0bd80b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdd878700 1 -- 172.21.15.83:0/1346168077 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd80b0280 con 0x7f0bd815ae20 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdd878700 1 --2- 172.21.15.83:0/1346168077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd815ae20 0x7f0bd80b29a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40098a0 tx=0x7f0bd4009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a0154ed52d9ca3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdc876700 1 -- 172.21.15.83:0/1346168077 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8001670 con 0x7f0bd815ae20 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdc876700 1 -- 172.21.15.83:0/1346168077 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd4004030 con 0x7f0bd815ae20 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdc876700 1 -- 172.21.15.83:0/1346168077 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40043c0 con 0x7f0bd815ae20 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdfadc700 1 -- 172.21.15.83:0/1346168077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd815ae20 msgr2=0x7f0bd80b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdfadc700 1 --2- 172.21.15.83:0/1346168077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd815ae20 0x7f0bd80b29a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40098a0 tx=0x7f0bd4009bb0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdfadc700 1 -- 172.21.15.83:0/1346168077 shutdown_connections 2023-11-26T07:14:51.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.798+0000 7f0bdfadc700 1 --2- 172.21.15.83:0/1346168077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd815ae20 0x7f0bd80b29a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.799+0000 7f0bdfadc700 1 -- 172.21.15.83:0/1346168077 >> 172.21.15.83:0/1346168077 conn(0x7f0bd80b57b0 msgr2=0x7f0bd80b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:51.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.799+0000 7f0bdfadc700 1 -- 172.21.15.83:0/1346168077 shutdown_connections 2023-11-26T07:14:51.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.799+0000 7f0bdfadc700 1 -- 172.21.15.83:0/1346168077 wait complete. 2023-11-26T07:14:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.800+0000 7f0bdfadc700 1 Processor -- start 2023-11-26T07:14:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.800+0000 7f0bdfadc700 1 -- start start 2023-11-26T07:14:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.800+0000 7f0bdfadc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bd80b1a70 0x7f0bd8156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.800+0000 7f0bdfadc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd815ae20 0x7f0bd81549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.800+0000 7f0bdfadc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd815b200 0x7f0bd80b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.800+0000 7f0bdfadc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd81f0640 con 0x7f0bd815b200 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bdfadc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd8157120 con 0x7f0bd815ae20 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bdfadc700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd81ef9c0 con 0x7f0bd80b1a70 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bde079700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd815b200 0x7f0bd80b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bde079700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd815b200 0x7f0bd80b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38246/0 (socket says 172.21.15.83:38246) 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bde079700 1 -- 172.21.15.83:0/83843470 learned_addr learned my addr 172.21.15.83:0/83843470 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bdd077700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd815ae20 0x7f0bd81549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bdd878700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bd80b1a70 0x7f0bd8156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bde079700 1 -- 172.21.15.83:0/83843470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bd80b1a70 msgr2=0x7f0bd8156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bde079700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bd80b1a70 0x7f0bd8156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bde079700 1 -- 172.21.15.83:0/83843470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd815ae20 msgr2=0x7f0bd81549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bde079700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd815ae20 0x7f0bd81549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:51.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bde079700 1 -- 172.21.15.83:0/83843470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd40094a0 con 0x7f0bd815b200 2023-11-26T07:14:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bdd077700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd815ae20 0x7f0bd81549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bdd878700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bd80b1a70 0x7f0bd8156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bde079700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd815b200 0x7f0bd80b0980 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000cbc0 tx=0x7f0bd000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bce7fc700 1 -- 172.21.15.83:0/83843470 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0003c00 con 0x7f0bd815b200 2023-11-26T07:14:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bce7fc700 1 -- 172.21.15.83:0/83843470 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0003d60 con 0x7f0bd815b200 2023-11-26T07:14:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bdfadc700 1 -- 172.21.15.83:0/83843470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd81efca0 con 0x7f0bd815b200 2023-11-26T07:14:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bce7fc700 1 -- 172.21.15.83:0/83843470 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000abf0 con 0x7f0bd815b200 2023-11-26T07:14:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.801+0000 7f0bdfadc700 1 -- 172.21.15.83:0/83843470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd81f0260 con 0x7f0bd815b200 2023-11-26T07:14:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.802+0000 7f0bdfadc700 1 -- 172.21.15.83:0/83843470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd8067f60 con 0x7f0bd815b200 2023-11-26T07:14:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.803+0000 7f0bce7fc700 1 -- 172.21.15.83:0/83843470 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0bd000ad50 con 0x7f0bd815b200 2023-11-26T07:14:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.803+0000 7f0bce7fc700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0bbc06b080 0x7f0bbc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.803+0000 7f0bce7fc700 1 -- 172.21.15.83:0/83843470 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0bd0089780 con 0x7f0bd815b200 2023-11-26T07:14:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.803+0000 7f0bdd878700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0bbc06b080 0x7f0bbc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.803+0000 7f0bdd878700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0bbc06b080 0x7f0bbc06d540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40090b0 tx=0x7f0bd401b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:51.808+0000 7f0bce7fc700 1 -- 172.21.15.83:0/83843470 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0bd0054880 con 0x7f0bd815b200 2023-11-26T07:14:52.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.000+0000 7f0bdfadc700 1 -- 172.21.15.83:0/83843470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bd80cde80 con 0x7f0bd815b200 2023-11-26T07:14:52.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.001+0000 7f0bce7fc700 1 -- 172.21.15.83:0/83843470 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+12845 (secure 0 0 0) 0x7f0bd0057ea0 con 0x7f0bd815b200 2023-11-26T07:14:52.003 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:52.003 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":54,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:14:51.245842+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:14:52.005 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 -- 172.21.15.83:0/83843470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0bbc06b080 msgr2=0x7f0bbc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0bbc06b080 0x7f0bbc06d540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40090b0 tx=0x7f0bd401b040 comp rx=0 tx=0).stop 2023-11-26T07:14:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 -- 172.21.15.83:0/83843470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd815b200 msgr2=0x7f0bd80b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd815b200 0x7f0bd80b0980 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000cbc0 tx=0x7f0bd000cf80 comp rx=0 tx=0).stop 2023-11-26T07:14:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 -- 172.21.15.83:0/83843470 shutdown_connections 2023-11-26T07:14:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd815b200 0x7f0bd80b0980 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0bbc06b080 0x7f0bbc06d540 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0bd80b1a70 0x7f0bd8156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 --2- 172.21.15.83:0/83843470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bd815ae20 0x7f0bd81549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 -- 172.21.15.83:0/83843470 >> 172.21.15.83:0/83843470 conn(0x7f0bd80b57b0 msgr2=0x7f0bd8155560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 -- 172.21.15.83:0/83843470 shutdown_connections 2023-11-26T07:14:52.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.003+0000 7f0bdfadc700 1 -- 172.21.15.83:0/83843470 wait complete. 2023-11-26T07:14:52.017 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:14:52.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:14:52.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.139+0000 7fac9e7a3700 1 Processor -- start 2023-11-26T07:14:52.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.139+0000 7fac9e7a3700 1 -- start start 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac9e7a3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac9815b370 0x7fac9815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac9e7a3700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fac980b3c60 0x7fac980b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac9e7a3700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac980ae5e0 con 0x7fac981587f0 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac9e7a3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980b4630 con 0x7fac9815b370 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac9e7a3700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9815be30 con 0x7fac980b3c60 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac977fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac9815b370 0x7fac9815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac977fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac9815b370 0x7fac9815b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38252/0 (socket says 172.21.15.83:38252) 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac977fe700 1 -- 172.21.15.83:0/3721861927 learned_addr learned my addr 172.21.15.83:0/3721861927 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac977fe700 1 -- 172.21.15.83:0/3721861927 >> v2:172.21.15.203:3300/0 conn(0x7fac980b3c60 msgr2=0x7fac980b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac977fe700 1 --2- 172.21.15.83:0/3721861927 >> v2:172.21.15.203:3300/0 conn(0x7fac980b3c60 0x7fac980b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac977fe700 1 -- 172.21.15.83:0/3721861927 >> v1:172.21.15.203:6789/0 conn(0x7fac981587f0 legacy=0x7fac9806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac977fe700 1 -- 172.21.15.83:0/3721861927 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9815c110 con 0x7fac9815b370 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac977fe700 1 --2- 172.21.15.83:0/3721861927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac9815b370 0x7fac9815b800 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fac880098f0 tx=0x7fac88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ddccd55defe340 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac96ffd700 1 -- 172.21.15.83:0/3721861927 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88004230 con 0x7fac9815b370 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac96ffd700 1 -- 172.21.15.83:0/3721861927 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac88004390 con 0x7fac9815b370 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.140+0000 7fac96ffd700 1 -- 172.21.15.83:0/3721861927 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac880036a0 con 0x7fac9815b370 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.141+0000 7fac9e7a3700 1 -- 172.21.15.83:0/3721861927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac9815b370 msgr2=0x7fac9815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.141+0000 7fac9e7a3700 1 --2- 172.21.15.83:0/3721861927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac9815b370 0x7fac9815b800 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fac880098f0 tx=0x7fac88009c00 comp rx=0 tx=0).stop 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.141+0000 7fac9e7a3700 1 -- 172.21.15.83:0/3721861927 shutdown_connections 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.141+0000 7fac9e7a3700 1 --2- 172.21.15.83:0/3721861927 >> v2:172.21.15.203:3300/0 conn(0x7fac980b3c60 0x7fac980b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.141+0000 7fac9e7a3700 1 --2- 172.21.15.83:0/3721861927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac9815b370 0x7fac9815b800 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.141+0000 7fac9e7a3700 1 -- 172.21.15.83:0/3721861927 >> 172.21.15.83:0/3721861927 conn(0x7fac980d0bf0 msgr2=0x7fac980d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.141+0000 7fac9e7a3700 1 -- 172.21.15.83:0/3721861927 shutdown_connections 2023-11-26T07:14:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.141+0000 7fac9e7a3700 1 -- 172.21.15.83:0/3721861927 wait complete. 2023-11-26T07:14:52.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.142+0000 7fac9e7a3700 1 Processor -- start 2023-11-26T07:14:52.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.142+0000 7fac9e7a3700 1 -- start start 2023-11-26T07:14:52.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.143+0000 7fac9e7a3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac980b3c60 0x7fac981f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.143+0000 7fac9e7a3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac981587f0 0x7fac981f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.143+0000 7fac9e7a3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac9815b370 0x7fac980cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.143+0000 7fac9e7a3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac981f5de0 con 0x7fac9815b370 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.143+0000 7fac9e7a3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac981e3930 con 0x7fac980b3c60 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.143+0000 7fac9e7a3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac981f4b10 con 0x7fac981587f0 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.143+0000 7fac9cd40700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac9815b370 0x7fac980cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.143+0000 7fac9cd40700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac9815b370 0x7fac980cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38260/0 (socket says 172.21.15.83:38260) 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.143+0000 7fac9cd40700 1 -- 172.21.15.83:0/1200372444 learned_addr learned my addr 172.21.15.83:0/1200372444 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac97fff700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac980b3c60 0x7fac981f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac977fe700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac981587f0 0x7fac981f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac9cd40700 1 -- 172.21.15.83:0/1200372444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac981587f0 msgr2=0x7fac981f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac9cd40700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac981587f0 0x7fac981f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac9cd40700 1 -- 172.21.15.83:0/1200372444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac980b3c60 msgr2=0x7fac981f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac9cd40700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac980b3c60 0x7fac981f1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac9cd40700 1 -- 172.21.15.83:0/1200372444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88009550 con 0x7fac9815b370 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac97fff700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac980b3c60 0x7fac981f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac977fe700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac981587f0 0x7fac981f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac9cd40700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac9815b370 0x7fac980cf4d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fac9000b970 tx=0x7fac9000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac94ff9700 1 -- 172.21.15.83:0/1200372444 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000c9b0 con 0x7fac9815b370 2023-11-26T07:14:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac94ff9700 1 -- 172.21.15.83:0/1200372444 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9000cb30 con 0x7fac9815b370 2023-11-26T07:14:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac9e7a3700 1 -- 172.21.15.83:0/1200372444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac981f51d0 con 0x7fac9815b370 2023-11-26T07:14:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac94ff9700 1 -- 172.21.15.83:0/1200372444 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac90014440 con 0x7fac9815b370 2023-11-26T07:14:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.144+0000 7fac9e7a3700 1 -- 172.21.15.83:0/1200372444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac981f5790 con 0x7fac9815b370 2023-11-26T07:14:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.145+0000 7fac9e7a3700 1 -- 172.21.15.83:0/1200372444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac981f5f50 con 0x7fac9815b370 2023-11-26T07:14:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.145+0000 7fac94ff9700 1 -- 172.21.15.83:0/1200372444 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fac9001d3b0 con 0x7fac9815b370 2023-11-26T07:14:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.146+0000 7fac94ff9700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fac8406b080 0x7fac8406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.146+0000 7fac94ff9700 1 -- 172.21.15.83:0/1200372444 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fac90006e40 con 0x7fac9815b370 2023-11-26T07:14:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.146+0000 7fac97fff700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fac8406b080 0x7fac8406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.146+0000 7fac97fff700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fac8406b080 0x7fac8406d540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fac98153f20 tx=0x7fac8c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:52.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.150+0000 7fac94ff9700 1 -- 172.21.15.83:0/1200372444 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fac90054510 con 0x7fac9815b370 2023-11-26T07:14:52.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.341+0000 7fac9e7a3700 1 -- 172.21.15.83:0/1200372444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac98068fb0 con 0x7fac9815b370 2023-11-26T07:14:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.341+0000 7fac94ff9700 1 -- 172.21.15.83:0/1200372444 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac900190a0 con 0x7fac9815b370 2023-11-26T07:14:52.344 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:14:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 -- 172.21.15.83:0/1200372444 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fac8406b080 msgr2=0x7fac8406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fac8406b080 0x7fac8406d540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fac98153f20 tx=0x7fac8c009040 comp rx=0 tx=0).stop 2023-11-26T07:14:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 -- 172.21.15.83:0/1200372444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac9815b370 msgr2=0x7fac980cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac9815b370 0x7fac980cf4d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fac9000b970 tx=0x7fac9000bd30 comp rx=0 tx=0).stop 2023-11-26T07:14:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 -- 172.21.15.83:0/1200372444 shutdown_connections 2023-11-26T07:14:52.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac9815b370 0x7fac980cf4d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac980b3c60 0x7fac981f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fac8406b080 0x7fac8406d540 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 --2- 172.21.15.83:0/1200372444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac981587f0 0x7fac981f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 -- 172.21.15.83:0/1200372444 >> 172.21.15.83:0/1200372444 conn(0x7fac980d0bf0 msgr2=0x7fac9815cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:52.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 -- 172.21.15.83:0/1200372444 shutdown_connections 2023-11-26T07:14:52.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.343+0000 7fac9e7a3700 1 -- 172.21.15.83:0/1200372444 wait complete. 2023-11-26T07:14:52.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:14:52.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.476+0000 7eff26f0e700 1 Processor -- start 2023-11-26T07:14:52.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.476+0000 7eff26f0e700 1 -- start start 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff26f0e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff200b56e0 0x7eff20164f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff26f0e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff200b3640 con 0x7eff20159f60 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff26f0e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff200b3920 con 0x7eff200b30d0 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff26f0e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff200b3c00 con 0x7eff200b56e0 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff254ab700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff200b56e0 0x7eff20164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff254ab700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff200b56e0 0x7eff20164f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38270/0 (socket says 172.21.15.83:38270) 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff254ab700 1 -- 172.21.15.83:0/3694635206 learned_addr learned my addr 172.21.15.83:0/3694635206 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff254ab700 1 -- 172.21.15.83:0/3694635206 >> v1:172.21.15.203:6789/0 conn(0x7eff200b30d0 legacy=0x7eff200b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff254ab700 1 -- 172.21.15.83:0/3694635206 >> v1:172.21.15.83:6789/0 conn(0x7eff20159f60 legacy=0x7eff2015a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff254ab700 1 -- 172.21.15.83:0/3694635206 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff200b3ee0 con 0x7eff200b56e0 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff1f7fe700 1 -- 172.21.15.83:0/3694635206 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468522498 0 0) 0x7eff200b3640 con 0x7eff20159f60 2023-11-26T07:14:52.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff254ab700 1 --2- 172.21.15.83:0/3694635206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff200b56e0 0x7eff20164f50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7eff1800af60 tx=0x7eff1800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6e1df4c72c7964 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.477+0000 7eff1f7fe700 1 -- 172.21.15.83:0/3694635206 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1800f070 con 0x7eff200b56e0 2023-11-26T07:14:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.478+0000 7eff1f7fe700 1 -- 172.21.15.83:0/3694635206 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1800a2e0 con 0x7eff200b56e0 2023-11-26T07:14:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.478+0000 7eff1f7fe700 1 -- 172.21.15.83:0/3694635206 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1800cc70 con 0x7eff200b56e0 2023-11-26T07:14:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.478+0000 7eff26f0e700 1 -- 172.21.15.83:0/3694635206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff200b56e0 msgr2=0x7eff20164f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.478+0000 7eff26f0e700 1 --2- 172.21.15.83:0/3694635206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff200b56e0 0x7eff20164f50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7eff1800af60 tx=0x7eff1800af90 comp rx=0 tx=0).stop 2023-11-26T07:14:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.478+0000 7eff26f0e700 1 -- 172.21.15.83:0/3694635206 shutdown_connections 2023-11-26T07:14:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.478+0000 7eff26f0e700 1 --2- 172.21.15.83:0/3694635206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff200b56e0 0x7eff20164f50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.478+0000 7eff26f0e700 1 -- 172.21.15.83:0/3694635206 >> 172.21.15.83:0/3694635206 conn(0x7eff2006cec0 msgr2=0x7eff2006d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.478+0000 7eff26f0e700 1 -- 172.21.15.83:0/3694635206 shutdown_connections 2023-11-26T07:14:52.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.478+0000 7eff26f0e700 1 -- 172.21.15.83:0/3694635206 wait complete. 2023-11-26T07:14:52.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.480+0000 7eff26f0e700 1 Processor -- start 2023-11-26T07:14:52.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.480+0000 7eff26f0e700 1 -- start start 2023-11-26T07:14:52.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff26f0e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff200b30d0 0x7eff20158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff26f0e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff200b56e0 0x7eff200a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff26f0e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff20159f60 0x7eff200aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff26f0e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff200aa920 con 0x7eff200b56e0 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff26f0e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2015b910 con 0x7eff20159f60 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff26f0e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2015bbf0 con 0x7eff200b30d0 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff1ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff200b56e0 0x7eff200a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff1ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff200b56e0 0x7eff200a9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38274/0 (socket says 172.21.15.83:38274) 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff1ffff700 1 -- 172.21.15.83:0/382038167 learned_addr learned my addr 172.21.15.83:0/382038167 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff254ab700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff20159f60 0x7eff200aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff24caa700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff200b30d0 0x7eff20158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff1ffff700 1 -- 172.21.15.83:0/382038167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff200b30d0 msgr2=0x7eff20158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff1ffff700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff200b30d0 0x7eff20158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff1ffff700 1 -- 172.21.15.83:0/382038167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff20159f60 msgr2=0x7eff200aa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff1ffff700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff20159f60 0x7eff200aa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff1ffff700 1 -- 172.21.15.83:0/382038167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1800ac40 con 0x7eff200b56e0 2023-11-26T07:14:52.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff1ffff700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff200b56e0 0x7eff200a9ea0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00c4a0 tx=0x7eff0c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:52.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.481+0000 7eff1d7fa700 1 -- 172.21.15.83:0/382038167 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0054e0 con 0x7eff200b56e0 2023-11-26T07:14:52.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.482+0000 7eff1d7fa700 1 -- 172.21.15.83:0/382038167 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0c003680 con 0x7eff200b56e0 2023-11-26T07:14:52.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.482+0000 7eff26f0e700 1 -- 172.21.15.83:0/382038167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2015bed0 con 0x7eff200b56e0 2023-11-26T07:14:52.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.482+0000 7eff1d7fa700 1 -- 172.21.15.83:0/382038167 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c003a30 con 0x7eff200b56e0 2023-11-26T07:14:52.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.482+0000 7eff26f0e700 1 -- 172.21.15.83:0/382038167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff20158d50 con 0x7eff200b56e0 2023-11-26T07:14:52.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.482+0000 7eff26f0e700 1 -- 172.21.15.83:0/382038167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff20158f10 con 0x7eff200b56e0 2023-11-26T07:14:52.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.483+0000 7eff1d7fa700 1 -- 172.21.15.83:0/382038167 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7eff0c003b90 con 0x7eff200b56e0 2023-11-26T07:14:52.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.483+0000 7eff1d7fa700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff0406b080 0x7eff0406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.483+0000 7eff1d7fa700 1 -- 172.21.15.83:0/382038167 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7eff0c088a80 con 0x7eff200b56e0 2023-11-26T07:14:52.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.483+0000 7eff24caa700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff0406b080 0x7eff0406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.484+0000 7eff24caa700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff0406b080 0x7eff0406d540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7eff14000f60 tx=0x7eff14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:52.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.488+0000 7eff1d7fa700 1 -- 172.21.15.83:0/382038167 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff0c053c90 con 0x7eff200b56e0 2023-11-26T07:14:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.682+0000 7eff26f0e700 1 -- 172.21.15.83:0/382038167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7eff20067f60 con 0x7eff200b56e0 2023-11-26T07:14:52.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.683+0000 7eff1d7fa700 1 -- 172.21.15.83:0/382038167 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) v1 ==== 84+0+65 (secure 0 0 0) 0x7eff0c017020 con 0x7eff200b56e0 2023-11-26T07:14:52.686 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:52.686 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:14:52.686 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:14:52.686 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2023-11-26T07:14:52.686 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2023-11-26T07:14:52.686 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:14:52.686 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:14:52.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 -- 172.21.15.83:0/382038167 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff0406b080 msgr2=0x7eff0406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff0406b080 0x7eff0406d540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7eff14000f60 tx=0x7eff14009040 comp rx=0 tx=0).stop 2023-11-26T07:14:52.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 -- 172.21.15.83:0/382038167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff200b56e0 msgr2=0x7eff200a9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff200b56e0 0x7eff200a9ea0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00c4a0 tx=0x7eff0c00c860 comp rx=0 tx=0).stop 2023-11-26T07:14:52.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 -- 172.21.15.83:0/382038167 shutdown_connections 2023-11-26T07:14:52.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff20159f60 0x7eff200aa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff0406b080 0x7eff0406d540 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff200b30d0 0x7eff20158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 --2- 172.21.15.83:0/382038167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff200b56e0 0x7eff200a9ea0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 -- 172.21.15.83:0/382038167 >> 172.21.15.83:0/382038167 conn(0x7eff2006cec0 msgr2=0x7eff20154500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:52.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 -- 172.21.15.83:0/382038167 shutdown_connections 2023-11-26T07:14:52.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.685+0000 7eff26f0e700 1 -- 172.21.15.83:0/382038167 wait complete. 2023-11-26T07:14:52.699 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:14:52.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:14:52.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.822+0000 7f6669b87700 1 Processor -- start 2023-11-26T07:14:52.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.822+0000 7f6669b87700 1 -- start start 2023-11-26T07:14:52.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.823+0000 7f6669b87700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415df20 0x7f666406c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.823+0000 7f6669b87700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666406bed0 con 0x7f6664162820 2023-11-26T07:14:52.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.823+0000 7f6669b87700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666406bba0 con 0x7f66641622b0 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.823+0000 7f6669b87700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6664070060 con 0x7f666415df20 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.823+0000 7f6662ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6664162820 0x7f6664161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58320/0 (socket says 172.21.15.83:58320) 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.823+0000 7f6662ffd700 1 -- 172.21.15.83:0/2246481387 learned_addr learned my addr 172.21.15.83:0/2246481387 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.823+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232220277 0 0) 0x7f666406bed0 con 0x7f6664162820 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66500035b0 con 0x7f6664162820 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66637fe700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415df20 0x7f666406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1799661410 0 0) 0x7f66500035b0 con 0x7f6664162820 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415df20 msgr2=0x7f666406c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415df20 0x7f666406c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 >> v1:172.21.15.203:6789/0 conn(0x7f66641622b0 legacy=0x7f66640cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666406faa0 con 0x7f6664162820 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f6654001670 con 0x7f6664162820 2023-11-26T07:14:52.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 >> v1:172.21.15.83:6789/0 conn(0x7f6664162820 legacy=0x7f6664161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6650003a30 0x7f6650003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650004850 0x7f6650004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415df20 0x7f6650005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6664070060 con 0x7f6650003a30 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665000bac0 con 0x7f6650004850 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665000bc80 con 0x7f666415df20 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66637fe700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6650003a30 0x7f6650003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f6662ffd700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650004850 0x7f6650004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f6663fff700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415df20 0x7f6650005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66637fe700 1 -- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415df20 msgr2=0x7f6650005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66637fe700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415df20 0x7f6650005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66637fe700 1 -- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650004850 msgr2=0x7f6650004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:52.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66637fe700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650004850 0x7f6650004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66637fe700 1 -- 172.21.15.83:0/2246481387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66500035b0 con 0x7f6650003a30 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f6663fff700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415df20 0x7f6650005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.824+0000 7f66637fe700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6650003a30 0x7f6650003e50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f664c009b30 tx=0x7f6650005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cdb87290dca2f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.825+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c00e6c0 con 0x7f6650003a30 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.825+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c00e820 con 0x7f6650003a30 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.825+0000 7f66627fc700 1 -- 172.21.15.83:0/2246481387 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c00eb90 con 0x7f6650003a30 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.825+0000 7f6669b87700 1 -- 172.21.15.83:0/2246481387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6650003a30 msgr2=0x7f6650003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.825+0000 7f6669b87700 1 --2- 172.21.15.83:0/2246481387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6650003a30 0x7f6650003e50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f664c009b30 tx=0x7f6650005880 comp rx=0 tx=0).stop 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.825+0000 7f6663fff700 1 -- 172.21.15.83:0/2246481387 reap_dead start 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.825+0000 7f6669b87700 1 -- 172.21.15.83:0/2246481387 shutdown_connections 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.825+0000 7f6669b87700 1 -- 172.21.15.83:0/2246481387 >> 172.21.15.83:0/2246481387 conn(0x7f666406dc80 msgr2=0x7f666409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.825+0000 7f6669b87700 1 -- 172.21.15.83:0/2246481387 shutdown_connections 2023-11-26T07:14:52.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.825+0000 7f6669b87700 1 -- 172.21.15.83:0/2246481387 wait complete. 2023-11-26T07:14:52.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.827+0000 7f6669b87700 1 Processor -- start 2023-11-26T07:14:52.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.827+0000 7f6669b87700 1 -- start start 2023-11-26T07:14:52.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6669b87700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650003a30 0x7f66641f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6669b87700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66640b4ca0 0x7f66640b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6669b87700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415cce0 0x7f666415d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6669b87700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66640b5660 con 0x7f66640b4ca0 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6669b87700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66641e9c60 con 0x7f6650003a30 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6669b87700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66640c6ac0 con 0x7f666415cce0 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6662ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66640b4ca0 0x7f66640b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6662ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66640b4ca0 0x7f66640b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38294/0 (socket says 172.21.15.83:38294) 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6662ffd700 1 -- 172.21.15.83:0/1970577361 learned_addr learned my addr 172.21.15.83:0/1970577361 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f66637fe700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650003a30 0x7f66641f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6663fff700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415cce0 0x7f666415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6662ffd700 1 -- 172.21.15.83:0/1970577361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415cce0 msgr2=0x7f666415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6662ffd700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415cce0 0x7f666415d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6662ffd700 1 -- 172.21.15.83:0/1970577361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650003a30 msgr2=0x7f66641f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6662ffd700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650003a30 0x7f66641f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:52.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6662ffd700 1 -- 172.21.15.83:0/1970577361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66641e9980 con 0x7f66640b4ca0 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f66637fe700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650003a30 0x7f66641f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f6662ffd700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66640b4ca0 0x7f66640b5120 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f66540072c0 tx=0x7f6654007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f665bfff700 1 -- 172.21.15.83:0/1970577361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66540014c0 con 0x7f66640b4ca0 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.828+0000 7f665bfff700 1 -- 172.21.15.83:0/1970577361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665400f850 con 0x7f66640b4ca0 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.829+0000 7f665bfff700 1 -- 172.21.15.83:0/1970577361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665400fc00 con 0x7f66640b4ca0 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.829+0000 7f6669b87700 1 -- 172.21.15.83:0/1970577361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66640c6d60 con 0x7f66640b4ca0 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.829+0000 7f6669b87700 1 -- 172.21.15.83:0/1970577361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666415d6b0 con 0x7f66640b4ca0 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.829+0000 7f6669b87700 1 -- 172.21.15.83:0/1970577361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66640c73b0 con 0x7f66640b4ca0 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.830+0000 7f665bfff700 1 -- 172.21.15.83:0/1970577361 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f665400fd60 con 0x7f66640b4ca0 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.830+0000 7f665bfff700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f664406b080 0x7f664406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.830+0000 7f66637fe700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f664406b080 0x7f664406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.830+0000 7f665bfff700 1 -- 172.21.15.83:0/1970577361 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6654089660 con 0x7f66640b4ca0 2023-11-26T07:14:52.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.831+0000 7f66637fe700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f664406b080 0x7f664406d540 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f664c0090b0 tx=0x7f664c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:52.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:52.835+0000 7f665bfff700 1 -- 172.21.15.83:0/1970577361 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6654054750 con 0x7f66640b4ca0 2023-11-26T07:14:53.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.025+0000 7f6669b87700 1 -- 172.21.15.83:0/1970577361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6664067f60 con 0x7f66640b4ca0 2023-11-26T07:14:53.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.528+0000 7f665bfff700 1 -- 172.21.15.83:0/1970577361 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6654057d70 con 0x7f66640b4ca0 2023-11-26T07:14:53.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.531+0000 7f6669b87700 1 -- 172.21.15.83:0/1970577361 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f664406b080 msgr2=0x7f664406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:53.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.531+0000 7f6669b87700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f664406b080 0x7f664406d540 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f664c0090b0 tx=0x7f664c008030 comp rx=0 tx=0).stop 2023-11-26T07:14:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.531+0000 7f6669b87700 1 -- 172.21.15.83:0/1970577361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66640b4ca0 msgr2=0x7f66640b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.531+0000 7f6669b87700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66640b4ca0 0x7f66640b5120 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f66540072c0 tx=0x7f6654007680 comp rx=0 tx=0).stop 2023-11-26T07:14:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.531+0000 7f6669b87700 1 -- 172.21.15.83:0/1970577361 shutdown_connections 2023-11-26T07:14:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.531+0000 7f6669b87700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f664406b080 0x7f664406d540 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.531+0000 7f6669b87700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f666415cce0 0x7f666415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.531+0000 7f6669b87700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6650003a30 0x7f66641f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.531+0000 7f6669b87700 1 --2- 172.21.15.83:0/1970577361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66640b4ca0 0x7f66640b5120 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:53.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.531+0000 7f6669b87700 1 -- 172.21.15.83:0/1970577361 >> 172.21.15.83:0/1970577361 conn(0x7f666406dc80 msgr2=0x7f66640ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:53.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.531+0000 7f6669b87700 1 -- 172.21.15.83:0/1970577361 shutdown_connections 2023-11-26T07:14:53.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.532+0000 7f6669b87700 1 -- 172.21.15.83:0/1970577361 wait complete. 2023-11-26T07:14:53.535 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2023-11-26T07:14:53.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:14:53.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.680+0000 7f6bbbd12700 1 Processor -- start 2023-11-26T07:14:53.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.680+0000 7f6bbbd12700 1 -- start start 2023-11-26T07:14:53.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bbbd12700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6bb415e160 0x7f6bb40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:53.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bbbd12700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40746a0 0x7f6bb406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:53.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bbbd12700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb40b3c60 con 0x7f6bb40afc90 2023-11-26T07:14:53.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bbbd12700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb40b4500 con 0x7f6bb415e160 2023-11-26T07:14:53.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bbbd12700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb40b58c0 con 0x7f6bb40746a0 2023-11-26T07:14:53.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bb9aae700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6bb415e160 0x7f6bb40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:53.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bb92ad700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6bb40afc90 0x7f6bb4073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58342/0 (socket says 172.21.15.83:58342) 2023-11-26T07:14:53.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bb9aae700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6bb415e160 0x7f6bb40b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38300/0 (socket says 172.21.15.83:38300) 2023-11-26T07:14:53.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bb92ad700 1 -- 172.21.15.83:0/1299210305 learned_addr learned my addr 172.21.15.83:0/1299210305 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:53.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bb9aae700 1 -- 172.21.15.83:0/1299210305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40746a0 msgr2=0x7f6bb406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bb9aae700 1 --2- 172.21.15.83:0/1299210305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40746a0 0x7f6bb406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bb9aae700 1 -- 172.21.15.83:0/1299210305 >> v1:172.21.15.83:6789/0 conn(0x7f6bb40afc90 legacy=0x7f6bb4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bb8aac700 1 -- 172.21.15.83:0/1299210305 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383627861 0 0) 0x7f6bb40b3c60 con 0x7f6bb40afc90 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.681+0000 7f6bb9aae700 1 -- 172.21.15.83:0/1299210305 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb40b0140 con 0x7f6bb415e160 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bb9aae700 1 --2- 172.21.15.83:0/1299210305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb415e160 0x7f6bb40b5da0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009840 tx=0x7f6ba4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851b8edb9414756 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bb8aac700 1 -- 172.21.15.83:0/1299210305 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4011830 con 0x7f6bb415e160 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bb8aac700 1 -- 172.21.15.83:0/1299210305 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba4011990 con 0x7f6bb415e160 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bb8aac700 1 -- 172.21.15.83:0/1299210305 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4011d20 con 0x7f6bb415e160 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bbbd12700 1 -- 172.21.15.83:0/1299210305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb415e160 msgr2=0x7f6bb40b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bbbd12700 1 --2- 172.21.15.83:0/1299210305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb415e160 0x7f6bb40b5da0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009840 tx=0x7f6ba4009b50 comp rx=0 tx=0).stop 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bbbd12700 1 -- 172.21.15.83:0/1299210305 shutdown_connections 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bbbd12700 1 --2- 172.21.15.83:0/1299210305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb40746a0 0x7f6bb406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:53.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bbbd12700 1 --2- 172.21.15.83:0/1299210305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb415e160 0x7f6bb40b5da0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bbbd12700 1 -- 172.21.15.83:0/1299210305 >> 172.21.15.83:0/1299210305 conn(0x7f6bb40d0bf0 msgr2=0x7f6bb40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bbbd12700 1 -- 172.21.15.83:0/1299210305 shutdown_connections 2023-11-26T07:14:53.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.682+0000 7f6bbbd12700 1 -- 172.21.15.83:0/1299210305 wait complete. 2023-11-26T07:14:53.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.684+0000 7f6bbbd12700 1 Processor -- start 2023-11-26T07:14:53.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.684+0000 7f6bbbd12700 1 -- start start 2023-11-26T07:14:53.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.684+0000 7f6bbbd12700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb40746a0 0x7f6bb415e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:53.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.684+0000 7f6bbbd12700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb40afc90 0x7f6bb41601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:53.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.684+0000 7f6bbbd12700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb415e160 0x7f6bb41f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.684+0000 7f6bbbd12700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb41520a0 con 0x7f6bb40746a0 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.684+0000 7f6bbbd12700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb415f100 con 0x7f6bb40afc90 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.684+0000 7f6bbbd12700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb415f3e0 con 0x7f6bb415e160 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.684+0000 7f6bb9aae700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb40746a0 0x7f6bb415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.684+0000 7f6bb9aae700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb40746a0 0x7f6bb415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38306/0 (socket says 172.21.15.83:38306) 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.684+0000 7f6bb9aae700 1 -- 172.21.15.83:0/3858971956 learned_addr learned my addr 172.21.15.83:0/3858971956 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6bba2af700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb415e160 0x7f6bb41f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6bb92ad700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb40afc90 0x7f6bb41601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6bb9aae700 1 -- 172.21.15.83:0/3858971956 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb415e160 msgr2=0x7f6bb41f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6bb9aae700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb415e160 0x7f6bb41f3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6bb9aae700 1 -- 172.21.15.83:0/3858971956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb40afc90 msgr2=0x7f6bb41601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:53.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6bb9aae700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb40afc90 0x7f6bb41601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6bb9aae700 1 -- 172.21.15.83:0/3858971956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba40094a0 con 0x7f6bb40746a0 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6bb9aae700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb40746a0 0x7f6bb415e720 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4014040 tx=0x7f6ba4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6baa7fc700 1 -- 172.21.15.83:0/3858971956 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40045f0 con 0x7f6bb40746a0 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6baa7fc700 1 -- 172.21.15.83:0/3858971956 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba401e070 con 0x7f6bb40746a0 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6bbbd12700 1 -- 172.21.15.83:0/3858971956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb415f6c0 con 0x7f6bb40746a0 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6baa7fc700 1 -- 172.21.15.83:0/3858971956 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4003b00 con 0x7f6bb40746a0 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.685+0000 7f6bbbd12700 1 -- 172.21.15.83:0/3858971956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb415fc80 con 0x7f6bb40746a0 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.686+0000 7f6bbbd12700 1 -- 172.21.15.83:0/3858971956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb415fe10 con 0x7f6bb40746a0 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.687+0000 7f6baa7fc700 1 -- 172.21.15.83:0/3858971956 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6ba4003c60 con 0x7f6bb40746a0 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.687+0000 7f6baa7fc700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b9806b080 0x7f6b9806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.687+0000 7f6baa7fc700 1 -- 172.21.15.83:0/3858971956 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6ba408a880 con 0x7f6bb40746a0 2023-11-26T07:14:53.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.687+0000 7f6bb92ad700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b9806b080 0x7f6b9806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:53.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.687+0000 7f6bb92ad700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b9806b080 0x7f6b9806d540 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40aeee0 tx=0x7f6bb0007be0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:53.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.692+0000 7f6baa7fc700 1 -- 172.21.15.83:0/3858971956 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ba4055920 con 0x7f6bb40746a0 2023-11-26T07:14:53.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:53.884+0000 7f6bbbd12700 1 -- 172.21.15.83:0/3858971956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6bb4068fb0 con 0x7f6bb40746a0 2023-11-26T07:14:54.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.534+0000 7f6baa7fc700 1 -- 172.21.15.83:0/3858971956 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6ba4020070 con 0x7f6bb40746a0 2023-11-26T07:14:54.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.537+0000 7f6bbbd12700 1 -- 172.21.15.83:0/3858971956 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b9806b080 msgr2=0x7f6b9806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:54.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.537+0000 7f6bbbd12700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b9806b080 0x7f6b9806d540 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40aeee0 tx=0x7f6bb0007be0 comp rx=0 tx=0).stop 2023-11-26T07:14:54.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.537+0000 7f6bbbd12700 1 -- 172.21.15.83:0/3858971956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb40746a0 msgr2=0x7f6bb415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:54.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.537+0000 7f6bbbd12700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb40746a0 0x7f6bb415e720 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4014040 tx=0x7f6ba4004750 comp rx=0 tx=0).stop 2023-11-26T07:14:54.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.538+0000 7f6bbbd12700 1 -- 172.21.15.83:0/3858971956 shutdown_connections 2023-11-26T07:14:54.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.538+0000 7f6bbbd12700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f6b9806b080 0x7f6b9806d540 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:54.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.538+0000 7f6bbbd12700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bb415e160 0x7f6bb41f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:54.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.538+0000 7f6bbbd12700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bb40746a0 0x7f6bb415e720 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:54.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.538+0000 7f6bbbd12700 1 --2- 172.21.15.83:0/3858971956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb40afc90 0x7f6bb41601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:54.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.538+0000 7f6bbbd12700 1 -- 172.21.15.83:0/3858971956 >> 172.21.15.83:0/3858971956 conn(0x7f6bb40d0bf0 msgr2=0x7f6bb415bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:54.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.538+0000 7f6bbbd12700 1 -- 172.21.15.83:0/3858971956 shutdown_connections 2023-11-26T07:14:54.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.538+0000 7f6bbbd12700 1 -- 172.21.15.83:0/3858971956 wait complete. 2023-11-26T07:14:54.547 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2023-11-26T07:14:54.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:14:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.687+0000 7eff9b419700 1 Processor -- start 2023-11-26T07:14:54.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.687+0000 7eff9b419700 1 -- start start 2023-11-26T07:14:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff9b419700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff940b5da0 0x7eff940afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff9b419700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff940b3c60 con 0x7eff940746a0 2023-11-26T07:14:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff9b419700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff940b58c0 con 0x7eff9415e160 2023-11-26T07:14:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff9b419700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff940b4500 con 0x7eff940b5da0 2023-11-26T07:14:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff999b6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff940b5da0 0x7eff940afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff989b4700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7eff940746a0 0x7eff9406d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58354/0 (socket says 172.21.15.83:58354) 2023-11-26T07:14:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff999b6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff940b5da0 0x7eff940afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38316/0 (socket says 172.21.15.83:38316) 2023-11-26T07:14:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff989b4700 1 -- 172.21.15.83:0/1772214449 learned_addr learned my addr 172.21.15.83:0/1772214449 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff999b6700 1 -- 172.21.15.83:0/1772214449 >> v1:172.21.15.203:6789/0 conn(0x7eff9415e160 legacy=0x7eff94073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff999b6700 1 -- 172.21.15.83:0/1772214449 >> v1:172.21.15.83:6789/0 conn(0x7eff940746a0 legacy=0x7eff9406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:54.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff999b6700 1 -- 172.21.15.83:0/1772214449 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9406bdc0 con 0x7eff940b5da0 2023-11-26T07:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff999b6700 1 --2- 172.21.15.83:0/1772214449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff940b5da0 0x7eff940afc90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009d90 tx=0x7eff8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a56d2d8caffc376 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff8bfff700 1 -- 172.21.15.83:0/1772214449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00e070 con 0x7eff940b5da0 2023-11-26T07:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff8bfff700 1 -- 172.21.15.83:0/1772214449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c00bb90 con 0x7eff940b5da0 2023-11-26T07:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.688+0000 7eff8bfff700 1 -- 172.21.15.83:0/1772214449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c003a20 con 0x7eff940b5da0 2023-11-26T07:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.689+0000 7eff9b419700 1 -- 172.21.15.83:0/1772214449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff940b5da0 msgr2=0x7eff940afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.689+0000 7eff9b419700 1 --2- 172.21.15.83:0/1772214449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff940b5da0 0x7eff940afc90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009d90 tx=0x7eff8c00b330 comp rx=0 tx=0).stop 2023-11-26T07:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.689+0000 7eff9b419700 1 -- 172.21.15.83:0/1772214449 shutdown_connections 2023-11-26T07:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.689+0000 7eff9b419700 1 --2- 172.21.15.83:0/1772214449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff940b5da0 0x7eff940afc90 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.689+0000 7eff9b419700 1 -- 172.21.15.83:0/1772214449 >> 172.21.15.83:0/1772214449 conn(0x7eff940d0bf0 msgr2=0x7eff940d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.689+0000 7eff9b419700 1 -- 172.21.15.83:0/1772214449 shutdown_connections 2023-11-26T07:14:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.689+0000 7eff9b419700 1 -- 172.21.15.83:0/1772214449 wait complete. 2023-11-26T07:14:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.691+0000 7eff9b419700 1 Processor -- start 2023-11-26T07:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.691+0000 7eff9b419700 1 -- start start 2023-11-26T07:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff9b419700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff940746a0 0x7eff94148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff9b419700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff940b5da0 0x7eff9415f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff9b419700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9415e160 0x7eff9415fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff9b419700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff94160ad0 con 0x7eff9415e160 2023-11-26T07:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff9b419700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff941e7d80 con 0x7eff940746a0 2023-11-26T07:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff9b419700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff94160230 con 0x7eff940b5da0 2023-11-26T07:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff999b6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9415e160 0x7eff9415fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff999b6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9415e160 0x7eff9415fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38322/0 (socket says 172.21.15.83:38322) 2023-11-26T07:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff999b6700 1 -- 172.21.15.83:0/336862392 learned_addr learned my addr 172.21.15.83:0/336862392 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff991b5700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff940746a0 0x7eff94148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff989b4700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff940b5da0 0x7eff9415f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff999b6700 1 -- 172.21.15.83:0/336862392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff940b5da0 msgr2=0x7eff9415f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff999b6700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff940b5da0 0x7eff9415f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff999b6700 1 -- 172.21.15.83:0/336862392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff940746a0 msgr2=0x7eff94148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff999b6700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff940746a0 0x7eff94148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff999b6700 1 -- 172.21.15.83:0/336862392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c009a70 con 0x7eff9415e160 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff999b6700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9415e160 0x7eff9415fbd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7eff94072ed0 tx=0x7eff8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff89ffb700 1 -- 172.21.15.83:0/336862392 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c012630 con 0x7eff9415e160 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff89ffb700 1 -- 172.21.15.83:0/336862392 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c0127b0 con 0x7eff9415e160 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff9b419700 1 -- 172.21.15.83:0/336862392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff94160510 con 0x7eff9415e160 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.692+0000 7eff89ffb700 1 -- 172.21.15.83:0/336862392 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c012b60 con 0x7eff9415e160 2023-11-26T07:14:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.693+0000 7eff9b419700 1 -- 172.21.15.83:0/336862392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff94149840 con 0x7eff9415e160 2023-11-26T07:14:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.693+0000 7eff9b419700 1 -- 172.21.15.83:0/336862392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff941e7b50 con 0x7eff9415e160 2023-11-26T07:14:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.694+0000 7eff89ffb700 1 -- 172.21.15.83:0/336862392 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7eff8c012cc0 con 0x7eff9415e160 2023-11-26T07:14:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.695+0000 7eff89ffb700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff7806b080 0x7eff7806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.695+0000 7eff89ffb700 1 -- 172.21.15.83:0/336862392 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5609+0+0 (secure 0 0 0) 0x7eff8c08c160 con 0x7eff9415e160 2023-11-26T07:14:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.695+0000 7eff991b5700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff7806b080 0x7eff7806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.695+0000 7eff991b5700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff7806b080 0x7eff7806d540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7eff94159e00 tx=0x7eff90009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.699+0000 7eff89ffb700 1 -- 172.21.15.83:0/336862392 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff8c057170 con 0x7eff9415e160 2023-11-26T07:14:54.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:54.934+0000 7eff9b419700 1 -- 172.21.15.83:0/336862392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7eff941e7b50 con 0x7eff9415e160 2023-11-26T07:14:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.535+0000 7eff89ffb700 1 -- 172.21.15.83:0/336862392 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v33) v1 ==== 172+0+0 (secure 0 0 0) 0x7eff941e7b50 con 0x7eff9415e160 2023-11-26T07:14:55.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 -- 172.21.15.83:0/336862392 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff7806b080 msgr2=0x7eff7806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:55.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff7806b080 0x7eff7806d540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7eff94159e00 tx=0x7eff90009040 comp rx=0 tx=0).stop 2023-11-26T07:14:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 -- 172.21.15.83:0/336862392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9415e160 msgr2=0x7eff9415fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9415e160 0x7eff9415fbd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7eff94072ed0 tx=0x7eff8c004ad0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 -- 172.21.15.83:0/336862392 shutdown_connections 2023-11-26T07:14:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff9415e160 0x7eff9415fbd0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff940746a0 0x7eff94148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff7806b080 0x7eff7806d540 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 --2- 172.21.15.83:0/336862392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff940b5da0 0x7eff9415f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 -- 172.21.15.83:0/336862392 >> 172.21.15.83:0/336862392 conn(0x7eff940d0bf0 msgr2=0x7eff940d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 -- 172.21.15.83:0/336862392 shutdown_connections 2023-11-26T07:14:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.539+0000 7eff9b419700 1 -- 172.21.15.83:0/336862392 wait complete. 2023-11-26T07:14:55.543 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 8 and data pool 9 2023-11-26T07:14:55.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:14:55.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.691+0000 7f47e560a700 1 Processor -- start 2023-11-26T07:14:55.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.691+0000 7f47e560a700 1 -- start start 2023-11-26T07:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47e560a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f47e0145760 0x7f47e006e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47e560a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e015ffb0 0x7f47e00cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47e560a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e006ee70 con 0x7f47e006df20 2023-11-26T07:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47e560a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e006d4d0 con 0x7f47e0145760 2023-11-26T07:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47e560a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e00cc5e0 con 0x7f47e015ffb0 2023-11-26T07:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47de7fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f47e006df20 0x7f47e015f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58378/0 (socket says 172.21.15.83:58378) 2023-11-26T07:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47de7fc700 1 -- 172.21.15.83:0/1113675008 learned_addr learned my addr 172.21.15.83:0/1113675008 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508720481 0 0) 0x7f47e006ee70 con 0x7f47e006df20 2023-11-26T07:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47df7fe700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e015ffb0 0x7f47e00cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c80035b0 con 0x7f47e006df20 2023-11-26T07:14:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47deffd700 1 --2- 172.21.15.83:0/1113675008 >> v2:172.21.15.203:3300/0 conn(0x7f47e0145760 0x7f47e006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1708633635 0 0) 0x7f47c80035b0 con 0x7f47e006df20 2023-11-26T07:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e015ffb0 msgr2=0x7f47e00cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e015ffb0 0x7f47e00cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 >> v2:172.21.15.203:3300/0 conn(0x7f47e0145760 msgr2=0x7f47e006e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 --2- 172.21.15.83:0/1113675008 >> v2:172.21.15.203:3300/0 conn(0x7f47e0145760 0x7f47e006e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e00cc8c0 con 0x7f47e006df20 2023-11-26T07:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f47d0001670 con 0x7f47e006df20 2023-11-26T07:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 >> v1:172.21.15.83:6789/0 conn(0x7f47e006df20 legacy=0x7f47e015f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e015ffb0 0x7f47c8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47c8003f80 0x7f47c8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.692+0000 7f47ddffb700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c8004d90 0x7f47c8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e006d4d0 con 0x7f47c8004d90 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e00cc5e0 con 0x7f47c8003f80 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c800ba10 con 0x7f47e015ffb0 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47df7fe700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c8004d90 0x7f47c8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47deffd700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e015ffb0 0x7f47c8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47de7fc700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47c8003f80 0x7f47c8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47df7fe700 1 -- 172.21.15.83:0/1113675008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e015ffb0 msgr2=0x7f47c8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47df7fe700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e015ffb0 0x7f47c8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47df7fe700 1 -- 172.21.15.83:0/1113675008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47c8003f80 msgr2=0x7f47c8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47df7fe700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47c8003f80 0x7f47c8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47df7fe700 1 -- 172.21.15.83:0/1113675008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c80035b0 con 0x7f47c8004d90 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47df7fe700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c8004d90 0x7f47c8005210 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009fc0 tx=0x7f47c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b844cae8782aff server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d800ebd0 con 0x7f47c8004d90 2023-11-26T07:14:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d800f040 con 0x7f47c8004d90 2023-11-26T07:14:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.693+0000 7f47ddffb700 1 -- 172.21.15.83:0/1113675008 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8014480 con 0x7f47c8004d90 2023-11-26T07:14:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.694+0000 7f47e560a700 1 -- 172.21.15.83:0/1113675008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c8004d90 msgr2=0x7f47c8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.694+0000 7f47e560a700 1 --2- 172.21.15.83:0/1113675008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47c8004d90 0x7f47c8005210 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009fc0 tx=0x7f47c8005880 comp rx=0 tx=0).stop 2023-11-26T07:14:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.694+0000 7f47df7fe700 1 -- 172.21.15.83:0/1113675008 reap_dead start 2023-11-26T07:14:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.694+0000 7f47e560a700 1 -- 172.21.15.83:0/1113675008 shutdown_connections 2023-11-26T07:14:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.694+0000 7f47e560a700 1 -- 172.21.15.83:0/1113675008 >> 172.21.15.83:0/1113675008 conn(0x7f47e00b1740 msgr2=0x7f47e00b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.694+0000 7f47e560a700 1 -- 172.21.15.83:0/1113675008 shutdown_connections 2023-11-26T07:14:55.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.694+0000 7f47e560a700 1 -- 172.21.15.83:0/1113675008 wait complete. 2023-11-26T07:14:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.695+0000 7f47e560a700 1 Processor -- start 2023-11-26T07:14:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.695+0000 7f47e560a700 1 -- start start 2023-11-26T07:14:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47e560a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e00abc60 0x7f47e012efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47e560a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e00ac040 0x7f47e00ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47e560a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e00aec60 0x7f47e0143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47e560a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0142df0 con 0x7f47e00abc60 2023-11-26T07:14:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47e560a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e013f920 con 0x7f47e00ac040 2023-11-26T07:14:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47e560a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e00af0e0 con 0x7f47e00aec60 2023-11-26T07:14:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47deffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e00abc60 0x7f47e012efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47deffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e00abc60 0x7f47e012efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38350/0 (socket says 172.21.15.83:38350) 2023-11-26T07:14:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47deffd700 1 -- 172.21.15.83:0/3617099018 learned_addr learned my addr 172.21.15.83:0/3617099018 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47de7fc700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e00ac040 0x7f47e00ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47df7fe700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e00aec60 0x7f47e0143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47deffd700 1 -- 172.21.15.83:0/3617099018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e00aec60 msgr2=0x7f47e0143210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47deffd700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e00aec60 0x7f47e0143210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47deffd700 1 -- 172.21.15.83:0/3617099018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e00ac040 msgr2=0x7f47e00ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47deffd700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e00ac040 0x7f47e00ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47deffd700 1 -- 172.21.15.83:0/3617099018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e00af320 con 0x7f47e00abc60 2023-11-26T07:14:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47de7fc700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e00ac040 0x7f47e00ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47deffd700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e00abc60 0x7f47e012efb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f47d4011aa0 tx=0x7f47d4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47c77fe700 1 -- 172.21.15.83:0/3617099018 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d400f980 con 0x7f47e00abc60 2023-11-26T07:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47c77fe700 1 -- 172.21.15.83:0/3617099018 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d400fae0 con 0x7f47e00abc60 2023-11-26T07:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47e560a700 1 -- 172.21.15.83:0/3617099018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e0142450 con 0x7f47e00abc60 2023-11-26T07:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.696+0000 7f47c77fe700 1 -- 172.21.15.83:0/3617099018 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d400fe90 con 0x7f47e00abc60 2023-11-26T07:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.697+0000 7f47e560a700 1 -- 172.21.15.83:0/3617099018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e0142a10 con 0x7f47e00abc60 2023-11-26T07:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.697+0000 7f47e560a700 1 -- 172.21.15.83:0/3617099018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e00751e0 con 0x7f47e00abc60 2023-11-26T07:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.698+0000 7f47c77fe700 1 -- 172.21.15.83:0/3617099018 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f47d4014440 con 0x7f47e00abc60 2023-11-26T07:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.698+0000 7f47c77fe700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47cc06b0b0 0x7f47cc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.698+0000 7f47c77fe700 1 -- 172.21.15.83:0/3617099018 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47d408a010 con 0x7f47e00abc60 2023-11-26T07:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.699+0000 7f47de7fc700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47cc06b0b0 0x7f47cc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.699+0000 7f47de7fc700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47cc06b0b0 0x7f47cc06d570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f47d00048d0 tx=0x7f47d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.703+0000 7f47c77fe700 1 -- 172.21.15.83:0/3617099018 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47d4054ed0 con 0x7f47e00abc60 2023-11-26T07:14:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.898+0000 7f47e560a700 1 -- 172.21.15.83:0/3617099018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47e0142f30 con 0x7f47e00abc60 2023-11-26T07:14:55.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.899+0000 7f47c77fe700 1 -- 172.21.15.83:0/3617099018 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16566 (secure 0 0 0) 0x7f47e0142f30 con 0x7f47e00abc60 2023-11-26T07:14:55.902 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:55.902 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":57,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:14:54.936051+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:14:53.044716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:14:53.904013+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:14:55.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.901+0000 7f47e560a700 1 -- 172.21.15.83:0/3617099018 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47cc06b0b0 msgr2=0x7f47cc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:55.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.901+0000 7f47e560a700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47cc06b0b0 0x7f47cc06d570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f47d00048d0 tx=0x7f47d0009040 comp rx=0 tx=0).stop 2023-11-26T07:14:55.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.901+0000 7f47e560a700 1 -- 172.21.15.83:0/3617099018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e00abc60 msgr2=0x7f47e012efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:55.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.901+0000 7f47e560a700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e00abc60 0x7f47e012efb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f47d4011aa0 tx=0x7f47d4011e60 comp rx=0 tx=0).stop 2023-11-26T07:14:55.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.901+0000 7f47e560a700 1 -- 172.21.15.83:0/3617099018 shutdown_connections 2023-11-26T07:14:55.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.901+0000 7f47e560a700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f47cc06b0b0 0x7f47cc06d570 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.901+0000 7f47e560a700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e00aec60 0x7f47e0143210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.901+0000 7f47e560a700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e00abc60 0x7f47e012efb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.901+0000 7f47e560a700 1 --2- 172.21.15.83:0/3617099018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e00ac040 0x7f47e00ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:55.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.901+0000 7f47e560a700 1 -- 172.21.15.83:0/3617099018 >> 172.21.15.83:0/3617099018 conn(0x7f47e00b1740 msgr2=0x7f47e00b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:55.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.902+0000 7f47e560a700 1 -- 172.21.15.83:0/3617099018 shutdown_connections 2023-11-26T07:14:55.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:55.902+0000 7f47e560a700 1 -- 172.21.15.83:0/3617099018 wait complete. 2023-11-26T07:14:55.916 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:14:56.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.033+0000 7fae4b36c700 1 Processor -- start 2023-11-26T07:14:56.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.034+0000 7fae4b36c700 1 -- start start 2023-11-26T07:14:56.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.034+0000 7fae4b36c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae4415ddd0 0x7fae440b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.034+0000 7fae4b36c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae440d2240 0x7fae440cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.034+0000 7fae4b36c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae440b1d40 con 0x7fae4415e1b0 2023-11-26T07:14:56.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.034+0000 7fae4b36c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae440b28c0 con 0x7fae440d2240 2023-11-26T07:14:56.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.034+0000 7fae4b36c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4406c000 con 0x7fae4415ddd0 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.034+0000 7fae49909700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae440d2240 0x7fae440cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae48907700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fae4415e1b0 0x7fae440d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58398/0 (socket says 172.21.15.83:58398) 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae49909700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae440d2240 0x7fae440cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38360/0 (socket says 172.21.15.83:38360) 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae48907700 1 -- 172.21.15.83:0/1646603742 learned_addr learned my addr 172.21.15.83:0/1646603742 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae49108700 1 --2- 172.21.15.83:0/1646603742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae4415ddd0 0x7fae440b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae49909700 1 -- 172.21.15.83:0/1646603742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae4415ddd0 msgr2=0x7fae440b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae49909700 1 --2- 172.21.15.83:0/1646603742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae4415ddd0 0x7fae440b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae49909700 1 -- 172.21.15.83:0/1646603742 >> v1:172.21.15.83:6789/0 conn(0x7fae4415e1b0 legacy=0x7fae440d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae49909700 1 -- 172.21.15.83:0/1646603742 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440b1f90 con 0x7fae440d2240 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae37fff700 1 -- 172.21.15.83:0/1646603742 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180928992 0 0) 0x7fae440b1d40 con 0x7fae4415e1b0 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae49909700 1 --2- 172.21.15.83:0/1646603742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae440d2240 0x7fae440cf000 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009d90 tx=0x7fae3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215bf68fb3c1597f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:56.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae37fff700 1 -- 172.21.15.83:0/1646603742 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00e070 con 0x7fae440d2240 2023-11-26T07:14:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae37fff700 1 -- 172.21.15.83:0/1646603742 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c00bb20 con 0x7fae440d2240 2023-11-26T07:14:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.035+0000 7fae37fff700 1 -- 172.21.15.83:0/1646603742 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c012430 con 0x7fae440d2240 2023-11-26T07:14:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.036+0000 7fae4b36c700 1 -- 172.21.15.83:0/1646603742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae440d2240 msgr2=0x7fae440cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.036+0000 7fae4b36c700 1 --2- 172.21.15.83:0/1646603742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae440d2240 0x7fae440cf000 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009d90 tx=0x7fae3c00b330 comp rx=0 tx=0).stop 2023-11-26T07:14:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.036+0000 7fae4b36c700 1 -- 172.21.15.83:0/1646603742 shutdown_connections 2023-11-26T07:14:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.036+0000 7fae4b36c700 1 --2- 172.21.15.83:0/1646603742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae440d2240 0x7fae440cf000 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.036+0000 7fae4b36c700 1 --2- 172.21.15.83:0/1646603742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae4415ddd0 0x7fae440b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.036+0000 7fae4b36c700 1 -- 172.21.15.83:0/1646603742 >> 172.21.15.83:0/1646603742 conn(0x7fae440731a0 msgr2=0x7fae4409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.036+0000 7fae4b36c700 1 -- 172.21.15.83:0/1646603742 shutdown_connections 2023-11-26T07:14:56.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.036+0000 7fae4b36c700 1 -- 172.21.15.83:0/1646603742 wait complete. 2023-11-26T07:14:56.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.037+0000 7fae4b36c700 1 Processor -- start 2023-11-26T07:14:56.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.037+0000 7fae4b36c700 1 -- start start 2023-11-26T07:14:56.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae4b36c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae440d2240 0x7fae4409c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae4b36c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae4415ddd0 0x7fae440b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae4b36c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae4415e1b0 0x7fae440b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae4b36c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4409cb70 con 0x7fae4415ddd0 2023-11-26T07:14:56.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae4b36c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440b5640 con 0x7fae440d2240 2023-11-26T07:14:56.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae4b36c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae440b5920 con 0x7fae4415e1b0 2023-11-26T07:14:56.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae48907700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae4415ddd0 0x7fae440b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae48907700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae4415ddd0 0x7fae440b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38364/0 (socket says 172.21.15.83:38364) 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae48907700 1 -- 172.21.15.83:0/1052796823 learned_addr learned my addr 172.21.15.83:0/1052796823 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae49108700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae440d2240 0x7fae4409c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae49909700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae4415e1b0 0x7fae440b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae48907700 1 -- 172.21.15.83:0/1052796823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae4415e1b0 msgr2=0x7fae440b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae48907700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae4415e1b0 0x7fae440b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae48907700 1 -- 172.21.15.83:0/1052796823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae440d2240 msgr2=0x7fae4409c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae48907700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae440d2240 0x7fae4409c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae48907700 1 -- 172.21.15.83:0/1052796823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c009a70 con 0x7fae4415ddd0 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae48907700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae4415ddd0 0x7fae440b3a90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fae400011e0 tx=0x7fae4000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae35ffb700 1 -- 172.21.15.83:0/1052796823 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40004c80 con 0x7fae4415ddd0 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae35ffb700 1 -- 172.21.15.83:0/1052796823 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae40004e00 con 0x7fae4415ddd0 2023-11-26T07:14:56.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae4b36c700 1 -- 172.21.15.83:0/1052796823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae440b2ed0 con 0x7fae4415ddd0 2023-11-26T07:14:56.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.038+0000 7fae35ffb700 1 -- 172.21.15.83:0/1052796823 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40007310 con 0x7fae4415ddd0 2023-11-26T07:14:56.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.039+0000 7fae4b36c700 1 -- 172.21.15.83:0/1052796823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae440b5ee0 con 0x7fae4415ddd0 2023-11-26T07:14:56.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.040+0000 7fae4b36c700 1 -- 172.21.15.83:0/1052796823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae44067f60 con 0x7fae4415ddd0 2023-11-26T07:14:56.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.040+0000 7fae35ffb700 1 -- 172.21.15.83:0/1052796823 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fae40007470 con 0x7fae4415ddd0 2023-11-26T07:14:56.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.040+0000 7fae35ffb700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b080 0x7fae2806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.040+0000 7fae35ffb700 1 -- 172.21.15.83:0/1052796823 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae400896a0 con 0x7fae4415ddd0 2023-11-26T07:14:56.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.040+0000 7fae49108700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b080 0x7fae2806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.041+0000 7fae49108700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b080 0x7fae2806d540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fae38009b70 tx=0x7fae38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:56.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.045+0000 7fae35ffb700 1 -- 172.21.15.83:0/1052796823 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae400544d0 con 0x7fae4415ddd0 2023-11-26T07:14:56.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.235+0000 7fae4b36c700 1 -- 172.21.15.83:0/1052796823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae440b7c60 con 0x7fae4415ddd0 2023-11-26T07:14:56.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.236+0000 7fae35ffb700 1 -- 172.21.15.83:0/1052796823 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16566 (secure 0 0 0) 0x7fae40057af0 con 0x7fae4415ddd0 2023-11-26T07:14:56.285 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:56.285 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":57,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:14:54.936051+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:14:53.044716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:14:53.904013+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:14:56.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 -- 172.21.15.83:0/1052796823 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b080 msgr2=0x7fae2806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:56.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b080 0x7fae2806d540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fae38009b70 tx=0x7fae38008040 comp rx=0 tx=0).stop 2023-11-26T07:14:56.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 -- 172.21.15.83:0/1052796823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae4415ddd0 msgr2=0x7fae440b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:56.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae4415ddd0 0x7fae440b3a90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fae400011e0 tx=0x7fae4000cae0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 -- 172.21.15.83:0/1052796823 shutdown_connections 2023-11-26T07:14:56.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b080 0x7fae2806d540 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae4415e1b0 0x7fae440b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae440d2240 0x7fae4409c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 --2- 172.21.15.83:0/1052796823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae4415ddd0 0x7fae440b3a90 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 -- 172.21.15.83:0/1052796823 >> 172.21.15.83:0/1052796823 conn(0x7fae440731a0 msgr2=0x7fae441606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:56.287 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 -- 172.21.15.83:0/1052796823 shutdown_connections 2023-11-26T07:14:56.288 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.238+0000 7fae4b36c700 1 -- 172.21.15.83:0/1052796823 wait complete. 2023-11-26T07:14:56.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:14:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.408+0000 7fe75b7ad700 1 Processor -- start 2023-11-26T07:14:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.409+0000 7fe75b7ad700 1 -- start start 2023-11-26T07:14:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.409+0000 7fe75b7ad700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7540cf610 0x7fe7540add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.409+0000 7fe75b7ad700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe75415f9e0 0x7fe7540b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe75b7ad700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe754073f90 0x7fe754074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe75b7ad700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe754074960 con 0x7fe754073f90 2023-11-26T07:14:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe75b7ad700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75406b080 con 0x7fe75415f9e0 2023-11-26T07:14:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe75b7ad700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75415c290 con 0x7fe7540cf610 2023-11-26T07:14:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe759d4a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe754073f90 0x7fe754074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe759d4a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe754073f90 0x7fe754074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38374/0 (socket says 172.21.15.83:38374) 2023-11-26T07:14:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe759d4a700 1 -- 172.21.15.83:0/1732956104 learned_addr learned my addr 172.21.15.83:0/1732956104 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe759549700 1 --2- 172.21.15.83:0/1732956104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7540cf610 0x7fe7540add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe758d48700 1 --2- 172.21.15.83:0/1732956104 >> v2:172.21.15.203:3300/0 conn(0x7fe75415f9e0 0x7fe7540b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe759d4a700 1 -- 172.21.15.83:0/1732956104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7540cf610 msgr2=0x7fe7540add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe759d4a700 1 --2- 172.21.15.83:0/1732956104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7540cf610 0x7fe7540add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe759d4a700 1 -- 172.21.15.83:0/1732956104 >> v2:172.21.15.203:3300/0 conn(0x7fe75415f9e0 msgr2=0x7fe7540b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe759d4a700 1 --2- 172.21.15.83:0/1732956104 >> v2:172.21.15.203:3300/0 conn(0x7fe75415f9e0 0x7fe7540b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe759d4a700 1 -- 172.21.15.83:0/1732956104 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7540cf9f0 con 0x7fe754073f90 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe759d4a700 1 --2- 172.21.15.83:0/1732956104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe754073f90 0x7fe754074420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe754075070 tx=0x7fe74c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fff61a36ecb536c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe74bfff700 1 -- 172.21.15.83:0/1732956104 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00e070 con 0x7fe754073f90 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe74bfff700 1 -- 172.21.15.83:0/1732956104 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c00b9c0 con 0x7fe754073f90 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.410+0000 7fe74bfff700 1 -- 172.21.15.83:0/1732956104 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00bd70 con 0x7fe754073f90 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.411+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1732956104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe754073f90 msgr2=0x7fe754074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.411+0000 7fe75b7ad700 1 --2- 172.21.15.83:0/1732956104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe754073f90 0x7fe754074420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe754075070 tx=0x7fe74c00b330 comp rx=0 tx=0).stop 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.411+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1732956104 shutdown_connections 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.411+0000 7fe75b7ad700 1 --2- 172.21.15.83:0/1732956104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe754073f90 0x7fe754074420 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.411+0000 7fe75b7ad700 1 --2- 172.21.15.83:0/1732956104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7540cf610 0x7fe7540add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.411+0000 7fe75b7ad700 1 --2- 172.21.15.83:0/1732956104 >> v2:172.21.15.203:3300/0 conn(0x7fe75415f9e0 0x7fe7540b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.411+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1732956104 >> 172.21.15.83:0/1732956104 conn(0x7fe7540d0bf0 msgr2=0x7fe7540d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.411+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1732956104 shutdown_connections 2023-11-26T07:14:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.411+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1732956104 wait complete. 2023-11-26T07:14:56.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.412+0000 7fe75b7ad700 1 Processor -- start 2023-11-26T07:14:56.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.412+0000 7fe75b7ad700 1 -- start start 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe75b7ad700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe754073f90 0x7fe75415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe75b7ad700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540cf610 0x7fe75413ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe75b7ad700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe75415f9e0 0x7fe754151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe75b7ad700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe754151a30 con 0x7fe754073f90 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe75b7ad700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe754153ab0 con 0x7fe7540cf610 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe75b7ad700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7541517b0 con 0x7fe75415f9e0 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe759549700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe754073f90 0x7fe75415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe759549700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe754073f90 0x7fe75415ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38376/0 (socket says 172.21.15.83:38376) 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe759549700 1 -- 172.21.15.83:0/1625035859 learned_addr learned my addr 172.21.15.83:0/1625035859 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe759549700 1 -- 172.21.15.83:0/1625035859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe75415f9e0 msgr2=0x7fe754151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe759549700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe75415f9e0 0x7fe754151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe758d48700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540cf610 0x7fe75413ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe759549700 1 -- 172.21.15.83:0/1625035859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540cf610 msgr2=0x7fe75413ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe759549700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540cf610 0x7fe75413ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe759549700 1 -- 172.21.15.83:0/1625035859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c009b30 con 0x7fe754073f90 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.413+0000 7fe759549700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe754073f90 0x7fe75415ef20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe74400e840 tx=0x7fe74400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.414+0000 7fe749ffb700 1 -- 172.21.15.83:0/1625035859 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744013610 con 0x7fe754073f90 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.414+0000 7fe749ffb700 1 -- 172.21.15.83:0/1625035859 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe744013770 con 0x7fe754073f90 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.414+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1625035859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe754152ee0 con 0x7fe754073f90 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.414+0000 7fe749ffb700 1 -- 172.21.15.83:0/1625035859 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744013b20 con 0x7fe754073f90 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.414+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1625035859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe754153460 con 0x7fe754073f90 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.415+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1625035859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe754068fb0 con 0x7fe754073f90 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.415+0000 7fe749ffb700 1 -- 172.21.15.83:0/1625035859 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fe744013c80 con 0x7fe754073f90 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.415+0000 7fe749ffb700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe73806b080 0x7fe73806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:56.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.416+0000 7fe749ffb700 1 -- 172.21.15.83:0/1625035859 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe744089590 con 0x7fe754073f90 2023-11-26T07:14:56.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.416+0000 7fe758d48700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe73806b080 0x7fe73806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:56.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.416+0000 7fe758d48700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe73806b080 0x7fe73806d540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe75415f460 tx=0x7fe750008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.420+0000 7fe749ffb700 1 -- 172.21.15.83:0/1625035859 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe744054340 con 0x7fe754073f90 2023-11-26T07:14:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:56.655+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1625035859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe75406c240 con 0x7fe754073f90 2023-11-26T07:14:57.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:14:57.099 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:14:57.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.574+0000 7fe749ffb700 1 -- 172.21.15.83:0/1625035859 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe744057960 con 0x7fe754073f90 2023-11-26T07:14:57.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1625035859 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe73806b080 msgr2=0x7fe73806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:57.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe73806b080 0x7fe73806d540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe75415f460 tx=0x7fe750008040 comp rx=0 tx=0).stop 2023-11-26T07:14:57.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1625035859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe754073f90 msgr2=0x7fe75415ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:57.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe754073f90 0x7fe75415ef20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe74400e840 tx=0x7fe74400ec00 comp rx=0 tx=0).stop 2023-11-26T07:14:57.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1625035859 shutdown_connections 2023-11-26T07:14:57.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe73806b080 0x7fe73806d540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:57.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe75415f9e0 0x7fe754151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:57.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe754073f90 0x7fe75415ef20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:57.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 --2- 172.21.15.83:0/1625035859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7540cf610 0x7fe75413ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:57.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1625035859 >> 172.21.15.83:0/1625035859 conn(0x7fe7540d0bf0 msgr2=0x7fe7540b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:57.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1625035859 shutdown_connections 2023-11-26T07:14:57.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.578+0000 7fe75b7ad700 1 -- 172.21.15.83:0/1625035859 wait complete. 2023-11-26T07:14:57.596 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:14:57.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:14:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.727+0000 7f1f98e3a700 1 Processor -- start 2023-11-26T07:14:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.727+0000 7f1f98e3a700 1 -- start start 2023-11-26T07:14:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.728+0000 7f1f98e3a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f940b5da0 0x7f1f940afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.728+0000 7f1f98e3a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f940b3c60 con 0x7f1f9415e160 2023-11-26T07:14:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.728+0000 7f1f98e3a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f940b58c0 con 0x7f1f940746a0 2023-11-26T07:14:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.728+0000 7f1f98e3a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f940b4500 con 0x7f1f940b5da0 2023-11-26T07:14:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.728+0000 7f1f92d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f940b5da0 0x7f1f940afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.728+0000 7f1f9259c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1f9415e160 0x7f1f94073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58424/0 (socket says 172.21.15.83:58424) 2023-11-26T07:14:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.728+0000 7f1f9259c700 1 -- 172.21.15.83:0/597615556 learned_addr learned my addr 172.21.15.83:0/597615556 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.728+0000 7f1f92d9d700 1 -- 172.21.15.83:0/597615556 >> v1:172.21.15.203:6789/0 conn(0x7f1f940746a0 legacy=0x7f1f9406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.728+0000 7f1f92d9d700 1 -- 172.21.15.83:0/597615556 >> v1:172.21.15.83:6789/0 conn(0x7f1f9415e160 legacy=0x7f1f94073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.728+0000 7f1f92d9d700 1 -- 172.21.15.83:0/597615556 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9406bdc0 con 0x7f1f940b5da0 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.728+0000 7f1f9159a700 1 -- 172.21.15.83:0/597615556 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576718424 0 0) 0x7f1f940b3c60 con 0x7f1f9415e160 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.729+0000 7f1f92d9d700 1 --2- 172.21.15.83:0/597615556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f940b5da0 0x7f1f940afc90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009d90 tx=0x7f1f8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124203898ee4b6a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.729+0000 7f1f9159a700 1 -- 172.21.15.83:0/597615556 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00e070 con 0x7f1f940b5da0 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.729+0000 7f1f9159a700 1 -- 172.21.15.83:0/597615556 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c00bb90 con 0x7f1f940b5da0 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.729+0000 7f1f9159a700 1 -- 172.21.15.83:0/597615556 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c003a20 con 0x7f1f940b5da0 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.729+0000 7f1f98e3a700 1 -- 172.21.15.83:0/597615556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f940b5da0 msgr2=0x7f1f940afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.729+0000 7f1f98e3a700 1 --2- 172.21.15.83:0/597615556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f940b5da0 0x7f1f940afc90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009d90 tx=0x7f1f8c00b330 comp rx=0 tx=0).stop 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.729+0000 7f1f98e3a700 1 -- 172.21.15.83:0/597615556 shutdown_connections 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.729+0000 7f1f98e3a700 1 --2- 172.21.15.83:0/597615556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f940b5da0 0x7f1f940afc90 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.729+0000 7f1f98e3a700 1 -- 172.21.15.83:0/597615556 >> 172.21.15.83:0/597615556 conn(0x7f1f940d0bf0 msgr2=0x7f1f940d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:57.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.729+0000 7f1f98e3a700 1 -- 172.21.15.83:0/597615556 shutdown_connections 2023-11-26T07:14:57.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.729+0000 7f1f98e3a700 1 -- 172.21.15.83:0/597615556 wait complete. 2023-11-26T07:14:57.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.731+0000 7f1f98e3a700 1 Processor -- start 2023-11-26T07:14:57.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.731+0000 7f1f98e3a700 1 -- start start 2023-11-26T07:14:57.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.731+0000 7f1f98e3a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f940746a0 0x7f1f941f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.731+0000 7f1f98e3a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f940b5da0 0x7f1f941547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.731+0000 7f1f98e3a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f9415e160 0x7f1f94154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.731+0000 7f1f98e3a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f94154310 con 0x7f1f940b5da0 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.731+0000 7f1f98e3a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f941f7b10 con 0x7f1f940746a0 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.731+0000 7f1f98e3a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f94153700 con 0x7f1f9415e160 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f91d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f940b5da0 0x7f1f941547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f91d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f940b5da0 0x7f1f941547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38392/0 (socket says 172.21.15.83:38392) 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f91d9b700 1 -- 172.21.15.83:0/443481130 learned_addr learned my addr 172.21.15.83:0/443481130 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f9259c700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f940746a0 0x7f1f941f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f92d9d700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f9415e160 0x7f1f94154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f91d9b700 1 -- 172.21.15.83:0/443481130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f9415e160 msgr2=0x7f1f94154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:57.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f91d9b700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f9415e160 0x7f1f94154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f91d9b700 1 -- 172.21.15.83:0/443481130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f940746a0 msgr2=0x7f1f941f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f91d9b700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f940746a0 0x7f1f941f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f91d9b700 1 -- 172.21.15.83:0/443481130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c009a70 con 0x7f1f940b5da0 2023-11-26T07:14:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f9259c700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f940746a0 0x7f1f941f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f91d9b700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f940b5da0 0x7f1f941547a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400b4a0 tx=0x7f1f8400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f82ffd700 1 -- 172.21.15.83:0/443481130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84011830 con 0x7f1f940b5da0 2023-11-26T07:14:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f82ffd700 1 -- 172.21.15.83:0/443481130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f84011990 con 0x7f1f940b5da0 2023-11-26T07:14:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f98e3a700 1 -- 172.21.15.83:0/443481130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f941539e0 con 0x7f1f940b5da0 2023-11-26T07:14:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f82ffd700 1 -- 172.21.15.83:0/443481130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84011d40 con 0x7f1f940b5da0 2023-11-26T07:14:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.732+0000 7f1f98e3a700 1 -- 172.21.15.83:0/443481130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f94153fa0 con 0x7f1f940b5da0 2023-11-26T07:14:57.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.733+0000 7f1f98e3a700 1 -- 172.21.15.83:0/443481130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f94068fb0 con 0x7f1f940b5da0 2023-11-26T07:14:57.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.734+0000 7f1f82ffd700 1 -- 172.21.15.83:0/443481130 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f1f8401a400 con 0x7f1f940b5da0 2023-11-26T07:14:57.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.734+0000 7f1f82ffd700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f7406b080 0x7f1f7406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:57.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.734+0000 7f1f9259c700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f7406b080 0x7f1f7406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:57.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.734+0000 7f1f82ffd700 1 -- 172.21.15.83:0/443481130 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f840895d0 con 0x7f1f940b5da0 2023-11-26T07:14:57.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.735+0000 7f1f9259c700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f7406b080 0x7f1f7406d540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1f88000f60 tx=0x7f1f88007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:57.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.739+0000 7f1f82ffd700 1 -- 172.21.15.83:0/443481130 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f84054400 con 0x7f1f940b5da0 2023-11-26T07:14:57.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:57.973+0000 7f1f98e3a700 1 -- 172.21.15.83:0/443481130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1f9406c140 con 0x7f1f940b5da0 2023-11-26T07:14:58.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.580+0000 7f1f82ffd700 1 -- 172.21.15.83:0/443481130 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f84057a20 con 0x7f1f940b5da0 2023-11-26T07:14:58.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.582+0000 7f1f98e3a700 1 -- 172.21.15.83:0/443481130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f7406b080 msgr2=0x7f1f7406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:58.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.582+0000 7f1f98e3a700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f7406b080 0x7f1f7406d540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1f88000f60 tx=0x7f1f88007ca0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.582+0000 7f1f98e3a700 1 -- 172.21.15.83:0/443481130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f940b5da0 msgr2=0x7f1f941547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:58.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.582+0000 7f1f98e3a700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f940b5da0 0x7f1f941547a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400b4a0 tx=0x7f1f8400b7b0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.582+0000 7f1f98e3a700 1 -- 172.21.15.83:0/443481130 shutdown_connections 2023-11-26T07:14:58.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.582+0000 7f1f98e3a700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f1f7406b080 0x7f1f7406d540 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.582+0000 7f1f98e3a700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f9415e160 0x7f1f94154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.582+0000 7f1f98e3a700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f940746a0 0x7f1f941f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.582+0000 7f1f98e3a700 1 --2- 172.21.15.83:0/443481130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f940b5da0 0x7f1f941547a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.582+0000 7f1f98e3a700 1 -- 172.21.15.83:0/443481130 >> 172.21.15.83:0/443481130 conn(0x7f1f940d0bf0 msgr2=0x7f1f94161ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:58.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.583+0000 7f1f98e3a700 1 -- 172.21.15.83:0/443481130 shutdown_connections 2023-11-26T07:14:58.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.583+0000 7f1f98e3a700 1 -- 172.21.15.83:0/443481130 wait complete. 2023-11-26T07:14:58.598 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:14:58.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:14:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.721+0000 7f037c4d4700 1 Processor -- start 2023-11-26T07:14:58.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.721+0000 7f037c4d4700 1 -- start start 2023-11-26T07:14:58.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037c4d4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f037415e1b0 0x7f03740b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:58.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037c4d4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03740b1d40 con 0x7f037415ddd0 2023-11-26T07:14:58.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037c4d4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03740b28c0 con 0x7f03740d2240 2023-11-26T07:14:58.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037c4d4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03740b1f90 con 0x7f037415e1b0 2023-11-26T07:14:58.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037a270700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f037415ddd0 0x7f03740d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58440/0 (socket says 172.21.15.83:58440) 2023-11-26T07:14:58.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037a270700 1 -- 172.21.15.83:0/285790343 learned_addr learned my addr 172.21.15.83:0/285790343 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:14:58.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782926433 0 0) 0x7f03740b1d40 con 0x7f037415ddd0 2023-11-26T07:14:58.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03600035b0 con 0x7f037415ddd0 2023-11-26T07:14:58.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037aa71700 1 --2- 172.21.15.83:0/285790343 >> v2:172.21.15.203:3300/0 conn(0x7f037415e1b0 0x7f03740b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:58.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 238871714 0 0) 0x7f03600035b0 con 0x7f037415ddd0 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 >> v2:172.21.15.203:3300/0 conn(0x7f037415e1b0 msgr2=0x7f03740b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 --2- 172.21.15.83:0/285790343 >> v2:172.21.15.203:3300/0 conn(0x7f037415e1b0 0x7f03740b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 >> v1:172.21.15.203:6789/0 conn(0x7f03740d2240 legacy=0x7f037406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03740b2270 con 0x7f037415ddd0 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f0364001670 con 0x7f037415ddd0 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 >> v1:172.21.15.83:6789/0 conn(0x7f037415ddd0 legacy=0x7f03740d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0360003a00 0x7f0360003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0360004850 0x7f0360004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0360005330 0x7f036000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03740b1f90 con 0x7f0360005330 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036000c200 con 0x7f0360003a00 2023-11-26T07:14:58.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036000c3c0 con 0x7f0360004850 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.722+0000 7f037aa71700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0360005330 0x7f036000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037a270700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0360003a00 0x7f0360003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f0379a6f700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0360004850 0x7f0360004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037aa71700 1 -- 172.21.15.83:0/285790343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0360004850 msgr2=0x7f0360004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037aa71700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0360004850 0x7f0360004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037aa71700 1 -- 172.21.15.83:0/285790343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0360003a00 msgr2=0x7f0360003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037aa71700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0360003a00 0x7f0360003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037aa71700 1 -- 172.21.15.83:0/285790343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03600035b0 con 0x7f0360005330 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f0379a6f700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0360004850 0x7f0360004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037aa71700 1 -- 172.21.15.83:0/285790343 reap_dead start 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037aa71700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0360005330 0x7f036000ba10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f036c009f90 tx=0x7f0360005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5876f1366fd1fe98 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:58.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c003a20 con 0x7f0360005330 2023-11-26T07:14:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036c00f040 con 0x7f0360005330 2023-11-26T07:14:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037926e700 1 -- 172.21.15.83:0/285790343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c003dc0 con 0x7f0360005330 2023-11-26T07:14:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037c4d4700 1 -- 172.21.15.83:0/285790343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0360005330 msgr2=0x7f036000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.723+0000 7f037c4d4700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0360005330 0x7f036000ba10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f036c009f90 tx=0x7f0360005880 comp rx=0 tx=0).stop 2023-11-26T07:14:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.724+0000 7f037c4d4700 1 -- 172.21.15.83:0/285790343 shutdown_connections 2023-11-26T07:14:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.724+0000 7f037c4d4700 1 --2- 172.21.15.83:0/285790343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0360005330 0x7f036000ba10 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.724+0000 7f037c4d4700 1 -- 172.21.15.83:0/285790343 >> 172.21.15.83:0/285790343 conn(0x7f03740731a0 msgr2=0x7f037409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.724+0000 7f037c4d4700 1 -- 172.21.15.83:0/285790343 shutdown_connections 2023-11-26T07:14:58.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.724+0000 7f037c4d4700 1 -- 172.21.15.83:0/285790343 wait complete. 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.725+0000 7f037c4d4700 1 Processor -- start 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.725+0000 7f037c4d4700 1 -- start start 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f037c4d4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0360005330 0x7f037406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f037c4d4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037409bc00 0x7f037414cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f037c4d4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f037409c080 0x7f037414d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f037c4d4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037409c570 con 0x7f037409bc00 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f037c4d4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037409c850 con 0x7f037409c080 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f037c4d4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037409cb30 con 0x7f0360005330 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f0379a6f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037409bc00 0x7f037414cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f0379a6f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037409bc00 0x7f037414cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38412/0 (socket says 172.21.15.83:38412) 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f0379a6f700 1 -- 172.21.15.83:0/960223132 learned_addr learned my addr 172.21.15.83:0/960223132 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f037a270700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0360005330 0x7f037406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f037aa71700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f037409c080 0x7f037414d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f0379a6f700 1 -- 172.21.15.83:0/960223132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0360005330 msgr2=0x7f037406ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f0379a6f700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0360005330 0x7f037406ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f0379a6f700 1 -- 172.21.15.83:0/960223132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f037409c080 msgr2=0x7f037414d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f0379a6f700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f037409c080 0x7f037414d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f0379a6f700 1 -- 172.21.15.83:0/960223132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03740b2ed0 con 0x7f037409bc00 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f0379a6f700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037409bc00 0x7f037414cf40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f037000e550 tx=0x7f037000e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f036affd700 1 -- 172.21.15.83:0/960223132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037000ee80 con 0x7f037409bc00 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f036affd700 1 -- 172.21.15.83:0/960223132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03700052e0 con 0x7f037409bc00 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f037c4d4700 1 -- 172.21.15.83:0/960223132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037414ad20 con 0x7f037409bc00 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f036affd700 1 -- 172.21.15.83:0/960223132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370003680 con 0x7f037409bc00 2023-11-26T07:14:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.726+0000 7f037c4d4700 1 -- 172.21.15.83:0/960223132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03740704c0 con 0x7f037409bc00 2023-11-26T07:14:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.727+0000 7f037c4d4700 1 -- 172.21.15.83:0/960223132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0374067f60 con 0x7f037409bc00 2023-11-26T07:14:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.728+0000 7f036affd700 1 -- 172.21.15.83:0/960223132 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f03700037e0 con 0x7f037409bc00 2023-11-26T07:14:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.728+0000 7f036affd700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f035806b080 0x7f035806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.728+0000 7f036affd700 1 -- 172.21.15.83:0/960223132 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0370088e00 con 0x7f037409bc00 2023-11-26T07:14:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.728+0000 7f037a270700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f035806b080 0x7f035806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.728+0000 7f037a270700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f035806b080 0x7f035806d540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0364007cd0 tx=0x7f0364000f80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.732+0000 7f036affd700 1 -- 172.21.15.83:0/960223132 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0370053ce0 con 0x7f037409bc00 2023-11-26T07:14:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.966+0000 7f037c4d4700 1 -- 172.21.15.83:0/960223132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03740b7c60 con 0x7f037409bc00 2023-11-26T07:14:58.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.967+0000 7f036affd700 1 -- 172.21.15.83:0/960223132 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f0370019020 con 0x7f037409bc00 2023-11-26T07:14:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:58.968 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/126258309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":126258309},{"type":"v1","addr":"172.21.15.203:6834","nonce":126258309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4203521524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":4203521524},{"type":"v1","addr":"172.21.15.203:6835","nonce":4203521524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1267615204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1267615204},{"type":"v1","addr":"172.21.15.83:6836","nonce":1267615204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:14:54.936357+0000","modified":"2023-11-26T07:14:57.974394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6102":{"gid":6102,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/83108157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":83108157},{"type":"v1","addr":"172.21.15.83:6837","nonce":83108157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:14:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 -- 172.21.15.83:0/960223132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f035806b080 msgr2=0x7f035806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f035806b080 0x7f035806d540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0364007cd0 tx=0x7f0364000f80 comp rx=0 tx=0).stop 2023-11-26T07:14:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 -- 172.21.15.83:0/960223132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037409bc00 msgr2=0x7f037414cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037409bc00 0x7f037414cf40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f037000e550 tx=0x7f037000e860 comp rx=0 tx=0).stop 2023-11-26T07:14:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 -- 172.21.15.83:0/960223132 shutdown_connections 2023-11-26T07:14:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f037409c080 0x7f037414d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f035806b080 0x7f035806d540 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0360005330 0x7f037406ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 --2- 172.21.15.83:0/960223132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f037409bc00 0x7f037414cf40 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 -- 172.21.15.83:0/960223132 >> 172.21.15.83:0/960223132 conn(0x7f03740731a0 msgr2=0x7f03740af250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 -- 172.21.15.83:0/960223132 shutdown_connections 2023-11-26T07:14:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:58.969+0000 7f037c4d4700 1 -- 172.21.15.83:0/960223132 wait complete. 2023-11-26T07:14:58.972 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 37 2023-11-26T07:14:58.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:14:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.101+0000 7fbf55844700 1 Processor -- start 2023-11-26T07:14:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.102+0000 7fbf55844700 1 -- start start 2023-11-26T07:14:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.102+0000 7fbf55844700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf500b0fa0 0x7fbf500afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.102+0000 7fbf55844700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf500d0c40 0x7fbf500ca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.102+0000 7fbf55844700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf500d10a0 con 0x7fbf500b1380 2023-11-26T07:14:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.102+0000 7fbf55844700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf500ca6d0 con 0x7fbf500d0c40 2023-11-26T07:14:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.102+0000 7fbf55844700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf500622a0 con 0x7fbf500b0fa0 2023-11-26T07:14:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.102+0000 7fbf4e7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf500d0c40 0x7fbf500ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4e7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbf500d0c40 0x7fbf500ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38418/0 (socket says 172.21.15.83:38418) 2023-11-26T07:14:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4e7fc700 1 -- 172.21.15.83:0/3961002338 learned_addr learned my addr 172.21.15.83:0/3961002338 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4effd700 1 --2- 172.21.15.83:0/3961002338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf500b0fa0 0x7fbf500afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4e7fc700 1 -- 172.21.15.83:0/3961002338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf500b0fa0 msgr2=0x7fbf500afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4e7fc700 1 --2- 172.21.15.83:0/3961002338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf500b0fa0 0x7fbf500afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4e7fc700 1 -- 172.21.15.83:0/3961002338 >> v1:172.21.15.203:6789/0 conn(0x7fbf500b1380 legacy=0x7fbf500ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4dffb700 1 -- 172.21.15.83:0/3961002338 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90625591 0 0) 0x7fbf500d10a0 con 0x7fbf500b1380 2023-11-26T07:14:59.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4e7fc700 1 -- 172.21.15.83:0/3961002338 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf50062580 con 0x7fbf500d0c40 2023-11-26T07:14:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4e7fc700 1 --2- 172.21.15.83:0/3961002338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf500d0c40 0x7fbf500ca190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbf400098f0 tx=0x7fbf40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccedd9aa73268d7b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4dffb700 1 -- 172.21.15.83:0/3961002338 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40011830 con 0x7fbf500d0c40 2023-11-26T07:14:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4dffb700 1 -- 172.21.15.83:0/3961002338 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf40011990 con 0x7fbf500d0c40 2023-11-26T07:14:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.103+0000 7fbf4dffb700 1 -- 172.21.15.83:0/3961002338 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40011d20 con 0x7fbf500d0c40 2023-11-26T07:14:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.104+0000 7fbf55844700 1 -- 172.21.15.83:0/3961002338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf500d0c40 msgr2=0x7fbf500ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.104+0000 7fbf55844700 1 --2- 172.21.15.83:0/3961002338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf500d0c40 0x7fbf500ca190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbf400098f0 tx=0x7fbf40009c00 comp rx=0 tx=0).stop 2023-11-26T07:14:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.104+0000 7fbf55844700 1 -- 172.21.15.83:0/3961002338 shutdown_connections 2023-11-26T07:14:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.104+0000 7fbf55844700 1 --2- 172.21.15.83:0/3961002338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf500b0fa0 0x7fbf500afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.104+0000 7fbf55844700 1 --2- 172.21.15.83:0/3961002338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf500d0c40 0x7fbf500ca190 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.104+0000 7fbf55844700 1 -- 172.21.15.83:0/3961002338 >> 172.21.15.83:0/3961002338 conn(0x7fbf500b57b0 msgr2=0x7fbf500b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.104+0000 7fbf55844700 1 -- 172.21.15.83:0/3961002338 shutdown_connections 2023-11-26T07:14:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.104+0000 7fbf55844700 1 -- 172.21.15.83:0/3961002338 wait complete. 2023-11-26T07:14:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.105+0000 7fbf55844700 1 Processor -- start 2023-11-26T07:14:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.105+0000 7fbf55844700 1 -- start start 2023-11-26T07:14:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf55844700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf500b0fa0 0x7fbf500cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf55844700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf500b1380 0x7fbf50157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf55844700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf500d0c40 0x7fbf501f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf55844700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf50157ea0 con 0x7fbf500b0fa0 2023-11-26T07:14:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf55844700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf501596d0 con 0x7fbf500b1380 2023-11-26T07:14:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf55844700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5015bb40 con 0x7fbf500d0c40 2023-11-26T07:14:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf4effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf500b0fa0 0x7fbf500cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf4effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf500b0fa0 0x7fbf500cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38426/0 (socket says 172.21.15.83:38426) 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf4effd700 1 -- 172.21.15.83:0/1187459551 learned_addr learned my addr 172.21.15.83:0/1187459551 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf4effd700 1 -- 172.21.15.83:0/1187459551 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf500d0c40 msgr2=0x7fbf501f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf4effd700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf500d0c40 0x7fbf501f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf4effd700 1 -- 172.21.15.83:0/1187459551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf500b1380 msgr2=0x7fbf50157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf4effd700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf500b1380 0x7fbf50157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf4effd700 1 -- 172.21.15.83:0/1187459551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf40009550 con 0x7fbf500b0fa0 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.106+0000 7fbf4effd700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf500b0fa0 0x7fbf500cd6f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9b0 tx=0x7fbf4400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.107+0000 7fbf37fff700 1 -- 172.21.15.83:0/1187459551 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44009710 con 0x7fbf500b0fa0 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.107+0000 7fbf37fff700 1 -- 172.21.15.83:0/1187459551 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4400c850 con 0x7fbf500b0fa0 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.107+0000 7fbf55844700 1 -- 172.21.15.83:0/1187459551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5015be20 con 0x7fbf500b0fa0 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.107+0000 7fbf37fff700 1 -- 172.21.15.83:0/1187459551 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400cbc0 con 0x7fbf500b0fa0 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.107+0000 7fbf55844700 1 -- 172.21.15.83:0/1187459551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf501599b0 con 0x7fbf500b0fa0 2023-11-26T07:14:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.108+0000 7fbf55844700 1 -- 172.21.15.83:0/1187459551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf50067f60 con 0x7fbf500b0fa0 2023-11-26T07:14:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.108+0000 7fbf37fff700 1 -- 172.21.15.83:0/1187459551 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fbf4400cd20 con 0x7fbf500b0fa0 2023-11-26T07:14:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.108+0000 7fbf37fff700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf3c06b080 0x7fbf3c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.108+0000 7fbf37fff700 1 -- 172.21.15.83:0/1187459551 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf44089a40 con 0x7fbf500b0fa0 2023-11-26T07:14:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.108+0000 7fbf4e7fc700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf3c06b080 0x7fbf3c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.109+0000 7fbf4e7fc700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf3c06b080 0x7fbf3c06d540 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbf500ca3a0 tx=0x7fbf40010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.113+0000 7fbf37fff700 1 -- 172.21.15.83:0/1187459551 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf4408be60 con 0x7fbf500b0fa0 2023-11-26T07:14:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.304+0000 7fbf55844700 1 -- 172.21.15.83:0/1187459551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf500cf940 con 0x7fbf500b0fa0 2023-11-26T07:14:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.305+0000 7fbf37fff700 1 -- 172.21.15.83:0/1187459551 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fbf44021090 con 0x7fbf500b0fa0 2023-11-26T07:14:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:59.307 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":58,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:14:56.536771+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:14:53.044716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:14:53.904013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:14:59.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.307+0000 7fbf55844700 1 -- 172.21.15.83:0/1187459551 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf3c06b080 msgr2=0x7fbf3c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.307+0000 7fbf55844700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf3c06b080 0x7fbf3c06d540 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbf500ca3a0 tx=0x7fbf40010000 comp rx=0 tx=0).stop 2023-11-26T07:14:59.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.307+0000 7fbf55844700 1 -- 172.21.15.83:0/1187459551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf500b0fa0 msgr2=0x7fbf500cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.307+0000 7fbf55844700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf500b0fa0 0x7fbf500cd6f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9b0 tx=0x7fbf4400ed70 comp rx=0 tx=0).stop 2023-11-26T07:14:59.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.307+0000 7fbf55844700 1 -- 172.21.15.83:0/1187459551 shutdown_connections 2023-11-26T07:14:59.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.307+0000 7fbf55844700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf3c06b080 0x7fbf3c06d540 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.307+0000 7fbf55844700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf500d0c40 0x7fbf501f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.307+0000 7fbf55844700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf500b0fa0 0x7fbf500cd6f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.307+0000 7fbf55844700 1 --2- 172.21.15.83:0/1187459551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf500b1380 0x7fbf50157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.307+0000 7fbf55844700 1 -- 172.21.15.83:0/1187459551 >> 172.21.15.83:0/1187459551 conn(0x7fbf500b57b0 msgr2=0x7fbf5006ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:59.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.308+0000 7fbf55844700 1 -- 172.21.15.83:0/1187459551 shutdown_connections 2023-11-26T07:14:59.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.308+0000 7fbf55844700 1 -- 172.21.15.83:0/1187459551 wait complete. 2023-11-26T07:14:59.321 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:14:59.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:14:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.440+0000 7f4a841d1700 1 Processor -- start 2023-11-26T07:14:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.440+0000 7f4a841d1700 1 -- start start 2023-11-26T07:14:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a841d1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a7c0b5dd0 0x7f4a7c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a841d1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4a7c0b4180 0x7f4a7c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a841d1700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a7c0b0290 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a841d1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a7c0b4610 con 0x7f4a7c0b4180 2023-11-26T07:14:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a841d1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0b4820 con 0x7f4a7c0b5dd0 2023-11-26T07:14:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a8276e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4a7c0b4180 0x7f4a7c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a8276e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4a7c0b4180 0x7f4a7c0af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38436/0 (socket says 172.21.15.83:38436) 2023-11-26T07:14:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a8276e700 1 -- 172.21.15.83:0/1618743088 learned_addr learned my addr 172.21.15.83:0/1618743088 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a8176c700 1 --2- 172.21.15.83:0/1618743088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a7c0b5dd0 0x7f4a7c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a8276e700 1 -- 172.21.15.83:0/1618743088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a7c0b5dd0 msgr2=0x7f4a7c0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a8276e700 1 --2- 172.21.15.83:0/1618743088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a7c0b5dd0 0x7f4a7c0b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a8276e700 1 -- 172.21.15.83:0/1618743088 >> v1:172.21.15.203:6789/0 conn(0x7f4a7c0b59f0 legacy=0x7f4a7c06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a8276e700 1 -- 172.21.15.83:0/1618743088 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c0b4b00 con 0x7f4a7c0b4180 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.441+0000 7f4a8276e700 1 --2- 172.21.15.83:0/1618743088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c0b4180 0x7f4a7c0af190 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009e50 tx=0x7f4a7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd13f4b85e02fc90 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.442+0000 7f4a80f6b700 1 -- 172.21.15.83:0/1618743088 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400e070 con 0x7f4a7c0b4180 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.442+0000 7f4a80f6b700 1 -- 172.21.15.83:0/1618743088 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7400b910 con 0x7f4a7c0b4180 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.442+0000 7f4a80f6b700 1 -- 172.21.15.83:0/1618743088 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400bcc0 con 0x7f4a7c0b4180 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.442+0000 7f4a841d1700 1 -- 172.21.15.83:0/1618743088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c0b4180 msgr2=0x7f4a7c0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.442+0000 7f4a841d1700 1 --2- 172.21.15.83:0/1618743088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c0b4180 0x7f4a7c0af190 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009e50 tx=0x7f4a7400b330 comp rx=0 tx=0).stop 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.442+0000 7f4a841d1700 1 -- 172.21.15.83:0/1618743088 shutdown_connections 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.442+0000 7f4a841d1700 1 --2- 172.21.15.83:0/1618743088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c0b4180 0x7f4a7c0af190 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.442+0000 7f4a841d1700 1 --2- 172.21.15.83:0/1618743088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a7c0b5dd0 0x7f4a7c0b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.442+0000 7f4a841d1700 1 -- 172.21.15.83:0/1618743088 >> 172.21.15.83:0/1618743088 conn(0x7f4a7c072990 msgr2=0x7f4a7c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.442+0000 7f4a841d1700 1 -- 172.21.15.83:0/1618743088 shutdown_connections 2023-11-26T07:14:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.442+0000 7f4a841d1700 1 -- 172.21.15.83:0/1618743088 wait complete. 2023-11-26T07:14:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.443+0000 7f4a841d1700 1 Processor -- start 2023-11-26T07:14:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a841d1700 1 -- start start 2023-11-26T07:14:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a841d1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a7c0b4180 0x7f4a7c154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a841d1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c0b59f0 0x7f4a7c0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a841d1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a7c0b5dd0 0x7f4a7c147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a841d1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0cd400 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a841d1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0cd610 con 0x7f4a7c0b5dd0 2023-11-26T07:14:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a841d1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0cf530 con 0x7f4a7c0b4180 2023-11-26T07:14:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a8176c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c0b59f0 0x7f4a7c0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a8176c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c0b59f0 0x7f4a7c0ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38440/0 (socket says 172.21.15.83:38440) 2023-11-26T07:14:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a8176c700 1 -- 172.21.15.83:0/2202440112 learned_addr learned my addr 172.21.15.83:0/2202440112 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a81f6d700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a7c0b4180 0x7f4a7c154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.444+0000 7f4a8276e700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a7c0b5dd0 0x7f4a7c147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.445+0000 7f4a8176c700 1 -- 172.21.15.83:0/2202440112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a7c0b4180 msgr2=0x7f4a7c154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.445+0000 7f4a8176c700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a7c0b4180 0x7f4a7c154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.445+0000 7f4a8176c700 1 -- 172.21.15.83:0/2202440112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a7c0b5dd0 msgr2=0x7f4a7c147450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.445+0000 7f4a8176c700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a7c0b5dd0 0x7f4a7c147450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.445+0000 7f4a8176c700 1 -- 172.21.15.83:0/2202440112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a74009b30 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.445+0000 7f4a8176c700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c0b59f0 0x7f4a7c0ccec0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00e840 tx=0x7f4a6c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.445+0000 7f4a727fc700 1 -- 172.21.15.83:0/2202440112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c009590 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.445+0000 7f4a727fc700 1 -- 172.21.15.83:0/2202440112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6c013680 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.445+0000 7f4a841d1700 1 -- 172.21.15.83:0/2202440112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7c0cf780 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.445+0000 7f4a727fc700 1 -- 172.21.15.83:0/2202440112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c013a10 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.445+0000 7f4a841d1700 1 -- 172.21.15.83:0/2202440112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7c0cfd40 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.446+0000 7f4a841d1700 1 -- 172.21.15.83:0/2202440112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a7c067f60 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.446+0000 7f4a727fc700 1 -- 172.21.15.83:0/2202440112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4a6c013b70 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.447+0000 7f4a727fc700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4a6006b080 0x7f4a6006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.447+0000 7f4a727fc700 1 -- 172.21.15.83:0/2202440112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a6c089760 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.447+0000 7f4a81f6d700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4a6006b080 0x7f4a6006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.447+0000 7f4a81f6d700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4a6006b080 0x7f4a6006d540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4a78007fd0 tx=0x7f4a78009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.451+0000 7f4a727fc700 1 -- 172.21.15.83:0/2202440112 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a6c054500 con 0x7f4a7c0b59f0 2023-11-26T07:14:59.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.640+0000 7f4a841d1700 1 -- 172.21.15.83:0/2202440112 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a7c155460 con 0x7f4a6006b080 2023-11-26T07:14:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.648+0000 7f4a727fc700 1 -- 172.21.15.83:0/2202440112 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284140 (secure 0 0 0) 0x7f4a78001560 con 0x7f4a6006b080 2023-11-26T07:14:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:14:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 -- 172.21.15.83:0/2202440112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4a6006b080 msgr2=0x7f4a6006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4a6006b080 0x7f4a6006d540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4a78007fd0 tx=0x7f4a78009040 comp rx=0 tx=0).stop 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 -- 172.21.15.83:0/2202440112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c0b59f0 msgr2=0x7f4a7c0ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c0b59f0 0x7f4a7c0ccec0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00e840 tx=0x7f4a6c00eb50 comp rx=0 tx=0).stop 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 -- 172.21.15.83:0/2202440112 shutdown_connections 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a7c0b5dd0 0x7f4a7c147450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f4a6006b080 0x7f4a6006d540 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a7c0b4180 0x7f4a7c154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 --2- 172.21.15.83:0/2202440112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a7c0b59f0 0x7f4a7c0ccec0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 -- 172.21.15.83:0/2202440112 >> 172.21.15.83:0/2202440112 conn(0x7f4a7c072990 msgr2=0x7f4a7c161c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 -- 172.21.15.83:0/2202440112 shutdown_connections 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.651+0000 7f4a841d1700 1 -- 172.21.15.83:0/2202440112 wait complete. 2023-11-26T07:14:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:14:59.665 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2023-11-26T07:14:58.353214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222576,"kb_used_data":6372,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752144,"statfs":{"total":773094113280,"available":772866195456,"internally_reserved":0,"allocated":6524928,"data_stored":3375775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.822689"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.549358+0000","last_change":"2023-11-26T07:14:54.549358+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.549358+0000","last_undegraded":"2023-11-26T07:14:54.549358+0000","last_fullsized":"2023-11-26T07:14:54.549358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547389+0000","last_change":"2023-11-26T07:14:54.547389+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547389+0000","last_undegraded":"2023-11-26T07:14:54.547389+0000","last_fullsized":"2023-11-26T07:14:54.547389+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.542861+0000","last_change":"2023-11-26T07:14:54.542861+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.542861+0000","last_undegraded":"2023-11-26T07:14:54.542861+0000","last_fullsized":"2023-11-26T07:14:54.542861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548630+0000","last_change":"2023-11-26T07:14:54.548630+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548630+0000","last_undegraded":"2023-11-26T07:14:54.548630+0000","last_fullsized":"2023-11-26T07:14:54.548630+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.550269+0000","last_change":"2023-11-26T07:14:54.550269+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.550269+0000","last_undegraded":"2023-11-26T07:14:54.550269+0000","last_fullsized":"2023-11-26T07:14:54.550269+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541277+0000","last_change":"2023-11-26T07:14:54.541277+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541277+0000","last_undegraded":"2023-11-26T07:14:54.541277+0000","last_fullsized":"2023-11-26T07:14:54.541277+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546451+0000","last_change":"2023-11-26T07:14:54.546451+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546451+0000","last_undegraded":"2023-11-26T07:14:54.546451+0000","last_fullsized":"2023-11-26T07:14:54.546451+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544395+0000","last_change":"2023-11-26T07:14:54.544395+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544395+0000","last_undegraded":"2023-11-26T07:14:54.544395+0000","last_fullsized":"2023-11-26T07:14:54.544395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546689+0000","last_change":"2023-11-26T07:14:54.546689+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546689+0000","last_undegraded":"2023-11-26T07:14:54.546689+0000","last_fullsized":"2023-11-26T07:14:54.546689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546856+0000","last_change":"2023-11-26T07:14:54.546856+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546856+0000","last_undegraded":"2023-11-26T07:14:54.546856+0000","last_fullsized":"2023-11-26T07:14:54.546856+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548182+0000","last_change":"2023-11-26T07:14:54.548182+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548182+0000","last_undegraded":"2023-11-26T07:14:54.548182+0000","last_fullsized":"2023-11-26T07:14:54.548182+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547812+0000","last_change":"2023-11-26T07:14:54.547812+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547812+0000","last_undegraded":"2023-11-26T07:14:54.547812+0000","last_fullsized":"2023-11-26T07:14:54.547812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541763+0000","last_change":"2023-11-26T07:14:54.541763+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541763+0000","last_undegraded":"2023-11-26T07:14:54.541763+0000","last_fullsized":"2023-11-26T07:14:54.541763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546199+0000","last_change":"2023-11-26T07:14:54.546199+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546199+0000","last_undegraded":"2023-11-26T07:14:54.546199+0000","last_fullsized":"2023-11-26T07:14:54.546199+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.545707+0000","last_change":"2023-11-26T07:14:54.545707+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.545707+0000","last_undegraded":"2023-11-26T07:14:54.545707+0000","last_fullsized":"2023-11-26T07:14:54.545707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.542346+0000","last_change":"2023-11-26T07:14:54.542346+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.542346+0000","last_undegraded":"2023-11-26T07:14:54.542346+0000","last_fullsized":"2023-11-26T07:14:54.542346+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547654+0000","last_change":"2023-11-26T07:14:54.547654+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547654+0000","last_undegraded":"2023-11-26T07:14:54.547654+0000","last_fullsized":"2023-11-26T07:14:54.547654+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.543362+0000","last_change":"2023-11-26T07:14:54.543362+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.543362+0000","last_undegraded":"2023-11-26T07:14:54.543362+0000","last_fullsized":"2023-11-26T07:14:54.543362+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.551714+0000","last_change":"2023-11-26T07:14:54.551714+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.551714+0000","last_undegraded":"2023-11-26T07:14:54.551714+0000","last_fullsized":"2023-11-26T07:14:54.551714+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544508+0000","last_change":"2023-11-26T07:14:54.544508+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544508+0000","last_undegraded":"2023-11-26T07:14:54.544508+0000","last_fullsized":"2023-11-26T07:14:54.544508+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541946+0000","last_change":"2023-11-26T07:14:54.541946+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541946+0000","last_undegraded":"2023-11-26T07:14:54.541946+0000","last_fullsized":"2023-11-26T07:14:54.541946+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544631+0000","last_change":"2023-11-26T07:14:54.544631+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544631+0000","last_undegraded":"2023-11-26T07:14:54.544631+0000","last_fullsized":"2023-11-26T07:14:54.544631+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540712+0000","last_change":"2023-11-26T07:14:54.540712+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540712+0000","last_undegraded":"2023-11-26T07:14:54.540712+0000","last_fullsized":"2023-11-26T07:14:54.540712+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541275+0000","last_change":"2023-11-26T07:14:54.541275+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541275+0000","last_undegraded":"2023-11-26T07:14:54.541275+0000","last_fullsized":"2023-11-26T07:14:54.541275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548834+0000","last_change":"2023-11-26T07:14:54.548834+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548834+0000","last_undegraded":"2023-11-26T07:14:54.548834+0000","last_fullsized":"2023-11-26T07:14:54.548834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548311+0000","last_change":"2023-11-26T07:14:54.548311+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548311+0000","last_undegraded":"2023-11-26T07:14:54.548311+0000","last_fullsized":"2023-11-26T07:14:54.548311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.542044+0000","last_change":"2023-11-26T07:14:54.542044+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.542044+0000","last_undegraded":"2023-11-26T07:14:54.542044+0000","last_fullsized":"2023-11-26T07:14:54.542044+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.549007+0000","last_change":"2023-11-26T07:14:54.549007+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.549007+0000","last_undegraded":"2023-11-26T07:14:54.549007+0000","last_fullsized":"2023-11-26T07:14:54.549007+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.545371+0000","last_change":"2023-11-26T07:14:54.545371+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.545371+0000","last_undegraded":"2023-11-26T07:14:54.545371+0000","last_fullsized":"2023-11-26T07:14:54.545371+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540768+0000","last_change":"2023-11-26T07:14:54.540768+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540768+0000","last_undegraded":"2023-11-26T07:14:54.540768+0000","last_fullsized":"2023-11-26T07:14:54.540768+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.551840+0000","last_change":"2023-11-26T07:14:54.551840+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.551840+0000","last_undegraded":"2023-11-26T07:14:54.551840+0000","last_fullsized":"2023-11-26T07:14:54.551840+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541070+0000","last_change":"2023-11-26T07:14:54.541070+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541070+0000","last_undegraded":"2023-11-26T07:14:54.541070+0000","last_fullsized":"2023-11-26T07:14:54.541070+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541058+0000","last_change":"2023-11-26T07:14:54.541058+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541058+0000","last_undegraded":"2023-11-26T07:14:54.541058+0000","last_fullsized":"2023-11-26T07:14:54.541058+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.553923+0000","last_change":"2023-11-26T07:14:54.551197+0000","last_active":"2023-11-26T07:14:54.553923+0000","last_peered":"2023-11-26T07:14:54.553923+0000","last_clean":"2023-11-26T07:14:54.553923+0000","last_became_active":"2023-11-26T07:14:54.550930+0000","last_became_peered":"2023-11-26T07:14:54.550930+0000","last_unstale":"2023-11-26T07:14:54.553923+0000","last_undegraded":"2023-11-26T07:14:54.553923+0000","last_fullsized":"2023-11-26T07:14:54.553923+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:28.707932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.549795+0000","last_change":"2023-11-26T07:14:54.549795+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.549795+0000","last_undegraded":"2023-11-26T07:14:54.549795+0000","last_fullsized":"2023-11-26T07:14:54.549795+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559392+0000","last_change":"2023-11-26T07:14:54.556263+0000","last_active":"2023-11-26T07:14:54.559392+0000","last_peered":"2023-11-26T07:14:54.559392+0000","last_clean":"2023-11-26T07:14:54.559392+0000","last_became_active":"2023-11-26T07:14:54.555815+0000","last_became_peered":"2023-11-26T07:14:54.555815+0000","last_unstale":"2023-11-26T07:14:54.559392+0000","last_undegraded":"2023-11-26T07:14:54.559392+0000","last_fullsized":"2023-11-26T07:14:54.559392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:19.097609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541498+0000","last_change":"2023-11-26T07:14:54.541498+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541498+0000","last_undegraded":"2023-11-26T07:14:54.541498+0000","last_fullsized":"2023-11-26T07:14:54.541498+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559206+0000","last_change":"2023-11-26T07:14:54.553820+0000","last_active":"2023-11-26T07:14:54.559206+0000","last_peered":"2023-11-26T07:14:54.559206+0000","last_clean":"2023-11-26T07:14:54.559206+0000","last_became_active":"2023-11-26T07:14:54.553552+0000","last_became_peered":"2023-11-26T07:14:54.553552+0000","last_unstale":"2023-11-26T07:14:54.559206+0000","last_undegraded":"2023-11-26T07:14:54.559206+0000","last_fullsized":"2023-11-26T07:14:54.559206+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:25:04.899521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541758+0000","last_change":"2023-11-26T07:14:54.541758+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541758+0000","last_undegraded":"2023-11-26T07:14:54.541758+0000","last_fullsized":"2023-11-26T07:14:54.541758+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559898+0000","last_change":"2023-11-26T07:14:54.555832+0000","last_active":"2023-11-26T07:14:54.559898+0000","last_peered":"2023-11-26T07:14:54.559898+0000","last_clean":"2023-11-26T07:14:54.559898+0000","last_became_active":"2023-11-26T07:14:54.555514+0000","last_became_peered":"2023-11-26T07:14:54.555514+0000","last_unstale":"2023-11-26T07:14:54.559898+0000","last_undegraded":"2023-11-26T07:14:54.559898+0000","last_fullsized":"2023-11-26T07:14:54.559898+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:43.781381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544223+0000","last_change":"2023-11-26T07:14:54.544223+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544223+0000","last_undegraded":"2023-11-26T07:14:54.544223+0000","last_fullsized":"2023-11-26T07:14:54.544223+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559889+0000","last_change":"2023-11-26T07:14:54.554968+0000","last_active":"2023-11-26T07:14:54.559889+0000","last_peered":"2023-11-26T07:14:54.559889+0000","last_clean":"2023-11-26T07:14:54.559889+0000","last_became_active":"2023-11-26T07:14:54.554664+0000","last_became_peered":"2023-11-26T07:14:54.554664+0000","last_unstale":"2023-11-26T07:14:54.559889+0000","last_undegraded":"2023-11-26T07:14:54.559889+0000","last_fullsized":"2023-11-26T07:14:54.559889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:16:04.759914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544911+0000","last_change":"2023-11-26T07:14:54.544911+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544911+0000","last_undegraded":"2023-11-26T07:14:54.544911+0000","last_fullsized":"2023-11-26T07:14:54.544911+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.548357+0000","last_change":"2023-11-26T07:14:54.545826+0000","last_active":"2023-11-26T07:14:54.548357+0000","last_peered":"2023-11-26T07:14:54.548357+0000","last_clean":"2023-11-26T07:14:54.548357+0000","last_became_active":"2023-11-26T07:14:54.545558+0000","last_became_peered":"2023-11-26T07:14:54.545558+0000","last_unstale":"2023-11-26T07:14:54.548357+0000","last_undegraded":"2023-11-26T07:14:54.548357+0000","last_fullsized":"2023-11-26T07:14:54.548357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:10:48.528921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540783+0000","last_change":"2023-11-26T07:14:54.540783+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540783+0000","last_undegraded":"2023-11-26T07:14:54.540783+0000","last_fullsized":"2023-11-26T07:14:54.540783+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559922+0000","last_change":"2023-11-26T07:14:54.555979+0000","last_active":"2023-11-26T07:14:54.559922+0000","last_peered":"2023-11-26T07:14:54.559922+0000","last_clean":"2023-11-26T07:14:54.559922+0000","last_became_active":"2023-11-26T07:14:54.555658+0000","last_became_peered":"2023-11-26T07:14:54.555658+0000","last_unstale":"2023-11-26T07:14:54.559922+0000","last_undegraded":"2023-11-26T07:14:54.559922+0000","last_fullsized":"2023-11-26T07:14:54.559922+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:43:49.044638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.545961+0000","last_change":"2023-11-26T07:14:54.545961+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.545961+0000","last_undegraded":"2023-11-26T07:14:54.545961+0000","last_fullsized":"2023-11-26T07:14:54.545961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.560027+0000","last_change":"2023-11-26T07:14:54.556906+0000","last_active":"2023-11-26T07:14:54.560027+0000","last_peered":"2023-11-26T07:14:54.560027+0000","last_clean":"2023-11-26T07:14:54.560027+0000","last_became_active":"2023-11-26T07:14:54.556633+0000","last_became_peered":"2023-11-26T07:14:54.556633+0000","last_unstale":"2023-11-26T07:14:54.560027+0000","last_undegraded":"2023-11-26T07:14:54.560027+0000","last_fullsized":"2023-11-26T07:14:54.560027+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:44.510723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.545819+0000","last_change":"2023-11-26T07:14:54.545819+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.545819+0000","last_undegraded":"2023-11-26T07:14:54.545819+0000","last_fullsized":"2023-11-26T07:14:54.545819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.555070+0000","last_change":"2023-11-26T07:14:54.551547+0000","last_active":"2023-11-26T07:14:54.555070+0000","last_peered":"2023-11-26T07:14:54.555070+0000","last_clean":"2023-11-26T07:14:54.555070+0000","last_became_active":"2023-11-26T07:14:54.551059+0000","last_became_peered":"2023-11-26T07:14:54.551059+0000","last_unstale":"2023-11-26T07:14:54.555070+0000","last_undegraded":"2023-11-26T07:14:54.555070+0000","last_fullsized":"2023-11-26T07:14:54.555070+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:41:47.910749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.543111+0000","last_change":"2023-11-26T07:14:54.543111+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.543111+0000","last_undegraded":"2023-11-26T07:14:54.543111+0000","last_fullsized":"2023-11-26T07:14:54.543111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559368+0000","last_change":"2023-11-26T07:14:54.556255+0000","last_active":"2023-11-26T07:14:54.559368+0000","last_peered":"2023-11-26T07:14:54.559368+0000","last_clean":"2023-11-26T07:14:54.559368+0000","last_became_active":"2023-11-26T07:14:54.555863+0000","last_became_peered":"2023-11-26T07:14:54.555863+0000","last_unstale":"2023-11-26T07:14:54.559368+0000","last_undegraded":"2023-11-26T07:14:54.559368+0000","last_fullsized":"2023-11-26T07:14:54.559368+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:12.852910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548484+0000","last_change":"2023-11-26T07:14:54.548484+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548484+0000","last_undegraded":"2023-11-26T07:14:54.548484+0000","last_fullsized":"2023-11-26T07:14:54.548484+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559325+0000","last_change":"2023-11-26T07:14:54.553992+0000","last_active":"2023-11-26T07:14:54.559325+0000","last_peered":"2023-11-26T07:14:54.559325+0000","last_clean":"2023-11-26T07:14:54.559325+0000","last_became_active":"2023-11-26T07:14:54.553718+0000","last_became_peered":"2023-11-26T07:14:54.553718+0000","last_unstale":"2023-11-26T07:14:54.559325+0000","last_undegraded":"2023-11-26T07:14:54.559325+0000","last_fullsized":"2023-11-26T07:14:54.559325+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:56.667181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.542413+0000","last_change":"2023-11-26T07:14:54.542413+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.542413+0000","last_undegraded":"2023-11-26T07:14:54.542413+0000","last_fullsized":"2023-11-26T07:14:54.542413+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.560033+0000","last_change":"2023-11-26T07:14:54.557028+0000","last_active":"2023-11-26T07:14:54.560033+0000","last_peered":"2023-11-26T07:14:54.560033+0000","last_clean":"2023-11-26T07:14:54.560033+0000","last_became_active":"2023-11-26T07:14:54.556758+0000","last_became_peered":"2023-11-26T07:14:54.556758+0000","last_unstale":"2023-11-26T07:14:54.560033+0000","last_undegraded":"2023-11-26T07:14:54.560033+0000","last_fullsized":"2023-11-26T07:14:54.560033+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:27:59.258743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547804+0000","last_change":"2023-11-26T07:14:54.547804+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547804+0000","last_undegraded":"2023-11-26T07:14:54.547804+0000","last_fullsized":"2023-11-26T07:14:54.547804+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.558868+0000","last_change":"2023-11-26T07:14:54.554993+0000","last_active":"2023-11-26T07:14:54.558868+0000","last_peered":"2023-11-26T07:14:54.558868+0000","last_clean":"2023-11-26T07:14:54.558868+0000","last_became_active":"2023-11-26T07:14:54.554750+0000","last_became_peered":"2023-11-26T07:14:54.554750+0000","last_unstale":"2023-11-26T07:14:54.558868+0000","last_undegraded":"2023-11-26T07:14:54.558868+0000","last_fullsized":"2023-11-26T07:14:54.558868+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:42.288092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541496+0000","last_change":"2023-11-26T07:14:54.541496+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541496+0000","last_undegraded":"2023-11-26T07:14:54.541496+0000","last_fullsized":"2023-11-26T07:14:54.541496+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559486+0000","last_change":"2023-11-26T07:14:54.552221+0000","last_active":"2023-11-26T07:14:54.559486+0000","last_peered":"2023-11-26T07:14:54.559486+0000","last_clean":"2023-11-26T07:14:54.559486+0000","last_became_active":"2023-11-26T07:14:54.551932+0000","last_became_peered":"2023-11-26T07:14:54.551932+0000","last_unstale":"2023-11-26T07:14:54.559486+0000","last_undegraded":"2023-11-26T07:14:54.559486+0000","last_fullsized":"2023-11-26T07:14:54.559486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:00:25.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546874+0000","last_change":"2023-11-26T07:14:54.546874+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546874+0000","last_undegraded":"2023-11-26T07:14:54.546874+0000","last_fullsized":"2023-11-26T07:14:54.546874+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559385+0000","last_change":"2023-11-26T07:14:54.553341+0000","last_active":"2023-11-26T07:14:54.559385+0000","last_peered":"2023-11-26T07:14:54.559385+0000","last_clean":"2023-11-26T07:14:54.559385+0000","last_became_active":"2023-11-26T07:14:54.551872+0000","last_became_peered":"2023-11-26T07:14:54.551872+0000","last_unstale":"2023-11-26T07:14:54.559385+0000","last_undegraded":"2023-11-26T07:14:54.559385+0000","last_fullsized":"2023-11-26T07:14:54.559385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:28.707932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541415+0000","last_change":"2023-11-26T07:14:54.541415+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541415+0000","last_undegraded":"2023-11-26T07:14:54.541415+0000","last_fullsized":"2023-11-26T07:14:54.541415+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.555091+0000","last_change":"2023-11-26T07:14:54.551340+0000","last_active":"2023-11-26T07:14:54.555091+0000","last_peered":"2023-11-26T07:14:54.555091+0000","last_clean":"2023-11-26T07:14:54.555091+0000","last_became_active":"2023-11-26T07:14:54.551030+0000","last_became_peered":"2023-11-26T07:14:54.551030+0000","last_unstale":"2023-11-26T07:14:54.555091+0000","last_undegraded":"2023-11-26T07:14:54.555091+0000","last_fullsized":"2023-11-26T07:14:54.555091+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:39.725578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540954+0000","last_change":"2023-11-26T07:14:54.540954+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540954+0000","last_undegraded":"2023-11-26T07:14:54.540954+0000","last_fullsized":"2023-11-26T07:14:54.540954+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.561926+0000","last_change":"2023-11-26T07:14:54.558605+0000","last_active":"2023-11-26T07:14:54.561926+0000","last_peered":"2023-11-26T07:14:54.561926+0000","last_clean":"2023-11-26T07:14:54.561926+0000","last_became_active":"2023-11-26T07:14:54.558324+0000","last_became_peered":"2023-11-26T07:14:54.558324+0000","last_unstale":"2023-11-26T07:14:54.561926+0000","last_undegraded":"2023-11-26T07:14:54.561926+0000","last_fullsized":"2023-11-26T07:14:54.561926+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:54:56.750302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547311+0000","last_change":"2023-11-26T07:14:54.547311+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547311+0000","last_undegraded":"2023-11-26T07:14:54.547311+0000","last_fullsized":"2023-11-26T07:14:54.547311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.561539+0000","last_change":"2023-11-26T07:14:54.556982+0000","last_active":"2023-11-26T07:14:54.561539+0000","last_peered":"2023-11-26T07:14:54.561539+0000","last_clean":"2023-11-26T07:14:54.561539+0000","last_became_active":"2023-11-26T07:14:54.556728+0000","last_became_peered":"2023-11-26T07:14:54.556728+0000","last_unstale":"2023-11-26T07:14:54.561539+0000","last_undegraded":"2023-11-26T07:14:54.561539+0000","last_fullsized":"2023-11-26T07:14:54.561539+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:58:50.594707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544068+0000","last_change":"2023-11-26T07:14:54.544068+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544068+0000","last_undegraded":"2023-11-26T07:14:54.544068+0000","last_fullsized":"2023-11-26T07:14:54.544068+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559409+0000","last_change":"2023-11-26T07:14:54.553413+0000","last_active":"2023-11-26T07:14:54.559409+0000","last_peered":"2023-11-26T07:14:54.559409+0000","last_clean":"2023-11-26T07:14:54.559409+0000","last_became_active":"2023-11-26T07:14:54.552941+0000","last_became_peered":"2023-11-26T07:14:54.552941+0000","last_unstale":"2023-11-26T07:14:54.559409+0000","last_undegraded":"2023-11-26T07:14:54.559409+0000","last_fullsized":"2023-11-26T07:14:54.559409+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:10:48.528921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540897+0000","last_change":"2023-11-26T07:14:54.540897+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540897+0000","last_undegraded":"2023-11-26T07:14:54.540897+0000","last_fullsized":"2023-11-26T07:14:54.540897+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.548825+0000","last_change":"2023-11-26T07:14:54.546741+0000","last_active":"2023-11-26T07:14:54.548825+0000","last_peered":"2023-11-26T07:14:54.548825+0000","last_clean":"2023-11-26T07:14:54.548825+0000","last_became_active":"2023-11-26T07:14:54.546479+0000","last_became_peered":"2023-11-26T07:14:54.546479+0000","last_unstale":"2023-11-26T07:14:54.548825+0000","last_undegraded":"2023-11-26T07:14:54.548825+0000","last_fullsized":"2023-11-26T07:14:54.548825+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:02:51.060031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547369+0000","last_change":"2023-11-26T07:14:54.547369+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547369+0000","last_undegraded":"2023-11-26T07:14:54.547369+0000","last_fullsized":"2023-11-26T07:14:54.547369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.558889+0000","last_change":"2023-11-26T07:14:54.555163+0000","last_active":"2023-11-26T07:14:54.558889+0000","last_peered":"2023-11-26T07:14:54.558889+0000","last_clean":"2023-11-26T07:14:54.558889+0000","last_became_active":"2023-11-26T07:14:54.554331+0000","last_became_peered":"2023-11-26T07:14:54.554331+0000","last_unstale":"2023-11-26T07:14:54.558889+0000","last_undegraded":"2023-11-26T07:14:54.558889+0000","last_fullsized":"2023-11-26T07:14:54.558889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:24.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541601+0000","last_change":"2023-11-26T07:14:54.541601+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541601+0000","last_undegraded":"2023-11-26T07:14:54.541601+0000","last_fullsized":"2023-11-26T07:14:54.541601+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.557606+0000","last_change":"2023-11-26T07:14:54.554798+0000","last_active":"2023-11-26T07:14:54.557606+0000","last_peered":"2023-11-26T07:14:54.557606+0000","last_clean":"2023-11-26T07:14:54.557606+0000","last_became_active":"2023-11-26T07:14:54.554545+0000","last_became_peered":"2023-11-26T07:14:54.554545+0000","last_unstale":"2023-11-26T07:14:54.557606+0000","last_undegraded":"2023-11-26T07:14:54.557606+0000","last_fullsized":"2023-11-26T07:14:54.557606+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:16:58.335520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541803+0000","last_change":"2023-11-26T07:14:54.541803+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541803+0000","last_undegraded":"2023-11-26T07:14:54.541803+0000","last_fullsized":"2023-11-26T07:14:54.541803+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559962+0000","last_change":"2023-11-26T07:14:54.556775+0000","last_active":"2023-11-26T07:14:54.559962+0000","last_peered":"2023-11-26T07:14:54.559962+0000","last_clean":"2023-11-26T07:14:54.559962+0000","last_became_active":"2023-11-26T07:14:54.556477+0000","last_became_peered":"2023-11-26T07:14:54.556477+0000","last_unstale":"2023-11-26T07:14:54.559962+0000","last_undegraded":"2023-11-26T07:14:54.559962+0000","last_fullsized":"2023-11-26T07:14:54.559962+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:58:50.594707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546092+0000","last_change":"2023-11-26T07:14:54.546092+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546092+0000","last_undegraded":"2023-11-26T07:14:54.546092+0000","last_fullsized":"2023-11-26T07:14:54.546092+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.562046+0000","last_change":"2023-11-26T07:14:54.559238+0000","last_active":"2023-11-26T07:14:54.562046+0000","last_peered":"2023-11-26T07:14:54.562046+0000","last_clean":"2023-11-26T07:14:54.562046+0000","last_became_active":"2023-11-26T07:14:54.558934+0000","last_became_peered":"2023-11-26T07:14:54.558934+0000","last_unstale":"2023-11-26T07:14:54.562046+0000","last_undegraded":"2023-11-26T07:14:54.562046+0000","last_fullsized":"2023-11-26T07:14:54.562046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:31:47.905726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546852+0000","last_change":"2023-11-26T07:14:54.546852+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546852+0000","last_undegraded":"2023-11-26T07:14:54.546852+0000","last_fullsized":"2023-11-26T07:14:54.546852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.551219+0000","last_change":"2023-11-26T07:14:54.550263+0000","last_active":"2023-11-26T07:14:54.551219+0000","last_peered":"2023-11-26T07:14:54.551219+0000","last_clean":"2023-11-26T07:14:54.551219+0000","last_became_active":"2023-11-26T07:14:54.549971+0000","last_became_peered":"2023-11-26T07:14:54.549971+0000","last_unstale":"2023-11-26T07:14:54.551219+0000","last_undegraded":"2023-11-26T07:14:54.551219+0000","last_fullsized":"2023-11-26T07:14:54.551219+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:43:49.044638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548069+0000","last_change":"2023-11-26T07:14:54.548069+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548069+0000","last_undegraded":"2023-11-26T07:14:54.548069+0000","last_fullsized":"2023-11-26T07:14:54.548069+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559548+0000","last_change":"2023-11-26T07:14:54.552589+0000","last_active":"2023-11-26T07:14:54.559548+0000","last_peered":"2023-11-26T07:14:54.559548+0000","last_clean":"2023-11-26T07:14:54.559548+0000","last_became_active":"2023-11-26T07:14:54.552302+0000","last_became_peered":"2023-11-26T07:14:54.552302+0000","last_unstale":"2023-11-26T07:14:54.559548+0000","last_undegraded":"2023-11-26T07:14:54.559548+0000","last_fullsized":"2023-11-26T07:14:54.559548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:59.018549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541119+0000","last_change":"2023-11-26T07:14:54.541119+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541119+0000","last_undegraded":"2023-11-26T07:14:54.541119+0000","last_fullsized":"2023-11-26T07:14:54.541119+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559918+0000","last_change":"2023-11-26T07:14:54.555982+0000","last_active":"2023-11-26T07:14:54.559918+0000","last_peered":"2023-11-26T07:14:54.559918+0000","last_clean":"2023-11-26T07:14:54.559918+0000","last_became_active":"2023-11-26T07:14:54.555678+0000","last_became_peered":"2023-11-26T07:14:54.555678+0000","last_unstale":"2023-11-26T07:14:54.559918+0000","last_undegraded":"2023-11-26T07:14:54.559918+0000","last_fullsized":"2023-11-26T07:14:54.559918+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:10:26.659226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.543043+0000","last_change":"2023-11-26T07:14:54.543043+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.543043+0000","last_undegraded":"2023-11-26T07:14:54.543043+0000","last_fullsized":"2023-11-26T07:14:54.543043+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.561812+0000","last_change":"2023-11-26T07:14:54.556883+0000","last_active":"2023-11-26T07:14:54.561812+0000","last_peered":"2023-11-26T07:14:54.561812+0000","last_clean":"2023-11-26T07:14:54.561812+0000","last_became_active":"2023-11-26T07:14:54.556654+0000","last_became_peered":"2023-11-26T07:14:54.556654+0000","last_unstale":"2023-11-26T07:14:54.561812+0000","last_undegraded":"2023-11-26T07:14:54.561812+0000","last_fullsized":"2023-11-26T07:14:54.561812+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:44.324311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541030+0000","last_change":"2023-11-26T07:14:54.541030+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541030+0000","last_undegraded":"2023-11-26T07:14:54.541030+0000","last_fullsized":"2023-11-26T07:14:54.541030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559901+0000","last_change":"2023-11-26T07:14:54.555216+0000","last_active":"2023-11-26T07:14:54.559901+0000","last_peered":"2023-11-26T07:14:54.559901+0000","last_clean":"2023-11-26T07:14:54.559901+0000","last_became_active":"2023-11-26T07:14:54.554577+0000","last_became_peered":"2023-11-26T07:14:54.554577+0000","last_unstale":"2023-11-26T07:14:54.559901+0000","last_undegraded":"2023-11-26T07:14:54.559901+0000","last_fullsized":"2023-11-26T07:14:54.559901+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:58.113764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546852+0000","last_change":"2023-11-26T07:14:54.546852+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546852+0000","last_undegraded":"2023-11-26T07:14:54.546852+0000","last_fullsized":"2023-11-26T07:14:54.546852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.561825+0000","last_change":"2023-11-26T07:14:54.556996+0000","last_active":"2023-11-26T07:14:54.561825+0000","last_peered":"2023-11-26T07:14:54.561825+0000","last_clean":"2023-11-26T07:14:54.561825+0000","last_became_active":"2023-11-26T07:14:54.556722+0000","last_became_peered":"2023-11-26T07:14:54.556722+0000","last_unstale":"2023-11-26T07:14:54.561825+0000","last_undegraded":"2023-11-26T07:14:54.561825+0000","last_fullsized":"2023-11-26T07:14:54.561825+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:21.473873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.542996+0000","last_change":"2023-11-26T07:14:54.542996+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.542996+0000","last_undegraded":"2023-11-26T07:14:54.542996+0000","last_fullsized":"2023-11-26T07:14:54.542996+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.555720+0000","last_change":"2023-11-26T07:14:54.550363+0000","last_active":"2023-11-26T07:14:54.555720+0000","last_peered":"2023-11-26T07:14:54.555720+0000","last_clean":"2023-11-26T07:14:54.555720+0000","last_became_active":"2023-11-26T07:14:54.550093+0000","last_became_peered":"2023-11-26T07:14:54.550093+0000","last_unstale":"2023-11-26T07:14:54.555720+0000","last_undegraded":"2023-11-26T07:14:54.555720+0000","last_fullsized":"2023-11-26T07:14:54.555720+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:30.521505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.543958+0000","last_change":"2023-11-26T07:14:54.543958+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.543958+0000","last_undegraded":"2023-11-26T07:14:54.543958+0000","last_fullsized":"2023-11-26T07:14:54.543958+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559599+0000","last_change":"2023-11-26T07:14:54.552852+0000","last_active":"2023-11-26T07:14:54.559599+0000","last_peered":"2023-11-26T07:14:54.559599+0000","last_clean":"2023-11-26T07:14:54.559599+0000","last_became_active":"2023-11-26T07:14:54.552543+0000","last_became_peered":"2023-11-26T07:14:54.552543+0000","last_unstale":"2023-11-26T07:14:54.559599+0000","last_undegraded":"2023-11-26T07:14:54.559599+0000","last_fullsized":"2023-11-26T07:14:54.559599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:30.337901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-26T07:14:53.538375+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:14:53.538375+0000","last_peered":"2023-11-26T07:14:53.538375+0000","last_clean":"2023-11-26T07:14:53.538375+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:14:53.538375+0000","last_undegraded":"2023-11-26T07:14:53.538375+0000","last_fullsized":"2023-11-26T07:14:53.538375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":867098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.55900000000000005},"min":{"1min":0.44,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.52200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.47299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73799999999999999,"5min":1.048,"15min":1.048},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.497,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.65200000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.495,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.621,"15min":0.622},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75600000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63800000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70499999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.623},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77800000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.59299999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76700000000000002,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.41499999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.37,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79500000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79000000000000004,"5min":1.171,"15min":1.171},"last":0.52100000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.74199999999999999,"5min":1.361,"15min":1.361},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78000000000000003,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":276730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.39200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.73699999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76600000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.49099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.69799999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.45800000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72299999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82599999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.753,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70399999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56799999999999995,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67900000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.51200000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.747,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73399999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":270463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.502,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.34899999999999998,"15min":0.34599999999999997},"min":{"1min":0.20000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.5,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.25900000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34000000000000002,"15min":0.33400000000000002},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.27200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53000000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.39500000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.252,"5min":0.246,"15min":0.246},"max":{"1min":0.57499999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.32300000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.877,"15min":0.877},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.44900000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.623,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":0.995,"15min":0.995},"last":0.499},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.46800000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88800000000000001,"5min":1.129,"15min":1.129},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.627,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47999999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47399999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83599999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":867098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.221,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54200000000000004,"5min":0.623,"15min":0.623},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.311}]},{"osd":2,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.629,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.374},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.28599999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.36099999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61699999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.437,"15min":0.439},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41299999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67000000000000004,"15min":0.67300000000000004},"min":{"1min":0.499,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96599999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60499999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.499},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57699999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.625},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83799999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.51200000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45100000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55700000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":659456,"data_stored":270463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36099999999999999,"15min":0.36899999999999999},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.373},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.497,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.27700000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.27900000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58799999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.27400000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.56499999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.31},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42599999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54000000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.29599999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64800000000000002,"15min":0.67600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85799999999999998,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79300000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.621,"15min":0.624},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81399999999999995,"5min":1.075,"15min":1.075},"last":0.443},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.38200000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80100000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.47099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.77900000000000003,"5min":1.03,"15min":1.03},"last":0.52700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.65900000000000003},"min":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.622,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82799999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.502}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":671744,"data_stored":276730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.218,"5min":0.188,"15min":0.188},"max":{"1min":0.42199999999999999,"5min":1.284,"15min":1.284},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.44900000000000001,"15min":0.47999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.55500000000000005,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.47499999999999998,"15min":0.52700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61799999999999999,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.46500000000000002,"15min":0.498},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.50700000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.50800000000000001},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.59299999999999997,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52700000000000002,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.44}]},{"osd":4,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.68000000000000005},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85299999999999998,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88200000000000001,"5min":1.405,"15min":1.405},"last":0.66300000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.50700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83799999999999997,"5min":1.377,"15min":1.377},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":0.80900000000000005},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91100000000000003,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.60899999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66200000000000003,"15min":0.67500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92500000000000004,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.75600000000000001,"15min":0.80200000000000005},"min":{"1min":0.51700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.871,"5min":4.883,"15min":4.883},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.75700000000000001,"15min":0.81899999999999995},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80200000000000005,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71599999999999997,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85299999999999998,"5min":2.923,"15min":2.923},"last":0.55200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":276730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.41499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77500000000000002,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.43099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.36499999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.53000000000000003,"15min":0.51200000000000001},"min":{"1min":0.44700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77500000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.63400000000000001,"15min":0.66400000000000003},"min":{"1min":0.45600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71099999999999997,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.41499999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54300000000000004,"15min":0.52500000000000002},"min":{"1min":0.47899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.68600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.64700000000000002,"15min":0.67900000000000005},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74299999999999999,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.51000000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.40500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75900000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73399999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.58399999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.54300000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.497},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72999999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71699999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.625,"15min":0.63600000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79600000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":270463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.39200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.49099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.35999999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.48799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.73799999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.42399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.44800000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.45900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64700000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.498,"15min":0.5},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.61599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67300000000000004,"5min":0.752,"15min":0.752},"last":0.49299999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.65800000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70899999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.66000000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63},{"interface":"front","average":{"1min":0.625,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76600000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:14:59.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:14:59.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.824+0000 7f5097383700 1 Processor -- start 2023-11-26T07:14:59.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.824+0000 7f5097383700 1 -- start start 2023-11-26T07:14:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f5097383700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509015e080 0x7f50900b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f5097383700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50900af4d0 0x7f50900afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f5097383700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50900ae430 con 0x7f509006c000 2023-11-26T07:14:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f5097383700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50900af0b0 con 0x7f50900af4d0 2023-11-26T07:14:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f5097383700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50900b05a0 con 0x7f509015e080 2023-11-26T07:14:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f509491e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50900af4d0 0x7f50900afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f509491e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50900af4d0 0x7f50900afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38448/0 (socket says 172.21.15.83:38448) 2023-11-26T07:14:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f509491e700 1 -- 172.21.15.83:0/2413704320 learned_addr learned my addr 172.21.15.83:0/2413704320 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f509511f700 1 --2- 172.21.15.83:0/2413704320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509015e080 0x7f50900b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f509491e700 1 -- 172.21.15.83:0/2413704320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509015e080 msgr2=0x7f50900b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f509491e700 1 --2- 172.21.15.83:0/2413704320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509015e080 0x7f50900b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f509491e700 1 -- 172.21.15.83:0/2413704320 >> v1:172.21.15.203:6789/0 conn(0x7f509006c000 legacy=0x7f5090073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f509491e700 1 -- 172.21.15.83:0/2413704320 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50900ae710 con 0x7f50900af4d0 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.825+0000 7f509491e700 1 --2- 172.21.15.83:0/2413704320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50900af4d0 0x7f50900afb90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5080009920 tx=0x7f5080009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4e1aef665e1f7b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.826+0000 7f5087fff700 1 -- 172.21.15.83:0/2413704320 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080004230 con 0x7f50900af4d0 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.826+0000 7f5087fff700 1 -- 172.21.15.83:0/2413704320 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5080004390 con 0x7f50900af4d0 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.826+0000 7f5087fff700 1 -- 172.21.15.83:0/2413704320 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080009e70 con 0x7f50900af4d0 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.826+0000 7f5097383700 1 -- 172.21.15.83:0/2413704320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50900af4d0 msgr2=0x7f50900afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.826+0000 7f5097383700 1 --2- 172.21.15.83:0/2413704320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50900af4d0 0x7f50900afb90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5080009920 tx=0x7f5080009c30 comp rx=0 tx=0).stop 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.826+0000 7f5097383700 1 -- 172.21.15.83:0/2413704320 shutdown_connections 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.826+0000 7f5097383700 1 --2- 172.21.15.83:0/2413704320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509015e080 0x7f50900b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.826+0000 7f5097383700 1 --2- 172.21.15.83:0/2413704320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50900af4d0 0x7f50900afb90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.826+0000 7f5097383700 1 -- 172.21.15.83:0/2413704320 >> 172.21.15.83:0/2413704320 conn(0x7f5090071b90 msgr2=0x7f5090071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:14:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.826+0000 7f5097383700 1 -- 172.21.15.83:0/2413704320 shutdown_connections 2023-11-26T07:14:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.826+0000 7f5097383700 1 -- 172.21.15.83:0/2413704320 wait complete. 2023-11-26T07:14:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.827+0000 7f5097383700 1 Processor -- start 2023-11-26T07:14:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.827+0000 7f5097383700 1 -- start start 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f5097383700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509006c000 0x7f50900cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f5097383700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50900af4d0 0x7f50901528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f5097383700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509015e080 0x7f50901450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f5097383700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50900d0050 con 0x7f509006c000 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f5097383700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5090153580 con 0x7f50900af4d0 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f5097383700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50900cf550 con 0x7f509015e080 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f509511f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509006c000 0x7f50900cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f509511f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509006c000 0x7f50900cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38452/0 (socket says 172.21.15.83:38452) 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f509511f700 1 -- 172.21.15.83:0/1902526508 learned_addr learned my addr 172.21.15.83:0/1902526508 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f509491e700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50900af4d0 0x7f50901528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f5095920700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509015e080 0x7f50901450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f509511f700 1 -- 172.21.15.83:0/1902526508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509015e080 msgr2=0x7f50901450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f509511f700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509015e080 0x7f50901450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f509511f700 1 -- 172.21.15.83:0/1902526508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50900af4d0 msgr2=0x7f50901528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f509511f700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50900af4d0 0x7f50901528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.828+0000 7f509511f700 1 -- 172.21.15.83:0/1902526508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5080009550 con 0x7f509006c000 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.829+0000 7f5095920700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509015e080 0x7f50901450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.829+0000 7f509491e700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50900af4d0 0x7f50901528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.829+0000 7f509511f700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509006c000 0x7f50900cf010 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f508c00c7f0 tx=0x7f508c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.829+0000 7f5085ffb700 1 -- 172.21.15.83:0/1902526508 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c011800 con 0x7f509006c000 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.829+0000 7f5085ffb700 1 -- 172.21.15.83:0/1902526508 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c011960 con 0x7f509006c000 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.829+0000 7f5097383700 1 -- 172.21.15.83:0/1902526508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50900cf7b0 con 0x7f509006c000 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.829+0000 7f5085ffb700 1 -- 172.21.15.83:0/1902526508 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c011cf0 con 0x7f509006c000 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.829+0000 7f5097383700 1 -- 172.21.15.83:0/1902526508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50900cfd70 con 0x7f509006c000 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.830+0000 7f5097383700 1 -- 172.21.15.83:0/1902526508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50900753f0 con 0x7f509006c000 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.830+0000 7f5085ffb700 1 -- 172.21.15.83:0/1902526508 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f508c018360 con 0x7f509006c000 2023-11-26T07:14:59.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.831+0000 7f5085ffb700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f507406b080 0x7f507406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:14:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.831+0000 7f5085ffb700 1 -- 172.21.15.83:0/1902526508 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f508c0899b0 con 0x7f509006c000 2023-11-26T07:14:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.831+0000 7f509491e700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f507406b080 0x7f507406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:14:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.831+0000 7f509491e700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f507406b080 0x7f507406d540 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5080009920 tx=0x7f5080008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:14:59.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:14:59.835+0000 7f5085ffb700 1 -- 172.21.15.83:0/1902526508 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f508c054640 con 0x7f509006c000 2023-11-26T07:15:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.022+0000 7f5097383700 1 -- 172.21.15.83:0/1902526508 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50900aecd0 con 0x7f507406b080 2023-11-26T07:15:00.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.030+0000 7f5085ffb700 1 -- 172.21.15.83:0/1902526508 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284140 (secure 0 0 0) 0x7f5080020070 con 0x7f507406b080 2023-11-26T07:15:00.035 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.033+0000 7f5097383700 1 -- 172.21.15.83:0/1902526508 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f507406b080 msgr2=0x7f507406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.033+0000 7f5097383700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f507406b080 0x7f507406d540 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5080009920 tx=0x7f5080008030 comp rx=0 tx=0).stop 2023-11-26T07:15:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.033+0000 7f5097383700 1 -- 172.21.15.83:0/1902526508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509006c000 msgr2=0x7f50900cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.033+0000 7f5097383700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509006c000 0x7f50900cf010 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f508c00c7f0 tx=0x7f508c00cbb0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.033+0000 7f5097383700 1 -- 172.21.15.83:0/1902526508 shutdown_connections 2023-11-26T07:15:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.033+0000 7f5097383700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f507406b080 0x7f507406d540 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.033+0000 7f5097383700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f509015e080 0x7f50901450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.033+0000 7f5097383700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509006c000 0x7f50900cf010 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.033+0000 7f5097383700 1 --2- 172.21.15.83:0/1902526508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50900af4d0 0x7f50901528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.033+0000 7f5097383700 1 -- 172.21.15.83:0/1902526508 >> 172.21.15.83:0/1902526508 conn(0x7f5090071b90 msgr2=0x7f50900b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.033+0000 7f5097383700 1 -- 172.21.15.83:0/1902526508 shutdown_connections 2023-11-26T07:15:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.034+0000 7f5097383700 1 -- 172.21.15.83:0/1902526508 wait complete. 2023-11-26T07:15:00.038 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:15:00.047 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2023-11-26T07:14:58.353214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222576,"kb_used_data":6372,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752144,"statfs":{"total":773094113280,"available":772866195456,"internally_reserved":0,"allocated":6524928,"data_stored":3375775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.822689"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.549358+0000","last_change":"2023-11-26T07:14:54.549358+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.549358+0000","last_undegraded":"2023-11-26T07:14:54.549358+0000","last_fullsized":"2023-11-26T07:14:54.549358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547389+0000","last_change":"2023-11-26T07:14:54.547389+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547389+0000","last_undegraded":"2023-11-26T07:14:54.547389+0000","last_fullsized":"2023-11-26T07:14:54.547389+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.542861+0000","last_change":"2023-11-26T07:14:54.542861+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.542861+0000","last_undegraded":"2023-11-26T07:14:54.542861+0000","last_fullsized":"2023-11-26T07:14:54.542861+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548630+0000","last_change":"2023-11-26T07:14:54.548630+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548630+0000","last_undegraded":"2023-11-26T07:14:54.548630+0000","last_fullsized":"2023-11-26T07:14:54.548630+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.550269+0000","last_change":"2023-11-26T07:14:54.550269+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.550269+0000","last_undegraded":"2023-11-26T07:14:54.550269+0000","last_fullsized":"2023-11-26T07:14:54.550269+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541277+0000","last_change":"2023-11-26T07:14:54.541277+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541277+0000","last_undegraded":"2023-11-26T07:14:54.541277+0000","last_fullsized":"2023-11-26T07:14:54.541277+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546451+0000","last_change":"2023-11-26T07:14:54.546451+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546451+0000","last_undegraded":"2023-11-26T07:14:54.546451+0000","last_fullsized":"2023-11-26T07:14:54.546451+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544395+0000","last_change":"2023-11-26T07:14:54.544395+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544395+0000","last_undegraded":"2023-11-26T07:14:54.544395+0000","last_fullsized":"2023-11-26T07:14:54.544395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546689+0000","last_change":"2023-11-26T07:14:54.546689+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546689+0000","last_undegraded":"2023-11-26T07:14:54.546689+0000","last_fullsized":"2023-11-26T07:14:54.546689+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546856+0000","last_change":"2023-11-26T07:14:54.546856+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546856+0000","last_undegraded":"2023-11-26T07:14:54.546856+0000","last_fullsized":"2023-11-26T07:14:54.546856+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548182+0000","last_change":"2023-11-26T07:14:54.548182+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548182+0000","last_undegraded":"2023-11-26T07:14:54.548182+0000","last_fullsized":"2023-11-26T07:14:54.548182+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547812+0000","last_change":"2023-11-26T07:14:54.547812+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547812+0000","last_undegraded":"2023-11-26T07:14:54.547812+0000","last_fullsized":"2023-11-26T07:14:54.547812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541763+0000","last_change":"2023-11-26T07:14:54.541763+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541763+0000","last_undegraded":"2023-11-26T07:14:54.541763+0000","last_fullsized":"2023-11-26T07:14:54.541763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546199+0000","last_change":"2023-11-26T07:14:54.546199+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546199+0000","last_undegraded":"2023-11-26T07:14:54.546199+0000","last_fullsized":"2023-11-26T07:14:54.546199+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.545707+0000","last_change":"2023-11-26T07:14:54.545707+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.545707+0000","last_undegraded":"2023-11-26T07:14:54.545707+0000","last_fullsized":"2023-11-26T07:14:54.545707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.542346+0000","last_change":"2023-11-26T07:14:54.542346+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.542346+0000","last_undegraded":"2023-11-26T07:14:54.542346+0000","last_fullsized":"2023-11-26T07:14:54.542346+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547654+0000","last_change":"2023-11-26T07:14:54.547654+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547654+0000","last_undegraded":"2023-11-26T07:14:54.547654+0000","last_fullsized":"2023-11-26T07:14:54.547654+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.543362+0000","last_change":"2023-11-26T07:14:54.543362+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.543362+0000","last_undegraded":"2023-11-26T07:14:54.543362+0000","last_fullsized":"2023-11-26T07:14:54.543362+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.551714+0000","last_change":"2023-11-26T07:14:54.551714+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.551714+0000","last_undegraded":"2023-11-26T07:14:54.551714+0000","last_fullsized":"2023-11-26T07:14:54.551714+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544508+0000","last_change":"2023-11-26T07:14:54.544508+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544508+0000","last_undegraded":"2023-11-26T07:14:54.544508+0000","last_fullsized":"2023-11-26T07:14:54.544508+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541946+0000","last_change":"2023-11-26T07:14:54.541946+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541946+0000","last_undegraded":"2023-11-26T07:14:54.541946+0000","last_fullsized":"2023-11-26T07:14:54.541946+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544631+0000","last_change":"2023-11-26T07:14:54.544631+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544631+0000","last_undegraded":"2023-11-26T07:14:54.544631+0000","last_fullsized":"2023-11-26T07:14:54.544631+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540712+0000","last_change":"2023-11-26T07:14:54.540712+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540712+0000","last_undegraded":"2023-11-26T07:14:54.540712+0000","last_fullsized":"2023-11-26T07:14:54.540712+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541275+0000","last_change":"2023-11-26T07:14:54.541275+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541275+0000","last_undegraded":"2023-11-26T07:14:54.541275+0000","last_fullsized":"2023-11-26T07:14:54.541275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548834+0000","last_change":"2023-11-26T07:14:54.548834+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548834+0000","last_undegraded":"2023-11-26T07:14:54.548834+0000","last_fullsized":"2023-11-26T07:14:54.548834+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548311+0000","last_change":"2023-11-26T07:14:54.548311+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548311+0000","last_undegraded":"2023-11-26T07:14:54.548311+0000","last_fullsized":"2023-11-26T07:14:54.548311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.542044+0000","last_change":"2023-11-26T07:14:54.542044+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.542044+0000","last_undegraded":"2023-11-26T07:14:54.542044+0000","last_fullsized":"2023-11-26T07:14:54.542044+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.549007+0000","last_change":"2023-11-26T07:14:54.549007+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.549007+0000","last_undegraded":"2023-11-26T07:14:54.549007+0000","last_fullsized":"2023-11-26T07:14:54.549007+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.545371+0000","last_change":"2023-11-26T07:14:54.545371+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.545371+0000","last_undegraded":"2023-11-26T07:14:54.545371+0000","last_fullsized":"2023-11-26T07:14:54.545371+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540768+0000","last_change":"2023-11-26T07:14:54.540768+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540768+0000","last_undegraded":"2023-11-26T07:14:54.540768+0000","last_fullsized":"2023-11-26T07:14:54.540768+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.551840+0000","last_change":"2023-11-26T07:14:54.551840+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.551840+0000","last_undegraded":"2023-11-26T07:14:54.551840+0000","last_fullsized":"2023-11-26T07:14:54.551840+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541070+0000","last_change":"2023-11-26T07:14:54.541070+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541070+0000","last_undegraded":"2023-11-26T07:14:54.541070+0000","last_fullsized":"2023-11-26T07:14:54.541070+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541058+0000","last_change":"2023-11-26T07:14:54.541058+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541058+0000","last_undegraded":"2023-11-26T07:14:54.541058+0000","last_fullsized":"2023-11-26T07:14:54.541058+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.553923+0000","last_change":"2023-11-26T07:14:54.551197+0000","last_active":"2023-11-26T07:14:54.553923+0000","last_peered":"2023-11-26T07:14:54.553923+0000","last_clean":"2023-11-26T07:14:54.553923+0000","last_became_active":"2023-11-26T07:14:54.550930+0000","last_became_peered":"2023-11-26T07:14:54.550930+0000","last_unstale":"2023-11-26T07:14:54.553923+0000","last_undegraded":"2023-11-26T07:14:54.553923+0000","last_fullsized":"2023-11-26T07:14:54.553923+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:28.707932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.549795+0000","last_change":"2023-11-26T07:14:54.549795+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.549795+0000","last_undegraded":"2023-11-26T07:14:54.549795+0000","last_fullsized":"2023-11-26T07:14:54.549795+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559392+0000","last_change":"2023-11-26T07:14:54.556263+0000","last_active":"2023-11-26T07:14:54.559392+0000","last_peered":"2023-11-26T07:14:54.559392+0000","last_clean":"2023-11-26T07:14:54.559392+0000","last_became_active":"2023-11-26T07:14:54.555815+0000","last_became_peered":"2023-11-26T07:14:54.555815+0000","last_unstale":"2023-11-26T07:14:54.559392+0000","last_undegraded":"2023-11-26T07:14:54.559392+0000","last_fullsized":"2023-11-26T07:14:54.559392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:19.097609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541498+0000","last_change":"2023-11-26T07:14:54.541498+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541498+0000","last_undegraded":"2023-11-26T07:14:54.541498+0000","last_fullsized":"2023-11-26T07:14:54.541498+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559206+0000","last_change":"2023-11-26T07:14:54.553820+0000","last_active":"2023-11-26T07:14:54.559206+0000","last_peered":"2023-11-26T07:14:54.559206+0000","last_clean":"2023-11-26T07:14:54.559206+0000","last_became_active":"2023-11-26T07:14:54.553552+0000","last_became_peered":"2023-11-26T07:14:54.553552+0000","last_unstale":"2023-11-26T07:14:54.559206+0000","last_undegraded":"2023-11-26T07:14:54.559206+0000","last_fullsized":"2023-11-26T07:14:54.559206+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:25:04.899521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541758+0000","last_change":"2023-11-26T07:14:54.541758+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541758+0000","last_undegraded":"2023-11-26T07:14:54.541758+0000","last_fullsized":"2023-11-26T07:14:54.541758+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559898+0000","last_change":"2023-11-26T07:14:54.555832+0000","last_active":"2023-11-26T07:14:54.559898+0000","last_peered":"2023-11-26T07:14:54.559898+0000","last_clean":"2023-11-26T07:14:54.559898+0000","last_became_active":"2023-11-26T07:14:54.555514+0000","last_became_peered":"2023-11-26T07:14:54.555514+0000","last_unstale":"2023-11-26T07:14:54.559898+0000","last_undegraded":"2023-11-26T07:14:54.559898+0000","last_fullsized":"2023-11-26T07:14:54.559898+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:43.781381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544223+0000","last_change":"2023-11-26T07:14:54.544223+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544223+0000","last_undegraded":"2023-11-26T07:14:54.544223+0000","last_fullsized":"2023-11-26T07:14:54.544223+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559889+0000","last_change":"2023-11-26T07:14:54.554968+0000","last_active":"2023-11-26T07:14:54.559889+0000","last_peered":"2023-11-26T07:14:54.559889+0000","last_clean":"2023-11-26T07:14:54.559889+0000","last_became_active":"2023-11-26T07:14:54.554664+0000","last_became_peered":"2023-11-26T07:14:54.554664+0000","last_unstale":"2023-11-26T07:14:54.559889+0000","last_undegraded":"2023-11-26T07:14:54.559889+0000","last_fullsized":"2023-11-26T07:14:54.559889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:16:04.759914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544911+0000","last_change":"2023-11-26T07:14:54.544911+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544911+0000","last_undegraded":"2023-11-26T07:14:54.544911+0000","last_fullsized":"2023-11-26T07:14:54.544911+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.548357+0000","last_change":"2023-11-26T07:14:54.545826+0000","last_active":"2023-11-26T07:14:54.548357+0000","last_peered":"2023-11-26T07:14:54.548357+0000","last_clean":"2023-11-26T07:14:54.548357+0000","last_became_active":"2023-11-26T07:14:54.545558+0000","last_became_peered":"2023-11-26T07:14:54.545558+0000","last_unstale":"2023-11-26T07:14:54.548357+0000","last_undegraded":"2023-11-26T07:14:54.548357+0000","last_fullsized":"2023-11-26T07:14:54.548357+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:10:48.528921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540783+0000","last_change":"2023-11-26T07:14:54.540783+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540783+0000","last_undegraded":"2023-11-26T07:14:54.540783+0000","last_fullsized":"2023-11-26T07:14:54.540783+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559922+0000","last_change":"2023-11-26T07:14:54.555979+0000","last_active":"2023-11-26T07:14:54.559922+0000","last_peered":"2023-11-26T07:14:54.559922+0000","last_clean":"2023-11-26T07:14:54.559922+0000","last_became_active":"2023-11-26T07:14:54.555658+0000","last_became_peered":"2023-11-26T07:14:54.555658+0000","last_unstale":"2023-11-26T07:14:54.559922+0000","last_undegraded":"2023-11-26T07:14:54.559922+0000","last_fullsized":"2023-11-26T07:14:54.559922+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:43:49.044638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.545961+0000","last_change":"2023-11-26T07:14:54.545961+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.545961+0000","last_undegraded":"2023-11-26T07:14:54.545961+0000","last_fullsized":"2023-11-26T07:14:54.545961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.560027+0000","last_change":"2023-11-26T07:14:54.556906+0000","last_active":"2023-11-26T07:14:54.560027+0000","last_peered":"2023-11-26T07:14:54.560027+0000","last_clean":"2023-11-26T07:14:54.560027+0000","last_became_active":"2023-11-26T07:14:54.556633+0000","last_became_peered":"2023-11-26T07:14:54.556633+0000","last_unstale":"2023-11-26T07:14:54.560027+0000","last_undegraded":"2023-11-26T07:14:54.560027+0000","last_fullsized":"2023-11-26T07:14:54.560027+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:44.510723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.545819+0000","last_change":"2023-11-26T07:14:54.545819+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.545819+0000","last_undegraded":"2023-11-26T07:14:54.545819+0000","last_fullsized":"2023-11-26T07:14:54.545819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.555070+0000","last_change":"2023-11-26T07:14:54.551547+0000","last_active":"2023-11-26T07:14:54.555070+0000","last_peered":"2023-11-26T07:14:54.555070+0000","last_clean":"2023-11-26T07:14:54.555070+0000","last_became_active":"2023-11-26T07:14:54.551059+0000","last_became_peered":"2023-11-26T07:14:54.551059+0000","last_unstale":"2023-11-26T07:14:54.555070+0000","last_undegraded":"2023-11-26T07:14:54.555070+0000","last_fullsized":"2023-11-26T07:14:54.555070+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:41:47.910749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.543111+0000","last_change":"2023-11-26T07:14:54.543111+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.543111+0000","last_undegraded":"2023-11-26T07:14:54.543111+0000","last_fullsized":"2023-11-26T07:14:54.543111+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559368+0000","last_change":"2023-11-26T07:14:54.556255+0000","last_active":"2023-11-26T07:14:54.559368+0000","last_peered":"2023-11-26T07:14:54.559368+0000","last_clean":"2023-11-26T07:14:54.559368+0000","last_became_active":"2023-11-26T07:14:54.555863+0000","last_became_peered":"2023-11-26T07:14:54.555863+0000","last_unstale":"2023-11-26T07:14:54.559368+0000","last_undegraded":"2023-11-26T07:14:54.559368+0000","last_fullsized":"2023-11-26T07:14:54.559368+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:12.852910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548484+0000","last_change":"2023-11-26T07:14:54.548484+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548484+0000","last_undegraded":"2023-11-26T07:14:54.548484+0000","last_fullsized":"2023-11-26T07:14:54.548484+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559325+0000","last_change":"2023-11-26T07:14:54.553992+0000","last_active":"2023-11-26T07:14:54.559325+0000","last_peered":"2023-11-26T07:14:54.559325+0000","last_clean":"2023-11-26T07:14:54.559325+0000","last_became_active":"2023-11-26T07:14:54.553718+0000","last_became_peered":"2023-11-26T07:14:54.553718+0000","last_unstale":"2023-11-26T07:14:54.559325+0000","last_undegraded":"2023-11-26T07:14:54.559325+0000","last_fullsized":"2023-11-26T07:14:54.559325+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:56.667181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.542413+0000","last_change":"2023-11-26T07:14:54.542413+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.542413+0000","last_undegraded":"2023-11-26T07:14:54.542413+0000","last_fullsized":"2023-11-26T07:14:54.542413+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.560033+0000","last_change":"2023-11-26T07:14:54.557028+0000","last_active":"2023-11-26T07:14:54.560033+0000","last_peered":"2023-11-26T07:14:54.560033+0000","last_clean":"2023-11-26T07:14:54.560033+0000","last_became_active":"2023-11-26T07:14:54.556758+0000","last_became_peered":"2023-11-26T07:14:54.556758+0000","last_unstale":"2023-11-26T07:14:54.560033+0000","last_undegraded":"2023-11-26T07:14:54.560033+0000","last_fullsized":"2023-11-26T07:14:54.560033+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:27:59.258743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547804+0000","last_change":"2023-11-26T07:14:54.547804+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547804+0000","last_undegraded":"2023-11-26T07:14:54.547804+0000","last_fullsized":"2023-11-26T07:14:54.547804+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.558868+0000","last_change":"2023-11-26T07:14:54.554993+0000","last_active":"2023-11-26T07:14:54.558868+0000","last_peered":"2023-11-26T07:14:54.558868+0000","last_clean":"2023-11-26T07:14:54.558868+0000","last_became_active":"2023-11-26T07:14:54.554750+0000","last_became_peered":"2023-11-26T07:14:54.554750+0000","last_unstale":"2023-11-26T07:14:54.558868+0000","last_undegraded":"2023-11-26T07:14:54.558868+0000","last_fullsized":"2023-11-26T07:14:54.558868+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:42.288092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541496+0000","last_change":"2023-11-26T07:14:54.541496+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541496+0000","last_undegraded":"2023-11-26T07:14:54.541496+0000","last_fullsized":"2023-11-26T07:14:54.541496+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559486+0000","last_change":"2023-11-26T07:14:54.552221+0000","last_active":"2023-11-26T07:14:54.559486+0000","last_peered":"2023-11-26T07:14:54.559486+0000","last_clean":"2023-11-26T07:14:54.559486+0000","last_became_active":"2023-11-26T07:14:54.551932+0000","last_became_peered":"2023-11-26T07:14:54.551932+0000","last_unstale":"2023-11-26T07:14:54.559486+0000","last_undegraded":"2023-11-26T07:14:54.559486+0000","last_fullsized":"2023-11-26T07:14:54.559486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:00:25.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546874+0000","last_change":"2023-11-26T07:14:54.546874+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546874+0000","last_undegraded":"2023-11-26T07:14:54.546874+0000","last_fullsized":"2023-11-26T07:14:54.546874+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559385+0000","last_change":"2023-11-26T07:14:54.553341+0000","last_active":"2023-11-26T07:14:54.559385+0000","last_peered":"2023-11-26T07:14:54.559385+0000","last_clean":"2023-11-26T07:14:54.559385+0000","last_became_active":"2023-11-26T07:14:54.551872+0000","last_became_peered":"2023-11-26T07:14:54.551872+0000","last_unstale":"2023-11-26T07:14:54.559385+0000","last_undegraded":"2023-11-26T07:14:54.559385+0000","last_fullsized":"2023-11-26T07:14:54.559385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:28.707932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541415+0000","last_change":"2023-11-26T07:14:54.541415+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541415+0000","last_undegraded":"2023-11-26T07:14:54.541415+0000","last_fullsized":"2023-11-26T07:14:54.541415+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.555091+0000","last_change":"2023-11-26T07:14:54.551340+0000","last_active":"2023-11-26T07:14:54.555091+0000","last_peered":"2023-11-26T07:14:54.555091+0000","last_clean":"2023-11-26T07:14:54.555091+0000","last_became_active":"2023-11-26T07:14:54.551030+0000","last_became_peered":"2023-11-26T07:14:54.551030+0000","last_unstale":"2023-11-26T07:14:54.555091+0000","last_undegraded":"2023-11-26T07:14:54.555091+0000","last_fullsized":"2023-11-26T07:14:54.555091+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:39.725578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540954+0000","last_change":"2023-11-26T07:14:54.540954+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540954+0000","last_undegraded":"2023-11-26T07:14:54.540954+0000","last_fullsized":"2023-11-26T07:14:54.540954+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.561926+0000","last_change":"2023-11-26T07:14:54.558605+0000","last_active":"2023-11-26T07:14:54.561926+0000","last_peered":"2023-11-26T07:14:54.561926+0000","last_clean":"2023-11-26T07:14:54.561926+0000","last_became_active":"2023-11-26T07:14:54.558324+0000","last_became_peered":"2023-11-26T07:14:54.558324+0000","last_unstale":"2023-11-26T07:14:54.561926+0000","last_undegraded":"2023-11-26T07:14:54.561926+0000","last_fullsized":"2023-11-26T07:14:54.561926+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:54:56.750302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547311+0000","last_change":"2023-11-26T07:14:54.547311+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547311+0000","last_undegraded":"2023-11-26T07:14:54.547311+0000","last_fullsized":"2023-11-26T07:14:54.547311+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.561539+0000","last_change":"2023-11-26T07:14:54.556982+0000","last_active":"2023-11-26T07:14:54.561539+0000","last_peered":"2023-11-26T07:14:54.561539+0000","last_clean":"2023-11-26T07:14:54.561539+0000","last_became_active":"2023-11-26T07:14:54.556728+0000","last_became_peered":"2023-11-26T07:14:54.556728+0000","last_unstale":"2023-11-26T07:14:54.561539+0000","last_undegraded":"2023-11-26T07:14:54.561539+0000","last_fullsized":"2023-11-26T07:14:54.561539+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:58:50.594707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.544068+0000","last_change":"2023-11-26T07:14:54.544068+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.544068+0000","last_undegraded":"2023-11-26T07:14:54.544068+0000","last_fullsized":"2023-11-26T07:14:54.544068+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559409+0000","last_change":"2023-11-26T07:14:54.553413+0000","last_active":"2023-11-26T07:14:54.559409+0000","last_peered":"2023-11-26T07:14:54.559409+0000","last_clean":"2023-11-26T07:14:54.559409+0000","last_became_active":"2023-11-26T07:14:54.552941+0000","last_became_peered":"2023-11-26T07:14:54.552941+0000","last_unstale":"2023-11-26T07:14:54.559409+0000","last_undegraded":"2023-11-26T07:14:54.559409+0000","last_fullsized":"2023-11-26T07:14:54.559409+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:10:48.528921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540897+0000","last_change":"2023-11-26T07:14:54.540897+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540897+0000","last_undegraded":"2023-11-26T07:14:54.540897+0000","last_fullsized":"2023-11-26T07:14:54.540897+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.548825+0000","last_change":"2023-11-26T07:14:54.546741+0000","last_active":"2023-11-26T07:14:54.548825+0000","last_peered":"2023-11-26T07:14:54.548825+0000","last_clean":"2023-11-26T07:14:54.548825+0000","last_became_active":"2023-11-26T07:14:54.546479+0000","last_became_peered":"2023-11-26T07:14:54.546479+0000","last_unstale":"2023-11-26T07:14:54.548825+0000","last_undegraded":"2023-11-26T07:14:54.548825+0000","last_fullsized":"2023-11-26T07:14:54.548825+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:02:51.060031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.547369+0000","last_change":"2023-11-26T07:14:54.547369+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.547369+0000","last_undegraded":"2023-11-26T07:14:54.547369+0000","last_fullsized":"2023-11-26T07:14:54.547369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.558889+0000","last_change":"2023-11-26T07:14:54.555163+0000","last_active":"2023-11-26T07:14:54.558889+0000","last_peered":"2023-11-26T07:14:54.558889+0000","last_clean":"2023-11-26T07:14:54.558889+0000","last_became_active":"2023-11-26T07:14:54.554331+0000","last_became_peered":"2023-11-26T07:14:54.554331+0000","last_unstale":"2023-11-26T07:14:54.558889+0000","last_undegraded":"2023-11-26T07:14:54.558889+0000","last_fullsized":"2023-11-26T07:14:54.558889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:24.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541601+0000","last_change":"2023-11-26T07:14:54.541601+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541601+0000","last_undegraded":"2023-11-26T07:14:54.541601+0000","last_fullsized":"2023-11-26T07:14:54.541601+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.557606+0000","last_change":"2023-11-26T07:14:54.554798+0000","last_active":"2023-11-26T07:14:54.557606+0000","last_peered":"2023-11-26T07:14:54.557606+0000","last_clean":"2023-11-26T07:14:54.557606+0000","last_became_active":"2023-11-26T07:14:54.554545+0000","last_became_peered":"2023-11-26T07:14:54.554545+0000","last_unstale":"2023-11-26T07:14:54.557606+0000","last_undegraded":"2023-11-26T07:14:54.557606+0000","last_fullsized":"2023-11-26T07:14:54.557606+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:16:58.335520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541803+0000","last_change":"2023-11-26T07:14:54.541803+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541803+0000","last_undegraded":"2023-11-26T07:14:54.541803+0000","last_fullsized":"2023-11-26T07:14:54.541803+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559962+0000","last_change":"2023-11-26T07:14:54.556775+0000","last_active":"2023-11-26T07:14:54.559962+0000","last_peered":"2023-11-26T07:14:54.559962+0000","last_clean":"2023-11-26T07:14:54.559962+0000","last_became_active":"2023-11-26T07:14:54.556477+0000","last_became_peered":"2023-11-26T07:14:54.556477+0000","last_unstale":"2023-11-26T07:14:54.559962+0000","last_undegraded":"2023-11-26T07:14:54.559962+0000","last_fullsized":"2023-11-26T07:14:54.559962+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:58:50.594707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546092+0000","last_change":"2023-11-26T07:14:54.546092+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546092+0000","last_undegraded":"2023-11-26T07:14:54.546092+0000","last_fullsized":"2023-11-26T07:14:54.546092+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.562046+0000","last_change":"2023-11-26T07:14:54.559238+0000","last_active":"2023-11-26T07:14:54.562046+0000","last_peered":"2023-11-26T07:14:54.562046+0000","last_clean":"2023-11-26T07:14:54.562046+0000","last_became_active":"2023-11-26T07:14:54.558934+0000","last_became_peered":"2023-11-26T07:14:54.558934+0000","last_unstale":"2023-11-26T07:14:54.562046+0000","last_undegraded":"2023-11-26T07:14:54.562046+0000","last_fullsized":"2023-11-26T07:14:54.562046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:31:47.905726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546852+0000","last_change":"2023-11-26T07:14:54.546852+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546852+0000","last_undegraded":"2023-11-26T07:14:54.546852+0000","last_fullsized":"2023-11-26T07:14:54.546852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.551219+0000","last_change":"2023-11-26T07:14:54.550263+0000","last_active":"2023-11-26T07:14:54.551219+0000","last_peered":"2023-11-26T07:14:54.551219+0000","last_clean":"2023-11-26T07:14:54.551219+0000","last_became_active":"2023-11-26T07:14:54.549971+0000","last_became_peered":"2023-11-26T07:14:54.549971+0000","last_unstale":"2023-11-26T07:14:54.551219+0000","last_undegraded":"2023-11-26T07:14:54.551219+0000","last_fullsized":"2023-11-26T07:14:54.551219+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:43:49.044638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.548069+0000","last_change":"2023-11-26T07:14:54.548069+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.548069+0000","last_undegraded":"2023-11-26T07:14:54.548069+0000","last_fullsized":"2023-11-26T07:14:54.548069+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559548+0000","last_change":"2023-11-26T07:14:54.552589+0000","last_active":"2023-11-26T07:14:54.559548+0000","last_peered":"2023-11-26T07:14:54.559548+0000","last_clean":"2023-11-26T07:14:54.559548+0000","last_became_active":"2023-11-26T07:14:54.552302+0000","last_became_peered":"2023-11-26T07:14:54.552302+0000","last_unstale":"2023-11-26T07:14:54.559548+0000","last_undegraded":"2023-11-26T07:14:54.559548+0000","last_fullsized":"2023-11-26T07:14:54.559548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:59.018549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541119+0000","last_change":"2023-11-26T07:14:54.541119+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541119+0000","last_undegraded":"2023-11-26T07:14:54.541119+0000","last_fullsized":"2023-11-26T07:14:54.541119+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559918+0000","last_change":"2023-11-26T07:14:54.555982+0000","last_active":"2023-11-26T07:14:54.559918+0000","last_peered":"2023-11-26T07:14:54.559918+0000","last_clean":"2023-11-26T07:14:54.559918+0000","last_became_active":"2023-11-26T07:14:54.555678+0000","last_became_peered":"2023-11-26T07:14:54.555678+0000","last_unstale":"2023-11-26T07:14:54.559918+0000","last_undegraded":"2023-11-26T07:14:54.559918+0000","last_fullsized":"2023-11-26T07:14:54.559918+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:10:26.659226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.543043+0000","last_change":"2023-11-26T07:14:54.543043+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.543043+0000","last_undegraded":"2023-11-26T07:14:54.543043+0000","last_fullsized":"2023-11-26T07:14:54.543043+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.561812+0000","last_change":"2023-11-26T07:14:54.556883+0000","last_active":"2023-11-26T07:14:54.561812+0000","last_peered":"2023-11-26T07:14:54.561812+0000","last_clean":"2023-11-26T07:14:54.561812+0000","last_became_active":"2023-11-26T07:14:54.556654+0000","last_became_peered":"2023-11-26T07:14:54.556654+0000","last_unstale":"2023-11-26T07:14:54.561812+0000","last_undegraded":"2023-11-26T07:14:54.561812+0000","last_fullsized":"2023-11-26T07:14:54.561812+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:44.324311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541030+0000","last_change":"2023-11-26T07:14:54.541030+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541030+0000","last_undegraded":"2023-11-26T07:14:54.541030+0000","last_fullsized":"2023-11-26T07:14:54.541030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559901+0000","last_change":"2023-11-26T07:14:54.555216+0000","last_active":"2023-11-26T07:14:54.559901+0000","last_peered":"2023-11-26T07:14:54.559901+0000","last_clean":"2023-11-26T07:14:54.559901+0000","last_became_active":"2023-11-26T07:14:54.554577+0000","last_became_peered":"2023-11-26T07:14:54.554577+0000","last_unstale":"2023-11-26T07:14:54.559901+0000","last_undegraded":"2023-11-26T07:14:54.559901+0000","last_fullsized":"2023-11-26T07:14:54.559901+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:58.113764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.546852+0000","last_change":"2023-11-26T07:14:54.546852+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.546852+0000","last_undegraded":"2023-11-26T07:14:54.546852+0000","last_fullsized":"2023-11-26T07:14:54.546852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.561825+0000","last_change":"2023-11-26T07:14:54.556996+0000","last_active":"2023-11-26T07:14:54.561825+0000","last_peered":"2023-11-26T07:14:54.561825+0000","last_clean":"2023-11-26T07:14:54.561825+0000","last_became_active":"2023-11-26T07:14:54.556722+0000","last_became_peered":"2023-11-26T07:14:54.556722+0000","last_unstale":"2023-11-26T07:14:54.561825+0000","last_undegraded":"2023-11-26T07:14:54.561825+0000","last_fullsized":"2023-11-26T07:14:54.561825+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:21.473873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.542996+0000","last_change":"2023-11-26T07:14:54.542996+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.542996+0000","last_undegraded":"2023-11-26T07:14:54.542996+0000","last_fullsized":"2023-11-26T07:14:54.542996+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.555720+0000","last_change":"2023-11-26T07:14:54.550363+0000","last_active":"2023-11-26T07:14:54.555720+0000","last_peered":"2023-11-26T07:14:54.555720+0000","last_clean":"2023-11-26T07:14:54.555720+0000","last_became_active":"2023-11-26T07:14:54.550093+0000","last_became_peered":"2023-11-26T07:14:54.550093+0000","last_unstale":"2023-11-26T07:14:54.555720+0000","last_undegraded":"2023-11-26T07:14:54.555720+0000","last_fullsized":"2023-11-26T07:14:54.555720+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:30.521505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.543958+0000","last_change":"2023-11-26T07:14:54.543958+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.543958+0000","last_undegraded":"2023-11-26T07:14:54.543958+0000","last_fullsized":"2023-11-26T07:14:54.543958+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559599+0000","last_change":"2023-11-26T07:14:54.552852+0000","last_active":"2023-11-26T07:14:54.559599+0000","last_peered":"2023-11-26T07:14:54.559599+0000","last_clean":"2023-11-26T07:14:54.559599+0000","last_became_active":"2023-11-26T07:14:54.552543+0000","last_became_peered":"2023-11-26T07:14:54.552543+0000","last_unstale":"2023-11-26T07:14:54.559599+0000","last_undegraded":"2023-11-26T07:14:54.559599+0000","last_fullsized":"2023-11-26T07:14:54.559599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:30.337901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-11-26T07:14:53.538375+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:14:53.538375+0000","last_peered":"2023-11-26T07:14:53.538375+0000","last_clean":"2023-11-26T07:14:53.538375+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:14:53.538375+0000","last_undegraded":"2023-11-26T07:14:53.538375+0000","last_fullsized":"2023-11-26T07:14:53.538375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":867098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.55900000000000005},"min":{"1min":0.44,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.52200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.47299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73799999999999999,"5min":1.048,"15min":1.048},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.497,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.65200000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.495,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.621,"15min":0.622},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75600000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63800000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70499999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.623},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77800000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.59299999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76700000000000002,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.41499999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.37,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79500000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79000000000000004,"5min":1.171,"15min":1.171},"last":0.52100000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.74199999999999999,"5min":1.361,"15min":1.361},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78000000000000003,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":276730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.39200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.73699999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76600000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.49099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.69799999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.45800000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72299999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82599999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.753,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70399999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56799999999999995,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67900000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.51200000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.747,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73399999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":270463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.502,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.34899999999999998,"15min":0.34599999999999997},"min":{"1min":0.20000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.5,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.25900000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34000000000000002,"15min":0.33400000000000002},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.27200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53000000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.39500000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.252,"5min":0.246,"15min":0.246},"max":{"1min":0.57499999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.434,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.32300000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.877,"15min":0.877},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.44900000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.623,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":0.995,"15min":0.995},"last":0.499},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.46800000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55400000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88800000000000001,"5min":1.129,"15min":1.129},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.627,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47999999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47399999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83599999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1265664,"data_stored":867098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.221,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54200000000000004,"5min":0.623,"15min":0.623},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.311}]},{"osd":2,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.629,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.374},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.28599999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.36099999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61699999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.437,"15min":0.439},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41299999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67000000000000004,"15min":0.67300000000000004},"min":{"1min":0.499,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96599999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60499999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.499},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57699999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.625},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83799999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.51200000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45100000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55700000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":659456,"data_stored":270463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36099999999999999,"15min":0.36899999999999999},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.373},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.497,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.27700000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.27900000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58799999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.27400000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.56499999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.31},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42599999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54000000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.29599999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64800000000000002,"15min":0.67600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85799999999999998,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79300000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.621,"15min":0.624},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81399999999999995,"5min":1.075,"15min":1.075},"last":0.443},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.38200000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80100000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.47099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.77900000000000003,"5min":1.03,"15min":1.03},"last":0.52700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.65900000000000003},"min":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.622,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82799999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.502}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":671744,"data_stored":276730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.218,"5min":0.188,"15min":0.188},"max":{"1min":0.42199999999999999,"5min":1.284,"15min":1.284},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.44900000000000001,"15min":0.47999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.55500000000000005,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.47499999999999998,"15min":0.52700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61799999999999999,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.46500000000000002,"15min":0.498},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.50700000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.48799999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.50800000000000001},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.59299999999999997,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52700000000000002,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.44}]},{"osd":4,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.68000000000000005},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85299999999999998,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88200000000000001,"5min":1.405,"15min":1.405},"last":0.66300000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.50700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83799999999999997,"5min":1.377,"15min":1.377},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":0.80900000000000005},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91100000000000003,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.60899999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66200000000000003,"15min":0.67500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92500000000000004,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.75600000000000001,"15min":0.80200000000000005},"min":{"1min":0.51700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.871,"5min":4.883,"15min":4.883},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.75700000000000001,"15min":0.81899999999999995},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80200000000000005,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71599999999999997,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85299999999999998,"5min":2.923,"15min":2.923},"last":0.55200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":276730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.41499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77500000000000002,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.43099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.36499999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.53000000000000003,"15min":0.51200000000000001},"min":{"1min":0.44700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77500000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.63400000000000001,"15min":0.66400000000000003},"min":{"1min":0.45600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71099999999999997,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.41499999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54300000000000004,"15min":0.52500000000000002},"min":{"1min":0.47899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.68600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.64700000000000002,"15min":0.67900000000000005},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74299999999999999,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.51000000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.40500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75900000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73399999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.58399999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.54300000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.497},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72999999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71699999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.625,"15min":0.63600000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79600000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":270463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.39200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.49099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.35999999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.48799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.73799999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.42399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.44800000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.45900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64700000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.498,"15min":0.5},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.61599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67300000000000004,"5min":0.752,"15min":0.752},"last":0.49299999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.65800000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70899999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.66000000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63},{"interface":"front","average":{"1min":0.625,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76600000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:15:00.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:15:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.206+0000 7fb08dc4e700 1 Processor -- start 2023-11-26T07:15:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.206+0000 7fb08dc4e700 1 -- start start 2023-11-26T07:15:00.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb08dc4e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb0880af4d0 0x7fb0880b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb08dc4e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0880ae430 con 0x7fb08806c710 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb08dc4e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0880ae710 con 0x7fb08815e080 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb08dc4e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0880af0b0 con 0x7fb0880af4d0 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb086ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb08806c710 0x7fb08806e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58502/0 (socket says 172.21.15.83:58502) 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb086ffd700 1 -- 172.21.15.83:0/3581792511 learned_addr learned my addr 172.21.15.83:0/3581792511 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613627438 0 0) 0x7fb0880ae430 con 0x7fb08806c710 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0700035b0 con 0x7fb08806c710 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb087fff700 1 --2- 172.21.15.83:0/3581792511 >> v2:172.21.15.203:3300/0 conn(0x7fb0880af4d0 0x7fb0880b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3497643779 0 0) 0x7fb0700035b0 con 0x7fb08806c710 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 >> v2:172.21.15.203:3300/0 conn(0x7fb0880af4d0 msgr2=0x7fb0880b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:00.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 --2- 172.21.15.83:0/3581792511 >> v2:172.21.15.203:3300/0 conn(0x7fb0880af4d0 0x7fb0880b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 >> v1:172.21.15.203:6789/0 conn(0x7fb08815e080 legacy=0x7fb08806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0880ae9f0 con 0x7fb08806c710 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fb07c001670 con 0x7fb08806c710 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb087fff700 1 --2- 172.21.15.83:0/3581792511 >> v2:172.21.15.203:3300/0 conn(0x7fb0880af4d0 0x7fb0880b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 >> v1:172.21.15.83:6789/0 conn(0x7fb08806c710 legacy=0x7fb08806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb070003a00 0x7fb070003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb070004850 0x7fb070004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070005210 0x7fb07000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880af0b0 con 0x7fb070005210 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07000c160 con 0x7fb070004850 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07000c320 con 0x7fb070003a00 2023-11-26T07:15:00.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.207+0000 7fb087fff700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070005210 0x7fb07000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb086ffd700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb070004850 0x7fb070004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb0877fe700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb070003a00 0x7fb070003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb087fff700 1 -- 172.21.15.83:0/3581792511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb070003a00 msgr2=0x7fb070003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb087fff700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb070003a00 0x7fb070003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb087fff700 1 -- 172.21.15.83:0/3581792511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb070004850 msgr2=0x7fb070004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb087fff700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb070004850 0x7fb070004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb087fff700 1 -- 172.21.15.83:0/3581792511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0700035b0 con 0x7fb070005210 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb0877fe700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb070003a00 0x7fb070003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb087fff700 1 -- 172.21.15.83:0/3581792511 reap_dead start 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb087fff700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070005210 0x7fb07000ba10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb0800073e0 tx=0x7fb070005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560737be8f9df768 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080009490 con 0x7fb070005210 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080014400 con 0x7fb070005210 2023-11-26T07:15:00.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.208+0000 7fb0867fc700 1 -- 172.21.15.83:0/3581792511 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0800147b0 con 0x7fb070005210 2023-11-26T07:15:00.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.209+0000 7fb08dc4e700 1 -- 172.21.15.83:0/3581792511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070005210 msgr2=0x7fb07000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.209+0000 7fb08dc4e700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070005210 0x7fb07000ba10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb0800073e0 tx=0x7fb070005880 comp rx=0 tx=0).stop 2023-11-26T07:15:00.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.209+0000 7fb08dc4e700 1 -- 172.21.15.83:0/3581792511 shutdown_connections 2023-11-26T07:15:00.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.209+0000 7fb08dc4e700 1 --2- 172.21.15.83:0/3581792511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070005210 0x7fb07000ba10 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.209+0000 7fb08dc4e700 1 -- 172.21.15.83:0/3581792511 >> 172.21.15.83:0/3581792511 conn(0x7fb088071b90 msgr2=0x7fb088071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:00.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.209+0000 7fb08dc4e700 1 -- 172.21.15.83:0/3581792511 shutdown_connections 2023-11-26T07:15:00.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.209+0000 7fb08dc4e700 1 -- 172.21.15.83:0/3581792511 wait complete. 2023-11-26T07:15:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.210+0000 7fb08dc4e700 1 Processor -- start 2023-11-26T07:15:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.210+0000 7fb08dc4e700 1 -- start start 2023-11-26T07:15:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.211+0000 7fb08dc4e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb070005210 0x7fb088151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.211+0000 7fb08dc4e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0880d0540 0x7fb08815b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.211+0000 7fb08dc4e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0881567e0 0x7fb088142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.211+0000 7fb08dc4e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb088155510 con 0x7fb0881567e0 2023-11-26T07:15:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.211+0000 7fb08dc4e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb088155740 con 0x7fb070005210 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.211+0000 7fb08dc4e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb088155a20 con 0x7fb0880d0540 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.211+0000 7fb087fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0881567e0 0x7fb088142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.211+0000 7fb087fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0881567e0 0x7fb088142220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38476/0 (socket says 172.21.15.83:38476) 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.211+0000 7fb087fff700 1 -- 172.21.15.83:0/2095583839 learned_addr learned my addr 172.21.15.83:0/2095583839 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb0877fe700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb070005210 0x7fb088151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb086ffd700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0880d0540 0x7fb08815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb087fff700 1 -- 172.21.15.83:0/2095583839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0880d0540 msgr2=0x7fb08815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb087fff700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0880d0540 0x7fb08815b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb087fff700 1 -- 172.21.15.83:0/2095583839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb070005210 msgr2=0x7fb088151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb087fff700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb070005210 0x7fb088151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb087fff700 1 -- 172.21.15.83:0/2095583839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb088155d00 con 0x7fb0881567e0 2023-11-26T07:15:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb087fff700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0881567e0 0x7fb088142220 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb088161a70 tx=0x7fb080004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb06ffff700 1 -- 172.21.15.83:0/2095583839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080007000 con 0x7fb0881567e0 2023-11-26T07:15:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb06ffff700 1 -- 172.21.15.83:0/2095583839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080009450 con 0x7fb0881567e0 2023-11-26T07:15:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb08dc4e700 1 -- 172.21.15.83:0/2095583839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb088155fe0 con 0x7fb0881567e0 2023-11-26T07:15:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb06ffff700 1 -- 172.21.15.83:0/2095583839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080003e10 con 0x7fb0881567e0 2023-11-26T07:15:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.212+0000 7fb08dc4e700 1 -- 172.21.15.83:0/2095583839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0881515d0 con 0x7fb0881567e0 2023-11-26T07:15:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.213+0000 7fb08dc4e700 1 -- 172.21.15.83:0/2095583839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0880753f0 con 0x7fb0881567e0 2023-11-26T07:15:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.213+0000 7fb06ffff700 1 -- 172.21.15.83:0/2095583839 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fb08001b040 con 0x7fb0881567e0 2023-11-26T07:15:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.214+0000 7fb06ffff700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07406b080 0x7fb07406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.214+0000 7fb06ffff700 1 -- 172.21.15.83:0/2095583839 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb08008ad60 con 0x7fb0881567e0 2023-11-26T07:15:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.214+0000 7fb0877fe700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07406b080 0x7fb07406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.214+0000 7fb0877fe700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07406b080 0x7fb07406d540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb08815bb70 tx=0x7fb078009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.218+0000 7fb06ffff700 1 -- 172.21.15.83:0/2095583839 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb080055d40 con 0x7fb0881567e0 2023-11-26T07:15:00.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.407+0000 7fb08dc4e700 1 -- 172.21.15.83:0/2095583839 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb088155b90 con 0x7fb07406b080 2023-11-26T07:15:00.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.415+0000 7fb06ffff700 1 -- 172.21.15.83:0/2095583839 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289151 (secure 0 0 0) 0x7fb088155b90 con 0x7fb07406b080 2023-11-26T07:15:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:00.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 -- 172.21.15.83:0/2095583839 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07406b080 msgr2=0x7fb07406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07406b080 0x7fb07406d540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb08815bb70 tx=0x7fb078009040 comp rx=0 tx=0).stop 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 -- 172.21.15.83:0/2095583839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0881567e0 msgr2=0x7fb088142220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0881567e0 0x7fb088142220 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb088161a70 tx=0x7fb080004b40 comp rx=0 tx=0).stop 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 -- 172.21.15.83:0/2095583839 shutdown_connections 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0881567e0 0x7fb088142220 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb070005210 0x7fb088151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb07406b080 0x7fb07406d540 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 --2- 172.21.15.83:0/2095583839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0880d0540 0x7fb08815b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 -- 172.21.15.83:0/2095583839 >> 172.21.15.83:0/2095583839 conn(0x7fb088071b90 msgr2=0x7fb088073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 -- 172.21.15.83:0/2095583839 shutdown_connections 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.418+0000 7fb08dc4e700 1 -- 172.21.15.83:0/2095583839 wait complete. 2023-11-26T07:15:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:15:00.431 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-11-26T07:15:00.353498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222832,"kb_used_data":6640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751888,"statfs":{"total":773094113280,"available":772865933312,"internally_reserved":0,"allocated":6799360,"data_stored":3490921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000895"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542403+0000","last_change":"2023-11-26T07:14:55.565782+0000","last_active":"2023-11-26T07:14:56.542403+0000","last_peered":"2023-11-26T07:14:56.542403+0000","last_clean":"2023-11-26T07:14:56.542403+0000","last_became_active":"2023-11-26T07:14:55.565549+0000","last_became_peered":"2023-11-26T07:14:55.565549+0000","last_unstale":"2023-11-26T07:14:56.542403+0000","last_undegraded":"2023-11-26T07:14:56.542403+0000","last_fullsized":"2023-11-26T07:14:56.542403+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:31:48.907743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877008+0000","last_change":"2023-11-26T07:14:55.563586+0000","last_active":"2023-11-26T07:14:56.877008+0000","last_peered":"2023-11-26T07:14:56.877008+0000","last_clean":"2023-11-26T07:14:56.877008+0000","last_became_active":"2023-11-26T07:14:55.563306+0000","last_became_peered":"2023-11-26T07:14:55.563306+0000","last_unstale":"2023-11-26T07:14:56.877008+0000","last_undegraded":"2023-11-26T07:14:56.877008+0000","last_fullsized":"2023-11-26T07:14:56.877008+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:04:44.976495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541137+0000","last_change":"2023-11-26T07:14:55.547529+0000","last_active":"2023-11-26T07:14:56.541137+0000","last_peered":"2023-11-26T07:14:56.541137+0000","last_clean":"2023-11-26T07:14:56.541137+0000","last_became_active":"2023-11-26T07:14:55.547248+0000","last_became_peered":"2023-11-26T07:14:55.547248+0000","last_unstale":"2023-11-26T07:14:56.541137+0000","last_undegraded":"2023-11-26T07:14:56.541137+0000","last_fullsized":"2023-11-26T07:14:56.541137+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:48:48.988066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877177+0000","last_change":"2023-11-26T07:14:55.556871+0000","last_active":"2023-11-26T07:14:56.877177+0000","last_peered":"2023-11-26T07:14:56.877177+0000","last_clean":"2023-11-26T07:14:56.877177+0000","last_became_active":"2023-11-26T07:14:55.556600+0000","last_became_peered":"2023-11-26T07:14:55.556600+0000","last_unstale":"2023-11-26T07:14:56.877177+0000","last_undegraded":"2023-11-26T07:14:56.877177+0000","last_fullsized":"2023-11-26T07:14:56.877177+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:57.935087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542195+0000","last_change":"2023-11-26T07:14:55.556264+0000","last_active":"2023-11-26T07:14:56.542195+0000","last_peered":"2023-11-26T07:14:56.542195+0000","last_clean":"2023-11-26T07:14:56.542195+0000","last_became_active":"2023-11-26T07:14:55.555983+0000","last_became_peered":"2023-11-26T07:14:55.555983+0000","last_unstale":"2023-11-26T07:14:56.542195+0000","last_undegraded":"2023-11-26T07:14:56.542195+0000","last_fullsized":"2023-11-26T07:14:56.542195+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:16:59.337538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876724+0000","last_change":"2023-11-26T07:14:55.557549+0000","last_active":"2023-11-26T07:14:56.876724+0000","last_peered":"2023-11-26T07:14:56.876724+0000","last_clean":"2023-11-26T07:14:56.876724+0000","last_became_active":"2023-11-26T07:14:55.557282+0000","last_became_peered":"2023-11-26T07:14:55.557282+0000","last_unstale":"2023-11-26T07:14:56.876724+0000","last_undegraded":"2023-11-26T07:14:56.876724+0000","last_fullsized":"2023-11-26T07:14:56.876724+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:03.665466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877024+0000","last_change":"2023-11-26T07:14:55.554176+0000","last_active":"2023-11-26T07:14:56.877024+0000","last_peered":"2023-11-26T07:14:56.877024+0000","last_clean":"2023-11-26T07:14:56.877024+0000","last_became_active":"2023-11-26T07:14:55.553858+0000","last_became_peered":"2023-11-26T07:14:55.553858+0000","last_unstale":"2023-11-26T07:14:56.877024+0000","last_undegraded":"2023-11-26T07:14:56.877024+0000","last_fullsized":"2023-11-26T07:14:56.877024+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:39:44.105610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876382+0000","last_change":"2023-11-26T07:14:55.561614+0000","last_active":"2023-11-26T07:14:56.876382+0000","last_peered":"2023-11-26T07:14:56.876382+0000","last_clean":"2023-11-26T07:14:56.876382+0000","last_became_active":"2023-11-26T07:14:55.561004+0000","last_became_peered":"2023-11-26T07:14:55.561004+0000","last_unstale":"2023-11-26T07:14:56.876382+0000","last_undegraded":"2023-11-26T07:14:56.876382+0000","last_fullsized":"2023-11-26T07:14:56.876382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:36.265114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877062+0000","last_change":"2023-11-26T07:14:55.568602+0000","last_active":"2023-11-26T07:14:56.877062+0000","last_peered":"2023-11-26T07:14:56.877062+0000","last_clean":"2023-11-26T07:14:56.877062+0000","last_became_active":"2023-11-26T07:14:55.568272+0000","last_became_peered":"2023-11-26T07:14:55.568272+0000","last_unstale":"2023-11-26T07:14:56.877062+0000","last_undegraded":"2023-11-26T07:14:56.877062+0000","last_fullsized":"2023-11-26T07:14:56.877062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:40.727595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877245+0000","last_change":"2023-11-26T07:14:55.555237+0000","last_active":"2023-11-26T07:14:56.877245+0000","last_peered":"2023-11-26T07:14:56.877245+0000","last_clean":"2023-11-26T07:14:56.877245+0000","last_became_active":"2023-11-26T07:14:55.554973+0000","last_became_peered":"2023-11-26T07:14:55.554973+0000","last_unstale":"2023-11-26T07:14:56.877245+0000","last_undegraded":"2023-11-26T07:14:56.877245+0000","last_fullsized":"2023-11-26T07:14:56.877245+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:01:53.326813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877295+0000","last_change":"2023-11-26T07:14:55.567135+0000","last_active":"2023-11-26T07:14:56.877295+0000","last_peered":"2023-11-26T07:14:56.877295+0000","last_clean":"2023-11-26T07:14:56.877295+0000","last_became_active":"2023-11-26T07:14:55.566885+0000","last_became_peered":"2023-11-26T07:14:55.566885+0000","last_unstale":"2023-11-26T07:14:56.877295+0000","last_undegraded":"2023-11-26T07:14:56.877295+0000","last_fullsized":"2023-11-26T07:14:56.877295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:25:05.901539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877351+0000","last_change":"2023-11-26T07:14:55.555316+0000","last_active":"2023-11-26T07:14:56.877351+0000","last_peered":"2023-11-26T07:14:56.877351+0000","last_clean":"2023-11-26T07:14:56.877351+0000","last_became_active":"2023-11-26T07:14:55.555043+0000","last_became_peered":"2023-11-26T07:14:55.555043+0000","last_unstale":"2023-11-26T07:14:56.877351+0000","last_undegraded":"2023-11-26T07:14:56.877351+0000","last_fullsized":"2023-11-26T07:14:56.877351+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:01.838876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541763+0000","last_change":"2023-11-26T07:14:54.541763+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541763+0000","last_undegraded":"2023-11-26T07:14:54.541763+0000","last_fullsized":"2023-11-26T07:14:54.541763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876769+0000","last_change":"2023-11-26T07:14:55.557930+0000","last_active":"2023-11-26T07:14:56.876769+0000","last_peered":"2023-11-26T07:14:56.876769+0000","last_clean":"2023-11-26T07:14:56.876769+0000","last_became_active":"2023-11-26T07:14:55.557647+0000","last_became_peered":"2023-11-26T07:14:55.557647+0000","last_unstale":"2023-11-26T07:14:56.876769+0000","last_undegraded":"2023-11-26T07:14:56.876769+0000","last_fullsized":"2023-11-26T07:14:56.876769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:00:31.201849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877113+0000","last_change":"2023-11-26T07:14:55.559368+0000","last_active":"2023-11-26T07:14:56.877113+0000","last_peered":"2023-11-26T07:14:56.877113+0000","last_clean":"2023-11-26T07:14:56.877113+0000","last_became_active":"2023-11-26T07:14:55.558965+0000","last_became_peered":"2023-11-26T07:14:55.558965+0000","last_unstale":"2023-11-26T07:14:56.877113+0000","last_undegraded":"2023-11-26T07:14:56.877113+0000","last_fullsized":"2023-11-26T07:14:56.877113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:41:48.912767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876397+0000","last_change":"2023-11-26T07:14:55.558759+0000","last_active":"2023-11-26T07:14:56.876397+0000","last_peered":"2023-11-26T07:14:56.876397+0000","last_clean":"2023-11-26T07:14:56.876397+0000","last_became_active":"2023-11-26T07:14:55.558507+0000","last_became_peered":"2023-11-26T07:14:55.558507+0000","last_unstale":"2023-11-26T07:14:56.876397+0000","last_undegraded":"2023-11-26T07:14:56.876397+0000","last_fullsized":"2023-11-26T07:14:56.876397+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:57.951234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876827+0000","last_change":"2023-11-26T07:14:55.562951+0000","last_active":"2023-11-26T07:14:56.876827+0000","last_peered":"2023-11-26T07:14:56.876827+0000","last_clean":"2023-11-26T07:14:56.876827+0000","last_became_active":"2023-11-26T07:14:55.562289+0000","last_became_peered":"2023-11-26T07:14:55.562289+0000","last_unstale":"2023-11-26T07:14:56.876827+0000","last_undegraded":"2023-11-26T07:14:56.876827+0000","last_fullsized":"2023-11-26T07:14:56.876827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:47:37.463547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541881+0000","last_change":"2023-11-26T07:14:55.570981+0000","last_active":"2023-11-26T07:14:56.541881+0000","last_peered":"2023-11-26T07:14:56.541881+0000","last_clean":"2023-11-26T07:14:56.541881+0000","last_became_active":"2023-11-26T07:14:55.570671+0000","last_became_peered":"2023-11-26T07:14:55.570671+0000","last_unstale":"2023-11-26T07:14:56.541881+0000","last_undegraded":"2023-11-26T07:14:56.541881+0000","last_fullsized":"2023-11-26T07:14:56.541881+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:15:46.689885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542209+0000","last_change":"2023-11-26T07:14:55.565302+0000","last_active":"2023-11-26T07:14:56.542209+0000","last_peered":"2023-11-26T07:14:56.542209+0000","last_clean":"2023-11-26T07:14:56.542209+0000","last_became_active":"2023-11-26T07:14:55.565070+0000","last_became_peered":"2023-11-26T07:14:55.565070+0000","last_unstale":"2023-11-26T07:14:56.542209+0000","last_undegraded":"2023-11-26T07:14:56.542209+0000","last_fullsized":"2023-11-26T07:14:56.542209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:45.326329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877055+0000","last_change":"2023-11-26T07:14:55.568585+0000","last_active":"2023-11-26T07:14:56.877055+0000","last_peered":"2023-11-26T07:14:56.877055+0000","last_clean":"2023-11-26T07:14:56.877055+0000","last_became_active":"2023-11-26T07:14:55.568162+0000","last_became_peered":"2023-11-26T07:14:55.568162+0000","last_unstale":"2023-11-26T07:14:56.877055+0000","last_undegraded":"2023-11-26T07:14:56.877055+0000","last_fullsized":"2023-11-26T07:14:56.877055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:34.558173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541424+0000","last_change":"2023-11-26T07:14:55.550793+0000","last_active":"2023-11-26T07:14:56.541424+0000","last_peered":"2023-11-26T07:14:56.541424+0000","last_clean":"2023-11-26T07:14:56.541424+0000","last_became_active":"2023-11-26T07:14:55.550509+0000","last_became_peered":"2023-11-26T07:14:55.550509+0000","last_unstale":"2023-11-26T07:14:56.541424+0000","last_undegraded":"2023-11-26T07:14:56.541424+0000","last_fullsized":"2023-11-26T07:14:56.541424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:44:43.988623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876289+0000","last_change":"2023-11-26T07:14:55.563684+0000","last_active":"2023-11-26T07:14:56.876289+0000","last_peered":"2023-11-26T07:14:56.876289+0000","last_clean":"2023-11-26T07:14:56.876289+0000","last_became_active":"2023-11-26T07:14:55.562596+0000","last_became_peered":"2023-11-26T07:14:55.562596+0000","last_unstale":"2023-11-26T07:14:56.876289+0000","last_undegraded":"2023-11-26T07:14:56.876289+0000","last_fullsized":"2023-11-26T07:14:56.876289+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:00.018524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542146+0000","last_change":"2023-11-26T07:14:55.565993+0000","last_active":"2023-11-26T07:14:56.542146+0000","last_peered":"2023-11-26T07:14:56.542146+0000","last_clean":"2023-11-26T07:14:56.542146+0000","last_became_active":"2023-11-26T07:14:55.565743+0000","last_became_peered":"2023-11-26T07:14:55.565743+0000","last_unstale":"2023-11-26T07:14:56.542146+0000","last_undegraded":"2023-11-26T07:14:56.542146+0000","last_fullsized":"2023-11-26T07:14:56.542146+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:36.265114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541275+0000","last_change":"2023-11-26T07:14:54.541275+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541275+0000","last_undegraded":"2023-11-26T07:14:54.541275+0000","last_fullsized":"2023-11-26T07:14:54.541275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542478+0000","last_change":"2023-11-26T07:14:55.560177+0000","last_active":"2023-11-26T07:14:56.542478+0000","last_peered":"2023-11-26T07:14:56.542478+0000","last_clean":"2023-11-26T07:14:56.542478+0000","last_became_active":"2023-11-26T07:14:55.559948+0000","last_became_peered":"2023-11-26T07:14:55.559948+0000","last_unstale":"2023-11-26T07:14:56.542478+0000","last_undegraded":"2023-11-26T07:14:56.542478+0000","last_fullsized":"2023-11-26T07:14:56.542478+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:22:36.861785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877238+0000","last_change":"2023-11-26T07:14:55.559369+0000","last_active":"2023-11-26T07:14:56.877238+0000","last_peered":"2023-11-26T07:14:56.877238+0000","last_clean":"2023-11-26T07:14:56.877238+0000","last_became_active":"2023-11-26T07:14:55.559088+0000","last_became_peered":"2023-11-26T07:14:55.559088+0000","last_unstale":"2023-11-26T07:14:56.877238+0000","last_undegraded":"2023-11-26T07:14:56.877238+0000","last_fullsized":"2023-11-26T07:14:56.877238+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:02.536615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877673+0000","last_change":"2023-11-26T07:14:55.569078+0000","last_active":"2023-11-26T07:14:56.877673+0000","last_peered":"2023-11-26T07:14:56.877673+0000","last_clean":"2023-11-26T07:14:56.877673+0000","last_became_active":"2023-11-26T07:14:55.568579+0000","last_became_peered":"2023-11-26T07:14:55.568579+0000","last_unstale":"2023-11-26T07:14:56.877673+0000","last_undegraded":"2023-11-26T07:14:56.877673+0000","last_fullsized":"2023-11-26T07:14:56.877673+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:01.838876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542281+0000","last_change":"2023-11-26T07:14:55.562903+0000","last_active":"2023-11-26T07:14:56.542281+0000","last_peered":"2023-11-26T07:14:56.542281+0000","last_clean":"2023-11-26T07:14:56.542281+0000","last_became_active":"2023-11-26T07:14:55.562637+0000","last_became_peered":"2023-11-26T07:14:55.562637+0000","last_unstale":"2023-11-26T07:14:56.542281+0000","last_undegraded":"2023-11-26T07:14:56.542281+0000","last_fullsized":"2023-11-26T07:14:56.542281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:22.475891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876656+0000","last_change":"2023-11-26T07:14:55.566349+0000","last_active":"2023-11-26T07:14:56.876656+0000","last_peered":"2023-11-26T07:14:56.876656+0000","last_clean":"2023-11-26T07:14:56.876656+0000","last_became_active":"2023-11-26T07:14:55.566043+0000","last_became_peered":"2023-11-26T07:14:55.566043+0000","last_unstale":"2023-11-26T07:14:56.876656+0000","last_undegraded":"2023-11-26T07:14:56.876656+0000","last_fullsized":"2023-11-26T07:14:56.876656+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:07:34.241609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541503+0000","last_change":"2023-11-26T07:14:55.547718+0000","last_active":"2023-11-26T07:14:56.541503+0000","last_peered":"2023-11-26T07:14:56.541503+0000","last_clean":"2023-11-26T07:14:56.541503+0000","last_became_active":"2023-11-26T07:14:55.547432+0000","last_became_peered":"2023-11-26T07:14:55.547432+0000","last_unstale":"2023-11-26T07:14:56.541503+0000","last_undegraded":"2023-11-26T07:14:56.541503+0000","last_fullsized":"2023-11-26T07:14:56.541503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:45:56.436888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542055+0000","last_change":"2023-11-26T07:14:55.560364+0000","last_active":"2023-11-26T07:14:56.542055+0000","last_peered":"2023-11-26T07:14:56.542055+0000","last_clean":"2023-11-26T07:14:56.542055+0000","last_became_active":"2023-11-26T07:14:55.559929+0000","last_became_peered":"2023-11-26T07:14:55.559929+0000","last_unstale":"2023-11-26T07:14:56.542055+0000","last_undegraded":"2023-11-26T07:14:56.542055+0000","last_fullsized":"2023-11-26T07:14:56.542055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:24:20.997710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876524+0000","last_change":"2023-11-26T07:14:55.563485+0000","last_active":"2023-11-26T07:14:56.876524+0000","last_peered":"2023-11-26T07:14:56.876524+0000","last_clean":"2023-11-26T07:14:56.876524+0000","last_became_active":"2023-11-26T07:14:55.563222+0000","last_became_peered":"2023-11-26T07:14:55.563222+0000","last_unstale":"2023-11-26T07:14:56.876524+0000","last_undegraded":"2023-11-26T07:14:56.876524+0000","last_fullsized":"2023-11-26T07:14:56.876524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:57.935087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876779+0000","last_change":"2023-11-26T07:14:55.569614+0000","last_active":"2023-11-26T07:14:56.876779+0000","last_peered":"2023-11-26T07:14:56.876779+0000","last_clean":"2023-11-26T07:14:56.876779+0000","last_became_active":"2023-11-26T07:14:55.567970+0000","last_became_peered":"2023-11-26T07:14:55.567970+0000","last_unstale":"2023-11-26T07:14:56.876779+0000","last_undegraded":"2023-11-26T07:14:56.876779+0000","last_fullsized":"2023-11-26T07:14:56.876779+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:02.536615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.569319+0000","last_change":"2023-11-26T07:14:54.551197+0000","last_active":"2023-11-26T07:14:55.569319+0000","last_peered":"2023-11-26T07:14:55.569319+0000","last_clean":"2023-11-26T07:14:55.569319+0000","last_became_active":"2023-11-26T07:14:54.550930+0000","last_became_peered":"2023-11-26T07:14:54.550930+0000","last_unstale":"2023-11-26T07:14:55.569319+0000","last_undegraded":"2023-11-26T07:14:55.569319+0000","last_fullsized":"2023-11-26T07:14:55.569319+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:24.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877481+0000","last_change":"2023-11-26T07:14:55.554217+0000","last_active":"2023-11-26T07:14:56.877481+0000","last_peered":"2023-11-26T07:14:56.877481+0000","last_clean":"2023-11-26T07:14:56.877481+0000","last_became_active":"2023-11-26T07:14:55.553893+0000","last_became_peered":"2023-11-26T07:14:55.553893+0000","last_unstale":"2023-11-26T07:14:56.877481+0000","last_undegraded":"2023-11-26T07:14:56.877481+0000","last_fullsized":"2023-11-26T07:14:56.877481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:02:52.062049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.566035+0000","last_change":"2023-11-26T07:14:54.556263+0000","last_active":"2023-11-26T07:14:55.566035+0000","last_peered":"2023-11-26T07:14:55.566035+0000","last_clean":"2023-11-26T07:14:55.566035+0000","last_became_active":"2023-11-26T07:14:54.555815+0000","last_became_peered":"2023-11-26T07:14:54.555815+0000","last_unstale":"2023-11-26T07:14:55.566035+0000","last_undegraded":"2023-11-26T07:14:55.566035+0000","last_fullsized":"2023-11-26T07:14:55.566035+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:13:38.502900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876491+0000","last_change":"2023-11-26T07:14:55.570096+0000","last_active":"2023-11-26T07:14:56.876491+0000","last_peered":"2023-11-26T07:14:56.876491+0000","last_clean":"2023-11-26T07:14:56.876491+0000","last_became_active":"2023-11-26T07:14:55.567546+0000","last_became_peered":"2023-11-26T07:14:55.567546+0000","last_unstale":"2023-11-26T07:14:56.876491+0000","last_undegraded":"2023-11-26T07:14:56.876491+0000","last_fullsized":"2023-11-26T07:14:56.876491+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:38.960124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.568272+0000","last_change":"2023-11-26T07:14:54.553820+0000","last_active":"2023-11-26T07:14:55.568272+0000","last_peered":"2023-11-26T07:14:55.568272+0000","last_clean":"2023-11-26T07:14:55.568272+0000","last_became_active":"2023-11-26T07:14:54.553552+0000","last_became_peered":"2023-11-26T07:14:54.553552+0000","last_unstale":"2023-11-26T07:14:55.568272+0000","last_undegraded":"2023-11-26T07:14:55.568272+0000","last_fullsized":"2023-11-26T07:14:55.568272+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:19.097609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877677+0000","last_change":"2023-11-26T07:14:55.568824+0000","last_active":"2023-11-26T07:14:56.877677+0000","last_peered":"2023-11-26T07:14:56.877677+0000","last_clean":"2023-11-26T07:14:56.877677+0000","last_became_active":"2023-11-26T07:14:55.568239+0000","last_became_peered":"2023-11-26T07:14:55.568239+0000","last_unstale":"2023-11-26T07:14:56.877677+0000","last_undegraded":"2023-11-26T07:14:56.877677+0000","last_fullsized":"2023-11-26T07:14:56.877677+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:48:48.988066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.561647+0000","last_change":"2023-11-26T07:14:54.555832+0000","last_active":"2023-11-26T07:14:55.561647+0000","last_peered":"2023-11-26T07:14:55.561647+0000","last_clean":"2023-11-26T07:14:55.561647+0000","last_became_active":"2023-11-26T07:14:54.555514+0000","last_became_peered":"2023-11-26T07:14:54.555514+0000","last_unstale":"2023-11-26T07:14:55.561647+0000","last_undegraded":"2023-11-26T07:14:55.561647+0000","last_fullsized":"2023-11-26T07:14:55.561647+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:24.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876443+0000","last_change":"2023-11-26T07:14:55.562781+0000","last_active":"2023-11-26T07:14:56.876443+0000","last_peered":"2023-11-26T07:14:56.876443+0000","last_clean":"2023-11-26T07:14:56.876443+0000","last_became_active":"2023-11-26T07:14:55.562323+0000","last_became_peered":"2023-11-26T07:14:55.562323+0000","last_unstale":"2023-11-26T07:14:56.876443+0000","last_undegraded":"2023-11-26T07:14:56.876443+0000","last_fullsized":"2023-11-26T07:14:56.876443+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:13.425773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.566845+0000","last_change":"2023-11-26T07:14:54.554968+0000","last_active":"2023-11-26T07:14:55.566845+0000","last_peered":"2023-11-26T07:14:55.566845+0000","last_clean":"2023-11-26T07:14:55.566845+0000","last_became_active":"2023-11-26T07:14:54.554664+0000","last_became_peered":"2023-11-26T07:14:54.554664+0000","last_unstale":"2023-11-26T07:14:55.566845+0000","last_undegraded":"2023-11-26T07:14:55.566845+0000","last_fullsized":"2023-11-26T07:14:55.566845+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:28.707932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876789+0000","last_change":"2023-11-26T07:14:55.568976+0000","last_active":"2023-11-26T07:14:56.876789+0000","last_peered":"2023-11-26T07:14:56.876789+0000","last_clean":"2023-11-26T07:14:56.876789+0000","last_became_active":"2023-11-26T07:14:55.568601+0000","last_became_peered":"2023-11-26T07:14:55.568601+0000","last_unstale":"2023-11-26T07:14:56.876789+0000","last_undegraded":"2023-11-26T07:14:56.876789+0000","last_fullsized":"2023-11-26T07:14:56.876789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:09.743461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.567289+0000","last_change":"2023-11-26T07:14:54.545826+0000","last_active":"2023-11-26T07:14:55.567289+0000","last_peered":"2023-11-26T07:14:55.567289+0000","last_clean":"2023-11-26T07:14:55.567289+0000","last_became_active":"2023-11-26T07:14:54.545558+0000","last_became_peered":"2023-11-26T07:14:54.545558+0000","last_unstale":"2023-11-26T07:14:55.567289+0000","last_undegraded":"2023-11-26T07:14:55.567289+0000","last_fullsized":"2023-11-26T07:14:55.567289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:56.667181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540783+0000","last_change":"2023-11-26T07:14:54.540783+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540783+0000","last_undegraded":"2023-11-26T07:14:54.540783+0000","last_fullsized":"2023-11-26T07:14:54.540783+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.548344+0000","last_change":"2023-11-26T07:14:54.555979+0000","last_active":"2023-11-26T07:14:55.548344+0000","last_peered":"2023-11-26T07:14:55.548344+0000","last_clean":"2023-11-26T07:14:55.548344+0000","last_became_active":"2023-11-26T07:14:54.555658+0000","last_became_peered":"2023-11-26T07:14:54.555658+0000","last_unstale":"2023-11-26T07:14:55.548344+0000","last_undegraded":"2023-11-26T07:14:55.548344+0000","last_fullsized":"2023-11-26T07:14:55.548344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:12.852910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876522+0000","last_change":"2023-11-26T07:14:55.564714+0000","last_active":"2023-11-26T07:14:56.876522+0000","last_peered":"2023-11-26T07:14:56.876522+0000","last_clean":"2023-11-26T07:14:56.876522+0000","last_became_active":"2023-11-26T07:14:55.564443+0000","last_became_peered":"2023-11-26T07:14:55.564443+0000","last_unstale":"2023-11-26T07:14:56.876522+0000","last_undegraded":"2023-11-26T07:14:56.876522+0000","last_fullsized":"2023-11-26T07:14:56.876522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:07:30.647501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.541291+0000","last_change":"2023-11-26T07:14:54.556906+0000","last_active":"2023-11-26T07:14:55.541291+0000","last_peered":"2023-11-26T07:14:55.541291+0000","last_clean":"2023-11-26T07:14:55.541291+0000","last_became_active":"2023-11-26T07:14:54.556633+0000","last_became_peered":"2023-11-26T07:14:54.556633+0000","last_unstale":"2023-11-26T07:14:55.541291+0000","last_undegraded":"2023-11-26T07:14:55.541291+0000","last_fullsized":"2023-11-26T07:14:55.541291+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:10:48.528921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877149+0000","last_change":"2023-11-26T07:14:55.561086+0000","last_active":"2023-11-26T07:14:56.877149+0000","last_peered":"2023-11-26T07:14:56.877149+0000","last_clean":"2023-11-26T07:14:56.877149+0000","last_became_active":"2023-11-26T07:14:55.560811+0000","last_became_peered":"2023-11-26T07:14:55.560811+0000","last_unstale":"2023-11-26T07:14:56.877149+0000","last_undegraded":"2023-11-26T07:14:56.877149+0000","last_fullsized":"2023-11-26T07:14:56.877149+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:57.669199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.541992+0000","last_change":"2023-11-26T07:14:54.551547+0000","last_active":"2023-11-26T07:14:55.541992+0000","last_peered":"2023-11-26T07:14:55.541992+0000","last_clean":"2023-11-26T07:14:55.541992+0000","last_became_active":"2023-11-26T07:14:54.551059+0000","last_became_peered":"2023-11-26T07:14:54.551059+0000","last_unstale":"2023-11-26T07:14:55.541992+0000","last_undegraded":"2023-11-26T07:14:55.541992+0000","last_fullsized":"2023-11-26T07:14:55.541992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:56.933069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541907+0000","last_change":"2023-11-26T07:14:55.554669+0000","last_active":"2023-11-26T07:14:56.541907+0000","last_peered":"2023-11-26T07:14:56.541907+0000","last_clean":"2023-11-26T07:14:56.541907+0000","last_became_active":"2023-11-26T07:14:55.554402+0000","last_became_peered":"2023-11-26T07:14:55.554402+0000","last_unstale":"2023-11-26T07:14:56.541907+0000","last_undegraded":"2023-11-26T07:14:56.541907+0000","last_fullsized":"2023-11-26T07:14:56.541907+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:01:27.228579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.548485+0000","last_change":"2023-11-26T07:14:54.556255+0000","last_active":"2023-11-26T07:14:55.548485+0000","last_peered":"2023-11-26T07:14:55.548485+0000","last_clean":"2023-11-26T07:14:55.548485+0000","last_became_active":"2023-11-26T07:14:54.555863+0000","last_became_peered":"2023-11-26T07:14:54.555863+0000","last_unstale":"2023-11-26T07:14:55.548485+0000","last_undegraded":"2023-11-26T07:14:55.548485+0000","last_fullsized":"2023-11-26T07:14:55.548485+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:49.214175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876808+0000","last_change":"2023-11-26T07:14:55.555431+0000","last_active":"2023-11-26T07:14:56.876808+0000","last_peered":"2023-11-26T07:14:56.876808+0000","last_clean":"2023-11-26T07:14:56.876808+0000","last_became_active":"2023-11-26T07:14:55.555090+0000","last_became_peered":"2023-11-26T07:14:55.555090+0000","last_unstale":"2023-11-26T07:14:56.876808+0000","last_undegraded":"2023-11-26T07:14:56.876808+0000","last_fullsized":"2023-11-26T07:14:56.876808+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:28:00.260761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.571669+0000","last_change":"2023-11-26T07:14:54.553992+0000","last_active":"2023-11-26T07:14:55.571669+0000","last_peered":"2023-11-26T07:14:55.571669+0000","last_clean":"2023-11-26T07:14:55.571669+0000","last_became_active":"2023-11-26T07:14:54.553718+0000","last_became_peered":"2023-11-26T07:14:54.553718+0000","last_unstale":"2023-11-26T07:14:55.571669+0000","last_undegraded":"2023-11-26T07:14:55.571669+0000","last_fullsized":"2023-11-26T07:14:55.571669+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:02.663448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541322+0000","last_change":"2023-11-26T07:14:55.556639+0000","last_active":"2023-11-26T07:14:56.541322+0000","last_peered":"2023-11-26T07:14:56.541322+0000","last_clean":"2023-11-26T07:14:56.541322+0000","last_became_active":"2023-11-26T07:14:55.556441+0000","last_became_peered":"2023-11-26T07:14:55.556441+0000","last_unstale":"2023-11-26T07:14:56.541322+0000","last_undegraded":"2023-11-26T07:14:56.541322+0000","last_fullsized":"2023-11-26T07:14:56.541322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:59.370564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.574692+0000","last_change":"2023-11-26T07:14:54.557028+0000","last_active":"2023-11-26T07:14:55.574692+0000","last_peered":"2023-11-26T07:14:55.574692+0000","last_clean":"2023-11-26T07:14:55.574692+0000","last_became_active":"2023-11-26T07:14:54.556758+0000","last_became_peered":"2023-11-26T07:14:54.556758+0000","last_unstale":"2023-11-26T07:14:55.574692+0000","last_undegraded":"2023-11-26T07:14:55.574692+0000","last_fullsized":"2023-11-26T07:14:55.574692+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:58:50.594707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876740+0000","last_change":"2023-11-26T07:14:55.562711+0000","last_active":"2023-11-26T07:14:56.876740+0000","last_peered":"2023-11-26T07:14:56.876740+0000","last_clean":"2023-11-26T07:14:56.876740+0000","last_became_active":"2023-11-26T07:14:55.562505+0000","last_became_peered":"2023-11-26T07:14:55.562505+0000","last_unstale":"2023-11-26T07:14:56.876740+0000","last_undegraded":"2023-11-26T07:14:56.876740+0000","last_fullsized":"2023-11-26T07:14:56.876740+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:57.951234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.573668+0000","last_change":"2023-11-26T07:14:54.554993+0000","last_active":"2023-11-26T07:14:55.573668+0000","last_peered":"2023-11-26T07:14:55.573668+0000","last_clean":"2023-11-26T07:14:55.573668+0000","last_became_active":"2023-11-26T07:14:54.554750+0000","last_became_peered":"2023-11-26T07:14:54.554750+0000","last_unstale":"2023-11-26T07:14:55.573668+0000","last_undegraded":"2023-11-26T07:14:55.573668+0000","last_fullsized":"2023-11-26T07:14:55.573668+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:15:45.687867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541558+0000","last_change":"2023-11-26T07:14:55.552259+0000","last_active":"2023-11-26T07:14:56.541558+0000","last_peered":"2023-11-26T07:14:56.541558+0000","last_clean":"2023-11-26T07:14:56.541558+0000","last_became_active":"2023-11-26T07:14:55.551888+0000","last_became_peered":"2023-11-26T07:14:55.551888+0000","last_unstale":"2023-11-26T07:14:56.541558+0000","last_undegraded":"2023-11-26T07:14:56.541558+0000","last_fullsized":"2023-11-26T07:14:56.541558+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:13:39.504918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559486+0000","last_change":"2023-11-26T07:14:54.552221+0000","last_active":"2023-11-26T07:14:54.559486+0000","last_peered":"2023-11-26T07:14:54.559486+0000","last_clean":"2023-11-26T07:14:54.559486+0000","last_became_active":"2023-11-26T07:14:54.551932+0000","last_became_peered":"2023-11-26T07:14:54.551932+0000","last_unstale":"2023-11-26T07:14:54.559486+0000","last_undegraded":"2023-11-26T07:14:54.559486+0000","last_fullsized":"2023-11-26T07:14:54.559486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:00:25.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876587+0000","last_change":"2023-11-26T07:14:55.553875+0000","last_active":"2023-11-26T07:14:56.876587+0000","last_peered":"2023-11-26T07:14:56.876587+0000","last_clean":"2023-11-26T07:14:56.876587+0000","last_became_active":"2023-11-26T07:14:55.553581+0000","last_became_peered":"2023-11-26T07:14:55.553581+0000","last_unstale":"2023-11-26T07:14:56.876587+0000","last_undegraded":"2023-11-26T07:14:56.876587+0000","last_fullsized":"2023-11-26T07:14:56.876587+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:42.254062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.548315+0000","last_change":"2023-11-26T07:14:54.553341+0000","last_active":"2023-11-26T07:14:55.548315+0000","last_peered":"2023-11-26T07:14:55.548315+0000","last_clean":"2023-11-26T07:14:55.548315+0000","last_became_active":"2023-11-26T07:14:54.551872+0000","last_became_peered":"2023-11-26T07:14:54.551872+0000","last_unstale":"2023-11-26T07:14:55.548315+0000","last_undegraded":"2023-11-26T07:14:55.548315+0000","last_fullsized":"2023-11-26T07:14:55.548315+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:42.288092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541270+0000","last_change":"2023-11-26T07:14:55.566027+0000","last_active":"2023-11-26T07:14:56.541270+0000","last_peered":"2023-11-26T07:14:56.541270+0000","last_clean":"2023-11-26T07:14:56.541270+0000","last_became_active":"2023-11-26T07:14:55.565770+0000","last_became_peered":"2023-11-26T07:14:55.565770+0000","last_unstale":"2023-11-26T07:14:56.541270+0000","last_undegraded":"2023-11-26T07:14:56.541270+0000","last_fullsized":"2023-11-26T07:14:56.541270+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:37:15.036624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.566051+0000","last_change":"2023-11-26T07:14:54.551340+0000","last_active":"2023-11-26T07:14:55.566051+0000","last_peered":"2023-11-26T07:14:55.566051+0000","last_clean":"2023-11-26T07:14:55.566051+0000","last_became_active":"2023-11-26T07:14:54.551030+0000","last_became_peered":"2023-11-26T07:14:54.551030+0000","last_unstale":"2023-11-26T07:14:55.566051+0000","last_undegraded":"2023-11-26T07:14:55.566051+0000","last_fullsized":"2023-11-26T07:14:55.566051+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:24.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541944+0000","last_change":"2023-11-26T07:14:55.547115+0000","last_active":"2023-11-26T07:14:56.541944+0000","last_peered":"2023-11-26T07:14:56.541944+0000","last_clean":"2023-11-26T07:14:56.541944+0000","last_became_active":"2023-11-26T07:14:55.546831+0000","last_became_peered":"2023-11-26T07:14:55.546831+0000","last_unstale":"2023-11-26T07:14:56.541944+0000","last_undegraded":"2023-11-26T07:14:56.541944+0000","last_fullsized":"2023-11-26T07:14:56.541944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:00.018524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.569880+0000","last_change":"2023-11-26T07:14:54.558605+0000","last_active":"2023-11-26T07:14:55.569880+0000","last_peered":"2023-11-26T07:14:55.569880+0000","last_clean":"2023-11-26T07:14:55.569880+0000","last_became_active":"2023-11-26T07:14:54.558324+0000","last_became_peered":"2023-11-26T07:14:54.558324+0000","last_unstale":"2023-11-26T07:14:55.569880+0000","last_undegraded":"2023-11-26T07:14:55.569880+0000","last_fullsized":"2023-11-26T07:14:55.569880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:45:55.434870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877336+0000","last_change":"2023-11-26T07:14:55.562179+0000","last_active":"2023-11-26T07:14:56.877336+0000","last_peered":"2023-11-26T07:14:56.877336+0000","last_clean":"2023-11-26T07:14:56.877336+0000","last_became_active":"2023-11-26T07:14:55.561908+0000","last_became_peered":"2023-11-26T07:14:55.561908+0000","last_unstale":"2023-11-26T07:14:56.877336+0000","last_undegraded":"2023-11-26T07:14:56.877336+0000","last_fullsized":"2023-11-26T07:14:56.877336+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:01:50.191654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.567782+0000","last_change":"2023-11-26T07:14:54.556982+0000","last_active":"2023-11-26T07:14:55.567782+0000","last_peered":"2023-11-26T07:14:55.567782+0000","last_clean":"2023-11-26T07:14:55.567782+0000","last_became_active":"2023-11-26T07:14:54.556728+0000","last_became_peered":"2023-11-26T07:14:54.556728+0000","last_unstale":"2023-11-26T07:14:55.567782+0000","last_undegraded":"2023-11-26T07:14:55.567782+0000","last_fullsized":"2023-11-26T07:14:55.567782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:42.288092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877055+0000","last_change":"2023-11-26T07:14:55.564847+0000","last_active":"2023-11-26T07:14:56.877055+0000","last_peered":"2023-11-26T07:14:56.877055+0000","last_clean":"2023-11-26T07:14:56.877055+0000","last_became_active":"2023-11-26T07:14:55.564561+0000","last_became_peered":"2023-11-26T07:14:55.564561+0000","last_unstale":"2023-11-26T07:14:56.877055+0000","last_undegraded":"2023-11-26T07:14:56.877055+0000","last_fullsized":"2023-11-26T07:14:56.877055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:01:52.510162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.561778+0000","last_change":"2023-11-26T07:14:54.553413+0000","last_active":"2023-11-26T07:14:55.561778+0000","last_peered":"2023-11-26T07:14:55.561778+0000","last_clean":"2023-11-26T07:14:55.561778+0000","last_became_active":"2023-11-26T07:14:54.552941+0000","last_became_peered":"2023-11-26T07:14:54.552941+0000","last_unstale":"2023-11-26T07:14:55.561778+0000","last_undegraded":"2023-11-26T07:14:55.561778+0000","last_fullsized":"2023-11-26T07:14:55.561778+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:15:18.343075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541477+0000","last_change":"2023-11-26T07:14:55.559136+0000","last_active":"2023-11-26T07:14:56.541477+0000","last_peered":"2023-11-26T07:14:56.541477+0000","last_clean":"2023-11-26T07:14:56.541477+0000","last_became_active":"2023-11-26T07:14:55.558850+0000","last_became_peered":"2023-11-26T07:14:55.558850+0000","last_unstale":"2023-11-26T07:14:56.541477+0000","last_undegraded":"2023-11-26T07:14:56.541477+0000","last_fullsized":"2023-11-26T07:14:56.541477+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:50.216193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.554475+0000","last_change":"2023-11-26T07:14:54.546741+0000","last_active":"2023-11-26T07:14:55.554475+0000","last_peered":"2023-11-26T07:14:55.554475+0000","last_clean":"2023-11-26T07:14:55.554475+0000","last_became_active":"2023-11-26T07:14:54.546479+0000","last_became_peered":"2023-11-26T07:14:54.546479+0000","last_unstale":"2023-11-26T07:14:55.554475+0000","last_undegraded":"2023-11-26T07:14:55.554475+0000","last_fullsized":"2023-11-26T07:14:55.554475+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:08.574978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876882+0000","last_change":"2023-11-26T07:14:55.563245+0000","last_active":"2023-11-26T07:14:56.876882+0000","last_peered":"2023-11-26T07:14:56.876882+0000","last_clean":"2023-11-26T07:14:56.876882+0000","last_became_active":"2023-11-26T07:14:55.562937+0000","last_became_peered":"2023-11-26T07:14:55.562937+0000","last_unstale":"2023-11-26T07:14:56.876882+0000","last_undegraded":"2023-11-26T07:14:56.876882+0000","last_fullsized":"2023-11-26T07:14:56.876882+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:38.960124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.570157+0000","last_change":"2023-11-26T07:14:54.555163+0000","last_active":"2023-11-26T07:14:55.570157+0000","last_peered":"2023-11-26T07:14:55.570157+0000","last_clean":"2023-11-26T07:14:55.570157+0000","last_became_active":"2023-11-26T07:14:54.554331+0000","last_became_peered":"2023-11-26T07:14:54.554331+0000","last_unstale":"2023-11-26T07:14:55.570157+0000","last_undegraded":"2023-11-26T07:14:55.570157+0000","last_fullsized":"2023-11-26T07:14:55.570157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:32:22.345038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876915+0000","last_change":"2023-11-26T07:14:55.567533+0000","last_active":"2023-11-26T07:14:56.876915+0000","last_peered":"2023-11-26T07:14:56.876915+0000","last_clean":"2023-11-26T07:14:56.876915+0000","last_became_active":"2023-11-26T07:14:55.567256+0000","last_became_peered":"2023-11-26T07:14:55.567256+0000","last_unstale":"2023-11-26T07:14:56.876915+0000","last_undegraded":"2023-11-26T07:14:56.876915+0000","last_fullsized":"2023-11-26T07:14:56.876915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:01:50.191654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.542772+0000","last_change":"2023-11-26T07:14:54.554798+0000","last_active":"2023-11-26T07:14:55.542772+0000","last_peered":"2023-11-26T07:14:55.542772+0000","last_clean":"2023-11-26T07:14:55.542772+0000","last_became_active":"2023-11-26T07:14:54.554545+0000","last_became_peered":"2023-11-26T07:14:54.554545+0000","last_unstale":"2023-11-26T07:14:55.542772+0000","last_undegraded":"2023-11-26T07:14:55.542772+0000","last_fullsized":"2023-11-26T07:14:55.542772+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:22:35.859767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876276+0000","last_change":"2023-11-26T07:14:55.553523+0000","last_active":"2023-11-26T07:14:56.876276+0000","last_peered":"2023-11-26T07:14:56.876276+0000","last_clean":"2023-11-26T07:14:56.876276+0000","last_became_active":"2023-11-26T07:14:55.553227+0000","last_became_peered":"2023-11-26T07:14:55.553227+0000","last_unstale":"2023-11-26T07:14:56.876276+0000","last_undegraded":"2023-11-26T07:14:56.876276+0000","last_fullsized":"2023-11-26T07:14:56.876276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:39:06.952707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.547474+0000","last_change":"2023-11-26T07:14:54.556775+0000","last_active":"2023-11-26T07:14:55.547474+0000","last_peered":"2023-11-26T07:14:55.547474+0000","last_clean":"2023-11-26T07:14:55.547474+0000","last_became_active":"2023-11-26T07:14:54.556477+0000","last_became_peered":"2023-11-26T07:14:54.556477+0000","last_unstale":"2023-11-26T07:14:55.547474+0000","last_undegraded":"2023-11-26T07:14:55.547474+0000","last_fullsized":"2023-11-26T07:14:55.547474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:41:47.910749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877407+0000","last_change":"2023-11-26T07:14:55.556529+0000","last_active":"2023-11-26T07:14:56.877407+0000","last_peered":"2023-11-26T07:14:56.877407+0000","last_clean":"2023-11-26T07:14:56.877407+0000","last_became_active":"2023-11-26T07:14:55.556258+0000","last_became_peered":"2023-11-26T07:14:55.556258+0000","last_unstale":"2023-11-26T07:14:56.877407+0000","last_undegraded":"2023-11-26T07:14:56.877407+0000","last_fullsized":"2023-11-26T07:14:56.877407+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:48:48.988066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.567977+0000","last_change":"2023-11-26T07:14:54.559238+0000","last_active":"2023-11-26T07:14:55.567977+0000","last_peered":"2023-11-26T07:14:55.567977+0000","last_clean":"2023-11-26T07:14:55.567977+0000","last_became_active":"2023-11-26T07:14:54.558934+0000","last_became_peered":"2023-11-26T07:14:54.558934+0000","last_unstale":"2023-11-26T07:14:55.567977+0000","last_undegraded":"2023-11-26T07:14:55.567977+0000","last_fullsized":"2023-11-26T07:14:55.567977+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:44.510723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877355+0000","last_change":"2023-11-26T07:14:55.552256+0000","last_active":"2023-11-26T07:14:56.877355+0000","last_peered":"2023-11-26T07:14:56.877355+0000","last_clean":"2023-11-26T07:14:56.877355+0000","last_became_active":"2023-11-26T07:14:55.551982+0000","last_became_peered":"2023-11-26T07:14:55.551982+0000","last_unstale":"2023-11-26T07:14:56.877355+0000","last_undegraded":"2023-11-26T07:14:56.877355+0000","last_fullsized":"2023-11-26T07:14:56.877355+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:03.665466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876918+0000","last_change":"2023-11-26T07:14:54.550263+0000","last_active":"2023-11-26T07:14:56.876918+0000","last_peered":"2023-11-26T07:14:56.876918+0000","last_clean":"2023-11-26T07:14:56.876918+0000","last_became_active":"2023-11-26T07:14:54.549971+0000","last_became_peered":"2023-11-26T07:14:54.549971+0000","last_unstale":"2023-11-26T07:14:56.876918+0000","last_undegraded":"2023-11-26T07:14:56.876918+0000","last_fullsized":"2023-11-26T07:14:56.876918+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:12.852910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876936+0000","last_change":"2023-11-26T07:14:55.556173+0000","last_active":"2023-11-26T07:14:56.876936+0000","last_peered":"2023-11-26T07:14:56.876936+0000","last_clean":"2023-11-26T07:14:56.876936+0000","last_became_active":"2023-11-26T07:14:55.555920+0000","last_became_peered":"2023-11-26T07:14:55.555920+0000","last_unstale":"2023-11-26T07:14:56.876936+0000","last_undegraded":"2023-11-26T07:14:56.876936+0000","last_fullsized":"2023-11-26T07:14:56.876936+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:53.832663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559548+0000","last_change":"2023-11-26T07:14:54.552589+0000","last_active":"2023-11-26T07:14:54.559548+0000","last_peered":"2023-11-26T07:14:54.559548+0000","last_clean":"2023-11-26T07:14:54.559548+0000","last_became_active":"2023-11-26T07:14:54.552302+0000","last_became_peered":"2023-11-26T07:14:54.552302+0000","last_unstale":"2023-11-26T07:14:54.559548+0000","last_undegraded":"2023-11-26T07:14:54.559548+0000","last_fullsized":"2023-11-26T07:14:54.559548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:59.018549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877765+0000","last_change":"2023-11-26T07:14:55.567850+0000","last_active":"2023-11-26T07:14:56.877765+0000","last_peered":"2023-11-26T07:14:56.877765+0000","last_clean":"2023-11-26T07:14:56.877765+0000","last_became_active":"2023-11-26T07:14:55.567191+0000","last_became_peered":"2023-11-26T07:14:55.567191+0000","last_unstale":"2023-11-26T07:14:56.877765+0000","last_undegraded":"2023-11-26T07:14:56.877765+0000","last_fullsized":"2023-11-26T07:14:56.877765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:37:15.036624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.551783+0000","last_change":"2023-11-26T07:14:54.555982+0000","last_active":"2023-11-26T07:14:55.551783+0000","last_peered":"2023-11-26T07:14:55.551783+0000","last_clean":"2023-11-26T07:14:55.551783+0000","last_became_active":"2023-11-26T07:14:54.555678+0000","last_became_peered":"2023-11-26T07:14:54.555678+0000","last_unstale":"2023-11-26T07:14:55.551783+0000","last_undegraded":"2023-11-26T07:14:55.551783+0000","last_fullsized":"2023-11-26T07:14:55.551783+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:08.574978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876572+0000","last_change":"2023-11-26T07:14:55.562592+0000","last_active":"2023-11-26T07:14:56.876572+0000","last_peered":"2023-11-26T07:14:56.876572+0000","last_clean":"2023-11-26T07:14:56.876572+0000","last_became_active":"2023-11-26T07:14:55.562283+0000","last_became_peered":"2023-11-26T07:14:55.562283+0000","last_unstale":"2023-11-26T07:14:56.876572+0000","last_undegraded":"2023-11-26T07:14:56.876572+0000","last_fullsized":"2023-11-26T07:14:56.876572+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:40:50.418915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.542809+0000","last_change":"2023-11-26T07:14:54.556883+0000","last_active":"2023-11-26T07:14:55.542809+0000","last_peered":"2023-11-26T07:14:55.542809+0000","last_clean":"2023-11-26T07:14:55.542809+0000","last_became_active":"2023-11-26T07:14:54.556654+0000","last_became_peered":"2023-11-26T07:14:54.556654+0000","last_unstale":"2023-11-26T07:14:55.542809+0000","last_undegraded":"2023-11-26T07:14:55.542809+0000","last_fullsized":"2023-11-26T07:14:55.542809+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:00:25.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541680+0000","last_change":"2023-11-26T07:14:55.567327+0000","last_active":"2023-11-26T07:14:56.541680+0000","last_peered":"2023-11-26T07:14:56.541680+0000","last_clean":"2023-11-26T07:14:56.541680+0000","last_became_active":"2023-11-26T07:14:55.567068+0000","last_became_peered":"2023-11-26T07:14:55.567068+0000","last_unstale":"2023-11-26T07:14:56.541680+0000","last_undegraded":"2023-11-26T07:14:56.541680+0000","last_fullsized":"2023-11-26T07:14:56.541680+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:32:23.347056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.542458+0000","last_change":"2023-11-26T07:14:54.555216+0000","last_active":"2023-11-26T07:14:55.542458+0000","last_peered":"2023-11-26T07:14:55.542458+0000","last_clean":"2023-11-26T07:14:55.542458+0000","last_became_active":"2023-11-26T07:14:54.554577+0000","last_became_peered":"2023-11-26T07:14:54.554577+0000","last_unstale":"2023-11-26T07:14:55.542458+0000","last_undegraded":"2023-11-26T07:14:55.542458+0000","last_fullsized":"2023-11-26T07:14:55.542458+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:43.781381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876106+0000","last_change":"2023-11-26T07:14:55.558942+0000","last_active":"2023-11-26T07:14:56.876106+0000","last_peered":"2023-11-26T07:14:56.876106+0000","last_clean":"2023-11-26T07:14:56.876106+0000","last_became_active":"2023-11-26T07:14:55.558687+0000","last_became_peered":"2023-11-26T07:14:55.558687+0000","last_unstale":"2023-11-26T07:14:56.876106+0000","last_undegraded":"2023-11-26T07:14:56.876106+0000","last_fullsized":"2023-11-26T07:14:56.876106+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:16:05.761932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.543069+0000","last_change":"2023-11-26T07:14:54.556996+0000","last_active":"2023-11-26T07:14:55.543069+0000","last_peered":"2023-11-26T07:14:55.543069+0000","last_clean":"2023-11-26T07:14:55.543069+0000","last_became_active":"2023-11-26T07:14:54.556722+0000","last_became_peered":"2023-11-26T07:14:54.556722+0000","last_unstale":"2023-11-26T07:14:55.543069+0000","last_undegraded":"2023-11-26T07:14:55.543069+0000","last_fullsized":"2023-11-26T07:14:55.543069+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:05.353039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876662+0000","last_change":"2023-11-26T07:14:55.565201+0000","last_active":"2023-11-26T07:14:56.876662+0000","last_peered":"2023-11-26T07:14:56.876662+0000","last_clean":"2023-11-26T07:14:56.876662+0000","last_became_active":"2023-11-26T07:14:55.564935+0000","last_became_peered":"2023-11-26T07:14:55.564935+0000","last_unstale":"2023-11-26T07:14:56.876662+0000","last_undegraded":"2023-11-26T07:14:56.876662+0000","last_fullsized":"2023-11-26T07:14:56.876662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:51:19.407229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.549059+0000","last_change":"2023-11-26T07:14:54.550363+0000","last_active":"2023-11-26T07:14:55.549059+0000","last_peered":"2023-11-26T07:14:55.549059+0000","last_clean":"2023-11-26T07:14:55.549059+0000","last_became_active":"2023-11-26T07:14:54.550093+0000","last_became_peered":"2023-11-26T07:14:54.550093+0000","last_unstale":"2023-11-26T07:14:55.549059+0000","last_undegraded":"2023-11-26T07:14:55.549059+0000","last_fullsized":"2023-11-26T07:14:55.549059+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:01:26.226561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876495+0000","last_change":"2023-11-26T07:14:55.561975+0000","last_active":"2023-11-26T07:14:56.876495+0000","last_peered":"2023-11-26T07:14:56.876495+0000","last_clean":"2023-11-26T07:14:56.876495+0000","last_became_active":"2023-11-26T07:14:55.561323+0000","last_became_peered":"2023-11-26T07:14:55.561323+0000","last_unstale":"2023-11-26T07:14:56.876495+0000","last_undegraded":"2023-11-26T07:14:56.876495+0000","last_fullsized":"2023-11-26T07:14:56.876495+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:13:03.157321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559599+0000","last_change":"2023-11-26T07:14:54.552852+0000","last_active":"2023-11-26T07:14:54.559599+0000","last_peered":"2023-11-26T07:14:54.559599+0000","last_clean":"2023-11-26T07:14:54.559599+0000","last_became_active":"2023-11-26T07:14:54.552543+0000","last_became_peered":"2023-11-26T07:14:54.552543+0000","last_unstale":"2023-11-26T07:14:54.559599+0000","last_undegraded":"2023-11-26T07:14:54.559599+0000","last_fullsized":"2023-11-26T07:14:54.559599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:30.337901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.540389+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:14:55.540389+0000","last_peered":"2023-11-26T07:14:55.540389+0000","last_clean":"2023-11-26T07:14:55.540389+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:14:55.540389+0000","last_undegraded":"2023-11-26T07:14:55.540389+0000","last_fullsized":"2023-11-26T07:14:55.540389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1298432,"data_stored":880333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.55900000000000005},"min":{"1min":0.44,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.52200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.53800000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.47299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73799999999999999,"5min":1.048,"15min":1.048},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.497,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.5}]},{"osd":2,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.495,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.37},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.621,"15min":0.622},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75600000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51700000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70499999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77800000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76700000000000002,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.46999999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.37,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79500000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79000000000000004,"5min":1.171,"15min":1.171},"last":0.63100000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.74199999999999999,"5min":1.361,"15min":1.361},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78000000000000003,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":276730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.39200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.73699999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76600000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.49099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.69799999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.45800000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72299999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82599999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.753,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70399999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56799999999999995,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67900000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.51200000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.747,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73399999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":289895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.502,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.34899999999999998,"15min":0.34599999999999997},"min":{"1min":0.20000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.5,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.379}]},{"osd":2,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34000000000000002,"15min":0.33400000000000002},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.27200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53000000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.33700000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.252,"5min":0.246,"15min":0.246},"max":{"1min":0.57499999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.432},{"interface":"front","average":{"1min":0.434,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.877,"15min":0.877},"last":0.499},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.623,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":0.995,"15min":0.995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.46800000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53200000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88800000000000001,"5min":1.129,"15min":1.129},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.627,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57399999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.628},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47399999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83599999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.61799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":882367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.221,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54200000000000004,"5min":0.623,"15min":0.623},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.443,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.34799999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.629,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.28599999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.29799999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61699999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.437,"15min":0.439},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42799999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67000000000000004,"15min":0.67300000000000004},"min":{"1min":0.499,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96599999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52500000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.625},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83799999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55100000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45100000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.503}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36099999999999999,"15min":0.36899999999999999},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.497,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.41899999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.27900000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58799999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.498},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.47199999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.27400000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.56499999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42599999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54000000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.44800000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64800000000000002,"15min":0.67600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85799999999999998,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79300000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.78900000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.621,"15min":0.624},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81399999999999995,"5min":1.075,"15min":1.075},"last":0.624},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.72799999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80100000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.47099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.77900000000000003,"5min":1.03,"15min":1.03},"last":0.77100000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.65900000000000003},"min":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.622,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82799999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.74399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.218,"5min":0.188,"15min":0.188},"max":{"1min":0.42199999999999999,"5min":1.284,"15min":1.284},"last":0.313},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.44900000000000001,"15min":0.47999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.55500000000000005,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.29999999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.47499999999999998,"15min":0.52700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61799999999999999,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.46500000000000002,"15min":0.498},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.50700000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.374}]},{"osd":2,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.50800000000000001},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.59299999999999997,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52700000000000002,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.35999999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.68000000000000005},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85299999999999998,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88200000000000001,"5min":1.405,"15min":1.405},"last":0.56200000000000006}]},{"osd":5,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.50700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83799999999999997,"5min":1.377,"15min":1.377},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":0.80900000000000005},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91100000000000003,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66200000000000003,"15min":0.67500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92500000000000004,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.75600000000000001,"15min":0.80200000000000005},"min":{"1min":0.51700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.871,"5min":4.883,"15min":4.883},"last":0.64800000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.75700000000000001,"15min":0.81899999999999995},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80200000000000005,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71599999999999997,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85299999999999998,"5min":2.923,"15min":2.923},"last":0.66200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":291417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.41499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77500000000000002,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.43099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45500000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.53000000000000003,"15min":0.51200000000000001},"min":{"1min":0.44700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77500000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.63400000000000001,"15min":0.66400000000000003},"min":{"1min":0.45600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71099999999999997,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.58099999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54300000000000004,"15min":0.52500000000000002},"min":{"1min":0.47899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.68600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.64700000000000002,"15min":0.67900000000000005},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74299999999999999,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.625}]},{"osd":3,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.40500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75900000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73399999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.498,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.751}]},{"osd":6,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.54300000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72999999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.73899999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71699999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.625,"15min":0.63600000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79600000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.82899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":290457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.39200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.49099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57499999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.48799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.73799999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69699999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.42399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.45900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64100000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.498,"15min":0.5},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.61599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67300000000000004,"5min":0.752,"15min":0.752},"last":0.60899999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.65800000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70899999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.62}]},{"osd":7,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.629},{"interface":"front","average":{"1min":0.625,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76600000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.746}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:15:00.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2023-11-26T07:15:00.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:14:54.541763+0000', 'last_change': '2023-11-26T07:14:54.541763+0000', 'last_active': '2023-11-26T07:14:54.530381+0000', 'last_peered': '2023-11-26T07:14:54.530381+0000', 'last_clean': '2023-11-26T07:14:54.530381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:14:54.541763+0000', 'last_undegraded': '2023-11-26T07:14:54.541763+0000', 'last_fullsized': '2023-11-26T07:14:54.541763+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:14:54.530381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:14:54.530381+0000', 'last_clean_scrub_stamp': '2023-11-26T07:14:54.530381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:15:00.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2023-11-26T07:15:00.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:14:54.541275+0000', 'last_change': '2023-11-26T07:14:54.541275+0000', 'last_active': '2023-11-26T07:14:54.530381+0000', 'last_peered': '2023-11-26T07:14:54.530381+0000', 'last_clean': '2023-11-26T07:14:54.530381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:14:54.541275+0000', 'last_undegraded': '2023-11-26T07:14:54.541275+0000', 'last_fullsized': '2023-11-26T07:14:54.541275+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:14:54.530381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:14:54.530381+0000', 'last_clean_scrub_stamp': '2023-11-26T07:14:54.530381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:15:00.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-11-26T07:15:00.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 56, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:14:54.540783+0000', 'last_change': '2023-11-26T07:14:54.540783+0000', 'last_active': '2023-11-26T07:14:54.530381+0000', 'last_peered': '2023-11-26T07:14:54.530381+0000', 'last_clean': '2023-11-26T07:14:54.530381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:14:54.540783+0000', 'last_undegraded': '2023-11-26T07:14:54.540783+0000', 'last_fullsized': '2023-11-26T07:14:54.540783+0000', 'mapping_epoch': 56, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 56, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:14:54.530381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:14:54.530381+0000', 'last_clean_scrub_stamp': '2023-11-26T07:14:54.530381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:15:00.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-26T07:15:00.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.590+0000 7f34a885c700 1 Processor -- start 2023-11-26T07:15:00.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.590+0000 7f34a885c700 1 -- start start 2023-11-26T07:15:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.591+0000 7f34a885c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f34a40afc80 0x7f34a40d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.591+0000 7f34a885c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a40ca620 con 0x7f34a40b0fa0 2023-11-26T07:15:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.591+0000 7f34a885c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a40622a0 con 0x7f34a40ca190 2023-11-26T07:15:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.591+0000 7f34a885c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a4062580 con 0x7f34a40afc80 2023-11-26T07:15:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.591+0000 7f34a1d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f34a40afc80 0x7f34a40d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.591+0000 7f34a259c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f34a40b0fa0 0x7f34a40b1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58520/0 (socket says 172.21.15.83:58520) 2023-11-26T07:15:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.591+0000 7f34a259c700 1 -- 172.21.15.83:0/2895888308 learned_addr learned my addr 172.21.15.83:0/2895888308 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.591+0000 7f34a1d9b700 1 -- 172.21.15.83:0/2895888308 >> v1:172.21.15.203:6789/0 conn(0x7f34a40ca190 legacy=0x7f34a40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.591+0000 7f34a1d9b700 1 -- 172.21.15.83:0/2895888308 >> v1:172.21.15.83:6789/0 conn(0x7f34a40b0fa0 legacy=0x7f34a40b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.591+0000 7f34a1d9b700 1 -- 172.21.15.83:0/2895888308 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a40c96b0 con 0x7f34a40afc80 2023-11-26T07:15:00.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.592+0000 7f34a1d9b700 1 --2- 172.21.15.83:0/2895888308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a40afc80 0x7f34a40d0c40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f348c009840 tx=0x7f348c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e4d8e64d270e81 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:00.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.592+0000 7f34a159a700 1 -- 172.21.15.83:0/2895888308 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c004230 con 0x7f34a40afc80 2023-11-26T07:15:00.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.592+0000 7f34a159a700 1 -- 172.21.15.83:0/2895888308 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c004390 con 0x7f34a40afc80 2023-11-26T07:15:00.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.592+0000 7f34a159a700 1 -- 172.21.15.83:0/2895888308 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c0036a0 con 0x7f34a40afc80 2023-11-26T07:15:00.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.592+0000 7f34a885c700 1 -- 172.21.15.83:0/2895888308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a40afc80 msgr2=0x7f34a40d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.592+0000 7f34a885c700 1 --2- 172.21.15.83:0/2895888308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a40afc80 0x7f34a40d0c40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f348c009840 tx=0x7f348c009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:00.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.592+0000 7f34a885c700 1 -- 172.21.15.83:0/2895888308 shutdown_connections 2023-11-26T07:15:00.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.592+0000 7f34a885c700 1 --2- 172.21.15.83:0/2895888308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a40afc80 0x7f34a40d0c40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.592+0000 7f34a885c700 1 -- 172.21.15.83:0/2895888308 >> 172.21.15.83:0/2895888308 conn(0x7f34a40b57b0 msgr2=0x7f34a40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:00.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.592+0000 7f34a885c700 1 -- 172.21.15.83:0/2895888308 shutdown_connections 2023-11-26T07:15:00.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.592+0000 7f34a885c700 1 -- 172.21.15.83:0/2895888308 wait complete. 2023-11-26T07:15:00.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.594+0000 7f34a885c700 1 Processor -- start 2023-11-26T07:15:00.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.594+0000 7f34a885c700 1 -- start start 2023-11-26T07:15:00.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.594+0000 7f34a885c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a40afc80 0x7f34a41505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.594+0000 7f34a885c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34a40b0fa0 0x7f34a40cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.594+0000 7f34a885c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34a40ca190 0x7f34a4156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.594+0000 7f34a885c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a4156580 con 0x7f34a40afc80 2023-11-26T07:15:00.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a885c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a41567b0 con 0x7f34a40ca190 2023-11-26T07:15:00.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a885c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a4156a90 con 0x7f34a40b0fa0 2023-11-26T07:15:00.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a40afc80 0x7f34a41505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a40afc80 0x7f34a41505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38486/0 (socket says 172.21.15.83:38486) 2023-11-26T07:15:00.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a259c700 1 -- 172.21.15.83:0/4214065242 learned_addr learned my addr 172.21.15.83:0/4214065242 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:00.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a2d9d700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34a40ca190 0x7f34a4156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a1d9b700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34a40b0fa0 0x7f34a40cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a259c700 1 -- 172.21.15.83:0/4214065242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34a40b0fa0 msgr2=0x7f34a40cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a259c700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34a40b0fa0 0x7f34a40cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a259c700 1 -- 172.21.15.83:0/4214065242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34a40ca190 msgr2=0x7f34a4156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a259c700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34a40ca190 0x7f34a4156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a259c700 1 -- 172.21.15.83:0/4214065242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0094a0 con 0x7f34a40afc80 2023-11-26T07:15:00.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a2d9d700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34a40ca190 0x7f34a4156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:00.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a259c700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a40afc80 0x7f34a41505a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f349c00cb10 tx=0x7f349c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:00.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f349affd700 1 -- 172.21.15.83:0/4214065242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c00ab00 con 0x7f34a40afc80 2023-11-26T07:15:00.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f349affd700 1 -- 172.21.15.83:0/4214065242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349c00ac60 con 0x7f34a40afc80 2023-11-26T07:15:00.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f349affd700 1 -- 172.21.15.83:0/4214065242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c012400 con 0x7f34a40afc80 2023-11-26T07:15:00.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a885c700 1 -- 172.21.15.83:0/4214065242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a4156d70 con 0x7f34a40afc80 2023-11-26T07:15:00.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.595+0000 7f34a885c700 1 -- 172.21.15.83:0/4214065242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a4157330 con 0x7f34a40afc80 2023-11-26T07:15:00.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.596+0000 7f34a885c700 1 -- 172.21.15.83:0/4214065242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a4067f60 con 0x7f34a40afc80 2023-11-26T07:15:00.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.597+0000 7f349affd700 1 -- 172.21.15.83:0/4214065242 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f349c0113a0 con 0x7f34a40afc80 2023-11-26T07:15:00.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.597+0000 7f349affd700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f348406b080 0x7f348406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:00.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.597+0000 7f349affd700 1 -- 172.21.15.83:0/4214065242 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f349c089770 con 0x7f34a40afc80 2023-11-26T07:15:00.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.597+0000 7f34a1d9b700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f348406b080 0x7f348406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:00.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.598+0000 7f34a1d9b700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f348406b080 0x7f348406d540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f34a40cc2e0 tx=0x7f348c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:00.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.602+0000 7f349affd700 1 -- 172.21.15.83:0/4214065242 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f349c054510 con 0x7f34a40afc80 2023-11-26T07:15:00.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.872+0000 7f34a885c700 1 -- 172.21.15.83:0/4214065242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f34a40b1380 con 0x7f34a40afc80 2023-11-26T07:15:00.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.872+0000 7f349affd700 1 -- 172.21.15.83:0/4214065242 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1098 (secure 0 0 0) 0x7f349c017080 con 0x7f34a40afc80 2023-11-26T07:15:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:00.874 INFO:teuthology.orchestra.run.smithi083.stdout:{"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":238,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1700982674,"num_in_osds":8,"osd_in_since":1700982665,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":94},{"state_name":"creating+peering","count":3}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":228179968,"bytes_avail":772865933312,"bytes_total":773094113280,"inactive_pgs_ratio":0.030927835032343864,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":37,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":6102}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-11-26T07:14:02.341096+0000","services":{}},"progress_events":{}} 2023-11-26T07:15:00.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 -- 172.21.15.83:0/4214065242 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f348406b080 msgr2=0x7f348406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f348406b080 0x7f348406d540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f34a40cc2e0 tx=0x7f348c008030 comp rx=0 tx=0).stop 2023-11-26T07:15:00.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 -- 172.21.15.83:0/4214065242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a40afc80 msgr2=0x7f34a41505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:00.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a40afc80 0x7f34a41505a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f349c00cb10 tx=0x7f349c00ced0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 -- 172.21.15.83:0/4214065242 shutdown_connections 2023-11-26T07:15:00.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34a40ca190 0x7f34a4156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a40afc80 0x7f34a41505a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f348406b080 0x7f348406d540 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 --2- 172.21.15.83:0/4214065242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34a40b0fa0 0x7f34a40cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:00.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 -- 172.21.15.83:0/4214065242 >> 172.21.15.83:0/4214065242 conn(0x7f34a40b57b0 msgr2=0x7f34a40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:00.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 -- 172.21.15.83:0/4214065242 shutdown_connections 2023-11-26T07:15:00.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:00.874+0000 7f34a885c700 1 -- 172.21.15.83:0/4214065242 wait complete. 2023-11-26T07:15:00.888 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-26T07:15:00.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:15:01.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.007+0000 7f529bc31700 1 Processor -- start 2023-11-26T07:15:01.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.007+0000 7f529bc31700 1 -- start start 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f529bc31700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f529406fa40 0x7f5294073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f529bc31700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f529406fef0 0x7f52940744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f529bc31700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52940af090 con 0x7f52940afc10 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f529bc31700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529406dda0 con 0x7f529406fa40 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f529bc31700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529406e060 con 0x7f529406fef0 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52991cc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f529406fa40 0x7f5294073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52999cd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f52940afc10 0x7f529415e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58534/0 (socket says 172.21.15.83:58534) 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52999cd700 1 -- 172.21.15.83:0/1086554319 learned_addr learned my addr 172.21.15.83:0/1086554319 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f529a1ce700 1 --2- 172.21.15.83:0/1086554319 >> v2:172.21.15.203:3300/0 conn(0x7f529406fef0 0x7f52940744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52991cc700 1 -- 172.21.15.83:0/1086554319 >> v2:172.21.15.203:3300/0 conn(0x7f529406fef0 msgr2=0x7f52940744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52991cc700 1 --2- 172.21.15.83:0/1086554319 >> v2:172.21.15.203:3300/0 conn(0x7f529406fef0 0x7f52940744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52991cc700 1 -- 172.21.15.83:0/1086554319 >> v1:172.21.15.83:6789/0 conn(0x7f52940afc10 legacy=0x7f529415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:01.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52989cb700 1 -- 172.21.15.83:0/1086554319 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68210199 0 0) 0x7f52940af090 con 0x7f52940afc10 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52991cc700 1 -- 172.21.15.83:0/1086554319 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529406ec70 con 0x7f529406fa40 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52991cc700 1 --2- 172.21.15.83:0/1086554319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529406fa40 0x7f5294073f90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f52940b07d0 tx=0x7f5284009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a83fe54154dd350 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52989cb700 1 -- 172.21.15.83:0/1086554319 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284011830 con 0x7f529406fa40 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52989cb700 1 -- 172.21.15.83:0/1086554319 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284011990 con 0x7f529406fa40 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.008+0000 7f52989cb700 1 -- 172.21.15.83:0/1086554319 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284011d20 con 0x7f529406fa40 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.009+0000 7f529bc31700 1 -- 172.21.15.83:0/1086554319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529406fa40 msgr2=0x7f5294073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.009+0000 7f529bc31700 1 --2- 172.21.15.83:0/1086554319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529406fa40 0x7f5294073f90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f52940b07d0 tx=0x7f5284009b20 comp rx=0 tx=0).stop 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.009+0000 7f529bc31700 1 -- 172.21.15.83:0/1086554319 shutdown_connections 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.009+0000 7f529bc31700 1 --2- 172.21.15.83:0/1086554319 >> v2:172.21.15.203:3300/0 conn(0x7f529406fef0 0x7f52940744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.009+0000 7f529bc31700 1 --2- 172.21.15.83:0/1086554319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529406fa40 0x7f5294073f90 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.009+0000 7f529bc31700 1 -- 172.21.15.83:0/1086554319 >> 172.21.15.83:0/1086554319 conn(0x7f529406aff0 msgr2=0x7f529406b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.009+0000 7f529bc31700 1 -- 172.21.15.83:0/1086554319 shutdown_connections 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.009+0000 7f529bc31700 1 -- 172.21.15.83:0/1086554319 wait complete. 2023-11-26T07:15:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.010+0000 7f529bc31700 1 Processor -- start 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.010+0000 7f529bc31700 1 -- start start 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f529bc31700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529406fa40 0x7f5294144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f529bc31700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f529406fef0 0x7f529415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f529bc31700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52940afc10 0x7f52940b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f529bc31700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52941f4e00 con 0x7f529406fa40 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f529bc31700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52941f50e0 con 0x7f52940afc10 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f529bc31700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52941f53c0 con 0x7f529406fef0 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f52999cd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529406fa40 0x7f5294144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f52999cd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529406fa40 0x7f5294144340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38500/0 (socket says 172.21.15.83:38500) 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f52999cd700 1 -- 172.21.15.83:0/1625146864 learned_addr learned my addr 172.21.15.83:0/1625146864 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f52991cc700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f529406fef0 0x7f529415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f52999cd700 1 -- 172.21.15.83:0/1625146864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f529406fef0 msgr2=0x7f529415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:01.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f529a1ce700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52940afc10 0x7f52940b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f52999cd700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f529406fef0 0x7f529415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f52999cd700 1 -- 172.21.15.83:0/1625146864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52940afc10 msgr2=0x7f52940b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f52999cd700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52940afc10 0x7f52940b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f52999cd700 1 -- 172.21.15.83:0/1625146864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52840094a0 con 0x7f529406fa40 2023-11-26T07:15:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f52991cc700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f529406fef0 0x7f529415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.011+0000 7f52999cd700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529406fa40 0x7f5294144340 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f529000c550 tx=0x7f529000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.012+0000 7f528a7fc700 1 -- 172.21.15.83:0/1625146864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290004c80 con 0x7f529406fa40 2023-11-26T07:15:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.012+0000 7f528a7fc700 1 -- 172.21.15.83:0/1625146864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5290004e00 con 0x7f529406fa40 2023-11-26T07:15:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.012+0000 7f529bc31700 1 -- 172.21.15.83:0/1625146864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52941f56a0 con 0x7f529406fa40 2023-11-26T07:15:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.012+0000 7f528a7fc700 1 -- 172.21.15.83:0/1625146864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290007310 con 0x7f529406fa40 2023-11-26T07:15:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.012+0000 7f529bc31700 1 -- 172.21.15.83:0/1625146864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52941f4920 con 0x7f529406fa40 2023-11-26T07:15:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.012+0000 7f529bc31700 1 -- 172.21.15.83:0/1625146864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52941f4ba0 con 0x7f529406fa40 2023-11-26T07:15:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.013+0000 7f528a7fc700 1 -- 172.21.15.83:0/1625146864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f5290007470 con 0x7f529406fa40 2023-11-26T07:15:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.014+0000 7f528a7fc700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527806b080 0x7f527806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:01.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.014+0000 7f528a7fc700 1 -- 172.21.15.83:0/1625146864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5290089b10 con 0x7f529406fa40 2023-11-26T07:15:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.014+0000 7f52991cc700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527806b080 0x7f527806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:01.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.014+0000 7f52991cc700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527806b080 0x7f527806d540 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f52840090b0 tx=0x7f5284008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:01.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.018+0000 7f528a7fc700 1 -- 172.21.15.83:0/1625146864 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5290054ab0 con 0x7f529406fa40 2023-11-26T07:15:01.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.204+0000 7f529bc31700 1 -- 172.21.15.83:0/1625146864 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f529406e900 con 0x7f527806b080 2023-11-26T07:15:01.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.212+0000 7f528a7fc700 1 -- 172.21.15.83:0/1625146864 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289151 (secure 0 0 0) 0x7f5284011b00 con 0x7f527806b080 2023-11-26T07:15:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 -- 172.21.15.83:0/1625146864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527806b080 msgr2=0x7f527806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527806b080 0x7f527806d540 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f52840090b0 tx=0x7f5284008030 comp rx=0 tx=0).stop 2023-11-26T07:15:01.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 -- 172.21.15.83:0/1625146864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529406fa40 msgr2=0x7f5294144340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529406fa40 0x7f5294144340 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f529000c550 tx=0x7f529000c910 comp rx=0 tx=0).stop 2023-11-26T07:15:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 -- 172.21.15.83:0/1625146864 shutdown_connections 2023-11-26T07:15:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52940afc10 0x7f52940b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529406fa40 0x7f5294144340 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f527806b080 0x7f527806d540 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 --2- 172.21.15.83:0/1625146864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f529406fef0 0x7f529415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 -- 172.21.15.83:0/1625146864 >> 172.21.15.83:0/1625146864 conn(0x7f529406aff0 msgr2=0x7f529415e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:01.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 -- 172.21.15.83:0/1625146864 shutdown_connections 2023-11-26T07:15:01.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:01.215+0000 7f529bc31700 1 -- 172.21.15.83:0/1625146864 wait complete. 2023-11-26T07:15:01.223 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:15:01.229 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-11-26T07:15:00.353498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222832,"kb_used_data":6640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751888,"statfs":{"total":773094113280,"available":772865933312,"internally_reserved":0,"allocated":6799360,"data_stored":3490921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000895"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542403+0000","last_change":"2023-11-26T07:14:55.565782+0000","last_active":"2023-11-26T07:14:56.542403+0000","last_peered":"2023-11-26T07:14:56.542403+0000","last_clean":"2023-11-26T07:14:56.542403+0000","last_became_active":"2023-11-26T07:14:55.565549+0000","last_became_peered":"2023-11-26T07:14:55.565549+0000","last_unstale":"2023-11-26T07:14:56.542403+0000","last_undegraded":"2023-11-26T07:14:56.542403+0000","last_fullsized":"2023-11-26T07:14:56.542403+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:31:48.907743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877008+0000","last_change":"2023-11-26T07:14:55.563586+0000","last_active":"2023-11-26T07:14:56.877008+0000","last_peered":"2023-11-26T07:14:56.877008+0000","last_clean":"2023-11-26T07:14:56.877008+0000","last_became_active":"2023-11-26T07:14:55.563306+0000","last_became_peered":"2023-11-26T07:14:55.563306+0000","last_unstale":"2023-11-26T07:14:56.877008+0000","last_undegraded":"2023-11-26T07:14:56.877008+0000","last_fullsized":"2023-11-26T07:14:56.877008+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:04:44.976495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541137+0000","last_change":"2023-11-26T07:14:55.547529+0000","last_active":"2023-11-26T07:14:56.541137+0000","last_peered":"2023-11-26T07:14:56.541137+0000","last_clean":"2023-11-26T07:14:56.541137+0000","last_became_active":"2023-11-26T07:14:55.547248+0000","last_became_peered":"2023-11-26T07:14:55.547248+0000","last_unstale":"2023-11-26T07:14:56.541137+0000","last_undegraded":"2023-11-26T07:14:56.541137+0000","last_fullsized":"2023-11-26T07:14:56.541137+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:48:48.988066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877177+0000","last_change":"2023-11-26T07:14:55.556871+0000","last_active":"2023-11-26T07:14:56.877177+0000","last_peered":"2023-11-26T07:14:56.877177+0000","last_clean":"2023-11-26T07:14:56.877177+0000","last_became_active":"2023-11-26T07:14:55.556600+0000","last_became_peered":"2023-11-26T07:14:55.556600+0000","last_unstale":"2023-11-26T07:14:56.877177+0000","last_undegraded":"2023-11-26T07:14:56.877177+0000","last_fullsized":"2023-11-26T07:14:56.877177+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:57.935087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542195+0000","last_change":"2023-11-26T07:14:55.556264+0000","last_active":"2023-11-26T07:14:56.542195+0000","last_peered":"2023-11-26T07:14:56.542195+0000","last_clean":"2023-11-26T07:14:56.542195+0000","last_became_active":"2023-11-26T07:14:55.555983+0000","last_became_peered":"2023-11-26T07:14:55.555983+0000","last_unstale":"2023-11-26T07:14:56.542195+0000","last_undegraded":"2023-11-26T07:14:56.542195+0000","last_fullsized":"2023-11-26T07:14:56.542195+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:16:59.337538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876724+0000","last_change":"2023-11-26T07:14:55.557549+0000","last_active":"2023-11-26T07:14:56.876724+0000","last_peered":"2023-11-26T07:14:56.876724+0000","last_clean":"2023-11-26T07:14:56.876724+0000","last_became_active":"2023-11-26T07:14:55.557282+0000","last_became_peered":"2023-11-26T07:14:55.557282+0000","last_unstale":"2023-11-26T07:14:56.876724+0000","last_undegraded":"2023-11-26T07:14:56.876724+0000","last_fullsized":"2023-11-26T07:14:56.876724+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:03.665466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877024+0000","last_change":"2023-11-26T07:14:55.554176+0000","last_active":"2023-11-26T07:14:56.877024+0000","last_peered":"2023-11-26T07:14:56.877024+0000","last_clean":"2023-11-26T07:14:56.877024+0000","last_became_active":"2023-11-26T07:14:55.553858+0000","last_became_peered":"2023-11-26T07:14:55.553858+0000","last_unstale":"2023-11-26T07:14:56.877024+0000","last_undegraded":"2023-11-26T07:14:56.877024+0000","last_fullsized":"2023-11-26T07:14:56.877024+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:39:44.105610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876382+0000","last_change":"2023-11-26T07:14:55.561614+0000","last_active":"2023-11-26T07:14:56.876382+0000","last_peered":"2023-11-26T07:14:56.876382+0000","last_clean":"2023-11-26T07:14:56.876382+0000","last_became_active":"2023-11-26T07:14:55.561004+0000","last_became_peered":"2023-11-26T07:14:55.561004+0000","last_unstale":"2023-11-26T07:14:56.876382+0000","last_undegraded":"2023-11-26T07:14:56.876382+0000","last_fullsized":"2023-11-26T07:14:56.876382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:36.265114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877062+0000","last_change":"2023-11-26T07:14:55.568602+0000","last_active":"2023-11-26T07:14:56.877062+0000","last_peered":"2023-11-26T07:14:56.877062+0000","last_clean":"2023-11-26T07:14:56.877062+0000","last_became_active":"2023-11-26T07:14:55.568272+0000","last_became_peered":"2023-11-26T07:14:55.568272+0000","last_unstale":"2023-11-26T07:14:56.877062+0000","last_undegraded":"2023-11-26T07:14:56.877062+0000","last_fullsized":"2023-11-26T07:14:56.877062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:40.727595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877245+0000","last_change":"2023-11-26T07:14:55.555237+0000","last_active":"2023-11-26T07:14:56.877245+0000","last_peered":"2023-11-26T07:14:56.877245+0000","last_clean":"2023-11-26T07:14:56.877245+0000","last_became_active":"2023-11-26T07:14:55.554973+0000","last_became_peered":"2023-11-26T07:14:55.554973+0000","last_unstale":"2023-11-26T07:14:56.877245+0000","last_undegraded":"2023-11-26T07:14:56.877245+0000","last_fullsized":"2023-11-26T07:14:56.877245+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:01:53.326813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877295+0000","last_change":"2023-11-26T07:14:55.567135+0000","last_active":"2023-11-26T07:14:56.877295+0000","last_peered":"2023-11-26T07:14:56.877295+0000","last_clean":"2023-11-26T07:14:56.877295+0000","last_became_active":"2023-11-26T07:14:55.566885+0000","last_became_peered":"2023-11-26T07:14:55.566885+0000","last_unstale":"2023-11-26T07:14:56.877295+0000","last_undegraded":"2023-11-26T07:14:56.877295+0000","last_fullsized":"2023-11-26T07:14:56.877295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:25:05.901539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877351+0000","last_change":"2023-11-26T07:14:55.555316+0000","last_active":"2023-11-26T07:14:56.877351+0000","last_peered":"2023-11-26T07:14:56.877351+0000","last_clean":"2023-11-26T07:14:56.877351+0000","last_became_active":"2023-11-26T07:14:55.555043+0000","last_became_peered":"2023-11-26T07:14:55.555043+0000","last_unstale":"2023-11-26T07:14:56.877351+0000","last_undegraded":"2023-11-26T07:14:56.877351+0000","last_fullsized":"2023-11-26T07:14:56.877351+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:01.838876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541763+0000","last_change":"2023-11-26T07:14:54.541763+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541763+0000","last_undegraded":"2023-11-26T07:14:54.541763+0000","last_fullsized":"2023-11-26T07:14:54.541763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876769+0000","last_change":"2023-11-26T07:14:55.557930+0000","last_active":"2023-11-26T07:14:56.876769+0000","last_peered":"2023-11-26T07:14:56.876769+0000","last_clean":"2023-11-26T07:14:56.876769+0000","last_became_active":"2023-11-26T07:14:55.557647+0000","last_became_peered":"2023-11-26T07:14:55.557647+0000","last_unstale":"2023-11-26T07:14:56.876769+0000","last_undegraded":"2023-11-26T07:14:56.876769+0000","last_fullsized":"2023-11-26T07:14:56.876769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:00:31.201849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877113+0000","last_change":"2023-11-26T07:14:55.559368+0000","last_active":"2023-11-26T07:14:56.877113+0000","last_peered":"2023-11-26T07:14:56.877113+0000","last_clean":"2023-11-26T07:14:56.877113+0000","last_became_active":"2023-11-26T07:14:55.558965+0000","last_became_peered":"2023-11-26T07:14:55.558965+0000","last_unstale":"2023-11-26T07:14:56.877113+0000","last_undegraded":"2023-11-26T07:14:56.877113+0000","last_fullsized":"2023-11-26T07:14:56.877113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:41:48.912767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876397+0000","last_change":"2023-11-26T07:14:55.558759+0000","last_active":"2023-11-26T07:14:56.876397+0000","last_peered":"2023-11-26T07:14:56.876397+0000","last_clean":"2023-11-26T07:14:56.876397+0000","last_became_active":"2023-11-26T07:14:55.558507+0000","last_became_peered":"2023-11-26T07:14:55.558507+0000","last_unstale":"2023-11-26T07:14:56.876397+0000","last_undegraded":"2023-11-26T07:14:56.876397+0000","last_fullsized":"2023-11-26T07:14:56.876397+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:57.951234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876827+0000","last_change":"2023-11-26T07:14:55.562951+0000","last_active":"2023-11-26T07:14:56.876827+0000","last_peered":"2023-11-26T07:14:56.876827+0000","last_clean":"2023-11-26T07:14:56.876827+0000","last_became_active":"2023-11-26T07:14:55.562289+0000","last_became_peered":"2023-11-26T07:14:55.562289+0000","last_unstale":"2023-11-26T07:14:56.876827+0000","last_undegraded":"2023-11-26T07:14:56.876827+0000","last_fullsized":"2023-11-26T07:14:56.876827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:47:37.463547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541881+0000","last_change":"2023-11-26T07:14:55.570981+0000","last_active":"2023-11-26T07:14:56.541881+0000","last_peered":"2023-11-26T07:14:56.541881+0000","last_clean":"2023-11-26T07:14:56.541881+0000","last_became_active":"2023-11-26T07:14:55.570671+0000","last_became_peered":"2023-11-26T07:14:55.570671+0000","last_unstale":"2023-11-26T07:14:56.541881+0000","last_undegraded":"2023-11-26T07:14:56.541881+0000","last_fullsized":"2023-11-26T07:14:56.541881+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:15:46.689885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542209+0000","last_change":"2023-11-26T07:14:55.565302+0000","last_active":"2023-11-26T07:14:56.542209+0000","last_peered":"2023-11-26T07:14:56.542209+0000","last_clean":"2023-11-26T07:14:56.542209+0000","last_became_active":"2023-11-26T07:14:55.565070+0000","last_became_peered":"2023-11-26T07:14:55.565070+0000","last_unstale":"2023-11-26T07:14:56.542209+0000","last_undegraded":"2023-11-26T07:14:56.542209+0000","last_fullsized":"2023-11-26T07:14:56.542209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:45.326329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877055+0000","last_change":"2023-11-26T07:14:55.568585+0000","last_active":"2023-11-26T07:14:56.877055+0000","last_peered":"2023-11-26T07:14:56.877055+0000","last_clean":"2023-11-26T07:14:56.877055+0000","last_became_active":"2023-11-26T07:14:55.568162+0000","last_became_peered":"2023-11-26T07:14:55.568162+0000","last_unstale":"2023-11-26T07:14:56.877055+0000","last_undegraded":"2023-11-26T07:14:56.877055+0000","last_fullsized":"2023-11-26T07:14:56.877055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:34.558173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541424+0000","last_change":"2023-11-26T07:14:55.550793+0000","last_active":"2023-11-26T07:14:56.541424+0000","last_peered":"2023-11-26T07:14:56.541424+0000","last_clean":"2023-11-26T07:14:56.541424+0000","last_became_active":"2023-11-26T07:14:55.550509+0000","last_became_peered":"2023-11-26T07:14:55.550509+0000","last_unstale":"2023-11-26T07:14:56.541424+0000","last_undegraded":"2023-11-26T07:14:56.541424+0000","last_fullsized":"2023-11-26T07:14:56.541424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:44:43.988623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876289+0000","last_change":"2023-11-26T07:14:55.563684+0000","last_active":"2023-11-26T07:14:56.876289+0000","last_peered":"2023-11-26T07:14:56.876289+0000","last_clean":"2023-11-26T07:14:56.876289+0000","last_became_active":"2023-11-26T07:14:55.562596+0000","last_became_peered":"2023-11-26T07:14:55.562596+0000","last_unstale":"2023-11-26T07:14:56.876289+0000","last_undegraded":"2023-11-26T07:14:56.876289+0000","last_fullsized":"2023-11-26T07:14:56.876289+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:00.018524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542146+0000","last_change":"2023-11-26T07:14:55.565993+0000","last_active":"2023-11-26T07:14:56.542146+0000","last_peered":"2023-11-26T07:14:56.542146+0000","last_clean":"2023-11-26T07:14:56.542146+0000","last_became_active":"2023-11-26T07:14:55.565743+0000","last_became_peered":"2023-11-26T07:14:55.565743+0000","last_unstale":"2023-11-26T07:14:56.542146+0000","last_undegraded":"2023-11-26T07:14:56.542146+0000","last_fullsized":"2023-11-26T07:14:56.542146+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:36.265114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.541275+0000","last_change":"2023-11-26T07:14:54.541275+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.541275+0000","last_undegraded":"2023-11-26T07:14:54.541275+0000","last_fullsized":"2023-11-26T07:14:54.541275+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542478+0000","last_change":"2023-11-26T07:14:55.560177+0000","last_active":"2023-11-26T07:14:56.542478+0000","last_peered":"2023-11-26T07:14:56.542478+0000","last_clean":"2023-11-26T07:14:56.542478+0000","last_became_active":"2023-11-26T07:14:55.559948+0000","last_became_peered":"2023-11-26T07:14:55.559948+0000","last_unstale":"2023-11-26T07:14:56.542478+0000","last_undegraded":"2023-11-26T07:14:56.542478+0000","last_fullsized":"2023-11-26T07:14:56.542478+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:22:36.861785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877238+0000","last_change":"2023-11-26T07:14:55.559369+0000","last_active":"2023-11-26T07:14:56.877238+0000","last_peered":"2023-11-26T07:14:56.877238+0000","last_clean":"2023-11-26T07:14:56.877238+0000","last_became_active":"2023-11-26T07:14:55.559088+0000","last_became_peered":"2023-11-26T07:14:55.559088+0000","last_unstale":"2023-11-26T07:14:56.877238+0000","last_undegraded":"2023-11-26T07:14:56.877238+0000","last_fullsized":"2023-11-26T07:14:56.877238+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:02.536615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877673+0000","last_change":"2023-11-26T07:14:55.569078+0000","last_active":"2023-11-26T07:14:56.877673+0000","last_peered":"2023-11-26T07:14:56.877673+0000","last_clean":"2023-11-26T07:14:56.877673+0000","last_became_active":"2023-11-26T07:14:55.568579+0000","last_became_peered":"2023-11-26T07:14:55.568579+0000","last_unstale":"2023-11-26T07:14:56.877673+0000","last_undegraded":"2023-11-26T07:14:56.877673+0000","last_fullsized":"2023-11-26T07:14:56.877673+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:01.838876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542281+0000","last_change":"2023-11-26T07:14:55.562903+0000","last_active":"2023-11-26T07:14:56.542281+0000","last_peered":"2023-11-26T07:14:56.542281+0000","last_clean":"2023-11-26T07:14:56.542281+0000","last_became_active":"2023-11-26T07:14:55.562637+0000","last_became_peered":"2023-11-26T07:14:55.562637+0000","last_unstale":"2023-11-26T07:14:56.542281+0000","last_undegraded":"2023-11-26T07:14:56.542281+0000","last_fullsized":"2023-11-26T07:14:56.542281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:22.475891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876656+0000","last_change":"2023-11-26T07:14:55.566349+0000","last_active":"2023-11-26T07:14:56.876656+0000","last_peered":"2023-11-26T07:14:56.876656+0000","last_clean":"2023-11-26T07:14:56.876656+0000","last_became_active":"2023-11-26T07:14:55.566043+0000","last_became_peered":"2023-11-26T07:14:55.566043+0000","last_unstale":"2023-11-26T07:14:56.876656+0000","last_undegraded":"2023-11-26T07:14:56.876656+0000","last_fullsized":"2023-11-26T07:14:56.876656+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:07:34.241609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541503+0000","last_change":"2023-11-26T07:14:55.547718+0000","last_active":"2023-11-26T07:14:56.541503+0000","last_peered":"2023-11-26T07:14:56.541503+0000","last_clean":"2023-11-26T07:14:56.541503+0000","last_became_active":"2023-11-26T07:14:55.547432+0000","last_became_peered":"2023-11-26T07:14:55.547432+0000","last_unstale":"2023-11-26T07:14:56.541503+0000","last_undegraded":"2023-11-26T07:14:56.541503+0000","last_fullsized":"2023-11-26T07:14:56.541503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:45:56.436888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542055+0000","last_change":"2023-11-26T07:14:55.560364+0000","last_active":"2023-11-26T07:14:56.542055+0000","last_peered":"2023-11-26T07:14:56.542055+0000","last_clean":"2023-11-26T07:14:56.542055+0000","last_became_active":"2023-11-26T07:14:55.559929+0000","last_became_peered":"2023-11-26T07:14:55.559929+0000","last_unstale":"2023-11-26T07:14:56.542055+0000","last_undegraded":"2023-11-26T07:14:56.542055+0000","last_fullsized":"2023-11-26T07:14:56.542055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:24:20.997710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876524+0000","last_change":"2023-11-26T07:14:55.563485+0000","last_active":"2023-11-26T07:14:56.876524+0000","last_peered":"2023-11-26T07:14:56.876524+0000","last_clean":"2023-11-26T07:14:56.876524+0000","last_became_active":"2023-11-26T07:14:55.563222+0000","last_became_peered":"2023-11-26T07:14:55.563222+0000","last_unstale":"2023-11-26T07:14:56.876524+0000","last_undegraded":"2023-11-26T07:14:56.876524+0000","last_fullsized":"2023-11-26T07:14:56.876524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:57.935087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876779+0000","last_change":"2023-11-26T07:14:55.569614+0000","last_active":"2023-11-26T07:14:56.876779+0000","last_peered":"2023-11-26T07:14:56.876779+0000","last_clean":"2023-11-26T07:14:56.876779+0000","last_became_active":"2023-11-26T07:14:55.567970+0000","last_became_peered":"2023-11-26T07:14:55.567970+0000","last_unstale":"2023-11-26T07:14:56.876779+0000","last_undegraded":"2023-11-26T07:14:56.876779+0000","last_fullsized":"2023-11-26T07:14:56.876779+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:02.536615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.569319+0000","last_change":"2023-11-26T07:14:54.551197+0000","last_active":"2023-11-26T07:14:55.569319+0000","last_peered":"2023-11-26T07:14:55.569319+0000","last_clean":"2023-11-26T07:14:55.569319+0000","last_became_active":"2023-11-26T07:14:54.550930+0000","last_became_peered":"2023-11-26T07:14:54.550930+0000","last_unstale":"2023-11-26T07:14:55.569319+0000","last_undegraded":"2023-11-26T07:14:55.569319+0000","last_fullsized":"2023-11-26T07:14:55.569319+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:24.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877481+0000","last_change":"2023-11-26T07:14:55.554217+0000","last_active":"2023-11-26T07:14:56.877481+0000","last_peered":"2023-11-26T07:14:56.877481+0000","last_clean":"2023-11-26T07:14:56.877481+0000","last_became_active":"2023-11-26T07:14:55.553893+0000","last_became_peered":"2023-11-26T07:14:55.553893+0000","last_unstale":"2023-11-26T07:14:56.877481+0000","last_undegraded":"2023-11-26T07:14:56.877481+0000","last_fullsized":"2023-11-26T07:14:56.877481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:02:52.062049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.566035+0000","last_change":"2023-11-26T07:14:54.556263+0000","last_active":"2023-11-26T07:14:55.566035+0000","last_peered":"2023-11-26T07:14:55.566035+0000","last_clean":"2023-11-26T07:14:55.566035+0000","last_became_active":"2023-11-26T07:14:54.555815+0000","last_became_peered":"2023-11-26T07:14:54.555815+0000","last_unstale":"2023-11-26T07:14:55.566035+0000","last_undegraded":"2023-11-26T07:14:55.566035+0000","last_fullsized":"2023-11-26T07:14:55.566035+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:13:38.502900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876491+0000","last_change":"2023-11-26T07:14:55.570096+0000","last_active":"2023-11-26T07:14:56.876491+0000","last_peered":"2023-11-26T07:14:56.876491+0000","last_clean":"2023-11-26T07:14:56.876491+0000","last_became_active":"2023-11-26T07:14:55.567546+0000","last_became_peered":"2023-11-26T07:14:55.567546+0000","last_unstale":"2023-11-26T07:14:56.876491+0000","last_undegraded":"2023-11-26T07:14:56.876491+0000","last_fullsized":"2023-11-26T07:14:56.876491+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:38.960124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.568272+0000","last_change":"2023-11-26T07:14:54.553820+0000","last_active":"2023-11-26T07:14:55.568272+0000","last_peered":"2023-11-26T07:14:55.568272+0000","last_clean":"2023-11-26T07:14:55.568272+0000","last_became_active":"2023-11-26T07:14:54.553552+0000","last_became_peered":"2023-11-26T07:14:54.553552+0000","last_unstale":"2023-11-26T07:14:55.568272+0000","last_undegraded":"2023-11-26T07:14:55.568272+0000","last_fullsized":"2023-11-26T07:14:55.568272+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:19.097609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877677+0000","last_change":"2023-11-26T07:14:55.568824+0000","last_active":"2023-11-26T07:14:56.877677+0000","last_peered":"2023-11-26T07:14:56.877677+0000","last_clean":"2023-11-26T07:14:56.877677+0000","last_became_active":"2023-11-26T07:14:55.568239+0000","last_became_peered":"2023-11-26T07:14:55.568239+0000","last_unstale":"2023-11-26T07:14:56.877677+0000","last_undegraded":"2023-11-26T07:14:56.877677+0000","last_fullsized":"2023-11-26T07:14:56.877677+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:48:48.988066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.561647+0000","last_change":"2023-11-26T07:14:54.555832+0000","last_active":"2023-11-26T07:14:55.561647+0000","last_peered":"2023-11-26T07:14:55.561647+0000","last_clean":"2023-11-26T07:14:55.561647+0000","last_became_active":"2023-11-26T07:14:54.555514+0000","last_became_peered":"2023-11-26T07:14:54.555514+0000","last_unstale":"2023-11-26T07:14:55.561647+0000","last_undegraded":"2023-11-26T07:14:55.561647+0000","last_fullsized":"2023-11-26T07:14:55.561647+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:24.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876443+0000","last_change":"2023-11-26T07:14:55.562781+0000","last_active":"2023-11-26T07:14:56.876443+0000","last_peered":"2023-11-26T07:14:56.876443+0000","last_clean":"2023-11-26T07:14:56.876443+0000","last_became_active":"2023-11-26T07:14:55.562323+0000","last_became_peered":"2023-11-26T07:14:55.562323+0000","last_unstale":"2023-11-26T07:14:56.876443+0000","last_undegraded":"2023-11-26T07:14:56.876443+0000","last_fullsized":"2023-11-26T07:14:56.876443+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:13.425773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.566845+0000","last_change":"2023-11-26T07:14:54.554968+0000","last_active":"2023-11-26T07:14:55.566845+0000","last_peered":"2023-11-26T07:14:55.566845+0000","last_clean":"2023-11-26T07:14:55.566845+0000","last_became_active":"2023-11-26T07:14:54.554664+0000","last_became_peered":"2023-11-26T07:14:54.554664+0000","last_unstale":"2023-11-26T07:14:55.566845+0000","last_undegraded":"2023-11-26T07:14:55.566845+0000","last_fullsized":"2023-11-26T07:14:55.566845+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:28.707932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876789+0000","last_change":"2023-11-26T07:14:55.568976+0000","last_active":"2023-11-26T07:14:56.876789+0000","last_peered":"2023-11-26T07:14:56.876789+0000","last_clean":"2023-11-26T07:14:56.876789+0000","last_became_active":"2023-11-26T07:14:55.568601+0000","last_became_peered":"2023-11-26T07:14:55.568601+0000","last_unstale":"2023-11-26T07:14:56.876789+0000","last_undegraded":"2023-11-26T07:14:56.876789+0000","last_fullsized":"2023-11-26T07:14:56.876789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:09.743461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.567289+0000","last_change":"2023-11-26T07:14:54.545826+0000","last_active":"2023-11-26T07:14:55.567289+0000","last_peered":"2023-11-26T07:14:55.567289+0000","last_clean":"2023-11-26T07:14:55.567289+0000","last_became_active":"2023-11-26T07:14:54.545558+0000","last_became_peered":"2023-11-26T07:14:54.545558+0000","last_unstale":"2023-11-26T07:14:55.567289+0000","last_undegraded":"2023-11-26T07:14:55.567289+0000","last_fullsized":"2023-11-26T07:14:55.567289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:56.667181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":56,"state":"creating+peering","last_fresh":"2023-11-26T07:14:54.540783+0000","last_change":"2023-11-26T07:14:54.540783+0000","last_active":"2023-11-26T07:14:54.530381+0000","last_peered":"2023-11-26T07:14:54.530381+0000","last_clean":"2023-11-26T07:14:54.530381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:14:54.540783+0000","last_undegraded":"2023-11-26T07:14:54.540783+0000","last_fullsized":"2023-11-26T07:14:54.540783+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.548344+0000","last_change":"2023-11-26T07:14:54.555979+0000","last_active":"2023-11-26T07:14:55.548344+0000","last_peered":"2023-11-26T07:14:55.548344+0000","last_clean":"2023-11-26T07:14:55.548344+0000","last_became_active":"2023-11-26T07:14:54.555658+0000","last_became_peered":"2023-11-26T07:14:54.555658+0000","last_unstale":"2023-11-26T07:14:55.548344+0000","last_undegraded":"2023-11-26T07:14:55.548344+0000","last_fullsized":"2023-11-26T07:14:55.548344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:12.852910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876522+0000","last_change":"2023-11-26T07:14:55.564714+0000","last_active":"2023-11-26T07:14:56.876522+0000","last_peered":"2023-11-26T07:14:56.876522+0000","last_clean":"2023-11-26T07:14:56.876522+0000","last_became_active":"2023-11-26T07:14:55.564443+0000","last_became_peered":"2023-11-26T07:14:55.564443+0000","last_unstale":"2023-11-26T07:14:56.876522+0000","last_undegraded":"2023-11-26T07:14:56.876522+0000","last_fullsized":"2023-11-26T07:14:56.876522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:07:30.647501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.541291+0000","last_change":"2023-11-26T07:14:54.556906+0000","last_active":"2023-11-26T07:14:55.541291+0000","last_peered":"2023-11-26T07:14:55.541291+0000","last_clean":"2023-11-26T07:14:55.541291+0000","last_became_active":"2023-11-26T07:14:54.556633+0000","last_became_peered":"2023-11-26T07:14:54.556633+0000","last_unstale":"2023-11-26T07:14:55.541291+0000","last_undegraded":"2023-11-26T07:14:55.541291+0000","last_fullsized":"2023-11-26T07:14:55.541291+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:10:48.528921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877149+0000","last_change":"2023-11-26T07:14:55.561086+0000","last_active":"2023-11-26T07:14:56.877149+0000","last_peered":"2023-11-26T07:14:56.877149+0000","last_clean":"2023-11-26T07:14:56.877149+0000","last_became_active":"2023-11-26T07:14:55.560811+0000","last_became_peered":"2023-11-26T07:14:55.560811+0000","last_unstale":"2023-11-26T07:14:56.877149+0000","last_undegraded":"2023-11-26T07:14:56.877149+0000","last_fullsized":"2023-11-26T07:14:56.877149+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:57.669199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.541992+0000","last_change":"2023-11-26T07:14:54.551547+0000","last_active":"2023-11-26T07:14:55.541992+0000","last_peered":"2023-11-26T07:14:55.541992+0000","last_clean":"2023-11-26T07:14:55.541992+0000","last_became_active":"2023-11-26T07:14:54.551059+0000","last_became_peered":"2023-11-26T07:14:54.551059+0000","last_unstale":"2023-11-26T07:14:55.541992+0000","last_undegraded":"2023-11-26T07:14:55.541992+0000","last_fullsized":"2023-11-26T07:14:55.541992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:56.933069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541907+0000","last_change":"2023-11-26T07:14:55.554669+0000","last_active":"2023-11-26T07:14:56.541907+0000","last_peered":"2023-11-26T07:14:56.541907+0000","last_clean":"2023-11-26T07:14:56.541907+0000","last_became_active":"2023-11-26T07:14:55.554402+0000","last_became_peered":"2023-11-26T07:14:55.554402+0000","last_unstale":"2023-11-26T07:14:56.541907+0000","last_undegraded":"2023-11-26T07:14:56.541907+0000","last_fullsized":"2023-11-26T07:14:56.541907+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:01:27.228579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.548485+0000","last_change":"2023-11-26T07:14:54.556255+0000","last_active":"2023-11-26T07:14:55.548485+0000","last_peered":"2023-11-26T07:14:55.548485+0000","last_clean":"2023-11-26T07:14:55.548485+0000","last_became_active":"2023-11-26T07:14:54.555863+0000","last_became_peered":"2023-11-26T07:14:54.555863+0000","last_unstale":"2023-11-26T07:14:55.548485+0000","last_undegraded":"2023-11-26T07:14:55.548485+0000","last_fullsized":"2023-11-26T07:14:55.548485+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:49.214175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876808+0000","last_change":"2023-11-26T07:14:55.555431+0000","last_active":"2023-11-26T07:14:56.876808+0000","last_peered":"2023-11-26T07:14:56.876808+0000","last_clean":"2023-11-26T07:14:56.876808+0000","last_became_active":"2023-11-26T07:14:55.555090+0000","last_became_peered":"2023-11-26T07:14:55.555090+0000","last_unstale":"2023-11-26T07:14:56.876808+0000","last_undegraded":"2023-11-26T07:14:56.876808+0000","last_fullsized":"2023-11-26T07:14:56.876808+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:28:00.260761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.571669+0000","last_change":"2023-11-26T07:14:54.553992+0000","last_active":"2023-11-26T07:14:55.571669+0000","last_peered":"2023-11-26T07:14:55.571669+0000","last_clean":"2023-11-26T07:14:55.571669+0000","last_became_active":"2023-11-26T07:14:54.553718+0000","last_became_peered":"2023-11-26T07:14:54.553718+0000","last_unstale":"2023-11-26T07:14:55.571669+0000","last_undegraded":"2023-11-26T07:14:55.571669+0000","last_fullsized":"2023-11-26T07:14:55.571669+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:02.663448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541322+0000","last_change":"2023-11-26T07:14:55.556639+0000","last_active":"2023-11-26T07:14:56.541322+0000","last_peered":"2023-11-26T07:14:56.541322+0000","last_clean":"2023-11-26T07:14:56.541322+0000","last_became_active":"2023-11-26T07:14:55.556441+0000","last_became_peered":"2023-11-26T07:14:55.556441+0000","last_unstale":"2023-11-26T07:14:56.541322+0000","last_undegraded":"2023-11-26T07:14:56.541322+0000","last_fullsized":"2023-11-26T07:14:56.541322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:59.370564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.574692+0000","last_change":"2023-11-26T07:14:54.557028+0000","last_active":"2023-11-26T07:14:55.574692+0000","last_peered":"2023-11-26T07:14:55.574692+0000","last_clean":"2023-11-26T07:14:55.574692+0000","last_became_active":"2023-11-26T07:14:54.556758+0000","last_became_peered":"2023-11-26T07:14:54.556758+0000","last_unstale":"2023-11-26T07:14:55.574692+0000","last_undegraded":"2023-11-26T07:14:55.574692+0000","last_fullsized":"2023-11-26T07:14:55.574692+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:58:50.594707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876740+0000","last_change":"2023-11-26T07:14:55.562711+0000","last_active":"2023-11-26T07:14:56.876740+0000","last_peered":"2023-11-26T07:14:56.876740+0000","last_clean":"2023-11-26T07:14:56.876740+0000","last_became_active":"2023-11-26T07:14:55.562505+0000","last_became_peered":"2023-11-26T07:14:55.562505+0000","last_unstale":"2023-11-26T07:14:56.876740+0000","last_undegraded":"2023-11-26T07:14:56.876740+0000","last_fullsized":"2023-11-26T07:14:56.876740+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:57.951234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.573668+0000","last_change":"2023-11-26T07:14:54.554993+0000","last_active":"2023-11-26T07:14:55.573668+0000","last_peered":"2023-11-26T07:14:55.573668+0000","last_clean":"2023-11-26T07:14:55.573668+0000","last_became_active":"2023-11-26T07:14:54.554750+0000","last_became_peered":"2023-11-26T07:14:54.554750+0000","last_unstale":"2023-11-26T07:14:55.573668+0000","last_undegraded":"2023-11-26T07:14:55.573668+0000","last_fullsized":"2023-11-26T07:14:55.573668+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:15:45.687867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541558+0000","last_change":"2023-11-26T07:14:55.552259+0000","last_active":"2023-11-26T07:14:56.541558+0000","last_peered":"2023-11-26T07:14:56.541558+0000","last_clean":"2023-11-26T07:14:56.541558+0000","last_became_active":"2023-11-26T07:14:55.551888+0000","last_became_peered":"2023-11-26T07:14:55.551888+0000","last_unstale":"2023-11-26T07:14:56.541558+0000","last_undegraded":"2023-11-26T07:14:56.541558+0000","last_fullsized":"2023-11-26T07:14:56.541558+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:13:39.504918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559486+0000","last_change":"2023-11-26T07:14:54.552221+0000","last_active":"2023-11-26T07:14:54.559486+0000","last_peered":"2023-11-26T07:14:54.559486+0000","last_clean":"2023-11-26T07:14:54.559486+0000","last_became_active":"2023-11-26T07:14:54.551932+0000","last_became_peered":"2023-11-26T07:14:54.551932+0000","last_unstale":"2023-11-26T07:14:54.559486+0000","last_undegraded":"2023-11-26T07:14:54.559486+0000","last_fullsized":"2023-11-26T07:14:54.559486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:00:25.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876587+0000","last_change":"2023-11-26T07:14:55.553875+0000","last_active":"2023-11-26T07:14:56.876587+0000","last_peered":"2023-11-26T07:14:56.876587+0000","last_clean":"2023-11-26T07:14:56.876587+0000","last_became_active":"2023-11-26T07:14:55.553581+0000","last_became_peered":"2023-11-26T07:14:55.553581+0000","last_unstale":"2023-11-26T07:14:56.876587+0000","last_undegraded":"2023-11-26T07:14:56.876587+0000","last_fullsized":"2023-11-26T07:14:56.876587+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:42.254062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.548315+0000","last_change":"2023-11-26T07:14:54.553341+0000","last_active":"2023-11-26T07:14:55.548315+0000","last_peered":"2023-11-26T07:14:55.548315+0000","last_clean":"2023-11-26T07:14:55.548315+0000","last_became_active":"2023-11-26T07:14:54.551872+0000","last_became_peered":"2023-11-26T07:14:54.551872+0000","last_unstale":"2023-11-26T07:14:55.548315+0000","last_undegraded":"2023-11-26T07:14:55.548315+0000","last_fullsized":"2023-11-26T07:14:55.548315+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:42.288092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541270+0000","last_change":"2023-11-26T07:14:55.566027+0000","last_active":"2023-11-26T07:14:56.541270+0000","last_peered":"2023-11-26T07:14:56.541270+0000","last_clean":"2023-11-26T07:14:56.541270+0000","last_became_active":"2023-11-26T07:14:55.565770+0000","last_became_peered":"2023-11-26T07:14:55.565770+0000","last_unstale":"2023-11-26T07:14:56.541270+0000","last_undegraded":"2023-11-26T07:14:56.541270+0000","last_fullsized":"2023-11-26T07:14:56.541270+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:37:15.036624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.566051+0000","last_change":"2023-11-26T07:14:54.551340+0000","last_active":"2023-11-26T07:14:55.566051+0000","last_peered":"2023-11-26T07:14:55.566051+0000","last_clean":"2023-11-26T07:14:55.566051+0000","last_became_active":"2023-11-26T07:14:54.551030+0000","last_became_peered":"2023-11-26T07:14:54.551030+0000","last_unstale":"2023-11-26T07:14:55.566051+0000","last_undegraded":"2023-11-26T07:14:55.566051+0000","last_fullsized":"2023-11-26T07:14:55.566051+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:24.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541944+0000","last_change":"2023-11-26T07:14:55.547115+0000","last_active":"2023-11-26T07:14:56.541944+0000","last_peered":"2023-11-26T07:14:56.541944+0000","last_clean":"2023-11-26T07:14:56.541944+0000","last_became_active":"2023-11-26T07:14:55.546831+0000","last_became_peered":"2023-11-26T07:14:55.546831+0000","last_unstale":"2023-11-26T07:14:56.541944+0000","last_undegraded":"2023-11-26T07:14:56.541944+0000","last_fullsized":"2023-11-26T07:14:56.541944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:00.018524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.569880+0000","last_change":"2023-11-26T07:14:54.558605+0000","last_active":"2023-11-26T07:14:55.569880+0000","last_peered":"2023-11-26T07:14:55.569880+0000","last_clean":"2023-11-26T07:14:55.569880+0000","last_became_active":"2023-11-26T07:14:54.558324+0000","last_became_peered":"2023-11-26T07:14:54.558324+0000","last_unstale":"2023-11-26T07:14:55.569880+0000","last_undegraded":"2023-11-26T07:14:55.569880+0000","last_fullsized":"2023-11-26T07:14:55.569880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:45:55.434870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877336+0000","last_change":"2023-11-26T07:14:55.562179+0000","last_active":"2023-11-26T07:14:56.877336+0000","last_peered":"2023-11-26T07:14:56.877336+0000","last_clean":"2023-11-26T07:14:56.877336+0000","last_became_active":"2023-11-26T07:14:55.561908+0000","last_became_peered":"2023-11-26T07:14:55.561908+0000","last_unstale":"2023-11-26T07:14:56.877336+0000","last_undegraded":"2023-11-26T07:14:56.877336+0000","last_fullsized":"2023-11-26T07:14:56.877336+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:01:50.191654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.567782+0000","last_change":"2023-11-26T07:14:54.556982+0000","last_active":"2023-11-26T07:14:55.567782+0000","last_peered":"2023-11-26T07:14:55.567782+0000","last_clean":"2023-11-26T07:14:55.567782+0000","last_became_active":"2023-11-26T07:14:54.556728+0000","last_became_peered":"2023-11-26T07:14:54.556728+0000","last_unstale":"2023-11-26T07:14:55.567782+0000","last_undegraded":"2023-11-26T07:14:55.567782+0000","last_fullsized":"2023-11-26T07:14:55.567782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:42.288092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877055+0000","last_change":"2023-11-26T07:14:55.564847+0000","last_active":"2023-11-26T07:14:56.877055+0000","last_peered":"2023-11-26T07:14:56.877055+0000","last_clean":"2023-11-26T07:14:56.877055+0000","last_became_active":"2023-11-26T07:14:55.564561+0000","last_became_peered":"2023-11-26T07:14:55.564561+0000","last_unstale":"2023-11-26T07:14:56.877055+0000","last_undegraded":"2023-11-26T07:14:56.877055+0000","last_fullsized":"2023-11-26T07:14:56.877055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:01:52.510162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.561778+0000","last_change":"2023-11-26T07:14:54.553413+0000","last_active":"2023-11-26T07:14:55.561778+0000","last_peered":"2023-11-26T07:14:55.561778+0000","last_clean":"2023-11-26T07:14:55.561778+0000","last_became_active":"2023-11-26T07:14:54.552941+0000","last_became_peered":"2023-11-26T07:14:54.552941+0000","last_unstale":"2023-11-26T07:14:55.561778+0000","last_undegraded":"2023-11-26T07:14:55.561778+0000","last_fullsized":"2023-11-26T07:14:55.561778+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:15:18.343075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541477+0000","last_change":"2023-11-26T07:14:55.559136+0000","last_active":"2023-11-26T07:14:56.541477+0000","last_peered":"2023-11-26T07:14:56.541477+0000","last_clean":"2023-11-26T07:14:56.541477+0000","last_became_active":"2023-11-26T07:14:55.558850+0000","last_became_peered":"2023-11-26T07:14:55.558850+0000","last_unstale":"2023-11-26T07:14:56.541477+0000","last_undegraded":"2023-11-26T07:14:56.541477+0000","last_fullsized":"2023-11-26T07:14:56.541477+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:50.216193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.554475+0000","last_change":"2023-11-26T07:14:54.546741+0000","last_active":"2023-11-26T07:14:55.554475+0000","last_peered":"2023-11-26T07:14:55.554475+0000","last_clean":"2023-11-26T07:14:55.554475+0000","last_became_active":"2023-11-26T07:14:54.546479+0000","last_became_peered":"2023-11-26T07:14:54.546479+0000","last_unstale":"2023-11-26T07:14:55.554475+0000","last_undegraded":"2023-11-26T07:14:55.554475+0000","last_fullsized":"2023-11-26T07:14:55.554475+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:08.574978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876882+0000","last_change":"2023-11-26T07:14:55.563245+0000","last_active":"2023-11-26T07:14:56.876882+0000","last_peered":"2023-11-26T07:14:56.876882+0000","last_clean":"2023-11-26T07:14:56.876882+0000","last_became_active":"2023-11-26T07:14:55.562937+0000","last_became_peered":"2023-11-26T07:14:55.562937+0000","last_unstale":"2023-11-26T07:14:56.876882+0000","last_undegraded":"2023-11-26T07:14:56.876882+0000","last_fullsized":"2023-11-26T07:14:56.876882+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:38.960124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.570157+0000","last_change":"2023-11-26T07:14:54.555163+0000","last_active":"2023-11-26T07:14:55.570157+0000","last_peered":"2023-11-26T07:14:55.570157+0000","last_clean":"2023-11-26T07:14:55.570157+0000","last_became_active":"2023-11-26T07:14:54.554331+0000","last_became_peered":"2023-11-26T07:14:54.554331+0000","last_unstale":"2023-11-26T07:14:55.570157+0000","last_undegraded":"2023-11-26T07:14:55.570157+0000","last_fullsized":"2023-11-26T07:14:55.570157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:32:22.345038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876915+0000","last_change":"2023-11-26T07:14:55.567533+0000","last_active":"2023-11-26T07:14:56.876915+0000","last_peered":"2023-11-26T07:14:56.876915+0000","last_clean":"2023-11-26T07:14:56.876915+0000","last_became_active":"2023-11-26T07:14:55.567256+0000","last_became_peered":"2023-11-26T07:14:55.567256+0000","last_unstale":"2023-11-26T07:14:56.876915+0000","last_undegraded":"2023-11-26T07:14:56.876915+0000","last_fullsized":"2023-11-26T07:14:56.876915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:01:50.191654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.542772+0000","last_change":"2023-11-26T07:14:54.554798+0000","last_active":"2023-11-26T07:14:55.542772+0000","last_peered":"2023-11-26T07:14:55.542772+0000","last_clean":"2023-11-26T07:14:55.542772+0000","last_became_active":"2023-11-26T07:14:54.554545+0000","last_became_peered":"2023-11-26T07:14:54.554545+0000","last_unstale":"2023-11-26T07:14:55.542772+0000","last_undegraded":"2023-11-26T07:14:55.542772+0000","last_fullsized":"2023-11-26T07:14:55.542772+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:22:35.859767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876276+0000","last_change":"2023-11-26T07:14:55.553523+0000","last_active":"2023-11-26T07:14:56.876276+0000","last_peered":"2023-11-26T07:14:56.876276+0000","last_clean":"2023-11-26T07:14:56.876276+0000","last_became_active":"2023-11-26T07:14:55.553227+0000","last_became_peered":"2023-11-26T07:14:55.553227+0000","last_unstale":"2023-11-26T07:14:56.876276+0000","last_undegraded":"2023-11-26T07:14:56.876276+0000","last_fullsized":"2023-11-26T07:14:56.876276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:39:06.952707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.547474+0000","last_change":"2023-11-26T07:14:54.556775+0000","last_active":"2023-11-26T07:14:55.547474+0000","last_peered":"2023-11-26T07:14:55.547474+0000","last_clean":"2023-11-26T07:14:55.547474+0000","last_became_active":"2023-11-26T07:14:54.556477+0000","last_became_peered":"2023-11-26T07:14:54.556477+0000","last_unstale":"2023-11-26T07:14:55.547474+0000","last_undegraded":"2023-11-26T07:14:55.547474+0000","last_fullsized":"2023-11-26T07:14:55.547474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:41:47.910749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877407+0000","last_change":"2023-11-26T07:14:55.556529+0000","last_active":"2023-11-26T07:14:56.877407+0000","last_peered":"2023-11-26T07:14:56.877407+0000","last_clean":"2023-11-26T07:14:56.877407+0000","last_became_active":"2023-11-26T07:14:55.556258+0000","last_became_peered":"2023-11-26T07:14:55.556258+0000","last_unstale":"2023-11-26T07:14:56.877407+0000","last_undegraded":"2023-11-26T07:14:56.877407+0000","last_fullsized":"2023-11-26T07:14:56.877407+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:48:48.988066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.567977+0000","last_change":"2023-11-26T07:14:54.559238+0000","last_active":"2023-11-26T07:14:55.567977+0000","last_peered":"2023-11-26T07:14:55.567977+0000","last_clean":"2023-11-26T07:14:55.567977+0000","last_became_active":"2023-11-26T07:14:54.558934+0000","last_became_peered":"2023-11-26T07:14:54.558934+0000","last_unstale":"2023-11-26T07:14:55.567977+0000","last_undegraded":"2023-11-26T07:14:55.567977+0000","last_fullsized":"2023-11-26T07:14:55.567977+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:44.510723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877355+0000","last_change":"2023-11-26T07:14:55.552256+0000","last_active":"2023-11-26T07:14:56.877355+0000","last_peered":"2023-11-26T07:14:56.877355+0000","last_clean":"2023-11-26T07:14:56.877355+0000","last_became_active":"2023-11-26T07:14:55.551982+0000","last_became_peered":"2023-11-26T07:14:55.551982+0000","last_unstale":"2023-11-26T07:14:56.877355+0000","last_undegraded":"2023-11-26T07:14:56.877355+0000","last_fullsized":"2023-11-26T07:14:56.877355+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:03.665466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876918+0000","last_change":"2023-11-26T07:14:54.550263+0000","last_active":"2023-11-26T07:14:56.876918+0000","last_peered":"2023-11-26T07:14:56.876918+0000","last_clean":"2023-11-26T07:14:56.876918+0000","last_became_active":"2023-11-26T07:14:54.549971+0000","last_became_peered":"2023-11-26T07:14:54.549971+0000","last_unstale":"2023-11-26T07:14:56.876918+0000","last_undegraded":"2023-11-26T07:14:56.876918+0000","last_fullsized":"2023-11-26T07:14:56.876918+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:12.852910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876936+0000","last_change":"2023-11-26T07:14:55.556173+0000","last_active":"2023-11-26T07:14:56.876936+0000","last_peered":"2023-11-26T07:14:56.876936+0000","last_clean":"2023-11-26T07:14:56.876936+0000","last_became_active":"2023-11-26T07:14:55.555920+0000","last_became_peered":"2023-11-26T07:14:55.555920+0000","last_unstale":"2023-11-26T07:14:56.876936+0000","last_undegraded":"2023-11-26T07:14:56.876936+0000","last_fullsized":"2023-11-26T07:14:56.876936+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:53.832663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559548+0000","last_change":"2023-11-26T07:14:54.552589+0000","last_active":"2023-11-26T07:14:54.559548+0000","last_peered":"2023-11-26T07:14:54.559548+0000","last_clean":"2023-11-26T07:14:54.559548+0000","last_became_active":"2023-11-26T07:14:54.552302+0000","last_became_peered":"2023-11-26T07:14:54.552302+0000","last_unstale":"2023-11-26T07:14:54.559548+0000","last_undegraded":"2023-11-26T07:14:54.559548+0000","last_fullsized":"2023-11-26T07:14:54.559548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:59.018549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877765+0000","last_change":"2023-11-26T07:14:55.567850+0000","last_active":"2023-11-26T07:14:56.877765+0000","last_peered":"2023-11-26T07:14:56.877765+0000","last_clean":"2023-11-26T07:14:56.877765+0000","last_became_active":"2023-11-26T07:14:55.567191+0000","last_became_peered":"2023-11-26T07:14:55.567191+0000","last_unstale":"2023-11-26T07:14:56.877765+0000","last_undegraded":"2023-11-26T07:14:56.877765+0000","last_fullsized":"2023-11-26T07:14:56.877765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:37:15.036624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.551783+0000","last_change":"2023-11-26T07:14:54.555982+0000","last_active":"2023-11-26T07:14:55.551783+0000","last_peered":"2023-11-26T07:14:55.551783+0000","last_clean":"2023-11-26T07:14:55.551783+0000","last_became_active":"2023-11-26T07:14:54.555678+0000","last_became_peered":"2023-11-26T07:14:54.555678+0000","last_unstale":"2023-11-26T07:14:55.551783+0000","last_undegraded":"2023-11-26T07:14:55.551783+0000","last_fullsized":"2023-11-26T07:14:55.551783+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:08.574978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876572+0000","last_change":"2023-11-26T07:14:55.562592+0000","last_active":"2023-11-26T07:14:56.876572+0000","last_peered":"2023-11-26T07:14:56.876572+0000","last_clean":"2023-11-26T07:14:56.876572+0000","last_became_active":"2023-11-26T07:14:55.562283+0000","last_became_peered":"2023-11-26T07:14:55.562283+0000","last_unstale":"2023-11-26T07:14:56.876572+0000","last_undegraded":"2023-11-26T07:14:56.876572+0000","last_fullsized":"2023-11-26T07:14:56.876572+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:40:50.418915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.542809+0000","last_change":"2023-11-26T07:14:54.556883+0000","last_active":"2023-11-26T07:14:55.542809+0000","last_peered":"2023-11-26T07:14:55.542809+0000","last_clean":"2023-11-26T07:14:55.542809+0000","last_became_active":"2023-11-26T07:14:54.556654+0000","last_became_peered":"2023-11-26T07:14:54.556654+0000","last_unstale":"2023-11-26T07:14:55.542809+0000","last_undegraded":"2023-11-26T07:14:55.542809+0000","last_fullsized":"2023-11-26T07:14:55.542809+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:00:25.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541680+0000","last_change":"2023-11-26T07:14:55.567327+0000","last_active":"2023-11-26T07:14:56.541680+0000","last_peered":"2023-11-26T07:14:56.541680+0000","last_clean":"2023-11-26T07:14:56.541680+0000","last_became_active":"2023-11-26T07:14:55.567068+0000","last_became_peered":"2023-11-26T07:14:55.567068+0000","last_unstale":"2023-11-26T07:14:56.541680+0000","last_undegraded":"2023-11-26T07:14:56.541680+0000","last_fullsized":"2023-11-26T07:14:56.541680+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:32:23.347056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.542458+0000","last_change":"2023-11-26T07:14:54.555216+0000","last_active":"2023-11-26T07:14:55.542458+0000","last_peered":"2023-11-26T07:14:55.542458+0000","last_clean":"2023-11-26T07:14:55.542458+0000","last_became_active":"2023-11-26T07:14:54.554577+0000","last_became_peered":"2023-11-26T07:14:54.554577+0000","last_unstale":"2023-11-26T07:14:55.542458+0000","last_undegraded":"2023-11-26T07:14:55.542458+0000","last_fullsized":"2023-11-26T07:14:55.542458+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:43.781381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876106+0000","last_change":"2023-11-26T07:14:55.558942+0000","last_active":"2023-11-26T07:14:56.876106+0000","last_peered":"2023-11-26T07:14:56.876106+0000","last_clean":"2023-11-26T07:14:56.876106+0000","last_became_active":"2023-11-26T07:14:55.558687+0000","last_became_peered":"2023-11-26T07:14:55.558687+0000","last_unstale":"2023-11-26T07:14:56.876106+0000","last_undegraded":"2023-11-26T07:14:56.876106+0000","last_fullsized":"2023-11-26T07:14:56.876106+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:16:05.761932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.543069+0000","last_change":"2023-11-26T07:14:54.556996+0000","last_active":"2023-11-26T07:14:55.543069+0000","last_peered":"2023-11-26T07:14:55.543069+0000","last_clean":"2023-11-26T07:14:55.543069+0000","last_became_active":"2023-11-26T07:14:54.556722+0000","last_became_peered":"2023-11-26T07:14:54.556722+0000","last_unstale":"2023-11-26T07:14:55.543069+0000","last_undegraded":"2023-11-26T07:14:55.543069+0000","last_fullsized":"2023-11-26T07:14:55.543069+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:05.353039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876662+0000","last_change":"2023-11-26T07:14:55.565201+0000","last_active":"2023-11-26T07:14:56.876662+0000","last_peered":"2023-11-26T07:14:56.876662+0000","last_clean":"2023-11-26T07:14:56.876662+0000","last_became_active":"2023-11-26T07:14:55.564935+0000","last_became_peered":"2023-11-26T07:14:55.564935+0000","last_unstale":"2023-11-26T07:14:56.876662+0000","last_undegraded":"2023-11-26T07:14:56.876662+0000","last_fullsized":"2023-11-26T07:14:56.876662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:51:19.407229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.549059+0000","last_change":"2023-11-26T07:14:54.550363+0000","last_active":"2023-11-26T07:14:55.549059+0000","last_peered":"2023-11-26T07:14:55.549059+0000","last_clean":"2023-11-26T07:14:55.549059+0000","last_became_active":"2023-11-26T07:14:54.550093+0000","last_became_peered":"2023-11-26T07:14:54.550093+0000","last_unstale":"2023-11-26T07:14:55.549059+0000","last_undegraded":"2023-11-26T07:14:55.549059+0000","last_fullsized":"2023-11-26T07:14:55.549059+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:01:26.226561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876495+0000","last_change":"2023-11-26T07:14:55.561975+0000","last_active":"2023-11-26T07:14:56.876495+0000","last_peered":"2023-11-26T07:14:56.876495+0000","last_clean":"2023-11-26T07:14:56.876495+0000","last_became_active":"2023-11-26T07:14:55.561323+0000","last_became_peered":"2023-11-26T07:14:55.561323+0000","last_unstale":"2023-11-26T07:14:56.876495+0000","last_undegraded":"2023-11-26T07:14:56.876495+0000","last_fullsized":"2023-11-26T07:14:56.876495+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:13:03.157321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:14:54.559599+0000","last_change":"2023-11-26T07:14:54.552852+0000","last_active":"2023-11-26T07:14:54.559599+0000","last_peered":"2023-11-26T07:14:54.559599+0000","last_clean":"2023-11-26T07:14:54.559599+0000","last_became_active":"2023-11-26T07:14:54.552543+0000","last_became_peered":"2023-11-26T07:14:54.552543+0000","last_unstale":"2023-11-26T07:14:54.559599+0000","last_undegraded":"2023-11-26T07:14:54.559599+0000","last_fullsized":"2023-11-26T07:14:54.559599+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:30.337901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.540389+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:14:55.540389+0000","last_peered":"2023-11-26T07:14:55.540389+0000","last_clean":"2023-11-26T07:14:55.540389+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:14:55.540389+0000","last_undegraded":"2023-11-26T07:14:55.540389+0000","last_fullsized":"2023-11-26T07:14:55.540389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1298432,"data_stored":880333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.55900000000000005},"min":{"1min":0.44,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.52200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.53800000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.47299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73799999999999999,"5min":1.048,"15min":1.048},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.497,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.5}]},{"osd":2,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.495,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.37},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.621,"15min":0.622},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75600000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51700000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70499999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77800000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76700000000000002,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.46999999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.37,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79500000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79000000000000004,"5min":1.171,"15min":1.171},"last":0.63100000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.74199999999999999,"5min":1.361,"15min":1.361},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78000000000000003,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":276730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.39200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.73699999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76600000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.49099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.69799999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.45800000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72299999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82599999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.753,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70399999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56799999999999995,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67900000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.51200000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.747,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73399999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":289895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.502,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.34899999999999998,"15min":0.34599999999999997},"min":{"1min":0.20000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.5,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.379}]},{"osd":2,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34000000000000002,"15min":0.33400000000000002},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.27200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53000000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.33700000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.252,"5min":0.246,"15min":0.246},"max":{"1min":0.57499999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.432},{"interface":"front","average":{"1min":0.434,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.877,"15min":0.877},"last":0.499},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.623,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":0.995,"15min":0.995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.46800000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53200000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88800000000000001,"5min":1.129,"15min":1.129},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.627,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57399999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.628},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47399999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83599999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.61799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":882367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.221,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54200000000000004,"5min":0.623,"15min":0.623},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.443,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.34799999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.629,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.28599999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.29799999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61699999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.437,"15min":0.439},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42799999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67000000000000004,"15min":0.67300000000000004},"min":{"1min":0.499,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96599999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52500000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.625},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83799999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.55100000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45100000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.503}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36099999999999999,"15min":0.36899999999999999},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.497,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.41899999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.27900000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58799999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.498},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.47199999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.27400000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.56499999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42599999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54000000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.44800000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64800000000000002,"15min":0.67600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85799999999999998,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79300000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.78900000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.621,"15min":0.624},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81399999999999995,"5min":1.075,"15min":1.075},"last":0.624},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.72799999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80100000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.47099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.77900000000000003,"5min":1.03,"15min":1.03},"last":0.77100000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.65900000000000003},"min":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.622,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82799999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.74399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.218,"5min":0.188,"15min":0.188},"max":{"1min":0.42199999999999999,"5min":1.284,"15min":1.284},"last":0.313},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.44900000000000001,"15min":0.47999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.55500000000000005,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.29999999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.47499999999999998,"15min":0.52700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61799999999999999,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.46500000000000002,"15min":0.498},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.50700000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.374}]},{"osd":2,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.50800000000000001},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.59299999999999997,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52700000000000002,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.35999999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.68000000000000005},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85299999999999998,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88200000000000001,"5min":1.405,"15min":1.405},"last":0.56200000000000006}]},{"osd":5,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.50700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83799999999999997,"5min":1.377,"15min":1.377},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":0.80900000000000005},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91100000000000003,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66200000000000003,"15min":0.67500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92500000000000004,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.75600000000000001,"15min":0.80200000000000005},"min":{"1min":0.51700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.871,"5min":4.883,"15min":4.883},"last":0.64800000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.75700000000000001,"15min":0.81899999999999995},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80200000000000005,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71599999999999997,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85299999999999998,"5min":2.923,"15min":2.923},"last":0.66200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":291417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.41499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77500000000000002,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.43099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45500000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.53000000000000003,"15min":0.51200000000000001},"min":{"1min":0.44700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77500000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.63400000000000001,"15min":0.66400000000000003},"min":{"1min":0.45600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71099999999999997,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.58099999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54300000000000004,"15min":0.52500000000000002},"min":{"1min":0.47899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.68600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.64700000000000002,"15min":0.67900000000000005},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74299999999999999,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.625}]},{"osd":3,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.40500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75900000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73399999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.498,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.751}]},{"osd":6,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.54300000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72999999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.73899999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71699999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.625,"15min":0.63600000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79600000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.82899999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":290457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.39200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.49099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57499999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.48799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.73799999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69699999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.42399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.45900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64100000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.498,"15min":0.5},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.61599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67300000000000004,"5min":0.752,"15min":0.752},"last":0.60899999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.65800000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70899999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.62}]},{"osd":7,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.629},{"interface":"front","average":{"1min":0.625,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76600000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.746}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:15:01.243 INFO:tasks.ceph.mds.a.smithi083.stderr:2023-11-26T07:15:01.242+0000 7fa61fefb700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:15:04.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:15:08.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.292+0000 7fc1fed55700 1 Processor -- start 2023-11-26T07:15:08.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.293+0000 7fc1fed55700 1 -- start start 2023-11-26T07:15:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fed55700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1f815bed0 0x7fc1f80b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fed55700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1f80d2160 0x7fc1f80d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fed55700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f80d0770 con 0x7fc1f81603c0 2023-11-26T07:15:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fed55700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f815c2b0 con 0x7fc1f80d2160 2023-11-26T07:15:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fed55700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80cd100 con 0x7fc1f815bed0 2023-11-26T07:15:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1f7fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc1f81603c0 0x7fc1f80d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58550/0 (socket says 172.21.15.83:58550) 2023-11-26T07:15:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1f7fff700 1 -- 172.21.15.83:0/1637727148 learned_addr learned my addr 172.21.15.83:0/1637727148 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:08.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fd2f2700 1 --2- 172.21.15.83:0/1637727148 >> v2:172.21.15.83:3300/0 conn(0x7fc1f80d2160 0x7fc1f80d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fcaf1700 1 --2- 172.21.15.83:0/1637727148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1f815bed0 0x7fc1f80b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fd2f2700 1 -- 172.21.15.83:0/1637727148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1f815bed0 msgr2=0x7fc1f80b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fd2f2700 1 --2- 172.21.15.83:0/1637727148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1f815bed0 0x7fc1f80b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fd2f2700 1 -- 172.21.15.83:0/1637727148 >> v1:172.21.15.83:6789/0 conn(0x7fc1f81603c0 legacy=0x7fc1f80d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fd2f2700 1 -- 172.21.15.83:0/1637727148 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f80d09a0 con 0x7fc1f80d2160 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1f77fe700 1 -- 172.21.15.83:0/1637727148 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906439535 0 0) 0x7fc1f80d0770 con 0x7fc1f81603c0 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1fd2f2700 1 --2- 172.21.15.83:0/1637727148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80d2160 0x7fc1f80d0230 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009d90 tx=0x7fc1f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48946c60ee5e4816 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.294+0000 7fc1f77fe700 1 -- 172.21.15.83:0/1637727148 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f000e070 con 0x7fc1f80d2160 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.295+0000 7fc1f77fe700 1 -- 172.21.15.83:0/1637727148 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f000bbd0 con 0x7fc1f80d2160 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.295+0000 7fc1f77fe700 1 -- 172.21.15.83:0/1637727148 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0012430 con 0x7fc1f80d2160 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.295+0000 7fc1fed55700 1 -- 172.21.15.83:0/1637727148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80d2160 msgr2=0x7fc1f80d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:08.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.295+0000 7fc1fed55700 1 --2- 172.21.15.83:0/1637727148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80d2160 0x7fc1f80d0230 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009d90 tx=0x7fc1f000b330 comp rx=0 tx=0).stop 2023-11-26T07:15:08.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.295+0000 7fc1fed55700 1 -- 172.21.15.83:0/1637727148 shutdown_connections 2023-11-26T07:15:08.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.295+0000 7fc1fed55700 1 --2- 172.21.15.83:0/1637727148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80d2160 0x7fc1f80d0230 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.295+0000 7fc1fed55700 1 --2- 172.21.15.83:0/1637727148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1f815bed0 0x7fc1f80b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.295+0000 7fc1fed55700 1 -- 172.21.15.83:0/1637727148 >> 172.21.15.83:0/1637727148 conn(0x7fc1f8073130 msgr2=0x7fc1f809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:08.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.295+0000 7fc1fed55700 1 -- 172.21.15.83:0/1637727148 shutdown_connections 2023-11-26T07:15:08.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.295+0000 7fc1fed55700 1 -- 172.21.15.83:0/1637727148 wait complete. 2023-11-26T07:15:08.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.297+0000 7fc1fed55700 1 Processor -- start 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.297+0000 7fc1fed55700 1 -- start start 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fed55700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80d2160 0x7fc1f81f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fed55700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1f815bed0 0x7fc1f815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fed55700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f81603c0 0x7fc1f81f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fed55700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f815e6c0 con 0x7fc1f80d2160 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fed55700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f81f5ac0 con 0x7fc1f81603c0 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fed55700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f814fba0 con 0x7fc1f815bed0 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fcaf1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80d2160 0x7fc1f81f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fcaf1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80d2160 0x7fc1f81f4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38514/0 (socket says 172.21.15.83:38514) 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fcaf1700 1 -- 172.21.15.83:0/1744599838 learned_addr learned my addr 172.21.15.83:0/1744599838 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fd2f2700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f81603c0 0x7fc1f81f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1f7fff700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1f815bed0 0x7fc1f815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fcaf1700 1 -- 172.21.15.83:0/1744599838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1f815bed0 msgr2=0x7fc1f815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fcaf1700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1f815bed0 0x7fc1f815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fcaf1700 1 -- 172.21.15.83:0/1744599838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f81603c0 msgr2=0x7fc1f81f0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fcaf1700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f81603c0 0x7fc1f81f0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fcaf1700 1 -- 172.21.15.83:0/1744599838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f0009a70 con 0x7fc1f80d2160 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1fcaf1700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80d2160 0x7fc1f81f4b80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00e840 tx=0x7fc1ec00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.298+0000 7fc1f57fa700 1 -- 172.21.15.83:0/1744599838 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec013610 con 0x7fc1f80d2160 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.299+0000 7fc1f57fa700 1 -- 172.21.15.83:0/1744599838 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ec013770 con 0x7fc1f80d2160 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.299+0000 7fc1fed55700 1 -- 172.21.15.83:0/1744599838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f80ca150 con 0x7fc1f80d2160 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.299+0000 7fc1f57fa700 1 -- 172.21.15.83:0/1744599838 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec013b20 con 0x7fc1f80d2160 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.299+0000 7fc1fed55700 1 -- 172.21.15.83:0/1744599838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f81f5500 con 0x7fc1f80d2160 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.300+0000 7fc1fed55700 1 -- 172.21.15.83:0/1744599838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f80c9f40 con 0x7fc1f80d2160 2023-11-26T07:15:08.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.300+0000 7fc1f57fa700 1 -- 172.21.15.83:0/1744599838 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fc1ec013c80 con 0x7fc1f80d2160 2023-11-26T07:15:08.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.300+0000 7fc1f57fa700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc1dc06b080 0x7fc1dc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.300+0000 7fc1f57fa700 1 -- 172.21.15.83:0/1744599838 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1ec089690 con 0x7fc1f80d2160 2023-11-26T07:15:08.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.300+0000 7fc1f7fff700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc1dc06b080 0x7fc1dc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.301+0000 7fc1f7fff700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc1dc06b080 0x7fc1dc06d540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc1f81f4ae0 tx=0x7fc1e4007be0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:08.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.305+0000 7fc1f57fa700 1 -- 172.21.15.83:0/1744599838 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc1ec0543b0 con 0x7fc1f80d2160 2023-11-26T07:15:08.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.497+0000 7fc1fed55700 1 -- 172.21.15.83:0/1744599838 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc1f80ccab0 con 0x7fc1dc06b080 2023-11-26T07:15:08.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.505+0000 7fc1f57fa700 1 -- 172.21.15.83:0/1744599838 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289289 (secure 0 0 0) 0x7fc1e40012b0 con 0x7fc1dc06b080 2023-11-26T07:15:08.509 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 -- 172.21.15.83:0/1744599838 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc1dc06b080 msgr2=0x7fc1dc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc1dc06b080 0x7fc1dc06d540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc1f81f4ae0 tx=0x7fc1e4007be0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 -- 172.21.15.83:0/1744599838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80d2160 msgr2=0x7fc1f81f4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80d2160 0x7fc1f81f4b80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00e840 tx=0x7fc1ec00ec00 comp rx=0 tx=0).stop 2023-11-26T07:15:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 -- 172.21.15.83:0/1744599838 shutdown_connections 2023-11-26T07:15:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1f81603c0 0x7fc1f81f0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80d2160 0x7fc1f81f4b80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc1dc06b080 0x7fc1dc06d540 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 --2- 172.21.15.83:0/1744599838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1f815bed0 0x7fc1f815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 -- 172.21.15.83:0/1744599838 >> 172.21.15.83:0/1744599838 conn(0x7fc1f8073130 msgr2=0x7fc1f8159360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 -- 172.21.15.83:0/1744599838 shutdown_connections 2023-11-26T07:15:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.508+0000 7fc1fed55700 1 -- 172.21.15.83:0/1744599838 wait complete. 2023-11-26T07:15:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:15:08.521 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2023-11-26T07:15:08.355553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222852,"kb_used_data":6664,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751868,"statfs":{"total":773094113280,"available":772865912832,"internally_reserved":0,"allocated":6823936,"data_stored":3503996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.814640"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542403+0000","last_change":"2023-11-26T07:14:55.565782+0000","last_active":"2023-11-26T07:14:56.542403+0000","last_peered":"2023-11-26T07:14:56.542403+0000","last_clean":"2023-11-26T07:14:56.542403+0000","last_became_active":"2023-11-26T07:14:55.565549+0000","last_became_peered":"2023-11-26T07:14:55.565549+0000","last_unstale":"2023-11-26T07:14:56.542403+0000","last_undegraded":"2023-11-26T07:14:56.542403+0000","last_fullsized":"2023-11-26T07:14:56.542403+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:31:48.907743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877008+0000","last_change":"2023-11-26T07:14:55.563586+0000","last_active":"2023-11-26T07:14:56.877008+0000","last_peered":"2023-11-26T07:14:56.877008+0000","last_clean":"2023-11-26T07:14:56.877008+0000","last_became_active":"2023-11-26T07:14:55.563306+0000","last_became_peered":"2023-11-26T07:14:55.563306+0000","last_unstale":"2023-11-26T07:14:56.877008+0000","last_undegraded":"2023-11-26T07:14:56.877008+0000","last_fullsized":"2023-11-26T07:14:56.877008+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:04:44.976495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541137+0000","last_change":"2023-11-26T07:14:55.547529+0000","last_active":"2023-11-26T07:14:56.541137+0000","last_peered":"2023-11-26T07:14:56.541137+0000","last_clean":"2023-11-26T07:14:56.541137+0000","last_became_active":"2023-11-26T07:14:55.547248+0000","last_became_peered":"2023-11-26T07:14:55.547248+0000","last_unstale":"2023-11-26T07:14:56.541137+0000","last_undegraded":"2023-11-26T07:14:56.541137+0000","last_fullsized":"2023-11-26T07:14:56.541137+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:48:48.988066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877177+0000","last_change":"2023-11-26T07:14:55.556871+0000","last_active":"2023-11-26T07:14:56.877177+0000","last_peered":"2023-11-26T07:14:56.877177+0000","last_clean":"2023-11-26T07:14:56.877177+0000","last_became_active":"2023-11-26T07:14:55.556600+0000","last_became_peered":"2023-11-26T07:14:55.556600+0000","last_unstale":"2023-11-26T07:14:56.877177+0000","last_undegraded":"2023-11-26T07:14:56.877177+0000","last_fullsized":"2023-11-26T07:14:56.877177+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:57.935087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542195+0000","last_change":"2023-11-26T07:14:55.556264+0000","last_active":"2023-11-26T07:14:56.542195+0000","last_peered":"2023-11-26T07:14:56.542195+0000","last_clean":"2023-11-26T07:14:56.542195+0000","last_became_active":"2023-11-26T07:14:55.555983+0000","last_became_peered":"2023-11-26T07:14:55.555983+0000","last_unstale":"2023-11-26T07:14:56.542195+0000","last_undegraded":"2023-11-26T07:14:56.542195+0000","last_fullsized":"2023-11-26T07:14:56.542195+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:16:59.337538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876724+0000","last_change":"2023-11-26T07:14:55.557549+0000","last_active":"2023-11-26T07:14:56.876724+0000","last_peered":"2023-11-26T07:14:56.876724+0000","last_clean":"2023-11-26T07:14:56.876724+0000","last_became_active":"2023-11-26T07:14:55.557282+0000","last_became_peered":"2023-11-26T07:14:55.557282+0000","last_unstale":"2023-11-26T07:14:56.876724+0000","last_undegraded":"2023-11-26T07:14:56.876724+0000","last_fullsized":"2023-11-26T07:14:56.876724+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:03.665466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877024+0000","last_change":"2023-11-26T07:14:55.554176+0000","last_active":"2023-11-26T07:14:56.877024+0000","last_peered":"2023-11-26T07:14:56.877024+0000","last_clean":"2023-11-26T07:14:56.877024+0000","last_became_active":"2023-11-26T07:14:55.553858+0000","last_became_peered":"2023-11-26T07:14:55.553858+0000","last_unstale":"2023-11-26T07:14:56.877024+0000","last_undegraded":"2023-11-26T07:14:56.877024+0000","last_fullsized":"2023-11-26T07:14:56.877024+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:39:44.105610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876382+0000","last_change":"2023-11-26T07:14:55.561614+0000","last_active":"2023-11-26T07:14:56.876382+0000","last_peered":"2023-11-26T07:14:56.876382+0000","last_clean":"2023-11-26T07:14:56.876382+0000","last_became_active":"2023-11-26T07:14:55.561004+0000","last_became_peered":"2023-11-26T07:14:55.561004+0000","last_unstale":"2023-11-26T07:14:56.876382+0000","last_undegraded":"2023-11-26T07:14:56.876382+0000","last_fullsized":"2023-11-26T07:14:56.876382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:36.265114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877062+0000","last_change":"2023-11-26T07:14:55.568602+0000","last_active":"2023-11-26T07:14:56.877062+0000","last_peered":"2023-11-26T07:14:56.877062+0000","last_clean":"2023-11-26T07:14:56.877062+0000","last_became_active":"2023-11-26T07:14:55.568272+0000","last_became_peered":"2023-11-26T07:14:55.568272+0000","last_unstale":"2023-11-26T07:14:56.877062+0000","last_undegraded":"2023-11-26T07:14:56.877062+0000","last_fullsized":"2023-11-26T07:14:56.877062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:40.727595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877245+0000","last_change":"2023-11-26T07:14:55.555237+0000","last_active":"2023-11-26T07:14:56.877245+0000","last_peered":"2023-11-26T07:14:56.877245+0000","last_clean":"2023-11-26T07:14:56.877245+0000","last_became_active":"2023-11-26T07:14:55.554973+0000","last_became_peered":"2023-11-26T07:14:55.554973+0000","last_unstale":"2023-11-26T07:14:56.877245+0000","last_undegraded":"2023-11-26T07:14:56.877245+0000","last_fullsized":"2023-11-26T07:14:56.877245+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:01:53.326813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877295+0000","last_change":"2023-11-26T07:14:55.567135+0000","last_active":"2023-11-26T07:14:56.877295+0000","last_peered":"2023-11-26T07:14:56.877295+0000","last_clean":"2023-11-26T07:14:56.877295+0000","last_became_active":"2023-11-26T07:14:55.566885+0000","last_became_peered":"2023-11-26T07:14:55.566885+0000","last_unstale":"2023-11-26T07:14:56.877295+0000","last_undegraded":"2023-11-26T07:14:56.877295+0000","last_fullsized":"2023-11-26T07:14:56.877295+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:25:05.901539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877351+0000","last_change":"2023-11-26T07:14:55.555316+0000","last_active":"2023-11-26T07:14:56.877351+0000","last_peered":"2023-11-26T07:14:56.877351+0000","last_clean":"2023-11-26T07:14:56.877351+0000","last_became_active":"2023-11-26T07:14:55.555043+0000","last_became_peered":"2023-11-26T07:14:55.555043+0000","last_unstale":"2023-11-26T07:14:56.877351+0000","last_undegraded":"2023-11-26T07:14:56.877351+0000","last_fullsized":"2023-11-26T07:14:56.877351+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:01.838876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541997+0000","last_change":"2023-11-26T07:14:55.570324+0000","last_active":"2023-11-26T07:14:56.541997+0000","last_peered":"2023-11-26T07:14:56.541997+0000","last_clean":"2023-11-26T07:14:56.541997+0000","last_became_active":"2023-11-26T07:14:55.570025+0000","last_became_peered":"2023-11-26T07:14:55.570025+0000","last_unstale":"2023-11-26T07:14:56.541997+0000","last_undegraded":"2023-11-26T07:14:56.541997+0000","last_fullsized":"2023-11-26T07:14:56.541997+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:01:53.326813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876769+0000","last_change":"2023-11-26T07:14:55.557930+0000","last_active":"2023-11-26T07:14:56.876769+0000","last_peered":"2023-11-26T07:14:56.876769+0000","last_clean":"2023-11-26T07:14:56.876769+0000","last_became_active":"2023-11-26T07:14:55.557647+0000","last_became_peered":"2023-11-26T07:14:55.557647+0000","last_unstale":"2023-11-26T07:14:56.876769+0000","last_undegraded":"2023-11-26T07:14:56.876769+0000","last_fullsized":"2023-11-26T07:14:56.876769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:00:31.201849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877113+0000","last_change":"2023-11-26T07:14:55.559368+0000","last_active":"2023-11-26T07:14:56.877113+0000","last_peered":"2023-11-26T07:14:56.877113+0000","last_clean":"2023-11-26T07:14:56.877113+0000","last_became_active":"2023-11-26T07:14:55.558965+0000","last_became_peered":"2023-11-26T07:14:55.558965+0000","last_unstale":"2023-11-26T07:14:56.877113+0000","last_undegraded":"2023-11-26T07:14:56.877113+0000","last_fullsized":"2023-11-26T07:14:56.877113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:41:48.912767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876397+0000","last_change":"2023-11-26T07:14:55.558759+0000","last_active":"2023-11-26T07:14:56.876397+0000","last_peered":"2023-11-26T07:14:56.876397+0000","last_clean":"2023-11-26T07:14:56.876397+0000","last_became_active":"2023-11-26T07:14:55.558507+0000","last_became_peered":"2023-11-26T07:14:55.558507+0000","last_unstale":"2023-11-26T07:14:56.876397+0000","last_undegraded":"2023-11-26T07:14:56.876397+0000","last_fullsized":"2023-11-26T07:14:56.876397+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:57.951234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876827+0000","last_change":"2023-11-26T07:14:55.562951+0000","last_active":"2023-11-26T07:14:56.876827+0000","last_peered":"2023-11-26T07:14:56.876827+0000","last_clean":"2023-11-26T07:14:56.876827+0000","last_became_active":"2023-11-26T07:14:55.562289+0000","last_became_peered":"2023-11-26T07:14:55.562289+0000","last_unstale":"2023-11-26T07:14:56.876827+0000","last_undegraded":"2023-11-26T07:14:56.876827+0000","last_fullsized":"2023-11-26T07:14:56.876827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:47:37.463547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541881+0000","last_change":"2023-11-26T07:14:55.570981+0000","last_active":"2023-11-26T07:14:56.541881+0000","last_peered":"2023-11-26T07:14:56.541881+0000","last_clean":"2023-11-26T07:14:56.541881+0000","last_became_active":"2023-11-26T07:14:55.570671+0000","last_became_peered":"2023-11-26T07:14:55.570671+0000","last_unstale":"2023-11-26T07:14:56.541881+0000","last_undegraded":"2023-11-26T07:14:56.541881+0000","last_fullsized":"2023-11-26T07:14:56.541881+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:15:46.689885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542209+0000","last_change":"2023-11-26T07:14:55.565302+0000","last_active":"2023-11-26T07:14:56.542209+0000","last_peered":"2023-11-26T07:14:56.542209+0000","last_clean":"2023-11-26T07:14:56.542209+0000","last_became_active":"2023-11-26T07:14:55.565070+0000","last_became_peered":"2023-11-26T07:14:55.565070+0000","last_unstale":"2023-11-26T07:14:56.542209+0000","last_undegraded":"2023-11-26T07:14:56.542209+0000","last_fullsized":"2023-11-26T07:14:56.542209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:45.326329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877055+0000","last_change":"2023-11-26T07:14:55.568585+0000","last_active":"2023-11-26T07:14:56.877055+0000","last_peered":"2023-11-26T07:14:56.877055+0000","last_clean":"2023-11-26T07:14:56.877055+0000","last_became_active":"2023-11-26T07:14:55.568162+0000","last_became_peered":"2023-11-26T07:14:55.568162+0000","last_unstale":"2023-11-26T07:14:56.877055+0000","last_undegraded":"2023-11-26T07:14:56.877055+0000","last_fullsized":"2023-11-26T07:14:56.877055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:34.558173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541424+0000","last_change":"2023-11-26T07:14:55.550793+0000","last_active":"2023-11-26T07:14:56.541424+0000","last_peered":"2023-11-26T07:14:56.541424+0000","last_clean":"2023-11-26T07:14:56.541424+0000","last_became_active":"2023-11-26T07:14:55.550509+0000","last_became_peered":"2023-11-26T07:14:55.550509+0000","last_unstale":"2023-11-26T07:14:56.541424+0000","last_undegraded":"2023-11-26T07:14:56.541424+0000","last_fullsized":"2023-11-26T07:14:56.541424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:44:43.988623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876289+0000","last_change":"2023-11-26T07:14:55.563684+0000","last_active":"2023-11-26T07:14:56.876289+0000","last_peered":"2023-11-26T07:14:56.876289+0000","last_clean":"2023-11-26T07:14:56.876289+0000","last_became_active":"2023-11-26T07:14:55.562596+0000","last_became_peered":"2023-11-26T07:14:55.562596+0000","last_unstale":"2023-11-26T07:14:56.876289+0000","last_undegraded":"2023-11-26T07:14:56.876289+0000","last_fullsized":"2023-11-26T07:14:56.876289+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:00.018524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542146+0000","last_change":"2023-11-26T07:14:55.565993+0000","last_active":"2023-11-26T07:14:56.542146+0000","last_peered":"2023-11-26T07:14:56.542146+0000","last_clean":"2023-11-26T07:14:56.542146+0000","last_became_active":"2023-11-26T07:14:55.565743+0000","last_became_peered":"2023-11-26T07:14:55.565743+0000","last_unstale":"2023-11-26T07:14:56.542146+0000","last_undegraded":"2023-11-26T07:14:56.542146+0000","last_fullsized":"2023-11-26T07:14:56.542146+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:36.265114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541266+0000","last_change":"2023-11-26T07:14:55.570312+0000","last_active":"2023-11-26T07:14:56.541266+0000","last_peered":"2023-11-26T07:14:56.541266+0000","last_clean":"2023-11-26T07:14:56.541266+0000","last_became_active":"2023-11-26T07:14:55.569989+0000","last_became_peered":"2023-11-26T07:14:55.569989+0000","last_unstale":"2023-11-26T07:14:56.541266+0000","last_undegraded":"2023-11-26T07:14:56.541266+0000","last_fullsized":"2023-11-26T07:14:56.541266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:28:00.260761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542478+0000","last_change":"2023-11-26T07:14:55.560177+0000","last_active":"2023-11-26T07:14:56.542478+0000","last_peered":"2023-11-26T07:14:56.542478+0000","last_clean":"2023-11-26T07:14:56.542478+0000","last_became_active":"2023-11-26T07:14:55.559948+0000","last_became_peered":"2023-11-26T07:14:55.559948+0000","last_unstale":"2023-11-26T07:14:56.542478+0000","last_undegraded":"2023-11-26T07:14:56.542478+0000","last_fullsized":"2023-11-26T07:14:56.542478+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:22:36.861785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877238+0000","last_change":"2023-11-26T07:14:55.559369+0000","last_active":"2023-11-26T07:14:56.877238+0000","last_peered":"2023-11-26T07:14:56.877238+0000","last_clean":"2023-11-26T07:14:56.877238+0000","last_became_active":"2023-11-26T07:14:55.559088+0000","last_became_peered":"2023-11-26T07:14:55.559088+0000","last_unstale":"2023-11-26T07:14:56.877238+0000","last_undegraded":"2023-11-26T07:14:56.877238+0000","last_fullsized":"2023-11-26T07:14:56.877238+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:02.536615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877673+0000","last_change":"2023-11-26T07:14:55.569078+0000","last_active":"2023-11-26T07:14:56.877673+0000","last_peered":"2023-11-26T07:14:56.877673+0000","last_clean":"2023-11-26T07:14:56.877673+0000","last_became_active":"2023-11-26T07:14:55.568579+0000","last_became_peered":"2023-11-26T07:14:55.568579+0000","last_unstale":"2023-11-26T07:14:56.877673+0000","last_undegraded":"2023-11-26T07:14:56.877673+0000","last_fullsized":"2023-11-26T07:14:56.877673+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:01.838876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542281+0000","last_change":"2023-11-26T07:14:55.562903+0000","last_active":"2023-11-26T07:14:56.542281+0000","last_peered":"2023-11-26T07:14:56.542281+0000","last_clean":"2023-11-26T07:14:56.542281+0000","last_became_active":"2023-11-26T07:14:55.562637+0000","last_became_peered":"2023-11-26T07:14:55.562637+0000","last_unstale":"2023-11-26T07:14:56.542281+0000","last_undegraded":"2023-11-26T07:14:56.542281+0000","last_fullsized":"2023-11-26T07:14:56.542281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:22.475891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876656+0000","last_change":"2023-11-26T07:14:55.566349+0000","last_active":"2023-11-26T07:14:56.876656+0000","last_peered":"2023-11-26T07:14:56.876656+0000","last_clean":"2023-11-26T07:14:56.876656+0000","last_became_active":"2023-11-26T07:14:55.566043+0000","last_became_peered":"2023-11-26T07:14:55.566043+0000","last_unstale":"2023-11-26T07:14:56.876656+0000","last_undegraded":"2023-11-26T07:14:56.876656+0000","last_fullsized":"2023-11-26T07:14:56.876656+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:07:34.241609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541503+0000","last_change":"2023-11-26T07:14:55.547718+0000","last_active":"2023-11-26T07:14:56.541503+0000","last_peered":"2023-11-26T07:14:56.541503+0000","last_clean":"2023-11-26T07:14:56.541503+0000","last_became_active":"2023-11-26T07:14:55.547432+0000","last_became_peered":"2023-11-26T07:14:55.547432+0000","last_unstale":"2023-11-26T07:14:56.541503+0000","last_undegraded":"2023-11-26T07:14:56.541503+0000","last_fullsized":"2023-11-26T07:14:56.541503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:45:56.436888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.542055+0000","last_change":"2023-11-26T07:14:55.560364+0000","last_active":"2023-11-26T07:14:56.542055+0000","last_peered":"2023-11-26T07:14:56.542055+0000","last_clean":"2023-11-26T07:14:56.542055+0000","last_became_active":"2023-11-26T07:14:55.559929+0000","last_became_peered":"2023-11-26T07:14:55.559929+0000","last_unstale":"2023-11-26T07:14:56.542055+0000","last_undegraded":"2023-11-26T07:14:56.542055+0000","last_fullsized":"2023-11-26T07:14:56.542055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:24:20.997710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876524+0000","last_change":"2023-11-26T07:14:55.563485+0000","last_active":"2023-11-26T07:14:56.876524+0000","last_peered":"2023-11-26T07:14:56.876524+0000","last_clean":"2023-11-26T07:14:56.876524+0000","last_became_active":"2023-11-26T07:14:55.563222+0000","last_became_peered":"2023-11-26T07:14:55.563222+0000","last_unstale":"2023-11-26T07:14:56.876524+0000","last_undegraded":"2023-11-26T07:14:56.876524+0000","last_fullsized":"2023-11-26T07:14:56.876524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:57.935087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876779+0000","last_change":"2023-11-26T07:14:55.569614+0000","last_active":"2023-11-26T07:14:56.876779+0000","last_peered":"2023-11-26T07:14:56.876779+0000","last_clean":"2023-11-26T07:14:56.876779+0000","last_became_active":"2023-11-26T07:14:55.567970+0000","last_became_peered":"2023-11-26T07:14:55.567970+0000","last_unstale":"2023-11-26T07:14:56.876779+0000","last_undegraded":"2023-11-26T07:14:56.876779+0000","last_fullsized":"2023-11-26T07:14:56.876779+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:02.536615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.569319+0000","last_change":"2023-11-26T07:14:54.551197+0000","last_active":"2023-11-26T07:14:55.569319+0000","last_peered":"2023-11-26T07:14:55.569319+0000","last_clean":"2023-11-26T07:14:55.569319+0000","last_became_active":"2023-11-26T07:14:54.550930+0000","last_became_peered":"2023-11-26T07:14:54.550930+0000","last_unstale":"2023-11-26T07:14:55.569319+0000","last_undegraded":"2023-11-26T07:14:55.569319+0000","last_fullsized":"2023-11-26T07:14:55.569319+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:24.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877481+0000","last_change":"2023-11-26T07:14:55.554217+0000","last_active":"2023-11-26T07:14:56.877481+0000","last_peered":"2023-11-26T07:14:56.877481+0000","last_clean":"2023-11-26T07:14:56.877481+0000","last_became_active":"2023-11-26T07:14:55.553893+0000","last_became_peered":"2023-11-26T07:14:55.553893+0000","last_unstale":"2023-11-26T07:14:56.877481+0000","last_undegraded":"2023-11-26T07:14:56.877481+0000","last_fullsized":"2023-11-26T07:14:56.877481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:02:52.062049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.566035+0000","last_change":"2023-11-26T07:14:54.556263+0000","last_active":"2023-11-26T07:14:55.566035+0000","last_peered":"2023-11-26T07:14:55.566035+0000","last_clean":"2023-11-26T07:14:55.566035+0000","last_became_active":"2023-11-26T07:14:54.555815+0000","last_became_peered":"2023-11-26T07:14:54.555815+0000","last_unstale":"2023-11-26T07:14:55.566035+0000","last_undegraded":"2023-11-26T07:14:55.566035+0000","last_fullsized":"2023-11-26T07:14:55.566035+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:13:38.502900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876491+0000","last_change":"2023-11-26T07:14:55.570096+0000","last_active":"2023-11-26T07:14:56.876491+0000","last_peered":"2023-11-26T07:14:56.876491+0000","last_clean":"2023-11-26T07:14:56.876491+0000","last_became_active":"2023-11-26T07:14:55.567546+0000","last_became_peered":"2023-11-26T07:14:55.567546+0000","last_unstale":"2023-11-26T07:14:56.876491+0000","last_undegraded":"2023-11-26T07:14:56.876491+0000","last_fullsized":"2023-11-26T07:14:56.876491+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:38.960124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.568272+0000","last_change":"2023-11-26T07:14:54.553820+0000","last_active":"2023-11-26T07:14:55.568272+0000","last_peered":"2023-11-26T07:14:55.568272+0000","last_clean":"2023-11-26T07:14:55.568272+0000","last_became_active":"2023-11-26T07:14:54.553552+0000","last_became_peered":"2023-11-26T07:14:54.553552+0000","last_unstale":"2023-11-26T07:14:55.568272+0000","last_undegraded":"2023-11-26T07:14:55.568272+0000","last_fullsized":"2023-11-26T07:14:55.568272+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:19.097609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877677+0000","last_change":"2023-11-26T07:14:55.568824+0000","last_active":"2023-11-26T07:14:56.877677+0000","last_peered":"2023-11-26T07:14:56.877677+0000","last_clean":"2023-11-26T07:14:56.877677+0000","last_became_active":"2023-11-26T07:14:55.568239+0000","last_became_peered":"2023-11-26T07:14:55.568239+0000","last_unstale":"2023-11-26T07:14:56.877677+0000","last_undegraded":"2023-11-26T07:14:56.877677+0000","last_fullsized":"2023-11-26T07:14:56.877677+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:48:48.988066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.561647+0000","last_change":"2023-11-26T07:14:54.555832+0000","last_active":"2023-11-26T07:14:55.561647+0000","last_peered":"2023-11-26T07:14:55.561647+0000","last_clean":"2023-11-26T07:14:55.561647+0000","last_became_active":"2023-11-26T07:14:54.555514+0000","last_became_peered":"2023-11-26T07:14:54.555514+0000","last_unstale":"2023-11-26T07:14:55.561647+0000","last_undegraded":"2023-11-26T07:14:55.561647+0000","last_fullsized":"2023-11-26T07:14:55.561647+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:24.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876443+0000","last_change":"2023-11-26T07:14:55.562781+0000","last_active":"2023-11-26T07:14:56.876443+0000","last_peered":"2023-11-26T07:14:56.876443+0000","last_clean":"2023-11-26T07:14:56.876443+0000","last_became_active":"2023-11-26T07:14:55.562323+0000","last_became_peered":"2023-11-26T07:14:55.562323+0000","last_unstale":"2023-11-26T07:14:56.876443+0000","last_undegraded":"2023-11-26T07:14:56.876443+0000","last_fullsized":"2023-11-26T07:14:56.876443+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:13.425773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.566845+0000","last_change":"2023-11-26T07:14:54.554968+0000","last_active":"2023-11-26T07:14:55.566845+0000","last_peered":"2023-11-26T07:14:55.566845+0000","last_clean":"2023-11-26T07:14:55.566845+0000","last_became_active":"2023-11-26T07:14:54.554664+0000","last_became_peered":"2023-11-26T07:14:54.554664+0000","last_unstale":"2023-11-26T07:14:55.566845+0000","last_undegraded":"2023-11-26T07:14:55.566845+0000","last_fullsized":"2023-11-26T07:14:55.566845+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:28.707932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876789+0000","last_change":"2023-11-26T07:14:55.568976+0000","last_active":"2023-11-26T07:14:56.876789+0000","last_peered":"2023-11-26T07:14:56.876789+0000","last_clean":"2023-11-26T07:14:56.876789+0000","last_became_active":"2023-11-26T07:14:55.568601+0000","last_became_peered":"2023-11-26T07:14:55.568601+0000","last_unstale":"2023-11-26T07:14:56.876789+0000","last_undegraded":"2023-11-26T07:14:56.876789+0000","last_fullsized":"2023-11-26T07:14:56.876789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:09.743461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.567289+0000","last_change":"2023-11-26T07:14:54.545826+0000","last_active":"2023-11-26T07:14:55.567289+0000","last_peered":"2023-11-26T07:14:55.567289+0000","last_clean":"2023-11-26T07:14:55.567289+0000","last_became_active":"2023-11-26T07:14:54.545558+0000","last_became_peered":"2023-11-26T07:14:54.545558+0000","last_unstale":"2023-11-26T07:14:55.567289+0000","last_undegraded":"2023-11-26T07:14:55.567289+0000","last_fullsized":"2023-11-26T07:14:55.567289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:56.667181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541442+0000","last_change":"2023-11-26T07:14:55.558261+0000","last_active":"2023-11-26T07:14:56.541442+0000","last_peered":"2023-11-26T07:14:56.541442+0000","last_clean":"2023-11-26T07:14:56.541442+0000","last_became_active":"2023-11-26T07:14:55.557967+0000","last_became_peered":"2023-11-26T07:14:55.557967+0000","last_unstale":"2023-11-26T07:14:56.541442+0000","last_undegraded":"2023-11-26T07:14:56.541442+0000","last_fullsized":"2023-11-26T07:14:56.541442+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:53.832663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.548344+0000","last_change":"2023-11-26T07:14:54.555979+0000","last_active":"2023-11-26T07:14:55.548344+0000","last_peered":"2023-11-26T07:14:55.548344+0000","last_clean":"2023-11-26T07:14:55.548344+0000","last_became_active":"2023-11-26T07:14:54.555658+0000","last_became_peered":"2023-11-26T07:14:54.555658+0000","last_unstale":"2023-11-26T07:14:55.548344+0000","last_undegraded":"2023-11-26T07:14:55.548344+0000","last_fullsized":"2023-11-26T07:14:55.548344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:12.852910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876522+0000","last_change":"2023-11-26T07:14:55.564714+0000","last_active":"2023-11-26T07:14:56.876522+0000","last_peered":"2023-11-26T07:14:56.876522+0000","last_clean":"2023-11-26T07:14:56.876522+0000","last_became_active":"2023-11-26T07:14:55.564443+0000","last_became_peered":"2023-11-26T07:14:55.564443+0000","last_unstale":"2023-11-26T07:14:56.876522+0000","last_undegraded":"2023-11-26T07:14:56.876522+0000","last_fullsized":"2023-11-26T07:14:56.876522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:07:30.647501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.541291+0000","last_change":"2023-11-26T07:14:54.556906+0000","last_active":"2023-11-26T07:14:55.541291+0000","last_peered":"2023-11-26T07:14:55.541291+0000","last_clean":"2023-11-26T07:14:55.541291+0000","last_became_active":"2023-11-26T07:14:54.556633+0000","last_became_peered":"2023-11-26T07:14:54.556633+0000","last_unstale":"2023-11-26T07:14:55.541291+0000","last_undegraded":"2023-11-26T07:14:55.541291+0000","last_fullsized":"2023-11-26T07:14:55.541291+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:10:48.528921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877149+0000","last_change":"2023-11-26T07:14:55.561086+0000","last_active":"2023-11-26T07:14:56.877149+0000","last_peered":"2023-11-26T07:14:56.877149+0000","last_clean":"2023-11-26T07:14:56.877149+0000","last_became_active":"2023-11-26T07:14:55.560811+0000","last_became_peered":"2023-11-26T07:14:55.560811+0000","last_unstale":"2023-11-26T07:14:56.877149+0000","last_undegraded":"2023-11-26T07:14:56.877149+0000","last_fullsized":"2023-11-26T07:14:56.877149+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:57.669199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.541992+0000","last_change":"2023-11-26T07:14:54.551547+0000","last_active":"2023-11-26T07:14:55.541992+0000","last_peered":"2023-11-26T07:14:55.541992+0000","last_clean":"2023-11-26T07:14:55.541992+0000","last_became_active":"2023-11-26T07:14:54.551059+0000","last_became_peered":"2023-11-26T07:14:54.551059+0000","last_unstale":"2023-11-26T07:14:55.541992+0000","last_undegraded":"2023-11-26T07:14:55.541992+0000","last_fullsized":"2023-11-26T07:14:55.541992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:56.933069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541907+0000","last_change":"2023-11-26T07:14:55.554669+0000","last_active":"2023-11-26T07:14:56.541907+0000","last_peered":"2023-11-26T07:14:56.541907+0000","last_clean":"2023-11-26T07:14:56.541907+0000","last_became_active":"2023-11-26T07:14:55.554402+0000","last_became_peered":"2023-11-26T07:14:55.554402+0000","last_unstale":"2023-11-26T07:14:56.541907+0000","last_undegraded":"2023-11-26T07:14:56.541907+0000","last_fullsized":"2023-11-26T07:14:56.541907+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:01:27.228579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.548485+0000","last_change":"2023-11-26T07:14:54.556255+0000","last_active":"2023-11-26T07:14:55.548485+0000","last_peered":"2023-11-26T07:14:55.548485+0000","last_clean":"2023-11-26T07:14:55.548485+0000","last_became_active":"2023-11-26T07:14:54.555863+0000","last_became_peered":"2023-11-26T07:14:54.555863+0000","last_unstale":"2023-11-26T07:14:55.548485+0000","last_undegraded":"2023-11-26T07:14:55.548485+0000","last_fullsized":"2023-11-26T07:14:55.548485+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:49.214175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876808+0000","last_change":"2023-11-26T07:14:55.555431+0000","last_active":"2023-11-26T07:14:56.876808+0000","last_peered":"2023-11-26T07:14:56.876808+0000","last_clean":"2023-11-26T07:14:56.876808+0000","last_became_active":"2023-11-26T07:14:55.555090+0000","last_became_peered":"2023-11-26T07:14:55.555090+0000","last_unstale":"2023-11-26T07:14:56.876808+0000","last_undegraded":"2023-11-26T07:14:56.876808+0000","last_fullsized":"2023-11-26T07:14:56.876808+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:28:00.260761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.571669+0000","last_change":"2023-11-26T07:14:54.553992+0000","last_active":"2023-11-26T07:14:55.571669+0000","last_peered":"2023-11-26T07:14:55.571669+0000","last_clean":"2023-11-26T07:14:55.571669+0000","last_became_active":"2023-11-26T07:14:54.553718+0000","last_became_peered":"2023-11-26T07:14:54.553718+0000","last_unstale":"2023-11-26T07:14:55.571669+0000","last_undegraded":"2023-11-26T07:14:55.571669+0000","last_fullsized":"2023-11-26T07:14:55.571669+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:02.663448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541322+0000","last_change":"2023-11-26T07:14:55.556639+0000","last_active":"2023-11-26T07:14:56.541322+0000","last_peered":"2023-11-26T07:14:56.541322+0000","last_clean":"2023-11-26T07:14:56.541322+0000","last_became_active":"2023-11-26T07:14:55.556441+0000","last_became_peered":"2023-11-26T07:14:55.556441+0000","last_unstale":"2023-11-26T07:14:56.541322+0000","last_undegraded":"2023-11-26T07:14:56.541322+0000","last_fullsized":"2023-11-26T07:14:56.541322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:59.370564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.574692+0000","last_change":"2023-11-26T07:14:54.557028+0000","last_active":"2023-11-26T07:14:55.574692+0000","last_peered":"2023-11-26T07:14:55.574692+0000","last_clean":"2023-11-26T07:14:55.574692+0000","last_became_active":"2023-11-26T07:14:54.556758+0000","last_became_peered":"2023-11-26T07:14:54.556758+0000","last_unstale":"2023-11-26T07:14:55.574692+0000","last_undegraded":"2023-11-26T07:14:55.574692+0000","last_fullsized":"2023-11-26T07:14:55.574692+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:58:50.594707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876740+0000","last_change":"2023-11-26T07:14:55.562711+0000","last_active":"2023-11-26T07:14:56.876740+0000","last_peered":"2023-11-26T07:14:56.876740+0000","last_clean":"2023-11-26T07:14:56.876740+0000","last_became_active":"2023-11-26T07:14:55.562505+0000","last_became_peered":"2023-11-26T07:14:55.562505+0000","last_unstale":"2023-11-26T07:14:56.876740+0000","last_undegraded":"2023-11-26T07:14:56.876740+0000","last_fullsized":"2023-11-26T07:14:56.876740+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:57.951234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.573668+0000","last_change":"2023-11-26T07:14:54.554993+0000","last_active":"2023-11-26T07:14:55.573668+0000","last_peered":"2023-11-26T07:14:55.573668+0000","last_clean":"2023-11-26T07:14:55.573668+0000","last_became_active":"2023-11-26T07:14:54.554750+0000","last_became_peered":"2023-11-26T07:14:54.554750+0000","last_unstale":"2023-11-26T07:14:55.573668+0000","last_undegraded":"2023-11-26T07:14:55.573668+0000","last_fullsized":"2023-11-26T07:14:55.573668+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:15:45.687867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541558+0000","last_change":"2023-11-26T07:14:55.552259+0000","last_active":"2023-11-26T07:14:56.541558+0000","last_peered":"2023-11-26T07:14:56.541558+0000","last_clean":"2023-11-26T07:14:56.541558+0000","last_became_active":"2023-11-26T07:14:55.551888+0000","last_became_peered":"2023-11-26T07:14:55.551888+0000","last_unstale":"2023-11-26T07:14:56.541558+0000","last_undegraded":"2023-11-26T07:14:56.541558+0000","last_fullsized":"2023-11-26T07:14:56.541558+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:13:39.504918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.547350+0000","last_change":"2023-11-26T07:14:54.552221+0000","last_active":"2023-11-26T07:14:55.547350+0000","last_peered":"2023-11-26T07:14:55.547350+0000","last_clean":"2023-11-26T07:14:55.547350+0000","last_became_active":"2023-11-26T07:14:54.551932+0000","last_became_peered":"2023-11-26T07:14:54.551932+0000","last_unstale":"2023-11-26T07:14:55.547350+0000","last_undegraded":"2023-11-26T07:14:55.547350+0000","last_fullsized":"2023-11-26T07:14:55.547350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:05.353039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876587+0000","last_change":"2023-11-26T07:14:55.553875+0000","last_active":"2023-11-26T07:14:56.876587+0000","last_peered":"2023-11-26T07:14:56.876587+0000","last_clean":"2023-11-26T07:14:56.876587+0000","last_became_active":"2023-11-26T07:14:55.553581+0000","last_became_peered":"2023-11-26T07:14:55.553581+0000","last_unstale":"2023-11-26T07:14:56.876587+0000","last_undegraded":"2023-11-26T07:14:56.876587+0000","last_fullsized":"2023-11-26T07:14:56.876587+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:42.254062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.548315+0000","last_change":"2023-11-26T07:14:54.553341+0000","last_active":"2023-11-26T07:14:55.548315+0000","last_peered":"2023-11-26T07:14:55.548315+0000","last_clean":"2023-11-26T07:14:55.548315+0000","last_became_active":"2023-11-26T07:14:54.551872+0000","last_became_peered":"2023-11-26T07:14:54.551872+0000","last_unstale":"2023-11-26T07:14:55.548315+0000","last_undegraded":"2023-11-26T07:14:55.548315+0000","last_fullsized":"2023-11-26T07:14:55.548315+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:42.288092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541270+0000","last_change":"2023-11-26T07:14:55.566027+0000","last_active":"2023-11-26T07:14:56.541270+0000","last_peered":"2023-11-26T07:14:56.541270+0000","last_clean":"2023-11-26T07:14:56.541270+0000","last_became_active":"2023-11-26T07:14:55.565770+0000","last_became_peered":"2023-11-26T07:14:55.565770+0000","last_unstale":"2023-11-26T07:14:56.541270+0000","last_undegraded":"2023-11-26T07:14:56.541270+0000","last_fullsized":"2023-11-26T07:14:56.541270+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:37:15.036624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.566051+0000","last_change":"2023-11-26T07:14:54.551340+0000","last_active":"2023-11-26T07:14:55.566051+0000","last_peered":"2023-11-26T07:14:55.566051+0000","last_clean":"2023-11-26T07:14:55.566051+0000","last_became_active":"2023-11-26T07:14:54.551030+0000","last_became_peered":"2023-11-26T07:14:54.551030+0000","last_unstale":"2023-11-26T07:14:55.566051+0000","last_undegraded":"2023-11-26T07:14:55.566051+0000","last_fullsized":"2023-11-26T07:14:55.566051+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:24.612671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541944+0000","last_change":"2023-11-26T07:14:55.547115+0000","last_active":"2023-11-26T07:14:56.541944+0000","last_peered":"2023-11-26T07:14:56.541944+0000","last_clean":"2023-11-26T07:14:56.541944+0000","last_became_active":"2023-11-26T07:14:55.546831+0000","last_became_peered":"2023-11-26T07:14:55.546831+0000","last_unstale":"2023-11-26T07:14:56.541944+0000","last_undegraded":"2023-11-26T07:14:56.541944+0000","last_fullsized":"2023-11-26T07:14:56.541944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:00.018524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.569880+0000","last_change":"2023-11-26T07:14:54.558605+0000","last_active":"2023-11-26T07:14:55.569880+0000","last_peered":"2023-11-26T07:14:55.569880+0000","last_clean":"2023-11-26T07:14:55.569880+0000","last_became_active":"2023-11-26T07:14:54.558324+0000","last_became_peered":"2023-11-26T07:14:54.558324+0000","last_unstale":"2023-11-26T07:14:55.569880+0000","last_undegraded":"2023-11-26T07:14:55.569880+0000","last_fullsized":"2023-11-26T07:14:55.569880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:45:55.434870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877336+0000","last_change":"2023-11-26T07:14:55.562179+0000","last_active":"2023-11-26T07:14:56.877336+0000","last_peered":"2023-11-26T07:14:56.877336+0000","last_clean":"2023-11-26T07:14:56.877336+0000","last_became_active":"2023-11-26T07:14:55.561908+0000","last_became_peered":"2023-11-26T07:14:55.561908+0000","last_unstale":"2023-11-26T07:14:56.877336+0000","last_undegraded":"2023-11-26T07:14:56.877336+0000","last_fullsized":"2023-11-26T07:14:56.877336+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:01:50.191654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.567782+0000","last_change":"2023-11-26T07:14:54.556982+0000","last_active":"2023-11-26T07:14:55.567782+0000","last_peered":"2023-11-26T07:14:55.567782+0000","last_clean":"2023-11-26T07:14:55.567782+0000","last_became_active":"2023-11-26T07:14:54.556728+0000","last_became_peered":"2023-11-26T07:14:54.556728+0000","last_unstale":"2023-11-26T07:14:55.567782+0000","last_undegraded":"2023-11-26T07:14:55.567782+0000","last_fullsized":"2023-11-26T07:14:55.567782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:58:42.288092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877055+0000","last_change":"2023-11-26T07:14:55.564847+0000","last_active":"2023-11-26T07:14:56.877055+0000","last_peered":"2023-11-26T07:14:56.877055+0000","last_clean":"2023-11-26T07:14:56.877055+0000","last_became_active":"2023-11-26T07:14:55.564561+0000","last_became_peered":"2023-11-26T07:14:55.564561+0000","last_unstale":"2023-11-26T07:14:56.877055+0000","last_undegraded":"2023-11-26T07:14:56.877055+0000","last_fullsized":"2023-11-26T07:14:56.877055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:01:52.510162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.561778+0000","last_change":"2023-11-26T07:14:54.553413+0000","last_active":"2023-11-26T07:14:55.561778+0000","last_peered":"2023-11-26T07:14:55.561778+0000","last_clean":"2023-11-26T07:14:55.561778+0000","last_became_active":"2023-11-26T07:14:54.552941+0000","last_became_peered":"2023-11-26T07:14:54.552941+0000","last_unstale":"2023-11-26T07:14:55.561778+0000","last_undegraded":"2023-11-26T07:14:55.561778+0000","last_fullsized":"2023-11-26T07:14:55.561778+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:15:18.343075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541477+0000","last_change":"2023-11-26T07:14:55.559136+0000","last_active":"2023-11-26T07:14:56.541477+0000","last_peered":"2023-11-26T07:14:56.541477+0000","last_clean":"2023-11-26T07:14:56.541477+0000","last_became_active":"2023-11-26T07:14:55.558850+0000","last_became_peered":"2023-11-26T07:14:55.558850+0000","last_unstale":"2023-11-26T07:14:56.541477+0000","last_undegraded":"2023-11-26T07:14:56.541477+0000","last_fullsized":"2023-11-26T07:14:56.541477+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:36:50.216193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.554475+0000","last_change":"2023-11-26T07:14:54.546741+0000","last_active":"2023-11-26T07:14:55.554475+0000","last_peered":"2023-11-26T07:14:55.554475+0000","last_clean":"2023-11-26T07:14:55.554475+0000","last_became_active":"2023-11-26T07:14:54.546479+0000","last_became_peered":"2023-11-26T07:14:54.546479+0000","last_unstale":"2023-11-26T07:14:55.554475+0000","last_undegraded":"2023-11-26T07:14:55.554475+0000","last_fullsized":"2023-11-26T07:14:55.554475+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:08.574978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876882+0000","last_change":"2023-11-26T07:14:55.563245+0000","last_active":"2023-11-26T07:14:56.876882+0000","last_peered":"2023-11-26T07:14:56.876882+0000","last_clean":"2023-11-26T07:14:56.876882+0000","last_became_active":"2023-11-26T07:14:55.562937+0000","last_became_peered":"2023-11-26T07:14:55.562937+0000","last_unstale":"2023-11-26T07:14:56.876882+0000","last_undegraded":"2023-11-26T07:14:56.876882+0000","last_fullsized":"2023-11-26T07:14:56.876882+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:38.960124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.570157+0000","last_change":"2023-11-26T07:14:54.555163+0000","last_active":"2023-11-26T07:14:55.570157+0000","last_peered":"2023-11-26T07:14:55.570157+0000","last_clean":"2023-11-26T07:14:55.570157+0000","last_became_active":"2023-11-26T07:14:54.554331+0000","last_became_peered":"2023-11-26T07:14:54.554331+0000","last_unstale":"2023-11-26T07:14:55.570157+0000","last_undegraded":"2023-11-26T07:14:55.570157+0000","last_fullsized":"2023-11-26T07:14:55.570157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:32:22.345038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876915+0000","last_change":"2023-11-26T07:14:55.567533+0000","last_active":"2023-11-26T07:14:56.876915+0000","last_peered":"2023-11-26T07:14:56.876915+0000","last_clean":"2023-11-26T07:14:56.876915+0000","last_became_active":"2023-11-26T07:14:55.567256+0000","last_became_peered":"2023-11-26T07:14:55.567256+0000","last_unstale":"2023-11-26T07:14:56.876915+0000","last_undegraded":"2023-11-26T07:14:56.876915+0000","last_fullsized":"2023-11-26T07:14:56.876915+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:01:50.191654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.542772+0000","last_change":"2023-11-26T07:14:54.554798+0000","last_active":"2023-11-26T07:14:55.542772+0000","last_peered":"2023-11-26T07:14:55.542772+0000","last_clean":"2023-11-26T07:14:55.542772+0000","last_became_active":"2023-11-26T07:14:54.554545+0000","last_became_peered":"2023-11-26T07:14:54.554545+0000","last_unstale":"2023-11-26T07:14:55.542772+0000","last_undegraded":"2023-11-26T07:14:55.542772+0000","last_fullsized":"2023-11-26T07:14:55.542772+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:22:35.859767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876276+0000","last_change":"2023-11-26T07:14:55.553523+0000","last_active":"2023-11-26T07:14:56.876276+0000","last_peered":"2023-11-26T07:14:56.876276+0000","last_clean":"2023-11-26T07:14:56.876276+0000","last_became_active":"2023-11-26T07:14:55.553227+0000","last_became_peered":"2023-11-26T07:14:55.553227+0000","last_unstale":"2023-11-26T07:14:56.876276+0000","last_undegraded":"2023-11-26T07:14:56.876276+0000","last_fullsized":"2023-11-26T07:14:56.876276+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:39:06.952707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.547474+0000","last_change":"2023-11-26T07:14:54.556775+0000","last_active":"2023-11-26T07:14:55.547474+0000","last_peered":"2023-11-26T07:14:55.547474+0000","last_clean":"2023-11-26T07:14:55.547474+0000","last_became_active":"2023-11-26T07:14:54.556477+0000","last_became_peered":"2023-11-26T07:14:54.556477+0000","last_unstale":"2023-11-26T07:14:55.547474+0000","last_undegraded":"2023-11-26T07:14:55.547474+0000","last_fullsized":"2023-11-26T07:14:55.547474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:41:47.910749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877407+0000","last_change":"2023-11-26T07:14:55.556529+0000","last_active":"2023-11-26T07:14:56.877407+0000","last_peered":"2023-11-26T07:14:56.877407+0000","last_clean":"2023-11-26T07:14:56.877407+0000","last_became_active":"2023-11-26T07:14:55.556258+0000","last_became_peered":"2023-11-26T07:14:55.556258+0000","last_unstale":"2023-11-26T07:14:56.877407+0000","last_undegraded":"2023-11-26T07:14:56.877407+0000","last_fullsized":"2023-11-26T07:14:56.877407+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:48:48.988066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.567977+0000","last_change":"2023-11-26T07:14:54.559238+0000","last_active":"2023-11-26T07:14:55.567977+0000","last_peered":"2023-11-26T07:14:55.567977+0000","last_clean":"2023-11-26T07:14:55.567977+0000","last_became_active":"2023-11-26T07:14:54.558934+0000","last_became_peered":"2023-11-26T07:14:54.558934+0000","last_unstale":"2023-11-26T07:14:55.567977+0000","last_undegraded":"2023-11-26T07:14:55.567977+0000","last_fullsized":"2023-11-26T07:14:55.567977+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:44.510723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877355+0000","last_change":"2023-11-26T07:14:55.552256+0000","last_active":"2023-11-26T07:14:56.877355+0000","last_peered":"2023-11-26T07:14:56.877355+0000","last_clean":"2023-11-26T07:14:56.877355+0000","last_became_active":"2023-11-26T07:14:55.551982+0000","last_became_peered":"2023-11-26T07:14:55.551982+0000","last_unstale":"2023-11-26T07:14:56.877355+0000","last_undegraded":"2023-11-26T07:14:56.877355+0000","last_fullsized":"2023-11-26T07:14:56.877355+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:03.665466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876918+0000","last_change":"2023-11-26T07:14:54.550263+0000","last_active":"2023-11-26T07:14:56.876918+0000","last_peered":"2023-11-26T07:14:56.876918+0000","last_clean":"2023-11-26T07:14:56.876918+0000","last_became_active":"2023-11-26T07:14:54.549971+0000","last_became_peered":"2023-11-26T07:14:54.549971+0000","last_unstale":"2023-11-26T07:14:56.876918+0000","last_undegraded":"2023-11-26T07:14:56.876918+0000","last_fullsized":"2023-11-26T07:14:56.876918+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:42:12.852910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876936+0000","last_change":"2023-11-26T07:14:55.556173+0000","last_active":"2023-11-26T07:14:56.876936+0000","last_peered":"2023-11-26T07:14:56.876936+0000","last_clean":"2023-11-26T07:14:56.876936+0000","last_became_active":"2023-11-26T07:14:55.555920+0000","last_became_peered":"2023-11-26T07:14:55.555920+0000","last_unstale":"2023-11-26T07:14:56.876936+0000","last_undegraded":"2023-11-26T07:14:56.876936+0000","last_fullsized":"2023-11-26T07:14:56.876936+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:50:53.832663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.547480+0000","last_change":"2023-11-26T07:14:54.552589+0000","last_active":"2023-11-26T07:14:55.547480+0000","last_peered":"2023-11-26T07:14:55.547480+0000","last_clean":"2023-11-26T07:14:55.547480+0000","last_became_active":"2023-11-26T07:14:54.552302+0000","last_became_peered":"2023-11-26T07:14:54.552302+0000","last_unstale":"2023-11-26T07:14:55.547480+0000","last_undegraded":"2023-11-26T07:14:55.547480+0000","last_fullsized":"2023-11-26T07:14:55.547480+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:25:40.381443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.877765+0000","last_change":"2023-11-26T07:14:55.567850+0000","last_active":"2023-11-26T07:14:56.877765+0000","last_peered":"2023-11-26T07:14:56.877765+0000","last_clean":"2023-11-26T07:14:56.877765+0000","last_became_active":"2023-11-26T07:14:55.567191+0000","last_became_peered":"2023-11-26T07:14:55.567191+0000","last_unstale":"2023-11-26T07:14:56.877765+0000","last_undegraded":"2023-11-26T07:14:56.877765+0000","last_fullsized":"2023-11-26T07:14:56.877765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:37:15.036624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.551783+0000","last_change":"2023-11-26T07:14:54.555982+0000","last_active":"2023-11-26T07:14:55.551783+0000","last_peered":"2023-11-26T07:14:55.551783+0000","last_clean":"2023-11-26T07:14:55.551783+0000","last_became_active":"2023-11-26T07:14:54.555678+0000","last_became_peered":"2023-11-26T07:14:54.555678+0000","last_unstale":"2023-11-26T07:14:55.551783+0000","last_undegraded":"2023-11-26T07:14:55.551783+0000","last_fullsized":"2023-11-26T07:14:55.551783+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:08.574978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876572+0000","last_change":"2023-11-26T07:14:55.562592+0000","last_active":"2023-11-26T07:14:56.876572+0000","last_peered":"2023-11-26T07:14:56.876572+0000","last_clean":"2023-11-26T07:14:56.876572+0000","last_became_active":"2023-11-26T07:14:55.562283+0000","last_became_peered":"2023-11-26T07:14:55.562283+0000","last_unstale":"2023-11-26T07:14:56.876572+0000","last_undegraded":"2023-11-26T07:14:56.876572+0000","last_fullsized":"2023-11-26T07:14:56.876572+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:40:50.418915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.542809+0000","last_change":"2023-11-26T07:14:54.556883+0000","last_active":"2023-11-26T07:14:55.542809+0000","last_peered":"2023-11-26T07:14:55.542809+0000","last_clean":"2023-11-26T07:14:55.542809+0000","last_became_active":"2023-11-26T07:14:54.556654+0000","last_became_peered":"2023-11-26T07:14:54.556654+0000","last_unstale":"2023-11-26T07:14:55.542809+0000","last_undegraded":"2023-11-26T07:14:55.542809+0000","last_fullsized":"2023-11-26T07:14:55.542809+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:00:25.665994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.541680+0000","last_change":"2023-11-26T07:14:55.567327+0000","last_active":"2023-11-26T07:14:56.541680+0000","last_peered":"2023-11-26T07:14:56.541680+0000","last_clean":"2023-11-26T07:14:56.541680+0000","last_became_active":"2023-11-26T07:14:55.567068+0000","last_became_peered":"2023-11-26T07:14:55.567068+0000","last_unstale":"2023-11-26T07:14:56.541680+0000","last_undegraded":"2023-11-26T07:14:56.541680+0000","last_fullsized":"2023-11-26T07:14:56.541680+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:32:23.347056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.542458+0000","last_change":"2023-11-26T07:14:54.555216+0000","last_active":"2023-11-26T07:14:55.542458+0000","last_peered":"2023-11-26T07:14:55.542458+0000","last_clean":"2023-11-26T07:14:55.542458+0000","last_became_active":"2023-11-26T07:14:54.554577+0000","last_became_peered":"2023-11-26T07:14:54.554577+0000","last_unstale":"2023-11-26T07:14:55.542458+0000","last_undegraded":"2023-11-26T07:14:55.542458+0000","last_fullsized":"2023-11-26T07:14:55.542458+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:44:43.781381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876106+0000","last_change":"2023-11-26T07:14:55.558942+0000","last_active":"2023-11-26T07:14:56.876106+0000","last_peered":"2023-11-26T07:14:56.876106+0000","last_clean":"2023-11-26T07:14:56.876106+0000","last_became_active":"2023-11-26T07:14:55.558687+0000","last_became_peered":"2023-11-26T07:14:55.558687+0000","last_unstale":"2023-11-26T07:14:56.876106+0000","last_undegraded":"2023-11-26T07:14:56.876106+0000","last_fullsized":"2023-11-26T07:14:56.876106+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:16:05.761932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.543069+0000","last_change":"2023-11-26T07:14:54.556996+0000","last_active":"2023-11-26T07:14:55.543069+0000","last_peered":"2023-11-26T07:14:55.543069+0000","last_clean":"2023-11-26T07:14:55.543069+0000","last_became_active":"2023-11-26T07:14:54.556722+0000","last_became_peered":"2023-11-26T07:14:54.556722+0000","last_unstale":"2023-11-26T07:14:55.543069+0000","last_undegraded":"2023-11-26T07:14:55.543069+0000","last_fullsized":"2023-11-26T07:14:55.543069+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:49:05.353039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876662+0000","last_change":"2023-11-26T07:14:55.565201+0000","last_active":"2023-11-26T07:14:56.876662+0000","last_peered":"2023-11-26T07:14:56.876662+0000","last_clean":"2023-11-26T07:14:56.876662+0000","last_became_active":"2023-11-26T07:14:55.564935+0000","last_became_peered":"2023-11-26T07:14:55.564935+0000","last_unstale":"2023-11-26T07:14:56.876662+0000","last_undegraded":"2023-11-26T07:14:56.876662+0000","last_fullsized":"2023-11-26T07:14:56.876662+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:51:19.407229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.549059+0000","last_change":"2023-11-26T07:14:54.550363+0000","last_active":"2023-11-26T07:14:55.549059+0000","last_peered":"2023-11-26T07:14:55.549059+0000","last_clean":"2023-11-26T07:14:55.549059+0000","last_became_active":"2023-11-26T07:14:54.550093+0000","last_became_peered":"2023-11-26T07:14:54.550093+0000","last_unstale":"2023-11-26T07:14:55.549059+0000","last_undegraded":"2023-11-26T07:14:55.549059+0000","last_fullsized":"2023-11-26T07:14:55.549059+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:01:26.226561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-26T07:14:56.876495+0000","last_change":"2023-11-26T07:14:55.561975+0000","last_active":"2023-11-26T07:14:56.876495+0000","last_peered":"2023-11-26T07:14:56.876495+0000","last_clean":"2023-11-26T07:14:56.876495+0000","last_became_active":"2023-11-26T07:14:55.561323+0000","last_became_peered":"2023-11-26T07:14:55.561323+0000","last_unstale":"2023-11-26T07:14:56.876495+0000","last_undegraded":"2023-11-26T07:14:56.876495+0000","last_fullsized":"2023-11-26T07:14:56.876495+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:54.530381+0000","last_clean_scrub_stamp":"2023-11-26T07:14:54.530381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:13:03.157321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.547508+0000","last_change":"2023-11-26T07:14:54.552852+0000","last_active":"2023-11-26T07:14:55.547508+0000","last_peered":"2023-11-26T07:14:55.547508+0000","last_clean":"2023-11-26T07:14:55.547508+0000","last_became_active":"2023-11-26T07:14:54.552543+0000","last_became_peered":"2023-11-26T07:14:54.552543+0000","last_unstale":"2023-11-26T07:14:55.547508+0000","last_undegraded":"2023-11-26T07:14:55.547508+0000","last_fullsized":"2023-11-26T07:14:55.547508+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:14:53.528363+0000","last_clean_scrub_stamp":"2023-11-26T07:14:53.528363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:32.134484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:14:55.540389+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:14:55.540389+0000","last_peered":"2023-11-26T07:14:55.540389+0000","last_clean":"2023-11-26T07:14:55.540389+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:14:55.540389+0000","last_undegraded":"2023-11-26T07:14:55.540389+0000","last_fullsized":"2023-11-26T07:14:55.540389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1298432,"data_stored":880333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56799999999999995,"15min":0.55900000000000005},"min":{"1min":0.44,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.52200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68899999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.629}]},{"osd":1,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.47299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73799999999999999,"5min":1.048,"15min":1.048},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.497,"5min":0.315,"15min":0.315},"max":{"1min":0.76100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.752}]},{"osd":2,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.495,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.621,"15min":0.622},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75600000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70499999999999996,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63500000000000001,"15min":0.63800000000000001},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77800000000000002,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.77600000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76700000000000002,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.754,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.45700000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.37,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79500000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79000000000000004,"5min":1.171,"15min":1.171},"last":0.51100000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.74199999999999999,"5min":1.361,"15min":1.361},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78000000000000003,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.60599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.39200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.52900000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.79900000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.56999999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76600000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.49099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.69799999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61599999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.45800000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72299999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82599999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.55600000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.753,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.374},{"interface":"front","average":{"1min":0.505,"5min":0.55300000000000005,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70399999999999996,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42799999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65900000000000003,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56799999999999995,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67900000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46800000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.747,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61399999999999999,"15min":0.624},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73399999999999999,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.495}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":700416,"data_stored":289895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33700000000000002,"15min":0.33100000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.502,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.34899999999999998,"15min":0.34599999999999997},"min":{"1min":0.20000000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.5,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.31}]},{"osd":2,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.34000000000000002,"15min":0.33400000000000002},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.27200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53000000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.34699999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.39300000000000002,"15min":0.38700000000000001},"min":{"1min":0.252,"5min":0.246,"15min":0.246},"max":{"1min":0.57499999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.40999999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79100000000000004,"5min":0.877,"15min":0.877},"last":0.629},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.44500000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.49299999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.623,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78700000000000003,"5min":0.995,"15min":0.995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.46800000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.57699999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.621,"15min":0.621},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.88800000000000001,"5min":1.129,"15min":1.129},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.627,"15min":0.625},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52100000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.52400000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.64600000000000002},"min":{"1min":0.47399999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83599999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343544,"statfs":{"total":96636764160,"available":96607789056,"internally_reserved":0,"allocated":1306624,"data_stored":882367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.221,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54200000000000004,"5min":0.623,"15min":0.623},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.376,"15min":0.36899999999999999},"min":{"1min":0.26400000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.40200000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.629,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.443},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.28599999999999998,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.32600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.40200000000000002,"15min":0.40400000000000003},"min":{"1min":0.29799999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61699999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.437,"15min":0.439},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67000000000000004,"15min":0.67300000000000004},"min":{"1min":0.499,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96599999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.48199999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89700000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.496}]},{"osd":6,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.625},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83799999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59799999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45100000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.437,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.61499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.36099999999999999,"15min":0.36899999999999999},"min":{"1min":0.23699999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.497,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47499999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.38200000000000001,"15min":0.39000000000000001},"min":{"1min":0.27900000000000003,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58799999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.496},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36399999999999999},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.443}]},{"osd":3,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.27400000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.56499999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.41799999999999998,"15min":0.42599999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54000000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.55700000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64800000000000002,"15min":0.67600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85799999999999998,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79300000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.76900000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.621,"15min":0.624},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81399999999999995,"5min":1.075,"15min":1.075},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82399999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.78400000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80100000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.47099999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.77900000000000003,"5min":1.03,"15min":1.03},"last":0.65200000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.64500000000000002,"15min":0.65900000000000003},"min":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.622,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82799999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":289861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.218,"5min":0.188,"15min":0.188},"max":{"1min":0.42199999999999999,"5min":1.284,"15min":1.284},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.44900000000000001,"15min":0.47999999999999998},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.55500000000000005,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.27800000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.47499999999999998,"15min":0.52700000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61799999999999999,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.25},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.46500000000000002,"15min":0.498},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.50700000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.33800000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.50800000000000001},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.59299999999999997,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41599999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52700000000000002,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.311}]},{"osd":4,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66400000000000003,"15min":0.68000000000000005},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85299999999999998,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.48099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88200000000000001,"5min":1.405,"15min":1.405},"last":0.56299999999999994}]},{"osd":5,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.50700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83799999999999997,"5min":1.377,"15min":1.377},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":0.80900000000000005},"min":{"1min":0.49399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91100000000000003,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.52600000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66200000000000003,"15min":0.67500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92500000000000004,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.75600000000000001,"15min":0.80200000000000005},"min":{"1min":0.51700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.871,"5min":4.883,"15min":4.883},"last":0.45800000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.75700000000000001,"15min":0.81899999999999995},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80200000000000005,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71599999999999997,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85299999999999998,"5min":2.923,"15min":2.923},"last":0.44900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":291417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.41499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77500000000000002,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.628},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.43099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.60599999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.53000000000000003,"15min":0.51200000000000001},"min":{"1min":0.44700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77500000000000002,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.63400000000000001,"15min":0.66400000000000003},"min":{"1min":0.45600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71099999999999997,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.72199999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54300000000000004,"15min":0.52500000000000002},"min":{"1min":0.47899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.68600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.64700000000000002,"15min":0.67900000000000005},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74299999999999999,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.67400000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.40500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75900000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.40400000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.752,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.317,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73399999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.48999999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.54300000000000004,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.441},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72999999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Nov 26 07:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71699999999999997,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.748},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.625,"15min":0.63600000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79600000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":290457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.39200000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.49099999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.748,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.48799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.73799999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.57899999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.80700000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.53800000000000003},"min":{"1min":0.42399999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52600000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.45900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60499999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.498,"15min":0.5},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.61599999999999999,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.55000000000000004,"15min":0.55700000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.67300000000000004,"5min":0.752,"15min":0.752},"last":0.373}]},{"osd":6,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57199999999999995,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.65800000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.35999999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70899999999999996,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.42699999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59399999999999997,"15min":0.59699999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.64800000000000002,"15min":0.65200000000000002},"min":{"1min":0.45200000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76600000000000001,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.39800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:15:08.526 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:15:08.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:15:08.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.688+0000 7f0d603d3700 1 Processor -- start 2023-11-26T07:15:08.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.688+0000 7f0d603d3700 1 -- start start 2023-11-26T07:15:08.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d603d3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d5815e160 0x7f0d580b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d603d3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d580746a0 0x7f0d5806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d603d3700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d580b3c60 con 0x7f0d580afc90 2023-11-26T07:15:08.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d603d3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d580b4500 con 0x7f0d580746a0 2023-11-26T07:15:08.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d603d3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d580b58c0 con 0x7f0d5815e160 2023-11-26T07:15:08.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d5e970700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d580746a0 0x7f0d5806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d5e970700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0d580746a0 0x7f0d5806d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38526/0 (socket says 172.21.15.83:38526) 2023-11-26T07:15:08.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d5e970700 1 -- 172.21.15.83:0/2702023312 learned_addr learned my addr 172.21.15.83:0/2702023312 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:08.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d5e16f700 1 --2- 172.21.15.83:0/2702023312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d5815e160 0x7f0d580b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d5e970700 1 -- 172.21.15.83:0/2702023312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d5815e160 msgr2=0x7f0d580b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:08.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d5e970700 1 --2- 172.21.15.83:0/2702023312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d5815e160 0x7f0d580b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d5e970700 1 -- 172.21.15.83:0/2702023312 >> v1:172.21.15.203:6789/0 conn(0x7f0d580afc90 legacy=0x7f0d58073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d5e970700 1 -- 172.21.15.83:0/2702023312 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5806bdc0 con 0x7f0d580746a0 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d5e970700 1 --2- 172.21.15.83:0/2702023312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d580746a0 0x7f0d5806d930 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009e50 tx=0x7f0d5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febb57d1801b6e7c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.689+0000 7f0d5d16d700 1 -- 172.21.15.83:0/2702023312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000e070 con 0x7f0d580746a0 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.690+0000 7f0d5d16d700 1 -- 172.21.15.83:0/2702023312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d50003a20 con 0x7f0d580746a0 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.690+0000 7f0d5d16d700 1 -- 172.21.15.83:0/2702023312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50003dd0 con 0x7f0d580746a0 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.690+0000 7f0d603d3700 1 -- 172.21.15.83:0/2702023312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d580746a0 msgr2=0x7f0d5806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.690+0000 7f0d603d3700 1 --2- 172.21.15.83:0/2702023312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d580746a0 0x7f0d5806d930 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009e50 tx=0x7f0d5000b330 comp rx=0 tx=0).stop 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.690+0000 7f0d603d3700 1 -- 172.21.15.83:0/2702023312 shutdown_connections 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.690+0000 7f0d603d3700 1 --2- 172.21.15.83:0/2702023312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d580746a0 0x7f0d5806d930 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.690+0000 7f0d603d3700 1 --2- 172.21.15.83:0/2702023312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d5815e160 0x7f0d580b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.690+0000 7f0d603d3700 1 -- 172.21.15.83:0/2702023312 >> 172.21.15.83:0/2702023312 conn(0x7f0d580d0bf0 msgr2=0x7f0d580d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:08.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.690+0000 7f0d603d3700 1 -- 172.21.15.83:0/2702023312 shutdown_connections 2023-11-26T07:15:08.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.690+0000 7f0d603d3700 1 -- 172.21.15.83:0/2702023312 wait complete. 2023-11-26T07:15:08.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.691+0000 7f0d603d3700 1 Processor -- start 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.691+0000 7f0d603d3700 1 -- start start 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.692+0000 7f0d603d3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d580746a0 0x7f0d58155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.692+0000 7f0d603d3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d580afc90 0x7f0d581583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.692+0000 7f0d603d3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d5815e160 0x7f0d58160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.692+0000 7f0d603d3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58156120 con 0x7f0d580afc90 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.692+0000 7f0d603d3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58156f10 con 0x7f0d580746a0 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.692+0000 7f0d603d3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58156300 con 0x7f0d5815e160 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.692+0000 7f0d5d96e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d580afc90 0x7f0d581583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.692+0000 7f0d5d96e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d580afc90 0x7f0d581583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38530/0 (socket says 172.21.15.83:38530) 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.692+0000 7f0d5d96e700 1 -- 172.21.15.83:0/3621081708 learned_addr learned my addr 172.21.15.83:0/3621081708 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.692+0000 7f0d5e970700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d5815e160 0x7f0d58160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.692+0000 7f0d5e16f700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d580746a0 0x7f0d58155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d5d96e700 1 -- 172.21.15.83:0/3621081708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d5815e160 msgr2=0x7f0d58160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d5d96e700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d5815e160 0x7f0d58160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d5d96e700 1 -- 172.21.15.83:0/3621081708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d580746a0 msgr2=0x7f0d58155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d5d96e700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d580746a0 0x7f0d58155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d5d96e700 1 -- 172.21.15.83:0/3621081708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d50009b30 con 0x7f0d580afc90 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d5e16f700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d580746a0 0x7f0d58155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d5e970700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d5815e160 0x7f0d58160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d5d96e700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d580afc90 0x7f0d581583b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400c4d0 tx=0x7f0d5400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d4effd700 1 -- 172.21.15.83:0/3621081708 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d54011790 con 0x7f0d580afc90 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d4effd700 1 -- 172.21.15.83:0/3621081708 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d540118f0 con 0x7f0d580afc90 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d603d3700 1 -- 172.21.15.83:0/3621081708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d581565e0 con 0x7f0d580afc90 2023-11-26T07:15:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d4effd700 1 -- 172.21.15.83:0/3621081708 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d54011ca0 con 0x7f0d580afc90 2023-11-26T07:15:08.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.693+0000 7f0d603d3700 1 -- 172.21.15.83:0/3621081708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d58156ba0 con 0x7f0d580afc90 2023-11-26T07:15:08.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.694+0000 7f0d603d3700 1 -- 172.21.15.83:0/3621081708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d581570a0 con 0x7f0d580afc90 2023-11-26T07:15:08.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.694+0000 7f0d4effd700 1 -- 172.21.15.83:0/3621081708 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0d5401c460 con 0x7f0d580afc90 2023-11-26T07:15:08.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.695+0000 7f0d4effd700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0d3c06b080 0x7f0d3c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:08.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.695+0000 7f0d4effd700 1 -- 172.21.15.83:0/3621081708 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d540896d0 con 0x7f0d580afc90 2023-11-26T07:15:08.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.695+0000 7f0d5e16f700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0d3c06b080 0x7f0d3c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:08.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.695+0000 7f0d5e16f700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0d3c06b080 0x7f0d3c06d540 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0d48007a50 tx=0x7f0d48008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:08.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.699+0000 7f0d4effd700 1 -- 172.21.15.83:0/3621081708 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d54054500 con 0x7f0d580afc90 2023-11-26T07:15:08.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.967+0000 7f0d603d3700 1 -- 172.21.15.83:0/3621081708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0d580afa80 con 0x7f0d580afc90 2023-11-26T07:15:08.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.969+0000 7f0d4effd700 1 -- 172.21.15.83:0/3621081708 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0d54057b20 con 0x7f0d580afc90 2023-11-26T07:15:08.970 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2023-11-26T07:15:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ== 2023-11-26T07:15:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2023-11-26T07:15:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2023-11-26T07:15:08.971 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:15:08.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 -- 172.21.15.83:0/3621081708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0d3c06b080 msgr2=0x7f0d3c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:08.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0d3c06b080 0x7f0d3c06d540 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0d48007a50 tx=0x7f0d48008040 comp rx=0 tx=0).stop 2023-11-26T07:15:08.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 -- 172.21.15.83:0/3621081708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d580afc90 msgr2=0x7f0d581583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:08.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d580afc90 0x7f0d581583b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400c4d0 tx=0x7f0d5400c890 comp rx=0 tx=0).stop 2023-11-26T07:15:08.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 -- 172.21.15.83:0/3621081708 shutdown_connections 2023-11-26T07:15:08.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0d3c06b080 0x7f0d3c06d540 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d5815e160 0x7f0d58160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d580746a0 0x7f0d58155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 --2- 172.21.15.83:0/3621081708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d580afc90 0x7f0d581583b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:08.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 -- 172.21.15.83:0/3621081708 >> 172.21.15.83:0/3621081708 conn(0x7f0d580d0bf0 msgr2=0x7f0d580aeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:08.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 -- 172.21.15.83:0/3621081708 shutdown_connections 2023-11-26T07:15:08.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:08.972+0000 7f0d603d3700 1 -- 172.21.15.83:0/3621081708 wait complete. 2023-11-26T07:15:08.975 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2023-11-26T07:15:08.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:15:09.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.104+0000 7f41a1b3d700 1 Processor -- start 2023-11-26T07:15:09.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.104+0000 7f41a1b3d700 1 -- start start 2023-11-26T07:15:09.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.105+0000 7f41a1b3d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f419c0adf00 0x7f419c06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.105+0000 7f41a1b3d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419c0af950 con 0x7f419c06c540 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.105+0000 7f41a1b3d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419c0afc30 con 0x7f419c1446a0 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.105+0000 7f41a1b3d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c144a80 con 0x7f419c0adf00 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f419c06c540 0x7f419c0b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58578/0 (socket says 172.21.15.83:58578) 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419bfff700 1 -- 172.21.15.83:0/1773291959 learned_addr learned my addr 172.21.15.83:0/1773291959 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419affd700 1 --2- 172.21.15.83:0/1773291959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f419c0adf00 0x7f419c06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419a7fc700 1 -- 172.21.15.83:0/1773291959 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380974727 0 0) 0x7f419c0af950 con 0x7f419c06c540 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419a7fc700 1 -- 172.21.15.83:0/1773291959 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41880057c0 con 0x7f419c06c540 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419a7fc700 1 -- 172.21.15.83:0/1773291959 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146598576 0 0) 0x7f419c0afc30 con 0x7f419c1446a0 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419a7fc700 1 -- 172.21.15.83:0/1773291959 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f419c0af950 con 0x7f419c1446a0 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419affd700 1 -- 172.21.15.83:0/1773291959 >> v1:172.21.15.203:6789/0 conn(0x7f419c1446a0 legacy=0x7f419c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419affd700 1 -- 172.21.15.83:0/1773291959 >> v1:172.21.15.83:6789/0 conn(0x7f419c06c540 legacy=0x7f419c0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419a7fc700 1 -- 172.21.15.83:0/1773291959 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 552558061 0 0) 0x7f41880057c0 con 0x7f419c06c540 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419affd700 1 -- 172.21.15.83:0/1773291959 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c0aff10 con 0x7f419c0adf00 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419affd700 1 --2- 172.21.15.83:0/1773291959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f419c0adf00 0x7f419c06c000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f418c009870 tx=0x7f418c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9722f39292c1cd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.106+0000 7f419a7fc700 1 -- 172.21.15.83:0/1773291959 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41940019f0 con 0x7f419c0adf00 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.107+0000 7f419a7fc700 1 -- 172.21.15.83:0/1773291959 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c004030 con 0x7f419c0adf00 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.107+0000 7f419a7fc700 1 -- 172.21.15.83:0/1773291959 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c0043c0 con 0x7f419c0adf00 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.107+0000 7f41a1b3d700 1 -- 172.21.15.83:0/1773291959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f419c0adf00 msgr2=0x7f419c06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.107+0000 7f41a1b3d700 1 --2- 172.21.15.83:0/1773291959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f419c0adf00 0x7f419c06c000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f418c009870 tx=0x7f418c009b80 comp rx=0 tx=0).stop 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.107+0000 7f41a1b3d700 1 -- 172.21.15.83:0/1773291959 shutdown_connections 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.107+0000 7f41a1b3d700 1 --2- 172.21.15.83:0/1773291959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f419c0adf00 0x7f419c06c000 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.107+0000 7f41a1b3d700 1 -- 172.21.15.83:0/1773291959 >> 172.21.15.83:0/1773291959 conn(0x7f419c071b90 msgr2=0x7f419c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:09.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.107+0000 7f41a1b3d700 1 -- 172.21.15.83:0/1773291959 shutdown_connections 2023-11-26T07:15:09.111 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.107+0000 7f41a1b3d700 1 -- 172.21.15.83:0/1773291959 wait complete. 2023-11-26T07:15:09.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.109+0000 7f41a1b3d700 1 Processor -- start 2023-11-26T07:15:09.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.109+0000 7f41a1b3d700 1 -- start start 2023-11-26T07:15:09.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f41a1b3d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f419c06c540 0x7f419c13b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f41a1b3d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f419c0adf00 0x7f419c1373d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f41a1b3d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419c1446a0 0x7f419c1400f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f41a1b3d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c142c50 con 0x7f419c1446a0 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f41a1b3d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c141cf0 con 0x7f419c0adf00 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f41a1b3d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c141fd0 con 0x7f419c06c540 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f419bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419c1446a0 0x7f419c1400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f419bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419c1446a0 0x7f419c1400f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38546/0 (socket says 172.21.15.83:38546) 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f419bfff700 1 -- 172.21.15.83:0/3108658978 learned_addr learned my addr 172.21.15.83:0/3108658978 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f419affd700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f419c0adf00 0x7f419c1373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f419b7fe700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f419c06c540 0x7f419c13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f419bfff700 1 -- 172.21.15.83:0/3108658978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f419c06c540 msgr2=0x7f419c13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f419bfff700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f419c06c540 0x7f419c13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f419bfff700 1 -- 172.21.15.83:0/3108658978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f419c0adf00 msgr2=0x7f419c1373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f419bfff700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f419c0adf00 0x7f419c1373d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f419bfff700 1 -- 172.21.15.83:0/3108658978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c0094a0 con 0x7f419c1446a0 2023-11-26T07:15:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f419bfff700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419c1446a0 0x7f419c1400f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f419400cc00 tx=0x7f419400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f4193fff700 1 -- 172.21.15.83:0/3108658978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419400aa80 con 0x7f419c1446a0 2023-11-26T07:15:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f4193fff700 1 -- 172.21.15.83:0/3108658978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419400abe0 con 0x7f419c1446a0 2023-11-26T07:15:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f41a1b3d700 1 -- 172.21.15.83:0/3108658978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419c1422b0 con 0x7f419c1446a0 2023-11-26T07:15:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.110+0000 7f4193fff700 1 -- 172.21.15.83:0/3108658978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41940070d0 con 0x7f419c1446a0 2023-11-26T07:15:09.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.111+0000 7f41a1b3d700 1 -- 172.21.15.83:0/3108658978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419c142870 con 0x7f419c1446a0 2023-11-26T07:15:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.112+0000 7f41a1b3d700 1 -- 172.21.15.83:0/3108658978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419c0753f0 con 0x7f419c1446a0 2023-11-26T07:15:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.112+0000 7f4193fff700 1 -- 172.21.15.83:0/3108658978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f4194007230 con 0x7f419c1446a0 2023-11-26T07:15:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.112+0000 7f4193fff700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f417c06b080 0x7f417c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.112+0000 7f4193fff700 1 -- 172.21.15.83:0/3108658978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4194089630 con 0x7f419c1446a0 2023-11-26T07:15:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.112+0000 7f419b7fe700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f417c06b080 0x7f417c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.113+0000 7f419b7fe700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f417c06b080 0x7f417c06d540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f41840011e0 tx=0x7f4184009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.117+0000 7f4193fff700 1 -- 172.21.15.83:0/3108658978 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41940544f0 con 0x7f419c1446a0 2023-11-26T07:15:09.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.346+0000 7f41a1b3d700 1 -- 172.21.15.83:0/3108658978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f419c072100 con 0x7f419c1446a0 2023-11-26T07:15:09.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.346+0000 7f4193fff700 1 -- 172.21.15.83:0/3108658978 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f4194057b10 con 0x7f419c1446a0 2023-11-26T07:15:09.348 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:09.348 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/126258309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":126258309},{"type":"v1","addr":"172.21.15.203:6834","nonce":126258309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4203521524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":4203521524},{"type":"v1","addr":"172.21.15.203:6835","nonce":4203521524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1267615204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1267615204},{"type":"v1","addr":"172.21.15.83:6836","nonce":1267615204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:14:54.936357+0000","modified":"2023-11-26T07:14:57.974394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6102":{"gid":6102,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/83108157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":83108157},{"type":"v1","addr":"172.21.15.83:6837","nonce":83108157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:15:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 -- 172.21.15.83:0/3108658978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f417c06b080 msgr2=0x7f417c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f417c06b080 0x7f417c06d540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f41840011e0 tx=0x7f4184009040 comp rx=0 tx=0).stop 2023-11-26T07:15:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 -- 172.21.15.83:0/3108658978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419c1446a0 msgr2=0x7f419c1400f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419c1446a0 0x7f419c1400f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f419400cc00 tx=0x7f419400cf10 comp rx=0 tx=0).stop 2023-11-26T07:15:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 -- 172.21.15.83:0/3108658978 shutdown_connections 2023-11-26T07:15:09.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f419c1446a0 0x7f419c1400f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f417c06b080 0x7f417c06d540 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f419c06c540 0x7f419c13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 --2- 172.21.15.83:0/3108658978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f419c0adf00 0x7f419c1373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 -- 172.21.15.83:0/3108658978 >> 172.21.15.83:0/3108658978 conn(0x7f419c071b90 msgr2=0x7f419c0b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 -- 172.21.15.83:0/3108658978 shutdown_connections 2023-11-26T07:15:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.348+0000 7f41a1b3d700 1 -- 172.21.15.83:0/3108658978 wait complete. 2023-11-26T07:15:09.351 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 37 2023-11-26T07:15:09.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:14:54.936357+0000', 'modified': '2023-11-26T07:14:57.974394+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6102}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6102': {'gid': 6102, 'name': 'a', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6837/83108157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6835', 'nonce': 83108157}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 83108157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:15:09.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:15:09.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-26T07:15:09.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.888+0000 7f12d0bd4700 1 Processor -- start 2023-11-26T07:15:09.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.888+0000 7f12d0bd4700 1 -- start start 2023-11-26T07:15:09.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12d0bd4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f12cc0adf00 0x7f12cc06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12d0bd4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 0x7f12cc0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12d0bd4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc0af950 con 0x7f12cc1446a0 2023-11-26T07:15:09.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12d0bd4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc144a80 con 0x7f12cc0adf00 2023-11-26T07:15:09.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12d0bd4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc06c9c0 con 0x7f12cc06c540 2023-11-26T07:15:09.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12c9d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f12cc0adf00 0x7f12cc06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12ca59c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f12cc1446a0 0x7f12cc0b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58590/0 (socket says 172.21.15.83:58590) 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12ca59c700 1 -- 172.21.15.83:0/2616017885 learned_addr learned my addr 172.21.15.83:0/2616017885 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12cad9d700 1 --2- 172.21.15.83:0/2616017885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 0x7f12cc0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12c9d9b700 1 -- 172.21.15.83:0/2616017885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 msgr2=0x7f12cc0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12c9d9b700 1 --2- 172.21.15.83:0/2616017885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 0x7f12cc0b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12c9d9b700 1 -- 172.21.15.83:0/2616017885 >> v1:172.21.15.83:6789/0 conn(0x7f12cc1446a0 legacy=0x7f12cc0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12c9d9b700 1 -- 172.21.15.83:0/2616017885 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0afc30 con 0x7f12cc0adf00 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12c959a700 1 -- 172.21.15.83:0/2616017885 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002688145 0 0) 0x7f12cc0af950 con 0x7f12cc1446a0 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12cad9d700 1 --2- 172.21.15.83:0/2616017885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 0x7f12cc0b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.889+0000 7f12c9d9b700 1 --2- 172.21.15.83:0/2616017885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc0adf00 0x7f12cc06c000 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009840 tx=0x7f12c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8637eaeb699fc82 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.890+0000 7f12c959a700 1 -- 172.21.15.83:0/2616017885 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0011830 con 0x7f12cc0adf00 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.890+0000 7f12c959a700 1 -- 172.21.15.83:0/2616017885 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c0011990 con 0x7f12cc0adf00 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.890+0000 7f12c959a700 1 -- 172.21.15.83:0/2616017885 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0011d20 con 0x7f12cc0adf00 2023-11-26T07:15:09.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.890+0000 7f12d0bd4700 1 -- 172.21.15.83:0/2616017885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc0adf00 msgr2=0x7f12cc06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.890+0000 7f12d0bd4700 1 --2- 172.21.15.83:0/2616017885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc0adf00 0x7f12cc06c000 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009840 tx=0x7f12c0009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:09.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.890+0000 7f12d0bd4700 1 -- 172.21.15.83:0/2616017885 shutdown_connections 2023-11-26T07:15:09.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.890+0000 7f12d0bd4700 1 --2- 172.21.15.83:0/2616017885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 0x7f12cc0b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.890+0000 7f12d0bd4700 1 --2- 172.21.15.83:0/2616017885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc0adf00 0x7f12cc06c000 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.890+0000 7f12d0bd4700 1 -- 172.21.15.83:0/2616017885 >> 172.21.15.83:0/2616017885 conn(0x7f12cc071b90 msgr2=0x7f12cc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:09.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.890+0000 7f12d0bd4700 1 -- 172.21.15.83:0/2616017885 shutdown_connections 2023-11-26T07:15:09.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.890+0000 7f12d0bd4700 1 -- 172.21.15.83:0/2616017885 wait complete. 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.892+0000 7f12d0bd4700 1 Processor -- start 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.892+0000 7f12d0bd4700 1 -- start start 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.892+0000 7f12d0bd4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 0x7f12cc136fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.892+0000 7f12d0bd4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12cc0adf00 0x7f12cc15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.892+0000 7f12d0bd4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc1446a0 0x7f12cc143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.892+0000 7f12d0bd4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc1374e0 con 0x7f12cc1446a0 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.892+0000 7f12d0bd4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc13a3b0 con 0x7f12cc0adf00 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.892+0000 7f12d0bd4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc15b880 con 0x7f12cc06c540 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.892+0000 7f12cad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc1446a0 0x7f12cc143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12cad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc1446a0 0x7f12cc143180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38558/0 (socket says 172.21.15.83:38558) 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12cad9d700 1 -- 172.21.15.83:0/3332327343 learned_addr learned my addr 172.21.15.83:0/3332327343 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:09.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12ca59c700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 0x7f12cc136fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12c9d9b700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12cc0adf00 0x7f12cc15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12cad9d700 1 -- 172.21.15.83:0/3332327343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 msgr2=0x7f12cc136fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12cad9d700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 0x7f12cc136fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12cad9d700 1 -- 172.21.15.83:0/3332327343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12cc0adf00 msgr2=0x7f12cc15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12cad9d700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12cc0adf00 0x7f12cc15b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12cad9d700 1 -- 172.21.15.83:0/3332327343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c00094a0 con 0x7f12cc1446a0 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12ca59c700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 0x7f12cc136fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12c9d9b700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12cc0adf00 0x7f12cc15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12cad9d700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc1446a0 0x7f12cc143180 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f12c400cef0 tx=0x7f12c400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12baffd700 1 -- 172.21.15.83:0/3332327343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400ae50 con 0x7f12cc1446a0 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12baffd700 1 -- 172.21.15.83:0/3332327343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c4015440 con 0x7f12cc1446a0 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12d0bd4700 1 -- 172.21.15.83:0/3332327343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc13a650 con 0x7f12cc1446a0 2023-11-26T07:15:09.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12baffd700 1 -- 172.21.15.83:0/3332327343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c40157f0 con 0x7f12cc1446a0 2023-11-26T07:15:09.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.893+0000 7f12d0bd4700 1 -- 172.21.15.83:0/3332327343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc142980 con 0x7f12cc1446a0 2023-11-26T07:15:09.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.894+0000 7f12baffd700 1 -- 172.21.15.83:0/3332327343 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f12c400d430 con 0x7f12cc1446a0 2023-11-26T07:15:09.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.895+0000 7f12baffd700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f12ac06b080 0x7f12ac06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.895+0000 7f12baffd700 1 -- 172.21.15.83:0/3332327343 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12c4089f60 con 0x7f12cc1446a0 2023-11-26T07:15:09.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.895+0000 7f12ca59c700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f12ac06b080 0x7f12ac06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.895+0000 7f12ca59c700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f12ac06b080 0x7f12ac06d540 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000f60 tx=0x7f12bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:09.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.897+0000 7f12dcdcdb80 1 Processor -- start 2023-11-26T07:15:09.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.897+0000 7f12dcdcdb80 1 -- start start 2023-11-26T07:15:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc4080 0x55ac5000a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac5000ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc1490 0x55ac4ffede70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4ff37770 con 0x55ac4ffc1490 2023-11-26T07:15:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4ff1e750 con 0x55ac4ffc4080 2023-11-26T07:15:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac4ff1c070 con 0x55ac4ffd9ed0 2023-11-26T07:15:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12cad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc1490 0x55ac4ffede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12cad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc1490 0x55ac4ffede70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38566/0 (socket says 172.21.15.83:38566) 2023-11-26T07:15:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12cad9d700 1 -- 172.21.15.83:0/3794898800 learned_addr learned my addr 172.21.15.83:0/3794898800 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12cad9d700 1 -- 172.21.15.83:0/3794898800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 msgr2=0x55ac5000ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:09.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12cad9d700 1 --2- 172.21.15.83:0/3794898800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac5000ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12cad9d700 1 -- 172.21.15.83:0/3794898800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc4080 msgr2=0x55ac5000a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12cad9d700 1 --2- 172.21.15.83:0/3794898800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc4080 0x55ac5000a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12cad9d700 1 -- 172.21.15.83:0/3794898800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac4fd945e0 con 0x55ac4ffc1490 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12cad9d700 1 --2- 172.21.15.83:0/3794898800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc1490 0x55ac4ffede70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f12c400cec0 tx=0x7f12c4058820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14d0cd98ddcbc9c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12abfff700 1 -- 172.21.15.83:0/3794898800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4060d50 con 0x55ac4ffc1490 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12abfff700 1 -- 172.21.15.83:0/3794898800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c40701c0 con 0x55ac4ffc1490 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.898+0000 7f12abfff700 1 -- 172.21.15.83:0/3794898800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4058ef0 con 0x55ac4ffc1490 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.899+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/3794898800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc1490 msgr2=0x55ac4ffede70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.899+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/3794898800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc1490 0x55ac4ffede70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f12c400cec0 tx=0x7f12c4058820 comp rx=0 tx=0).stop 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.899+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/3794898800 shutdown_connections 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.899+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/3794898800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc1490 0x55ac4ffede70 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.899+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/3794898800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc4080 0x55ac5000a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.899+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/3794898800 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac5000ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.899+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/3794898800 >> 172.21.15.83:0/3794898800 conn(0x55ac4fef2310 msgr2=0x55ac50008e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:09.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.899+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/3794898800 shutdown_connections 2023-11-26T07:15:09.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.899+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/3794898800 wait complete. 2023-11-26T07:15:09.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.900+0000 7f12dcdcdb80 1 -- start start 2023-11-26T07:15:09.904 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.900+0000 7f12dcdcdb80 1 Processor -- start 2023-11-26T07:15:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc1490 0x55ac4ffdc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac4ffd9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac4ffdee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4ff17960 con 0x55ac4ffc4080 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4fed1510 con 0x55ac4ffc1490 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac4fed11b0 con 0x55ac4ffd9ed0 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12dcdcdb80 10 client.0 _ll_register_callbacks cb 0x55ac4fdfca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12c9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac4ffd9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12c9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac4ffd9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38570/0 (socket says 172.21.15.83:38570) 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12c9d9b700 1 -- 172.21.15.83:0/4173092281 learned_addr learned my addr 172.21.15.83:0/4173092281 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12ca59c700 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc1490 0x55ac4ffdc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12cad9d700 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac4ffdee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12c9d9b700 1 -- 172.21.15.83:0/4173092281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 msgr2=0x55ac4ffdee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12c9d9b700 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac4ffdee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12c9d9b700 1 -- 172.21.15.83:0/4173092281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc1490 msgr2=0x55ac4ffdc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:09.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12c9d9b700 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc1490 0x55ac4ffdc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12c9d9b700 1 -- 172.21.15.83:0/4173092281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc142980 con 0x55ac4ffc4080 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12c9d9b700 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac4ffd9890 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f12c0000c00 tx=0x7f12c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1541de1d59b3a47c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.901+0000 7f12a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12a97fa700 1 -- 172.21.15.83:0/4173092281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0004450 con 0x55ac4ffc4080 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12a97fa700 1 -- 172.21.15.83:0/4173092281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12c00036a0 con 0x55ac4ffc4080 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4173092281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac4ffb6a40 con 0x55ac4ffc4080 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12a97fa700 1 -- 172.21.15.83:0/4173092281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12c00045b0 con 0x55ac4ffc4080 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12dcdcdb80 10 client.6288 fetch_fsmap learned FSMap version 37 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4173092281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55ac4fd40d90 con 0x55ac4ffc4080 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12a97fa700 1 -- 172.21.15.83:0/4173092281 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ac4ffb6a40 con 0x55ac4ffc4080 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12a97fa700 1 -- 172.21.15.83:0/4173092281 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f12c0003860 con 0x55ac4ffc4080 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12dcdcdb80 10 client.6288 fetch_fsmap finished waiting for FSMap version 37 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12dcdcdb80 10 client.6288 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:15:09.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12dcdcdb80 20 client.6288 populate_metadata read hostname 'smithi083' 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.83:6835/83108157,v1:172.21.15.83:6837/83108157] conn(0x55ac4ffb8760 0x55ac4fff0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12dcdcdb80 4 client.6288 mds_command: new command op to 6102 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4173092281 --> [v2:172.21.15.83:6835/83108157,v1:172.21.15.83:6837/83108157] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac4ff92840 con 0x55ac4ffb8760 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12ca59c700 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.83:6835/83108157,v1:172.21.15.83:6837/83108157] conn(0x55ac4ffb8760 0x55ac4fff0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12a97fa700 10 client.6288 ms_handle_connect on v2:172.21.15.83:6835/83108157 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.902+0000 7f12ca59c700 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.83:6835/83108157,v1:172.21.15.83:6837/83108157] conn(0x55ac4ffb8760 0x55ac4fff0060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aadfa8a757f73362 server_cookie=c849601ef16903e5 in_seq=0 out_seq=0 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12a97fa700 1 -- 172.21.15.83:0/4173092281 <== mds.0 v2:172.21.15.83:6835/83108157 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ac4ff92840 con 0x55ac4ffb8760 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12a97fa700 10 client.6288 handle_command_reply: tid=0 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 1 client.6288 shutdown 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4173092281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 msgr2=0x55ac4ffd9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac4ffd9890 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f12c0000c00 tx=0x7f12c0004750 comp rx=0 tx=0).stop 2023-11-26T07:15:09.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4173092281 shutdown_connections 2023-11-26T07:15:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac4ffdee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc1490 0x55ac4ffdc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.83:6835/83108157,v1:172.21.15.83:6837/83108157] conn(0x55ac4ffb8760 0x55ac4fff0060 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4173092281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac4ffd9890 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12a97fa700 0 client.6288 ms_handle_reset on v2:172.21.15.83:6835/83108157 2023-11-26T07:15:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4173092281 >> 172.21.15.83:0/4173092281 conn(0x55ac4fef2310 msgr2=0x55ac4ffee820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4173092281 shutdown_connections 2023-11-26T07:15:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4173092281 wait complete. 2023-11-26T07:15:09.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.903+0000 7f12dcdcdb80 20 client.6288 trim_cache size 0 max 16384 2023-11-26T07:15:09.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.921+0000 7f12dcdcdb80 1 Processor -- start 2023-11-26T07:15:09.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.921+0000 7f12dcdcdb80 1 -- start start 2023-11-26T07:15:09.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffc1490 0x55ac4ffbf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc4080 0x55ac4ffbf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffd9ed0 0x55ac4ffd7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4fd968d0 con 0x55ac4ffd9ed0 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac50005440 con 0x55ac4ffc4080 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac500066a0 con 0x55ac4ffc1490 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12cad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffd9ed0 0x55ac4ffd7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12cad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffd9ed0 0x55ac4ffd7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38580/0 (socket says 172.21.15.83:38580) 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12cad9d700 1 -- 172.21.15.83:0/478787061 learned_addr learned my addr 172.21.15.83:0/478787061 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12c9d9b700 1 --2- 172.21.15.83:0/478787061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc4080 0x55ac4ffbf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12ca59c700 1 --2- 172.21.15.83:0/478787061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffc1490 0x55ac4ffbf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12cad9d700 1 -- 172.21.15.83:0/478787061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffc1490 msgr2=0x55ac4ffbf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12cad9d700 1 --2- 172.21.15.83:0/478787061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffc1490 0x55ac4ffbf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12cad9d700 1 -- 172.21.15.83:0/478787061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc4080 msgr2=0x55ac4ffbf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12cad9d700 1 --2- 172.21.15.83:0/478787061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc4080 0x55ac4ffbf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12cad9d700 1 -- 172.21.15.83:0/478787061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac4fd29c10 con 0x55ac4ffd9ed0 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12ca59c700 1 --2- 172.21.15.83:0/478787061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffc1490 0x55ac4ffbf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.922+0000 7f12cad9d700 1 --2- 172.21.15.83:0/478787061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffd9ed0 0x55ac4ffd7a40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f12c400a950 tx=0x7f12c401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b6dfb6a75a677f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12b8ff9700 1 -- 172.21.15.83:0/478787061 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4083040 con 0x55ac4ffd9ed0 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12b8ff9700 1 -- 172.21.15.83:0/478787061 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c4070330 con 0x55ac4ffd9ed0 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12b8ff9700 1 -- 172.21.15.83:0/478787061 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4058180 con 0x55ac4ffd9ed0 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/478787061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffd9ed0 msgr2=0x55ac4ffd7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/478787061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffd9ed0 0x55ac4ffd7a40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f12c400a950 tx=0x7f12c401a000 comp rx=0 tx=0).stop 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/478787061 shutdown_connections 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/478787061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffd9ed0 0x55ac4ffd7a40 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/478787061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffc1490 0x55ac4ffbf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/478787061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc4080 0x55ac4ffbf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/478787061 >> 172.21.15.83:0/478787061 conn(0x55ac4fef2310 msgr2=0x55ac4ffee820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/478787061 shutdown_connections 2023-11-26T07:15:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.923+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/478787061 wait complete. 2023-11-26T07:15:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.924+0000 7f12dcdcdb80 1 -- start start 2023-11-26T07:15:09.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.924+0000 7f12dcdcdb80 1 Processor -- start 2023-11-26T07:15:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc1490 0x55ac5001a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac5001a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12dcdcdb80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac50008ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac50009ba0 con 0x55ac4ffc4080 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac4ffedfe0 con 0x55ac4ffc1490 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12dcdcdb80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac50071a70 con 0x55ac4ffd9ed0 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12dcdcdb80 10 client.0 _ll_register_callbacks cb 0x55ac4fdfca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12c9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac5001a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12c9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac5001a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38584/0 (socket says 172.21.15.83:38584) 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12c9d9b700 1 -- 172.21.15.83:0/4104364209 learned_addr learned my addr 172.21.15.83:0/4104364209 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12ca59c700 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc1490 0x55ac5001a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12cad9d700 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac50008ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12c9d9b700 1 -- 172.21.15.83:0/4104364209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 msgr2=0x55ac50008ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:09.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12c9d9b700 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac50008ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12c9d9b700 1 -- 172.21.15.83:0/4104364209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc1490 msgr2=0x55ac5001a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12c9d9b700 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc1490 0x55ac5001a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12c9d9b700 1 -- 172.21.15.83:0/4104364209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac4fd945e0 con 0x55ac4ffc4080 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12cad9d700 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac50008ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12c9d9b700 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac5001a9a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f12c00090b0 tx=0x7f12c0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf459b3b08917527 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12a97fa700 1 -- 172.21.15.83:0/4104364209 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0004660 con 0x55ac4ffc4080 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12a97fa700 1 -- 172.21.15.83:0/4104364209 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12c00276f0 con 0x55ac4ffc4080 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.925+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4104364209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12c40701c0 con 0x55ac4ffc4080 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12a97fa700 1 -- 172.21.15.83:0/4104364209 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12c0003d20 con 0x55ac4ffc4080 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12dcdcdb80 10 client.6294 fetch_fsmap learned FSMap version 37 2023-11-26T07:15:09.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4104364209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55ac4fd67200 con 0x55ac4ffc4080 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12a97fa700 1 -- 172.21.15.83:0/4104364209 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c0041020 con 0x55ac4ffc4080 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12a97fa700 1 -- 172.21.15.83:0/4104364209 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f12c0027020 con 0x55ac4ffc4080 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12dcdcdb80 10 client.6294 fetch_fsmap finished waiting for FSMap version 37 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12dcdcdb80 10 client.6294 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12dcdcdb80 20 client.6294 populate_metadata read hostname 'smithi083' 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.83:6835/83108157,v1:172.21.15.83:6837/83108157] conn(0x55ac500091e0 0x55ac500095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12dcdcdb80 4 client.6294 mds_command: new command op to 6102 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4104364209 --> [v2:172.21.15.83:6835/83108157,v1:172.21.15.83:6837/83108157] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ac4feddf80 con 0x55ac500091e0 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12ca59c700 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.83:6835/83108157,v1:172.21.15.83:6837/83108157] conn(0x55ac500091e0 0x55ac500095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12a97fa700 10 client.6294 ms_handle_connect on v2:172.21.15.83:6835/83108157 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.926+0000 7f12ca59c700 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.83:6835/83108157,v1:172.21.15.83:6837/83108157] conn(0x55ac500091e0 0x55ac500095c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fcce46aea1e2062 server_cookie=a3dd1df88443889b in_seq=0 out_seq=0 2023-11-26T07:15:09.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12a97fa700 1 -- 172.21.15.83:0/4104364209 <== mds.0 v2:172.21.15.83:6835/83108157 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55ac4feddf80 con 0x55ac500091e0 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12a97fa700 10 client.6294 handle_command_reply: tid=0 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 1 client.6294 shutdown 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4104364209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 msgr2=0x55ac5001a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac5001a9a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f12c00090b0 tx=0x7f12c0008030 comp rx=0 tx=0).stop 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4104364209 shutdown_connections 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ac4ffd9ed0 0x55ac50008ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac4ffc1490 0x55ac5001a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.83:6835/83108157,v1:172.21.15.83:6837/83108157] conn(0x55ac500091e0 0x55ac500095c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 1 --2- 172.21.15.83:0/4104364209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac4ffc4080 0x55ac5001a9a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12a97fa700 0 client.6294 ms_handle_reset on v2:172.21.15.83:6835/83108157 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4104364209 >> 172.21.15.83:0/4104364209 conn(0x55ac4fef2310 msgr2=0x55ac4ffbe230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:09.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4104364209 shutdown_connections 2023-11-26T07:15:09.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 1 -- 172.21.15.83:0/4104364209 wait complete. 2023-11-26T07:15:09.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12dcdcdb80 20 client.6294 trim_cache size 0 max 16384 2023-11-26T07:15:09.936 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:15:09.936 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "0aae1049-cc8c-4011-a01f-378673e57a96", 2023-11-26T07:15:09.936 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-26T07:15:09.936 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6102, 2023-11-26T07:15:09.936 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-26T07:15:09.936 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-26T07:15:09.936 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-26T07:15:09.936 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 14.38983867, 2023-11-26T07:15:09.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 37, 2023-11-26T07:15:09.937 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 58, 2023-11-26T07:15:09.937 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 58, 2023-11-26T07:15:09.937 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 19.611298304999998 2023-11-26T07:15:09.937 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:15:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12d0bd4700 1 -- 172.21.15.83:0/3332327343 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f12ac06b080 msgr2=0x7f12ac06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12d0bd4700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f12ac06b080 0x7f12ac06d540 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000f60 tx=0x7f12bc009040 comp rx=0 tx=0).stop 2023-11-26T07:15:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12d0bd4700 1 -- 172.21.15.83:0/3332327343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc1446a0 msgr2=0x7f12cc143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12d0bd4700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc1446a0 0x7f12cc143180 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f12c400cef0 tx=0x7f12c400a340 comp rx=0 tx=0).stop 2023-11-26T07:15:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12d0bd4700 1 -- 172.21.15.83:0/3332327343 shutdown_connections 2023-11-26T07:15:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12d0bd4700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc1446a0 0x7f12cc143180 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12d0bd4700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f12ac06b080 0x7f12ac06d540 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12d0bd4700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12cc06c540 0x7f12cc136fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12d0bd4700 1 --2- 172.21.15.83:0/3332327343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12cc0adf00 0x7f12cc15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:09.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.927+0000 7f12d0bd4700 1 -- 172.21.15.83:0/3332327343 >> 172.21.15.83:0/3332327343 conn(0x7f12cc071b90 msgr2=0x7f12cc073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:09.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.928+0000 7f12d0bd4700 1 -- 172.21.15.83:0/3332327343 shutdown_connections 2023-11-26T07:15:09.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:09.928+0000 7f12d0bd4700 1 -- 172.21.15.83:0/3332327343 wait complete. 2023-11-26T07:15:09.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:15:10.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.060+0000 7f611577d700 1 Processor -- start 2023-11-26T07:15:10.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.061+0000 7f611577d700 1 -- start start 2023-11-26T07:15:10.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.061+0000 7f611577d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6110140680 0x7f6110140c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:10.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.061+0000 7f611577d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f61100b3d40 0x7f61100b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.061+0000 7f611577d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611006d550 con 0x7f6110070d60 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.061+0000 7f611577d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61100b47c0 con 0x7f6110140680 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.061+0000 7f611577d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611006cc60 con 0x7f61100b3d40 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.061+0000 7f610effd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6110140680 0x7f6110140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.061+0000 7f610f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6110070d60 0x7f611006d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58634/0 (socket says 172.21.15.83:58634) 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.062+0000 7f610f7fe700 1 -- 172.21.15.83:0/2302071021 learned_addr learned my addr 172.21.15.83:0/2302071021 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.062+0000 7f610e7fc700 1 --2- 172.21.15.83:0/2302071021 >> v2:172.21.15.203:3300/0 conn(0x7f61100b3d40 0x7f61100b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.062+0000 7f610effd700 1 -- 172.21.15.83:0/2302071021 >> v2:172.21.15.203:3300/0 conn(0x7f61100b3d40 msgr2=0x7f61100b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.062+0000 7f610effd700 1 --2- 172.21.15.83:0/2302071021 >> v2:172.21.15.203:3300/0 conn(0x7f61100b3d40 0x7f61100b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.062+0000 7f610effd700 1 -- 172.21.15.83:0/2302071021 >> v1:172.21.15.83:6789/0 conn(0x7f6110070d60 legacy=0x7f611006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.062+0000 7f610dffb700 1 -- 172.21.15.83:0/2302071021 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458678181 0 0) 0x7f611006d550 con 0x7f6110070d60 2023-11-26T07:15:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.062+0000 7f610effd700 1 -- 172.21.15.83:0/2302071021 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611006cf00 con 0x7f6110140680 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.062+0000 7f610effd700 1 --2- 172.21.15.83:0/2302071021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6110140680 0x7f6110140c80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f610000a9e0 tx=0x7f610000acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c416e542e5f2a659 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.062+0000 7f610dffb700 1 -- 172.21.15.83:0/2302071021 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610000cb30 con 0x7f6110140680 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.062+0000 7f610dffb700 1 -- 172.21.15.83:0/2302071021 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610000cc90 con 0x7f6110140680 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.062+0000 7f610dffb700 1 -- 172.21.15.83:0/2302071021 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610000ae80 con 0x7f6110140680 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.063+0000 7f611577d700 1 -- 172.21.15.83:0/2302071021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6110140680 msgr2=0x7f6110140c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.063+0000 7f611577d700 1 --2- 172.21.15.83:0/2302071021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6110140680 0x7f6110140c80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f610000a9e0 tx=0x7f610000acf0 comp rx=0 tx=0).stop 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.063+0000 7f611577d700 1 -- 172.21.15.83:0/2302071021 shutdown_connections 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.063+0000 7f611577d700 1 --2- 172.21.15.83:0/2302071021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6110140680 0x7f6110140c80 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.063+0000 7f611577d700 1 --2- 172.21.15.83:0/2302071021 >> v2:172.21.15.203:3300/0 conn(0x7f61100b3d40 0x7f61100b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.063+0000 7f611577d700 1 -- 172.21.15.83:0/2302071021 >> 172.21.15.83:0/2302071021 conn(0x7f61100b1740 msgr2=0x7f61100b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.063+0000 7f611577d700 1 -- 172.21.15.83:0/2302071021 shutdown_connections 2023-11-26T07:15:10.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.063+0000 7f611577d700 1 -- 172.21.15.83:0/2302071021 wait complete. 2023-11-26T07:15:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.064+0000 7f611577d700 1 Processor -- start 2023-11-26T07:15:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.064+0000 7f611577d700 1 -- start start 2023-11-26T07:15:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f611577d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6110070d60 0x7f61101331e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f611577d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61100b3d40 0x7f6110154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f611577d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6110131760 0x7f6110159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f611577d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61101545b0 con 0x7f61100b3d40 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f611577d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6110132140 con 0x7f6110131760 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f611577d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6110132420 con 0x7f6110070d60 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61100b3d40 0x7f6110154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61100b3d40 0x7f6110154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38598/0 (socket says 172.21.15.83:38598) 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610e7fc700 1 -- 172.21.15.83:0/2731036856 learned_addr learned my addr 172.21.15.83:0/2731036856 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610effd700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6110070d60 0x7f61101331e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610f7fe700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6110131760 0x7f6110159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610e7fc700 1 -- 172.21.15.83:0/2731036856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6110070d60 msgr2=0x7f61101331e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610e7fc700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6110070d60 0x7f61101331e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610e7fc700 1 -- 172.21.15.83:0/2731036856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6110131760 msgr2=0x7f6110159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610e7fc700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6110131760 0x7f6110159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610e7fc700 1 -- 172.21.15.83:0/2731036856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610000a640 con 0x7f61100b3d40 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610f7fe700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6110131760 0x7f6110159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f610e7fc700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61100b3d40 0x7f6110154070 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f610400e870 tx=0x7f610400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.065+0000 7f60f77fe700 1 -- 172.21.15.83:0/2731036856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6104009590 con 0x7f61100b3d40 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.066+0000 7f60f77fe700 1 -- 172.21.15.83:0/2731036856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6104013680 con 0x7f61100b3d40 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.066+0000 7f611577d700 1 -- 172.21.15.83:0/2731036856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6110132700 con 0x7f61100b3d40 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.066+0000 7f60f77fe700 1 -- 172.21.15.83:0/2731036856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61040139f0 con 0x7f61100b3d40 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.066+0000 7f611577d700 1 -- 172.21.15.83:0/2731036856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6110132cc0 con 0x7f61100b3d40 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.066+0000 7f611577d700 1 -- 172.21.15.83:0/2731036856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6110133030 con 0x7f61100b3d40 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.067+0000 7f60f77fe700 1 -- 172.21.15.83:0/2731036856 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6104013b50 con 0x7f61100b3d40 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.067+0000 7f60f77fe700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f60fc06b080 0x7f60fc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.067+0000 7f60f77fe700 1 -- 172.21.15.83:0/2731036856 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61040897b0 con 0x7f61100b3d40 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.067+0000 7f610effd700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f60fc06b080 0x7f60fc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:10.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.068+0000 7f610effd700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f60fc06b080 0x7f60fc06d540 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f611004f460 tx=0x7f6100012000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.072+0000 7f60f77fe700 1 -- 172.21.15.83:0/2731036856 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61040545e0 con 0x7f61100b3d40 2023-11-26T07:15:10.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.308+0000 7f611577d700 1 -- 172.21.15.83:0/2731036856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f61100751e0 con 0x7f61100b3d40 2023-11-26T07:15:10.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.308+0000 7f60f77fe700 1 -- 172.21.15.83:0/2731036856 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6104057c00 con 0x7f61100b3d40 2023-11-26T07:15:10.309 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:15:10.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 -- 172.21.15.83:0/2731036856 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f60fc06b080 msgr2=0x7f60fc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:10.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f60fc06b080 0x7f60fc06d540 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f611004f460 tx=0x7f6100012000 comp rx=0 tx=0).stop 2023-11-26T07:15:10.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 -- 172.21.15.83:0/2731036856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61100b3d40 msgr2=0x7f6110154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:10.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61100b3d40 0x7f6110154070 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f610400e870 tx=0x7f610400eb80 comp rx=0 tx=0).stop 2023-11-26T07:15:10.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 -- 172.21.15.83:0/2731036856 shutdown_connections 2023-11-26T07:15:10.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6110131760 0x7f6110159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:10.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f60fc06b080 0x7f60fc06d540 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:10.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6110070d60 0x7f61101331e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:10.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 --2- 172.21.15.83:0/2731036856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61100b3d40 0x7f6110154070 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:10.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 -- 172.21.15.83:0/2731036856 >> 172.21.15.83:0/2731036856 conn(0x7f61100b1740 msgr2=0x7f61100a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:10.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 -- 172.21.15.83:0/2731036856 shutdown_connections 2023-11-26T07:15:10.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:10.310+0000 7f611577d700 1 -- 172.21.15.83:0/2731036856 wait complete. 2023-11-26T07:15:10.324 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:15:10.324 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:15:10.324 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-26T07:15:10.324 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:15:10.324 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:15:10.324 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:15:10.324 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:15:10.325 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:15:10.325 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:15:10.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:10.325 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-26T07:15:10.342 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:15:10.342 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:15:10.342 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:15:10.342 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:10.342 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-26T07:15:10.343 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:10.343 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:10.343 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:10.343 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:10.343 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:10.343 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:10.343 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:10.343 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:15:10.343 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:10.343 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:15:10.343 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42175sec preferred_lft 42175sec 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::3a41:fab:adc5:5d6c/64 scope link noprefixroute 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::54b0:6eff:fe33:bb15/64 scope link 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether fe:46:b0:72:7b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::fc46:b0ff:fe72:7b3f/64 scope link 2023-11-26T07:15:10.344 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:10.345 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:15:10.345 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:15:10.345 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::7808:fdff:fe1b:770c/64 scope link 2023-11-26T07:15:10.345 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:10.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:10.345 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:15:10.407 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-26T07:15:10.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:10.409 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-26T07:15:10.467 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-26T07:15:10.468 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:15:10.468 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:15:10.468 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:15:10.468 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:15:10.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:10.468 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:15:10.468 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:15:10.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:15:10.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:15:10.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:15:10.469 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:15:10.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:10.573 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:15:10.589 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:15:10.589 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:15:10.589 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:15:10.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:10.590 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-26T07:15:10.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:15:10.656 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:15:10.713 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:15:10.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:15:10.713 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:15:10.769 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:15:10.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:10.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:10.855 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:10.855 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:10.879 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:10.879 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:10.879 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:15:10.879 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:15:10.879 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:15:10.879 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:15:10.879 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:15:10.879 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:15:10.932 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:10.932 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:10.932 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:15:10.932 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:15:10.932 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:15:10.932 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:15:10.932 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:15:10.932 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:15:10.932 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:15:10.932 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:15:10.933 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:15:10.940 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:15:10.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:10.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:15:10.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:10.967 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:15:11.050 INFO:teuthology.orchestra.run.smithi083.stdout:client6306 2023-11-26T07:15:11.050 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:15:11.051 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:11.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:15:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.266+0000 7fbf80cd2700 1 Processor -- start 2023-11-26T07:15:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.266+0000 7fbf80cd2700 1 -- start start 2023-11-26T07:15:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.267+0000 7fbf80cd2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf7c0b3370 0x7fbf7c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.267+0000 7fbf80cd2700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbf7c0d1c70 0x7fbf7c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.267+0000 7fbf80cd2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7c0b22d0 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.267+0000 7fbf80cd2700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c0b2f50 con 0x7fbf7c0d1c70 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.267+0000 7fbf80cd2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0d2640 con 0x7fbf7c0b3370 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.267+0000 7fbf79d9b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf7c0b3370 0x7fbf7c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.267+0000 7fbf7ad9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbf7c0d1c70 0x7fbf7c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.267+0000 7fbf7a59c700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fbf7c15bdd0 0x7fbf7c0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:43266/0 (socket says 172.21.15.83:43266) 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.267+0000 7fbf7a59c700 1 -- 172.21.15.83:0/3392366155 learned_addr learned my addr 172.21.15.83:0/3392366155 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.268+0000 7fbf7959a700 1 -- 172.21.15.83:0/3392366155 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16556734 0 0) 0x7fbf7c0b22d0 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.268+0000 7fbf7959a700 1 -- 172.21.15.83:0/3392366155 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf640035b0 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.268+0000 7fbf79d9b700 1 -- 172.21.15.83:0/3392366155 >> v2:172.21.15.203:3300/0 conn(0x7fbf7c0d1c70 msgr2=0x7fbf7c0d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.268+0000 7fbf79d9b700 1 --2- 172.21.15.83:0/3392366155 >> v2:172.21.15.203:3300/0 conn(0x7fbf7c0d1c70 0x7fbf7c0d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.268+0000 7fbf79d9b700 1 -- 172.21.15.83:0/3392366155 >> v1:172.21.15.203:6789/0 conn(0x7fbf7c15bdd0 legacy=0x7fbf7c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.268+0000 7fbf79d9b700 1 -- 172.21.15.83:0/3392366155 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0b25b0 con 0x7fbf7c0b3370 2023-11-26T07:15:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.268+0000 7fbf7ad9d700 1 --2- 172.21.15.83:0/3392366155 >> v2:172.21.15.203:3300/0 conn(0x7fbf7c0d1c70 0x7fbf7c0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.268+0000 7fbf79d9b700 1 --2- 172.21.15.83:0/3392366155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf7c0b3370 0x7fbf7c1604e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009840 tx=0x7fbf6c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abc26f22a51584d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.268+0000 7fbf7959a700 1 -- 172.21.15.83:0/3392366155 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c011830 con 0x7fbf7c0b3370 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.268+0000 7fbf7959a700 1 -- 172.21.15.83:0/3392366155 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6c011990 con 0x7fbf7c0b3370 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.268+0000 7fbf7959a700 1 -- 172.21.15.83:0/3392366155 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c011d20 con 0x7fbf7c0b3370 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.269+0000 7fbf80cd2700 1 -- 172.21.15.83:0/3392366155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf7c0b3370 msgr2=0x7fbf7c1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.269+0000 7fbf80cd2700 1 --2- 172.21.15.83:0/3392366155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf7c0b3370 0x7fbf7c1604e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009840 tx=0x7fbf6c009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.269+0000 7fbf80cd2700 1 -- 172.21.15.83:0/3392366155 shutdown_connections 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.269+0000 7fbf80cd2700 1 --2- 172.21.15.83:0/3392366155 >> v2:172.21.15.203:3300/0 conn(0x7fbf7c0d1c70 0x7fbf7c0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.269+0000 7fbf80cd2700 1 --2- 172.21.15.83:0/3392366155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf7c0b3370 0x7fbf7c1604e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.269+0000 7fbf80cd2700 1 -- 172.21.15.83:0/3392366155 >> 172.21.15.83:0/3392366155 conn(0x7fbf7c0731a0 msgr2=0x7fbf7c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.269+0000 7fbf80cd2700 1 -- 172.21.15.83:0/3392366155 shutdown_connections 2023-11-26T07:15:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.269+0000 7fbf80cd2700 1 -- 172.21.15.83:0/3392366155 wait complete. 2023-11-26T07:15:11.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.270+0000 7fbf80cd2700 1 Processor -- start 2023-11-26T07:15:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.270+0000 7fbf80cd2700 1 -- start start 2023-11-26T07:15:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf80cd2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf7c0b3370 0x7fbf7c06ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf80cd2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf7c0d1c70 0x7fbf7c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf80cd2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf7c15bdd0 0x7fbf7c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf80cd2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0cf0c0 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf80cd2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0cd060 con 0x7fbf7c0b3370 2023-11-26T07:15:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf80cd2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c06c880 con 0x7fbf7c0d1c70 2023-11-26T07:15:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf7ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf7c15bdd0 0x7fbf7c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf7ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf7c15bdd0 0x7fbf7c0ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38618/0 (socket says 172.21.15.83:38618) 2023-11-26T07:15:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf7ad9d700 1 -- 172.21.15.83:0/2664259455 learned_addr learned my addr 172.21.15.83:0/2664259455 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf7a59c700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf7c0b3370 0x7fbf7c06ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf79d9b700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf7c0d1c70 0x7fbf7c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf7ad9d700 1 -- 172.21.15.83:0/2664259455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf7c0d1c70 msgr2=0x7fbf7c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf7ad9d700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf7c0d1c70 0x7fbf7c0cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf7ad9d700 1 -- 172.21.15.83:0/2664259455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf7c0b3370 msgr2=0x7fbf7c06ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf7ad9d700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf7c0b3370 0x7fbf7c06ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf7ad9d700 1 -- 172.21.15.83:0/2664259455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf74009a70 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.271+0000 7fbf7a59c700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf7c0b3370 0x7fbf7c06ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.272+0000 7fbf7ad9d700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf7c15bdd0 0x7fbf7c0ce020 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400efc0 tx=0x7fbf7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.272+0000 7fbf6affd700 1 -- 172.21.15.83:0/2664259455 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74009d70 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.272+0000 7fbf6affd700 1 -- 172.21.15.83:0/2664259455 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7400f040 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.272+0000 7fbf80cd2700 1 -- 172.21.15.83:0/2664259455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6c0094a0 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.272+0000 7fbf6affd700 1 -- 172.21.15.83:0/2664259455 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7400ce00 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.272+0000 7fbf80cd2700 1 -- 172.21.15.83:0/2664259455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c06ce40 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.273+0000 7fbf80cd2700 1 -- 172.21.15.83:0/2664259455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7c067f60 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.273+0000 7fbf6affd700 1 -- 172.21.15.83:0/2664259455 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fbf74014480 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.274+0000 7fbf6affd700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf5c06b080 0x7fbf5c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.274+0000 7fbf7a59c700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf5c06b080 0x7fbf5c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.274+0000 7fbf6affd700 1 -- 172.21.15.83:0/2664259455 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf7408a1d0 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.274+0000 7fbf7a59c700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf5c06b080 0x7fbf5c06d540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbf700011e0 tx=0x7fbf70009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:11.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.278+0000 7fbf6affd700 1 -- 172.21.15.83:0/2664259455 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf74054f00 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.512+0000 7fbf80cd2700 1 -- 172.21.15.83:0/2664259455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbf7c0b7c60 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.512+0000 7fbf6affd700 1 -- 172.21.15.83:0/2664259455 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbf74058520 con 0x7fbf7c15bdd0 2023-11-26T07:15:11.514 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:15:11.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 -- 172.21.15.83:0/2664259455 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf5c06b080 msgr2=0x7fbf5c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:11.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf5c06b080 0x7fbf5c06d540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbf700011e0 tx=0x7fbf70009040 comp rx=0 tx=0).stop 2023-11-26T07:15:11.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 -- 172.21.15.83:0/2664259455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf7c15bdd0 msgr2=0x7fbf7c0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:11.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf7c15bdd0 0x7fbf7c0ce020 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400efc0 tx=0x7fbf7400c3a0 comp rx=0 tx=0).stop 2023-11-26T07:15:11.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 -- 172.21.15.83:0/2664259455 shutdown_connections 2023-11-26T07:15:11.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf7c15bdd0 0x7fbf7c0ce020 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:11.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf7c0b3370 0x7fbf7c06ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:11.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbf5c06b080 0x7fbf5c06d540 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:11.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 --2- 172.21.15.83:0/2664259455 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf7c0d1c70 0x7fbf7c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:11.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 -- 172.21.15.83:0/2664259455 >> 172.21.15.83:0/2664259455 conn(0x7fbf7c0731a0 msgr2=0x7fbf7c070610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:11.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 -- 172.21.15.83:0/2664259455 shutdown_connections 2023-11-26T07:15:11.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:11.515+0000 7fbf80cd2700 1 -- 172.21.15.83:0/2664259455 wait complete. 2023-11-26T07:15:11.528 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:15:11.528 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:15:11.528 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2023-11-26T07:15:11.528 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:15:11.528 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:15:11.528 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:15:11.529 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:15:11.529 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:15:11.529 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:15:11.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:11.529 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2023-11-26T07:15:11.553 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:15:11.553 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:15:11.553 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:15:11.553 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:11.553 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-11-26T07:15:11.553 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:11.553 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:11.554 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:11.554 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:11.554 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:11.554 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:11.554 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:11.554 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:15:11.554 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:11.554 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:15:11.554 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42178sec preferred_lft 42178sec 2023-11-26T07:15:11.554 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::64b0:e04:3aa8:a78c/64 scope link noprefixroute 2023-11-26T07:15:11.554 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1092:a6ff:fe63:3da7/64 scope link 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::b88a:cff:fe60:2b6f/64 scope link 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:15:11.555 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether de:b3:89:ce:09:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:15:11.556 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::dcb3:89ff:fece:9cd/64 scope link 2023-11-26T07:15:11.556 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:11.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:11.556 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:15:11.626 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-11-26T07:15:11.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:11.627 DEBUG:teuthology.orchestra.run.smithi203:> route 2023-11-26T07:15:11.688 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2023-11-26T07:15:11.688 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:15:11.688 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:15:11.688 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:15:11.688 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:15:11.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:11.688 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:15:11.688 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:15:11.688 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:15:11.688 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:15:11.689 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:15:11.689 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:15:11.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:11.797 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:15:11.813 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:15:11.813 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:15:11.813 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:15:11.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:11.814 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2023-11-26T07:15:11.880 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:15:11.880 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:15:11.936 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:15:11.936 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:15:11.936 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:15:11.993 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:15:11.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:11.993 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:12.080 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:12.080 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:12.107 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:12.107 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:12.107 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:15:12.107 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:15:12.107 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:15:12.107 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:15:12.107 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:15:12.108 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:15:12.138 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:12.138 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:12.139 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:15:12.139 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:15:12.139 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:15:12.139 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:15:12.139 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:15:12.139 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:15:12.139 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:15:12.139 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:15:12.139 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:15:12.154 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:15:12.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:12.154 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:15:12.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:12.184 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:15:12.256 INFO:teuthology.orchestra.run.smithi203.stdout:client5791 2023-11-26T07:15:12.256 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:15:12.257 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:12.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:15:12.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.442+0000 7ff9093d6700 1 Processor -- start 2023-11-26T07:15:12.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.442+0000 7ff9093d6700 1 -- start start 2023-11-26T07:15:12.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff9093d6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff90415dfe0 0x7ff9040d3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:12.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff9093d6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff9040d39a0 0x7ff9040b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:12.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff9093d6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9040d1e60 con 0x7ff90415e3c0 2023-11-26T07:15:12.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff9093d6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9040d2ae0 con 0x7ff9040d39a0 2023-11-26T07:15:12.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff9093d6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9040d2140 con 0x7ff90415dfe0 2023-11-26T07:15:12.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff9027fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff9040d39a0 0x7ff9040b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff9037fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff90415e3c0 0x7ff90406aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58662/0 (socket says 172.21.15.83:58662) 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff9037fe700 1 -- 172.21.15.83:0/1581159308 learned_addr learned my addr 172.21.15.83:0/1581159308 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff902ffd700 1 --2- 172.21.15.83:0/1581159308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff90415dfe0 0x7ff9040d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff9027fc700 1 -- 172.21.15.83:0/1581159308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff90415dfe0 msgr2=0x7ff9040d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff9027fc700 1 --2- 172.21.15.83:0/1581159308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff90415dfe0 0x7ff9040d3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff9027fc700 1 -- 172.21.15.83:0/1581159308 >> v1:172.21.15.83:6789/0 conn(0x7ff90415e3c0 legacy=0x7ff90406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff9027fc700 1 -- 172.21.15.83:0/1581159308 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9040d2420 con 0x7ff9040d39a0 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.443+0000 7ff901ffb700 1 -- 172.21.15.83:0/1581159308 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914164467 0 0) 0x7ff9040d1e60 con 0x7ff90415e3c0 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff9027fc700 1 --2- 172.21.15.83:0/1581159308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9040d39a0 0x7ff9040b5930 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009840 tx=0x7ff8f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45847e44e528253 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff901ffb700 1 -- 172.21.15.83:0/1581159308 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4011830 con 0x7ff9040d39a0 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff901ffb700 1 -- 172.21.15.83:0/1581159308 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f4011990 con 0x7ff9040d39a0 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff901ffb700 1 -- 172.21.15.83:0/1581159308 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4011d20 con 0x7ff9040d39a0 2023-11-26T07:15:12.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff9093d6700 1 -- 172.21.15.83:0/1581159308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9040d39a0 msgr2=0x7ff9040b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:12.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff9093d6700 1 --2- 172.21.15.83:0/1581159308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9040d39a0 0x7ff9040b5930 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009840 tx=0x7ff8f4009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:12.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff9093d6700 1 -- 172.21.15.83:0/1581159308 shutdown_connections 2023-11-26T07:15:12.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff9093d6700 1 --2- 172.21.15.83:0/1581159308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff90415dfe0 0x7ff9040d3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:12.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff9093d6700 1 --2- 172.21.15.83:0/1581159308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9040d39a0 0x7ff9040b5930 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:12.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff9093d6700 1 -- 172.21.15.83:0/1581159308 >> 172.21.15.83:0/1581159308 conn(0x7ff9040d0b10 msgr2=0x7ff9040d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:12.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff9093d6700 1 -- 172.21.15.83:0/1581159308 shutdown_connections 2023-11-26T07:15:12.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.444+0000 7ff9093d6700 1 -- 172.21.15.83:0/1581159308 wait complete. 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.446+0000 7ff9093d6700 1 Processor -- start 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.446+0000 7ff9093d6700 1 -- start start 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.446+0000 7ff9093d6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9040d39a0 0x7ff9041f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.446+0000 7ff9093d6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90415dfe0 0x7ff9040cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.446+0000 7ff9093d6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff90415e3c0 0x7ff9040b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.446+0000 7ff9093d6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9040caf80 con 0x7ff90415dfe0 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.446+0000 7ff9093d6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff904158400 con 0x7ff9040d39a0 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.446+0000 7ff9093d6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9041f46c0 con 0x7ff90415e3c0 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9027fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90415dfe0 0x7ff9040cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9027fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90415dfe0 0x7ff9040cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38632/0 (socket says 172.21.15.83:38632) 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9027fc700 1 -- 172.21.15.83:0/2209623124 learned_addr learned my addr 172.21.15.83:0/2209623124 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:12.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9037fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff90415e3c0 0x7ff9040b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff902ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9040d39a0 0x7ff9041f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9027fc700 1 -- 172.21.15.83:0/2209623124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff90415e3c0 msgr2=0x7ff9040b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9027fc700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff90415e3c0 0x7ff9040b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9027fc700 1 -- 172.21.15.83:0/2209623124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9040d39a0 msgr2=0x7ff9041f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9027fc700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9040d39a0 0x7ff9041f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9027fc700 1 -- 172.21.15.83:0/2209623124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f40094a0 con 0x7ff90415dfe0 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9027fc700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90415dfe0 0x7ff9040cdcf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40090b0 tx=0x7ff8f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff8eb7fe700 1 -- 172.21.15.83:0/2209623124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40045d0 con 0x7ff90415dfe0 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff8eb7fe700 1 -- 172.21.15.83:0/2209623124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f401e070 con 0x7ff90415dfe0 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff8eb7fe700 1 -- 172.21.15.83:0/2209623124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40038c0 con 0x7ff90415dfe0 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9093d6700 1 -- 172.21.15.83:0/2209623124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9041f49a0 con 0x7ff90415dfe0 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.447+0000 7ff9093d6700 1 -- 172.21.15.83:0/2209623124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff904158090 con 0x7ff90415dfe0 2023-11-26T07:15:12.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.448+0000 7ff9093d6700 1 -- 172.21.15.83:0/2209623124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff904158590 con 0x7ff90415dfe0 2023-11-26T07:15:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.449+0000 7ff8eb7fe700 1 -- 172.21.15.83:0/2209623124 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7ff8f4003a20 con 0x7ff90415dfe0 2023-11-26T07:15:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.449+0000 7ff8eb7fe700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8f006b080 0x7ff8f006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.449+0000 7ff8eb7fe700 1 -- 172.21.15.83:0/2209623124 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8f408a8d0 con 0x7ff90415dfe0 2023-11-26T07:15:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.449+0000 7ff902ffd700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8f006b080 0x7ff8f006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:12.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.449+0000 7ff902ffd700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8f006b080 0x7ff8f006d540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8006b20 tx=0x7ff8f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:12.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.453+0000 7ff8eb7fe700 1 -- 172.21.15.83:0/2209623124 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff8f4055700 con 0x7ff90415dfe0 2023-11-26T07:15:12.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:12.691+0000 7ff9093d6700 1 -- 172.21.15.83:0/2209623124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff9040751e0 con 0x7ff90415dfe0 2023-11-26T07:15:13.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.614+0000 7ff8eb7fe700 1 -- 172.21.15.83:0/2209623124 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff8f40150c0 con 0x7ff90415dfe0 2023-11-26T07:15:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.616+0000 7ff9093d6700 1 -- 172.21.15.83:0/2209623124 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8f006b080 msgr2=0x7ff8f006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:13.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.616+0000 7ff9093d6700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8f006b080 0x7ff8f006d540 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8006b20 tx=0x7ff8f8008040 comp rx=0 tx=0).stop 2023-11-26T07:15:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.616+0000 7ff9093d6700 1 -- 172.21.15.83:0/2209623124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90415dfe0 msgr2=0x7ff9040cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.616+0000 7ff9093d6700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90415dfe0 0x7ff9040cdcf0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40090b0 tx=0x7ff8f4004750 comp rx=0 tx=0).stop 2023-11-26T07:15:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.617+0000 7ff9093d6700 1 -- 172.21.15.83:0/2209623124 shutdown_connections 2023-11-26T07:15:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.617+0000 7ff9093d6700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8f006b080 0x7ff8f006d540 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.617+0000 7ff9093d6700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff90415e3c0 0x7ff9040b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.617+0000 7ff9093d6700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9040d39a0 0x7ff9041f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.617+0000 7ff9093d6700 1 --2- 172.21.15.83:0/2209623124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90415dfe0 0x7ff9040cdcf0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.617+0000 7ff9093d6700 1 -- 172.21.15.83:0/2209623124 >> 172.21.15.83:0/2209623124 conn(0x7ff9040d0b10 msgr2=0x7ff9040b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.617+0000 7ff9093d6700 1 -- 172.21.15.83:0/2209623124 shutdown_connections 2023-11-26T07:15:13.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.617+0000 7ff9093d6700 1 -- 172.21.15.83:0/2209623124 wait complete. 2023-11-26T07:15:13.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:15:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.758+0000 7eff6029b700 1 Processor -- start 2023-11-26T07:15:13.761 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.759+0000 7eff6029b700 1 -- start start 2023-11-26T07:15:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff6029b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff5815c1e0 0x7eff580cc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff6029b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7eff580d0040 0x7eff580d0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff6029b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff580d09d0 con 0x7eff5815be00 2023-11-26T07:15:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff6029b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff580b00e0 con 0x7eff5815c1e0 2023-11-26T07:15:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff6029b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff580af200 con 0x7eff580d0040 2023-11-26T07:15:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5d836700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff5815c1e0 0x7eff580cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5d836700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff5815c1e0 0x7eff580cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38638/0 (socket says 172.21.15.83:38638) 2023-11-26T07:15:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5d836700 1 -- 172.21.15.83:0/392176708 learned_addr learned my addr 172.21.15.83:0/392176708 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5e838700 1 --2- 172.21.15.83:0/392176708 >> v2:172.21.15.203:3300/0 conn(0x7eff580d0040 0x7eff580d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5d836700 1 -- 172.21.15.83:0/392176708 >> v2:172.21.15.203:3300/0 conn(0x7eff580d0040 msgr2=0x7eff580d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5d836700 1 --2- 172.21.15.83:0/392176708 >> v2:172.21.15.203:3300/0 conn(0x7eff580d0040 0x7eff580d0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5d836700 1 -- 172.21.15.83:0/392176708 >> v1:172.21.15.203:6789/0 conn(0x7eff5815be00 legacy=0x7eff580aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5d836700 1 -- 172.21.15.83:0/392176708 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff580af4d0 con 0x7eff5815c1e0 2023-11-26T07:15:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5d836700 1 --2- 172.21.15.83:0/392176708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff5815c1e0 0x7eff580cc660 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7eff580b0050 tx=0x7eff48009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72a65bca454ef4d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5d035700 1 -- 172.21.15.83:0/392176708 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48004230 con 0x7eff5815c1e0 2023-11-26T07:15:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5d035700 1 -- 172.21.15.83:0/392176708 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff48004390 con 0x7eff5815c1e0 2023-11-26T07:15:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.760+0000 7eff5d035700 1 -- 172.21.15.83:0/392176708 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff480036a0 con 0x7eff5815c1e0 2023-11-26T07:15:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.761+0000 7eff6029b700 1 -- 172.21.15.83:0/392176708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff5815c1e0 msgr2=0x7eff580cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.761+0000 7eff6029b700 1 --2- 172.21.15.83:0/392176708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff5815c1e0 0x7eff580cc660 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7eff580b0050 tx=0x7eff48009bd0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.761+0000 7eff6029b700 1 -- 172.21.15.83:0/392176708 shutdown_connections 2023-11-26T07:15:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.761+0000 7eff6029b700 1 --2- 172.21.15.83:0/392176708 >> v2:172.21.15.203:3300/0 conn(0x7eff580d0040 0x7eff580d0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.761+0000 7eff6029b700 1 --2- 172.21.15.83:0/392176708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff5815c1e0 0x7eff580cc660 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.761+0000 7eff6029b700 1 -- 172.21.15.83:0/392176708 >> 172.21.15.83:0/392176708 conn(0x7eff5809aa00 msgr2=0x7eff5809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.761+0000 7eff6029b700 1 -- 172.21.15.83:0/392176708 shutdown_connections 2023-11-26T07:15:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.761+0000 7eff6029b700 1 -- 172.21.15.83:0/392176708 wait complete. 2023-11-26T07:15:13.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.762+0000 7eff6029b700 1 Processor -- start 2023-11-26T07:15:13.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.762+0000 7eff6029b700 1 -- start start 2023-11-26T07:15:13.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff6029b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff580d0040 0x7eff580cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:13.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff6029b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff5815be00 0x7eff580cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:13.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff6029b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5815c1e0 0x7eff580c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:13.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff6029b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5815d720 con 0x7eff5815be00 2023-11-26T07:15:13.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff6029b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff580cbad0 con 0x7eff5815c1e0 2023-11-26T07:15:13.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff6029b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5806ca10 con 0x7eff580d0040 2023-11-26T07:15:13.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5d836700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff5815be00 0x7eff580cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5d836700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff5815be00 0x7eff580cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38644/0 (socket says 172.21.15.83:38644) 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5d836700 1 -- 172.21.15.83:0/2635505664 learned_addr learned my addr 172.21.15.83:0/2635505664 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5e838700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5815c1e0 0x7eff580c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5e037700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff580d0040 0x7eff580cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5d836700 1 -- 172.21.15.83:0/2635505664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff580d0040 msgr2=0x7eff580cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5d836700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff580d0040 0x7eff580cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5d836700 1 -- 172.21.15.83:0/2635505664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5815c1e0 msgr2=0x7eff580c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5d836700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5815c1e0 0x7eff580c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5d836700 1 -- 172.21.15.83:0/2635505664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff48009550 con 0x7eff5815be00 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5e037700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff580d0040 0x7eff580cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff5d836700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff5815be00 0x7eff580cb590 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7eff48009130 tx=0x7eff48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:13.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.763+0000 7eff4effd700 1 -- 172.21.15.83:0/2635505664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48011630 con 0x7eff5815be00 2023-11-26T07:15:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.764+0000 7eff4effd700 1 -- 172.21.15.83:0/2635505664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff48011790 con 0x7eff5815be00 2023-11-26T07:15:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.764+0000 7eff6029b700 1 -- 172.21.15.83:0/2635505664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5806c730 con 0x7eff5815be00 2023-11-26T07:15:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.764+0000 7eff4effd700 1 -- 172.21.15.83:0/2635505664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48011b40 con 0x7eff5815be00 2023-11-26T07:15:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.764+0000 7eff6029b700 1 -- 172.21.15.83:0/2635505664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5815d340 con 0x7eff5815be00 2023-11-26T07:15:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.765+0000 7eff6029b700 1 -- 172.21.15.83:0/2635505664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff58067f60 con 0x7eff5815be00 2023-11-26T07:15:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.765+0000 7eff4effd700 1 -- 172.21.15.83:0/2635505664 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7eff48011ca0 con 0x7eff5815be00 2023-11-26T07:15:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.765+0000 7eff4effd700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff3c06b0b0 0x7eff3c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.766+0000 7eff5e037700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff3c06b0b0 0x7eff3c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:13.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.766+0000 7eff4effd700 1 -- 172.21.15.83:0/2635505664 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff4808ac70 con 0x7eff5815be00 2023-11-26T07:15:13.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.766+0000 7eff5e037700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff3c06b0b0 0x7eff3c06d570 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7eff54003f40 tx=0x7eff54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:13.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.770+0000 7eff4effd700 1 -- 172.21.15.83:0/2635505664 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff48055be0 con 0x7eff5815be00 2023-11-26T07:15:13.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.990+0000 7eff6029b700 1 -- 172.21.15.83:0/2635505664 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7eff580afd70 con 0x7eff3c06b0b0 2023-11-26T07:15:13.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.991+0000 7eff4effd700 1 -- 172.21.15.83:0/2635505664 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7eff54006860 con 0x7eff3c06b0b0 2023-11-26T07:15:13.994 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:15:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephfs" 2023-11-26T07:15:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:13.994 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:15:13.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.993+0000 7eff6029b700 1 -- 172.21.15.83:0/2635505664 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff3c06b0b0 msgr2=0x7eff3c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:13.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.993+0000 7eff6029b700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff3c06b0b0 0x7eff3c06d570 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7eff54003f40 tx=0x7eff54009040 comp rx=0 tx=0).stop 2023-11-26T07:15:13.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.993+0000 7eff6029b700 1 -- 172.21.15.83:0/2635505664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff5815be00 msgr2=0x7eff580cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:13.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.993+0000 7eff6029b700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff5815be00 0x7eff580cb590 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7eff48009130 tx=0x7eff48004750 comp rx=0 tx=0).stop 2023-11-26T07:15:13.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.993+0000 7eff6029b700 1 -- 172.21.15.83:0/2635505664 shutdown_connections 2023-11-26T07:15:13.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.993+0000 7eff6029b700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff5815c1e0 0x7eff580c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.994+0000 7eff6029b700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff3c06b0b0 0x7eff3c06d570 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.994+0000 7eff6029b700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff580d0040 0x7eff580cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.994+0000 7eff6029b700 1 --2- 172.21.15.83:0/2635505664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff5815be00 0x7eff580cb590 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:13.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.994+0000 7eff6029b700 1 -- 172.21.15.83:0/2635505664 >> 172.21.15.83:0/2635505664 conn(0x7eff5809aa00 msgr2=0x7eff580aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:13.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.994+0000 7eff6029b700 1 -- 172.21.15.83:0/2635505664 shutdown_connections 2023-11-26T07:15:13.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:13.994+0000 7eff6029b700 1 -- 172.21.15.83:0/2635505664 wait complete. 2023-11-26T07:15:14.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:15:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.128+0000 7f0790d47700 1 Processor -- start 2023-11-26T07:15:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.128+0000 7f0790d47700 1 -- start start 2023-11-26T07:15:14.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f0790d47700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f078c0b5da0 0x7f078c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f0790d47700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078c0b3c60 con 0x7f078c15e160 2023-11-26T07:15:14.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f0790d47700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078c0b58c0 con 0x7f078c0746a0 2023-11-26T07:15:14.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f0790d47700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078c0b4500 con 0x7f078c0b5da0 2023-11-26T07:15:14.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078a59c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f078c15e160 0x7f078c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58690/0 (socket says 172.21.15.83:58690) 2023-11-26T07:15:14.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078a59c700 1 -- 172.21.15.83:0/3186247761 learned_addr learned my addr 172.21.15.83:0/3186247761 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:14.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902722475 0 0) 0x7f078c0b3c60 con 0x7f078c15e160 2023-11-26T07:15:14.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07740035b0 con 0x7f078c15e160 2023-11-26T07:15:14.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078ad9d700 1 --2- 172.21.15.83:0/3186247761 >> v2:172.21.15.203:3300/0 conn(0x7f078c0b5da0 0x7f078c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3581390098 0 0) 0x7f07740035b0 con 0x7f078c15e160 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 >> v2:172.21.15.203:3300/0 conn(0x7f078c0b5da0 msgr2=0x7f078c0afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 --2- 172.21.15.83:0/3186247761 >> v2:172.21.15.203:3300/0 conn(0x7f078c0b5da0 0x7f078c0afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 >> v1:172.21.15.203:6789/0 conn(0x7f078c0746a0 legacy=0x7f078c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c06bdc0 con 0x7f078c15e160 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f077c001670 con 0x7f078c15e160 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 >> v1:172.21.15.83:6789/0 conn(0x7f078c15e160 legacy=0x7f078c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774003a00 0x7f0774003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0774004850 0x7f0774004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0774005210 0x7f077400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c0b4500 con 0x7f0774003a00 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077400c0b0 con 0x7f0774004850 2023-11-26T07:15:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077400c270 con 0x7f0774005210 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.129+0000 7f078a59c700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774003a00 0x7f0774003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f0789d9b700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0774004850 0x7f0774004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f078ad9d700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0774005210 0x7f077400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f078a59c700 1 -- 172.21.15.83:0/3186247761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0774005210 msgr2=0x7f077400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f078a59c700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0774005210 0x7f077400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f078a59c700 1 -- 172.21.15.83:0/3186247761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0774004850 msgr2=0x7f0774004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f078a59c700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0774004850 0x7f0774004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f078ad9d700 1 -- 172.21.15.83:0/3186247761 reap_dead start 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f078a59c700 1 -- 172.21.15.83:0/3186247761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07740035b0 con 0x7f0774003a00 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f0789d9b700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0774004850 0x7f0774004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f078a59c700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774003a00 0x7f0774003e20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f077c00c2a0 tx=0x7f0774005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a26671335d9229d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c007c10 con 0x7f0774003a00 2023-11-26T07:15:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c00cd60 con 0x7f0774003a00 2023-11-26T07:15:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f078959a700 1 -- 172.21.15.83:0/3186247761 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c005ba0 con 0x7f0774003a00 2023-11-26T07:15:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f0790d47700 1 -- 172.21.15.83:0/3186247761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774003a00 msgr2=0x7f0774003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.130+0000 7f0790d47700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774003a00 0x7f0774003e20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f077c00c2a0 tx=0x7f0774005880 comp rx=0 tx=0).stop 2023-11-26T07:15:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.131+0000 7f0790d47700 1 -- 172.21.15.83:0/3186247761 shutdown_connections 2023-11-26T07:15:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.131+0000 7f0790d47700 1 --2- 172.21.15.83:0/3186247761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774003a00 0x7f0774003e20 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.131+0000 7f0790d47700 1 -- 172.21.15.83:0/3186247761 >> 172.21.15.83:0/3186247761 conn(0x7f078c0d0bf0 msgr2=0x7f078c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.131+0000 7f0790d47700 1 -- 172.21.15.83:0/3186247761 shutdown_connections 2023-11-26T07:15:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.131+0000 7f0790d47700 1 -- 172.21.15.83:0/3186247761 wait complete. 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.132+0000 7f0790d47700 1 Processor -- start 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.132+0000 7f0790d47700 1 -- start start 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f0790d47700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0774003a00 0x7f078c15e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f0790d47700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f078c158840 0x7f078c14c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f0790d47700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f078c159130 0x7f078c1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f0790d47700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c159b20 con 0x7f078c159130 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f0790d47700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c15f990 con 0x7f078c158840 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f0790d47700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c14ce90 con 0x7f0774003a00 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f078ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f078c159130 0x7f078c1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f078ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f078c159130 0x7f078c1595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38666/0 (socket says 172.21.15.83:38666) 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f078ad9d700 1 -- 172.21.15.83:0/3737433090 learned_addr learned my addr 172.21.15.83:0/3737433090 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f0789d9b700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f078c158840 0x7f078c14c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f078a59c700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0774003a00 0x7f078c15e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f078ad9d700 1 -- 172.21.15.83:0/3737433090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0774003a00 msgr2=0x7f078c15e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f078ad9d700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0774003a00 0x7f078c15e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f078ad9d700 1 -- 172.21.15.83:0/3737433090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f078c158840 msgr2=0x7f078c14c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f078ad9d700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f078c158840 0x7f078c14c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f078ad9d700 1 -- 172.21.15.83:0/3737433090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c15ed80 con 0x7f078c159130 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f078ad9d700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f078c159130 0x7f078c1595e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f078c075050 tx=0x7f07840114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.133+0000 7f077affd700 1 -- 172.21.15.83:0/3737433090 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400e070 con 0x7f078c159130 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.134+0000 7f077affd700 1 -- 172.21.15.83:0/3737433090 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0784003c30 con 0x7f078c159130 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.134+0000 7f0790d47700 1 -- 172.21.15.83:0/3737433090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c15f060 con 0x7f078c159130 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.134+0000 7f077affd700 1 -- 172.21.15.83:0/3737433090 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784003e20 con 0x7f078c159130 2023-11-26T07:15:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.134+0000 7f0790d47700 1 -- 172.21.15.83:0/3737433090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c15f620 con 0x7f078c159130 2023-11-26T07:15:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.135+0000 7f0790d47700 1 -- 172.21.15.83:0/3737433090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c15fb40 con 0x7f078c159130 2023-11-26T07:15:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.135+0000 7f077affd700 1 -- 172.21.15.83:0/3737433090 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f0784011a10 con 0x7f078c159130 2023-11-26T07:15:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.135+0000 7f077affd700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f076c06b080 0x7f076c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.135+0000 7f078a59c700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f076c06b080 0x7f076c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.135+0000 7f077affd700 1 -- 172.21.15.83:0/3737433090 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0784089cb0 con 0x7f078c159130 2023-11-26T07:15:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.136+0000 7f078a59c700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f076c06b080 0x7f076c06d540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f077c000c00 tx=0x7f077c007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.140+0000 7f077affd700 1 -- 172.21.15.83:0/3737433090 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f078c15fb40 con 0x7f078c159130 2023-11-26T07:15:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.329+0000 7f0790d47700 1 -- 172.21.15.83:0/3737433090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f078c15fb00 con 0x7f078c159130 2023-11-26T07:15:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.329+0000 7f077affd700 1 -- 172.21.15.83:0/3737433090 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0784054a60 con 0x7f078c159130 2023-11-26T07:15:14.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.331+0000 7f0790d47700 1 -- 172.21.15.83:0/3737433090 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f076c06b080 msgr2=0x7f076c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.331+0000 7f0790d47700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f076c06b080 0x7f076c06d540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f077c000c00 tx=0x7f077c007000 comp rx=0 tx=0).stop 2023-11-26T07:15:14.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.331+0000 7f0790d47700 1 -- 172.21.15.83:0/3737433090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f078c159130 msgr2=0x7f078c1595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.331+0000 7f0790d47700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f078c159130 0x7f078c1595e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f078c075050 tx=0x7f07840114b0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.331+0000 7f0790d47700 1 -- 172.21.15.83:0/3737433090 shutdown_connections 2023-11-26T07:15:14.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.331+0000 7f0790d47700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f078c159130 0x7f078c1595e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.331+0000 7f0790d47700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f076c06b080 0x7f076c06d540 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.331+0000 7f0790d47700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0774003a00 0x7f078c15e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.331+0000 7f0790d47700 1 --2- 172.21.15.83:0/3737433090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f078c158840 0x7f078c14c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.331+0000 7f0790d47700 1 -- 172.21.15.83:0/3737433090 >> 172.21.15.83:0/3737433090 conn(0x7f078c0d0bf0 msgr2=0x7f078c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:14.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.332+0000 7f0790d47700 1 -- 172.21.15.83:0/3737433090 shutdown_connections 2023-11-26T07:15:14.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.332+0000 7f0790d47700 1 -- 172.21.15.83:0/3737433090 wait complete. 2023-11-26T07:15:14.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:14.346 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol86790) 2023-11-26T07:15:14.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:14.379 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-11-26T07:15:14.436 INFO:teuthology.orchestra.run.smithi083.stdout:cephfs_data 2023-11-26T07:15:14.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:14.437 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol86790) 2023-11-26T07:15:14.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol95121 2023-11-26T07:15:14.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.670+0000 7f10fb549700 1 Processor -- start 2023-11-26T07:15:14.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.670+0000 7f10fb549700 1 -- start start 2023-11-26T07:15:14.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.671+0000 7f10fb549700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10f40cccd0 0x7f10f40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.676 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.671+0000 7f10fb549700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f406fd30 con 0x7f10f415c320 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.671+0000 7f10fb549700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f406f770 con 0x7f10f415bf40 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.671+0000 7f10fb549700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f40cd6a0 con 0x7f10f40cccd0 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.671+0000 7f10f9ae6700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f10f415c320 0x7f10f415c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58712/0 (socket says 172.21.15.83:58712) 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.671+0000 7f10f9ae6700 1 -- 172.21.15.83:0/3361389378 learned_addr learned my addr 172.21.15.83:0/3361389378 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.671+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865266221 0 0) 0x7f10f406fd30 con 0x7f10f415c320 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e00035b0 con 0x7f10f415c320 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10f8ae4700 1 --2- 172.21.15.83:0/3361389378 >> v2:172.21.15.203:3300/0 conn(0x7f10f40cccd0 0x7f10f40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2161509252 0 0) 0x7f10e00035b0 con 0x7f10f415c320 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 >> v2:172.21.15.203:3300/0 conn(0x7f10f40cccd0 msgr2=0x7f10f40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 --2- 172.21.15.83:0/3361389378 >> v2:172.21.15.203:3300/0 conn(0x7f10f40cccd0 0x7f10f40cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 >> v1:172.21.15.203:6789/0 conn(0x7f10f415bf40 legacy=0x7f10f40af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f406fa50 con 0x7f10f415c320 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f10ec0019f0 con 0x7f10f415c320 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 >> v1:172.21.15.83:6789/0 conn(0x7f10f415c320 legacy=0x7f10f415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e0003a00 0x7f10e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10e0004850 0x7f10e0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e0005210 0x7f10e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f40cd6a0 con 0x7f10e0004850 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e000c0b0 con 0x7f10e0003a00 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e000c270 con 0x7f10e0005210 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10f8ae4700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10e0004850 0x7f10e0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10f92e5700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e0003a00 0x7f10e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10f9ae6700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e0005210 0x7f10e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10f8ae4700 1 -- 172.21.15.83:0/3361389378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e0005210 msgr2=0x7f10e000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10f8ae4700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e0005210 0x7f10e000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10f8ae4700 1 -- 172.21.15.83:0/3361389378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e0003a00 msgr2=0x7f10e0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10f8ae4700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e0003a00 0x7f10e0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10f8ae4700 1 -- 172.21.15.83:0/3361389378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e00035b0 con 0x7f10e0004850 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10f9ae6700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e0005210 0x7f10e000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.672+0000 7f10f9ae6700 1 -- 172.21.15.83:0/3361389378 reap_dead start 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.673+0000 7f10f8ae4700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10e0004850 0x7f10e0004cd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009b60 tx=0x7f10e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e39e07bbabd46b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.673+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f000e6c0 con 0x7f10e0004850 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.673+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f000e820 con 0x7f10e0004850 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.673+0000 7f10ebfff700 1 -- 172.21.15.83:0/3361389378 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f000eb90 con 0x7f10e0004850 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.673+0000 7f10fb549700 1 -- 172.21.15.83:0/3361389378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10e0004850 msgr2=0x7f10e0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.673+0000 7f10fb549700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10e0004850 0x7f10e0004cd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009b60 tx=0x7f10e0005880 comp rx=0 tx=0).stop 2023-11-26T07:15:14.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.673+0000 7f10fb549700 1 -- 172.21.15.83:0/3361389378 shutdown_connections 2023-11-26T07:15:14.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.673+0000 7f10fb549700 1 --2- 172.21.15.83:0/3361389378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10e0004850 0x7f10e0004cd0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.673+0000 7f10fb549700 1 -- 172.21.15.83:0/3361389378 >> 172.21.15.83:0/3361389378 conn(0x7f10f40b2050 msgr2=0x7f10f40b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:14.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.673+0000 7f10fb549700 1 -- 172.21.15.83:0/3361389378 shutdown_connections 2023-11-26T07:15:14.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.673+0000 7f10fb549700 1 -- 172.21.15.83:0/3361389378 wait complete. 2023-11-26T07:15:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.674+0000 7f10fb549700 1 Processor -- start 2023-11-26T07:15:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.675+0000 7f10fb549700 1 -- start start 2023-11-26T07:15:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.675+0000 7f10fb549700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e0004850 0x7f10f4157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.675+0000 7f10fb549700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10f4159510 0x7f10f4159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.675+0000 7f10fb549700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f4153a40 0x7f10f4153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.675+0000 7f10fb549700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f413eb20 con 0x7f10f4159510 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.675+0000 7f10fb549700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f4144f00 con 0x7f10f4153a40 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.675+0000 7f10fb549700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f4159ed0 con 0x7f10e0004850 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.675+0000 7f10f8ae4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10f4159510 0x7f10f4159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.675+0000 7f10f8ae4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10f4159510 0x7f10f4159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38684/0 (socket says 172.21.15.83:38684) 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.675+0000 7f10f8ae4700 1 -- 172.21.15.83:0/1545836830 learned_addr learned my addr 172.21.15.83:0/1545836830 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.675+0000 7f10f92e5700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e0004850 0x7f10f4157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10f9ae6700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f4153a40 0x7f10f4153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10f8ae4700 1 -- 172.21.15.83:0/1545836830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e0004850 msgr2=0x7f10f4157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10f8ae4700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e0004850 0x7f10f4157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10f8ae4700 1 -- 172.21.15.83:0/1545836830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f4153a40 msgr2=0x7f10f4153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10f8ae4700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f4153a40 0x7f10f4153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10f8ae4700 1 -- 172.21.15.83:0/1545836830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f4154400 con 0x7f10f4159510 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10f8ae4700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10f4159510 0x7f10f4159990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f10e000c080 tx=0x7f10f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10e9ffb700 1 -- 172.21.15.83:0/1545836830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f000e6c0 con 0x7f10f4159510 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10e9ffb700 1 -- 172.21.15.83:0/1545836830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f0003ad0 con 0x7f10f4159510 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10fb549700 1 -- 172.21.15.83:0/1545836830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f4154650 con 0x7f10f4159510 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10e9ffb700 1 -- 172.21.15.83:0/1545836830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0003e80 con 0x7f10f4159510 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.676+0000 7f10fb549700 1 -- 172.21.15.83:0/1545836830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f4154c10 con 0x7f10f4159510 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.677+0000 7f10fb549700 1 -- 172.21.15.83:0/1545836830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f4068d90 con 0x7f10f4159510 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.678+0000 7f10e9ffb700 1 -- 172.21.15.83:0/1545836830 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f10f0013470 con 0x7f10f4159510 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.678+0000 7f10e9ffb700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10d806b080 0x7f10d806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.678+0000 7f10f92e5700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10d806b080 0x7f10d806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.678+0000 7f10e9ffb700 1 -- 172.21.15.83:0/1545836830 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10f008b150 con 0x7f10f4159510 2023-11-26T07:15:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.679+0000 7f10f92e5700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10d806b080 0x7f10d806d540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f10e4000c00 tx=0x7f10e4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.683+0000 7f10e9ffb700 1 -- 172.21.15.83:0/1545836830 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f10f0056130 con 0x7f10f4159510 2023-11-26T07:15:14.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.903+0000 7f10fb549700 1 -- 172.21.15.83:0/1545836830 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol95121", "target": ["mon-mgr", ""]}) v1 -- 0x7f10f40b4770 con 0x7f10d806b080 2023-11-26T07:15:14.909 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:14.907+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:14.909 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:14.907+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:14.910 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:14.907+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:14.910 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:14.907+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:14.910 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:14.907+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.938+0000 7f10e9ffb700 1 -- 172.21.15.83:0/1545836830 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f10e4001590 con 0x7f10d806b080 2023-11-26T07:15:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 -- 172.21.15.83:0/1545836830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10d806b080 msgr2=0x7f10d806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10d806b080 0x7f10d806d540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f10e4000c00 tx=0x7f10e4009040 comp rx=0 tx=0).stop 2023-11-26T07:15:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 -- 172.21.15.83:0/1545836830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10f4159510 msgr2=0x7f10f4159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10f4159510 0x7f10f4159990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f10e000c080 tx=0x7f10f00047c0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 -- 172.21.15.83:0/1545836830 shutdown_connections 2023-11-26T07:15:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f4153a40 0x7f10f4153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f10d806b080 0x7f10d806d540 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e0004850 0x7f10f4157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 --2- 172.21.15.83:0/1545836830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10f4159510 0x7f10f4159990 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 -- 172.21.15.83:0/1545836830 >> 172.21.15.83:0/1545836830 conn(0x7f10f40b2050 msgr2=0x7f10f40b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 -- 172.21.15.83:0/1545836830 shutdown_connections 2023-11-26T07:15:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:14.940+0000 7f10fb549700 1 -- 172.21.15.83:0/1545836830 wait complete. 2023-11-26T07:15:14.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:14.955 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol95121/.meta ./volumes/_nogroup/subvol86790) 2023-11-26T07:15:15.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol86790 2023-11-26T07:15:15.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.447+0000 7fec2d905700 1 Processor -- start 2023-11-26T07:15:15.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.447+0000 7fec2d905700 1 -- start start 2023-11-26T07:15:15.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec2d905700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fec2806aff0 0x7fec2806b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec2d905700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fec280d2a20 0x7fec280b7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec2d905700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2806b910 con 0x7fec2815e1c0 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec2d905700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec280cf590 con 0x7fec2806aff0 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec2d905700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec280d2eb0 con 0x7fec280d2a20 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec26ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fec2806aff0 0x7fec2806b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec26ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fec2806aff0 0x7fec2806b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38708/0 (socket says 172.21.15.83:38708) 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec26ffd700 1 -- 172.21.15.83:0/394618658 learned_addr learned my addr 172.21.15.83:0/394618658 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec26ffd700 1 -- 172.21.15.83:0/394618658 >> v2:172.21.15.203:3300/0 conn(0x7fec280d2a20 msgr2=0x7fec280b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec277fe700 1 --2- 172.21.15.83:0/394618658 >> v2:172.21.15.203:3300/0 conn(0x7fec280d2a20 0x7fec280b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec26ffd700 1 --2- 172.21.15.83:0/394618658 >> v2:172.21.15.203:3300/0 conn(0x7fec280d2a20 0x7fec280b7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec26ffd700 1 -- 172.21.15.83:0/394618658 >> v1:172.21.15.203:6789/0 conn(0x7fec2815e1c0 legacy=0x7fec280cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:15.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec26ffd700 1 -- 172.21.15.83:0/394618658 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec280d3190 con 0x7fec2806aff0 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec277fe700 1 --2- 172.21.15.83:0/394618658 >> v2:172.21.15.203:3300/0 conn(0x7fec280d2a20 0x7fec280b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec26ffd700 1 --2- 172.21.15.83:0/394618658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2806aff0 0x7fec2806b3d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fec2804f8b0 tx=0x7fec18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b24df88f715b3f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec25ffb700 1 -- 172.21.15.83:0/394618658 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18004230 con 0x7fec2806aff0 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.448+0000 7fec25ffb700 1 -- 172.21.15.83:0/394618658 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec18004390 con 0x7fec2806aff0 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.449+0000 7fec25ffb700 1 -- 172.21.15.83:0/394618658 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec180036a0 con 0x7fec2806aff0 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.449+0000 7fec2d905700 1 -- 172.21.15.83:0/394618658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2806aff0 msgr2=0x7fec2806b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.449+0000 7fec2d905700 1 --2- 172.21.15.83:0/394618658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2806aff0 0x7fec2806b3d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fec2804f8b0 tx=0x7fec18009bd0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.449+0000 7fec2d905700 1 -- 172.21.15.83:0/394618658 shutdown_connections 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.449+0000 7fec2d905700 1 --2- 172.21.15.83:0/394618658 >> v2:172.21.15.203:3300/0 conn(0x7fec280d2a20 0x7fec280b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.449+0000 7fec2d905700 1 --2- 172.21.15.83:0/394618658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec2806aff0 0x7fec2806b3d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.449+0000 7fec2d905700 1 -- 172.21.15.83:0/394618658 >> 172.21.15.83:0/394618658 conn(0x7fec280d3e40 msgr2=0x7fec280d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:15.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.449+0000 7fec2d905700 1 -- 172.21.15.83:0/394618658 shutdown_connections 2023-11-26T07:15:15.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.449+0000 7fec2d905700 1 -- 172.21.15.83:0/394618658 wait complete. 2023-11-26T07:15:15.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.451+0000 7fec2d905700 1 Processor -- start 2023-11-26T07:15:15.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.451+0000 7fec2d905700 1 -- start start 2023-11-26T07:15:15.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec2d905700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2806aff0 0x7fec28142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec2d905700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec2815e1c0 0x7fec28157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec2d905700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec28142220 0x7fec28156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec2d905700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec28155c30 con 0x7fec28142220 2023-11-26T07:15:15.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec2d905700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec28155ef0 con 0x7fec2815e1c0 2023-11-26T07:15:15.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec2d905700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2815a3d0 con 0x7fec2806aff0 2023-11-26T07:15:15.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec277fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec28142220 0x7fec28156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec277fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec28142220 0x7fec28156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38716/0 (socket says 172.21.15.83:38716) 2023-11-26T07:15:15.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec277fe700 1 -- 172.21.15.83:0/1707713986 learned_addr learned my addr 172.21.15.83:0/1707713986 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:15.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec267fc700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec2815e1c0 0x7fec28157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec26ffd700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2806aff0 0x7fec28142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec277fe700 1 -- 172.21.15.83:0/1707713986 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2806aff0 msgr2=0x7fec28142b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec277fe700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2806aff0 0x7fec28142b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec277fe700 1 -- 172.21.15.83:0/1707713986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec2815e1c0 msgr2=0x7fec28157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec277fe700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec2815e1c0 0x7fec28157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec277fe700 1 -- 172.21.15.83:0/1707713986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec18009550 con 0x7fec28142220 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec277fe700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec28142220 0x7fec28156380 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fec2000cf20 tx=0x7fec2000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec0ffff700 1 -- 172.21.15.83:0/1707713986 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2000ad00 con 0x7fec28142220 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.452+0000 7fec0ffff700 1 -- 172.21.15.83:0/1707713986 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2000ae60 con 0x7fec28142220 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.453+0000 7fec2d905700 1 -- 172.21.15.83:0/1707713986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2815a640 con 0x7fec28142220 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.453+0000 7fec0ffff700 1 -- 172.21.15.83:0/1707713986 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20015680 con 0x7fec28142220 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.453+0000 7fec2d905700 1 -- 172.21.15.83:0/1707713986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec28157c90 con 0x7fec28142220 2023-11-26T07:15:15.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.453+0000 7fec2d905700 1 -- 172.21.15.83:0/1707713986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec28068fb0 con 0x7fec28142220 2023-11-26T07:15:15.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.454+0000 7fec0ffff700 1 -- 172.21.15.83:0/1707713986 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7fec2000d430 con 0x7fec28142220 2023-11-26T07:15:15.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.454+0000 7fec0ffff700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1406b080 0x7fec1406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.454+0000 7fec0ffff700 1 -- 172.21.15.83:0/1707713986 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec20089d90 con 0x7fec28142220 2023-11-26T07:15:15.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.454+0000 7fec26ffd700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1406b080 0x7fec1406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.455+0000 7fec26ffd700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1406b080 0x7fec1406d540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fec18009130 tx=0x7fec18008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:15.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.459+0000 7fec0ffff700 1 -- 172.21.15.83:0/1707713986 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec20054bc0 con 0x7fec28142220 2023-11-26T07:15:15.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.681+0000 7fec2d905700 1 -- 172.21.15.83:0/1707713986 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol86790", "target": ["mon-mgr", ""]}) v1 -- 0x7fec280d3750 con 0x7fec1406b080 2023-11-26T07:15:15.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.710+0000 7fec0ffff700 1 -- 172.21.15.83:0/1707713986 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7fec180041e0 con 0x7fec1406b080 2023-11-26T07:15:15.713 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/_nogroup/subvol86790 2023-11-26T07:15:15.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.712+0000 7fec2d905700 1 -- 172.21.15.83:0/1707713986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1406b080 msgr2=0x7fec1406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:15.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.712+0000 7fec2d905700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1406b080 0x7fec1406d540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fec18009130 tx=0x7fec18008030 comp rx=0 tx=0).stop 2023-11-26T07:15:15.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.712+0000 7fec2d905700 1 -- 172.21.15.83:0/1707713986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec28142220 msgr2=0x7fec28156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:15.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.712+0000 7fec2d905700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec28142220 0x7fec28156380 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fec2000cf20 tx=0x7fec2000cf50 comp rx=0 tx=0).stop 2023-11-26T07:15:15.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.713+0000 7fec2d905700 1 -- 172.21.15.83:0/1707713986 shutdown_connections 2023-11-26T07:15:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.713+0000 7fec2d905700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec28142220 0x7fec28156380 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.713+0000 7fec2d905700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fec1406b080 0x7fec1406d540 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.713+0000 7fec2d905700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec2806aff0 0x7fec28142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.713+0000 7fec2d905700 1 --2- 172.21.15.83:0/1707713986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec2815e1c0 0x7fec28157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.713+0000 7fec2d905700 1 -- 172.21.15.83:0/1707713986 >> 172.21.15.83:0/1707713986 conn(0x7fec280d3e40 msgr2=0x7fec281605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.713+0000 7fec2d905700 1 -- 172.21.15.83:0/1707713986 shutdown_connections 2023-11-26T07:15:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.713+0000 7fec2d905700 1 -- 172.21.15.83:0/1707713986 wait complete. 2023-11-26T07:15:15.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol86790 2023-11-26T07:15:15.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.848+0000 7f78fd596700 1 Processor -- start 2023-11-26T07:15:15.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.848+0000 7f78fd596700 1 -- start start 2023-11-26T07:15:15.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78fd596700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78f80d45d0 0x7f78f80b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78fd596700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f78f80d39d0 0x7f78f80b7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78fd596700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f80d2930 con 0x7f78f815e140 2023-11-26T07:15:15.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78fd596700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f815e520 con 0x7f78f80d39d0 2023-11-26T07:15:15.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78fd596700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f80d2c10 con 0x7f78f80d45d0 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78f77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f78f80d39d0 0x7f78f80b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78f77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f78f80d39d0 0x7f78f80b7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38730/0 (socket says 172.21.15.83:38730) 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78f77fe700 1 -- 172.21.15.83:0/4017253029 learned_addr learned my addr 172.21.15.83:0/4017253029 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78f77fe700 1 -- 172.21.15.83:0/4017253029 >> v2:172.21.15.203:3300/0 conn(0x7f78f80d45d0 msgr2=0x7f78f80b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78f67fc700 1 --2- 172.21.15.83:0/4017253029 >> v2:172.21.15.203:3300/0 conn(0x7f78f80d45d0 0x7f78f80b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78f77fe700 1 --2- 172.21.15.83:0/4017253029 >> v2:172.21.15.203:3300/0 conn(0x7f78f80d45d0 0x7f78f80b70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78f77fe700 1 -- 172.21.15.83:0/4017253029 >> v1:172.21.15.203:6789/0 conn(0x7f78f815e140 legacy=0x7f78f80d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78f77fe700 1 -- 172.21.15.83:0/4017253029 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f80d2ef0 con 0x7f78f80d39d0 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.849+0000 7f78f67fc700 1 --2- 172.21.15.83:0/4017253029 >> v2:172.21.15.203:3300/0 conn(0x7f78f80d45d0 0x7f78f80b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78f77fe700 1 --2- 172.21.15.83:0/4017253029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f80d39d0 0x7f78f80b7610 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f78f80d4b20 tx=0x7f78f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5128dc2d606bfa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78f5ffb700 1 -- 172.21.15.83:0/4017253029 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f000e070 con 0x7f78f80d39d0 2023-11-26T07:15:15.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78f5ffb700 1 -- 172.21.15.83:0/4017253029 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f0003a20 con 0x7f78f80d39d0 2023-11-26T07:15:15.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78f5ffb700 1 -- 172.21.15.83:0/4017253029 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0003dd0 con 0x7f78f80d39d0 2023-11-26T07:15:15.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78fd596700 1 -- 172.21.15.83:0/4017253029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f80d39d0 msgr2=0x7f78f80b7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:15.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78fd596700 1 --2- 172.21.15.83:0/4017253029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f80d39d0 0x7f78f80b7610 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f78f80d4b20 tx=0x7f78f000b330 comp rx=0 tx=0).stop 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78fd596700 1 -- 172.21.15.83:0/4017253029 shutdown_connections 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78fd596700 1 --2- 172.21.15.83:0/4017253029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f80d39d0 0x7f78f80b7610 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78fd596700 1 --2- 172.21.15.83:0/4017253029 >> v2:172.21.15.203:3300/0 conn(0x7f78f80d45d0 0x7f78f80b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78fd596700 1 -- 172.21.15.83:0/4017253029 >> 172.21.15.83:0/4017253029 conn(0x7f78f806b410 msgr2=0x7f78f806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78fd596700 1 -- 172.21.15.83:0/4017253029 shutdown_connections 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.850+0000 7f78fd596700 1 -- 172.21.15.83:0/4017253029 wait complete. 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.852+0000 7f78fd596700 1 Processor -- start 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.852+0000 7f78fd596700 1 -- start start 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.852+0000 7f78fd596700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f80d39d0 0x7f78f814d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.852+0000 7f78fd596700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f80d45d0 0x7f78f80cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.852+0000 7f78fd596700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78f815e140 0x7f78f80cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.852+0000 7f78fd596700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80c7ec0 con 0x7f78f80d45d0 2023-11-26T07:15:15.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.852+0000 7f78fd596700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f814d880 con 0x7f78f80d39d0 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.852+0000 7f78fd596700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80c8110 con 0x7f78f815e140 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f67fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f80d45d0 0x7f78f80cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f67fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f80d45d0 0x7f78f80cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38734/0 (socket says 172.21.15.83:38734) 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f67fc700 1 -- 172.21.15.83:0/58585299 learned_addr learned my addr 172.21.15.83:0/58585299 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f6ffd700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f80d39d0 0x7f78f814d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f77fe700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78f815e140 0x7f78f80cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f67fc700 1 -- 172.21.15.83:0/58585299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78f815e140 msgr2=0x7f78f80cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f67fc700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78f815e140 0x7f78f80cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f67fc700 1 -- 172.21.15.83:0/58585299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f80d39d0 msgr2=0x7f78f814d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f67fc700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f80d39d0 0x7f78f814d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f67fc700 1 -- 172.21.15.83:0/58585299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f0009b30 con 0x7f78f80d45d0 2023-11-26T07:15:15.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f6ffd700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f80d39d0 0x7f78f814d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:15.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78f67fc700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f80d45d0 0x7f78f80cb000 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f78e800c750 tx=0x7f78e800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:15.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78df7fe700 1 -- 172.21.15.83:0/58585299 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8011800 con 0x7f78f80d45d0 2023-11-26T07:15:15.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78df7fe700 1 -- 172.21.15.83:0/58585299 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e8011960 con 0x7f78f80d45d0 2023-11-26T07:15:15.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78fd596700 1 -- 172.21.15.83:0/58585299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f80c83f0 con 0x7f78f80d45d0 2023-11-26T07:15:15.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78df7fe700 1 -- 172.21.15.83:0/58585299 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8011cd0 con 0x7f78f80d45d0 2023-11-26T07:15:15.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.853+0000 7f78fd596700 1 -- 172.21.15.83:0/58585299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f80c89b0 con 0x7f78f80d45d0 2023-11-26T07:15:15.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.854+0000 7f78fd596700 1 -- 172.21.15.83:0/58585299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f80c8d20 con 0x7f78f80d45d0 2023-11-26T07:15:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.855+0000 7f78df7fe700 1 -- 172.21.15.83:0/58585299 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f78e8021350 con 0x7f78f80d45d0 2023-11-26T07:15:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.855+0000 7f78df7fe700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f78e406b080 0x7f78e406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.855+0000 7f78df7fe700 1 -- 172.21.15.83:0/58585299 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78e8089a30 con 0x7f78f80d45d0 2023-11-26T07:15:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.855+0000 7f78f6ffd700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f78e406b080 0x7f78e406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:15.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.855+0000 7f78f6ffd700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f78e406b080 0x7f78e406d540 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009930 tx=0x7f78ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:15.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:15.860+0000 7f78df7fe700 1 -- 172.21.15.83:0/58585299 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78e80546c0 con 0x7f78f80d45d0 2023-11-26T07:15:16.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.083+0000 7f78fd596700 1 -- 172.21.15.83:0/58585299 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol86790", "target": ["mon-mgr", ""]}) v1 -- 0x7f78f80d1be0 con 0x7f78e406b080 2023-11-26T07:15:16.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.088+0000 7f78df7fe700 1 -- 172.21.15.83:0/58585299 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f78ec009e00 con 0x7f78e406b080 2023-11-26T07:15:16.090 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/_nogroup/subvol86790 2023-11-26T07:15:16.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 -- 172.21.15.83:0/58585299 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f78e406b080 msgr2=0x7f78e406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:16.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f78e406b080 0x7f78e406d540 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009930 tx=0x7f78ec008040 comp rx=0 tx=0).stop 2023-11-26T07:15:16.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 -- 172.21.15.83:0/58585299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f80d45d0 msgr2=0x7f78f80cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:16.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f80d45d0 0x7f78f80cb000 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f78e800c750 tx=0x7f78e800cb10 comp rx=0 tx=0).stop 2023-11-26T07:15:16.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 -- 172.21.15.83:0/58585299 shutdown_connections 2023-11-26T07:15:16.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f78e406b080 0x7f78e406d540 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:16.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78f815e140 0x7f78f80cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:16.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78f80d39d0 0x7f78f814d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:16.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 --2- 172.21.15.83:0/58585299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f80d45d0 0x7f78f80cb000 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:16.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 -- 172.21.15.83:0/58585299 >> 172.21.15.83:0/58585299 conn(0x7f78f806b410 msgr2=0x7f78f815e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:16.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 -- 172.21.15.83:0/58585299 shutdown_connections 2023-11-26T07:15:16.094 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:16.090+0000 7f78fd596700 1 -- 172.21.15.83:0/58585299 wait complete. 2023-11-26T07:15:16.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:16.104 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/5082117df8ac12e1d1fe4fc71860f4e6.meta) 2023-11-26T07:15:20.327 INFO:teuthology.orchestra.run.smithi083.stdout:[GLOBAL] 2023-11-26T07:15:20.327 INFO:teuthology.orchestra.run.smithi083.stdout:version = 1 2023-11-26T07:15:20.327 INFO:teuthology.orchestra.run.smithi083.stdout:type = subvolume 2023-11-26T07:15:20.327 INFO:teuthology.orchestra.run.smithi083.stdout:path = /volumes/_nogroup/subvol86790 2023-11-26T07:15:20.327 INFO:teuthology.orchestra.run.smithi083.stdout:state = complete 2023-11-26T07:15:20.328 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:20.329 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol86790 alice 2023-11-26T07:15:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.463+0000 7f0fd1d1e700 1 Processor -- start 2023-11-26T07:15:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.463+0000 7f0fd1d1e700 1 -- start start 2023-11-26T07:15:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.463+0000 7f0fd1d1e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0fcc0d1ab0 0x7f0fcc0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.463+0000 7f0fd1d1e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fcc0d24a0 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.463+0000 7f0fd1d1e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fcc0d01c0 con 0x7f0fcc0cf040 2023-11-26T07:15:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.463+0000 7f0fd1d1e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fcc070910 con 0x7f0fcc0d1ab0 2023-11-26T07:15:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fcaffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0fcc0d1ab0 0x7f0fcc0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fcbfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0fcc0af3b0 0x7f0fcc0af7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58786/0 (socket says 172.21.15.83:58786) 2023-11-26T07:15:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fcaffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0fcc0d1ab0 0x7f0fcc0d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38744/0 (socket says 172.21.15.83:38744) 2023-11-26T07:15:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fcbfff700 1 -- 172.21.15.83:0/170693397 learned_addr learned my addr 172.21.15.83:0/170693397 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fcaffd700 1 -- 172.21.15.83:0/170693397 >> v1:172.21.15.203:6789/0 conn(0x7f0fcc0cf040 legacy=0x7f0fcc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fcaffd700 1 -- 172.21.15.83:0/170693397 >> v1:172.21.15.83:6789/0 conn(0x7f0fcc0af3b0 legacy=0x7f0fcc0af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fcaffd700 1 -- 172.21.15.83:0/170693397 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc070bf0 con 0x7f0fcc0d1ab0 2023-11-26T07:15:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fca7fc700 1 -- 172.21.15.83:0/170693397 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873880578 0 0) 0x7f0fcc0d24a0 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fcaffd700 1 --2- 172.21.15.83:0/170693397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fcc0d1ab0 0x7f0fcc0d1f60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009840 tx=0x7f0fbc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe605b64d6b62cf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fca7fc700 1 -- 172.21.15.83:0/170693397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc004230 con 0x7f0fcc0d1ab0 2023-11-26T07:15:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fca7fc700 1 -- 172.21.15.83:0/170693397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc004390 con 0x7f0fcc0d1ab0 2023-11-26T07:15:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.464+0000 7f0fca7fc700 1 -- 172.21.15.83:0/170693397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0036a0 con 0x7f0fcc0d1ab0 2023-11-26T07:15:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.465+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/170693397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fcc0d1ab0 msgr2=0x7f0fcc0d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.465+0000 7f0fd1d1e700 1 --2- 172.21.15.83:0/170693397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fcc0d1ab0 0x7f0fcc0d1f60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009840 tx=0x7f0fbc009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.465+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/170693397 shutdown_connections 2023-11-26T07:15:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.465+0000 7f0fd1d1e700 1 --2- 172.21.15.83:0/170693397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fcc0d1ab0 0x7f0fcc0d1f60 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.465+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/170693397 >> 172.21.15.83:0/170693397 conn(0x7f0fcc09aa00 msgr2=0x7f0fcc09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.465+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/170693397 shutdown_connections 2023-11-26T07:15:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.465+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/170693397 wait complete. 2023-11-26T07:15:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.466+0000 7f0fd1d1e700 1 Processor -- start 2023-11-26T07:15:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.466+0000 7f0fd1d1e700 1 -- start start 2023-11-26T07:15:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fd1d1e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fcc0af3b0 0x7f0fcc15bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fd1d1e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fcc0cf040 0x7f0fcc0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fd1d1e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fcc0d1ab0 0x7f0fcc15a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fd1d1e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0ca2a0 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fd1d1e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0b5840 con 0x7f0fcc0d1ab0 2023-11-26T07:15:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fd1d1e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0c9720 con 0x7f0fcc0cf040 2023-11-26T07:15:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fcb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fcc0af3b0 0x7f0fcc15bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fcb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fcc0af3b0 0x7f0fcc15bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38748/0 (socket says 172.21.15.83:38748) 2023-11-26T07:15:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fcb7fe700 1 -- 172.21.15.83:0/226446421 learned_addr learned my addr 172.21.15.83:0/226446421 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fcbfff700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fcc0d1ab0 0x7f0fcc15a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fcb7fe700 1 -- 172.21.15.83:0/226446421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fcc0cf040 msgr2=0x7f0fcc0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fcb7fe700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fcc0cf040 0x7f0fcc0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fcb7fe700 1 -- 172.21.15.83:0/226446421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fcc0d1ab0 msgr2=0x7f0fcc15a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fcb7fe700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fcc0d1ab0 0x7f0fcc15a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fcb7fe700 1 -- 172.21.15.83:0/226446421 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc0094a0 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fcb7fe700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fcc0af3b0 0x7f0fcc15bec0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c4d0 tx=0x7f0fc000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.467+0000 7f0fb3fff700 1 -- 172.21.15.83:0/226446421 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0011790 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.468+0000 7f0fb3fff700 1 -- 172.21.15.83:0/226446421 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc00118f0 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.468+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/226446421 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc0c9a00 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.468+0000 7f0fb3fff700 1 -- 172.21.15.83:0/226446421 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0011ca0 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.468+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/226446421 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc0cb950 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.468+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/226446421 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc0cbae0 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.469+0000 7f0fb3fff700 1 -- 172.21.15.83:0/226446421 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 10) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0fc001c460 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.469+0000 7f0fb3fff700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0fb806b110 0x7f0fb806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.469+0000 7f0fb3fff700 1 -- 172.21.15.83:0/226446421 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fc00897a0 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.469+0000 7f0fcaffd700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0fb806b110 0x7f0fb806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.470+0000 7f0fcaffd700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0fb806b110 0x7f0fb806d5d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0c8980 tx=0x7f0fbc013040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.474+0000 7f0fb3fff700 1 -- 172.21.15.83:0/226446421 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0fc00545c0 con 0x7f0fcc0af3b0 2023-11-26T07:15:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.696+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/226446421 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol86790", "auth_id": "alice", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fcc069000 con 0x7f0fb806b110 2023-11-26T07:15:20.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.738+0000 7f0fb3fff700 1 -- 172.21.15.83:0/226446421 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0fcc069000 con 0x7f0fb806b110 2023-11-26T07:15:20.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/226446421 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0fb806b110 msgr2=0x7f0fb806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:20.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0fb806b110 0x7f0fb806d5d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0c8980 tx=0x7f0fbc013040 comp rx=0 tx=0).stop 2023-11-26T07:15:20.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/226446421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fcc0af3b0 msgr2=0x7f0fcc15bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:20.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fcc0af3b0 0x7f0fcc15bec0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c4d0 tx=0x7f0fc000c890 comp rx=0 tx=0).stop 2023-11-26T07:15:20.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/226446421 shutdown_connections 2023-11-26T07:15:20.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0fcc0d1ab0 0x7f0fcc15a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fcc0af3b0 0x7f0fcc15bec0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f0fb806b110 0x7f0fb806d5d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 --2- 172.21.15.83:0/226446421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0fcc0cf040 0x7f0fcc0c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/226446421 >> 172.21.15.83:0/226446421 conn(0x7f0fcc09aa00 msgr2=0x7f0fcc0b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:20.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/226446421 shutdown_connections 2023-11-26T07:15:20.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.740+0000 7f0fd1d1e700 1 -- 172.21.15.83:0/226446421 wait complete. 2023-11-26T07:15:20.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2023-11-26T07:15:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.874+0000 7fcecbc71700 1 Processor -- start 2023-11-26T07:15:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.874+0000 7fcecbc71700 1 -- start start 2023-11-26T07:15:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.875+0000 7fcecbc71700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcec415e080 0x7fcec40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.875+0000 7fcecbc71700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec406c710 0x7fcec40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.875+0000 7fcecbc71700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec40ae430 con 0x7fcec40af4d0 2023-11-26T07:15:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.875+0000 7fcecbc71700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec40af0b0 con 0x7fcec415e080 2023-11-26T07:15:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.875+0000 7fcecbc71700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec40b05a0 con 0x7fcec406c710 2023-11-26T07:15:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.875+0000 7fcec920c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcec40af4d0 0x7fcec406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58800/0 (socket says 172.21.15.83:58800) 2023-11-26T07:15:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.875+0000 7fcec920c700 1 -- 172.21.15.83:0/595778134 learned_addr learned my addr 172.21.15.83:0/595778134 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.875+0000 7fceca20e700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec406c710 0x7fcec40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.875+0000 7fcec9a0d700 1 --2- 172.21.15.83:0/595778134 >> v2:172.21.15.203:3300/0 conn(0x7fcec415e080 0x7fcec40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.875+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904408412 0 0) 0x7fcec40ae430 con 0x7fcec40af4d0 2023-11-26T07:15:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.875+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceb00035b0 con 0x7fcec40af4d0 2023-11-26T07:15:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2168168494 0 0) 0x7fceb00035b0 con 0x7fcec40af4d0 2023-11-26T07:15:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec406c710 msgr2=0x7fcec40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec406c710 0x7fcec40afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 >> v2:172.21.15.203:3300/0 conn(0x7fcec415e080 msgr2=0x7fcec40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 --2- 172.21.15.83:0/595778134 >> v2:172.21.15.203:3300/0 conn(0x7fcec415e080 0x7fcec40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec40ae710 con 0x7fcec40af4d0 2023-11-26T07:15:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fcec0001670 con 0x7fcec40af4d0 2023-11-26T07:15:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 >> v1:172.21.15.83:6789/0 conn(0x7fcec40af4d0 legacy=0x7fcec406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec406c710 0x7fceb0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fceb0003fb0 0x7fceb0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb0004d90 0x7fceb0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec40af0b0 con 0x7fceb0004d90 2023-11-26T07:15:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec40b05a0 con 0x7fceb0003fb0 2023-11-26T07:15:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb000ba10 con 0x7fcec406c710 2023-11-26T07:15:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec9a0d700 1 --2- 172.21.15.83:0/595778134 >> v2:172.21.15.203:3300/0 conn(0x7fcec415e080 0x7fcec40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:15:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fceca20e700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb0004d90 0x7fceb0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec920c700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fceb0003fb0 0x7fceb0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fcec9a0d700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec406c710 0x7fceb0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fceca20e700 1 -- 172.21.15.83:0/595778134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec406c710 msgr2=0x7fceb0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fceca20e700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec406c710 0x7fceb0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fceca20e700 1 -- 172.21.15.83:0/595778134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fceb0003fb0 msgr2=0x7fceb0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fceca20e700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fceb0003fb0 0x7fceb0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fceca20e700 1 -- 172.21.15.83:0/595778134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb00035b0 con 0x7fceb0004d90 2023-11-26T07:15:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.876+0000 7fceca20e700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb0004d90 0x7fceb0005210 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009f90 tx=0x7fceb0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f16263511f636a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.877+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00ebd0 con 0x7fceb0004d90 2023-11-26T07:15:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.877+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc00f040 con 0x7fceb0004d90 2023-11-26T07:15:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.877+0000 7fcec8a0b700 1 -- 172.21.15.83:0/595778134 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc014480 con 0x7fceb0004d90 2023-11-26T07:15:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.877+0000 7fcecbc71700 1 -- 172.21.15.83:0/595778134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb0004d90 msgr2=0x7fceb0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.877+0000 7fcecbc71700 1 --2- 172.21.15.83:0/595778134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb0004d90 0x7fceb0005210 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009f90 tx=0x7fceb0005880 comp rx=0 tx=0).stop 2023-11-26T07:15:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.877+0000 7fceca20e700 1 -- 172.21.15.83:0/595778134 reap_dead start 2023-11-26T07:15:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.877+0000 7fcecbc71700 1 -- 172.21.15.83:0/595778134 shutdown_connections 2023-11-26T07:15:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.877+0000 7fcecbc71700 1 -- 172.21.15.83:0/595778134 >> 172.21.15.83:0/595778134 conn(0x7fcec4071b90 msgr2=0x7fcec4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.877+0000 7fcecbc71700 1 -- 172.21.15.83:0/595778134 shutdown_connections 2023-11-26T07:15:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.877+0000 7fcecbc71700 1 -- 172.21.15.83:0/595778134 wait complete. 2023-11-26T07:15:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.878+0000 7fcecbc71700 1 Processor -- start 2023-11-26T07:15:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.878+0000 7fcecbc71700 1 -- start start 2023-11-26T07:15:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.879+0000 7fcecbc71700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec4143540 0x7fcec40cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.879+0000 7fcecbc71700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec40cfd30 0x7fcec415e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.879+0000 7fcecbc71700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec415e9a0 0x7fcec4145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.879+0000 7fcecbc71700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec40d01b0 con 0x7fcec415e9a0 2023-11-26T07:15:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.879+0000 7fcecbc71700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec4153f90 con 0x7fcec4143540 2023-11-26T07:15:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.879+0000 7fcecbc71700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec4154200 con 0x7fcec40cfd30 2023-11-26T07:15:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.879+0000 7fceca20e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec415e9a0 0x7fcec4145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.879+0000 7fceca20e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec415e9a0 0x7fcec4145d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38774/0 (socket says 172.21.15.83:38774) 2023-11-26T07:15:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.879+0000 7fceca20e700 1 -- 172.21.15.83:0/2881434426 learned_addr learned my addr 172.21.15.83:0/2881434426 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.879+0000 7fcec920c700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec40cfd30 0x7fcec415e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.879+0000 7fcec9a0d700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec4143540 0x7fcec40cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.880+0000 7fceca20e700 1 -- 172.21.15.83:0/2881434426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec40cfd30 msgr2=0x7fcec415e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.880+0000 7fceca20e700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec40cfd30 0x7fcec415e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.880+0000 7fceca20e700 1 -- 172.21.15.83:0/2881434426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec4143540 msgr2=0x7fcec40cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.880+0000 7fceca20e700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec4143540 0x7fcec40cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.880+0000 7fceca20e700 1 -- 172.21.15.83:0/2881434426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec41544e0 con 0x7fcec415e9a0 2023-11-26T07:15:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.880+0000 7fceca20e700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec415e9a0 0x7fcec4145d60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fceb0004720 tx=0x7fcebc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.880+0000 7fceba7fc700 1 -- 172.21.15.83:0/2881434426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc007000 con 0x7fcec415e9a0 2023-11-26T07:15:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.880+0000 7fceba7fc700 1 -- 172.21.15.83:0/2881434426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc015b30 con 0x7fcec415e9a0 2023-11-26T07:15:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.880+0000 7fceba7fc700 1 -- 172.21.15.83:0/2881434426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc003a20 con 0x7fcec415e9a0 2023-11-26T07:15:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.880+0000 7fcecbc71700 1 -- 172.21.15.83:0/2881434426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec41547c0 con 0x7fcec415e9a0 2023-11-26T07:15:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.880+0000 7fcecbc71700 1 -- 172.21.15.83:0/2881434426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec415f190 con 0x7fcec415e9a0 2023-11-26T07:15:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.881+0000 7fcecbc71700 1 -- 172.21.15.83:0/2881434426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec4154980 con 0x7fcec415e9a0 2023-11-26T07:15:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.881+0000 7fceba7fc700 1 -- 172.21.15.83:0/2881434426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 10) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fcebc01e430 con 0x7fcec415e9a0 2023-11-26T07:15:20.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.882+0000 7fceba7fc700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcea806b110 0x7fcea806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:20.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.882+0000 7fceba7fc700 1 -- 172.21.15.83:0/2881434426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcebc024030 con 0x7fcec415e9a0 2023-11-26T07:15:20.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.882+0000 7fcec9a0d700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcea806b110 0x7fcea806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:20.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.882+0000 7fcec9a0d700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcea806b110 0x7fcea806d5d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009cc0 tx=0x7fceb4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:20.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:20.886+0000 7fceba7fc700 1 -- 172.21.15.83:0/2881434426 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcebc0563f0 con 0x7fcec415e9a0 2023-11-26T07:15:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.154+0000 7fcecbc71700 1 -- 172.21.15.83:0/2881434426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) v1 -- 0x7fcec40753f0 con 0x7fcec415e9a0 2023-11-26T07:15:21.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.155+0000 7fceba7fc700 1 -- 172.21.15.83:0/2881434426 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v18) v1 ==== 107+0+280 (secure 0 0 0) 0x7fcebc059a10 con 0x7fcec415e9a0 2023-11-26T07:15:21.157 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:21.158 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:15:21.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:21.158 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.alice", 2023-11-26T07:15:21.158 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCI8GJlGXHNKhAA5ZF4a69YcKlXkvwoR2gTSw==", 2023-11-26T07:15:21.158 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:21.158 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol86790", 2023-11-26T07:15:21.158 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2023-11-26T07:15:21.158 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-26T07:15:21.158 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:21.158 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:21.158 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:15:21.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 -- 172.21.15.83:0/2881434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcea806b110 msgr2=0x7fcea806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcea806b110 0x7fcea806d5d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009cc0 tx=0x7fceb4008040 comp rx=0 tx=0).stop 2023-11-26T07:15:21.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 -- 172.21.15.83:0/2881434426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec415e9a0 msgr2=0x7fcec4145d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec415e9a0 0x7fcec4145d60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fceb0004720 tx=0x7fcebc009750 comp rx=0 tx=0).stop 2023-11-26T07:15:21.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 -- 172.21.15.83:0/2881434426 shutdown_connections 2023-11-26T07:15:21.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec415e9a0 0x7fcec4145d60 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec4143540 0x7fcec40cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcea806b110 0x7fcea806d5d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 --2- 172.21.15.83:0/2881434426 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec40cfd30 0x7fcec415e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 -- 172.21.15.83:0/2881434426 >> 172.21.15.83:0/2881434426 conn(0x7fcec4071b90 msgr2=0x7fcec406bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:21.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 -- 172.21.15.83:0/2881434426 shutdown_connections 2023-11-26T07:15:21.161 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.157+0000 7fcecbc71700 1 -- 172.21.15.83:0/2881434426 wait complete. 2023-11-26T07:15:21.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol86790 2023-11-26T07:15:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.291+0000 7f589d59b700 1 Processor -- start 2023-11-26T07:15:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.291+0000 7f589d59b700 1 -- start start 2023-11-26T07:15:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.292+0000 7f589d59b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f58980cc540 0x7f589806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.292+0000 7f589d59b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f589806eb20 0x7f58980b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:21.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.292+0000 7f589d59b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589806ef50 con 0x7f589815a620 2023-11-26T07:15:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.292+0000 7f589d59b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58980b2f20 con 0x7f589806eb20 2023-11-26T07:15:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.292+0000 7f589d59b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589815aa00 con 0x7f58980cc540 2023-11-26T07:15:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.292+0000 7f58977fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f589806eb20 0x7f58980b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.292+0000 7f58977fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f589806eb20 0x7f58980b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38782/0 (socket says 172.21.15.83:38782) 2023-11-26T07:15:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.292+0000 7f58977fe700 1 -- 172.21.15.83:0/3502817839 learned_addr learned my addr 172.21.15.83:0/3502817839 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.293+0000 7f58967fc700 1 --2- 172.21.15.83:0/3502817839 >> v2:172.21.15.203:3300/0 conn(0x7f58980cc540 0x7f589806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.293+0000 7f58977fe700 1 -- 172.21.15.83:0/3502817839 >> v2:172.21.15.203:3300/0 conn(0x7f58980cc540 msgr2=0x7f589806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.293+0000 7f58977fe700 1 --2- 172.21.15.83:0/3502817839 >> v2:172.21.15.203:3300/0 conn(0x7f58980cc540 0x7f589806e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.293+0000 7f58977fe700 1 -- 172.21.15.83:0/3502817839 >> v1:172.21.15.203:6789/0 conn(0x7f589815a620 legacy=0x7f58980ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.293+0000 7f58977fe700 1 -- 172.21.15.83:0/3502817839 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58980ce960 con 0x7f589806eb20 2023-11-26T07:15:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.293+0000 7f58977fe700 1 --2- 172.21.15.83:0/3502817839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f589806eb20 0x7f58980b3350 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5898075600 tx=0x7f589000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6faedd9ec7fb3f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.293+0000 7f5895ffb700 1 -- 172.21.15.83:0/3502817839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890010070 con 0x7f589806eb20 2023-11-26T07:15:21.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.293+0000 7f5895ffb700 1 -- 172.21.15.83:0/3502817839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589000db40 con 0x7f589806eb20 2023-11-26T07:15:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.293+0000 7f5895ffb700 1 -- 172.21.15.83:0/3502817839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890003a20 con 0x7f589806eb20 2023-11-26T07:15:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.294+0000 7f589d59b700 1 -- 172.21.15.83:0/3502817839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f589806eb20 msgr2=0x7f58980b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.294+0000 7f589d59b700 1 --2- 172.21.15.83:0/3502817839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f589806eb20 0x7f58980b3350 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5898075600 tx=0x7f589000d430 comp rx=0 tx=0).stop 2023-11-26T07:15:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.294+0000 7f589d59b700 1 -- 172.21.15.83:0/3502817839 shutdown_connections 2023-11-26T07:15:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.294+0000 7f589d59b700 1 --2- 172.21.15.83:0/3502817839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f589806eb20 0x7f58980b3350 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.294+0000 7f589d59b700 1 --2- 172.21.15.83:0/3502817839 >> v2:172.21.15.203:3300/0 conn(0x7f58980cc540 0x7f589806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.294+0000 7f589d59b700 1 -- 172.21.15.83:0/3502817839 >> 172.21.15.83:0/3502817839 conn(0x7f58980b2050 msgr2=0x7f58980b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.294+0000 7f589d59b700 1 -- 172.21.15.83:0/3502817839 shutdown_connections 2023-11-26T07:15:21.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.294+0000 7f589d59b700 1 -- 172.21.15.83:0/3502817839 wait complete. 2023-11-26T07:15:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.295+0000 7f589d59b700 1 Processor -- start 2023-11-26T07:15:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.295+0000 7f589d59b700 1 -- start start 2023-11-26T07:15:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f589d59b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f589806eb20 0x7f5898149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f589d59b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58980cc540 0x7f5898153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:21.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f589d59b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f589815a620 0x7f58981527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f589d59b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5898149fc0 con 0x7f58980cc540 2023-11-26T07:15:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f589d59b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5898153dd0 con 0x7f589815a620 2023-11-26T07:15:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f589d59b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58981540b0 con 0x7f589806eb20 2023-11-26T07:15:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f58967fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58980cc540 0x7f5898153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f58967fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58980cc540 0x7f5898153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38786/0 (socket says 172.21.15.83:38786) 2023-11-26T07:15:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f58967fc700 1 -- 172.21.15.83:0/3208394676 learned_addr learned my addr 172.21.15.83:0/3208394676 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f5896ffd700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f589806eb20 0x7f5898149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f58977fe700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f589815a620 0x7f58981527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f58967fc700 1 -- 172.21.15.83:0/3208394676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f589806eb20 msgr2=0x7f5898149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f58967fc700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f589806eb20 0x7f5898149a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f58967fc700 1 -- 172.21.15.83:0/3208394676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f589815a620 msgr2=0x7f58981527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f58967fc700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f589815a620 0x7f58981527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f58967fc700 1 -- 172.21.15.83:0/3208394676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589000bd40 con 0x7f58980cc540 2023-11-26T07:15:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f5896ffd700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f589806eb20 0x7f5898149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f58967fc700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58980cc540 0x7f5898153580 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f588800c7f0 tx=0x7f588800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:21.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f587f7fe700 1 -- 172.21.15.83:0/3208394676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5888011800 con 0x7f58980cc540 2023-11-26T07:15:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f587f7fe700 1 -- 172.21.15.83:0/3208394676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5888011960 con 0x7f58980cc540 2023-11-26T07:15:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f589d59b700 1 -- 172.21.15.83:0/3208394676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5898154390 con 0x7f58980cc540 2023-11-26T07:15:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f587f7fe700 1 -- 172.21.15.83:0/3208394676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5888011cf0 con 0x7f58980cc540 2023-11-26T07:15:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.296+0000 7f589d59b700 1 -- 172.21.15.83:0/3208394676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5898152280 con 0x7f58980cc540 2023-11-26T07:15:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.297+0000 7f589d59b700 1 -- 172.21.15.83:0/3208394676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5898068d90 con 0x7f58980cc540 2023-11-26T07:15:21.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.298+0000 7f587f7fe700 1 -- 172.21.15.83:0/3208394676 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 10) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f5888018360 con 0x7f58980cc540 2023-11-26T07:15:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.298+0000 7f587f7fe700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f588406b110 0x7f588406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.298+0000 7f587f7fe700 1 -- 172.21.15.83:0/3208394676 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5888089ac0 con 0x7f58980cc540 2023-11-26T07:15:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.298+0000 7f5896ffd700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f588406b110 0x7f588406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.299+0000 7f5896ffd700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f588406b110 0x7f588406d5d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f588c009930 tx=0x7f588c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:21.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.303+0000 7f587f7fe700 1 -- 172.21.15.83:0/3208394676 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5888054710 con 0x7f58980cc540 2023-11-26T07:15:21.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.524+0000 7f589d59b700 1 -- 172.21.15.83:0/3208394676 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol86790", "target": ["mon-mgr", ""]}) v1 -- 0x7f589806f710 con 0x7f588406b110 2023-11-26T07:15:21.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.536+0000 7f587f7fe700 1 -- 172.21.15.83:0/3208394676 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f589806f710 con 0x7f588406b110 2023-11-26T07:15:21.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 -- 172.21.15.83:0/3208394676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f588406b110 msgr2=0x7f588406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f588406b110 0x7f588406d5d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f588c009930 tx=0x7f588c008040 comp rx=0 tx=0).stop 2023-11-26T07:15:21.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 -- 172.21.15.83:0/3208394676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58980cc540 msgr2=0x7f5898153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58980cc540 0x7f5898153580 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f588800c7f0 tx=0x7f588800cbb0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 -- 172.21.15.83:0/3208394676 shutdown_connections 2023-11-26T07:15:21.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f589815a620 0x7f58981527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f588406b110 0x7f588406d5d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f589806eb20 0x7f5898149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 --2- 172.21.15.83:0/3208394676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58980cc540 0x7f5898153580 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 -- 172.21.15.83:0/3208394676 >> 172.21.15.83:0/3208394676 conn(0x7f58980b2050 msgr2=0x7f58980af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:21.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 -- 172.21.15.83:0/3208394676 shutdown_connections 2023-11-26T07:15:21.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.539+0000 7f589d59b700 1 -- 172.21.15.83:0/3208394676 wait complete. 2023-11-26T07:15:21.543 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:21.540+0000 7f3780369700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:21.543 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:21.540+0000 7f3780369700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:21.543 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:21.540+0000 7f3780369700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:21.543 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:21.540+0000 7f3780369700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:21.543 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:21.540+0000 7f3780369700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:21.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol95121 2023-11-26T07:15:21.558 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:21.555+0000 7f378136b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:21.558 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:21.555+0000 7f378136b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:21.558 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:21.555+0000 7f378136b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:21.558 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:21.555+0000 7f378136b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:21.558 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:21.555+0000 7f378136b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.678+0000 7fbdef650700 1 Processor -- start 2023-11-26T07:15:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.678+0000 7fbdef650700 1 -- start start 2023-11-26T07:15:21.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.679+0000 7fbdef650700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbde80b4370 0x7fbde815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:21.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.679+0000 7fbdef650700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde80ae5e0 con 0x7fbde815b370 2023-11-26T07:15:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.679+0000 7fbdef650700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde815be30 con 0x7fbde81587f0 2023-11-26T07:15:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.679+0000 7fbdef650700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde815c110 con 0x7fbde80b4370 2023-11-26T07:15:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.679+0000 7fbdedbed700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbde80b4370 0x7fbde815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.679+0000 7fbdedbed700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbde80b4370 0x7fbde815b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38822/0 (socket says 172.21.15.83:38822) 2023-11-26T07:15:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.679+0000 7fbdecbeb700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbde815b370 0x7fbde80b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58860/0 (socket says 172.21.15.83:58860) 2023-11-26T07:15:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.679+0000 7fbdedbed700 1 -- 172.21.15.83:0/2018233274 learned_addr learned my addr 172.21.15.83:0/2018233274 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.680+0000 7fbdedbed700 1 -- 172.21.15.83:0/2018233274 >> v1:172.21.15.203:6789/0 conn(0x7fbde81587f0 legacy=0x7fbde806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.680+0000 7fbdedbed700 1 -- 172.21.15.83:0/2018233274 >> v1:172.21.15.83:6789/0 conn(0x7fbde815b370 legacy=0x7fbde80b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.680+0000 7fbdedbed700 1 -- 172.21.15.83:0/2018233274 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80ae860 con 0x7fbde80b4370 2023-11-26T07:15:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.680+0000 7fbddffff700 1 -- 172.21.15.83:0/2018233274 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230666730 0 0) 0x7fbde80ae5e0 con 0x7fbde815b370 2023-11-26T07:15:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.680+0000 7fbdedbed700 1 --2- 172.21.15.83:0/2018233274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80b4370 0x7fbde815b800 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009e80 tx=0x7fbde000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f78a5c8d5b83639 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.680+0000 7fbddffff700 1 -- 172.21.15.83:0/2018233274 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde000e070 con 0x7fbde80b4370 2023-11-26T07:15:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.680+0000 7fbddffff700 1 -- 172.21.15.83:0/2018233274 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde0003a20 con 0x7fbde80b4370 2023-11-26T07:15:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.680+0000 7fbddffff700 1 -- 172.21.15.83:0/2018233274 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0003dd0 con 0x7fbde80b4370 2023-11-26T07:15:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.680+0000 7fbdef650700 1 -- 172.21.15.83:0/2018233274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80b4370 msgr2=0x7fbde815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.680+0000 7fbdef650700 1 --2- 172.21.15.83:0/2018233274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80b4370 0x7fbde815b800 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009e80 tx=0x7fbde000b330 comp rx=0 tx=0).stop 2023-11-26T07:15:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.681+0000 7fbdef650700 1 -- 172.21.15.83:0/2018233274 shutdown_connections 2023-11-26T07:15:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.681+0000 7fbdef650700 1 --2- 172.21.15.83:0/2018233274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde80b4370 0x7fbde815b800 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.681+0000 7fbdef650700 1 -- 172.21.15.83:0/2018233274 >> 172.21.15.83:0/2018233274 conn(0x7fbde80d0bf0 msgr2=0x7fbde80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.681+0000 7fbdef650700 1 -- 172.21.15.83:0/2018233274 shutdown_connections 2023-11-26T07:15:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.681+0000 7fbdef650700 1 -- 172.21.15.83:0/2018233274 wait complete. 2023-11-26T07:15:21.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.682+0000 7fbdef650700 1 Processor -- start 2023-11-26T07:15:21.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.682+0000 7fbdef650700 1 -- start start 2023-11-26T07:15:21.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.682+0000 7fbdef650700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbde80b4370 0x7fbde814b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:21.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdef650700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde81587f0 0x7fbde809bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdef650700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde815b370 0x7fbde80b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdef650700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde809ab30 con 0x7fbde815b370 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdef650700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde809ae10 con 0x7fbde80b4370 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdef650700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde809b0f0 con 0x7fbde81587f0 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdedbed700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde815b370 0x7fbde80b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdedbed700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde815b370 0x7fbde80b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38828/0 (socket says 172.21.15.83:38828) 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdedbed700 1 -- 172.21.15.83:0/4195857136 learned_addr learned my addr 172.21.15.83:0/4195857136 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbded3ec700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbde80b4370 0x7fbde814b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdedbed700 1 -- 172.21.15.83:0/4195857136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde81587f0 msgr2=0x7fbde809bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdecbeb700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde81587f0 0x7fbde809bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdedbed700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde81587f0 0x7fbde809bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdedbed700 1 -- 172.21.15.83:0/4195857136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbde80b4370 msgr2=0x7fbde814b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdedbed700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbde80b4370 0x7fbde814b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdedbed700 1 -- 172.21.15.83:0/4195857136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde0009b30 con 0x7fbde815b370 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbded3ec700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbde80b4370 0x7fbde814b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdedbed700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde815b370 0x7fbde80b1920 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbde0015040 tx=0x7fbde0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdddffb700 1 -- 172.21.15.83:0/4195857136 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0004240 con 0x7fbde815b370 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdddffb700 1 -- 172.21.15.83:0/4195857136 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde00043e0 con 0x7fbde815b370 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdef650700 1 -- 172.21.15.83:0/4195857136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde809b3d0 con 0x7fbde815b370 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdddffb700 1 -- 172.21.15.83:0/4195857136 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0004790 con 0x7fbde815b370 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.683+0000 7fbdef650700 1 -- 172.21.15.83:0/4195857136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde814ba30 con 0x7fbde815b370 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.684+0000 7fbdef650700 1 -- 172.21.15.83:0/4195857136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde8068fb0 con 0x7fbde815b370 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.685+0000 7fbdddffb700 1 -- 172.21.15.83:0/4195857136 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 10) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fbde0019070 con 0x7fbde815b370 2023-11-26T07:15:21.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.685+0000 7fbdddffb700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdcc06b110 0x7fbdcc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:21.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.685+0000 7fbdddffb700 1 -- 172.21.15.83:0/4195857136 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbde008aad0 con 0x7fbde815b370 2023-11-26T07:15:21.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.685+0000 7fbded3ec700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdcc06b110 0x7fbdcc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:21.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.686+0000 7fbded3ec700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdcc06b110 0x7fbdcc06d5d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbde4003f40 tx=0x7fbde4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:21.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.690+0000 7fbdddffb700 1 -- 172.21.15.83:0/4195857136 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbde00557a0 con 0x7fbde815b370 2023-11-26T07:15:21.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.912+0000 7fbdef650700 1 -- 172.21.15.83:0/4195857136 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol95121", "target": ["mon-mgr", ""]}) v1 -- 0x7fbde809af80 con 0x7fbdcc06b110 2023-11-26T07:15:21.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.927+0000 7fbdddffb700 1 -- 172.21.15.83:0/4195857136 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbde809af80 con 0x7fbdcc06b110 2023-11-26T07:15:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 -- 172.21.15.83:0/4195857136 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdcc06b110 msgr2=0x7fbdcc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdcc06b110 0x7fbdcc06d5d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbde4003f40 tx=0x7fbde4009040 comp rx=0 tx=0).stop 2023-11-26T07:15:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 -- 172.21.15.83:0/4195857136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde815b370 msgr2=0x7fbde80b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde815b370 0x7fbde80b1920 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbde0015040 tx=0x7fbde0004ad0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 -- 172.21.15.83:0/4195857136 shutdown_connections 2023-11-26T07:15:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde815b370 0x7fbde80b1920 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbde80b4370 0x7fbde814b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbdcc06b110 0x7fbdcc06d5d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 --2- 172.21.15.83:0/4195857136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbde81587f0 0x7fbde809bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 -- 172.21.15.83:0/4195857136 >> 172.21.15.83:0/4195857136 conn(0x7fbde80d0bf0 msgr2=0x7fbde8061af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:21.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 -- 172.21.15.83:0/4195857136 shutdown_connections 2023-11-26T07:15:21.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:21.930+0000 7fbdef650700 1 -- 172.21.15.83:0/4195857136 wait complete. 2023-11-26T07:15:21.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:21.946 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-26T07:15:21.980 INFO:teuthology.orchestra.run.smithi083.stdout:2 2023-11-26T07:15:21.981 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:15:21.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:15:22.061 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:15:22.107 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:15:22.107 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:15:22.107 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:22.124 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:15:22.124 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:22.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.339+0000 7fa37d15c700 1 Processor -- start 2023-11-26T07:15:22.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.339+0000 7fa37d15c700 1 -- start start 2023-11-26T07:15:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.339+0000 7fa37d15c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815ddd0 0x7fa3780b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.339+0000 7fa37d15c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa37815e1b0 0x7fa3780cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.339+0000 7fa37d15c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3780b1d40 con 0x7fa3780d1b30 2023-11-26T07:15:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.339+0000 7fa37d15c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3780b28c0 con 0x7fa37815e1b0 2023-11-26T07:15:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.339+0000 7fa37d15c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37806c000 con 0x7fa37815ddd0 2023-11-26T07:15:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa376d9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815ddd0 0x7fa3780b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa37659c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa37815e1b0 0x7fa3780cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa37659c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa37815e1b0 0x7fa3780cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38284/0 (socket says 172.21.15.83:38284) 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa376d9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815ddd0 0x7fa3780b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:49290/0 (socket says 172.21.15.83:49290) 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa37659c700 1 -- 172.21.15.83:0/2147717801 learned_addr learned my addr 172.21.15.83:0/2147717801 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa37759e700 1 --1- 172.21.15.83:0/2147717801 >> v1:172.21.15.203:6789/0 conn(0x7fa3780d1b30 0x7fa378161c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:43490/0 (socket says 172.21.15.83:43490) 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa37659c700 1 -- 172.21.15.83:0/2147717801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815ddd0 msgr2=0x7fa3780b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa37659c700 1 --2- 172.21.15.83:0/2147717801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815ddd0 0x7fa3780b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa37659c700 1 -- 172.21.15.83:0/2147717801 >> v1:172.21.15.203:6789/0 conn(0x7fa3780d1b30 legacy=0x7fa378161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa37659c700 1 -- 172.21.15.83:0/2147717801 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3780b1f90 con 0x7fa37815e1b0 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa376d9d700 1 --2- 172.21.15.83:0/2147717801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815ddd0 0x7fa3780b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa37659c700 1 --2- 172.21.15.83:0/2147717801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa37815e1b0 0x7fa3780cf000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa3680098f0 tx=0x7fa368009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f002d7d0d5461dbe server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.340+0000 7fa375d9b700 1 -- 172.21.15.83:0/2147717801 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368011830 con 0x7fa37815e1b0 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.341+0000 7fa375d9b700 1 -- 172.21.15.83:0/2147717801 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa368011990 con 0x7fa37815e1b0 2023-11-26T07:15:22.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.341+0000 7fa375d9b700 1 -- 172.21.15.83:0/2147717801 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368011d20 con 0x7fa37815e1b0 2023-11-26T07:15:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.341+0000 7fa37d15c700 1 -- 172.21.15.83:0/2147717801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa37815e1b0 msgr2=0x7fa3780cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.341+0000 7fa37d15c700 1 --2- 172.21.15.83:0/2147717801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa37815e1b0 0x7fa3780cf000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa3680098f0 tx=0x7fa368009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.341+0000 7fa37d15c700 1 -- 172.21.15.83:0/2147717801 shutdown_connections 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.341+0000 7fa37d15c700 1 --2- 172.21.15.83:0/2147717801 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815ddd0 0x7fa3780b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.341+0000 7fa37d15c700 1 --2- 172.21.15.83:0/2147717801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa37815e1b0 0x7fa3780cf000 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.341+0000 7fa37d15c700 1 -- 172.21.15.83:0/2147717801 >> 172.21.15.83:0/2147717801 conn(0x7fa3780731a0 msgr2=0x7fa37809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.341+0000 7fa37d15c700 1 -- 172.21.15.83:0/2147717801 shutdown_connections 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.341+0000 7fa37d15c700 1 -- 172.21.15.83:0/2147717801 wait complete. 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.343+0000 7fa37d15c700 1 Processor -- start 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.343+0000 7fa37d15c700 1 -- start start 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.343+0000 7fa37d15c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3780d1b30 0x7fa37814cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.343+0000 7fa37d15c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa37815ddd0 0x7fa37815f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.343+0000 7fa37d15c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815e1b0 0x7fa37815fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.343+0000 7fa37d15c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37814dba0 con 0x7fa37815ddd0 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.343+0000 7fa37d15c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37814d300 con 0x7fa3780d1b30 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.343+0000 7fa37d15c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37814d5e0 con 0x7fa37815e1b0 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.343+0000 7fa37659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa37815ddd0 0x7fa37815f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.343+0000 7fa37659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa37815ddd0 0x7fa37815f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38840/0 (socket says 172.21.15.83:38840) 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.343+0000 7fa37659c700 1 -- 172.21.15.83:0/3596033145 learned_addr learned my addr 172.21.15.83:0/3596033145 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa37759e700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815e1b0 0x7fa37815fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa376d9d700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3780d1b30 0x7fa37814cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa37659c700 1 -- 172.21.15.83:0/3596033145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815e1b0 msgr2=0x7fa37815fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa37659c700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815e1b0 0x7fa37815fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa37659c700 1 -- 172.21.15.83:0/3596033145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3780d1b30 msgr2=0x7fa37814cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa37659c700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3780d1b30 0x7fa37814cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa37659c700 1 -- 172.21.15.83:0/3596033145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3600094a0 con 0x7fa37815ddd0 2023-11-26T07:15:22.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa37659c700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa37815ddd0 0x7fa37815f710 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa368000c00 tx=0x7fa368004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa36f7fe700 1 -- 172.21.15.83:0/3596033145 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368004530 con 0x7fa37815ddd0 2023-11-26T07:15:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa36f7fe700 1 -- 172.21.15.83:0/3596033145 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36801f070 con 0x7fa37815ddd0 2023-11-26T07:15:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa37d15c700 1 -- 172.21.15.83:0/3596033145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa368009550 con 0x7fa37815ddd0 2023-11-26T07:15:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa36f7fe700 1 -- 172.21.15.83:0/3596033145 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3680038e0 con 0x7fa37815ddd0 2023-11-26T07:15:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.344+0000 7fa37d15c700 1 -- 172.21.15.83:0/3596033145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3780b2ed0 con 0x7fa37815ddd0 2023-11-26T07:15:22.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.345+0000 7fa37d15c700 1 -- 172.21.15.83:0/3596033145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa378067f60 con 0x7fa37815ddd0 2023-11-26T07:15:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.345+0000 7fa36f7fe700 1 -- 172.21.15.83:0/3596033145 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 10) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa368021030 con 0x7fa37815ddd0 2023-11-26T07:15:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.346+0000 7fa36f7fe700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa35806b110 0x7fa35806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.346+0000 7fa36f7fe700 1 -- 172.21.15.83:0/3596033145 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa36808ad20 con 0x7fa37815ddd0 2023-11-26T07:15:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.346+0000 7fa376d9d700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa35806b110 0x7fa35806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.346+0000 7fa376d9d700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa35806b110 0x7fa35806d5d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa360009d00 tx=0x7fa360008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:22.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.350+0000 7fa36f7fe700 1 -- 172.21.15.83:0/3596033145 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa368055c20 con 0x7fa37815ddd0 2023-11-26T07:15:22.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.544+0000 7fa37d15c700 1 -- 172.21.15.83:0/3596033145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3780cf0e0 con 0x7fa37815ddd0 2023-11-26T07:15:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.545+0000 7fa36f7fe700 1 -- 172.21.15.83:0/3596033145 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fa368015090 con 0x7fa37815ddd0 2023-11-26T07:15:22.548 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:22.548 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":58,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:14:56.536771+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:14:53.044716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:14:53.904013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:22.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 -- 172.21.15.83:0/3596033145 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa35806b110 msgr2=0x7fa35806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa35806b110 0x7fa35806d5d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa360009d00 tx=0x7fa360008040 comp rx=0 tx=0).stop 2023-11-26T07:15:22.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 -- 172.21.15.83:0/3596033145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa37815ddd0 msgr2=0x7fa37815f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa37815ddd0 0x7fa37815f710 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa368000c00 tx=0x7fa368004750 comp rx=0 tx=0).stop 2023-11-26T07:15:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 -- 172.21.15.83:0/3596033145 shutdown_connections 2023-11-26T07:15:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa35806b110 0x7fa35806d5d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa37815e1b0 0x7fa37815fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3780d1b30 0x7fa37814cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 --2- 172.21.15.83:0/3596033145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa37815ddd0 0x7fa37815f710 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 -- 172.21.15.83:0/3596033145 >> 172.21.15.83:0/3596033145 conn(0x7fa3780731a0 msgr2=0x7fa37806ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 -- 172.21.15.83:0/3596033145 shutdown_connections 2023-11-26T07:15:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.547+0000 7fa37d15c700 1 -- 172.21.15.83:0/3596033145 wait complete. 2023-11-26T07:15:22.562 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:14:53.044716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:14:53.904013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:15:22.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:15:22.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.687+0000 7f4212b85700 1 Processor -- start 2023-11-26T07:15:22.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.687+0000 7f4212b85700 1 -- start start 2023-11-26T07:15:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.688+0000 7f4212b85700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 0x7f420c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.688+0000 7f4212b85700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f420c15b370 0x7f420c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.688+0000 7f4212b85700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f420c06b530 con 0x7f420c0b3c60 2023-11-26T07:15:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.688+0000 7f4212b85700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420c0ae5e0 con 0x7f420c15b370 2023-11-26T07:15:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.688+0000 7f4212b85700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c15be30 con 0x7f420c1587f0 2023-11-26T07:15:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.688+0000 7f4210921700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 0x7f420c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.688+0000 7f420bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f420c15b370 0x7f420c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.688+0000 7f4211122700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f420c0b3c60 0x7f420c0af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:43504/0 (socket says 172.21.15.83:43504) 2023-11-26T07:15:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.688+0000 7f420bfff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f420c15b370 0x7f420c15b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38298/0 (socket says 172.21.15.83:38298) 2023-11-26T07:15:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.688+0000 7f4210921700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 0x7f420c06aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:49304/0 (socket says 172.21.15.83:49304) 2023-11-26T07:15:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.688+0000 7f4211122700 1 -- 172.21.15.83:0/3506013783 learned_addr learned my addr 172.21.15.83:0/3506013783 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.689+0000 7f420bfff700 1 -- 172.21.15.83:0/3506013783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 msgr2=0x7f420c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.689+0000 7f420bfff700 1 --2- 172.21.15.83:0/3506013783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 0x7f420c06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.689+0000 7f420bfff700 1 -- 172.21.15.83:0/3506013783 >> v1:172.21.15.203:6789/0 conn(0x7f420c0b3c60 legacy=0x7f420c0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.689+0000 7f420b7fe700 1 -- 172.21.15.83:0/3506013783 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063127582 0 0) 0x7f420c06b530 con 0x7f420c0b3c60 2023-11-26T07:15:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.689+0000 7f420bfff700 1 -- 172.21.15.83:0/3506013783 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c15c110 con 0x7f420c15b370 2023-11-26T07:15:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.689+0000 7f4210921700 1 --2- 172.21.15.83:0/3506013783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 0x7f420c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:15:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.689+0000 7f420bfff700 1 --2- 172.21.15.83:0/3506013783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c15b370 0x7f420c15b7f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009840 tx=0x7f41f8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7596f2f4bbf75aaf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.689+0000 7f420b7fe700 1 -- 172.21.15.83:0/3506013783 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f8011830 con 0x7f420c15b370 2023-11-26T07:15:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.689+0000 7f420b7fe700 1 -- 172.21.15.83:0/3506013783 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f8011990 con 0x7f420c15b370 2023-11-26T07:15:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.689+0000 7f420b7fe700 1 -- 172.21.15.83:0/3506013783 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f8011d20 con 0x7f420c15b370 2023-11-26T07:15:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.690+0000 7f4212b85700 1 -- 172.21.15.83:0/3506013783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c15b370 msgr2=0x7f420c15b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.690+0000 7f4212b85700 1 --2- 172.21.15.83:0/3506013783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c15b370 0x7f420c15b7f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009840 tx=0x7f41f8009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.690+0000 7f4212b85700 1 -- 172.21.15.83:0/3506013783 shutdown_connections 2023-11-26T07:15:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.690+0000 7f4212b85700 1 --2- 172.21.15.83:0/3506013783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 0x7f420c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.690+0000 7f4212b85700 1 --2- 172.21.15.83:0/3506013783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c15b370 0x7f420c15b7f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.690+0000 7f4212b85700 1 -- 172.21.15.83:0/3506013783 >> 172.21.15.83:0/3506013783 conn(0x7f420c0d0bf0 msgr2=0x7f420c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.690+0000 7f4212b85700 1 -- 172.21.15.83:0/3506013783 shutdown_connections 2023-11-26T07:15:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.690+0000 7f4212b85700 1 -- 172.21.15.83:0/3506013783 wait complete. 2023-11-26T07:15:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.691+0000 7f4212b85700 1 Processor -- start 2023-11-26T07:15:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.691+0000 7f4212b85700 1 -- start start 2023-11-26T07:15:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4212b85700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c0b3c60 0x7f420c1546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4212b85700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 0x7f420c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4212b85700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c15b370 0x7f420c0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4212b85700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c0afe40 con 0x7f420c0b3c60 2023-11-26T07:15:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4212b85700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420c1f47e0 con 0x7f420c15b370 2023-11-26T07:15:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4212b85700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c1f4ac0 con 0x7f420c1587f0 2023-11-26T07:15:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4210921700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c0b3c60 0x7f420c1546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4210921700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c0b3c60 0x7f420c1546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38848/0 (socket says 172.21.15.83:38848) 2023-11-26T07:15:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4210921700 1 -- 172.21.15.83:0/3891849755 learned_addr learned my addr 172.21.15.83:0/3891849755 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f420bfff700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 0x7f420c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4211122700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c15b370 0x7f420c0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4210921700 1 -- 172.21.15.83:0/3891849755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 msgr2=0x7f420c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4210921700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 0x7f420c073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4210921700 1 -- 172.21.15.83:0/3891849755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c15b370 msgr2=0x7f420c0744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4210921700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c15b370 0x7f420c0744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.692+0000 7f4210921700 1 -- 172.21.15.83:0/3891849755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000094a0 con 0x7f420c0b3c60 2023-11-26T07:15:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.693+0000 7f4210921700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c0b3c60 0x7f420c1546b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f420000ebe0 tx=0x7f420000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.693+0000 7f42097fa700 1 -- 172.21.15.83:0/3891849755 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420000c850 con 0x7f420c0b3c60 2023-11-26T07:15:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.693+0000 7f42097fa700 1 -- 172.21.15.83:0/3891849755 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420000c9b0 con 0x7f420c0b3c60 2023-11-26T07:15:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.693+0000 7f4212b85700 1 -- 172.21.15.83:0/3891849755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f80094a0 con 0x7f420c0b3c60 2023-11-26T07:15:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.693+0000 7f42097fa700 1 -- 172.21.15.83:0/3891849755 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420000cd40 con 0x7f420c0b3c60 2023-11-26T07:15:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.693+0000 7f4212b85700 1 -- 172.21.15.83:0/3891849755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420c1f5080 con 0x7f420c0b3c60 2023-11-26T07:15:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.694+0000 7f4212b85700 1 -- 172.21.15.83:0/3891849755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420c068fb0 con 0x7f420c0b3c60 2023-11-26T07:15:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.694+0000 7f42097fa700 1 -- 172.21.15.83:0/3891849755 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f4200013470 con 0x7f420c0b3c60 2023-11-26T07:15:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.695+0000 7f42097fa700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f41f006b230 0x7f41f006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.695+0000 7f420bfff700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f41f006b230 0x7f41f006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:22.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.695+0000 7f42097fa700 1 -- 172.21.15.83:0/3891849755 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4200089ba0 con 0x7f420c0b3c60 2023-11-26T07:15:22.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.695+0000 7f420bfff700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f41f006b230 0x7f41f006d6f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f41f80090b0 tx=0x7f41f8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:22.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.699+0000 7f42097fa700 1 -- 172.21.15.83:0/3891849755 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4200054840 con 0x7f420c0b3c60 2023-11-26T07:15:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.886+0000 7f4212b85700 1 -- 172.21.15.83:0/3891849755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f420c06bc00 con 0x7f420c0b3c60 2023-11-26T07:15:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.886+0000 7f42097fa700 1 -- 172.21.15.83:0/3891849755 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f420000cea0 con 0x7f420c0b3c60 2023-11-26T07:15:22.890 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:15:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 -- 172.21.15.83:0/3891849755 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f41f006b230 msgr2=0x7f41f006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f41f006b230 0x7f41f006d6f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f41f80090b0 tx=0x7f41f8008030 comp rx=0 tx=0).stop 2023-11-26T07:15:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 -- 172.21.15.83:0/3891849755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c0b3c60 msgr2=0x7f420c1546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c0b3c60 0x7f420c1546b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f420000ebe0 tx=0x7f420000eef0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 -- 172.21.15.83:0/3891849755 shutdown_connections 2023-11-26T07:15:22.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f420c15b370 0x7f420c0744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c0b3c60 0x7f420c1546b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f41f006b230 0x7f41f006d6f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 --2- 172.21.15.83:0/3891849755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f420c1587f0 0x7f420c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:22.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 -- 172.21.15.83:0/3891849755 >> 172.21.15.83:0/3891849755 conn(0x7f420c0d0bf0 msgr2=0x7f420c06be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:22.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 -- 172.21.15.83:0/3891849755 shutdown_connections 2023-11-26T07:15:22.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:22.888+0000 7f4212b85700 1 -- 172.21.15.83:0/3891849755 wait complete. 2023-11-26T07:15:22.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:15:23.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.023+0000 7fbe55b1a700 1 Processor -- start 2023-11-26T07:15:23.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.023+0000 7fbe55b1a700 1 -- start start 2023-11-26T07:15:23.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.024+0000 7fbe55b1a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbe5015bf50 0x7fbe500ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.024+0000 7fbe55b1a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe500ce830 0x7fbe500cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.024+0000 7fbe55b1a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe500d0140 con 0x7fbe500cf1f0 2023-11-26T07:15:23.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.024+0000 7fbe55b1a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5015c330 con 0x7fbe5015bf50 2023-11-26T07:15:23.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.024+0000 7fbe55b1a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500d03e0 con 0x7fbe500ce830 2023-11-26T07:15:23.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4effd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe500ce830 0x7fbe500cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4f7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbe5015bf50 0x7fbe500ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4ffff700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fbe500cf1f0 0x7fbe500c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:43516/0 (socket says 172.21.15.83:43516) 2023-11-26T07:15:23.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4ffff700 1 -- 172.21.15.83:0/2169831669 learned_addr learned my addr 172.21.15.83:0/2169831669 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:23.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4effd700 1 -- 172.21.15.83:0/2169831669 >> v2:172.21.15.203:3300/0 conn(0x7fbe5015bf50 msgr2=0x7fbe500ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:23.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4effd700 1 --2- 172.21.15.83:0/2169831669 >> v2:172.21.15.203:3300/0 conn(0x7fbe5015bf50 0x7fbe500ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4effd700 1 -- 172.21.15.83:0/2169831669 >> v1:172.21.15.203:6789/0 conn(0x7fbe500cf1f0 legacy=0x7fbe500c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4effd700 1 -- 172.21.15.83:0/2169831669 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500d06c0 con 0x7fbe500ce830 2023-11-26T07:15:23.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4f7fe700 1 --2- 172.21.15.83:0/2169831669 >> v2:172.21.15.203:3300/0 conn(0x7fbe5015bf50 0x7fbe500ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:15:23.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4effd700 1 --2- 172.21.15.83:0/2169831669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe500ce830 0x7fbe500cecb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009840 tx=0x7fbe40009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa789ae4fe32978 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:23.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4e7fc700 1 -- 172.21.15.83:0/2169831669 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40011830 con 0x7fbe500ce830 2023-11-26T07:15:23.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4e7fc700 1 -- 172.21.15.83:0/2169831669 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe40011990 con 0x7fbe500ce830 2023-11-26T07:15:23.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.025+0000 7fbe4e7fc700 1 -- 172.21.15.83:0/2169831669 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40011d20 con 0x7fbe500ce830 2023-11-26T07:15:23.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.026+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2169831669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe500ce830 msgr2=0x7fbe500cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.026+0000 7fbe55b1a700 1 --2- 172.21.15.83:0/2169831669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe500ce830 0x7fbe500cecb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009840 tx=0x7fbe40009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:23.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.026+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2169831669 shutdown_connections 2023-11-26T07:15:23.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.026+0000 7fbe55b1a700 1 --2- 172.21.15.83:0/2169831669 >> v2:172.21.15.203:3300/0 conn(0x7fbe5015bf50 0x7fbe500ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.026+0000 7fbe55b1a700 1 --2- 172.21.15.83:0/2169831669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe500ce830 0x7fbe500cecb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.026+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2169831669 >> 172.21.15.83:0/2169831669 conn(0x7fbe500b3780 msgr2=0x7fbe500b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:23.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.026+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2169831669 shutdown_connections 2023-11-26T07:15:23.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.026+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2169831669 wait complete. 2023-11-26T07:15:23.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.027+0000 7fbe55b1a700 1 Processor -- start 2023-11-26T07:15:23.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.027+0000 7fbe55b1a700 1 -- start start 2023-11-26T07:15:23.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.028+0000 7fbe55b1a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe500ce830 0x7fbe501f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.028+0000 7fbe55b1a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe500cf1f0 0x7fbe500b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.028+0000 7fbe55b1a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe5015bf50 0x7fbe500cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.028+0000 7fbe55b1a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50142d40 con 0x7fbe5015bf50 2023-11-26T07:15:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.028+0000 7fbe55b1a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe501531e0 con 0x7fbe500ce830 2023-11-26T07:15:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.028+0000 7fbe55b1a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50142130 con 0x7fbe500cf1f0 2023-11-26T07:15:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.028+0000 7fbe4ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe5015bf50 0x7fbe500cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.028+0000 7fbe4ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe5015bf50 0x7fbe500cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38864/0 (socket says 172.21.15.83:38864) 2023-11-26T07:15:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.028+0000 7fbe4ffff700 1 -- 172.21.15.83:0/2307459235 learned_addr learned my addr 172.21.15.83:0/2307459235 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.028+0000 7fbe4effd700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe500cf1f0 0x7fbe500b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.028+0000 7fbe4f7fe700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe500ce830 0x7fbe501f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe4ffff700 1 -- 172.21.15.83:0/2307459235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe500cf1f0 msgr2=0x7fbe500b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe4ffff700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe500cf1f0 0x7fbe500b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe4ffff700 1 -- 172.21.15.83:0/2307459235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe500ce830 msgr2=0x7fbe501f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe4ffff700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe500ce830 0x7fbe501f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe4ffff700 1 -- 172.21.15.83:0/2307459235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380094a0 con 0x7fbe5015bf50 2023-11-26T07:15:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe4f7fe700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe500ce830 0x7fbe501f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe4ffff700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe5015bf50 0x7fbe500cb830 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800cbc0 tx=0x7fbe4800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe47fff700 1 -- 172.21.15.83:0/2307459235 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4800aaa0 con 0x7fbe5015bf50 2023-11-26T07:15:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe47fff700 1 -- 172.21.15.83:0/2307459235 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4800ac00 con 0x7fbe5015bf50 2023-11-26T07:15:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2307459235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe400094a0 con 0x7fbe5015bf50 2023-11-26T07:15:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe47fff700 1 -- 172.21.15.83:0/2307459235 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48012400 con 0x7fbe5015bf50 2023-11-26T07:15:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.029+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2307459235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe501426f0 con 0x7fbe5015bf50 2023-11-26T07:15:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.030+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2307459235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe50068fb0 con 0x7fbe5015bf50 2023-11-26T07:15:23.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.030+0000 7fbe47fff700 1 -- 172.21.15.83:0/2307459235 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fbe4801b3b0 con 0x7fbe5015bf50 2023-11-26T07:15:23.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.031+0000 7fbe47fff700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbe3006b230 0x7fbe3006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.031+0000 7fbe47fff700 1 -- 172.21.15.83:0/2307459235 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe480897e0 con 0x7fbe5015bf50 2023-11-26T07:15:23.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.031+0000 7fbe4f7fe700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbe3006b230 0x7fbe3006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.031+0000 7fbe4f7fe700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbe3006b230 0x7fbe3006d6f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009d20 tx=0x7fbe38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:23.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.035+0000 7fbe47fff700 1 -- 172.21.15.83:0/2307459235 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbe48054430 con 0x7fbe5015bf50 2023-11-26T07:15:23.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.224+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2307459235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbe50142e80 con 0x7fbe5015bf50 2023-11-26T07:15:23.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.224+0000 7fbe47fff700 1 -- 172.21.15.83:0/2307459235 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbe48057a50 con 0x7fbe5015bf50 2023-11-26T07:15:23.227 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:15:23.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2307459235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbe3006b230 msgr2=0x7fbe3006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbe3006b230 0x7fbe3006d6f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009d20 tx=0x7fbe38008040 comp rx=0 tx=0).stop 2023-11-26T07:15:23.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2307459235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe5015bf50 msgr2=0x7fbe500cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe5015bf50 0x7fbe500cb830 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800cbc0 tx=0x7fbe4800ced0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2307459235 shutdown_connections 2023-11-26T07:15:23.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe5015bf50 0x7fbe500cb830 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe500ce830 0x7fbe501f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbe3006b230 0x7fbe3006d6f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 --2- 172.21.15.83:0/2307459235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe500cf1f0 0x7fbe500b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2307459235 >> 172.21.15.83:0/2307459235 conn(0x7fbe500b3780 msgr2=0x7fbe500d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:23.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2307459235 shutdown_connections 2023-11-26T07:15:23.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.226+0000 7fbe55b1a700 1 -- 172.21.15.83:0/2307459235 wait complete. 2023-11-26T07:15:23.241 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:15:23.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.361+0000 7f3efdf82700 1 Processor -- start 2023-11-26T07:15:23.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.362+0000 7f3efdf82700 1 -- start start 2023-11-26T07:15:23.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.362+0000 7f3efdf82700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3ef815e160 0x7f3ef80b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.362+0000 7f3efdf82700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef80b3c60 con 0x7f3ef80746a0 2023-11-26T07:15:23.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.362+0000 7f3efdf82700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef80b58c0 con 0x7f3ef80afc90 2023-11-26T07:15:23.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.362+0000 7f3efdf82700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef80b4500 con 0x7f3ef815e160 2023-11-26T07:15:23.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.362+0000 7f3ef7fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3ef80746a0 0x7f3ef806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:58912/0 (socket says 172.21.15.83:58912) 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.362+0000 7f3ef7fff700 1 -- 172.21.15.83:0/1252653057 learned_addr learned my addr 172.21.15.83:0/1252653057 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef77fe700 1 --2- 172.21.15.83:0/1252653057 >> v2:172.21.15.203:3300/0 conn(0x7f3ef815e160 0x7f3ef80b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872229810 0 0) 0x7f3ef80b3c60 con 0x7f3ef80746a0 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ee00035b0 con 0x7f3ef80746a0 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1402158640 0 0) 0x7f3ee00035b0 con 0x7f3ef80746a0 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 >> v2:172.21.15.203:3300/0 conn(0x7f3ef815e160 msgr2=0x7f3ef80b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 --2- 172.21.15.83:0/1252653057 >> v2:172.21.15.203:3300/0 conn(0x7f3ef815e160 0x7f3ef80b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 >> v1:172.21.15.203:6789/0 conn(0x7f3ef80afc90 legacy=0x7f3ef8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef80b0140 con 0x7f3ef80746a0 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f3ef00019f0 con 0x7f3ef80746a0 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 >> v1:172.21.15.83:6789/0 conn(0x7f3ef80746a0 legacy=0x7f3ef806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 0x7f3ee0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ee0004850 0x7f3ee0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ee0005210 0x7f3ee000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef80b4500 con 0x7f3ee0003a00 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef77fe700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 0x7f3ee0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee000c0b0 con 0x7f3ee0004850 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee000c270 con 0x7f3ee0005210 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef6ffd700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ee0004850 0x7f3ee0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef7fff700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ee0005210 0x7f3ee000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef77fe700 1 -- 172.21.15.83:0/1252653057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ee0005210 msgr2=0x7f3ee000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef77fe700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ee0005210 0x7f3ee000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef77fe700 1 -- 172.21.15.83:0/1252653057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ee0004850 msgr2=0x7f3ee0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:23.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef77fe700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ee0004850 0x7f3ee0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef77fe700 1 -- 172.21.15.83:0/1252653057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee00035b0 con 0x7f3ee0003a00 2023-11-26T07:15:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef6ffd700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ee0004850 0x7f3ee0004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef7fff700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ee0005210 0x7f3ee000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.363+0000 7f3ef7fff700 1 -- 172.21.15.83:0/1252653057 reap_dead start 2023-11-26T07:15:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.364+0000 7f3ef77fe700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 0x7f3ee0003e20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009b60 tx=0x7f3ee0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c859fb397594691f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.364+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00e6c0 con 0x7f3ee0003a00 2023-11-26T07:15:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.364+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec00e820 con 0x7f3ee0003a00 2023-11-26T07:15:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.364+0000 7f3ef67fc700 1 -- 172.21.15.83:0/1252653057 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00eb90 con 0x7f3ee0003a00 2023-11-26T07:15:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.364+0000 7f3efdf82700 1 -- 172.21.15.83:0/1252653057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 msgr2=0x7f3ee0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.364+0000 7f3efdf82700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 0x7f3ee0003e20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009b60 tx=0x7f3ee0005880 comp rx=0 tx=0).stop 2023-11-26T07:15:23.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.364+0000 7f3efdf82700 1 -- 172.21.15.83:0/1252653057 shutdown_connections 2023-11-26T07:15:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.364+0000 7f3efdf82700 1 --2- 172.21.15.83:0/1252653057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 0x7f3ee0003e20 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.364+0000 7f3efdf82700 1 -- 172.21.15.83:0/1252653057 >> 172.21.15.83:0/1252653057 conn(0x7f3ef80d0bf0 msgr2=0x7f3ef80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.364+0000 7f3efdf82700 1 -- 172.21.15.83:0/1252653057 shutdown_connections 2023-11-26T07:15:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.364+0000 7f3efdf82700 1 -- 172.21.15.83:0/1252653057 wait complete. 2023-11-26T07:15:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.365+0000 7f3efdf82700 1 Processor -- start 2023-11-26T07:15:23.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.366+0000 7f3efdf82700 1 -- start start 2023-11-26T07:15:23.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.366+0000 7f3efdf82700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 0x7f3ef81f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.366+0000 7f3efdf82700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80d0050 0x7f3ef81f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.366+0000 7f3efdf82700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef81f77d0 0x7f3ef80b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.366+0000 7f3efdf82700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef81611c0 con 0x7f3ee0003a00 2023-11-26T07:15:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.366+0000 7f3efdf82700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef81f7e00 con 0x7f3ef81f77d0 2023-11-26T07:15:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.366+0000 7f3efdf82700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef81f80e0 con 0x7f3ef80d0050 2023-11-26T07:15:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 0x7f3ef81f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 0x7f3ef81f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38880/0 (socket says 172.21.15.83:38880) 2023-11-26T07:15:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef77fe700 1 -- 172.21.15.83:0/904992083 learned_addr learned my addr 172.21.15.83:0/904992083 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef6ffd700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80d0050 0x7f3ef81f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef7fff700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef81f77d0 0x7f3ef80b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef77fe700 1 -- 172.21.15.83:0/904992083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80d0050 msgr2=0x7f3ef81f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef77fe700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80d0050 0x7f3ef81f7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef77fe700 1 -- 172.21.15.83:0/904992083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef81f77d0 msgr2=0x7f3ef80b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef77fe700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef81f77d0 0x7f3ef80b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef77fe700 1 -- 172.21.15.83:0/904992083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef81f3d10 con 0x7f3ee0003a00 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef7fff700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef81f77d0 0x7f3ef80b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3ef77fe700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 0x7f3ef81f47f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c080 tx=0x7f3eec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3edffff700 1 -- 172.21.15.83:0/904992083 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00e6c0 con 0x7f3ee0003a00 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3edffff700 1 -- 172.21.15.83:0/904992083 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec003ad0 con 0x7f3ee0003a00 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3efdf82700 1 -- 172.21.15.83:0/904992083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef81f3ff0 con 0x7f3ee0003a00 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3edffff700 1 -- 172.21.15.83:0/904992083 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec003e80 con 0x7f3ee0003a00 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.367+0000 7f3efdf82700 1 -- 172.21.15.83:0/904992083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef80d04d0 con 0x7f3ee0003a00 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.368+0000 7f3efdf82700 1 -- 172.21.15.83:0/904992083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef81f4640 con 0x7f3ee0003a00 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.368+0000 7f3edffff700 1 -- 172.21.15.83:0/904992083 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f3eec013470 con 0x7f3ee0003a00 2023-11-26T07:15:23.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.369+0000 7f3edffff700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3ee406b230 0x7f3ee406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.369+0000 7f3edffff700 1 -- 172.21.15.83:0/904992083 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3eec08b600 con 0x7f3ee0003a00 2023-11-26T07:15:23.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.369+0000 7f3ef6ffd700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3ee406b230 0x7f3ee406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.369+0000 7f3ef6ffd700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3ee406b230 0x7f3ee406d6f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8000c00 tx=0x7f3ee8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:23.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.373+0000 7f3edffff700 1 -- 172.21.15.83:0/904992083 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3eec056450 con 0x7f3ee0003a00 2023-11-26T07:15:23.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.569+0000 7f3efdf82700 1 -- 172.21.15.83:0/904992083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ef8068fb0 con 0x7f3ee0003a00 2023-11-26T07:15:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.569+0000 7f3edffff700 1 -- 172.21.15.83:0/904992083 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3eec059a70 con 0x7f3ee0003a00 2023-11-26T07:15:23.572 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:15:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 -- 172.21.15.83:0/904992083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3ee406b230 msgr2=0x7f3ee406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3ee406b230 0x7f3ee406d6f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8000c00 tx=0x7f3ee8009040 comp rx=0 tx=0).stop 2023-11-26T07:15:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 -- 172.21.15.83:0/904992083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 msgr2=0x7f3ef81f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 0x7f3ef81f47f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c080 tx=0x7f3eec0047c0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 -- 172.21.15.83:0/904992083 shutdown_connections 2023-11-26T07:15:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef81f77d0 0x7f3ef80b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ee0003a00 0x7f3ef81f47f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3ee406b230 0x7f3ee406d6f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 --2- 172.21.15.83:0/904992083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80d0050 0x7f3ef81f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 -- 172.21.15.83:0/904992083 >> 172.21.15.83:0/904992083 conn(0x7f3ef80d0bf0 msgr2=0x7f3ef806e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 -- 172.21.15.83:0/904992083 shutdown_connections 2023-11-26T07:15:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.572+0000 7f3efdf82700 1 -- 172.21.15.83:0/904992083 wait complete. 2023-11-26T07:15:23.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:15:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.706+0000 7fae42956700 1 Processor -- start 2023-11-26T07:15:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.706+0000 7fae42956700 1 -- start start 2023-11-26T07:15:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.707+0000 7fae42956700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae3c15bdd0 0x7fae3c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.707+0000 7fae42956700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae3c0b3370 0x7fae3c0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.707+0000 7fae42956700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c0b2030 con 0x7fae3c0d1c70 2023-11-26T07:15:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.707+0000 7fae42956700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c0b2f50 con 0x7fae3c0b3370 2023-11-26T07:15:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.707+0000 7fae42956700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0b22d0 con 0x7fae3c15bdd0 2023-11-26T07:15:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.707+0000 7fae3b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae3c0b3370 0x7fae3c0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.707+0000 7fae3b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae3c0b3370 0x7fae3c0b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38890/0 (socket says 172.21.15.83:38890) 2023-11-26T07:15:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.707+0000 7fae3b7fe700 1 -- 172.21.15.83:0/35049568 learned_addr learned my addr 172.21.15.83:0/35049568 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.708+0000 7fae3b7fe700 1 -- 172.21.15.83:0/35049568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae3c15bdd0 msgr2=0x7fae3c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.708+0000 7fae3bfff700 1 --2- 172.21.15.83:0/35049568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae3c15bdd0 0x7fae3c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.708+0000 7fae3b7fe700 1 --2- 172.21.15.83:0/35049568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae3c15bdd0 0x7fae3c1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.708+0000 7fae3b7fe700 1 -- 172.21.15.83:0/35049568 >> v1:172.21.15.203:6789/0 conn(0x7fae3c0d1c70 legacy=0x7fae3c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.708+0000 7fae3b7fe700 1 -- 172.21.15.83:0/35049568 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c0b25b0 con 0x7fae3c0b3370 2023-11-26T07:15:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.708+0000 7fae3b7fe700 1 --2- 172.21.15.83:0/35049568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae3c0b3370 0x7fae3c0b1af0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098f0 tx=0x7fae2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa120ce785e7f78 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.708+0000 7fae3affd700 1 -- 172.21.15.83:0/35049568 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c004230 con 0x7fae3c0b3370 2023-11-26T07:15:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.708+0000 7fae3affd700 1 -- 172.21.15.83:0/35049568 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c004390 con 0x7fae3c0b3370 2023-11-26T07:15:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.708+0000 7fae3affd700 1 -- 172.21.15.83:0/35049568 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0036a0 con 0x7fae3c0b3370 2023-11-26T07:15:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.708+0000 7fae42956700 1 -- 172.21.15.83:0/35049568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae3c0b3370 msgr2=0x7fae3c0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.708+0000 7fae42956700 1 --2- 172.21.15.83:0/35049568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae3c0b3370 0x7fae3c0b1af0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098f0 tx=0x7fae2c009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.709+0000 7fae42956700 1 -- 172.21.15.83:0/35049568 shutdown_connections 2023-11-26T07:15:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.709+0000 7fae42956700 1 --2- 172.21.15.83:0/35049568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae3c15bdd0 0x7fae3c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.709+0000 7fae42956700 1 --2- 172.21.15.83:0/35049568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae3c0b3370 0x7fae3c0b1af0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.709+0000 7fae42956700 1 -- 172.21.15.83:0/35049568 >> 172.21.15.83:0/35049568 conn(0x7fae3c0731a0 msgr2=0x7fae3c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.709+0000 7fae42956700 1 -- 172.21.15.83:0/35049568 shutdown_connections 2023-11-26T07:15:23.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.709+0000 7fae42956700 1 -- 172.21.15.83:0/35049568 wait complete. 2023-11-26T07:15:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.710+0000 7fae42956700 1 Processor -- start 2023-11-26T07:15:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.710+0000 7fae42956700 1 -- start start 2023-11-26T07:15:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae42956700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae3c0b3370 0x7fae3c156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae42956700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae3c0d1c70 0x7fae3c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae42956700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae3c15bdd0 0x7fae3c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae42956700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c157350 con 0x7fae3c0b3370 2023-11-26T07:15:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae42956700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0b62a0 con 0x7fae3c15bdd0 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae42956700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0b5690 con 0x7fae3c0d1c70 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae3bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae3c0b3370 0x7fae3c156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae3bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae3c0b3370 0x7fae3c156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38894/0 (socket says 172.21.15.83:38894) 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae3bfff700 1 -- 172.21.15.83:0/2933100611 learned_addr learned my addr 172.21.15.83:0/2933100611 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae40ef3700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae3c15bdd0 0x7fae3c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae3b7fe700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae3c0d1c70 0x7fae3c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae3bfff700 1 -- 172.21.15.83:0/2933100611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae3c0d1c70 msgr2=0x7fae3c0b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae3bfff700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae3c0d1c70 0x7fae3c0b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae3bfff700 1 -- 172.21.15.83:0/2933100611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae3c15bdd0 msgr2=0x7fae3c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae3bfff700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae3c15bdd0 0x7fae3c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae3bfff700 1 -- 172.21.15.83:0/2933100611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c009550 con 0x7fae3c0b3370 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae3b7fe700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae3c0d1c70 0x7fae3c0b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae3bfff700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae3c0b3370 0x7fae3c156e10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c7f0 tx=0x7fae3000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae38ff9700 1 -- 172.21.15.83:0/2933100611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30011800 con 0x7fae3c0b3370 2023-11-26T07:15:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae38ff9700 1 -- 172.21.15.83:0/2933100611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae30011960 con 0x7fae3c0b3370 2023-11-26T07:15:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae38ff9700 1 -- 172.21.15.83:0/2933100611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30011cd0 con 0x7fae3c0b3370 2023-11-26T07:15:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae42956700 1 -- 172.21.15.83:0/2933100611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c0b5970 con 0x7fae3c0b3370 2023-11-26T07:15:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.711+0000 7fae42956700 1 -- 172.21.15.83:0/2933100611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c0b5f30 con 0x7fae3c0b3370 2023-11-26T07:15:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.712+0000 7fae42956700 1 -- 172.21.15.83:0/2933100611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c0b63e0 con 0x7fae3c0b3370 2023-11-26T07:15:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.713+0000 7fae38ff9700 1 -- 172.21.15.83:0/2933100611 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fae30010350 con 0x7fae3c0b3370 2023-11-26T07:15:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.713+0000 7fae38ff9700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b230 0x7fae2806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.713+0000 7fae38ff9700 1 -- 172.21.15.83:0/2933100611 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae30089700 con 0x7fae3c0b3370 2023-11-26T07:15:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.713+0000 7fae3b7fe700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b230 0x7fae2806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.714+0000 7fae3b7fe700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b230 0x7fae2806d6f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009130 tx=0x7fae2c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:23.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.718+0000 7fae38ff9700 1 -- 172.21.15.83:0/2933100611 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae300542e0 con 0x7fae3c0b3370 2023-11-26T07:15:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.925+0000 7fae42956700 1 -- 172.21.15.83:0/2933100611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fae3c067f60 con 0x7fae3c0b3370 2023-11-26T07:15:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.925+0000 7fae38ff9700 1 -- 172.21.15.83:0/2933100611 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7fae30057900 con 0x7fae3c0b3370 2023-11-26T07:15:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 -- 172.21.15.83:0/2933100611 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b230 msgr2=0x7fae2806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b230 0x7fae2806d6f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009130 tx=0x7fae2c008030 comp rx=0 tx=0).stop 2023-11-26T07:15:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 -- 172.21.15.83:0/2933100611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae3c0b3370 msgr2=0x7fae3c156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae3c0b3370 0x7fae3c156e10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c7f0 tx=0x7fae3000cb00 comp rx=0 tx=0).stop 2023-11-26T07:15:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 -- 172.21.15.83:0/2933100611 shutdown_connections 2023-11-26T07:15:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae3c15bdd0 0x7fae3c0cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae3c0b3370 0x7fae3c156e10 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fae2806b230 0x7fae2806d6f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 --2- 172.21.15.83:0/2933100611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae3c0d1c70 0x7fae3c0b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 -- 172.21.15.83:0/2933100611 >> 172.21.15.83:0/2933100611 conn(0x7fae3c0731a0 msgr2=0x7fae3c0b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 -- 172.21.15.83:0/2933100611 shutdown_connections 2023-11-26T07:15:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:23.927+0000 7fae42956700 1 -- 172.21.15.83:0/2933100611 wait complete. 2023-11-26T07:15:23.931 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:15:23.941 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:15:23.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:23.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:15:24.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:15:24.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:24.023 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:15:24.042 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:15:24.042 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:24.062 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:15:24.062 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:24.120 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:24.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.237+0000 7fc8276d9700 1 Processor -- start 2023-11-26T07:15:24.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.238+0000 7fc8276d9700 1 -- start start 2023-11-26T07:15:24.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.238+0000 7fc8276d9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8200ce4d0 0x7fc8200d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.238+0000 7fc8276d9700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8200d0150 con 0x7fc8200ce910 2023-11-26T07:15:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.238+0000 7fc8276d9700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8200cee50 con 0x7fc82015e080 2023-11-26T07:15:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.238+0000 7fc8276d9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82015e460 con 0x7fc8200ce4d0 2023-11-26T07:15:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.238+0000 7fc824c74700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8200ce4d0 0x7fc8200d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.239+0000 7fc824c74700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc8200ce4d0 0x7fc8200d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38908/0 (socket says 172.21.15.83:38908) 2023-11-26T07:15:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.239+0000 7fc824c74700 1 -- 172.21.15.83:0/2092969361 learned_addr learned my addr 172.21.15.83:0/2092969361 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.239+0000 7fc824c74700 1 -- 172.21.15.83:0/2092969361 >> v1:172.21.15.203:6789/0 conn(0x7fc82015e080 legacy=0x7fc8200cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.239+0000 7fc824c74700 1 -- 172.21.15.83:0/2092969361 >> v1:172.21.15.83:6789/0 conn(0x7fc8200ce910 legacy=0x7fc8200d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.239+0000 7fc824c74700 1 -- 172.21.15.83:0/2092969361 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8200cf130 con 0x7fc8200ce4d0 2023-11-26T07:15:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.239+0000 7fc824c74700 1 --2- 172.21.15.83:0/2092969361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8200ce4d0 0x7fc8200d1be0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0098f0 tx=0x7fc81c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9f08ccfb7c42fa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.239+0000 7fc817fff700 1 -- 172.21.15.83:0/2092969361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00b8a0 con 0x7fc8200ce4d0 2023-11-26T07:15:24.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.239+0000 7fc817fff700 1 -- 172.21.15.83:0/2092969361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c00ba00 con 0x7fc8200ce4d0 2023-11-26T07:15:24.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.239+0000 7fc817fff700 1 -- 172.21.15.83:0/2092969361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00bd90 con 0x7fc8200ce4d0 2023-11-26T07:15:24.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.240+0000 7fc8276d9700 1 -- 172.21.15.83:0/2092969361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8200ce4d0 msgr2=0x7fc8200d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.240+0000 7fc8276d9700 1 --2- 172.21.15.83:0/2092969361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8200ce4d0 0x7fc8200d1be0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0098f0 tx=0x7fc81c009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:24.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.240+0000 7fc8276d9700 1 -- 172.21.15.83:0/2092969361 shutdown_connections 2023-11-26T07:15:24.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.240+0000 7fc8276d9700 1 --2- 172.21.15.83:0/2092969361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8200ce4d0 0x7fc8200d1be0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.240+0000 7fc8276d9700 1 -- 172.21.15.83:0/2092969361 >> 172.21.15.83:0/2092969361 conn(0x7fc820073130 msgr2=0x7fc82009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:24.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.240+0000 7fc8276d9700 1 -- 172.21.15.83:0/2092969361 shutdown_connections 2023-11-26T07:15:24.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.240+0000 7fc8276d9700 1 -- 172.21.15.83:0/2092969361 wait complete. 2023-11-26T07:15:24.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.241+0000 7fc8276d9700 1 Processor -- start 2023-11-26T07:15:24.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.241+0000 7fc8276d9700 1 -- start start 2023-11-26T07:15:24.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc8276d9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8200ce4d0 0x7fc8200b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc8276d9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8200ce910 0x7fc8200b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc8276d9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82015e080 0x7fc8200b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc8276d9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8200c69c0 con 0x7fc82015e080 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc8276d9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8200b72b0 con 0x7fc8200ce4d0 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc8276d9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8201f41d0 con 0x7fc8200ce910 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc825c76700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82015e080 0x7fc8200b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc825c76700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82015e080 0x7fc8200b5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38916/0 (socket says 172.21.15.83:38916) 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc825c76700 1 -- 172.21.15.83:0/61542541 learned_addr learned my addr 172.21.15.83:0/61542541 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc824c74700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8200ce910 0x7fc8200b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc825475700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8200ce4d0 0x7fc8200b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc825c76700 1 -- 172.21.15.83:0/61542541 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8200ce910 msgr2=0x7fc8200b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc825c76700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8200ce910 0x7fc8200b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc825c76700 1 -- 172.21.15.83:0/61542541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8200ce4d0 msgr2=0x7fc8200b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc825c76700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8200ce4d0 0x7fc8200b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc825c76700 1 -- 172.21.15.83:0/61542541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c009550 con 0x7fc82015e080 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc825c76700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82015e080 0x7fc8200b5e00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc81800cc50 tx=0x7fc81800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc815ffb700 1 -- 172.21.15.83:0/61542541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818003c00 con 0x7fc82015e080 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc815ffb700 1 -- 172.21.15.83:0/61542541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc818003d60 con 0x7fc82015e080 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc8276d9700 1 -- 172.21.15.83:0/61542541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8200c6020 con 0x7fc82015e080 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.242+0000 7fc815ffb700 1 -- 172.21.15.83:0/61542541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800aab0 con 0x7fc82015e080 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.243+0000 7fc8276d9700 1 -- 172.21.15.83:0/61542541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8200c65e0 con 0x7fc82015e080 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.244+0000 7fc8276d9700 1 -- 172.21.15.83:0/61542541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8200c6c30 con 0x7fc82015e080 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.244+0000 7fc815ffb700 1 -- 172.21.15.83:0/61542541 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fc81800ac10 con 0x7fc82015e080 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.244+0000 7fc815ffb700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc80406b230 0x7fc80406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.244+0000 7fc815ffb700 1 -- 172.21.15.83:0/61542541 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc818089b70 con 0x7fc82015e080 2023-11-26T07:15:24.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.244+0000 7fc825475700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc80406b230 0x7fc80406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.245+0000 7fc825475700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc80406b230 0x7fc80406d6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc8200c8bb0 tx=0x7fc810009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:24.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.249+0000 7fc815ffb700 1 -- 172.21.15.83:0/61542541 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8180547a0 con 0x7fc82015e080 2023-11-26T07:15:24.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.441+0000 7fc8276d9700 1 -- 172.21.15.83:0/61542541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc820067f60 con 0x7fc82015e080 2023-11-26T07:15:24.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.442+0000 7fc815ffb700 1 -- 172.21.15.83:0/61542541 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fc818057dc0 con 0x7fc82015e080 2023-11-26T07:15:24.445 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:24.445 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":58,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:14:56.536771+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:14:53.044716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:14:53.904013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 -- 172.21.15.83:0/61542541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc80406b230 msgr2=0x7fc80406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc80406b230 0x7fc80406d6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc8200c8bb0 tx=0x7fc810009040 comp rx=0 tx=0).stop 2023-11-26T07:15:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 -- 172.21.15.83:0/61542541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82015e080 msgr2=0x7fc8200b5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82015e080 0x7fc8200b5e00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc81800cc50 tx=0x7fc81800cf60 comp rx=0 tx=0).stop 2023-11-26T07:15:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 -- 172.21.15.83:0/61542541 shutdown_connections 2023-11-26T07:15:24.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc82015e080 0x7fc8200b5e00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8200ce4d0 0x7fc8200b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc80406b230 0x7fc80406d6f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 --2- 172.21.15.83:0/61542541 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8200ce910 0x7fc8200b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 -- 172.21.15.83:0/61542541 >> 172.21.15.83:0/61542541 conn(0x7fc820073130 msgr2=0x7fc82015c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 -- 172.21.15.83:0/61542541 shutdown_connections 2023-11-26T07:15:24.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.444+0000 7fc8276d9700 1 -- 172.21.15.83:0/61542541 wait complete. 2023-11-26T07:15:24.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:14:53.044716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:14:53.904013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:15:24.459 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:15:24.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.581+0000 7f79d77ee700 1 Processor -- start 2023-11-26T07:15:24.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.581+0000 7f79d77ee700 1 -- start start 2023-11-26T07:15:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d77ee700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79d015e160 0x7f79d00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d77ee700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d00afc90 0x7f79d0073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d77ee700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d00b3c60 con 0x7f79d00744d0 2023-11-26T07:15:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d77ee700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d00b4500 con 0x7f79d015e160 2023-11-26T07:15:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d77ee700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d0074960 con 0x7f79d00afc90 2023-11-26T07:15:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d558a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79d015e160 0x7f79d00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d558a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79d015e160 0x7f79d00b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38920/0 (socket says 172.21.15.83:38920) 2023-11-26T07:15:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d558a700 1 -- 172.21.15.83:0/2225202113 learned_addr learned my addr 172.21.15.83:0/2225202113 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d4d89700 1 --2- 172.21.15.83:0/2225202113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d00afc90 0x7f79d0073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d558a700 1 -- 172.21.15.83:0/2225202113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d00afc90 msgr2=0x7f79d0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d558a700 1 --2- 172.21.15.83:0/2225202113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d00afc90 0x7f79d0073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d558a700 1 -- 172.21.15.83:0/2225202113 >> v1:172.21.15.203:6789/0 conn(0x7f79d00744d0 legacy=0x7f79d006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.582+0000 7f79d558a700 1 -- 172.21.15.83:0/2225202113 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d00b58c0 con 0x7f79d015e160 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79d558a700 1 --2- 172.21.15.83:0/2225202113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d015e160 0x7f79d00b5da0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f79c00098f0 tx=0x7f79c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96cfefec001b2a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79c7fff700 1 -- 172.21.15.83:0/2225202113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0004230 con 0x7f79d015e160 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79c7fff700 1 -- 172.21.15.83:0/2225202113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c0004390 con 0x7f79d015e160 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79c7fff700 1 -- 172.21.15.83:0/2225202113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c00036a0 con 0x7f79d015e160 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79d77ee700 1 -- 172.21.15.83:0/2225202113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d015e160 msgr2=0x7f79d00b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79d77ee700 1 --2- 172.21.15.83:0/2225202113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d015e160 0x7f79d00b5da0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f79c00098f0 tx=0x7f79c0009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79d77ee700 1 -- 172.21.15.83:0/2225202113 shutdown_connections 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79d77ee700 1 --2- 172.21.15.83:0/2225202113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d015e160 0x7f79d00b5da0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79d77ee700 1 --2- 172.21.15.83:0/2225202113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d00afc90 0x7f79d0073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79d77ee700 1 -- 172.21.15.83:0/2225202113 >> 172.21.15.83:0/2225202113 conn(0x7f79d00d0bf0 msgr2=0x7f79d00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79d77ee700 1 -- 172.21.15.83:0/2225202113 shutdown_connections 2023-11-26T07:15:24.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.583+0000 7f79d77ee700 1 -- 172.21.15.83:0/2225202113 wait complete. 2023-11-26T07:15:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.584+0000 7f79d77ee700 1 Processor -- start 2023-11-26T07:15:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.585+0000 7f79d77ee700 1 -- start start 2023-11-26T07:15:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.585+0000 7f79d77ee700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d00744d0 0x7f79d014e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.585+0000 7f79d77ee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d00afc90 0x7f79d014d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.585+0000 7f79d77ee700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d015e160 0x7f79d014a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.585+0000 7f79d77ee700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d01f38c0 con 0x7f79d00afc90 2023-11-26T07:15:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.585+0000 7f79d77ee700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d014dac0 con 0x7f79d00744d0 2023-11-26T07:15:24.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.585+0000 7f79d77ee700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d014ac60 con 0x7f79d015e160 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.585+0000 7f79d4d89700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d00afc90 0x7f79d014d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.585+0000 7f79d4d89700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d00afc90 0x7f79d014d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38928/0 (socket says 172.21.15.83:38928) 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.585+0000 7f79d4d89700 1 -- 172.21.15.83:0/591884661 learned_addr learned my addr 172.21.15.83:0/591884661 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79d558a700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d00744d0 0x7f79d014e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79d5d8b700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d015e160 0x7f79d014a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79d4d89700 1 -- 172.21.15.83:0/591884661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d015e160 msgr2=0x7f79d014a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79d4d89700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d015e160 0x7f79d014a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79d4d89700 1 -- 172.21.15.83:0/591884661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d00744d0 msgr2=0x7f79d014e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79d4d89700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d00744d0 0x7f79d014e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79d4d89700 1 -- 172.21.15.83:0/591884661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c0009550 con 0x7f79d00afc90 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79d558a700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d00744d0 0x7f79d014e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79d4d89700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d00afc90 0x7f79d014d580 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00e810 tx=0x7f79cc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:24.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79c5ffb700 1 -- 172.21.15.83:0/591884661 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc013610 con 0x7f79d00afc90 2023-11-26T07:15:24.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79c5ffb700 1 -- 172.21.15.83:0/591884661 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc013770 con 0x7f79d00afc90 2023-11-26T07:15:24.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79d77ee700 1 -- 172.21.15.83:0/591884661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d014af40 con 0x7f79d00afc90 2023-11-26T07:15:24.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79c5ffb700 1 -- 172.21.15.83:0/591884661 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc013b20 con 0x7f79d00afc90 2023-11-26T07:15:24.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.586+0000 7f79d77ee700 1 -- 172.21.15.83:0/591884661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d014b500 con 0x7f79d00afc90 2023-11-26T07:15:24.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.587+0000 7f79d77ee700 1 -- 172.21.15.83:0/591884661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d0068fb0 con 0x7f79d00afc90 2023-11-26T07:15:24.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.588+0000 7f79c5ffb700 1 -- 172.21.15.83:0/591884661 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f79cc013c80 con 0x7f79d00afc90 2023-11-26T07:15:24.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.588+0000 7f79c5ffb700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79b406b230 0x7f79b406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.588+0000 7f79c5ffb700 1 -- 172.21.15.83:0/591884661 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79cc0899f0 con 0x7f79d00afc90 2023-11-26T07:15:24.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.588+0000 7f79d558a700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79b406b230 0x7f79b406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.588+0000 7f79d558a700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79b406b230 0x7f79b406d6f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f79d01f6f50 tx=0x7f79c0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:24.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.592+0000 7f79c5ffb700 1 -- 172.21.15.83:0/591884661 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79cc054590 con 0x7f79d00afc90 2023-11-26T07:15:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.783+0000 7f79d77ee700 1 -- 172.21.15.83:0/591884661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79d006c140 con 0x7f79d00afc90 2023-11-26T07:15:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.783+0000 7f79c5ffb700 1 -- 172.21.15.83:0/591884661 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79cc057bb0 con 0x7f79d00afc90 2023-11-26T07:15:24.786 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 -- 172.21.15.83:0/591884661 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79b406b230 msgr2=0x7f79b406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79b406b230 0x7f79b406d6f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f79d01f6f50 tx=0x7f79c0008030 comp rx=0 tx=0).stop 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 -- 172.21.15.83:0/591884661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d00afc90 msgr2=0x7f79d014d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d00afc90 0x7f79d014d580 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00e810 tx=0x7f79cc00ebd0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 -- 172.21.15.83:0/591884661 shutdown_connections 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f79b406b230 0x7f79b406d6f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d015e160 0x7f79d014a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d00744d0 0x7f79d014e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 --2- 172.21.15.83:0/591884661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79d00afc90 0x7f79d014d580 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 -- 172.21.15.83:0/591884661 >> 172.21.15.83:0/591884661 conn(0x7f79d00d0bf0 msgr2=0x7f79d006ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 -- 172.21.15.83:0/591884661 shutdown_connections 2023-11-26T07:15:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.785+0000 7f79d77ee700 1 -- 172.21.15.83:0/591884661 wait complete. 2023-11-26T07:15:24.799 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:15:24.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.919+0000 7f2abd2c3700 1 Processor -- start 2023-11-26T07:15:24.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.919+0000 7f2abd2c3700 1 -- start start 2023-11-26T07:15:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2abd2c3700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ab815e140 0x7f2ab80d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2abd2c3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ab80d39d0 0x7f2ab80d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2abd2c3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80b98c0 0x7f2ab80b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2abd2c3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab815e520 con 0x7f2ab80d39d0 2023-11-26T07:15:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2abd2c3700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab80d2930 con 0x7f2ab815e140 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2abd2c3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80d2c10 con 0x7f2ab80b98c0 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab67fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ab80d39d0 0x7f2ab80d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab67fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ab80d39d0 0x7f2ab80d4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38936/0 (socket says 172.21.15.83:38936) 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab67fc700 1 -- 172.21.15.83:0/3059514628 learned_addr learned my addr 172.21.15.83:0/3059514628 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab6ffd700 1 --2- 172.21.15.83:0/3059514628 >> v2:172.21.15.203:3300/0 conn(0x7f2ab815e140 0x7f2ab80d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab77fe700 1 --2- 172.21.15.83:0/3059514628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80b98c0 0x7f2ab80b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab67fc700 1 -- 172.21.15.83:0/3059514628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80b98c0 msgr2=0x7f2ab80b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab67fc700 1 --2- 172.21.15.83:0/3059514628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80b98c0 0x7f2ab80b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab67fc700 1 -- 172.21.15.83:0/3059514628 >> v2:172.21.15.203:3300/0 conn(0x7f2ab815e140 msgr2=0x7f2ab80d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab67fc700 1 --2- 172.21.15.83:0/3059514628 >> v2:172.21.15.203:3300/0 conn(0x7f2ab815e140 0x7f2ab80d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab67fc700 1 -- 172.21.15.83:0/3059514628 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab80d2ef0 con 0x7f2ab80d39d0 2023-11-26T07:15:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab6ffd700 1 --2- 172.21.15.83:0/3059514628 >> v2:172.21.15.203:3300/0 conn(0x7f2ab815e140 0x7f2ab80d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab77fe700 1 --2- 172.21.15.83:0/3059514628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80b98c0 0x7f2ab80b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.920+0000 7f2ab67fc700 1 --2- 172.21.15.83:0/3059514628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab80d39d0 0x7f2ab80d4400 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80d5030 tx=0x7f2aa8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7d08db83b04224 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2ab5ffb700 1 -- 172.21.15.83:0/3059514628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800b870 con 0x7f2ab80d39d0 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2ab5ffb700 1 -- 172.21.15.83:0/3059514628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa800b9d0 con 0x7f2ab80d39d0 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2ab5ffb700 1 -- 172.21.15.83:0/3059514628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800bd60 con 0x7f2ab80d39d0 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2abd2c3700 1 -- 172.21.15.83:0/3059514628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab80d39d0 msgr2=0x7f2ab80d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2abd2c3700 1 --2- 172.21.15.83:0/3059514628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab80d39d0 0x7f2ab80d4400 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80d5030 tx=0x7f2aa8009bd0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2abd2c3700 1 -- 172.21.15.83:0/3059514628 shutdown_connections 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2abd2c3700 1 --2- 172.21.15.83:0/3059514628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80b98c0 0x7f2ab80b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2abd2c3700 1 --2- 172.21.15.83:0/3059514628 >> v2:172.21.15.203:3300/0 conn(0x7f2ab815e140 0x7f2ab80d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2abd2c3700 1 --2- 172.21.15.83:0/3059514628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab80d39d0 0x7f2ab80d4400 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2abd2c3700 1 -- 172.21.15.83:0/3059514628 >> 172.21.15.83:0/3059514628 conn(0x7f2ab806b410 msgr2=0x7f2ab806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2abd2c3700 1 -- 172.21.15.83:0/3059514628 shutdown_connections 2023-11-26T07:15:24.927 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.921+0000 7f2abd2c3700 1 -- 172.21.15.83:0/3059514628 wait complete. 2023-11-26T07:15:24.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.923+0000 7f2abd2c3700 1 Processor -- start 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.923+0000 7f2abd2c3700 1 -- start start 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2abd2c3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab80b98c0 0x7f2ab81f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2abd2c3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80d39d0 0x7f2ab815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2abd2c3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ab815e140 0x7f2ab80cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2abd2c3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80cc3d0 con 0x7f2ab80b98c0 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2abd2c3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab81f4820 con 0x7f2ab815e140 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2abd2c3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80cb330 con 0x7f2ab80d39d0 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab80b98c0 0x7f2ab81f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab80b98c0 0x7f2ab81f42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38940/0 (socket says 172.21.15.83:38940) 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab6ffd700 1 -- 172.21.15.83:0/4115174383 learned_addr learned my addr 172.21.15.83:0/4115174383 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab77fe700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ab815e140 0x7f2ab80cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab67fc700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80d39d0 0x7f2ab815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab6ffd700 1 -- 172.21.15.83:0/4115174383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80d39d0 msgr2=0x7f2ab815c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab6ffd700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80d39d0 0x7f2ab815c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab6ffd700 1 -- 172.21.15.83:0/4115174383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ab815e140 msgr2=0x7f2ab80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab6ffd700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ab815e140 0x7f2ab80cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab6ffd700 1 -- 172.21.15.83:0/4115174383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa8009550 con 0x7f2ab80b98c0 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab77fe700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ab815e140 0x7f2ab80cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab67fc700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80d39d0 0x7f2ab815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.924+0000 7f2ab6ffd700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab80b98c0 0x7f2ab81f42e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00c7f0 tx=0x7f2aac00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.925+0000 7f2a9f7fe700 1 -- 172.21.15.83:0/4115174383 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac011800 con 0x7f2ab80b98c0 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.925+0000 7f2a9f7fe700 1 -- 172.21.15.83:0/4115174383 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac011960 con 0x7f2ab80b98c0 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.925+0000 7f2a9f7fe700 1 -- 172.21.15.83:0/4115174383 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac011d10 con 0x7f2ab80b98c0 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.925+0000 7f2abd2c3700 1 -- 172.21.15.83:0/4115174383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab80cb610 con 0x7f2ab80b98c0 2023-11-26T07:15:24.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.925+0000 7f2abd2c3700 1 -- 172.21.15.83:0/4115174383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab80cbbd0 con 0x7f2ab80b98c0 2023-11-26T07:15:24.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.926+0000 7f2abd2c3700 1 -- 172.21.15.83:0/4115174383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab80d22f0 con 0x7f2ab80b98c0 2023-11-26T07:15:24.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.926+0000 7f2a9f7fe700 1 -- 172.21.15.83:0/4115174383 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f2aac018360 con 0x7f2ab80b98c0 2023-11-26T07:15:24.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.926+0000 7f2a9f7fe700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2aa406b230 0x7f2aa406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:24.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.926+0000 7f2a9f7fe700 1 -- 172.21.15.83:0/4115174383 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2aac089b40 con 0x7f2ab80b98c0 2023-11-26T07:15:24.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.926+0000 7f2ab67fc700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2aa406b230 0x7f2aa406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:24.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.927+0000 7f2ab67fc700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2aa406b230 0x7f2aa406d6f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009130 tx=0x7f2aa8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:24.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:24.931+0000 7f2a9f7fe700 1 -- 172.21.15.83:0/4115174383 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2aac054760 con 0x7f2ab80b98c0 2023-11-26T07:15:25.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.120+0000 7f2abd2c3700 1 -- 172.21.15.83:0/4115174383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ab80c8740 con 0x7f2ab80b98c0 2023-11-26T07:15:25.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.120+0000 7f2a9f7fe700 1 -- 172.21.15.83:0/4115174383 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2aac01f020 con 0x7f2ab80b98c0 2023-11-26T07:15:25.122 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:15:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 -- 172.21.15.83:0/4115174383 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2aa406b230 msgr2=0x7f2aa406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2aa406b230 0x7f2aa406d6f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009130 tx=0x7f2aa8008030 comp rx=0 tx=0).stop 2023-11-26T07:15:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 -- 172.21.15.83:0/4115174383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab80b98c0 msgr2=0x7f2ab81f42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab80b98c0 0x7f2ab81f42e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00c7f0 tx=0x7f2aac00cbb0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 -- 172.21.15.83:0/4115174383 shutdown_connections 2023-11-26T07:15:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ab815e140 0x7f2ab80cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab80b98c0 0x7f2ab81f42e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2aa406b230 0x7f2aa406d6f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 --2- 172.21.15.83:0/4115174383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ab80d39d0 0x7f2ab815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 -- 172.21.15.83:0/4115174383 >> 172.21.15.83:0/4115174383 conn(0x7f2ab806b410 msgr2=0x7f2ab815ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 -- 172.21.15.83:0/4115174383 shutdown_connections 2023-11-26T07:15:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.122+0000 7f2abd2c3700 1 -- 172.21.15.83:0/4115174383 wait complete. 2023-11-26T07:15:25.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:15:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.257+0000 7fd57312a700 1 Processor -- start 2023-11-26T07:15:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.257+0000 7fd57312a700 1 -- start start 2023-11-26T07:15:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd57312a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd56c15e080 0x7fd56c06bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd57312a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd56c0ce430 0x7fd56c0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd57312a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd56c0cedc0 con 0x7fd56c0b4310 2023-11-26T07:15:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd57312a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56c0d1200 con 0x7fd56c15e080 2023-11-26T07:15:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd57312a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56c15e460 con 0x7fd56c0ce430 2023-11-26T07:15:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd570ec6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd56c15e080 0x7fd56c06bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd570ec6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd56c15e080 0x7fd56c06bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38948/0 (socket says 172.21.15.83:38948) 2023-11-26T07:15:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd570ec6700 1 -- 172.21.15.83:0/2295164643 learned_addr learned my addr 172.21.15.83:0/2295164643 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd5716c7700 1 --2- 172.21.15.83:0/2295164643 >> v2:172.21.15.203:3300/0 conn(0x7fd56c0ce430 0x7fd56c0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd570ec6700 1 -- 172.21.15.83:0/2295164643 >> v2:172.21.15.203:3300/0 conn(0x7fd56c0ce430 msgr2=0x7fd56c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:25.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd570ec6700 1 --2- 172.21.15.83:0/2295164643 >> v2:172.21.15.203:3300/0 conn(0x7fd56c0ce430 0x7fd56c0ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd570ec6700 1 -- 172.21.15.83:0/2295164643 >> v1:172.21.15.203:6789/0 conn(0x7fd56c0b4310 legacy=0x7fd56c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd570ec6700 1 -- 172.21.15.83:0/2295164643 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c06ba50 con 0x7fd56c15e080 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd570ec6700 1 --2- 172.21.15.83:0/2295164643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56c15e080 0x7fd56c06bee0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd56c06e1c0 tx=0x7fd560009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3dd1ab9c299ff84 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd56b7fe700 1 -- 172.21.15.83:0/2295164643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560004230 con 0x7fd56c15e080 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd56b7fe700 1 -- 172.21.15.83:0/2295164643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd560004390 con 0x7fd56c15e080 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.258+0000 7fd56b7fe700 1 -- 172.21.15.83:0/2295164643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5600036a0 con 0x7fd56c15e080 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.259+0000 7fd57312a700 1 -- 172.21.15.83:0/2295164643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56c15e080 msgr2=0x7fd56c06bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.259+0000 7fd57312a700 1 --2- 172.21.15.83:0/2295164643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56c15e080 0x7fd56c06bee0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd56c06e1c0 tx=0x7fd560009bd0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.259+0000 7fd57312a700 1 -- 172.21.15.83:0/2295164643 shutdown_connections 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.259+0000 7fd57312a700 1 --2- 172.21.15.83:0/2295164643 >> v2:172.21.15.203:3300/0 conn(0x7fd56c0ce430 0x7fd56c0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.259+0000 7fd57312a700 1 --2- 172.21.15.83:0/2295164643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56c15e080 0x7fd56c06bee0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.259+0000 7fd57312a700 1 -- 172.21.15.83:0/2295164643 >> 172.21.15.83:0/2295164643 conn(0x7fd56c0b3780 msgr2=0x7fd56c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.259+0000 7fd57312a700 1 -- 172.21.15.83:0/2295164643 shutdown_connections 2023-11-26T07:15:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.259+0000 7fd57312a700 1 -- 172.21.15.83:0/2295164643 wait complete. 2023-11-26T07:15:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.260+0000 7fd57312a700 1 Processor -- start 2023-11-26T07:15:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.260+0000 7fd57312a700 1 -- start start 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.261+0000 7fd57312a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd56c0b4310 0x7fd56c161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.261+0000 7fd57312a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd56c0ce430 0x7fd56c15e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.261+0000 7fd57312a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56c15e080 0x7fd56c15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.261+0000 7fd57312a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c0c6cf0 con 0x7fd56c15e080 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.261+0000 7fd57312a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c0b70f0 con 0x7fd56c0b4310 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.261+0000 7fd57312a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c0b73d0 con 0x7fd56c0ce430 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.261+0000 7fd5716c7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56c15e080 0x7fd56c15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.261+0000 7fd5716c7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56c15e080 0x7fd56c15c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38958/0 (socket says 172.21.15.83:38958) 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.261+0000 7fd5716c7700 1 -- 172.21.15.83:0/4265308203 learned_addr learned my addr 172.21.15.83:0/4265308203 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.261+0000 7fd5716c7700 1 -- 172.21.15.83:0/4265308203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd56c0ce430 msgr2=0x7fd56c15e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.261+0000 7fd5716c7700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd56c0ce430 0x7fd56c15e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.262+0000 7fd5716c7700 1 -- 172.21.15.83:0/4265308203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd56c0b4310 msgr2=0x7fd56c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.262+0000 7fd5716c7700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd56c0b4310 0x7fd56c161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.262+0000 7fd5716c7700 1 -- 172.21.15.83:0/4265308203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd560009550 con 0x7fd56c15e080 2023-11-26T07:15:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.262+0000 7fd5716c7700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56c15e080 0x7fd56c15c660 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd564007e40 tx=0x7fd56400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.262+0000 7fd5697fa700 1 -- 172.21.15.83:0/4265308203 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56400eea0 con 0x7fd56c15e080 2023-11-26T07:15:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.262+0000 7fd5697fa700 1 -- 172.21.15.83:0/4265308203 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd564014420 con 0x7fd56c15e080 2023-11-26T07:15:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.262+0000 7fd57312a700 1 -- 172.21.15.83:0/4265308203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c0c6f90 con 0x7fd56c15e080 2023-11-26T07:15:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.262+0000 7fd5697fa700 1 -- 172.21.15.83:0/4265308203 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640147b0 con 0x7fd56c15e080 2023-11-26T07:15:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.262+0000 7fd57312a700 1 -- 172.21.15.83:0/4265308203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c161550 con 0x7fd56c15e080 2023-11-26T07:15:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.263+0000 7fd57312a700 1 -- 172.21.15.83:0/4265308203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56c068fb0 con 0x7fd56c15e080 2023-11-26T07:15:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.263+0000 7fd5697fa700 1 -- 172.21.15.83:0/4265308203 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fd564014910 con 0x7fd56c15e080 2023-11-26T07:15:25.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.264+0000 7fd5697fa700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd55006b230 0x7fd55006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.264+0000 7fd5697fa700 1 -- 172.21.15.83:0/4265308203 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd56408a3b0 con 0x7fd56c15e080 2023-11-26T07:15:25.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.264+0000 7fd570ec6700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd55006b230 0x7fd55006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:25.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.264+0000 7fd570ec6700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd55006b230 0x7fd55006d6f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0c6560 tx=0x7fd560010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:25.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.268+0000 7fd5697fa700 1 -- 172.21.15.83:0/4265308203 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd564055010 con 0x7fd56c15e080 2023-11-26T07:15:25.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.460+0000 7fd57312a700 1 -- 172.21.15.83:0/4265308203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd56c0b1070 con 0x7fd56c15e080 2023-11-26T07:15:25.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.461+0000 7fd5697fa700 1 -- 172.21.15.83:0/4265308203 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd564058630 con 0x7fd56c15e080 2023-11-26T07:15:25.463 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:15:25.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 -- 172.21.15.83:0/4265308203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd55006b230 msgr2=0x7fd55006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd55006b230 0x7fd55006d6f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0c6560 tx=0x7fd560010000 comp rx=0 tx=0).stop 2023-11-26T07:15:25.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 -- 172.21.15.83:0/4265308203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56c15e080 msgr2=0x7fd56c15c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56c15e080 0x7fd56c15c660 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd564007e40 tx=0x7fd56400e3c0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 -- 172.21.15.83:0/4265308203 shutdown_connections 2023-11-26T07:15:25.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd56c15e080 0x7fd56c15c660 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd56c0b4310 0x7fd56c161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd55006b230 0x7fd55006d6f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 --2- 172.21.15.83:0/4265308203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd56c0ce430 0x7fd56c15e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 -- 172.21.15.83:0/4265308203 >> 172.21.15.83:0/4265308203 conn(0x7fd56c0b3780 msgr2=0x7fd56c070270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:25.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 -- 172.21.15.83:0/4265308203 shutdown_connections 2023-11-26T07:15:25.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.463+0000 7fd57312a700 1 -- 172.21.15.83:0/4265308203 wait complete. 2023-11-26T07:15:25.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:15:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.596+0000 7fb4a51a8700 1 Processor -- start 2023-11-26T07:15:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.596+0000 7fb4a51a8700 1 -- start start 2023-11-26T07:15:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.597+0000 7fb4a51a8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00b1390 0x7fb4a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.597+0000 7fb4a51a8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4a00d2390 0x7fb4a006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.597+0000 7fb4a51a8700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a00b1c60 con 0x7fb4a00b0fb0 2023-11-26T07:15:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.597+0000 7fb4a51a8700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a00d0450 con 0x7fb4a00d2390 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.597+0000 7fb4a51a8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00d0720 con 0x7fb4a00b1390 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.597+0000 7fb49f59e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4a00d2390 0x7fb4a006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.597+0000 7fb49f59e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4a00d2390 0x7fb4a006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38966/0 (socket says 172.21.15.83:38966) 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.597+0000 7fb49f59e700 1 -- 172.21.15.83:0/3322216773 learned_addr learned my addr 172.21.15.83:0/3322216773 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.597+0000 7fb49e59c700 1 --2- 172.21.15.83:0/3322216773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00b1390 0x7fb4a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb49f59e700 1 -- 172.21.15.83:0/3322216773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00b1390 msgr2=0x7fb4a006b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb49f59e700 1 --2- 172.21.15.83:0/3322216773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00b1390 0x7fb4a006b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb49f59e700 1 -- 172.21.15.83:0/3322216773 >> v1:172.21.15.203:6789/0 conn(0x7fb4a00b0fb0 legacy=0x7fb4a00cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb49f59e700 1 -- 172.21.15.83:0/3322216773 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a00d0a00 con 0x7fb4a00d2390 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb49e59c700 1 --2- 172.21.15.83:0/3322216773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00b1390 0x7fb4a006b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb49f59e700 1 --2- 172.21.15.83:0/3322216773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a00d2390 0x7fb4a006bc90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb498009e50 tx=0x7fb49800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5d8da0f286af10 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb49dd9b700 1 -- 172.21.15.83:0/3322216773 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49800e070 con 0x7fb4a00d2390 2023-11-26T07:15:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb49dd9b700 1 -- 172.21.15.83:0/3322216773 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb498003a20 con 0x7fb4a00d2390 2023-11-26T07:15:25.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb49dd9b700 1 -- 172.21.15.83:0/3322216773 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498003dd0 con 0x7fb4a00d2390 2023-11-26T07:15:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3322216773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a00d2390 msgr2=0x7fb4a006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb4a51a8700 1 --2- 172.21.15.83:0/3322216773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a00d2390 0x7fb4a006bc90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb498009e50 tx=0x7fb49800b330 comp rx=0 tx=0).stop 2023-11-26T07:15:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3322216773 shutdown_connections 2023-11-26T07:15:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb4a51a8700 1 --2- 172.21.15.83:0/3322216773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a00d2390 0x7fb4a006bc90 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb4a51a8700 1 --2- 172.21.15.83:0/3322216773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00b1390 0x7fb4a006b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3322216773 >> 172.21.15.83:0/3322216773 conn(0x7fb4a009aa00 msgr2=0x7fb4a009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3322216773 shutdown_connections 2023-11-26T07:15:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.598+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3322216773 wait complete. 2023-11-26T07:15:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.600+0000 7fb4a51a8700 1 Processor -- start 2023-11-26T07:15:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.600+0000 7fb4a51a8700 1 -- start start 2023-11-26T07:15:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb4a51a8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00b0fb0 0x7fb4a01f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb4a51a8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a00b1390 0x7fb4a00c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb4a51a8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a00d2390 0x7fb4a00b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb4a51a8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00b6e20 con 0x7fb4a00b1390 2023-11-26T07:15:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb4a51a8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00cb6e0 con 0x7fb4a00d2390 2023-11-26T07:15:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb4a51a8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00b6210 con 0x7fb4a00b0fb0 2023-11-26T07:15:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a00b1390 0x7fb4a00c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a00b1390 0x7fb4a00c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38970/0 (socket says 172.21.15.83:38970) 2023-11-26T07:15:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49e59c700 1 -- 172.21.15.83:0/3405767449 learned_addr learned my addr 172.21.15.83:0/3405767449 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49f59e700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a00d2390 0x7fb4a00b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49ed9d700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00b0fb0 0x7fb4a01f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49e59c700 1 -- 172.21.15.83:0/3405767449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00b0fb0 msgr2=0x7fb4a01f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49e59c700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00b0fb0 0x7fb4a01f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49e59c700 1 -- 172.21.15.83:0/3405767449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a00d2390 msgr2=0x7fb4a00b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49e59c700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a00d2390 0x7fb4a00b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49e59c700 1 -- 172.21.15.83:0/3405767449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb498009b30 con 0x7fb4a00b1390 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49f59e700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a00d2390 0x7fb4a00b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb49e59c700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a00b1390 0x7fb4a00c7240 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb49400c750 tx=0x7fb49400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb4877fe700 1 -- 172.21.15.83:0/3405767449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb494011800 con 0x7fb4a00b1390 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb4877fe700 1 -- 172.21.15.83:0/3405767449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb494011960 con 0x7fb4a00b1390 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb4877fe700 1 -- 172.21.15.83:0/3405767449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb494011cd0 con 0x7fb4a00b1390 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.601+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3405767449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a00b64f0 con 0x7fb4a00b1390 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.602+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3405767449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a00b6ab0 con 0x7fb4a00b1390 2023-11-26T07:15:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.602+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3405767449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a0068480 con 0x7fb4a00b1390 2023-11-26T07:15:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.603+0000 7fb4877fe700 1 -- 172.21.15.83:0/3405767449 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fb494021350 con 0x7fb4a00b1390 2023-11-26T07:15:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.603+0000 7fb4877fe700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb48c06b230 0x7fb48c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.603+0000 7fb4877fe700 1 -- 172.21.15.83:0/3405767449 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb494089b40 con 0x7fb4a00b1390 2023-11-26T07:15:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.603+0000 7fb49ed9d700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb48c06b230 0x7fb48c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.604+0000 7fb49ed9d700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb48c06b230 0x7fb48c06d6f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb490004f30 tx=0x7fb490009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.608+0000 7fb4877fe700 1 -- 172.21.15.83:0/3405767449 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb494054760 con 0x7fb4a00b1390 2023-11-26T07:15:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.815+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3405767449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb4a00751e0 con 0x7fb4a00b1390 2023-11-26T07:15:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.815+0000 7fb4877fe700 1 -- 172.21.15.83:0/3405767449 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb49401e080 con 0x7fb4a00b1390 2023-11-26T07:15:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3405767449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb48c06b230 msgr2=0x7fb48c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb48c06b230 0x7fb48c06d6f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb490004f30 tx=0x7fb490009040 comp rx=0 tx=0).stop 2023-11-26T07:15:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3405767449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a00b1390 msgr2=0x7fb4a00c7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a00b1390 0x7fb4a00c7240 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb49400c750 tx=0x7fb49400cb10 comp rx=0 tx=0).stop 2023-11-26T07:15:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3405767449 shutdown_connections 2023-11-26T07:15:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4a00d2390 0x7fb4a00b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb48c06b230 0x7fb48c06d6f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4a00b0fb0 0x7fb4a01f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 --2- 172.21.15.83:0/3405767449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a00b1390 0x7fb4a00c7240 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3405767449 >> 172.21.15.83:0/3405767449 conn(0x7fb4a009aa00 msgr2=0x7fb4a00b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3405767449 shutdown_connections 2023-11-26T07:15:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:25.818+0000 7fb4a51a8700 1 -- 172.21.15.83:0/3405767449 wait complete. 2023-11-26T07:15:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:15:25.832 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:15:25.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:25.832 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:15:25.880 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:15:25.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:25.880 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:15:25.899 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:15:25.899 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:25.916 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:15:25.917 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:15:25.917 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:25.958 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:15:25.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:15:26.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.075+0000 7f8616a72700 1 Processor -- start 2023-11-26T07:15:26.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.075+0000 7f8616a72700 1 -- start start 2023-11-26T07:15:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f8616a72700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f86100d2240 0x7f86100b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f8616a72700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861015e1b0 0x7f86100cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f8616a72700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86100b1d40 con 0x7f861015ddd0 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f8616a72700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86100b28c0 con 0x7f86100d2240 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f8616a72700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861006c000 con 0x7f861015e1b0 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f860ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f86100d2240 0x7f86100b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f860ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f86100d2240 0x7f86100b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38978/0 (socket says 172.21.15.83:38978) 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f860ffff700 1 -- 172.21.15.83:0/658054804 learned_addr learned my addr 172.21.15.83:0/658054804 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f861500f700 1 --2- 172.21.15.83:0/658054804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861015e1b0 0x7f86100cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f860ffff700 1 -- 172.21.15.83:0/658054804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861015e1b0 msgr2=0x7f86100cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f860ffff700 1 --2- 172.21.15.83:0/658054804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861015e1b0 0x7f86100cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f860ffff700 1 -- 172.21.15.83:0/658054804 >> v1:172.21.15.203:6789/0 conn(0x7f861015ddd0 legacy=0x7f86100d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.076+0000 7f860ffff700 1 -- 172.21.15.83:0/658054804 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86100b1f90 con 0x7f86100d2240 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f860ffff700 1 --2- 172.21.15.83:0/658054804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86100d2240 0x7f86100b1680 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009920 tx=0x7f85fc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ae117ee34571f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f860f7fe700 1 -- 172.21.15.83:0/658054804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc004230 con 0x7f86100d2240 2023-11-26T07:15:26.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f860f7fe700 1 -- 172.21.15.83:0/658054804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc004390 con 0x7f86100d2240 2023-11-26T07:15:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f860f7fe700 1 -- 172.21.15.83:0/658054804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc009e70 con 0x7f86100d2240 2023-11-26T07:15:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f8616a72700 1 -- 172.21.15.83:0/658054804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86100d2240 msgr2=0x7f86100b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f8616a72700 1 --2- 172.21.15.83:0/658054804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86100d2240 0x7f86100b1680 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009920 tx=0x7f85fc009c30 comp rx=0 tx=0).stop 2023-11-26T07:15:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f8616a72700 1 -- 172.21.15.83:0/658054804 shutdown_connections 2023-11-26T07:15:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f8616a72700 1 --2- 172.21.15.83:0/658054804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861015e1b0 0x7f86100cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f8616a72700 1 --2- 172.21.15.83:0/658054804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86100d2240 0x7f86100b1680 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f8616a72700 1 -- 172.21.15.83:0/658054804 >> 172.21.15.83:0/658054804 conn(0x7f86100731a0 msgr2=0x7f861009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f8616a72700 1 -- 172.21.15.83:0/658054804 shutdown_connections 2023-11-26T07:15:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.077+0000 7f8616a72700 1 -- 172.21.15.83:0/658054804 wait complete. 2023-11-26T07:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.078+0000 7f8616a72700 1 Processor -- start 2023-11-26T07:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.079+0000 7f8616a72700 1 -- start start 2023-11-26T07:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.079+0000 7f8616a72700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86100d2240 0x7f86101521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.079+0000 7f8616a72700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f861015ddd0 0x7f86100b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.079+0000 7f8616a72700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861015e1b0 0x7f861006f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.079+0000 7f8616a72700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86100b6340 con 0x7f86100d2240 2023-11-26T07:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.079+0000 7f8616a72700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86100b6620 con 0x7f861015ddd0 2023-11-26T07:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.079+0000 7f8616a72700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86100b6900 con 0x7f861015e1b0 2023-11-26T07:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f861480e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86100d2240 0x7f86101521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f861480e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86100d2240 0x7f86101521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38982/0 (socket says 172.21.15.83:38982) 2023-11-26T07:15:26.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f861480e700 1 -- 172.21.15.83:0/1331300133 learned_addr learned my addr 172.21.15.83:0/1331300133 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f860ffff700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f861015ddd0 0x7f86100b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f861500f700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861015e1b0 0x7f861006f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f861480e700 1 -- 172.21.15.83:0/1331300133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861015e1b0 msgr2=0x7f861006f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f861480e700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861015e1b0 0x7f861006f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f861480e700 1 -- 172.21.15.83:0/1331300133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f861015ddd0 msgr2=0x7f86100b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f861480e700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f861015ddd0 0x7f86100b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f861480e700 1 -- 172.21.15.83:0/1331300133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc009550 con 0x7f86100d2240 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f861480e700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86100d2240 0x7f86101521f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f860400c4d0 tx=0x7f860400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f860d7fa700 1 -- 172.21.15.83:0/1331300133 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604011790 con 0x7f86100d2240 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f860d7fa700 1 -- 172.21.15.83:0/1331300133 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86040118f0 con 0x7f86100d2240 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f8616a72700 1 -- 172.21.15.83:0/1331300133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86100b2ed0 con 0x7f86100d2240 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f860d7fa700 1 -- 172.21.15.83:0/1331300133 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604011ca0 con 0x7f86100d2240 2023-11-26T07:15:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.080+0000 7f8616a72700 1 -- 172.21.15.83:0/1331300133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86100b6ec0 con 0x7f86100d2240 2023-11-26T07:15:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.081+0000 7f8616a72700 1 -- 172.21.15.83:0/1331300133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86100b7230 con 0x7f86100d2240 2023-11-26T07:15:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.081+0000 7f860d7fa700 1 -- 172.21.15.83:0/1331300133 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f860401c460 con 0x7f86100d2240 2023-11-26T07:15:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.082+0000 7f860d7fa700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85f406b260 0x7f85f406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.082+0000 7f860d7fa700 1 -- 172.21.15.83:0/1331300133 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8604089710 con 0x7f86100d2240 2023-11-26T07:15:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.082+0000 7f860ffff700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85f406b260 0x7f85f406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.082+0000 7f860ffff700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85f406b260 0x7f85f406d720 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009130 tx=0x7f85fc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:26.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.086+0000 7f860d7fa700 1 -- 172.21.15.83:0/1331300133 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86040544c0 con 0x7f86100d2240 2023-11-26T07:15:26.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.315+0000 7f8616a72700 1 -- 172.21.15.83:0/1331300133 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86100b7050 con 0x7f86100d2240 2023-11-26T07:15:26.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.316+0000 7f860d7fa700 1 -- 172.21.15.83:0/1331300133 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f8604057ae0 con 0x7f86100d2240 2023-11-26T07:15:26.318 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:26.318 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/126258309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":126258309},{"type":"v1","addr":"172.21.15.203:6834","nonce":126258309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4203521524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":4203521524},{"type":"v1","addr":"172.21.15.203:6835","nonce":4203521524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1267615204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1267615204},{"type":"v1","addr":"172.21.15.83:6836","nonce":1267615204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:14:54.936357+0000","modified":"2023-11-26T07:14:57.974394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6102":{"gid":6102,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/83108157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":83108157},{"type":"v1","addr":"172.21.15.83:6837","nonce":83108157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:15:26.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 -- 172.21.15.83:0/1331300133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85f406b260 msgr2=0x7f85f406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85f406b260 0x7f85f406d720 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009130 tx=0x7f85fc008030 comp rx=0 tx=0).stop 2023-11-26T07:15:26.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 -- 172.21.15.83:0/1331300133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86100d2240 msgr2=0x7f86101521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86100d2240 0x7f86101521f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f860400c4d0 tx=0x7f860400c7e0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 -- 172.21.15.83:0/1331300133 shutdown_connections 2023-11-26T07:15:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f85f406b260 0x7f85f406d720 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f861015e1b0 0x7f861006f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86100d2240 0x7f86101521f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 --2- 172.21.15.83:0/1331300133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f861015ddd0 0x7f86100b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 -- 172.21.15.83:0/1331300133 >> 172.21.15.83:0/1331300133 conn(0x7f86100731a0 msgr2=0x7f86100aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 -- 172.21.15.83:0/1331300133 shutdown_connections 2023-11-26T07:15:26.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.318+0000 7f8616a72700 1 -- 172.21.15.83:0/1331300133 wait complete. 2023-11-26T07:15:26.322 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 38 2023-11-26T07:15:26.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:26.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.460+0000 7f8b3da7d700 1 Processor -- start 2023-11-26T07:15:26.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.460+0000 7f8b3da7d700 1 -- start start 2023-11-26T07:15:26.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.461+0000 7f8b3da7d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8b380b49a0 0x7f8b380d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.461+0000 7f8b3da7d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3815c4b0 con 0x7f8b3806c710 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.461+0000 7f8b3da7d700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3815c790 con 0x7f8b38152cc0 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.461+0000 7f8b3da7d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3815ca70 con 0x7f8b380b49a0 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.461+0000 7f8b36ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8b3806c710 0x7f8b380cff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59032/0 (socket says 172.21.15.83:59032) 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.461+0000 7f8b36ffd700 1 -- 172.21.15.83:0/1650515431 learned_addr learned my addr 172.21.15.83:0/1650515431 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839530004 0 0) 0x7f8b3815c4b0 con 0x7f8b3806c710 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b37fff700 1 --2- 172.21.15.83:0/1650515431 >> v2:172.21.15.203:3300/0 conn(0x7f8b380b49a0 0x7f8b380d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b200035b0 con 0x7f8b3806c710 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 607667119 0 0) 0x7f8b200035b0 con 0x7f8b3806c710 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 >> v2:172.21.15.203:3300/0 conn(0x7f8b380b49a0 msgr2=0x7f8b380d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 --2- 172.21.15.83:0/1650515431 >> v2:172.21.15.203:3300/0 conn(0x7f8b380b49a0 0x7f8b380d0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 >> v1:172.21.15.203:6789/0 conn(0x7f8b38152cc0 legacy=0x7f8b3806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3815cd50 con 0x7f8b3806c710 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f8b28001670 con 0x7f8b3806c710 2023-11-26T07:15:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 >> v1:172.21.15.83:6789/0 conn(0x7f8b3806c710 legacy=0x7f8b380cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 --2- 172.21.15.83:0/1650515431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b20003a00 0x7f8b20003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 --2- 172.21.15.83:0/1650515431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b20004850 0x7f8b20004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 --2- 172.21.15.83:0/1650515431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 0x7f8b2000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3815ca70 con 0x7f8b20005210 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2000c1b0 con 0x7f8b20003a00 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2000c370 con 0x7f8b20004850 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b37fff700 1 --2- 172.21.15.83:0/1650515431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 0x7f8b2000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b377fe700 1 --2- 172.21.15.83:0/1650515431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b20003a00 0x7f8b20003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b36ffd700 1 --2- 172.21.15.83:0/1650515431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b20004850 0x7f8b20004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b37fff700 1 -- 172.21.15.83:0/1650515431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b20004850 msgr2=0x7f8b20004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b37fff700 1 --2- 172.21.15.83:0/1650515431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b20004850 0x7f8b20004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b37fff700 1 -- 172.21.15.83:0/1650515431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b20003a00 msgr2=0x7f8b20003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b37fff700 1 --2- 172.21.15.83:0/1650515431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b20003a00 0x7f8b20003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b37fff700 1 -- 172.21.15.83:0/1650515431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b200035b0 con 0x7f8b20005210 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b37fff700 1 -- 172.21.15.83:0/1650515431 reap_dead start 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.462+0000 7f8b37fff700 1 --2- 172.21.15.83:0/1650515431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 0x7f8b2000ba10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009fc0 tx=0x7f8b20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50147edaa2d7d366 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.463+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30003a20 con 0x7f8b20005210 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.463+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3000f040 con 0x7f8b20005210 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.463+0000 7f8b367fc700 1 -- 172.21.15.83:0/1650515431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30003de0 con 0x7f8b20005210 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.463+0000 7f8b3da7d700 1 -- 172.21.15.83:0/1650515431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 msgr2=0x7f8b2000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.463+0000 7f8b3da7d700 1 --2- 172.21.15.83:0/1650515431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 0x7f8b2000ba10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009fc0 tx=0x7f8b20005880 comp rx=0 tx=0).stop 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.463+0000 7f8b3da7d700 1 -- 172.21.15.83:0/1650515431 shutdown_connections 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.463+0000 7f8b3da7d700 1 --2- 172.21.15.83:0/1650515431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 0x7f8b2000ba10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.463+0000 7f8b3da7d700 1 -- 172.21.15.83:0/1650515431 >> 172.21.15.83:0/1650515431 conn(0x7f8b38071b90 msgr2=0x7f8b38071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.463+0000 7f8b3da7d700 1 -- 172.21.15.83:0/1650515431 shutdown_connections 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.463+0000 7f8b3da7d700 1 -- 172.21.15.83:0/1650515431 wait complete. 2023-11-26T07:15:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.464+0000 7f8b3da7d700 1 Processor -- start 2023-11-26T07:15:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.464+0000 7f8b3da7d700 1 -- start start 2023-11-26T07:15:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.465+0000 7f8b3da7d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 0x7f8b381474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.465+0000 7f8b3da7d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b38151480 0x7f8b381556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.465+0000 7f8b3da7d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b38151900 0x7f8b38155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.465+0000 7f8b3da7d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38159be0 con 0x7f8b20005210 2023-11-26T07:15:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.465+0000 7f8b3da7d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38158f60 con 0x7f8b38151480 2023-11-26T07:15:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.465+0000 7f8b3da7d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38159240 con 0x7f8b38151900 2023-11-26T07:15:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.465+0000 7f8b377fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 0x7f8b381474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.465+0000 7f8b377fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 0x7f8b381474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39002/0 (socket says 172.21.15.83:39002) 2023-11-26T07:15:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.465+0000 7f8b377fe700 1 -- 172.21.15.83:0/3114183352 learned_addr learned my addr 172.21.15.83:0/3114183352 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.465+0000 7f8b36ffd700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b38151480 0x7f8b381556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.465+0000 7f8b37fff700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b38151900 0x7f8b38155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.466+0000 7f8b377fe700 1 -- 172.21.15.83:0/3114183352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b38151900 msgr2=0x7f8b38155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.466+0000 7f8b377fe700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b38151900 0x7f8b38155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.466+0000 7f8b377fe700 1 -- 172.21.15.83:0/3114183352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b38151480 msgr2=0x7f8b381556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.466+0000 7f8b377fe700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b38151480 0x7f8b381556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.466+0000 7f8b377fe700 1 -- 172.21.15.83:0/3114183352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38159520 con 0x7f8b20005210 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.466+0000 7f8b377fe700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 0x7f8b381474f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00f820 tx=0x7f8b2c00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.466+0000 7f8b1ffff700 1 -- 172.21.15.83:0/3114183352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0052e0 con 0x7f8b20005210 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.466+0000 7f8b1ffff700 1 -- 172.21.15.83:0/3114183352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c005460 con 0x7f8b20005210 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.466+0000 7f8b3da7d700 1 -- 172.21.15.83:0/3114183352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b38159800 con 0x7f8b20005210 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.466+0000 7f8b1ffff700 1 -- 172.21.15.83:0/3114183352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0038c0 con 0x7f8b20005210 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.466+0000 7f8b3da7d700 1 -- 172.21.15.83:0/3114183352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b38158b60 con 0x7f8b20005210 2023-11-26T07:15:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.467+0000 7f8b3da7d700 1 -- 172.21.15.83:0/3114183352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b38153f60 con 0x7f8b20005210 2023-11-26T07:15:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.467+0000 7f8b1ffff700 1 -- 172.21.15.83:0/3114183352 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f8b2c003a20 con 0x7f8b20005210 2023-11-26T07:15:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.468+0000 7f8b1ffff700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b2406b230 0x7f8b2406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.468+0000 7f8b1ffff700 1 -- 172.21.15.83:0/3114183352 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b2c0897c0 con 0x7f8b20005210 2023-11-26T07:15:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.468+0000 7f8b36ffd700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b2406b230 0x7f8b2406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.468+0000 7f8b36ffd700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b2406b230 0x7f8b2406d6f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b380d04c0 tx=0x7f8b28009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.472+0000 7f8b1ffff700 1 -- 172.21.15.83:0/3114183352 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b2c0544f0 con 0x7f8b20005210 2023-11-26T07:15:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.666+0000 7f8b3da7d700 1 -- 172.21.15.83:0/3114183352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b380753f0 con 0x7f8b20005210 2023-11-26T07:15:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.668+0000 7f8b1ffff700 1 -- 172.21.15.83:0/3114183352 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f8b2c057b10 con 0x7f8b20005210 2023-11-26T07:15:26.670 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:26.671 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":58,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:14:56.536771+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:14:53.044716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:14:53.904013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 -- 172.21.15.83:0/3114183352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b2406b230 msgr2=0x7f8b2406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b2406b230 0x7f8b2406d6f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b380d04c0 tx=0x7f8b28009040 comp rx=0 tx=0).stop 2023-11-26T07:15:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 -- 172.21.15.83:0/3114183352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 msgr2=0x7f8b381474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 0x7f8b381474f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00f820 tx=0x7f8b2c00fbe0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 -- 172.21.15.83:0/3114183352 shutdown_connections 2023-11-26T07:15:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b2406b230 0x7f8b2406d6f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8b38151900 0x7f8b38155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b20005210 0x7f8b381474f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 --2- 172.21.15.83:0/3114183352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b38151480 0x7f8b381556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 -- 172.21.15.83:0/3114183352 >> 172.21.15.83:0/3114183352 conn(0x7f8b38071b90 msgr2=0x7f8b380b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 -- 172.21.15.83:0/3114183352 shutdown_connections 2023-11-26T07:15:26.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.670+0000 7f8b3da7d700 1 -- 172.21.15.83:0/3114183352 wait complete. 2023-11-26T07:15:26.684 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:14:53.044716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:14:53.904013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:15:26.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:15:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.805+0000 7f3aeb59e700 1 Processor -- start 2023-11-26T07:15:26.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.805+0000 7f3aeb59e700 1 -- start start 2023-11-26T07:15:26.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aeb59e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3aec158390 0x7f3aec0afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aeb59e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aec0ad870 0x7f3aec06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aeb59e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aec0ae610 con 0x7f3aec073f90 2023-11-26T07:15:26.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aeb59e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aec06beb0 con 0x7f3aec158390 2023-11-26T07:15:26.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aeb59e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec0ae050 con 0x7f3aec0ad870 2023-11-26T07:15:26.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aea59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3aec158390 0x7f3aec0afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aea59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3aec158390 0x7f3aec0afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39010/0 (socket says 172.21.15.83:39010) 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aea59c700 1 -- 172.21.15.83:0/2602088817 learned_addr learned my addr 172.21.15.83:0/2602088817 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aead9d700 1 --1- 172.21.15.83:0/2602088817 >> v1:172.21.15.83:6789/0 conn(0x7f3aec073f90 0x7f3aec15e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59054/0 (socket says 172.21.15.83:59054) 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3ae9d9b700 1 --2- 172.21.15.83:0/2602088817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aec0ad870 0x7f3aec06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aea59c700 1 -- 172.21.15.83:0/2602088817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aec0ad870 msgr2=0x7f3aec06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aea59c700 1 --2- 172.21.15.83:0/2602088817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aec0ad870 0x7f3aec06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aea59c700 1 -- 172.21.15.83:0/2602088817 >> v1:172.21.15.83:6789/0 conn(0x7f3aec073f90 legacy=0x7f3aec15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3aea59c700 1 -- 172.21.15.83:0/2602088817 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec0ae330 con 0x7f3aec158390 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.806+0000 7f3ae9d9b700 1 --2- 172.21.15.83:0/2602088817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aec0ad870 0x7f3aec06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3aea59c700 1 --2- 172.21.15.83:0/2602088817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3aec158390 0x7f3aec0afe50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00098f0 tx=0x7f3ae0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45bc2ab9726ce44 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3ae959a700 1 -- 172.21.15.83:0/2602088817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000b8a0 con 0x7f3aec158390 2023-11-26T07:15:26.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3ae959a700 1 -- 172.21.15.83:0/2602088817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae000ba00 con 0x7f3aec158390 2023-11-26T07:15:26.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3ae959a700 1 -- 172.21.15.83:0/2602088817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000bd90 con 0x7f3aec158390 2023-11-26T07:15:26.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3aeb59e700 1 -- 172.21.15.83:0/2602088817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3aec158390 msgr2=0x7f3aec0afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3aeb59e700 1 --2- 172.21.15.83:0/2602088817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3aec158390 0x7f3aec0afe50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00098f0 tx=0x7f3ae0009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:26.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3aeb59e700 1 -- 172.21.15.83:0/2602088817 shutdown_connections 2023-11-26T07:15:26.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3aeb59e700 1 --2- 172.21.15.83:0/2602088817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3aec158390 0x7f3aec0afe50 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3aeb59e700 1 --2- 172.21.15.83:0/2602088817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aec0ad870 0x7f3aec06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3aeb59e700 1 -- 172.21.15.83:0/2602088817 >> 172.21.15.83:0/2602088817 conn(0x7f3aec0d0d40 msgr2=0x7f3aec0d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:26.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3aeb59e700 1 -- 172.21.15.83:0/2602088817 shutdown_connections 2023-11-26T07:15:26.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.807+0000 7f3aeb59e700 1 -- 172.21.15.83:0/2602088817 wait complete. 2023-11-26T07:15:26.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.809+0000 7f3aeb59e700 1 Processor -- start 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.809+0000 7f3aeb59e700 1 -- start start 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.809+0000 7f3aeb59e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aec073f90 0x7f3aec1f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.809+0000 7f3aeb59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3aec0ad870 0x7f3aec06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.809+0000 7f3aeb59e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aec158390 0x7f3aec148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.809+0000 7f3aeb59e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec06f350 con 0x7f3aec0ad870 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.809+0000 7f3aeb59e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec146610 con 0x7f3aec158390 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.809+0000 7f3aeb59e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec06ed90 con 0x7f3aec073f90 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3ae9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3aec0ad870 0x7f3aec06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3ae9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3aec0ad870 0x7f3aec06e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39018/0 (socket says 172.21.15.83:39018) 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3ae9d9b700 1 -- 172.21.15.83:0/1165398548 learned_addr learned my addr 172.21.15.83:0/1165398548 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3aea59c700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aec073f90 0x7f3aec1f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3aead9d700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aec158390 0x7f3aec148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3ae9d9b700 1 -- 172.21.15.83:0/1165398548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aec073f90 msgr2=0x7f3aec1f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3ae9d9b700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aec073f90 0x7f3aec1f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3ae9d9b700 1 -- 172.21.15.83:0/1165398548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aec158390 msgr2=0x7f3aec148120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3ae9d9b700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aec158390 0x7f3aec148120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3ae9d9b700 1 -- 172.21.15.83:0/1165398548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0009550 con 0x7f3aec0ad870 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3ae9d9b700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3aec0ad870 0x7f3aec06e2a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00c750 tx=0x7f3adc00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3adaffd700 1 -- 172.21.15.83:0/1165398548 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc011800 con 0x7f3aec0ad870 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3adaffd700 1 -- 172.21.15.83:0/1165398548 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3adc011960 con 0x7f3aec0ad870 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3aeb59e700 1 -- 172.21.15.83:0/1165398548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aec06f070 con 0x7f3aec0ad870 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3adaffd700 1 -- 172.21.15.83:0/1165398548 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc011cd0 con 0x7f3aec0ad870 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.810+0000 7f3aeb59e700 1 -- 172.21.15.83:0/1165398548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aec14c370 con 0x7f3aec0ad870 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.811+0000 7f3aeb59e700 1 -- 172.21.15.83:0/1165398548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aec068480 con 0x7f3aec0ad870 2023-11-26T07:15:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.812+0000 7f3adaffd700 1 -- 172.21.15.83:0/1165398548 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f3adc021350 con 0x7f3aec0ad870 2023-11-26T07:15:26.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.812+0000 7f3adaffd700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3acc06b230 0x7f3acc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:26.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.812+0000 7f3adaffd700 1 -- 172.21.15.83:0/1165398548 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3adc0899a0 con 0x7f3aec0ad870 2023-11-26T07:15:26.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.812+0000 7f3aea59c700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3acc06b230 0x7f3acc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:26.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.812+0000 7f3aea59c700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3acc06b230 0x7f3acc06d6f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009c40 tx=0x7f3ae0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:26.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:26.816+0000 7f3adaffd700 1 -- 172.21.15.83:0/1165398548 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3adc0545c0 con 0x7f3aec0ad870 2023-11-26T07:15:27.006 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.003+0000 7f3aeb59e700 1 -- 172.21.15.83:0/1165398548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3aec146090 con 0x7f3aec0ad870 2023-11-26T07:15:27.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.004+0000 7f3adaffd700 1 -- 172.21.15.83:0/1165398548 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3aec146090 con 0x7f3aec0ad870 2023-11-26T07:15:27.014 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:15:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 -- 172.21.15.83:0/1165398548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3acc06b230 msgr2=0x7f3acc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3acc06b230 0x7f3acc06d6f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009c40 tx=0x7f3ae0008030 comp rx=0 tx=0).stop 2023-11-26T07:15:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 -- 172.21.15.83:0/1165398548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3aec0ad870 msgr2=0x7f3aec06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3aec0ad870 0x7f3aec06e2a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00c750 tx=0x7f3adc00ca60 comp rx=0 tx=0).stop 2023-11-26T07:15:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 -- 172.21.15.83:0/1165398548 shutdown_connections 2023-11-26T07:15:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aec158390 0x7f3aec148120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3acc06b230 0x7f3acc06d6f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aec073f90 0x7f3aec1f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 --2- 172.21.15.83:0/1165398548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3aec0ad870 0x7f3aec06e2a0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 -- 172.21.15.83:0/1165398548 >> 172.21.15.83:0/1165398548 conn(0x7f3aec0d0d40 msgr2=0x7f3aec0b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:27.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 -- 172.21.15.83:0/1165398548 shutdown_connections 2023-11-26T07:15:27.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.006+0000 7f3aeb59e700 1 -- 172.21.15.83:0/1165398548 wait complete. 2023-11-26T07:15:27.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:15:27.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.140+0000 7fb7d05cd700 1 Processor -- start 2023-11-26T07:15:27.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.140+0000 7fb7d05cd700 1 -- start start 2023-11-26T07:15:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7d05cd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb7c80cf610 0x7fb7c80add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7d05cd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7c815f9e0 0x7fb7c80b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7d05cd700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c806b080 con 0x7fb7c8073f90 2023-11-26T07:15:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7d05cd700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c815c290 con 0x7fb7c80cf610 2023-11-26T07:15:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7d05cd700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80cf9f0 con 0x7fb7c815f9e0 2023-11-26T07:15:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7cdb68700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7c815f9e0 0x7fb7c80b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7ce369700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb7c80cf610 0x7fb7c80add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7ce369700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb7c80cf610 0x7fb7c80add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38476/0 (socket says 172.21.15.83:38476) 2023-11-26T07:15:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7ce369700 1 -- 172.21.15.83:0/1492334424 learned_addr learned my addr 172.21.15.83:0/1492334424 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7cdb68700 1 -- 172.21.15.83:0/1492334424 >> v2:172.21.15.203:3300/0 conn(0x7fb7c80cf610 msgr2=0x7fb7c80add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7cdb68700 1 --2- 172.21.15.83:0/1492334424 >> v2:172.21.15.203:3300/0 conn(0x7fb7c80cf610 0x7fb7c80add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7cdb68700 1 -- 172.21.15.83:0/1492334424 >> v1:172.21.15.203:6789/0 conn(0x7fb7c8073f90 legacy=0x7fb7c806b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.141+0000 7fb7cdb68700 1 -- 172.21.15.83:0/1492334424 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c815ff80 con 0x7fb7c815f9e0 2023-11-26T07:15:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7cdb68700 1 --2- 172.21.15.83:0/1492334424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7c815f9e0 0x7fb7c80b3c60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009840 tx=0x7fb7b8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba2455466639c1b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:27.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7cd367700 1 -- 172.21.15.83:0/1492334424 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8011830 con 0x7fb7c815f9e0 2023-11-26T07:15:27.148 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7cd367700 1 -- 172.21.15.83:0/1492334424 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b8011990 con 0x7fb7c815f9e0 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7cd367700 1 -- 172.21.15.83:0/1492334424 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8011d20 con 0x7fb7c815f9e0 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7d05cd700 1 -- 172.21.15.83:0/1492334424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7c815f9e0 msgr2=0x7fb7c80b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7d05cd700 1 --2- 172.21.15.83:0/1492334424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7c815f9e0 0x7fb7c80b3c60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009840 tx=0x7fb7b8009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7d05cd700 1 -- 172.21.15.83:0/1492334424 shutdown_connections 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7d05cd700 1 --2- 172.21.15.83:0/1492334424 >> v2:172.21.15.203:3300/0 conn(0x7fb7c80cf610 0x7fb7c80add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7d05cd700 1 --2- 172.21.15.83:0/1492334424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7c815f9e0 0x7fb7c80b3c60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7d05cd700 1 -- 172.21.15.83:0/1492334424 >> 172.21.15.83:0/1492334424 conn(0x7fb7c80d0bf0 msgr2=0x7fb7c80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7d05cd700 1 -- 172.21.15.83:0/1492334424 shutdown_connections 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.142+0000 7fb7d05cd700 1 -- 172.21.15.83:0/1492334424 wait complete. 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.144+0000 7fb7d05cd700 1 Processor -- start 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.144+0000 7fb7d05cd700 1 -- start start 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.144+0000 7fb7d05cd700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7c8073f90 0x7fb7c81f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.144+0000 7fb7d05cd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7c80cf610 0x7fb7c80b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.144+0000 7fb7d05cd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7c815f9e0 0x7fb7c8154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.144+0000 7fb7d05cd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c81f41b0 con 0x7fb7c80cf610 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.144+0000 7fb7d05cd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c806dca0 con 0x7fb7c815f9e0 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.144+0000 7fb7d05cd700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c806df80 con 0x7fb7c8073f90 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.144+0000 7fb7cdb68700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7c80cf610 0x7fb7c80b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.144+0000 7fb7cdb68700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7c80cf610 0x7fb7c80b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39032/0 (socket says 172.21.15.83:39032) 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.144+0000 7fb7cdb68700 1 -- 172.21.15.83:0/3679082109 learned_addr learned my addr 172.21.15.83:0/3679082109 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7ceb6a700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7c815f9e0 0x7fb7c8154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7ce369700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7c8073f90 0x7fb7c81f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7cdb68700 1 -- 172.21.15.83:0/3679082109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7c8073f90 msgr2=0x7fb7c81f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7cdb68700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7c8073f90 0x7fb7c81f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7cdb68700 1 -- 172.21.15.83:0/3679082109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7c815f9e0 msgr2=0x7fb7c8154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7cdb68700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7c815f9e0 0x7fb7c8154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7cdb68700 1 -- 172.21.15.83:0/3679082109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b80094a0 con 0x7fb7c80cf610 2023-11-26T07:15:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7cdb68700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7c80cf610 0x7fb7c80b57a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009fd0 tx=0x7fb7b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7beffd700 1 -- 172.21.15.83:0/3679082109 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80036a0 con 0x7fb7c80cf610 2023-11-26T07:15:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7beffd700 1 -- 172.21.15.83:0/3679082109 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b801f070 con 0x7fb7c80cf610 2023-11-26T07:15:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7beffd700 1 -- 172.21.15.83:0/3679082109 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8003a60 con 0x7fb7c80cf610 2023-11-26T07:15:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7d05cd700 1 -- 172.21.15.83:0/3679082109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c806e260 con 0x7fb7c80cf610 2023-11-26T07:15:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.145+0000 7fb7d05cd700 1 -- 172.21.15.83:0/3679082109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c806e820 con 0x7fb7c80cf610 2023-11-26T07:15:27.151 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.146+0000 7fb7bcff9700 1 -- 172.21.15.83:0/3679082109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a40052e0 con 0x7fb7c80cf610 2023-11-26T07:15:27.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.147+0000 7fb7beffd700 1 -- 172.21.15.83:0/3679082109 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fb7b8003bc0 con 0x7fb7c80cf610 2023-11-26T07:15:27.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.147+0000 7fb7beffd700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7ac06b230 0x7fb7ac06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.147+0000 7fb7beffd700 1 -- 172.21.15.83:0/3679082109 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7b8021070 con 0x7fb7c80cf610 2023-11-26T07:15:27.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.147+0000 7fb7ce369700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7ac06b230 0x7fb7ac06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.152 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.147+0000 7fb7ce369700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7ac06b230 0x7fb7ac06d6f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009bd0 tx=0x7fb7c40092c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:27.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.151+0000 7fb7beffd700 1 -- 172.21.15.83:0/3679082109 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7b8055f90 con 0x7fb7c80cf610 2023-11-26T07:15:27.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:15:27.172 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:15:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.347+0000 7fb7bcff9700 1 -- 172.21.15.83:0/3679082109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7a4005b10 con 0x7fb7c80cf610 2023-11-26T07:15:27.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.347+0000 7fb7beffd700 1 -- 172.21.15.83:0/3679082109 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7b80595b0 con 0x7fb7c80cf610 2023-11-26T07:15:27.350 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:15:27.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 -- 172.21.15.83:0/3679082109 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7ac06b230 msgr2=0x7fb7ac06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7ac06b230 0x7fb7ac06d6f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009bd0 tx=0x7fb7c40092c0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 -- 172.21.15.83:0/3679082109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7c80cf610 msgr2=0x7fb7c80b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7c80cf610 0x7fb7c80b57a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009fd0 tx=0x7fb7b8004750 comp rx=0 tx=0).stop 2023-11-26T07:15:27.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 -- 172.21.15.83:0/3679082109 shutdown_connections 2023-11-26T07:15:27.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7c815f9e0 0x7fb7c8154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb7ac06b230 0x7fb7ac06d6f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7c8073f90 0x7fb7c81f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 --2- 172.21.15.83:0/3679082109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7c80cf610 0x7fb7c80b57a0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 -- 172.21.15.83:0/3679082109 >> 172.21.15.83:0/3679082109 conn(0x7fb7c80d0bf0 msgr2=0x7fb7c80b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:27.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 -- 172.21.15.83:0/3679082109 shutdown_connections 2023-11-26T07:15:27.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.350+0000 7fb7bcff9700 1 -- 172.21.15.83:0/3679082109 wait complete. 2023-11-26T07:15:27.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:15:27.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.485+0000 7ffb86fda700 1 Processor -- start 2023-11-26T07:15:27.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.485+0000 7ffb86fda700 1 -- start start 2023-11-26T07:15:27.490 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb86fda700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb801587f0 0x7ffb8006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb86fda700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb8015b370 0x7ffb8015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb86fda700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8006b530 con 0x7ffb800b3c60 2023-11-26T07:15:27.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb86fda700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb800ae5e0 con 0x7ffb8015b370 2023-11-26T07:15:27.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb86fda700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8015be30 con 0x7ffb801587f0 2023-11-26T07:15:27.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb7ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb8015b370 0x7ffb8015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb7ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb8015b370 0x7ffb8015b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39038/0 (socket says 172.21.15.83:39038) 2023-11-26T07:15:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb7ffff700 1 -- 172.21.15.83:0/1286940444 learned_addr learned my addr 172.21.15.83:0/1286940444 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb84d76700 1 --2- 172.21.15.83:0/1286940444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb801587f0 0x7ffb8006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb7ffff700 1 -- 172.21.15.83:0/1286940444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb801587f0 msgr2=0x7ffb8006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb7ffff700 1 --2- 172.21.15.83:0/1286940444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb801587f0 0x7ffb8006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb7ffff700 1 -- 172.21.15.83:0/1286940444 >> v1:172.21.15.83:6789/0 conn(0x7ffb800b3c60 legacy=0x7ffb800af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb7ffff700 1 -- 172.21.15.83:0/1286940444 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8015c110 con 0x7ffb8015b370 2023-11-26T07:15:27.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb7ffff700 1 --2- 172.21.15.83:0/1286940444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8015b370 0x7ffb8015b7f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0098f0 tx=0x7ffb6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db5a3d7495a6bfc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:27.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.486+0000 7ffb7f7fe700 1 -- 172.21.15.83:0/1286940444 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c00b8a0 con 0x7ffb8015b370 2023-11-26T07:15:27.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.487+0000 7ffb7f7fe700 1 -- 172.21.15.83:0/1286940444 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c00ba00 con 0x7ffb8015b370 2023-11-26T07:15:27.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.487+0000 7ffb7f7fe700 1 -- 172.21.15.83:0/1286940444 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c00bd90 con 0x7ffb8015b370 2023-11-26T07:15:27.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.487+0000 7ffb86fda700 1 -- 172.21.15.83:0/1286940444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8015b370 msgr2=0x7ffb8015b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.487+0000 7ffb86fda700 1 --2- 172.21.15.83:0/1286940444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8015b370 0x7ffb8015b7f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0098f0 tx=0x7ffb6c009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:27.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.487+0000 7ffb86fda700 1 -- 172.21.15.83:0/1286940444 shutdown_connections 2023-11-26T07:15:27.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.487+0000 7ffb86fda700 1 --2- 172.21.15.83:0/1286940444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb801587f0 0x7ffb8006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.487+0000 7ffb86fda700 1 --2- 172.21.15.83:0/1286940444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb8015b370 0x7ffb8015b7f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.487+0000 7ffb86fda700 1 -- 172.21.15.83:0/1286940444 >> 172.21.15.83:0/1286940444 conn(0x7ffb800d0bf0 msgr2=0x7ffb800d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:27.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.487+0000 7ffb86fda700 1 -- 172.21.15.83:0/1286940444 shutdown_connections 2023-11-26T07:15:27.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.487+0000 7ffb86fda700 1 -- 172.21.15.83:0/1286940444 wait complete. 2023-11-26T07:15:27.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.488+0000 7ffb86fda700 1 Processor -- start 2023-11-26T07:15:27.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.488+0000 7ffb86fda700 1 -- start start 2023-11-26T07:15:27.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.489+0000 7ffb86fda700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb800b3c60 0x7ffb80154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.489+0000 7ffb86fda700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb801587f0 0x7ffb801e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.489+0000 7ffb86fda700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb8015b370 0x7ffb801e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.489+0000 7ffb86fda700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8009eca0 con 0x7ffb801587f0 2023-11-26T07:15:27.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.489+0000 7ffb86fda700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8009e020 con 0x7ffb800b3c60 2023-11-26T07:15:27.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.489+0000 7ffb86fda700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8009e300 con 0x7ffb8015b370 2023-11-26T07:15:27.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.489+0000 7ffb7ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb801587f0 0x7ffb801e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.489+0000 7ffb7ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb801587f0 0x7ffb801e7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39046/0 (socket says 172.21.15.83:39046) 2023-11-26T07:15:27.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.489+0000 7ffb7ffff700 1 -- 172.21.15.83:0/645978517 learned_addr learned my addr 172.21.15.83:0/645978517 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:27.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.489+0000 7ffb84d76700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb800b3c60 0x7ffb80154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.489+0000 7ffb85577700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb8015b370 0x7ffb801e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.490+0000 7ffb7ffff700 1 -- 172.21.15.83:0/645978517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb8015b370 msgr2=0x7ffb801e7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.490+0000 7ffb7ffff700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb8015b370 0x7ffb801e7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.490+0000 7ffb7ffff700 1 -- 172.21.15.83:0/645978517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb800b3c60 msgr2=0x7ffb80154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.490+0000 7ffb7ffff700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb800b3c60 0x7ffb80154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.490+0000 7ffb7ffff700 1 -- 172.21.15.83:0/645978517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c009550 con 0x7ffb801587f0 2023-11-26T07:15:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.490+0000 7ffb7ffff700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb801587f0 0x7ffb801e7a10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c000c00 tx=0x7ffb6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.490+0000 7ffb7d7fa700 1 -- 172.21.15.83:0/645978517 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c004480 con 0x7ffb801587f0 2023-11-26T07:15:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.490+0000 7ffb7d7fa700 1 -- 172.21.15.83:0/645978517 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c01f070 con 0x7ffb801587f0 2023-11-26T07:15:27.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.490+0000 7ffb86fda700 1 -- 172.21.15.83:0/645978517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8009e5e0 con 0x7ffb801587f0 2023-11-26T07:15:27.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.490+0000 7ffb7d7fa700 1 -- 172.21.15.83:0/645978517 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0038c0 con 0x7ffb801587f0 2023-11-26T07:15:27.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.490+0000 7ffb86fda700 1 -- 172.21.15.83:0/645978517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb801547a0 con 0x7ffb801587f0 2023-11-26T07:15:27.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.491+0000 7ffb86fda700 1 -- 172.21.15.83:0/645978517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb80068fb0 con 0x7ffb801587f0 2023-11-26T07:15:27.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.491+0000 7ffb7d7fa700 1 -- 172.21.15.83:0/645978517 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7ffb6c021030 con 0x7ffb801587f0 2023-11-26T07:15:27.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.492+0000 7ffb7d7fa700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffb6406b230 0x7ffb6406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.492+0000 7ffb7d7fa700 1 -- 172.21.15.83:0/645978517 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb6c08af90 con 0x7ffb801587f0 2023-11-26T07:15:27.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.492+0000 7ffb84d76700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffb6406b230 0x7ffb6406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.492+0000 7ffb84d76700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffb6406b230 0x7ffb6406d6f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009930 tx=0x7ffb74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:27.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.496+0000 7ffb7d7fa700 1 -- 172.21.15.83:0/645978517 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb6c055de0 con 0x7ffb801587f0 2023-11-26T07:15:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.691+0000 7ffb86fda700 1 -- 172.21.15.83:0/645978517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb8015e410 con 0x7ffb801587f0 2023-11-26T07:15:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.692+0000 7ffb7d7fa700 1 -- 172.21.15.83:0/645978517 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb6c0150a0 con 0x7ffb801587f0 2023-11-26T07:15:27.694 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:15:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 -- 172.21.15.83:0/645978517 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffb6406b230 msgr2=0x7ffb6406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffb6406b230 0x7ffb6406d6f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009930 tx=0x7ffb74008040 comp rx=0 tx=0).stop 2023-11-26T07:15:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 -- 172.21.15.83:0/645978517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb801587f0 msgr2=0x7ffb801e7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb801587f0 0x7ffb801e7a10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c000c00 tx=0x7ffb6c004750 comp rx=0 tx=0).stop 2023-11-26T07:15:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 -- 172.21.15.83:0/645978517 shutdown_connections 2023-11-26T07:15:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ffb6406b230 0x7ffb6406d6f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb8015b370 0x7ffb801e7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb800b3c60 0x7ffb80154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 --2- 172.21.15.83:0/645978517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb801587f0 0x7ffb801e7a10 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 -- 172.21.15.83:0/645978517 >> 172.21.15.83:0/645978517 conn(0x7ffb800d0bf0 msgr2=0x7ffb800d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 -- 172.21.15.83:0/645978517 shutdown_connections 2023-11-26T07:15:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.694+0000 7ffb86fda700 1 -- 172.21.15.83:0/645978517 wait complete. 2023-11-26T07:15:27.708 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:15:27.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.826+0000 7fcb0e513700 1 Processor -- start 2023-11-26T07:15:27.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.826+0000 7fcb0e513700 1 -- start start 2023-11-26T07:15:27.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb0e513700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcb0815c160 0x7fcb080622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb0e513700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0815c540 con 0x7fcb080b04b0 2023-11-26T07:15:27.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb0e513700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb080b2ee0 con 0x7fcb080b2870 2023-11-26T07:15:27.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb0e513700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb080add40 con 0x7fcb0815c160 2023-11-26T07:15:27.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb0cab0700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcb080b04b0 0x7fcb08165c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59096/0 (socket says 172.21.15.83:59096) 2023-11-26T07:15:27.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb0cab0700 1 -- 172.21.15.83:0/3333658680 learned_addr learned my addr 172.21.15.83:0/3333658680 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:27.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb07fff700 1 --2- 172.21.15.83:0/3333658680 >> v2:172.21.15.203:3300/0 conn(0x7fcb0815c160 0x7fcb080622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270163967 0 0) 0x7fcb0815c540 con 0x7fcb080b04b0 2023-11-26T07:15:27.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf40035b0 con 0x7fcb080b04b0 2023-11-26T07:15:27.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477006413 0 0) 0x7fcaf40035b0 con 0x7fcb080b04b0 2023-11-26T07:15:27.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 >> v2:172.21.15.203:3300/0 conn(0x7fcb0815c160 msgr2=0x7fcb080622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:27.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb06ffd700 1 --2- 172.21.15.83:0/3333658680 >> v2:172.21.15.203:3300/0 conn(0x7fcb0815c160 0x7fcb080622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 >> v1:172.21.15.203:6789/0 conn(0x7fcb080b2870 legacy=0x7fcb080afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb080ad410 con 0x7fcb080b04b0 2023-11-26T07:15:27.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.827+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fcb000019f0 con 0x7fcb080b04b0 2023-11-26T07:15:27.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 >> v1:172.21.15.83:6789/0 conn(0x7fcb080b04b0 legacy=0x7fcb08165c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb06ffd700 1 --2- 172.21.15.83:0/3333658680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf4003a00 0x7fcaf4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb06ffd700 1 --2- 172.21.15.83:0/3333658680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 0x7fcaf4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb06ffd700 1 --2- 172.21.15.83:0/3333658680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf4005210 0x7fcaf400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb080add40 con 0x7fcaf4004850 2023-11-26T07:15:27.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf400c0b0 con 0x7fcaf4005210 2023-11-26T07:15:27.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf400c270 con 0x7fcaf4003a00 2023-11-26T07:15:27.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb077fe700 1 --2- 172.21.15.83:0/3333658680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 0x7fcaf4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb07fff700 1 --2- 172.21.15.83:0/3333658680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf4003a00 0x7fcaf4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb0cab0700 1 --2- 172.21.15.83:0/3333658680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf4005210 0x7fcaf400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb077fe700 1 -- 172.21.15.83:0/3333658680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf4003a00 msgr2=0x7fcaf4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:27.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb077fe700 1 --2- 172.21.15.83:0/3333658680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcaf4003a00 0x7fcaf4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb077fe700 1 -- 172.21.15.83:0/3333658680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf4005210 msgr2=0x7fcaf400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:27.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb077fe700 1 --2- 172.21.15.83:0/3333658680 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcaf4005210 0x7fcaf400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb077fe700 1 -- 172.21.15.83:0/3333658680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf40035b0 con 0x7fcaf4004850 2023-11-26T07:15:27.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb0cab0700 1 -- 172.21.15.83:0/3333658680 reap_dead start 2023-11-26T07:15:27.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb077fe700 1 --2- 172.21.15.83:0/3333658680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 0x7fcaf4004cd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8001620 tx=0x7fcaf4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3ccc0e4600d37f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:27.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80052d0 con 0x7fcaf4004850 2023-11-26T07:15:27.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf8005450 con 0x7fcaf4004850 2023-11-26T07:15:27.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.828+0000 7fcb06ffd700 1 -- 172.21.15.83:0/3333658680 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80038c0 con 0x7fcaf4004850 2023-11-26T07:15:27.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.829+0000 7fcb0e513700 1 -- 172.21.15.83:0/3333658680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 msgr2=0x7fcaf4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.829+0000 7fcb0e513700 1 --2- 172.21.15.83:0/3333658680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 0x7fcaf4004cd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8001620 tx=0x7fcaf4005880 comp rx=0 tx=0).stop 2023-11-26T07:15:27.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.829+0000 7fcb0e513700 1 -- 172.21.15.83:0/3333658680 shutdown_connections 2023-11-26T07:15:27.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.829+0000 7fcb0e513700 1 --2- 172.21.15.83:0/3333658680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 0x7fcaf4004cd0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.829+0000 7fcb0e513700 1 -- 172.21.15.83:0/3333658680 >> 172.21.15.83:0/3333658680 conn(0x7fcb080b57b0 msgr2=0x7fcb080b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:27.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.829+0000 7fcb0e513700 1 -- 172.21.15.83:0/3333658680 shutdown_connections 2023-11-26T07:15:27.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.829+0000 7fcb0e513700 1 -- 172.21.15.83:0/3333658680 wait complete. 2023-11-26T07:15:27.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.830+0000 7fcb0e513700 1 Processor -- start 2023-11-26T07:15:27.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.830+0000 7fcb0e513700 1 -- start start 2023-11-26T07:15:27.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb0e513700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 0x7fcb081f10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb0e513700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb0815ef70 0x7fcb0814a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb0e513700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb0815a0a0 0x7fcb0815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb0e513700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0815aa90 con 0x7fcaf4004850 2023-11-26T07:15:27.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb0e513700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb081f1620 con 0x7fcb0815a0a0 2023-11-26T07:15:27.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb0e513700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb08158300 con 0x7fcb0815ef70 2023-11-26T07:15:27.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb07fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 0x7fcb081f10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb07fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 0x7fcb081f10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39064/0 (socket says 172.21.15.83:39064) 2023-11-26T07:15:27.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb07fff700 1 -- 172.21.15.83:0/1407867384 learned_addr learned my addr 172.21.15.83:0/1407867384 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:27.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb07fff700 1 -- 172.21.15.83:0/1407867384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb0815ef70 msgr2=0x7fcb0814a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb07fff700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb0815ef70 0x7fcb0814a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb07fff700 1 -- 172.21.15.83:0/1407867384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb0815a0a0 msgr2=0x7fcb0815a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:27.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb07fff700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb0815a0a0 0x7fcb0815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:27.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb07fff700 1 -- 172.21.15.83:0/1407867384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb081585e0 con 0x7fcaf4004850 2023-11-26T07:15:27.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.831+0000 7fcb07fff700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 0x7fcb081f10e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0011aa0 tx=0x7fcaf0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:27.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.832+0000 7fcb04ff9700 1 -- 172.21.15.83:0/1407867384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000f900 con 0x7fcaf4004850 2023-11-26T07:15:27.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.832+0000 7fcb04ff9700 1 -- 172.21.15.83:0/1407867384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf000fa80 con 0x7fcaf4004850 2023-11-26T07:15:27.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.832+0000 7fcb0e513700 1 -- 172.21.15.83:0/1407867384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb081588c0 con 0x7fcaf4004850 2023-11-26T07:15:27.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.832+0000 7fcb04ff9700 1 -- 172.21.15.83:0/1407867384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000fe30 con 0x7fcaf4004850 2023-11-26T07:15:27.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.832+0000 7fcb0e513700 1 -- 172.21.15.83:0/1407867384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb08158e80 con 0x7fcaf4004850 2023-11-26T07:15:27.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.833+0000 7fcb0e513700 1 -- 172.21.15.83:0/1407867384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb08067f60 con 0x7fcaf4004850 2023-11-26T07:15:27.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.833+0000 7fcb04ff9700 1 -- 172.21.15.83:0/1407867384 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fcaf0014440 con 0x7fcaf4004850 2023-11-26T07:15:27.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.833+0000 7fcb04ff9700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcae806b260 0x7fcae806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:27.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.834+0000 7fcb04ff9700 1 -- 172.21.15.83:0/1407867384 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaf008a290 con 0x7fcaf4004850 2023-11-26T07:15:27.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.834+0000 7fcb077fe700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcae806b260 0x7fcae806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:27.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.834+0000 7fcb077fe700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcae806b260 0x7fcae806d720 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcb080b2800 tx=0x7fcaf8010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:27.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:27.838+0000 7fcb04ff9700 1 -- 172.21.15.83:0/1407867384 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcaf0055010 con 0x7fcaf4004850 2023-11-26T07:15:28.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.074+0000 7fcb0e513700 1 -- 172.21.15.83:0/1407867384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb08159070 con 0x7fcaf4004850 2023-11-26T07:15:28.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.074+0000 7fcb04ff9700 1 -- 172.21.15.83:0/1407867384 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4910 (secure 0 0 0) 0x7fcaf0022090 con 0x7fcaf4004850 2023-11-26T07:15:28.077 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:28.077 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/126258309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":126258309},{"type":"v1","addr":"172.21.15.203:6834","nonce":126258309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4203521524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":4203521524},{"type":"v1","addr":"172.21.15.203:6835","nonce":4203521524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1267615204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1267615204},{"type":"v1","addr":"172.21.15.83:6836","nonce":1267615204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:14:54.936357+0000","modified":"2023-11-26T07:14:57.974394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6102":{"gid":6102,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/83108157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":83108157},{"type":"v1","addr":"172.21.15.83:6837","nonce":83108157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:15:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.076+0000 7fcb0e513700 1 -- 172.21.15.83:0/1407867384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcae806b260 msgr2=0x7fcae806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.076+0000 7fcb0e513700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcae806b260 0x7fcae806d720 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcb080b2800 tx=0x7fcaf8010000 comp rx=0 tx=0).stop 2023-11-26T07:15:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.076+0000 7fcb0e513700 1 -- 172.21.15.83:0/1407867384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 msgr2=0x7fcb081f10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.076+0000 7fcb0e513700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 0x7fcb081f10e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0011aa0 tx=0x7fcaf0011e60 comp rx=0 tx=0).stop 2023-11-26T07:15:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.076+0000 7fcb0e513700 1 -- 172.21.15.83:0/1407867384 shutdown_connections 2023-11-26T07:15:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.076+0000 7fcb0e513700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb0815a0a0 0x7fcb0815a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.077+0000 7fcb0e513700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaf4004850 0x7fcb081f10e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.077+0000 7fcb0e513700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcae806b260 0x7fcae806d720 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.077+0000 7fcb0e513700 1 --2- 172.21.15.83:0/1407867384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb0815ef70 0x7fcb0814a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.077+0000 7fcb0e513700 1 -- 172.21.15.83:0/1407867384 >> 172.21.15.83:0/1407867384 conn(0x7fcb080b57b0 msgr2=0x7fcb080d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.077+0000 7fcb0e513700 1 -- 172.21.15.83:0/1407867384 shutdown_connections 2023-11-26T07:15:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.077+0000 7fcb0e513700 1 -- 172.21.15.83:0/1407867384 wait complete. 2023-11-26T07:15:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 38 2023-11-26T07:15:28.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:28.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.208+0000 7ff8d8014700 1 Processor -- start 2023-11-26T07:15:28.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.208+0000 7ff8d8014700 1 -- start start 2023-11-26T07:15:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d8014700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8d00adf00 0x7ff8d006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d8014700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d006c540 0x7ff8d00b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d8014700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d00af950 con 0x7ff8d01446a0 2023-11-26T07:15:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d8014700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d0144a80 con 0x7ff8d00adf00 2023-11-26T07:15:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d8014700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d006c9c0 con 0x7ff8d006c540 2023-11-26T07:15:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d55af700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8d00adf00 0x7ff8d006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d55af700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff8d00adf00 0x7ff8d006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39074/0 (socket says 172.21.15.83:39074) 2023-11-26T07:15:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d55af700 1 -- 172.21.15.83:0/3994469685 learned_addr learned my addr 172.21.15.83:0/3994469685 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d55af700 1 -- 172.21.15.83:0/3994469685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d006c540 msgr2=0x7ff8d00b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d65b1700 1 --2- 172.21.15.83:0/3994469685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d006c540 0x7ff8d00b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d55af700 1 --2- 172.21.15.83:0/3994469685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d006c540 0x7ff8d00b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d55af700 1 -- 172.21.15.83:0/3994469685 >> v1:172.21.15.83:6789/0 conn(0x7ff8d01446a0 legacy=0x7ff8d00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d55af700 1 -- 172.21.15.83:0/3994469685 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d00afc30 con 0x7ff8d00adf00 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d55af700 1 --2- 172.21.15.83:0/3994469685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d00adf00 0x7ff8d006c000 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000bb90 tx=0x7ff8c000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a683042f2d2a64b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.209+0000 7ff8d4dae700 1 -- 172.21.15.83:0/3994469685 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000dd50 con 0x7ff8d00adf00 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.210+0000 7ff8d4dae700 1 -- 172.21.15.83:0/3994469685 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c0010470 con 0x7ff8d00adf00 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.210+0000 7ff8d4dae700 1 -- 172.21.15.83:0/3994469685 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0010800 con 0x7ff8d00adf00 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.210+0000 7ff8d8014700 1 -- 172.21.15.83:0/3994469685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d00adf00 msgr2=0x7ff8d006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.210+0000 7ff8d8014700 1 --2- 172.21.15.83:0/3994469685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d00adf00 0x7ff8d006c000 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000bb90 tx=0x7ff8c000bea0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.210+0000 7ff8d8014700 1 -- 172.21.15.83:0/3994469685 shutdown_connections 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.210+0000 7ff8d8014700 1 --2- 172.21.15.83:0/3994469685 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d006c540 0x7ff8d00b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.210+0000 7ff8d8014700 1 --2- 172.21.15.83:0/3994469685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d00adf00 0x7ff8d006c000 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.210+0000 7ff8d8014700 1 -- 172.21.15.83:0/3994469685 >> 172.21.15.83:0/3994469685 conn(0x7ff8d0071b90 msgr2=0x7ff8d0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.210+0000 7ff8d8014700 1 -- 172.21.15.83:0/3994469685 shutdown_connections 2023-11-26T07:15:28.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.210+0000 7ff8d8014700 1 -- 172.21.15.83:0/3994469685 wait complete. 2023-11-26T07:15:28.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.212+0000 7ff8d8014700 1 Processor -- start 2023-11-26T07:15:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.212+0000 7ff8d8014700 1 -- start start 2023-11-26T07:15:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d8014700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d006c540 0x7ff8d0159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d8014700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d00adf00 0x7ff8d0140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d8014700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d01446a0 0x7ff8d0073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d8014700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0141220 con 0x7ff8d00adf00 2023-11-26T07:15:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d8014700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0141500 con 0x7ff8d01446a0 2023-11-26T07:15:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d8014700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00cd600 con 0x7ff8d006c540 2023-11-26T07:15:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d55af700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d00adf00 0x7ff8d0140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d55af700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d00adf00 0x7ff8d0140840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39080/0 (socket says 172.21.15.83:39080) 2023-11-26T07:15:28.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d55af700 1 -- 172.21.15.83:0/521908819 learned_addr learned my addr 172.21.15.83:0/521908819 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d5db0700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d006c540 0x7ff8d0159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d65b1700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d01446a0 0x7ff8d0073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d55af700 1 -- 172.21.15.83:0/521908819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d006c540 msgr2=0x7ff8d0159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d55af700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d006c540 0x7ff8d0159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d55af700 1 -- 172.21.15.83:0/521908819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d01446a0 msgr2=0x7ff8d0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d55af700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d01446a0 0x7ff8d0073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d55af700 1 -- 172.21.15.83:0/521908819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c000b7f0 con 0x7ff8d00adf00 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d65b1700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d01446a0 0x7ff8d0073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d55af700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d00adf00 0x7ff8d0140840 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0000c00 tx=0x7ff8c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8c67fc700 1 -- 172.21.15.83:0/521908819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c001d030 con 0x7ff8d00adf00 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8c67fc700 1 -- 172.21.15.83:0/521908819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c001e040 con 0x7ff8d00adf00 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d8014700 1 -- 172.21.15.83:0/521908819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d00cd8e0 con 0x7ff8d00adf00 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8c67fc700 1 -- 172.21.15.83:0/521908819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00036a0 con 0x7ff8d00adf00 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.213+0000 7ff8d8014700 1 -- 172.21.15.83:0/521908819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d0159dc0 con 0x7ff8d00adf00 2023-11-26T07:15:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.214+0000 7ff8d8014700 1 -- 172.21.15.83:0/521908819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d00753f0 con 0x7ff8d00adf00 2023-11-26T07:15:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.215+0000 7ff8c67fc700 1 -- 172.21.15.83:0/521908819 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7ff8c0003800 con 0x7ff8d00adf00 2023-11-26T07:15:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.215+0000 7ff8c67fc700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8b406b230 0x7ff8b406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.215+0000 7ff8d5db0700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8b406b230 0x7ff8b406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.215+0000 7ff8c67fc700 1 -- 172.21.15.83:0/521908819 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8c008ddf0 con 0x7ff8d00adf00 2023-11-26T07:15:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.216+0000 7ff8d5db0700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8b406b230 0x7ff8b406d6f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc001620 tx=0x7ff8cc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:28.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.220+0000 7ff8c67fc700 1 -- 172.21.15.83:0/521908819 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff8c0058a10 con 0x7ff8d00adf00 2023-11-26T07:15:28.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.411+0000 7ff8d8014700 1 -- 172.21.15.83:0/521908819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8d0067f60 con 0x7ff8d00adf00 2023-11-26T07:15:28.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.412+0000 7ff8c67fc700 1 -- 172.21.15.83:0/521908819 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16553 (secure 0 0 0) 0x7ff8c005c030 con 0x7ff8d00adf00 2023-11-26T07:15:28.414 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":58,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:14:56.536771+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:14:53.044716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:14:53.904013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 -- 172.21.15.83:0/521908819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8b406b230 msgr2=0x7ff8b406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8b406b230 0x7ff8b406d6f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc001620 tx=0x7ff8cc009040 comp rx=0 tx=0).stop 2023-11-26T07:15:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 -- 172.21.15.83:0/521908819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d00adf00 msgr2=0x7ff8d0140840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d00adf00 0x7ff8d0140840 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0000c00 tx=0x7ff8c0004750 comp rx=0 tx=0).stop 2023-11-26T07:15:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 -- 172.21.15.83:0/521908819 shutdown_connections 2023-11-26T07:15:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8d01446a0 0x7ff8d0073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff8b406b230 0x7ff8b406d6f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8d006c540 0x7ff8d0159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 --2- 172.21.15.83:0/521908819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d00adf00 0x7ff8d0140840 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 -- 172.21.15.83:0/521908819 >> 172.21.15.83:0/521908819 conn(0x7ff8d0071b90 msgr2=0x7ff8d00b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:28.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 -- 172.21.15.83:0/521908819 shutdown_connections 2023-11-26T07:15:28.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.414+0000 7ff8d8014700 1 -- 172.21.15.83:0/521908819 wait complete. 2023-11-26T07:15:28.428 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:15:28.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:15:28.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.546+0000 7f147e0ba700 1 Processor -- start 2023-11-26T07:15:28.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.546+0000 7f147e0ba700 1 -- start start 2023-11-26T07:15:28.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.547+0000 7f147e0ba700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f147815e2a0 0x7f147806ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.547+0000 7f147e0ba700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14780ce090 0x7f147806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.547+0000 7f147e0ba700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147806e7f0 con 0x7f1478073f90 2023-11-26T07:15:28.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.547+0000 7f147e0ba700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147806dec0 con 0x7f14780ce090 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.547+0000 7f147e0ba700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147806e1a0 con 0x7f147815e2a0 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.547+0000 7f1476ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14780ce090 0x7f147806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.547+0000 7f1476ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14780ce090 0x7f147806ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39090/0 (socket says 172.21.15.83:39090) 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.547+0000 7f1476ffd700 1 -- 172.21.15.83:0/455596531 learned_addr learned my addr 172.21.15.83:0/455596531 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f14777fe700 1 --2- 172.21.15.83:0/455596531 >> v2:172.21.15.203:3300/0 conn(0x7f147815e2a0 0x7f147806ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f1476ffd700 1 -- 172.21.15.83:0/455596531 >> v2:172.21.15.203:3300/0 conn(0x7f147815e2a0 msgr2=0x7f147806ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f1476ffd700 1 --2- 172.21.15.83:0/455596531 >> v2:172.21.15.203:3300/0 conn(0x7f147815e2a0 0x7f147806ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f1476ffd700 1 -- 172.21.15.83:0/455596531 >> v1:172.21.15.203:6789/0 conn(0x7f1478073f90 legacy=0x7f14780cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f1476ffd700 1 -- 172.21.15.83:0/455596531 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147806e480 con 0x7f14780ce090 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f1476ffd700 1 --2- 172.21.15.83:0/455596531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14780ce090 0x7f147806ec80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f147806d0c0 tx=0x7f1468009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562dd102db3452dd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f14767fc700 1 -- 172.21.15.83:0/455596531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468004230 con 0x7f14780ce090 2023-11-26T07:15:28.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f14767fc700 1 -- 172.21.15.83:0/455596531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1468004390 con 0x7f14780ce090 2023-11-26T07:15:28.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f14767fc700 1 -- 172.21.15.83:0/455596531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14680036a0 con 0x7f14780ce090 2023-11-26T07:15:28.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f147e0ba700 1 -- 172.21.15.83:0/455596531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14780ce090 msgr2=0x7f147806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f147e0ba700 1 --2- 172.21.15.83:0/455596531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14780ce090 0x7f147806ec80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f147806d0c0 tx=0x7f1468009bd0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f147e0ba700 1 -- 172.21.15.83:0/455596531 shutdown_connections 2023-11-26T07:15:28.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f147e0ba700 1 --2- 172.21.15.83:0/455596531 >> v2:172.21.15.203:3300/0 conn(0x7f147815e2a0 0x7f147806ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.548+0000 7f147e0ba700 1 --2- 172.21.15.83:0/455596531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14780ce090 0x7f147806ec80 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.549+0000 7f147e0ba700 1 -- 172.21.15.83:0/455596531 >> 172.21.15.83:0/455596531 conn(0x7f14780cd570 msgr2=0x7f14780cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:28.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.549+0000 7f147e0ba700 1 -- 172.21.15.83:0/455596531 shutdown_connections 2023-11-26T07:15:28.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.549+0000 7f147e0ba700 1 -- 172.21.15.83:0/455596531 wait complete. 2023-11-26T07:15:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.550+0000 7f147e0ba700 1 Processor -- start 2023-11-26T07:15:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.550+0000 7f147e0ba700 1 -- start start 2023-11-26T07:15:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f147e0ba700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1478073f90 0x7f14781f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f147e0ba700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14780ce090 0x7f1478160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f147e0ba700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f147815e2a0 0x7f1478155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f147e0ba700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1478161770 con 0x7f14780ce090 2023-11-26T07:15:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f147e0ba700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1478161490 con 0x7f147815e2a0 2023-11-26T07:15:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f147e0ba700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14781f7160 con 0x7f1478073f90 2023-11-26T07:15:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f1476ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14780ce090 0x7f1478160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f1476ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14780ce090 0x7f1478160c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39094/0 (socket says 172.21.15.83:39094) 2023-11-26T07:15:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f1476ffd700 1 -- 172.21.15.83:0/2283983101 learned_addr learned my addr 172.21.15.83:0/2283983101 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f14777fe700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1478073f90 0x7f14781f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f1477fff700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f147815e2a0 0x7f1478155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f1476ffd700 1 -- 172.21.15.83:0/2283983101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1478073f90 msgr2=0x7f14781f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f1476ffd700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1478073f90 0x7f14781f4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f1476ffd700 1 -- 172.21.15.83:0/2283983101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f147815e2a0 msgr2=0x7f1478155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f1476ffd700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f147815e2a0 0x7f1478155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f1476ffd700 1 -- 172.21.15.83:0/2283983101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1468009550 con 0x7f14780ce090 2023-11-26T07:15:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f1476ffd700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14780ce090 0x7f1478160c60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f1468009130 tx=0x7f1468004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f145ffff700 1 -- 172.21.15.83:0/2283983101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468003d80 con 0x7f14780ce090 2023-11-26T07:15:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f145ffff700 1 -- 172.21.15.83:0/2283983101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146801f070 con 0x7f14780ce090 2023-11-26T07:15:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f145ffff700 1 -- 172.21.15.83:0/2283983101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468011870 con 0x7f14780ce090 2023-11-26T07:15:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f147e0ba700 1 -- 172.21.15.83:0/2283983101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14781f7440 con 0x7f14780ce090 2023-11-26T07:15:28.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.551+0000 7f147e0ba700 1 -- 172.21.15.83:0/2283983101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14781611a0 con 0x7f14780ce090 2023-11-26T07:15:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.552+0000 7f147e0ba700 1 -- 172.21.15.83:0/2283983101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147806c1c0 con 0x7f14780ce090 2023-11-26T07:15:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.553+0000 7f145ffff700 1 -- 172.21.15.83:0/2283983101 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f1468021030 con 0x7f14780ce090 2023-11-26T07:15:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.553+0000 7f145ffff700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f146406b230 0x7f146406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.553+0000 7f145ffff700 1 -- 172.21.15.83:0/2283983101 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f146808b160 con 0x7f14780ce090 2023-11-26T07:15:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.553+0000 7f14777fe700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f146406b230 0x7f146406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.554+0000 7f14777fe700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f146406b230 0x7f146406d6f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1478072a80 tx=0x7f146c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:28.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.558+0000 7f145ffff700 1 -- 172.21.15.83:0/2283983101 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1468055fb0 con 0x7f14780ce090 2023-11-26T07:15:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.792+0000 7f147e0ba700 1 -- 172.21.15.83:0/2283983101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1478067f60 con 0x7f14780ce090 2023-11-26T07:15:28.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.793+0000 7f145ffff700 1 -- 172.21.15.83:0/2283983101 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f14680595d0 con 0x7f14780ce090 2023-11-26T07:15:28.795 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:28.795 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/126258309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":126258309},{"type":"v1","addr":"172.21.15.203:6834","nonce":126258309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4203521524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":4203521524},{"type":"v1","addr":"172.21.15.203:6835","nonce":4203521524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1267615204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1267615204},{"type":"v1","addr":"172.21.15.83:6836","nonce":1267615204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:14:54.936357+0000","modified":"2023-11-26T07:14:57.974394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6102":{"gid":6102,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/83108157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":83108157},{"type":"v1","addr":"172.21.15.83:6837","nonce":83108157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:15:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 -- 172.21.15.83:0/2283983101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f146406b230 msgr2=0x7f146406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f146406b230 0x7f146406d6f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1478072a80 tx=0x7f146c008040 comp rx=0 tx=0).stop 2023-11-26T07:15:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 -- 172.21.15.83:0/2283983101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14780ce090 msgr2=0x7f1478160c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14780ce090 0x7f1478160c60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f1468009130 tx=0x7f1468004750 comp rx=0 tx=0).stop 2023-11-26T07:15:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 -- 172.21.15.83:0/2283983101 shutdown_connections 2023-11-26T07:15:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f147815e2a0 0x7f1478155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f146406b230 0x7f146406d6f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1478073f90 0x7f14781f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 --2- 172.21.15.83:0/2283983101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14780ce090 0x7f1478160c60 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 -- 172.21.15.83:0/2283983101 >> 172.21.15.83:0/2283983101 conn(0x7f14780cd570 msgr2=0x7f14780cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 -- 172.21.15.83:0/2283983101 shutdown_connections 2023-11-26T07:15:28.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.795+0000 7f147e0ba700 1 -- 172.21.15.83:0/2283983101 wait complete. 2023-11-26T07:15:28.799 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 38 2023-11-26T07:15:28.810 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:15:28.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.927+0000 7f186d0d5700 1 Processor -- start 2023-11-26T07:15:28.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.927+0000 7f186d0d5700 1 -- start start 2023-11-26T07:15:28.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186d0d5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f186815e080 0x7f18680b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186d0d5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18680af4d0 0x7f18680afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186d0d5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f186806c000 0x7f186806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186d0d5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18680af0b0 con 0x7f18680af4d0 2023-11-26T07:15:28.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186d0d5700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186806c9f0 con 0x7f186815e080 2023-11-26T07:15:28.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186d0d5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18680b05a0 con 0x7f186806c000 2023-11-26T07:15:28.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18680af4d0 0x7f18680afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186659c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18680af4d0 0x7f18680afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39100/0 (socket says 172.21.15.83:39100) 2023-11-26T07:15:28.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186659c700 1 -- 172.21.15.83:0/2379995670 learned_addr learned my addr 172.21.15.83:0/2379995670 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:28.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f1866d9d700 1 --2- 172.21.15.83:0/2379995670 >> v2:172.21.15.203:3300/0 conn(0x7f186815e080 0x7f18680b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186759e700 1 --2- 172.21.15.83:0/2379995670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f186806c000 0x7f186806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186659c700 1 -- 172.21.15.83:0/2379995670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f186806c000 msgr2=0x7f186806c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:28.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186659c700 1 --2- 172.21.15.83:0/2379995670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f186806c000 0x7f186806c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186659c700 1 -- 172.21.15.83:0/2379995670 >> v2:172.21.15.203:3300/0 conn(0x7f186815e080 msgr2=0x7f18680b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:28.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186659c700 1 --2- 172.21.15.83:0/2379995670 >> v2:172.21.15.203:3300/0 conn(0x7f186815e080 0x7f18680b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186659c700 1 -- 172.21.15.83:0/2379995670 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18680ae430 con 0x7f18680af4d0 2023-11-26T07:15:28.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f1866d9d700 1 --2- 172.21.15.83:0/2379995670 >> v2:172.21.15.203:3300/0 conn(0x7f186815e080 0x7f18680b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:28.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186759e700 1 --2- 172.21.15.83:0/2379995670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f186806c000 0x7f186806c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:28.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f186659c700 1 --2- 172.21.15.83:0/2379995670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18680af4d0 0x7f18680afb90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1868072a90 tx=0x7f185c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a98f80d37275bcf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:28.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f1865d9b700 1 -- 172.21.15.83:0/2379995670 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00b870 con 0x7f18680af4d0 2023-11-26T07:15:28.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f1865d9b700 1 -- 172.21.15.83:0/2379995670 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c00b9d0 con 0x7f18680af4d0 2023-11-26T07:15:28.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.928+0000 7f1865d9b700 1 -- 172.21.15.83:0/2379995670 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00bd60 con 0x7f18680af4d0 2023-11-26T07:15:28.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.929+0000 7f186d0d5700 1 -- 172.21.15.83:0/2379995670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18680af4d0 msgr2=0x7f18680afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.929+0000 7f186d0d5700 1 --2- 172.21.15.83:0/2379995670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18680af4d0 0x7f18680afb90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1868072a90 tx=0x7f185c009bd0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.929+0000 7f186d0d5700 1 -- 172.21.15.83:0/2379995670 shutdown_connections 2023-11-26T07:15:28.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.929+0000 7f186d0d5700 1 --2- 172.21.15.83:0/2379995670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f186806c000 0x7f186806c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.929+0000 7f186d0d5700 1 --2- 172.21.15.83:0/2379995670 >> v2:172.21.15.203:3300/0 conn(0x7f186815e080 0x7f18680b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.929+0000 7f186d0d5700 1 --2- 172.21.15.83:0/2379995670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18680af4d0 0x7f18680afb90 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.929+0000 7f186d0d5700 1 -- 172.21.15.83:0/2379995670 >> 172.21.15.83:0/2379995670 conn(0x7f1868071b90 msgr2=0x7f1868071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:28.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.929+0000 7f186d0d5700 1 -- 172.21.15.83:0/2379995670 shutdown_connections 2023-11-26T07:15:28.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.929+0000 7f186d0d5700 1 -- 172.21.15.83:0/2379995670 wait complete. 2023-11-26T07:15:28.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.930+0000 7f186d0d5700 1 Processor -- start 2023-11-26T07:15:28.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.930+0000 7f186d0d5700 1 -- start start 2023-11-26T07:15:28.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.931+0000 7f186d0d5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f186806c000 0x7f18680cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.931+0000 7f186d0d5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18680af4d0 0x7f18681603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.931+0000 7f186d0d5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f186815e080 0x7f18681608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.931+0000 7f186d0d5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186815f810 con 0x7f186815e080 2023-11-26T07:15:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.931+0000 7f186d0d5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18681576f0 con 0x7f186806c000 2023-11-26T07:15:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.931+0000 7f186d0d5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18681f3640 con 0x7f18680af4d0 2023-11-26T07:15:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.931+0000 7f186759e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f186815e080 0x7f18681608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.931+0000 7f186759e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f186815e080 0x7f18681608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39110/0 (socket says 172.21.15.83:39110) 2023-11-26T07:15:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.931+0000 7f186759e700 1 -- 172.21.15.83:0/3166807095 learned_addr learned my addr 172.21.15.83:0/3166807095 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.931+0000 7f186659c700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18680af4d0 0x7f18681603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.931+0000 7f1866d9d700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f186806c000 0x7f18680cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f186759e700 1 -- 172.21.15.83:0/3166807095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18680af4d0 msgr2=0x7f18681603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f186759e700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18680af4d0 0x7f18681603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f186759e700 1 -- 172.21.15.83:0/3166807095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f186806c000 msgr2=0x7f18680cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f186759e700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f186806c000 0x7f18680cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f186759e700 1 -- 172.21.15.83:0/3166807095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009550 con 0x7f186815e080 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f1866d9d700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f186806c000 0x7f18680cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f186759e700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f186815e080 0x7f18681608f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f186000cf20 tx=0x7f186000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f184f7fe700 1 -- 172.21.15.83:0/3166807095 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186000ad50 con 0x7f186815e080 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f184f7fe700 1 -- 172.21.15.83:0/3166807095 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1860015440 con 0x7f186815e080 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f186d0d5700 1 -- 172.21.15.83:0/3166807095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186815f250 con 0x7f186815e080 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f184f7fe700 1 -- 172.21.15.83:0/3166807095 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18600157f0 con 0x7f186815e080 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.932+0000 7f186d0d5700 1 -- 172.21.15.83:0/3166807095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1868157380 con 0x7f186815e080 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.933+0000 7f186d0d5700 1 -- 172.21.15.83:0/3166807095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18680753f0 con 0x7f186815e080 2023-11-26T07:15:28.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.933+0000 7f184f7fe700 1 -- 172.21.15.83:0/3166807095 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f186000d430 con 0x7f186815e080 2023-11-26T07:15:28.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.934+0000 7f184f7fe700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f185406b230 0x7f185406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:28.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.934+0000 7f184f7fe700 1 -- 172.21.15.83:0/3166807095 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1860089fa0 con 0x7f186815e080 2023-11-26T07:15:28.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.934+0000 7f1866d9d700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f185406b230 0x7f185406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:28.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.934+0000 7f1866d9d700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f185406b230 0x7f185406d6f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f185800aa20 tx=0x7f1858008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:28.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:28.938+0000 7f184f7fe700 1 -- 172.21.15.83:0/3166807095 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1860054cd0 con 0x7f186815e080 2023-11-26T07:15:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.168+0000 7f186d0d5700 1 -- 172.21.15.83:0/3166807095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1868067f60 con 0x7f186815e080 2023-11-26T07:15:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.169+0000 7f184f7fe700 1 -- 172.21.15.83:0/3166807095 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f186001a020 con 0x7f186815e080 2023-11-26T07:15:29.172 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:29.172 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/126258309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":126258309},{"type":"v1","addr":"172.21.15.203:6834","nonce":126258309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4203521524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":4203521524},{"type":"v1","addr":"172.21.15.203:6835","nonce":4203521524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1267615204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1267615204},{"type":"v1","addr":"172.21.15.83:6836","nonce":1267615204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:14:54.936357+0000","modified":"2023-11-26T07:14:57.974394+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6102},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6102":{"gid":6102,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6837/83108157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":83108157},{"type":"v1","addr":"172.21.15.83:6837","nonce":83108157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:15:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 -- 172.21.15.83:0/3166807095 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f185406b230 msgr2=0x7f185406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f185406b230 0x7f185406d6f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f185800aa20 tx=0x7f1858008040 comp rx=0 tx=0).stop 2023-11-26T07:15:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 -- 172.21.15.83:0/3166807095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f186815e080 msgr2=0x7f18681608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f186815e080 0x7f18681608f0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f186000cf20 tx=0x7f186000cf50 comp rx=0 tx=0).stop 2023-11-26T07:15:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 -- 172.21.15.83:0/3166807095 shutdown_connections 2023-11-26T07:15:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f186815e080 0x7f18681608f0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f186806c000 0x7f18680cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f185406b230 0x7f185406d6f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 --2- 172.21.15.83:0/3166807095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18680af4d0 0x7f18681603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 -- 172.21.15.83:0/3166807095 >> 172.21.15.83:0/3166807095 conn(0x7f1868071b90 msgr2=0x7f18680d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 -- 172.21.15.83:0/3166807095 shutdown_connections 2023-11-26T07:15:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.171+0000 7f186d0d5700 1 -- 172.21.15.83:0/3166807095 wait complete. 2023-11-26T07:15:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 38 2023-11-26T07:15:29.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.302+0000 7f2273a1f700 1 Processor -- start 2023-11-26T07:15:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.303+0000 7f2273a1f700 1 -- start start 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.303+0000 7f2273a1f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f226c15bd60 0x7f226c0d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.303+0000 7f2273a1f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226c0cc780 con 0x7f226c0cf2b0 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.303+0000 7f2273a1f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226c0cc9b0 con 0x7f226c15c140 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.303+0000 7f2273a1f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226c0ccc90 con 0x7f226c15bd60 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.303+0000 7f2271fbc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f226c0cf2b0 0x7f226c0c57c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59158/0 (socket says 172.21.15.83:59158) 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2271fbc700 1 -- 172.21.15.83:0/4158898129 learned_addr learned my addr 172.21.15.83:0/4158898129 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361516296 0 0) 0x7f226c0cc780 con 0x7f226c0cf2b0 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22580035b0 con 0x7f226c0cf2b0 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f22717bb700 1 --2- 172.21.15.83:0/4158898129 >> v2:172.21.15.203:3300/0 conn(0x7f226c15bd60 0x7f226c0d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3647159799 0 0) 0x7f22580035b0 con 0x7f226c0cf2b0 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 >> v2:172.21.15.203:3300/0 conn(0x7f226c15bd60 msgr2=0x7f226c0d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 --2- 172.21.15.83:0/4158898129 >> v2:172.21.15.203:3300/0 conn(0x7f226c15bd60 0x7f226c0d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 >> v1:172.21.15.203:6789/0 conn(0x7f226c15c140 legacy=0x7f226c0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0ccf70 con 0x7f226c0cf2b0 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f22640019f0 con 0x7f226c0cf2b0 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 >> v1:172.21.15.83:6789/0 conn(0x7f226c0cf2b0 legacy=0x7f226c0c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 --2- 172.21.15.83:0/4158898129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2258003a00 0x7f2258003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 --2- 172.21.15.83:0/4158898129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2258004850 0x7f2258004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 --2- 172.21.15.83:0/4158898129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2258005210 0x7f225800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0ccc90 con 0x7f2258004850 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225800c0b0 con 0x7f2258003a00 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225800c270 con 0x7f2258005210 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2270fba700 1 --2- 172.21.15.83:0/4158898129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2258004850 0x7f2258004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f22717bb700 1 --2- 172.21.15.83:0/4158898129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2258003a00 0x7f2258003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.304+0000 7f2271fbc700 1 --2- 172.21.15.83:0/4158898129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2258005210 0x7f225800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2270fba700 1 -- 172.21.15.83:0/4158898129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2258005210 msgr2=0x7f225800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2270fba700 1 --2- 172.21.15.83:0/4158898129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2258005210 0x7f225800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2270fba700 1 -- 172.21.15.83:0/4158898129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2258003a00 msgr2=0x7f2258003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2270fba700 1 --2- 172.21.15.83:0/4158898129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2258003a00 0x7f2258003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2270fba700 1 -- 172.21.15.83:0/4158898129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22580035b0 con 0x7f2258004850 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2271fbc700 1 -- 172.21.15.83:0/4158898129 reap_dead start 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2270fba700 1 --2- 172.21.15.83:0/4158898129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2258004850 0x7f2258004cd0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f22680017c0 tx=0x7f2258005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c675094ec19b383f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22680052d0 con 0x7f2258004850 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2268005430 con 0x7f2258004850 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2263fff700 1 -- 172.21.15.83:0/4158898129 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22680038c0 con 0x7f2258004850 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2273a1f700 1 -- 172.21.15.83:0/4158898129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2258004850 msgr2=0x7f2258004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2273a1f700 1 --2- 172.21.15.83:0/4158898129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2258004850 0x7f2258004cd0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f22680017c0 tx=0x7f2258005880 comp rx=0 tx=0).stop 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2273a1f700 1 -- 172.21.15.83:0/4158898129 shutdown_connections 2023-11-26T07:15:29.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2273a1f700 1 --2- 172.21.15.83:0/4158898129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2258004850 0x7f2258004cd0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2273a1f700 1 -- 172.21.15.83:0/4158898129 >> 172.21.15.83:0/4158898129 conn(0x7f226c073130 msgr2=0x7f226c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:29.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2273a1f700 1 -- 172.21.15.83:0/4158898129 shutdown_connections 2023-11-26T07:15:29.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.305+0000 7f2273a1f700 1 -- 172.21.15.83:0/4158898129 wait complete. 2023-11-26T07:15:29.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.307+0000 7f2273a1f700 1 Processor -- start 2023-11-26T07:15:29.312 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.307+0000 7f2273a1f700 1 -- start start 2023-11-26T07:15:29.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.307+0000 7f2273a1f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2258004850 0x7f226c0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.307+0000 7f2273a1f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f226c159ce0 0x7f226c153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.307+0000 7f2273a1f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f226c15a160 0x7f226c153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.307+0000 7f2273a1f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c151df0 con 0x7f226c15a160 2023-11-26T07:15:29.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.307+0000 7f2273a1f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c151ac0 con 0x7f226c159ce0 2023-11-26T07:15:29.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.307+0000 7f2273a1f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c1545d0 con 0x7f2258004850 2023-11-26T07:15:29.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.307+0000 7f2271fbc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f226c15a160 0x7f226c153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.307+0000 7f2271fbc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f226c15a160 0x7f226c153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39130/0 (socket says 172.21.15.83:39130) 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.307+0000 7f2271fbc700 1 -- 172.21.15.83:0/966643830 learned_addr learned my addr 172.21.15.83:0/966643830 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2270fba700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f226c159ce0 0x7f226c153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f22717bb700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2258004850 0x7f226c0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2271fbc700 1 -- 172.21.15.83:0/966643830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2258004850 msgr2=0x7f226c0c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2271fbc700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2258004850 0x7f226c0c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2271fbc700 1 -- 172.21.15.83:0/966643830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f226c159ce0 msgr2=0x7f226c153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2271fbc700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f226c159ce0 0x7f226c153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2271fbc700 1 -- 172.21.15.83:0/966643830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c1548b0 con 0x7f226c15a160 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2271fbc700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f226c15a160 0x7f226c153fa0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f226c0cf720 tx=0x7f2264007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2261ffb700 1 -- 172.21.15.83:0/966643830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400eb40 con 0x7f226c15a160 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2261ffb700 1 -- 172.21.15.83:0/966643830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400ecc0 con 0x7f226c15a160 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2273a1f700 1 -- 172.21.15.83:0/966643830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226c153210 con 0x7f226c15a160 2023-11-26T07:15:29.315 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2261ffb700 1 -- 172.21.15.83:0/966643830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264013420 con 0x7f226c15a160 2023-11-26T07:15:29.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.308+0000 7f2273a1f700 1 -- 172.21.15.83:0/966643830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c154b90 con 0x7f226c15a160 2023-11-26T07:15:29.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.309+0000 7f2273a1f700 1 -- 172.21.15.83:0/966643830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c067f60 con 0x7f226c15a160 2023-11-26T07:15:29.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.309+0000 7f2261ffb700 1 -- 172.21.15.83:0/966643830 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f2264013580 con 0x7f226c15a160 2023-11-26T07:15:29.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.310+0000 7f2261ffb700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225006b230 0x7f225006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.310+0000 7f2261ffb700 1 -- 172.21.15.83:0/966643830 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2264089fc0 con 0x7f226c15a160 2023-11-26T07:15:29.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.310+0000 7f22717bb700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225006b230 0x7f225006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.310+0000 7f22717bb700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225006b230 0x7f225006d6f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f225c009ce0 tx=0x7f225c009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:29.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.314+0000 7f2261ffb700 1 -- 172.21.15.83:0/966643830 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2264054c60 con 0x7f226c15a160 2023-11-26T07:15:29.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.505+0000 7f2273a1f700 1 -- 172.21.15.83:0/966643830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f226c070ac0 con 0x7f226c15a160 2023-11-26T07:15:29.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.506+0000 7f2261ffb700 1 -- 172.21.15.83:0/966643830 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f2264058280 con 0x7f226c15a160 2023-11-26T07:15:29.509 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:29.509 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":58,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:14:56.536771+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:14:53.044716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:14:53.904013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:29.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 -- 172.21.15.83:0/966643830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225006b230 msgr2=0x7f225006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225006b230 0x7f225006d6f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f225c009ce0 tx=0x7f225c009210 comp rx=0 tx=0).stop 2023-11-26T07:15:29.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 -- 172.21.15.83:0/966643830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f226c15a160 msgr2=0x7f226c153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f226c15a160 0x7f226c153fa0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f226c0cf720 tx=0x7f2264007c90 comp rx=0 tx=0).stop 2023-11-26T07:15:29.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 -- 172.21.15.83:0/966643830 shutdown_connections 2023-11-26T07:15:29.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f226c15a160 0x7f226c153fa0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f225006b230 0x7f225006d6f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2258004850 0x7f226c0c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 --2- 172.21.15.83:0/966643830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f226c159ce0 0x7f226c153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 -- 172.21.15.83:0/966643830 >> 172.21.15.83:0/966643830 conn(0x7f226c073130 msgr2=0x7f226c06f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:29.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 -- 172.21.15.83:0/966643830 shutdown_connections 2023-11-26T07:15:29.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.508+0000 7f2273a1f700 1 -- 172.21.15.83:0/966643830 wait complete. 2023-11-26T07:15:29.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:15:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.642+0000 7f81f0968700 1 Processor -- start 2023-11-26T07:15:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.642+0000 7f81f0968700 1 -- start start 2023-11-26T07:15:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81f0968700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f81ec0b0ba0 0x7f81ec0af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81f0968700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81ec0b0f80 0x7f81ec0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81f0968700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81ec15ee90 0x7f81ec0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81f0968700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec0b2bd0 con 0x7f81ec15ee90 2023-11-26T07:15:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81f0968700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec0b1fc0 con 0x7f81ec0b0ba0 2023-11-26T07:15:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81f0968700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec0b22a0 con 0x7f81ec0b0f80 2023-11-26T07:15:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81ead9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81ec15ee90 0x7f81ec0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81ead9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81ec15ee90 0x7f81ec0ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39138/0 (socket says 172.21.15.83:39138) 2023-11-26T07:15:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81ead9d700 1 -- 172.21.15.83:0/668315922 learned_addr learned my addr 172.21.15.83:0/668315922 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81e9d9b700 1 --2- 172.21.15.83:0/668315922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81ec0b0f80 0x7f81ec0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81ea59c700 1 --2- 172.21.15.83:0/668315922 >> v2:172.21.15.203:3300/0 conn(0x7f81ec0b0ba0 0x7f81ec0af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81ead9d700 1 -- 172.21.15.83:0/668315922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81ec0b0f80 msgr2=0x7f81ec0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81ead9d700 1 --2- 172.21.15.83:0/668315922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81ec0b0f80 0x7f81ec0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81ead9d700 1 -- 172.21.15.83:0/668315922 >> v2:172.21.15.203:3300/0 conn(0x7f81ec0b0ba0 msgr2=0x7f81ec0af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81ead9d700 1 --2- 172.21.15.83:0/668315922 >> v2:172.21.15.203:3300/0 conn(0x7f81ec0b0ba0 0x7f81ec0af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81ead9d700 1 -- 172.21.15.83:0/668315922 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec0b2580 con 0x7f81ec15ee90 2023-11-26T07:15:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81e9d9b700 1 --2- 172.21.15.83:0/668315922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81ec0b0f80 0x7f81ec0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81ea59c700 1 --2- 172.21.15.83:0/668315922 >> v2:172.21.15.203:3300/0 conn(0x7f81ec0b0ba0 0x7f81ec0af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.643+0000 7f81ead9d700 1 --2- 172.21.15.83:0/668315922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec15ee90 0x7f81ec0ccca0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0d1340 tx=0x7f81e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2247bfa61fcffe72 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81e959a700 1 -- 172.21.15.83:0/668315922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400e070 con 0x7f81ec15ee90 2023-11-26T07:15:29.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81e959a700 1 -- 172.21.15.83:0/668315922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e400b9c0 con 0x7f81ec15ee90 2023-11-26T07:15:29.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81e959a700 1 -- 172.21.15.83:0/668315922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400bd70 con 0x7f81ec15ee90 2023-11-26T07:15:29.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81f0968700 1 -- 172.21.15.83:0/668315922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec15ee90 msgr2=0x7f81ec0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81f0968700 1 --2- 172.21.15.83:0/668315922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec15ee90 0x7f81ec0ccca0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0d1340 tx=0x7f81e400b330 comp rx=0 tx=0).stop 2023-11-26T07:15:29.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81f0968700 1 -- 172.21.15.83:0/668315922 shutdown_connections 2023-11-26T07:15:29.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81f0968700 1 --2- 172.21.15.83:0/668315922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec15ee90 0x7f81ec0ccca0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81f0968700 1 --2- 172.21.15.83:0/668315922 >> v2:172.21.15.203:3300/0 conn(0x7f81ec0b0ba0 0x7f81ec0af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81f0968700 1 --2- 172.21.15.83:0/668315922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81ec0b0f80 0x7f81ec0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81f0968700 1 -- 172.21.15.83:0/668315922 >> 172.21.15.83:0/668315922 conn(0x7f81ec0b57b0 msgr2=0x7f81ec0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81f0968700 1 -- 172.21.15.83:0/668315922 shutdown_connections 2023-11-26T07:15:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.644+0000 7f81f0968700 1 -- 172.21.15.83:0/668315922 wait complete. 2023-11-26T07:15:29.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.646+0000 7f81f0968700 1 Processor -- start 2023-11-26T07:15:29.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.646+0000 7f81f0968700 1 -- start start 2023-11-26T07:15:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81f0968700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81ec0b0ba0 0x7f81ec062450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81f0968700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81ec0b0f80 0x7f81ec159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81f0968700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec15ee90 0x7f81ec0cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81f0968700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec062990 con 0x7f81ec15ee90 2023-11-26T07:15:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81f0968700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec15c2a0 con 0x7f81ec0b0ba0 2023-11-26T07:15:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81f0968700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec15b690 con 0x7f81ec0b0f80 2023-11-26T07:15:29.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81ead9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec15ee90 0x7f81ec0cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81ead9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec15ee90 0x7f81ec0cac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39142/0 (socket says 172.21.15.83:39142) 2023-11-26T07:15:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81ead9d700 1 -- 172.21.15.83:0/2479987983 learned_addr learned my addr 172.21.15.83:0/2479987983 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81ea59c700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81ec0b0ba0 0x7f81ec062450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81e9d9b700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81ec0b0f80 0x7f81ec159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81ead9d700 1 -- 172.21.15.83:0/2479987983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81ec0b0f80 msgr2=0x7f81ec159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81ead9d700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81ec0b0f80 0x7f81ec159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81ead9d700 1 -- 172.21.15.83:0/2479987983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81ec0b0ba0 msgr2=0x7f81ec062450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81ead9d700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81ec0b0ba0 0x7f81ec062450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81ead9d700 1 -- 172.21.15.83:0/2479987983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e4009b30 con 0x7f81ec15ee90 2023-11-26T07:15:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81ea59c700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81ec0b0ba0 0x7f81ec062450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.647+0000 7f81ead9d700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec15ee90 0x7f81ec0cac20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b370 tx=0x7f81e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.648+0000 7f81daffd700 1 -- 172.21.15.83:0/2479987983 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4004310 con 0x7f81ec15ee90 2023-11-26T07:15:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.648+0000 7f81daffd700 1 -- 172.21.15.83:0/2479987983 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e4004470 con 0x7f81ec15ee90 2023-11-26T07:15:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.648+0000 7f81daffd700 1 -- 172.21.15.83:0/2479987983 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4004820 con 0x7f81ec15ee90 2023-11-26T07:15:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.648+0000 7f81f0968700 1 -- 172.21.15.83:0/2479987983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec15b970 con 0x7f81ec15ee90 2023-11-26T07:15:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.648+0000 7f81f0968700 1 -- 172.21.15.83:0/2479987983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec15bf30 con 0x7f81ec15ee90 2023-11-26T07:15:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.649+0000 7f81f0968700 1 -- 172.21.15.83:0/2479987983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec067f60 con 0x7f81ec15ee90 2023-11-26T07:15:29.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.649+0000 7f81daffd700 1 -- 172.21.15.83:0/2479987983 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f81e401b430 con 0x7f81ec15ee90 2023-11-26T07:15:29.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.649+0000 7f81daffd700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f81cc06b230 0x7f81cc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:29.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.649+0000 7f81daffd700 1 -- 172.21.15.83:0/2479987983 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81e408b170 con 0x7f81ec15ee90 2023-11-26T07:15:29.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.650+0000 7f81ea59c700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f81cc06b230 0x7f81cc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:29.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.650+0000 7f81ea59c700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f81cc06b230 0x7f81cc06d6f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f81dc006ba0 tx=0x7f81dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:29.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.654+0000 7f81daffd700 1 -- 172.21.15.83:0/2479987983 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f81e4055d90 con 0x7f81ec15ee90 2023-11-26T07:15:29.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.889+0000 7f81f0968700 1 -- 172.21.15.83:0/2479987983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f81ec15c0c0 con 0x7f81ec15ee90 2023-11-26T07:15:29.892 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-26T07:15:29.890+0000 7fcd25823700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:15:29.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.891+0000 7f81daffd700 1 -- 172.21.15.83:0/2479987983 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f81e40593b0 con 0x7f81ec15ee90 2023-11-26T07:15:29.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.894+0000 7f81f0968700 1 -- 172.21.15.83:0/2479987983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f81cc06b230 msgr2=0x7f81cc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.894+0000 7f81f0968700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f81cc06b230 0x7f81cc06d6f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f81dc006ba0 tx=0x7f81dc008040 comp rx=0 tx=0).stop 2023-11-26T07:15:29.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.894+0000 7f81f0968700 1 -- 172.21.15.83:0/2479987983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec15ee90 msgr2=0x7f81ec0cac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:29.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.894+0000 7f81f0968700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec15ee90 0x7f81ec0cac20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f81e400b370 tx=0x7f81e4004ad0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.895+0000 7f81f0968700 1 -- 172.21.15.83:0/2479987983 shutdown_connections 2023-11-26T07:15:29.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.895+0000 7f81f0968700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec15ee90 0x7f81ec0cac20 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.895+0000 7f81f0968700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81ec0b0ba0 0x7f81ec062450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.895+0000 7f81f0968700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f81cc06b230 0x7f81cc06d6f0 secure :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f81dc006ba0 tx=0x7f81dc008040 comp rx=0 tx=0).stop 2023-11-26T07:15:29.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.895+0000 7f81f0968700 1 --2- 172.21.15.83:0/2479987983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81ec0b0f80 0x7f81ec159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:29.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.895+0000 7f81f0968700 1 -- 172.21.15.83:0/2479987983 >> 172.21.15.83:0/2479987983 conn(0x7f81ec0b57b0 msgr2=0x7f81ec06e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:29.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.896+0000 7f81f0968700 1 -- 172.21.15.83:0/2479987983 shutdown_connections 2023-11-26T07:15:29.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:29.896+0000 7f81f0968700 1 -- 172.21.15.83:0/2479987983 wait complete. 2023-11-26T07:15:29.899 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:15:29.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:15:29.934 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-26T07:15:30.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.031+0000 7fb4be555700 1 Processor -- start 2023-11-26T07:15:30.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.032+0000 7fb4be555700 1 -- start start 2023-11-26T07:15:30.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.032+0000 7fb4be555700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4b80c6c30 0x7fb4b80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:30.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.032+0000 7fb4be555700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b80d0140 con 0x7fb4b815bf50 2023-11-26T07:15:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.032+0000 7fb4be555700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b80cf3d0 con 0x7fb4b80cef40 2023-11-26T07:15:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.032+0000 7fb4be555700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b815c330 con 0x7fb4b80c6c30 2023-11-26T07:15:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.032+0000 7fb4b77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4b80c6c30 0x7fb4b80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.032+0000 7fb4b7fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb4b815bf50 0x7fb4b80ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59200/0 (socket says 172.21.15.83:59200) 2023-11-26T07:15:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.032+0000 7fb4b7fff700 1 -- 172.21.15.83:0/3470680620 learned_addr learned my addr 172.21.15.83:0/3470680620 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:30.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.033+0000 7fb4b77fe700 1 -- 172.21.15.83:0/3470680620 >> v1:172.21.15.203:6789/0 conn(0x7fb4b80cef40 legacy=0x7fb4b80cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:30.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.033+0000 7fb4b77fe700 1 -- 172.21.15.83:0/3470680620 >> v1:172.21.15.83:6789/0 conn(0x7fb4b815bf50 legacy=0x7fb4b80ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:30.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.033+0000 7fb4b77fe700 1 -- 172.21.15.83:0/3470680620 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b80d03e0 con 0x7fb4b80c6c30 2023-11-26T07:15:30.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.033+0000 7fb4b6ffd700 1 -- 172.21.15.83:0/3470680620 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515443763 0 0) 0x7fb4b80d0140 con 0x7fb4b815bf50 2023-11-26T07:15:30.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.033+0000 7fb4b77fe700 1 --2- 172.21.15.83:0/3470680620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b80c6c30 0x7fb4b80cea00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009840 tx=0x7fb4ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b680010045c36 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:30.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.033+0000 7fb4b6ffd700 1 -- 172.21.15.83:0/3470680620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac004230 con 0x7fb4b80c6c30 2023-11-26T07:15:30.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.033+0000 7fb4b6ffd700 1 -- 172.21.15.83:0/3470680620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac004390 con 0x7fb4b80c6c30 2023-11-26T07:15:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.033+0000 7fb4b6ffd700 1 -- 172.21.15.83:0/3470680620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0036a0 con 0x7fb4b80c6c30 2023-11-26T07:15:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.034+0000 7fb4be555700 1 -- 172.21.15.83:0/3470680620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b80c6c30 msgr2=0x7fb4b80cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.034+0000 7fb4be555700 1 --2- 172.21.15.83:0/3470680620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b80c6c30 0x7fb4b80cea00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009840 tx=0x7fb4ac009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.034+0000 7fb4be555700 1 -- 172.21.15.83:0/3470680620 shutdown_connections 2023-11-26T07:15:30.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.034+0000 7fb4be555700 1 --2- 172.21.15.83:0/3470680620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b80c6c30 0x7fb4b80cea00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.034+0000 7fb4be555700 1 -- 172.21.15.83:0/3470680620 >> 172.21.15.83:0/3470680620 conn(0x7fb4b80b3780 msgr2=0x7fb4b80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.034+0000 7fb4be555700 1 -- 172.21.15.83:0/3470680620 shutdown_connections 2023-11-26T07:15:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.034+0000 7fb4be555700 1 -- 172.21.15.83:0/3470680620 wait complete. 2023-11-26T07:15:30.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.035+0000 7fb4be555700 1 Processor -- start 2023-11-26T07:15:30.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.035+0000 7fb4be555700 1 -- start start 2023-11-26T07:15:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4be555700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b80c6c30 0x7fb4b814ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4be555700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b80cef40 0x7fb4b806ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4be555700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b815bf50 0x7fb4b806bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4be555700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b806c9a0 con 0x7fb4b80cef40 2023-11-26T07:15:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4be555700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b806c480 con 0x7fb4b815bf50 2023-11-26T07:15:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4be555700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b806c6c0 con 0x7fb4b80c6c30 2023-11-26T07:15:30.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4b77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b80cef40 0x7fb4b806ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4b77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b80cef40 0x7fb4b806ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39168/0 (socket says 172.21.15.83:39168) 2023-11-26T07:15:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4b77fe700 1 -- 172.21.15.83:0/1278218288 learned_addr learned my addr 172.21.15.83:0/1278218288 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4b7fff700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b80c6c30 0x7fb4b814ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4bcaf2700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b815bf50 0x7fb4b806bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4b77fe700 1 -- 172.21.15.83:0/1278218288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b80c6c30 msgr2=0x7fb4b814ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4b77fe700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b80c6c30 0x7fb4b814ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:30.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4b77fe700 1 -- 172.21.15.83:0/1278218288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b815bf50 msgr2=0x7fb4b806bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:30.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4b77fe700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b815bf50 0x7fb4b806bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:30.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4b77fe700 1 -- 172.21.15.83:0/1278218288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac0094a0 con 0x7fb4b80cef40 2023-11-26T07:15:30.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4b7fff700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b80c6c30 0x7fb4b814ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:30.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.036+0000 7fb4b77fe700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b80cef40 0x7fb4b806ba00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0090b0 tx=0x7fb4ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:30.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.037+0000 7fb4b4ff9700 1 -- 172.21.15.83:0/1278218288 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac011830 con 0x7fb4b80cef40 2023-11-26T07:15:30.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.037+0000 7fb4b4ff9700 1 -- 172.21.15.83:0/1278218288 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac01e070 con 0x7fb4b80cef40 2023-11-26T07:15:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.037+0000 7fb4b4ff9700 1 -- 172.21.15.83:0/1278218288 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac011bf0 con 0x7fb4b80cef40 2023-11-26T07:15:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.037+0000 7fb4be555700 1 -- 172.21.15.83:0/1278218288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b80c9f90 con 0x7fb4b80cef40 2023-11-26T07:15:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.037+0000 7fb4be555700 1 -- 172.21.15.83:0/1278218288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b80ca550 con 0x7fb4b80cef40 2023-11-26T07:15:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.038+0000 7fb4be555700 1 -- 172.21.15.83:0/1278218288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b80c9df0 con 0x7fb4b80cef40 2023-11-26T07:15:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.038+0000 7fb4b4ff9700 1 -- 172.21.15.83:0/1278218288 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fb4ac011d50 con 0x7fb4b80cef40 2023-11-26T07:15:30.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.038+0000 7fb4b4ff9700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb4a406b230 0x7fb4a406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:30.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.038+0000 7fb4b4ff9700 1 -- 172.21.15.83:0/1278218288 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb4ac08af50 con 0x7fb4b80cef40 2023-11-26T07:15:30.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.038+0000 7fb4b7fff700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb4a406b230 0x7fb4a406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:30.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.039+0000 7fb4b7fff700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb4a406b230 0x7fb4a406d6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8000f60 tx=0x7fb4a8007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:30.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.043+0000 7fb4b4ff9700 1 -- 172.21.15.83:0/1278218288 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4b80c9df0 con 0x7fb4b80cef40 2023-11-26T07:15:30.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.276+0000 7fb4be555700 1 -- 172.21.15.83:0/1278218288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb4b80c9df0 con 0x7fb4b80cef40 2023-11-26T07:15:30.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.896+0000 7fb4b4ff9700 1 -- 172.21.15.83:0/1278218288 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb4b80c9df0 con 0x7fb4b80cef40 2023-11-26T07:15:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 -- 172.21.15.83:0/1278218288 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb4a406b230 msgr2=0x7fb4a406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb4a406b230 0x7fb4a406d6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8000f60 tx=0x7fb4a8007d30 comp rx=0 tx=0).stop 2023-11-26T07:15:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 -- 172.21.15.83:0/1278218288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b80cef40 msgr2=0x7fb4b806ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b80cef40 0x7fb4b806ba00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0090b0 tx=0x7fb4ac004750 comp rx=0 tx=0).stop 2023-11-26T07:15:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 -- 172.21.15.83:0/1278218288 shutdown_connections 2023-11-26T07:15:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b815bf50 0x7fb4b806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb4a406b230 0x7fb4a406d6f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b80c6c30 0x7fb4b814ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 --2- 172.21.15.83:0/1278218288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b80cef40 0x7fb4b806ba00 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 -- 172.21.15.83:0/1278218288 >> 172.21.15.83:0/1278218288 conn(0x7fb4b80b3780 msgr2=0x7fb4b80ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 -- 172.21.15.83:0/1278218288 shutdown_connections 2023-11-26T07:15:30.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:30.898+0000 7fb4be555700 1 -- 172.21.15.83:0/1278218288 wait complete. 2023-11-26T07:15:30.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:15:31.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.077+0000 7f546464f700 1 Processor -- start 2023-11-26T07:15:31.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.077+0000 7f546464f700 1 -- start start 2023-11-26T07:15:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f546464f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f545c0b01e0 0x7f545c06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f546464f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0746a0 0x7f545c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f546464f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0b05c0 con 0x7f545c0b59c0 2023-11-26T07:15:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f546464f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c1470a0 con 0x7f545c0b01e0 2023-11-26T07:15:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f546464f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0b5e70 con 0x7f545c0746a0 2023-11-26T07:15:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f54623eb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f545c0b01e0 0x7f545c06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f5461bea700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f545c0b59c0 0x7f545c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59216/0 (socket says 172.21.15.83:59216) 2023-11-26T07:15:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f5461bea700 1 -- 172.21.15.83:0/766273070 learned_addr learned my addr 172.21.15.83:0/766273070 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f54623eb700 1 -- 172.21.15.83:0/766273070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0746a0 msgr2=0x7f545c0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f54623eb700 1 --2- 172.21.15.83:0/766273070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0746a0 0x7f545c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f54623eb700 1 -- 172.21.15.83:0/766273070 >> v1:172.21.15.83:6789/0 conn(0x7f545c0b59c0 legacy=0x7f545c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f54623eb700 1 -- 172.21.15.83:0/766273070 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c0b4250 con 0x7f545c0b01e0 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f54613e9700 1 -- 172.21.15.83:0/766273070 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385456866 0 0) 0x7f545c0b05c0 con 0x7f545c0b59c0 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.078+0000 7f54623eb700 1 --2- 172.21.15.83:0/766273070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c0b01e0 0x7f545c06e9c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5458009870 tx=0x7f5458009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d39a34cf13b017b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.079+0000 7f54613e9700 1 -- 172.21.15.83:0/766273070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458011830 con 0x7f545c0b01e0 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.079+0000 7f54613e9700 1 -- 172.21.15.83:0/766273070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5458011990 con 0x7f545c0b01e0 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.079+0000 7f54613e9700 1 -- 172.21.15.83:0/766273070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458011d20 con 0x7f545c0b01e0 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.079+0000 7f546464f700 1 -- 172.21.15.83:0/766273070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c0b01e0 msgr2=0x7f545c06e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.079+0000 7f546464f700 1 --2- 172.21.15.83:0/766273070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c0b01e0 0x7f545c06e9c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5458009870 tx=0x7f5458009b80 comp rx=0 tx=0).stop 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.079+0000 7f546464f700 1 -- 172.21.15.83:0/766273070 shutdown_connections 2023-11-26T07:15:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.079+0000 7f546464f700 1 --2- 172.21.15.83:0/766273070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0746a0 0x7f545c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:31.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.079+0000 7f546464f700 1 --2- 172.21.15.83:0/766273070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c0b01e0 0x7f545c06e9c0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:31.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.079+0000 7f546464f700 1 -- 172.21.15.83:0/766273070 >> 172.21.15.83:0/766273070 conn(0x7f545c0d0b40 msgr2=0x7f545c0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:31.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.079+0000 7f546464f700 1 -- 172.21.15.83:0/766273070 shutdown_connections 2023-11-26T07:15:31.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.079+0000 7f546464f700 1 -- 172.21.15.83:0/766273070 wait complete. 2023-11-26T07:15:31.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.080+0000 7f546464f700 1 Processor -- start 2023-11-26T07:15:31.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.080+0000 7f546464f700 1 -- start start 2023-11-26T07:15:31.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f546464f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c0746a0 0x7f545c13f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:31.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f546464f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0b01e0 0x7f545c13b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:31.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f546464f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f545c0b59c0 0x7f545c13d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:31.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f546464f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c140a60 con 0x7f545c0746a0 2023-11-26T07:15:31.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f546464f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c13fb40 con 0x7f545c0b59c0 2023-11-26T07:15:31.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f546464f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c13fde0 con 0x7f545c0b01e0 2023-11-26T07:15:31.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f54623eb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c0746a0 0x7f545c13f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:31.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f54623eb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c0746a0 0x7f545c13f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39184/0 (socket says 172.21.15.83:39184) 2023-11-26T07:15:31.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f54623eb700 1 -- 172.21.15.83:0/1460098369 learned_addr learned my addr 172.21.15.83:0/1460098369 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f54623eb700 1 -- 172.21.15.83:0/1460098369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0b01e0 msgr2=0x7f545c13b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f5461bea700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0b01e0 0x7f545c13b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f54623eb700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0b01e0 0x7f545c13b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f54623eb700 1 -- 172.21.15.83:0/1460098369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f545c0b59c0 msgr2=0x7f545c13d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f54623eb700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f545c0b59c0 0x7f545c13d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.081+0000 7f54623eb700 1 -- 172.21.15.83:0/1460098369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54580094a0 con 0x7f545c0746a0 2023-11-26T07:15:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.082+0000 7f54623eb700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c0746a0 0x7f545c13f600 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f54580090b0 tx=0x7f5458004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.082+0000 7f5452ffd700 1 -- 172.21.15.83:0/1460098369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580038a0 con 0x7f545c0746a0 2023-11-26T07:15:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.082+0000 7f5452ffd700 1 -- 172.21.15.83:0/1460098369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545801f070 con 0x7f545c0746a0 2023-11-26T07:15:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.082+0000 7f546464f700 1 -- 172.21.15.83:0/1460098369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c1400c0 con 0x7f545c0746a0 2023-11-26T07:15:31.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.082+0000 7f5452ffd700 1 -- 172.21.15.83:0/1460098369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458003c40 con 0x7f545c0746a0 2023-11-26T07:15:31.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.082+0000 7f546464f700 1 -- 172.21.15.83:0/1460098369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c140680 con 0x7f545c0746a0 2023-11-26T07:15:31.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.083+0000 7f546464f700 1 -- 172.21.15.83:0/1460098369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c0b48b0 con 0x7f545c0746a0 2023-11-26T07:15:31.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.083+0000 7f5452ffd700 1 -- 172.21.15.83:0/1460098369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f5458026480 con 0x7f545c0746a0 2023-11-26T07:15:31.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.083+0000 7f5452ffd700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544006b230 0x7f544006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:31.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.084+0000 7f5452ffd700 1 -- 172.21.15.83:0/1460098369 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5458021070 con 0x7f545c0746a0 2023-11-26T07:15:31.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.084+0000 7f5461bea700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544006b230 0x7f544006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:31.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.084+0000 7f5461bea700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544006b230 0x7f544006d6f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f545c1f74f0 tx=0x7f544c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:31.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.088+0000 7f5452ffd700 1 -- 172.21.15.83:0/1460098369 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5458056350 con 0x7f545c0746a0 2023-11-26T07:15:31.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.278+0000 7f546464f700 1 -- 172.21.15.83:0/1460098369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f545c067f60 con 0x7f545c0746a0 2023-11-26T07:15:31.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.900+0000 7f5452ffd700 1 -- 172.21.15.83:0/1460098369 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5458059970 con 0x7f545c0746a0 2023-11-26T07:15:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.904+0000 7f546464f700 1 -- 172.21.15.83:0/1460098369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544006b230 msgr2=0x7f544006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.904+0000 7f546464f700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544006b230 0x7f544006d6f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f545c1f74f0 tx=0x7f544c009040 comp rx=0 tx=0).stop 2023-11-26T07:15:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.904+0000 7f546464f700 1 -- 172.21.15.83:0/1460098369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c0746a0 msgr2=0x7f545c13f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.904+0000 7f546464f700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c0746a0 0x7f545c13f600 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f54580090b0 tx=0x7f5458004750 comp rx=0 tx=0).stop 2023-11-26T07:15:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.904+0000 7f546464f700 1 -- 172.21.15.83:0/1460098369 shutdown_connections 2023-11-26T07:15:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.904+0000 7f546464f700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f545c0b59c0 0x7f545c13d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:31.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.904+0000 7f546464f700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c0746a0 0x7f545c13f600 secure :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f54580090b0 tx=0x7f5458004750 comp rx=0 tx=0).stop 2023-11-26T07:15:31.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.904+0000 7f546464f700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f544006b230 0x7f544006d6f0 secure :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f545c1f74f0 tx=0x7f544c009040 comp rx=0 tx=0).stop 2023-11-26T07:15:31.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.904+0000 7f546464f700 1 --2- 172.21.15.83:0/1460098369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f545c0b01e0 0x7f545c13b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:31.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.904+0000 7f546464f700 1 -- 172.21.15.83:0/1460098369 >> 172.21.15.83:0/1460098369 conn(0x7f545c0d0b40 msgr2=0x7f545c0b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:31.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.905+0000 7f546464f700 1 -- 172.21.15.83:0/1460098369 shutdown_connections 2023-11-26T07:15:31.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:31.905+0000 7f546464f700 1 -- 172.21.15.83:0/1460098369 wait complete. 2023-11-26T07:15:31.909 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:15:31.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:15:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.050+0000 7effabcdb700 1 Processor -- start 2023-11-26T07:15:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.050+0000 7effabcdb700 1 -- start start 2023-11-26T07:15:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.051+0000 7effabcdb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7effa40624b0 0x7effa415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.051+0000 7effabcdb700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa40d0ec0 con 0x7effa40cc820 2023-11-26T07:15:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.051+0000 7effabcdb700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa40d0380 con 0x7effa40b1020 2023-11-26T07:15:32.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.051+0000 7effabcdb700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa40d0620 con 0x7effa40624b0 2023-11-26T07:15:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.051+0000 7effaa278700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7effa40624b0 0x7effa415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.051+0000 7effa9276700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7effa40cc820 0x7effa415fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59230/0 (socket says 172.21.15.83:59230) 2023-11-26T07:15:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.051+0000 7effa9276700 1 -- 172.21.15.83:0/4026423525 learned_addr learned my addr 172.21.15.83:0/4026423525 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effaa278700 1 -- 172.21.15.83:0/4026423525 >> v1:172.21.15.203:6789/0 conn(0x7effa40b1020 legacy=0x7effa40b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effaa278700 1 -- 172.21.15.83:0/4026423525 >> v1:172.21.15.83:6789/0 conn(0x7effa40cc820 legacy=0x7effa415fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effaa278700 1 -- 172.21.15.83:0/4026423525 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa40d0900 con 0x7effa40624b0 2023-11-26T07:15:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effa8a75700 1 -- 172.21.15.83:0/4026423525 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376455276 0 0) 0x7effa40d0ec0 con 0x7effa40cc820 2023-11-26T07:15:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effaa278700 1 --2- 172.21.15.83:0/4026423525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa40624b0 0x7effa415c7f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009d90 tx=0x7eff9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5611ae63ed5bfa7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effa8a75700 1 -- 172.21.15.83:0/4026423525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00e070 con 0x7effa40624b0 2023-11-26T07:15:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effa8a75700 1 -- 172.21.15.83:0/4026423525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c00bb90 con 0x7effa40624b0 2023-11-26T07:15:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effa8a75700 1 -- 172.21.15.83:0/4026423525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c003a20 con 0x7effa40624b0 2023-11-26T07:15:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effabcdb700 1 -- 172.21.15.83:0/4026423525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa40624b0 msgr2=0x7effa415c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effabcdb700 1 --2- 172.21.15.83:0/4026423525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa40624b0 0x7effa415c7f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009d90 tx=0x7eff9c00b330 comp rx=0 tx=0).stop 2023-11-26T07:15:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effabcdb700 1 -- 172.21.15.83:0/4026423525 shutdown_connections 2023-11-26T07:15:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.052+0000 7effabcdb700 1 --2- 172.21.15.83:0/4026423525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa40624b0 0x7effa415c7f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.053+0000 7effabcdb700 1 -- 172.21.15.83:0/4026423525 >> 172.21.15.83:0/4026423525 conn(0x7effa4072710 msgr2=0x7effa4072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.053+0000 7effabcdb700 1 -- 172.21.15.83:0/4026423525 shutdown_connections 2023-11-26T07:15:32.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.053+0000 7effabcdb700 1 -- 172.21.15.83:0/4026423525 wait complete. 2023-11-26T07:15:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.054+0000 7effabcdb700 1 Processor -- start 2023-11-26T07:15:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.054+0000 7effabcdb700 1 -- start start 2023-11-26T07:15:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effabcdb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa40624b0 0x7effa40998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effabcdb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa40b1020 0x7effa40c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effabcdb700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effa40cc820 0x7effa40c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effabcdb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa40c85f0 con 0x7effa40624b0 2023-11-26T07:15:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effabcdb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa40c88d0 con 0x7effa40b1020 2023-11-26T07:15:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effabcdb700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa40c8bb0 con 0x7effa40cc820 2023-11-26T07:15:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effa9a77700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa40624b0 0x7effa40998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effa9a77700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa40624b0 0x7effa40998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39194/0 (socket says 172.21.15.83:39194) 2023-11-26T07:15:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effa9a77700 1 -- 172.21.15.83:0/1266216237 learned_addr learned my addr 172.21.15.83:0/1266216237 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effa9a77700 1 -- 172.21.15.83:0/1266216237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effa40cc820 msgr2=0x7effa40c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effa9a77700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effa40cc820 0x7effa40c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effa9a77700 1 -- 172.21.15.83:0/1266216237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa40b1020 msgr2=0x7effa40c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effa9a77700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa40b1020 0x7effa40c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.055+0000 7effa9a77700 1 -- 172.21.15.83:0/1266216237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c009a70 con 0x7effa40624b0 2023-11-26T07:15:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.056+0000 7effa9a77700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa40624b0 0x7effa40998c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7eff9400c4a0 tx=0x7eff9400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.056+0000 7eff9a7fc700 1 -- 172.21.15.83:0/1266216237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff940054e0 con 0x7effa40624b0 2023-11-26T07:15:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.056+0000 7eff9a7fc700 1 -- 172.21.15.83:0/1266216237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff94003680 con 0x7effa40624b0 2023-11-26T07:15:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.056+0000 7effabcdb700 1 -- 172.21.15.83:0/1266216237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa40c8e90 con 0x7effa40624b0 2023-11-26T07:15:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.056+0000 7eff9a7fc700 1 -- 172.21.15.83:0/1266216237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94003a30 con 0x7effa40624b0 2023-11-26T07:15:32.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.056+0000 7effabcdb700 1 -- 172.21.15.83:0/1266216237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa40b02a0 con 0x7effa40624b0 2023-11-26T07:15:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.057+0000 7effabcdb700 1 -- 172.21.15.83:0/1266216237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa40c9200 con 0x7effa40624b0 2023-11-26T07:15:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.057+0000 7eff9a7fc700 1 -- 172.21.15.83:0/1266216237 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7eff94003b90 con 0x7effa40624b0 2023-11-26T07:15:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.058+0000 7eff9a7fc700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff8806b230 0x7eff8806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.058+0000 7eff9a7fc700 1 -- 172.21.15.83:0/1266216237 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5357+0+0 (secure 0 0 0) 0x7eff94088f30 con 0x7effa40624b0 2023-11-26T07:15:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.058+0000 7effa9276700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff8806b230 0x7eff8806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.058+0000 7effa9276700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff8806b230 0x7eff8806d6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7effa40addc0 tx=0x7effa00079f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.062+0000 7eff9a7fc700 1 -- 172.21.15.83:0/1266216237 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff94053d30 con 0x7effa40624b0 2023-11-26T07:15:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.252+0000 7effabcdb700 1 -- 172.21.15.83:0/1266216237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7effa40c8730 con 0x7effa40624b0 2023-11-26T07:15:32.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.905+0000 7eff9a7fc700 1 -- 172.21.15.83:0/1266216237 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) v1 ==== 169+0+0 (secure 0 0 0) 0x7effa40c8730 con 0x7effa40624b0 2023-11-26T07:15:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.908+0000 7effabcdb700 1 -- 172.21.15.83:0/1266216237 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff8806b230 msgr2=0x7eff8806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.908+0000 7effabcdb700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff8806b230 0x7eff8806d6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7effa40addc0 tx=0x7effa00079f0 comp rx=0 tx=0).stop 2023-11-26T07:15:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.908+0000 7effabcdb700 1 -- 172.21.15.83:0/1266216237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa40624b0 msgr2=0x7effa40998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.908+0000 7effabcdb700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa40624b0 0x7effa40998c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7eff9400c4a0 tx=0x7eff9400c860 comp rx=0 tx=0).stop 2023-11-26T07:15:32.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.908+0000 7effabcdb700 1 -- 172.21.15.83:0/1266216237 shutdown_connections 2023-11-26T07:15:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.908+0000 7effabcdb700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7eff8806b230 0x7eff8806d6f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.908+0000 7effabcdb700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7effa40cc820 0x7effa40c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.908+0000 7effabcdb700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa40624b0 0x7effa40998c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.908+0000 7effabcdb700 1 --2- 172.21.15.83:0/1266216237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa40b1020 0x7effa40c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.908+0000 7effabcdb700 1 -- 172.21.15.83:0/1266216237 >> 172.21.15.83:0/1266216237 conn(0x7effa4072710 msgr2=0x7effa40aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.908+0000 7effabcdb700 1 -- 172.21.15.83:0/1266216237 shutdown_connections 2023-11-26T07:15:32.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:32.909+0000 7effabcdb700 1 -- 172.21.15.83:0/1266216237 wait complete. 2023-11-26T07:15:32.913 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2023-11-26T07:15:32.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:15:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.046+0000 7fb3d3532700 1 Processor -- start 2023-11-26T07:15:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.046+0000 7fb3d3532700 1 -- start start 2023-11-26T07:15:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d3532700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb3cc0746a0 0x7fb3cc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d3532700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb3cc0afc90 0x7fb3cc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d3532700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3cc0b3c60 con 0x7fb3cc15e160 2023-11-26T07:15:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d3532700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc0b4500 con 0x7fb3cc0afc90 2023-11-26T07:15:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d3532700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc0b58c0 con 0x7fb3cc0746a0 2023-11-26T07:15:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d1acf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb3cc0afc90 0x7fb3cc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d1acf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb3cc0afc90 0x7fb3cc06d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39206/0 (socket says 172.21.15.83:39206) 2023-11-26T07:15:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d1acf700 1 -- 172.21.15.83:0/2451317423 learned_addr learned my addr 172.21.15.83:0/2451317423 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:33.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d0acd700 1 --2- 172.21.15.83:0/2451317423 >> v2:172.21.15.203:3300/0 conn(0x7fb3cc0746a0 0x7fb3cc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d1acf700 1 -- 172.21.15.83:0/2451317423 >> v2:172.21.15.203:3300/0 conn(0x7fb3cc0746a0 msgr2=0x7fb3cc0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d1acf700 1 --2- 172.21.15.83:0/2451317423 >> v2:172.21.15.203:3300/0 conn(0x7fb3cc0746a0 0x7fb3cc0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d1acf700 1 -- 172.21.15.83:0/2451317423 >> v1:172.21.15.203:6789/0 conn(0x7fb3cc15e160 legacy=0x7fb3cc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d1acf700 1 -- 172.21.15.83:0/2451317423 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc06bdc0 con 0x7fb3cc0afc90 2023-11-26T07:15:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3d1acf700 1 --2- 172.21.15.83:0/2451317423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3cc0afc90 0x7fb3cc06d930 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0d0880 tx=0x7fb3c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13cdc3b004d30c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.047+0000 7fb3c3fff700 1 -- 172.21.15.83:0/2451317423 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400e070 con 0x7fb3cc0afc90 2023-11-26T07:15:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.048+0000 7fb3c3fff700 1 -- 172.21.15.83:0/2451317423 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c4003a20 con 0x7fb3cc0afc90 2023-11-26T07:15:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.048+0000 7fb3c3fff700 1 -- 172.21.15.83:0/2451317423 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4003dd0 con 0x7fb3cc0afc90 2023-11-26T07:15:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.048+0000 7fb3d3532700 1 -- 172.21.15.83:0/2451317423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3cc0afc90 msgr2=0x7fb3cc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.048+0000 7fb3d3532700 1 --2- 172.21.15.83:0/2451317423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3cc0afc90 0x7fb3cc06d930 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0d0880 tx=0x7fb3c400b330 comp rx=0 tx=0).stop 2023-11-26T07:15:33.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.048+0000 7fb3d3532700 1 -- 172.21.15.83:0/2451317423 shutdown_connections 2023-11-26T07:15:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.048+0000 7fb3d3532700 1 --2- 172.21.15.83:0/2451317423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3cc0afc90 0x7fb3cc06d930 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.048+0000 7fb3d3532700 1 --2- 172.21.15.83:0/2451317423 >> v2:172.21.15.203:3300/0 conn(0x7fb3cc0746a0 0x7fb3cc0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.048+0000 7fb3d3532700 1 -- 172.21.15.83:0/2451317423 >> 172.21.15.83:0/2451317423 conn(0x7fb3cc0d0bf0 msgr2=0x7fb3cc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.048+0000 7fb3d3532700 1 -- 172.21.15.83:0/2451317423 shutdown_connections 2023-11-26T07:15:33.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.048+0000 7fb3d3532700 1 -- 172.21.15.83:0/2451317423 wait complete. 2023-11-26T07:15:33.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.049+0000 7fb3d3532700 1 Processor -- start 2023-11-26T07:15:33.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.049+0000 7fb3d3532700 1 -- start start 2023-11-26T07:15:33.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d3532700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3cc0746a0 0x7fb3cc1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d3532700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3cc0afc90 0x7fb3cc160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d3532700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc15e160 0x7fb3cc14e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d3532700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0af590 con 0x7fb3cc0afc90 2023-11-26T07:15:33.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d3532700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc15fbb0 con 0x7fb3cc15e160 2023-11-26T07:15:33.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d3532700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0af060 con 0x7fb3cc0746a0 2023-11-26T07:15:33.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d0acd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3cc0afc90 0x7fb3cc160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:33.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d0acd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3cc0afc90 0x7fb3cc160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39210/0 (socket says 172.21.15.83:39210) 2023-11-26T07:15:33.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d0acd700 1 -- 172.21.15.83:0/4024671526 learned_addr learned my addr 172.21.15.83:0/4024671526 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:33.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d1acf700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc15e160 0x7fb3cc14e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:33.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d0acd700 1 -- 172.21.15.83:0/4024671526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3cc0746a0 msgr2=0x7fb3cc1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:33.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d0acd700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3cc0746a0 0x7fb3cc1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d0acd700 1 -- 172.21.15.83:0/4024671526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc15e160 msgr2=0x7fb3cc14e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:33.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d0acd700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc15e160 0x7fb3cc14e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.050+0000 7fb3d0acd700 1 -- 172.21.15.83:0/4024671526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c4009b30 con 0x7fb3cc0afc90 2023-11-26T07:15:33.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.051+0000 7fb3d0acd700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3cc0afc90 0x7fb3cc160930 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00c820 tx=0x7fb3bc00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:33.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.051+0000 7fb3c1ffb700 1 -- 172.21.15.83:0/4024671526 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc011800 con 0x7fb3cc0afc90 2023-11-26T07:15:33.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.051+0000 7fb3c1ffb700 1 -- 172.21.15.83:0/4024671526 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc011960 con 0x7fb3cc0afc90 2023-11-26T07:15:33.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.051+0000 7fb3d3532700 1 -- 172.21.15.83:0/4024671526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc0af2b0 con 0x7fb3cc0afc90 2023-11-26T07:15:33.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.051+0000 7fb3c1ffb700 1 -- 172.21.15.83:0/4024671526 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc011d10 con 0x7fb3cc0afc90 2023-11-26T07:15:33.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.051+0000 7fb3d3532700 1 -- 172.21.15.83:0/4024671526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc160e70 con 0x7fb3cc0afc90 2023-11-26T07:15:33.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.052+0000 7fb3d3532700 1 -- 172.21.15.83:0/4024671526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc15f630 con 0x7fb3cc0afc90 2023-11-26T07:15:33.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.052+0000 7fb3c1ffb700 1 -- 172.21.15.83:0/4024671526 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fb3bc018360 con 0x7fb3cc0afc90 2023-11-26T07:15:33.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.052+0000 7fb3c1ffb700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb3b006b230 0x7fb3b006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.053+0000 7fb3c1ffb700 1 -- 172.21.15.83:0/4024671526 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb3bc089850 con 0x7fb3cc0afc90 2023-11-26T07:15:33.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.053+0000 7fb3d12ce700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb3b006b230 0x7fb3b006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:33.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.053+0000 7fb3d12ce700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb3b006b230 0x7fb3b006d6f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80096f0 tx=0x7fb3c8006c50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:33.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.057+0000 7fb3c1ffb700 1 -- 172.21.15.83:0/4024671526 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3cc15f630 con 0x7fb3cc0afc90 2023-11-26T07:15:33.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.248+0000 7fb3d3532700 1 -- 172.21.15.83:0/4024671526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb3cc15f630 con 0x7fb3cc0afc90 2023-11-26T07:15:33.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.251+0000 7fb3c1ffb700 1 -- 172.21.15.83:0/4024671526 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb3cc15f630 con 0x7fb3cc0afc90 2023-11-26T07:15:33.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 -- 172.21.15.83:0/4024671526 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb3b006b230 msgr2=0x7fb3b006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:33.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb3b006b230 0x7fb3b006d6f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80096f0 tx=0x7fb3c8006c50 comp rx=0 tx=0).stop 2023-11-26T07:15:33.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 -- 172.21.15.83:0/4024671526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3cc0afc90 msgr2=0x7fb3cc160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:33.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3cc0afc90 0x7fb3cc160930 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00c820 tx=0x7fb3bc00cbe0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 -- 172.21.15.83:0/4024671526 shutdown_connections 2023-11-26T07:15:33.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3cc15e160 0x7fb3cc14e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb3b006b230 0x7fb3b006d6f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3cc0746a0 0x7fb3cc1603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 --2- 172.21.15.83:0/4024671526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3cc0afc90 0x7fb3cc160930 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 -- 172.21.15.83:0/4024671526 >> 172.21.15.83:0/4024671526 conn(0x7fb3cc0d0bf0 msgr2=0x7fb3cc0b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:33.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 -- 172.21.15.83:0/4024671526 shutdown_connections 2023-11-26T07:15:33.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.254+0000 7fb3d3532700 1 -- 172.21.15.83:0/4024671526 wait complete. 2023-11-26T07:15:33.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:15:33.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.387+0000 7fd5a2569700 1 Processor -- start 2023-11-26T07:15:33.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.387+0000 7fd5a2569700 1 -- start start 2023-11-26T07:15:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.387+0000 7fd5a2569700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd59c15f180 0x7fd59c0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.387+0000 7fd5a2569700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd59c0cfd40 0x7fd59c0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.387+0000 7fd5a2569700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59c0b4290 con 0x7fd59c13e000 2023-11-26T07:15:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.387+0000 7fd5a2569700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59c13e570 con 0x7fd59c15f180 2023-11-26T07:15:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.387+0000 7fd5a2569700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0d05d0 con 0x7fd59c0cfd40 2023-11-26T07:15:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd59c15f180 0x7fd59c0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd59c13e000 0x7fd59c15ea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59256/0 (socket says 172.21.15.83:59256) 2023-11-26T07:15:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59bfff700 1 -- 172.21.15.83:0/4060569250 learned_addr learned my addr 172.21.15.83:0/4060569250 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd5a0b06700 1 --2- 172.21.15.83:0/4060569250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd59c0cfd40 0x7fd59c0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59b7fe700 1 -- 172.21.15.83:0/4060569250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd59c0cfd40 msgr2=0x7fd59c0b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59b7fe700 1 --2- 172.21.15.83:0/4060569250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd59c0cfd40 0x7fd59c0b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59b7fe700 1 -- 172.21.15.83:0/4060569250 >> v1:172.21.15.83:6789/0 conn(0x7fd59c13e000 legacy=0x7fd59c15ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59b7fe700 1 -- 172.21.15.83:0/4060569250 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c0d08b0 con 0x7fd59c15f180 2023-11-26T07:15:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59affd700 1 -- 172.21.15.83:0/4060569250 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890890790 0 0) 0x7fd59c0b4290 con 0x7fd59c13e000 2023-11-26T07:15:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59b7fe700 1 --2- 172.21.15.83:0/4060569250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c15f180 0x7fd59c0cf800 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd590009840 tx=0x7fd590009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34f6894577aab7c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59affd700 1 -- 172.21.15.83:0/4060569250 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590011830 con 0x7fd59c15f180 2023-11-26T07:15:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59affd700 1 -- 172.21.15.83:0/4060569250 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590011990 con 0x7fd59c15f180 2023-11-26T07:15:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.388+0000 7fd59affd700 1 -- 172.21.15.83:0/4060569250 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590011d20 con 0x7fd59c15f180 2023-11-26T07:15:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.389+0000 7fd5a2569700 1 -- 172.21.15.83:0/4060569250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c15f180 msgr2=0x7fd59c0cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.389+0000 7fd5a2569700 1 --2- 172.21.15.83:0/4060569250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c15f180 0x7fd59c0cf800 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd590009840 tx=0x7fd590009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.389+0000 7fd5a2569700 1 -- 172.21.15.83:0/4060569250 shutdown_connections 2023-11-26T07:15:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.389+0000 7fd5a2569700 1 --2- 172.21.15.83:0/4060569250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd59c0cfd40 0x7fd59c0b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.389+0000 7fd5a2569700 1 --2- 172.21.15.83:0/4060569250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c15f180 0x7fd59c0cf800 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.389+0000 7fd5a2569700 1 -- 172.21.15.83:0/4060569250 >> 172.21.15.83:0/4060569250 conn(0x7fd59c072990 msgr2=0x7fd59c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.389+0000 7fd5a2569700 1 -- 172.21.15.83:0/4060569250 shutdown_connections 2023-11-26T07:15:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.389+0000 7fd5a2569700 1 -- 172.21.15.83:0/4060569250 wait complete. 2023-11-26T07:15:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.390+0000 7fd5a2569700 1 Processor -- start 2023-11-26T07:15:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.390+0000 7fd5a2569700 1 -- start start 2023-11-26T07:15:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.391+0000 7fd5a2569700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd59c0cfd40 0x7fd59c144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.391+0000 7fd5a2569700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd59c13e000 0x7fd59c06b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.391+0000 7fd5a2569700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c15f180 0x7fd59c06bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.391+0000 7fd5a2569700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c130070 con 0x7fd59c15f180 2023-11-26T07:15:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.391+0000 7fd5a2569700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c141760 con 0x7fd59c0cfd40 2023-11-26T07:15:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.391+0000 7fd5a2569700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c144650 con 0x7fd59c13e000 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.391+0000 7fd5a0b06700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c15f180 0x7fd59c06bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.391+0000 7fd5a0b06700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c15f180 0x7fd59c06bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39226/0 (socket says 172.21.15.83:39226) 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.391+0000 7fd5a0b06700 1 -- 172.21.15.83:0/2378154734 learned_addr learned my addr 172.21.15.83:0/2378154734 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.391+0000 7fd59bfff700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd59c0cfd40 0x7fd59c144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd59b7fe700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd59c13e000 0x7fd59c06b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd5a0b06700 1 -- 172.21.15.83:0/2378154734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd59c13e000 msgr2=0x7fd59c06b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd5a0b06700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd59c13e000 0x7fd59c06b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd5a0b06700 1 -- 172.21.15.83:0/2378154734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd59c0cfd40 msgr2=0x7fd59c144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd5a0b06700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd59c0cfd40 0x7fd59c144110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd5a0b06700 1 -- 172.21.15.83:0/2378154734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5900094a0 con 0x7fd59c15f180 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd5a0b06700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c15f180 0x7fd59c06bdd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd594007e40 tx=0x7fd59400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd598ff9700 1 -- 172.21.15.83:0/2378154734 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59400ed30 con 0x7fd59c15f180 2023-11-26T07:15:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd598ff9700 1 -- 172.21.15.83:0/2378154734 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd594014420 con 0x7fd59c15f180 2023-11-26T07:15:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd5a2569700 1 -- 172.21.15.83:0/2378154734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c12fd90 con 0x7fd59c15f180 2023-11-26T07:15:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd598ff9700 1 -- 172.21.15.83:0/2378154734 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940147d0 con 0x7fd59c15f180 2023-11-26T07:15:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.392+0000 7fd5a2569700 1 -- 172.21.15.83:0/2378154734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59c12d4d0 con 0x7fd59c15f180 2023-11-26T07:15:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.393+0000 7fd5a2569700 1 -- 172.21.15.83:0/2378154734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59c12d050 con 0x7fd59c15f180 2023-11-26T07:15:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.393+0000 7fd598ff9700 1 -- 172.21.15.83:0/2378154734 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fd594014930 con 0x7fd59c15f180 2023-11-26T07:15:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.394+0000 7fd598ff9700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd58806b230 0x7fd58806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.394+0000 7fd598ff9700 1 -- 172.21.15.83:0/2378154734 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd594089ed0 con 0x7fd59c15f180 2023-11-26T07:15:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.394+0000 7fd59bfff700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd58806b230 0x7fd58806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.394+0000 7fd59bfff700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd58806b230 0x7fd58806d6f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd58c007fd0 tx=0x7fd58c007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:33.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.398+0000 7fd598ff9700 1 -- 172.21.15.83:0/2378154734 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd59c12d050 con 0x7fd59c15f180 2023-11-26T07:15:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.675+0000 7fd5a2569700 1 -- 172.21.15.83:0/2378154734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7fd59c12d050 con 0x7fd59c15f180 2023-11-26T07:15:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.906+0000 7fd598ff9700 1 -- 172.21.15.83:0/2378154734 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v222) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd59c12d050 con 0x7fd59c15f180 2023-11-26T07:15:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.908+0000 7fd5a2569700 1 -- 172.21.15.83:0/2378154734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd58806b230 msgr2=0x7fd58806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.908+0000 7fd5a2569700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd58806b230 0x7fd58806d6f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd58c007fd0 tx=0x7fd58c007de0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.908+0000 7fd5a2569700 1 -- 172.21.15.83:0/2378154734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c15f180 msgr2=0x7fd59c06bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.908+0000 7fd5a2569700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c15f180 0x7fd59c06bdd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd594007e40 tx=0x7fd59400e3c0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.909+0000 7fd5a2569700 1 -- 172.21.15.83:0/2378154734 shutdown_connections 2023-11-26T07:15:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.909+0000 7fd5a2569700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd59c15f180 0x7fd59c06bdd0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.909+0000 7fd5a2569700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd59c0cfd40 0x7fd59c144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.909+0000 7fd5a2569700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fd58806b230 0x7fd58806d6f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.909+0000 7fd5a2569700 1 --2- 172.21.15.83:0/2378154734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd59c13e000 0x7fd59c06b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.909+0000 7fd5a2569700 1 -- 172.21.15.83:0/2378154734 >> 172.21.15.83:0/2378154734 conn(0x7fd59c072990 msgr2=0x7fd59c0b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.909+0000 7fd5a2569700 1 -- 172.21.15.83:0/2378154734 shutdown_connections 2023-11-26T07:15:33.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:33.909+0000 7fd5a2569700 1 -- 172.21.15.83:0/2378154734 wait complete. 2023-11-26T07:15:33.924 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2023-11-26T07:15:33.924 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:15:33.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:34.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.043+0000 7fccae119700 1 Processor -- start 2023-11-26T07:15:34.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.044+0000 7fccae119700 1 -- start start 2023-11-26T07:15:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.044+0000 7fccae119700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcca815e080 0x7fcca80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.044+0000 7fccae119700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcca806c710 0x7fcca80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.044+0000 7fccae119700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca80ae430 con 0x7fcca80af4d0 2023-11-26T07:15:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.044+0000 7fccae119700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca80af0b0 con 0x7fcca806c710 2023-11-26T07:15:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.044+0000 7fccae119700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca80b05a0 con 0x7fcca815e080 2023-11-26T07:15:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.044+0000 7fcca7fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcca806c710 0x7fcca80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.044+0000 7fcca6ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcca80af4d0 0x7fcca806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59270/0 (socket says 172.21.15.83:59270) 2023-11-26T07:15:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.044+0000 7fcca6ffd700 1 -- 172.21.15.83:0/3676325593 learned_addr learned my addr 172.21.15.83:0/3676325593 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.045+0000 7fcca77fe700 1 --2- 172.21.15.83:0/3676325593 >> v2:172.21.15.203:3300/0 conn(0x7fcca815e080 0x7fcca80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.045+0000 7fcca7fff700 1 -- 172.21.15.83:0/3676325593 >> v2:172.21.15.203:3300/0 conn(0x7fcca815e080 msgr2=0x7fcca80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.045+0000 7fcca7fff700 1 --2- 172.21.15.83:0/3676325593 >> v2:172.21.15.203:3300/0 conn(0x7fcca815e080 0x7fcca80b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.045+0000 7fcca7fff700 1 -- 172.21.15.83:0/3676325593 >> v1:172.21.15.83:6789/0 conn(0x7fcca80af4d0 legacy=0x7fcca806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.045+0000 7fcca7fff700 1 -- 172.21.15.83:0/3676325593 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca80ae710 con 0x7fcca806c710 2023-11-26T07:15:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.045+0000 7fcca7fff700 1 --2- 172.21.15.83:0/3676325593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca806c710 0x7fcca80afb90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fcca8072a90 tx=0x7fcca000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97ad17d76b69239 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.045+0000 7fcca67fc700 1 -- 172.21.15.83:0/3676325593 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca000e070 con 0x7fcca806c710 2023-11-26T07:15:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.045+0000 7fcca67fc700 1 -- 172.21.15.83:0/3676325593 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca000bb90 con 0x7fcca806c710 2023-11-26T07:15:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.045+0000 7fcca67fc700 1 -- 172.21.15.83:0/3676325593 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0012430 con 0x7fcca806c710 2023-11-26T07:15:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.046+0000 7fccae119700 1 -- 172.21.15.83:0/3676325593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca806c710 msgr2=0x7fcca80afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.046+0000 7fccae119700 1 --2- 172.21.15.83:0/3676325593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca806c710 0x7fcca80afb90 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fcca8072a90 tx=0x7fcca000b330 comp rx=0 tx=0).stop 2023-11-26T07:15:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.046+0000 7fccae119700 1 -- 172.21.15.83:0/3676325593 shutdown_connections 2023-11-26T07:15:34.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.046+0000 7fccae119700 1 --2- 172.21.15.83:0/3676325593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca806c710 0x7fcca80afb90 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.046+0000 7fccae119700 1 --2- 172.21.15.83:0/3676325593 >> v2:172.21.15.203:3300/0 conn(0x7fcca815e080 0x7fcca80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.046+0000 7fccae119700 1 -- 172.21.15.83:0/3676325593 >> 172.21.15.83:0/3676325593 conn(0x7fcca8071b90 msgr2=0x7fcca8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:34.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.046+0000 7fccae119700 1 -- 172.21.15.83:0/3676325593 shutdown_connections 2023-11-26T07:15:34.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.046+0000 7fccae119700 1 -- 172.21.15.83:0/3676325593 wait complete. 2023-11-26T07:15:34.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.047+0000 7fccae119700 1 Processor -- start 2023-11-26T07:15:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.047+0000 7fccae119700 1 -- start start 2023-11-26T07:15:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fccae119700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca806c710 0x7fcca8151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fccae119700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcca80af4d0 0x7fcca815e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fccae119700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca815e080 0x7fcca8153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fccae119700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8157c50 con 0x7fcca815e080 2023-11-26T07:15:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fccae119700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8157360 con 0x7fcca806c710 2023-11-26T07:15:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fccae119700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8157600 con 0x7fcca80af4d0 2023-11-26T07:15:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcca7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca815e080 0x7fcca8153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcca7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca815e080 0x7fcca8153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39240/0 (socket says 172.21.15.83:39240) 2023-11-26T07:15:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcca7fff700 1 -- 172.21.15.83:0/1843682917 learned_addr learned my addr 172.21.15.83:0/1843682917 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcca77fe700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca806c710 0x7fcca8151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcca6ffd700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcca80af4d0 0x7fcca815e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcca7fff700 1 -- 172.21.15.83:0/1843682917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcca80af4d0 msgr2=0x7fcca815e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcca7fff700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcca80af4d0 0x7fcca815e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcca7fff700 1 -- 172.21.15.83:0/1843682917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca806c710 msgr2=0x7fcca8151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcca7fff700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca806c710 0x7fcca8151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcca7fff700 1 -- 172.21.15.83:0/1843682917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca0009a70 con 0x7fcca815e080 2023-11-26T07:15:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcca7fff700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca815e080 0x7fcca8153b20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fcca000b370 tx=0x7fcca0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcc9ffff700 1 -- 172.21.15.83:0/1843682917 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0012430 con 0x7fcca815e080 2023-11-26T07:15:34.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcc9ffff700 1 -- 172.21.15.83:0/1843682917 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca0020070 con 0x7fcca815e080 2023-11-26T07:15:34.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fccae119700 1 -- 172.21.15.83:0/1843682917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca81578e0 con 0x7fcca815e080 2023-11-26T07:15:34.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.048+0000 7fcc9ffff700 1 -- 172.21.15.83:0/1843682917 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0004280 con 0x7fcca815e080 2023-11-26T07:15:34.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.049+0000 7fccae119700 1 -- 172.21.15.83:0/1843682917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca815e9a0 con 0x7fcca815e080 2023-11-26T07:15:34.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.049+0000 7fccae119700 1 -- 172.21.15.83:0/1843682917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca80753f0 con 0x7fcca815e080 2023-11-26T07:15:34.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.050+0000 7fcc9ffff700 1 -- 172.21.15.83:0/1843682917 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fcca00043e0 con 0x7fcca815e080 2023-11-26T07:15:34.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.050+0000 7fcc9ffff700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc8806b230 0x7fcc8806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.050+0000 7fcc9ffff700 1 -- 172.21.15.83:0/1843682917 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcca0022070 con 0x7fcca815e080 2023-11-26T07:15:34.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.050+0000 7fcca77fe700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc8806b230 0x7fcc8806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.051+0000 7fcca77fe700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc8806b230 0x7fcc8806d6f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800a950 tx=0x7fcc98008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:34.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.055+0000 7fcc9ffff700 1 -- 172.21.15.83:0/1843682917 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcca00569b0 con 0x7fcca815e080 2023-11-26T07:15:34.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.245+0000 7fccae119700 1 -- 172.21.15.83:0/1843682917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcca8067f60 con 0x7fcca815e080 2023-11-26T07:15:34.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.246+0000 7fcc9ffff700 1 -- 172.21.15.83:0/1843682917 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+12970 (secure 0 0 0) 0x7fcca0059fd0 con 0x7fcca815e080 2023-11-26T07:15:34.248 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:34.248 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":61,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:15:32.902223+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6837/83108157":"2023-11-27T07:15:29.891356+0000","172.21.15.83:6835/83108157":"2023-11-27T07:15:29.891356+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:34.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 -- 172.21.15.83:0/1843682917 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc8806b230 msgr2=0x7fcc8806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc8806b230 0x7fcc8806d6f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800a950 tx=0x7fcc98008040 comp rx=0 tx=0).stop 2023-11-26T07:15:34.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 -- 172.21.15.83:0/1843682917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca815e080 msgr2=0x7fcca8153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca815e080 0x7fcca8153b20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fcca000b370 tx=0x7fcca0004ad0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 -- 172.21.15.83:0/1843682917 shutdown_connections 2023-11-26T07:15:34.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcca815e080 0x7fcca8153b20 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcca806c710 0x7fcca8151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcc8806b230 0x7fcc8806d6f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 --2- 172.21.15.83:0/1843682917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcca80af4d0 0x7fcca815e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 -- 172.21.15.83:0/1843682917 >> 172.21.15.83:0/1843682917 conn(0x7fcca8071b90 msgr2=0x7fcca80b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:34.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 -- 172.21.15.83:0/1843682917 shutdown_connections 2023-11-26T07:15:34.252 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.248+0000 7fccae119700 1 -- 172.21.15.83:0/1843682917 wait complete. 2023-11-26T07:15:34.265 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:15:34.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:15:34.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.387+0000 7f09025ab700 1 Processor -- start 2023-11-26T07:15:34.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.387+0000 7f09025ab700 1 -- start start 2023-11-26T07:15:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f09025ab700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f08fc15e160 0x7f08fc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f09025ab700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f08fc0746a0 0x7f08fc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f09025ab700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08fc0b3c60 con 0x7f08fc0afc90 2023-11-26T07:15:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f09025ab700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc0b4500 con 0x7f08fc15e160 2023-11-26T07:15:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f09025ab700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc0b58c0 con 0x7f08fc0746a0 2023-11-26T07:15:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f08fbfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f08fc15e160 0x7f08fc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f08fb7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f08fc0afc90 0x7f08fc073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59284/0 (socket says 172.21.15.83:59284) 2023-11-26T07:15:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f08fb7fe700 1 -- 172.21.15.83:0/749741518 learned_addr learned my addr 172.21.15.83:0/749741518 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f0900b48700 1 --2- 172.21.15.83:0/749741518 >> v2:172.21.15.203:3300/0 conn(0x7f08fc0746a0 0x7f08fc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f08fbfff700 1 -- 172.21.15.83:0/749741518 >> v2:172.21.15.203:3300/0 conn(0x7f08fc0746a0 msgr2=0x7f08fc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f08fbfff700 1 --2- 172.21.15.83:0/749741518 >> v2:172.21.15.203:3300/0 conn(0x7f08fc0746a0 0x7f08fc06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f08fbfff700 1 -- 172.21.15.83:0/749741518 >> v1:172.21.15.83:6789/0 conn(0x7f08fc0afc90 legacy=0x7f08fc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f08fbfff700 1 -- 172.21.15.83:0/749741518 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc0b0140 con 0x7f08fc15e160 2023-11-26T07:15:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f08faffd700 1 -- 172.21.15.83:0/749741518 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564318427 0 0) 0x7f08fc0b3c60 con 0x7f08fc0afc90 2023-11-26T07:15:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.388+0000 7f08fbfff700 1 --2- 172.21.15.83:0/749741518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08fc15e160 0x7f08fc0b5da0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0d16e0 tx=0x7f08f0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b083efbc89de8b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.389+0000 7f08faffd700 1 -- 172.21.15.83:0/749741518 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f0011830 con 0x7f08fc15e160 2023-11-26T07:15:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.389+0000 7f08faffd700 1 -- 172.21.15.83:0/749741518 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f0011990 con 0x7f08fc15e160 2023-11-26T07:15:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.389+0000 7f08faffd700 1 -- 172.21.15.83:0/749741518 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f0011d20 con 0x7f08fc15e160 2023-11-26T07:15:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.389+0000 7f09025ab700 1 -- 172.21.15.83:0/749741518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08fc15e160 msgr2=0x7f08fc0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.389+0000 7f09025ab700 1 --2- 172.21.15.83:0/749741518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08fc15e160 0x7f08fc0b5da0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0d16e0 tx=0x7f08f0009b20 comp rx=0 tx=0).stop 2023-11-26T07:15:34.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.389+0000 7f09025ab700 1 -- 172.21.15.83:0/749741518 shutdown_connections 2023-11-26T07:15:34.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.389+0000 7f09025ab700 1 --2- 172.21.15.83:0/749741518 >> v2:172.21.15.203:3300/0 conn(0x7f08fc0746a0 0x7f08fc06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.389+0000 7f09025ab700 1 --2- 172.21.15.83:0/749741518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08fc15e160 0x7f08fc0b5da0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.389+0000 7f09025ab700 1 -- 172.21.15.83:0/749741518 >> 172.21.15.83:0/749741518 conn(0x7f08fc0d0bf0 msgr2=0x7f08fc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:34.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.389+0000 7f09025ab700 1 -- 172.21.15.83:0/749741518 shutdown_connections 2023-11-26T07:15:34.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.389+0000 7f09025ab700 1 -- 172.21.15.83:0/749741518 wait complete. 2023-11-26T07:15:34.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.390+0000 7f09025ab700 1 Processor -- start 2023-11-26T07:15:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.391+0000 7f09025ab700 1 -- start start 2023-11-26T07:15:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.391+0000 7f09025ab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08fc0746a0 0x7f08fc158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.391+0000 7f09025ab700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08fc0afc90 0x7f08fc0c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.391+0000 7f09025ab700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08fc15e160 0x7f08fc0c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.391+0000 7f09025ab700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc158e50 con 0x7f08fc0afc90 2023-11-26T07:15:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.391+0000 7f09025ab700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc15e8f0 con 0x7f08fc0746a0 2023-11-26T07:15:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.391+0000 7f09025ab700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0c76b0 con 0x7f08fc15e160 2023-11-26T07:15:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.391+0000 7f08fb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08fc0afc90 0x7f08fc0c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.397 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.391+0000 7f08fb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08fc0afc90 0x7f08fc0c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39250/0 (socket says 172.21.15.83:39250) 2023-11-26T07:15:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.391+0000 7f08fb7fe700 1 -- 172.21.15.83:0/1059475437 learned_addr learned my addr 172.21.15.83:0/1059475437 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f08fbfff700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08fc0746a0 0x7f08fc158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f0900b48700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08fc15e160 0x7f08fc0c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f08fb7fe700 1 -- 172.21.15.83:0/1059475437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08fc15e160 msgr2=0x7f08fc0c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f08fb7fe700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08fc15e160 0x7f08fc0c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f08fb7fe700 1 -- 172.21.15.83:0/1059475437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08fc0746a0 msgr2=0x7f08fc158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f08fb7fe700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08fc0746a0 0x7f08fc158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f08fb7fe700 1 -- 172.21.15.83:0/1059475437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f00094a0 con 0x7f08fc0afc90 2023-11-26T07:15:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f08fb7fe700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08fc0afc90 0x7f08fc0c6b60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0011e0 tx=0x7f08ec00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:34.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f08f8ff9700 1 -- 172.21.15.83:0/1059475437 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec004c80 con 0x7f08fc0afc90 2023-11-26T07:15:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f08f8ff9700 1 -- 172.21.15.83:0/1059475437 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec004e00 con 0x7f08fc0afc90 2023-11-26T07:15:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f09025ab700 1 -- 172.21.15.83:0/1059475437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc15f670 con 0x7f08fc0afc90 2023-11-26T07:15:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f08f8ff9700 1 -- 172.21.15.83:0/1059475437 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec007310 con 0x7f08fc0afc90 2023-11-26T07:15:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.392+0000 7f09025ab700 1 -- 172.21.15.83:0/1059475437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc15eb90 con 0x7f08fc0afc90 2023-11-26T07:15:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.393+0000 7f09025ab700 1 -- 172.21.15.83:0/1059475437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08fc0b4c60 con 0x7f08fc0afc90 2023-11-26T07:15:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.393+0000 7f08f8ff9700 1 -- 172.21.15.83:0/1059475437 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f08ec007470 con 0x7f08fc0afc90 2023-11-26T07:15:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.394+0000 7f08f8ff9700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f08e806b230 0x7f08e806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.394+0000 7f08f8ff9700 1 -- 172.21.15.83:0/1059475437 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f08ec089670 con 0x7f08fc0afc90 2023-11-26T07:15:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.394+0000 7f08fbfff700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f08e806b230 0x7f08e806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.394+0000 7f08fbfff700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f08e806b230 0x7f08e806d6f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f08f00090b0 tx=0x7f08f001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.398+0000 7f08f8ff9700 1 -- 172.21.15.83:0/1059475437 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08ec054560 con 0x7f08fc0afc90 2023-11-26T07:15:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.588+0000 7f09025ab700 1 -- 172.21.15.83:0/1059475437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08fc068fb0 con 0x7f08fc0afc90 2023-11-26T07:15:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.588+0000 7f08f8ff9700 1 -- 172.21.15.83:0/1059475437 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08ec057b80 con 0x7f08fc0afc90 2023-11-26T07:15:34.591 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:15:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.590+0000 7f09025ab700 1 -- 172.21.15.83:0/1059475437 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f08e806b230 msgr2=0x7f08e806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.590+0000 7f09025ab700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f08e806b230 0x7f08e806d6f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f08f00090b0 tx=0x7f08f001c040 comp rx=0 tx=0).stop 2023-11-26T07:15:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.590+0000 7f09025ab700 1 -- 172.21.15.83:0/1059475437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08fc0afc90 msgr2=0x7f08fc0c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.590+0000 7f09025ab700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08fc0afc90 0x7f08fc0c6b60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0011e0 tx=0x7f08ec00cae0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.591+0000 7f09025ab700 1 -- 172.21.15.83:0/1059475437 shutdown_connections 2023-11-26T07:15:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.591+0000 7f09025ab700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f08e806b230 0x7f08e806d6f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.591+0000 7f09025ab700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08fc15e160 0x7f08fc0c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.591+0000 7f09025ab700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08fc0746a0 0x7f08fc158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.591+0000 7f09025ab700 1 --2- 172.21.15.83:0/1059475437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08fc0afc90 0x7f08fc0c6b60 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.591+0000 7f09025ab700 1 -- 172.21.15.83:0/1059475437 >> 172.21.15.83:0/1059475437 conn(0x7f08fc0d0bf0 msgr2=0x7f08fc162550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.591+0000 7f09025ab700 1 -- 172.21.15.83:0/1059475437 shutdown_connections 2023-11-26T07:15:34.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.591+0000 7f09025ab700 1 -- 172.21.15.83:0/1059475437 wait complete. 2023-11-26T07:15:34.605 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2023-11-26T07:15:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.723+0000 7fc0ec7fa700 1 Processor -- start 2023-11-26T07:15:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.723+0000 7fc0ec7fa700 1 -- start start 2023-11-26T07:15:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.724+0000 7fc0ec7fa700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0e41473a0 0x7fc0e415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.724+0000 7fc0ec7fa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0e4147780 0x7fc0e406c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.724+0000 7fc0ec7fa700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc0e406dc20 0x7fc0e4073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.724+0000 7fc0ec7fa700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e406c9a0 con 0x7fc0e41473a0 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.724+0000 7fc0ec7fa700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e415eb00 con 0x7fc0e406dc20 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.724+0000 7fc0ec7fa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e415ede0 con 0x7fc0e4147780 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.724+0000 7fc0ea596700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0e41473a0 0x7fc0e415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.724+0000 7fc0ea596700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc0e41473a0 0x7fc0e415e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39256/0 (socket says 172.21.15.83:39256) 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.724+0000 7fc0ea596700 1 -- 172.21.15.83:0/805033976 learned_addr learned my addr 172.21.15.83:0/805033976 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.725+0000 7fc0ead97700 1 --2- 172.21.15.83:0/805033976 >> v2:172.21.15.203:3300/0 conn(0x7fc0e406dc20 0x7fc0e4073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.725+0000 7fc0e9d95700 1 --2- 172.21.15.83:0/805033976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0e4147780 0x7fc0e406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.725+0000 7fc0ea596700 1 -- 172.21.15.83:0/805033976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0e4147780 msgr2=0x7fc0e406c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.725+0000 7fc0ea596700 1 --2- 172.21.15.83:0/805033976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0e4147780 0x7fc0e406c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.725+0000 7fc0ea596700 1 -- 172.21.15.83:0/805033976 >> v2:172.21.15.203:3300/0 conn(0x7fc0e406dc20 msgr2=0x7fc0e4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.725+0000 7fc0ea596700 1 --2- 172.21.15.83:0/805033976 >> v2:172.21.15.203:3300/0 conn(0x7fc0e406dc20 0x7fc0e4073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.725+0000 7fc0ea596700 1 -- 172.21.15.83:0/805033976 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e415f0c0 con 0x7fc0e41473a0 2023-11-26T07:15:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.725+0000 7fc0ea596700 1 --2- 172.21.15.83:0/805033976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e41473a0 0x7fc0e415e4b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc0e415e9f0 tx=0x7fc0d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725a01f39d05e86d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.725+0000 7fc0e9594700 1 -- 172.21.15.83:0/805033976 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400b870 con 0x7fc0e41473a0 2023-11-26T07:15:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.725+0000 7fc0e9594700 1 -- 172.21.15.83:0/805033976 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d400b9d0 con 0x7fc0e41473a0 2023-11-26T07:15:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.725+0000 7fc0e9594700 1 -- 172.21.15.83:0/805033976 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400bd60 con 0x7fc0e41473a0 2023-11-26T07:15:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.726+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/805033976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e41473a0 msgr2=0x7fc0e415e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.726+0000 7fc0ec7fa700 1 --2- 172.21.15.83:0/805033976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e41473a0 0x7fc0e415e4b0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc0e415e9f0 tx=0x7fc0d4009bd0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.726+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/805033976 shutdown_connections 2023-11-26T07:15:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.726+0000 7fc0ec7fa700 1 --2- 172.21.15.83:0/805033976 >> v2:172.21.15.203:3300/0 conn(0x7fc0e406dc20 0x7fc0e4073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.726+0000 7fc0ec7fa700 1 --2- 172.21.15.83:0/805033976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e41473a0 0x7fc0e415e4b0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.726+0000 7fc0ec7fa700 1 --2- 172.21.15.83:0/805033976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0e4147780 0x7fc0e406c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.726+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/805033976 >> 172.21.15.83:0/805033976 conn(0x7fc0e40b3130 msgr2=0x7fc0e40d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.726+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/805033976 shutdown_connections 2023-11-26T07:15:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.726+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/805033976 wait complete. 2023-11-26T07:15:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.727+0000 7fc0ec7fa700 1 Processor -- start 2023-11-26T07:15:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.727+0000 7fc0ec7fa700 1 -- start start 2023-11-26T07:15:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ec7fa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e406dc20 0x7fc0e40cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ec7fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e41473a0 0x7fc0e40ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ec7fa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0e4147780 0x7fc0e40addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ec7fa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e415de60 con 0x7fc0e406dc20 2023-11-26T07:15:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ec7fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e4137240 con 0x7fc0e41473a0 2023-11-26T07:15:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ec7fa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40ce280 con 0x7fc0e4147780 2023-11-26T07:15:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ea596700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e406dc20 0x7fc0e40cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ea596700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e406dc20 0x7fc0e40cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39262/0 (socket says 172.21.15.83:39262) 2023-11-26T07:15:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ea596700 1 -- 172.21.15.83:0/27875950 learned_addr learned my addr 172.21.15.83:0/27875950 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0e9d95700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e41473a0 0x7fc0e40ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ead97700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0e4147780 0x7fc0e40addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ea596700 1 -- 172.21.15.83:0/27875950 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0e4147780 msgr2=0x7fc0e40addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ea596700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0e4147780 0x7fc0e40addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ea596700 1 -- 172.21.15.83:0/27875950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e41473a0 msgr2=0x7fc0e40ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ea596700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e41473a0 0x7fc0e40ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ea596700 1 -- 172.21.15.83:0/27875950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d4009550 con 0x7fc0e406dc20 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0e9d95700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e41473a0 0x7fc0e40ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ea596700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e406dc20 0x7fc0e40cd930 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4000c00 tx=0x7fc0d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0daffd700 1 -- 172.21.15.83:0/27875950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4004540 con 0x7fc0e406dc20 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0daffd700 1 -- 172.21.15.83:0/27875950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d40036a0 con 0x7fc0e406dc20 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/27875950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e40ce560 con 0x7fc0e406dc20 2023-11-26T07:15:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0daffd700 1 -- 172.21.15.83:0/27875950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4003a50 con 0x7fc0e406dc20 2023-11-26T07:15:34.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.728+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/27875950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e40ceb20 con 0x7fc0e406dc20 2023-11-26T07:15:34.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.729+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/27875950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e4067f60 con 0x7fc0e406dc20 2023-11-26T07:15:34.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.730+0000 7fc0daffd700 1 -- 172.21.15.83:0/27875950 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fc0d4021070 con 0x7fc0e406dc20 2023-11-26T07:15:34.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.730+0000 7fc0daffd700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc0c806b230 0x7fc0c806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:34.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.730+0000 7fc0daffd700 1 -- 172.21.15.83:0/27875950 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc0d408af00 con 0x7fc0e406dc20 2023-11-26T07:15:34.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.730+0000 7fc0e9d95700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc0c806b230 0x7fc0c806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:34.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.730+0000 7fc0e9d95700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc0c806b230 0x7fc0c806d6f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4147c00 tx=0x7fc0e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:34.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:34.734+0000 7fc0daffd700 1 -- 172.21.15.83:0/27875950 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0d4055f30 con 0x7fc0e406dc20 2023-11-26T07:15:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.008+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/27875950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7fc0e40cd4e0 con 0x7fc0e406dc20 2023-11-26T07:15:35.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.912+0000 7fc0daffd700 1 -- 172.21.15.83:0/27875950 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v224) v1 ==== 134+0+0 (secure 0 0 0) 0x7fc0d4015020 con 0x7fc0e406dc20 2023-11-26T07:15:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/27875950 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc0c806b230 msgr2=0x7fc0c806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc0c806b230 0x7fc0c806d6f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4147c00 tx=0x7fc0e0008040 comp rx=0 tx=0).stop 2023-11-26T07:15:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/27875950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e406dc20 msgr2=0x7fc0e40cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e406dc20 0x7fc0e40cd930 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4000c00 tx=0x7fc0d4004750 comp rx=0 tx=0).stop 2023-11-26T07:15:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/27875950 shutdown_connections 2023-11-26T07:15:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc0c806b230 0x7fc0c806d6f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:35.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0e4147780 0x7fc0e40addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:35.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0e406dc20 0x7fc0e40cd930 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:35.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 --2- 172.21.15.83:0/27875950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0e41473a0 0x7fc0e40ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:35.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/27875950 >> 172.21.15.83:0/27875950 conn(0x7fc0e40b3130 msgr2=0x7fc0e415f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:35.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/27875950 shutdown_connections 2023-11-26T07:15:35.919 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:35.915+0000 7fc0ec7fa700 1 -- 172.21.15.83:0/27875950 wait complete. 2023-11-26T07:15:35.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:15:36.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.060+0000 7f9a4ee46700 1 Processor -- start 2023-11-26T07:15:36.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.060+0000 7f9a4ee46700 1 -- start start 2023-11-26T07:15:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4ee46700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9a4815b370 0x7f9a4815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4ee46700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a480b3c60 0x7f9a480b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4ee46700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a480ae5e0 con 0x7f9a481587f0 2023-11-26T07:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4ee46700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a480b4630 con 0x7f9a480b3c60 2023-11-26T07:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4ee46700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a4815be30 con 0x7f9a4815b370 2023-11-26T07:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4d3e3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a480b3c60 0x7f9a480b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4d3e3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a480b3c60 0x7f9a480b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39274/0 (socket says 172.21.15.83:39274) 2023-11-26T07:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4d3e3700 1 -- 172.21.15.83:0/3290846709 learned_addr learned my addr 172.21.15.83:0/3290846709 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a47fff700 1 --2- 172.21.15.83:0/3290846709 >> v2:172.21.15.203:3300/0 conn(0x7f9a4815b370 0x7f9a4815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4d3e3700 1 -- 172.21.15.83:0/3290846709 >> v2:172.21.15.203:3300/0 conn(0x7f9a4815b370 msgr2=0x7f9a4815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4d3e3700 1 --2- 172.21.15.83:0/3290846709 >> v2:172.21.15.203:3300/0 conn(0x7f9a4815b370 0x7f9a4815b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4d3e3700 1 -- 172.21.15.83:0/3290846709 >> v1:172.21.15.203:6789/0 conn(0x7f9a481587f0 legacy=0x7f9a4806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4d3e3700 1 -- 172.21.15.83:0/3290846709 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4815c110 con 0x7f9a480b3c60 2023-11-26T07:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a4d3e3700 1 --2- 172.21.15.83:0/3290846709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a480b3c60 0x7f9a480b40f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000d150 tx=0x7f9a4000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f330b9e83b5d5bf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a477fe700 1 -- 172.21.15.83:0/3290846709 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40010070 con 0x7f9a480b3c60 2023-11-26T07:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.061+0000 7f9a477fe700 1 -- 172.21.15.83:0/3290846709 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4000dd80 con 0x7f9a480b3c60 2023-11-26T07:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.062+0000 7f9a477fe700 1 -- 172.21.15.83:0/3290846709 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000b310 con 0x7f9a480b3c60 2023-11-26T07:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.062+0000 7f9a4ee46700 1 -- 172.21.15.83:0/3290846709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a480b3c60 msgr2=0x7f9a480b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.062+0000 7f9a4ee46700 1 --2- 172.21.15.83:0/3290846709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a480b3c60 0x7f9a480b40f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000d150 tx=0x7f9a4000d460 comp rx=0 tx=0).stop 2023-11-26T07:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.062+0000 7f9a4ee46700 1 -- 172.21.15.83:0/3290846709 shutdown_connections 2023-11-26T07:15:36.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.062+0000 7f9a4ee46700 1 --2- 172.21.15.83:0/3290846709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a480b3c60 0x7f9a480b40f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.062+0000 7f9a4ee46700 1 --2- 172.21.15.83:0/3290846709 >> v2:172.21.15.203:3300/0 conn(0x7f9a4815b370 0x7f9a4815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.062+0000 7f9a4ee46700 1 -- 172.21.15.83:0/3290846709 >> 172.21.15.83:0/3290846709 conn(0x7f9a480d0bf0 msgr2=0x7f9a480d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.062+0000 7f9a4ee46700 1 -- 172.21.15.83:0/3290846709 shutdown_connections 2023-11-26T07:15:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.062+0000 7f9a4ee46700 1 -- 172.21.15.83:0/3290846709 wait complete. 2023-11-26T07:15:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.063+0000 7f9a4ee46700 1 Processor -- start 2023-11-26T07:15:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.063+0000 7f9a4ee46700 1 -- start start 2023-11-26T07:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a4ee46700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a480b3c60 0x7f9a48145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a4ee46700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a481587f0 0x7f9a480b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a4ee46700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a4815b370 0x7f9a48150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a4ee46700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a48145a70 con 0x7f9a481587f0 2023-11-26T07:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a4ee46700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a480b0fb0 con 0x7f9a480b3c60 2023-11-26T07:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a4ee46700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a480b1290 con 0x7f9a4815b370 2023-11-26T07:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a47fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a481587f0 0x7f9a480b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a47fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a481587f0 0x7f9a480b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39278/0 (socket says 172.21.15.83:39278) 2023-11-26T07:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a47fff700 1 -- 172.21.15.83:0/1248963144 learned_addr learned my addr 172.21.15.83:0/1248963144 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a4cbe2700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a480b3c60 0x7f9a48145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a47fff700 1 -- 172.21.15.83:0/1248963144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a4815b370 msgr2=0x7f9a48150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.064+0000 7f9a4d3e3700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a4815b370 0x7f9a48150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.065+0000 7f9a47fff700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a4815b370 0x7f9a48150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.065+0000 7f9a47fff700 1 -- 172.21.15.83:0/1248963144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a480b3c60 msgr2=0x7f9a48145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.065+0000 7f9a47fff700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a480b3c60 0x7f9a48145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.065+0000 7f9a47fff700 1 -- 172.21.15.83:0/1248963144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4000bd40 con 0x7f9a481587f0 2023-11-26T07:15:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.065+0000 7f9a4cbe2700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a480b3c60 0x7f9a48145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.065+0000 7f9a47fff700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a481587f0 0x7f9a480b2050 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400d800 tx=0x7f9a3400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.065+0000 7f9a457fa700 1 -- 172.21.15.83:0/1248963144 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a340125f0 con 0x7f9a481587f0 2023-11-26T07:15:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.065+0000 7f9a457fa700 1 -- 172.21.15.83:0/1248963144 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a34012750 con 0x7f9a481587f0 2023-11-26T07:15:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.065+0000 7f9a4ee46700 1 -- 172.21.15.83:0/1248963144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a480b1570 con 0x7f9a481587f0 2023-11-26T07:15:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.065+0000 7f9a457fa700 1 -- 172.21.15.83:0/1248963144 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34012b00 con 0x7f9a481587f0 2023-11-26T07:15:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.065+0000 7f9a4ee46700 1 -- 172.21.15.83:0/1248963144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a480b1b30 con 0x7f9a481587f0 2023-11-26T07:15:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.066+0000 7f9a4ee46700 1 -- 172.21.15.83:0/1248963144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a480b1cc0 con 0x7f9a481587f0 2023-11-26T07:15:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.066+0000 7f9a457fa700 1 -- 172.21.15.83:0/1248963144 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f9a34012c60 con 0x7f9a481587f0 2023-11-26T07:15:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.067+0000 7f9a457fa700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9a2c06b230 0x7f9a2c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.067+0000 7f9a457fa700 1 -- 172.21.15.83:0/1248963144 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9a34089c40 con 0x7f9a481587f0 2023-11-26T07:15:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.067+0000 7f9a4cbe2700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9a2c06b230 0x7f9a2c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.067+0000 7f9a4cbe2700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9a2c06b230 0x7f9a2c06d6f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c000f60 tx=0x7f9a3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.071+0000 7f9a457fa700 1 -- 172.21.15.83:0/1248963144 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a34054be0 con 0x7f9a481587f0 2023-11-26T07:15:36.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.260+0000 7f9a4ee46700 1 -- 172.21.15.83:0/1248963144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9a48068fb0 con 0x7f9a481587f0 2023-11-26T07:15:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.264+0000 7f9a457fa700 1 -- 172.21.15.83:0/1248963144 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9a34058200 con 0x7f9a481587f0 2023-11-26T07:15:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 -- 172.21.15.83:0/1248963144 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9a2c06b230 msgr2=0x7f9a2c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9a2c06b230 0x7f9a2c06d6f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c000f60 tx=0x7f9a3c009040 comp rx=0 tx=0).stop 2023-11-26T07:15:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 -- 172.21.15.83:0/1248963144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a481587f0 msgr2=0x7f9a480b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a481587f0 0x7f9a480b2050 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400d800 tx=0x7f9a3400dbc0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 -- 172.21.15.83:0/1248963144 shutdown_connections 2023-11-26T07:15:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f9a2c06b230 0x7f9a2c06d6f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a4815b370 0x7f9a48150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a480b3c60 0x7f9a48145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 --2- 172.21.15.83:0/1248963144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a481587f0 0x7f9a480b2050 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 -- 172.21.15.83:0/1248963144 >> 172.21.15.83:0/1248963144 conn(0x7f9a480d0bf0 msgr2=0x7f9a481569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 -- 172.21.15.83:0/1248963144 shutdown_connections 2023-11-26T07:15:36.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.266+0000 7f9a4ee46700 1 -- 172.21.15.83:0/1248963144 wait complete. 2023-11-26T07:15:36.281 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:15:36.281 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:15:36.327 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:15:36.377 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:15:36.378 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:15:36.378 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:36.395 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:15:36.395 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:15:36.395 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:36.414 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:15:36.414 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:15:36.415 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:36.452 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:15:36.452 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:15:36.452 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:36.474 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:15:36.474 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:15:36.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:15:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.609+0000 7f53bc6b2700 1 Processor -- start 2023-11-26T07:15:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.609+0000 7f53bc6b2700 1 -- start start 2023-11-26T07:15:36.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.610+0000 7f53bc6b2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b40c9fc0 0x7f53b41691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.610+0000 7f53bc6b2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b40cfff0 con 0x7f53b4158a50 2023-11-26T07:15:36.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.610+0000 7f53bc6b2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b40cfd10 con 0x7f53b4158670 2023-11-26T07:15:36.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.610+0000 7f53bc6b2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40629b0 con 0x7f53b40c9fc0 2023-11-26T07:15:36.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b9c4d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f53b4158a50 0x7f53b40c98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59324/0 (socket says 172.21.15.83:59324) 2023-11-26T07:15:36.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b9c4d700 1 -- 172.21.15.83:0/2629229795 learned_addr learned my addr 172.21.15.83:0/2629229795 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:36.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53bac4f700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b40c9fc0 0x7f53b41691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954474994 0 0) 0x7f53b40cfff0 con 0x7f53b4158a50 2023-11-26T07:15:36.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a00035b0 con 0x7f53b4158a50 2023-11-26T07:15:36.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794139412 0 0) 0x7f53b40cfd10 con 0x7f53b4158670 2023-11-26T07:15:36.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b40cfff0 con 0x7f53b4158670 2023-11-26T07:15:36.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2606129972 0 0) 0x7f53a00035b0 con 0x7f53b4158a50 2023-11-26T07:15:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b40c9fc0 msgr2=0x7f53b41691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b40c9fc0 0x7f53b41691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 >> v1:172.21.15.203:6789/0 conn(0x7f53b4158670 legacy=0x7f53b40622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4161f50 con 0x7f53b4158a50 2023-11-26T07:15:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f53b0001670 con 0x7f53b4158a50 2023-11-26T07:15:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 >> v1:172.21.15.83:6789/0 conn(0x7f53b4158a50 legacy=0x7f53b40c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53a0003bc0 0x7f53a0003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 0x7f53a0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b40c9fc0 0x7f53a0005310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40629b0 con 0x7f53a0004850 2023-11-26T07:15:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a000bc40 con 0x7f53a0003bc0 2023-11-26T07:15:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a000be00 con 0x7f53b40c9fc0 2023-11-26T07:15:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b9c4d700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 0x7f53a0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53bac4f700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b40c9fc0 0x7f53a0005310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53ba44e700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53a0003bc0 0x7f53a0003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b9c4d700 1 -- 172.21.15.83:0/2629229795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b40c9fc0 msgr2=0x7f53a0005310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b9c4d700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b40c9fc0 0x7f53a0005310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b9c4d700 1 -- 172.21.15.83:0/2629229795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53a0003bc0 msgr2=0x7f53a0003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.611+0000 7f53b9c4d700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53a0003bc0 0x7f53a0003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53b9c4d700 1 -- 172.21.15.83:0/2629229795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a00035b0 con 0x7f53a0004850 2023-11-26T07:15:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53b9c4d700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 0x7f53a0004cd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f53b415e870 tx=0x7f53a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e830a06600264fca server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b000cc50 con 0x7f53a0004850 2023-11-26T07:15:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b000cdb0 con 0x7f53a0004850 2023-11-26T07:15:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53b944c700 1 -- 172.21.15.83:0/2629229795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0005ba0 con 0x7f53a0004850 2023-11-26T07:15:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53bc6b2700 1 -- 172.21.15.83:0/2629229795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 msgr2=0x7f53a0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53bc6b2700 1 --2- 172.21.15.83:0/2629229795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 0x7f53a0004cd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f53b415e870 tx=0x7f53a0005880 comp rx=0 tx=0).stop 2023-11-26T07:15:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53bac4f700 1 -- 172.21.15.83:0/2629229795 reap_dead start 2023-11-26T07:15:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53bc6b2700 1 -- 172.21.15.83:0/2629229795 shutdown_connections 2023-11-26T07:15:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53bc6b2700 1 -- 172.21.15.83:0/2629229795 >> 172.21.15.83:0/2629229795 conn(0x7f53b40b57b0 msgr2=0x7f53b40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53bc6b2700 1 -- 172.21.15.83:0/2629229795 shutdown_connections 2023-11-26T07:15:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.612+0000 7f53bc6b2700 1 -- 172.21.15.83:0/2629229795 wait complete. 2023-11-26T07:15:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.614+0000 7f53bc6b2700 1 Processor -- start 2023-11-26T07:15:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.614+0000 7f53bc6b2700 1 -- start start 2023-11-26T07:15:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.614+0000 7f53bc6b2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 0x7f53b40b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.614+0000 7f53bc6b2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b4154a60 0x7f53b4149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.614+0000 7f53bc6b2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b40b3ac0 0x7f53b40b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.614+0000 7f53bc6b2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40b4480 con 0x7f53a0004850 2023-11-26T07:15:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.614+0000 7f53bc6b2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4155e10 con 0x7f53b40b3ac0 2023-11-26T07:15:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.614+0000 7f53bc6b2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4155200 con 0x7f53b4154a60 2023-11-26T07:15:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.614+0000 7f53ba44e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 0x7f53b40b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.614+0000 7f53ba44e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 0x7f53b40b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39296/0 (socket says 172.21.15.83:39296) 2023-11-26T07:15:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.614+0000 7f53ba44e700 1 -- 172.21.15.83:0/3955931672 learned_addr learned my addr 172.21.15.83:0/3955931672 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53b9c4d700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b4154a60 0x7f53b4149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53bac4f700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b40b3ac0 0x7f53b40b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53ba44e700 1 -- 172.21.15.83:0/3955931672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b4154a60 msgr2=0x7f53b4149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53ba44e700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b4154a60 0x7f53b4149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53ba44e700 1 -- 172.21.15.83:0/3955931672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b40b3ac0 msgr2=0x7f53b40b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53ba44e700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b40b3ac0 0x7f53b40b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53ba44e700 1 -- 172.21.15.83:0/3955931672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b41554e0 con 0x7f53a0004850 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53b9c4d700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b4154a60 0x7f53b4149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53bac4f700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b40b3ac0 0x7f53b40b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53ba44e700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 0x7f53b40b2f90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f53a400f7a0 tx=0x7f53a400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53aaffd700 1 -- 172.21.15.83:0/3955931672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4005320 con 0x7f53a0004850 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53aaffd700 1 -- 172.21.15.83:0/3955931672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a40054a0 con 0x7f53a0004850 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53bc6b2700 1 -- 172.21.15.83:0/3955931672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b41557c0 con 0x7f53a0004850 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53aaffd700 1 -- 172.21.15.83:0/3955931672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4006aa0 con 0x7f53a0004850 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.615+0000 7f53bc6b2700 1 -- 172.21.15.83:0/3955931672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b4154ee0 con 0x7f53a0004850 2023-11-26T07:15:36.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.616+0000 7f53bc6b2700 1 -- 172.21.15.83:0/3955931672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b41560f0 con 0x7f53a0004850 2023-11-26T07:15:36.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.616+0000 7f53aaffd700 1 -- 172.21.15.83:0/3955931672 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f53a4006c00 con 0x7f53a0004850 2023-11-26T07:15:36.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.617+0000 7f53aaffd700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f539806b230 0x7f539806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:36.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.617+0000 7f53b9c4d700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f539806b230 0x7f539806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:36.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.617+0000 7f53aaffd700 1 -- 172.21.15.83:0/3955931672 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f53a40890c0 con 0x7f53a0004850 2023-11-26T07:15:36.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.617+0000 7f53b9c4d700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f539806b230 0x7f539806d6f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f53b415e870 tx=0x7f53b0007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:36.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.621+0000 7f53aaffd700 1 -- 172.21.15.83:0/3955931672 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53a4054060 con 0x7f53a0004850 2023-11-26T07:15:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.855+0000 7f53bc6b2700 1 -- 172.21.15.83:0/3955931672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53b4067f60 con 0x7f53a0004850 2023-11-26T07:15:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.855+0000 7f53aaffd700 1 -- 172.21.15.83:0/3955931672 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f53a4057680 con 0x7f53a0004850 2023-11-26T07:15:36.858 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":40,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5674,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/126258309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":126258309},{"type":"v1","addr":"172.21.15.203:6834","nonce":126258309}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4203521524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":4203521524},{"type":"v1","addr":"172.21.15.203:6835","nonce":4203521524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6836/1267615204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1267615204},{"type":"v1","addr":"172.21.15.83:6836","nonce":1267615204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6519,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2513932820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2513932820},{"type":"v1","addr":"172.21.15.83:6837","nonce":2513932820}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[]} 2023-11-26T07:15:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.857+0000 7f53bc6b2700 1 -- 172.21.15.83:0/3955931672 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f539806b230 msgr2=0x7f539806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.857+0000 7f53bc6b2700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f539806b230 0x7f539806d6f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f53b415e870 tx=0x7f53b0007000 comp rx=0 tx=0).stop 2023-11-26T07:15:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.857+0000 7f53bc6b2700 1 -- 172.21.15.83:0/3955931672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 msgr2=0x7f53b40b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.857+0000 7f53bc6b2700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 0x7f53b40b2f90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f53a400f7a0 tx=0x7f53a400fb60 comp rx=0 tx=0).stop 2023-11-26T07:15:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.858+0000 7f53bc6b2700 1 -- 172.21.15.83:0/3955931672 shutdown_connections 2023-11-26T07:15:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.858+0000 7f53bc6b2700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b40b3ac0 0x7f53b40b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.858+0000 7f53bc6b2700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53a0004850 0x7f53b40b2f90 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.858+0000 7f53bc6b2700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f539806b230 0x7f539806d6f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.858+0000 7f53bc6b2700 1 --2- 172.21.15.83:0/3955931672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b4154a60 0x7f53b4149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.858+0000 7f53bc6b2700 1 -- 172.21.15.83:0/3955931672 >> 172.21.15.83:0/3955931672 conn(0x7f53b40b57b0 msgr2=0x7f53b40d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.858+0000 7f53bc6b2700 1 -- 172.21.15.83:0/3955931672 shutdown_connections 2023-11-26T07:15:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:36.858+0000 7f53bc6b2700 1 -- 172.21.15.83:0/3955931672 wait complete. 2023-11-26T07:15:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 40 2023-11-26T07:15:36.873 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:15:36.873 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:15:36.873 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:15:36.873 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:15:36.873 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:15:36.873 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:15:36.874 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:15:36.874 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:15:36.874 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:15:36.874 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:15:36.874 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:15:36.874 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:15:36.874 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:15:36.875 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:15:36.875 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:15:36.875 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:15:36.882 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:15:36.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:15:36.883 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:15:36.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:15:36.886 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:15:36.886 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:15:36.886 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:15:36.887 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:15:36.924 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:15:36.924 DEBUG:teuthology.parallel:result is None 2023-11-26T07:15:36.925 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:15:36.925 DEBUG:teuthology.parallel:result is None 2023-11-26T07:15:36.928 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:15:36.929 DEBUG:teuthology.parallel:result is None 2023-11-26T07:15:36.929 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:15:36.929 DEBUG:teuthology.parallel:result is None 2023-11-26T07:15:36.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:15:37.003 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-26T07:15:37.003 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-26T07:15:37.016 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-11-26T07:15:37.016 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-11-26T07:15:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.055+0000 7f797b217700 1 Processor -- start 2023-11-26T07:15:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.055+0000 7f797b217700 1 -- start start 2023-11-26T07:15:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f797b217700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f797406c710 0x7f79740b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f797b217700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79740af4d0 0x7f79740afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f797b217700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79740ae430 con 0x7f797415e080 2023-11-26T07:15:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f797b217700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79740af0b0 con 0x7f79740af4d0 2023-11-26T07:15:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f797b217700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79740b05a0 con 0x7f797406c710 2023-11-26T07:15:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f79797b4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79740af4d0 0x7f79740afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f79797b4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f79740af4d0 0x7f79740afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39332/0 (socket says 172.21.15.83:39332) 2023-11-26T07:15:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f79797b4700 1 -- 172.21.15.83:0/1551132126 learned_addr learned my addr 172.21.15.83:0/1551132126 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f7973fff700 1 --2- 172.21.15.83:0/1551132126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f797406c710 0x7f79740b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:37.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f79797b4700 1 -- 172.21.15.83:0/1551132126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f797406c710 msgr2=0x7f79740b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f79797b4700 1 --2- 172.21.15.83:0/1551132126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f797406c710 0x7f79740b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f79797b4700 1 -- 172.21.15.83:0/1551132126 >> v1:172.21.15.203:6789/0 conn(0x7f797415e080 legacy=0x7f797406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f79797b4700 1 -- 172.21.15.83:0/1551132126 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79740ae710 con 0x7f79740af4d0 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f79797b4700 1 --2- 172.21.15.83:0/1551132126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79740af4d0 0x7f79740afb90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f796c009e80 tx=0x7f796c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc72ee628938c296 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f79737fe700 1 -- 172.21.15.83:0/1551132126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c00e070 con 0x7f79740af4d0 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f79737fe700 1 -- 172.21.15.83:0/1551132126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c003a20 con 0x7f79740af4d0 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.056+0000 7f79737fe700 1 -- 172.21.15.83:0/1551132126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c003dd0 con 0x7f79740af4d0 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.057+0000 7f797b217700 1 -- 172.21.15.83:0/1551132126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79740af4d0 msgr2=0x7f79740afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.057+0000 7f797b217700 1 --2- 172.21.15.83:0/1551132126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79740af4d0 0x7f79740afb90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f796c009e80 tx=0x7f796c00b330 comp rx=0 tx=0).stop 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.057+0000 7f797b217700 1 -- 172.21.15.83:0/1551132126 shutdown_connections 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.057+0000 7f797b217700 1 --2- 172.21.15.83:0/1551132126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79740af4d0 0x7f79740afb90 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:37.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.057+0000 7f797b217700 1 --2- 172.21.15.83:0/1551132126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f797406c710 0x7f79740b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:37.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.057+0000 7f797b217700 1 -- 172.21.15.83:0/1551132126 >> 172.21.15.83:0/1551132126 conn(0x7f7974071b90 msgr2=0x7f7974071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:37.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.057+0000 7f797b217700 1 -- 172.21.15.83:0/1551132126 shutdown_connections 2023-11-26T07:15:37.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.057+0000 7f797b217700 1 -- 172.21.15.83:0/1551132126 wait complete. 2023-11-26T07:15:37.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.058+0000 7f797b217700 1 Processor -- start 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.058+0000 7f797b217700 1 -- start start 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f797b217700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f797406c710 0x7f7974157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f797b217700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79740af4d0 0x7f7974147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f797b217700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f797415e080 0x7f79741f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f797b217700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79741597e0 con 0x7f797406c710 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f797b217700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7974159a30 con 0x7f79740af4d0 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f797b217700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7974159d10 con 0x7f797415e080 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f7978fb3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f797406c710 0x7f7974157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f7978fb3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f797406c710 0x7f7974157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39334/0 (socket says 172.21.15.83:39334) 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f7978fb3700 1 -- 172.21.15.83:0/70542645 learned_addr learned my addr 172.21.15.83:0/70542645 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f79797b4700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f797415e080 0x7f79741f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f7973fff700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79740af4d0 0x7f7974147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f7978fb3700 1 -- 172.21.15.83:0/70542645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f797415e080 msgr2=0x7f79741f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f7978fb3700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f797415e080 0x7f79741f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f7978fb3700 1 -- 172.21.15.83:0/70542645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79740af4d0 msgr2=0x7f7974147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f7978fb3700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79740af4d0 0x7f7974147ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f7978fb3700 1 -- 172.21.15.83:0/70542645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c009b30 con 0x7f797406c710 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.059+0000 7f7973fff700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79740af4d0 0x7f7974147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.060+0000 7f7978fb3700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f797406c710 0x7f7974157250 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f796800b4a0 tx=0x7f796800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.060+0000 7f79717fa700 1 -- 172.21.15.83:0/70542645 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968011830 con 0x7f797406c710 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.060+0000 7f79717fa700 1 -- 172.21.15.83:0/70542645 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7968011990 con 0x7f797406c710 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.060+0000 7f797b217700 1 -- 172.21.15.83:0/70542645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7974159ff0 con 0x7f797406c710 2023-11-26T07:15:37.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.060+0000 7f79717fa700 1 -- 172.21.15.83:0/70542645 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968011d40 con 0x7f797406c710 2023-11-26T07:15:37.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.060+0000 7f797b217700 1 -- 172.21.15.83:0/70542645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7974148010 con 0x7f797406c710 2023-11-26T07:15:37.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.061+0000 7f797b217700 1 -- 172.21.15.83:0/70542645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79740753f0 con 0x7f797406c710 2023-11-26T07:15:37.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.061+0000 7f79717fa700 1 -- 172.21.15.83:0/70542645 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f796801a400 con 0x7f797406c710 2023-11-26T07:15:37.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.062+0000 7f79717fa700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f795806b260 0x7f795806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:37.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.062+0000 7f7973fff700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f795806b260 0x7f795806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:37.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.062+0000 7f79717fa700 1 -- 172.21.15.83:0/70542645 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7968089330 con 0x7f797406c710 2023-11-26T07:15:37.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.062+0000 7f7973fff700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f795806b260 0x7f795806d720 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f79741f8000 tx=0x7f7960008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:37.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.066+0000 7f79717fa700 1 -- 172.21.15.83:0/70542645 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7968054360 con 0x7f797406c710 2023-11-26T07:15:37.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:37.279+0000 7f797b217700 1 -- 172.21.15.83:0/70542645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7974067f60 con 0x7f797406c710 2023-11-26T07:15:38.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.278+0000 7f79717fa700 1 -- 172.21.15.83:0/70542645 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7968057980 con 0x7f797406c710 2023-11-26T07:15:38.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 -- 172.21.15.83:0/70542645 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f795806b260 msgr2=0x7f795806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:38.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f795806b260 0x7f795806d720 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f79741f8000 tx=0x7f7960008040 comp rx=0 tx=0).stop 2023-11-26T07:15:38.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 -- 172.21.15.83:0/70542645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f797406c710 msgr2=0x7f7974157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:38.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f797406c710 0x7f7974157250 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f796800b4a0 tx=0x7f796800b7b0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 -- 172.21.15.83:0/70542645 shutdown_connections 2023-11-26T07:15:38.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f795806b260 0x7f795806d720 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f797415e080 0x7f79741f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f797406c710 0x7f7974157250 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 --2- 172.21.15.83:0/70542645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79740af4d0 0x7f7974147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 -- 172.21.15.83:0/70542645 >> 172.21.15.83:0/70542645 conn(0x7f7974071b90 msgr2=0x7f797406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:38.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 -- 172.21.15.83:0/70542645 shutdown_connections 2023-11-26T07:15:38.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.281+0000 7f797b217700 1 -- 172.21.15.83:0/70542645 wait complete. 2023-11-26T07:15:38.285 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2023-11-26T07:15:38.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:15:38.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.423+0000 7fcfd258c700 1 Processor -- start 2023-11-26T07:15:38.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.423+0000 7fcfd258c700 1 -- start start 2023-11-26T07:15:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.423+0000 7fcfd258c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfcc15e1e0 0x7fcfcc0d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.423+0000 7fcfd258c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcfcc0d3a00 0x7fcfcc0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.423+0000 7fcfd258c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcfcc0d0db0 0x7fcfcc06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.423+0000 7fcfd258c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc0d3150 con 0x7fcfcc0d0db0 2023-11-26T07:15:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.423+0000 7fcfd258c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc0d1240 con 0x7fcfcc0d3a00 2023-11-26T07:15:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.423+0000 7fcfd258c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc15e5c0 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfd0b29700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcfcc0d0db0 0x7fcfcc06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfd0b29700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcfcc0d0db0 0x7fcfcc06aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39350/0 (socket says 172.21.15.83:39350) 2023-11-26T07:15:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfd0b29700 1 -- 172.21.15.83:0/3770230959 learned_addr learned my addr 172.21.15.83:0/3770230959 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:38.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfcbfff700 1 --2- 172.21.15.83:0/3770230959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfcc15e1e0 0x7fcfcc0d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfcb7fe700 1 --2- 172.21.15.83:0/3770230959 >> v2:172.21.15.203:3300/0 conn(0x7fcfcc0d3a00 0x7fcfcc0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfd0b29700 1 -- 172.21.15.83:0/3770230959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfcc15e1e0 msgr2=0x7fcfcc0d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfd0b29700 1 --2- 172.21.15.83:0/3770230959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfcc15e1e0 0x7fcfcc0d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfd0b29700 1 -- 172.21.15.83:0/3770230959 >> v2:172.21.15.203:3300/0 conn(0x7fcfcc0d3a00 msgr2=0x7fcfcc0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfd0b29700 1 --2- 172.21.15.83:0/3770230959 >> v2:172.21.15.203:3300/0 conn(0x7fcfcc0d3a00 0x7fcfcc0d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfd0b29700 1 -- 172.21.15.83:0/3770230959 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0d20b0 con 0x7fcfcc0d0db0 2023-11-26T07:15:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfd0b29700 1 --2- 172.21.15.83:0/3770230959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfcc0d0db0 0x7fcfcc06aff0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc08c1d0 tx=0x7fcfc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c92a6c94dbeb84d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfcaffd700 1 -- 172.21.15.83:0/3770230959 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc400e070 con 0x7fcfcc0d0db0 2023-11-26T07:15:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfcaffd700 1 -- 172.21.15.83:0/3770230959 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc400b980 con 0x7fcfcc0d0db0 2023-11-26T07:15:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.424+0000 7fcfcaffd700 1 -- 172.21.15.83:0/3770230959 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc400bd30 con 0x7fcfcc0d0db0 2023-11-26T07:15:38.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.425+0000 7fcfd258c700 1 -- 172.21.15.83:0/3770230959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfcc0d0db0 msgr2=0x7fcfcc06aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.425+0000 7fcfd258c700 1 --2- 172.21.15.83:0/3770230959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfcc0d0db0 0x7fcfcc06aff0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc08c1d0 tx=0x7fcfc400b330 comp rx=0 tx=0).stop 2023-11-26T07:15:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.425+0000 7fcfd258c700 1 -- 172.21.15.83:0/3770230959 shutdown_connections 2023-11-26T07:15:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.425+0000 7fcfd258c700 1 --2- 172.21.15.83:0/3770230959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfcc0d0db0 0x7fcfcc06aff0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.425+0000 7fcfd258c700 1 --2- 172.21.15.83:0/3770230959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfcc15e1e0 0x7fcfcc0d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.425+0000 7fcfd258c700 1 --2- 172.21.15.83:0/3770230959 >> v2:172.21.15.203:3300/0 conn(0x7fcfcc0d3a00 0x7fcfcc0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.425+0000 7fcfd258c700 1 -- 172.21.15.83:0/3770230959 >> 172.21.15.83:0/3770230959 conn(0x7fcfcc0b5b50 msgr2=0x7fcfcc0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.425+0000 7fcfd258c700 1 -- 172.21.15.83:0/3770230959 shutdown_connections 2023-11-26T07:15:38.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.425+0000 7fcfd258c700 1 -- 172.21.15.83:0/3770230959 wait complete. 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.426+0000 7fcfd258c700 1 Processor -- start 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.427+0000 7fcfd258c700 1 -- start start 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.427+0000 7fcfd258c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfcc0d0db0 0x7fcfcc1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.427+0000 7fcfd258c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfcc0d3a00 0x7fcfcc160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.427+0000 7fcfd258c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfcc15e1e0 0x7fcfcc159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.427+0000 7fcfd258c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0cc020 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.427+0000 7fcfd258c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc15f9f0 con 0x7fcfcc0d0db0 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.427+0000 7fcfd258c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc15fcd0 con 0x7fcfcc0d3a00 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfd0b29700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfcc15e1e0 0x7fcfcc159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfd0b29700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfcc15e1e0 0x7fcfcc159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39356/0 (socket says 172.21.15.83:39356) 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfd0b29700 1 -- 172.21.15.83:0/2701680777 learned_addr learned my addr 172.21.15.83:0/2701680777 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:38.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfcbfff700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfcc0d0db0 0x7fcfcc1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfcb7fe700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfcc0d3a00 0x7fcfcc160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfd0b29700 1 -- 172.21.15.83:0/2701680777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfcc0d3a00 msgr2=0x7fcfcc160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfd0b29700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfcc0d3a00 0x7fcfcc160d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfd0b29700 1 -- 172.21.15.83:0/2701680777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfcc0d0db0 msgr2=0x7fcfcc1530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfd0b29700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfcc0d0db0 0x7fcfcc1530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfd0b29700 1 -- 172.21.15.83:0/2701680777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc4009b30 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfd0b29700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfcc15e1e0 0x7fcfcc159bd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc08c570 tx=0x7fcfc4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfc8ff9700 1 -- 172.21.15.83:0/2701680777 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4004250 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfc8ff9700 1 -- 172.21.15.83:0/2701680777 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc40043b0 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfd258c700 1 -- 172.21.15.83:0/2701680777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc15ffb0 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfc8ff9700 1 -- 172.21.15.83:0/2701680777 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4004740 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.428+0000 7fcfd258c700 1 -- 172.21.15.83:0/2701680777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc160570 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.429+0000 7fcfd258c700 1 -- 172.21.15.83:0/2701680777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc0751e0 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.430+0000 7fcfc8ff9700 1 -- 172.21.15.83:0/2701680777 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fcfc401c440 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.430+0000 7fcfc8ff9700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcfb806b230 0x7fcfb806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.430+0000 7fcfc8ff9700 1 -- 172.21.15.83:0/2701680777 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcfc408aef0 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.430+0000 7fcfcbfff700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcfb806b230 0x7fcfb806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.430+0000 7fcfcbfff700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcfb806b230 0x7fcfb806d6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b930 tx=0x7fcfbc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:38.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.435+0000 7fcfc8ff9700 1 -- 172.21.15.83:0/2701680777 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfc4055dd0 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.702+0000 7fcfd258c700 1 -- 172.21.15.83:0/2701680777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcfcc15d550 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.703+0000 7fcfc8ff9700 1 -- 172.21.15.83:0/2701680777 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5617 (secure 0 0 0) 0x7fcfc402a020 con 0x7fcfcc15e1e0 2023-11-26T07:15:38.705 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:38.705 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:15:38.705 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlA8pxNxAAt5NuewAjm7yPThCguJ6sEA==", 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.706 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlHBgDCBAAk6egig5krY7kiMiHkMKrWg==", 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlbKEQAxAAM2tBBQwM+tt2d8JU37YURg==", 2023-11-26T07:15:38.707 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlI3ZHDxAArtLmBVGr/vAB/YGwEFfhGw==", 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:15:38.708 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBo72JlhuSTKRAA6LRDOHBQVl5TL5tIHzyfzA==", 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2023-11-26T07:15:38.709 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBr72JlteU6KRAAuuHTYosYpl+KptywVmaeHA==", 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBu72JlEb6dKBAAebp4pnu9XZsSRJpojTZcvQ==", 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:15:38.710 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBx72JlbTJ3KBAA1JUxxAoyCzTtfgubUxSJEA==", 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.711 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB372JlufqqLhAAm6ZUt9juWj5yAIDlnCXqWg==", 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB672Jl1L3vLRAAX06GcsntcmB+7L1CsfnJzw==", 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:15:38.712 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB972Jlv41HLRAALtuyuREvGmTuDZt/RT0UcA==", 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2023-11-26T07:15:38.713 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCA72JlKCHsLBAAAXDzA7AN36wcLjhK97nlkg==", 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ==", 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2023-11-26T07:15:38.714 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JleNpIGhAAKwoQj4zB9YOxpFMD28juCA==", 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.715 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JldAHlFhAAb3u9KljU9vssDl54zgno2Q==", 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JluiQUIBAA06mqpeCzgpbSLo2/fulfmw==", 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:15:38.716 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlD4jbGhAA05xtSS5EzQTuhBjVqS2JpA==", 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.717 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.alice", 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCI8GJlGXHNKhAA5ZF4a69YcKlXkvwoR2gTSw==", 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol86790", 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2023-11-26T07:15:38.718 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlsA0VNBAAfSlnVquhI+vpLFwmgLGGVA==", 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72Jl3qkjMBAA+jcxYuDG/UHhB7W7HDwmaw==", 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:15:38.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:38.720 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:38.720 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:15:38.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 -- 172.21.15.83:0/2701680777 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcfb806b230 msgr2=0x7fcfb806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:38.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcfb806b230 0x7fcfb806d6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b930 tx=0x7fcfbc008040 comp rx=0 tx=0).stop 2023-11-26T07:15:38.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 -- 172.21.15.83:0/2701680777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfcc15e1e0 msgr2=0x7fcfcc159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:38.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfcc15e1e0 0x7fcfcc159bd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc08c570 tx=0x7fcfc4004ad0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 -- 172.21.15.83:0/2701680777 shutdown_connections 2023-11-26T07:15:38.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfcc15e1e0 0x7fcfcc159bd0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfcc0d0db0 0x7fcfcc1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcfb806b230 0x7fcfb806d6f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 --2- 172.21.15.83:0/2701680777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfcc0d3a00 0x7fcfcc160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 -- 172.21.15.83:0/2701680777 >> 172.21.15.83:0/2701680777 conn(0x7fcfcc0b5b50 msgr2=0x7fcfcc0b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:38.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 -- 172.21.15.83:0/2701680777 shutdown_connections 2023-11-26T07:15:38.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.705+0000 7fcfd258c700 1 -- 172.21.15.83:0/2701680777 wait complete. 2023-11-26T07:15:38.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2023-11-26T07:15:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.841+0000 7f8934222700 1 Processor -- start 2023-11-26T07:15:38.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.841+0000 7f8934222700 1 -- start start 2023-11-26T07:15:38.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.842+0000 7f8934222700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f892c0b0f80 0x7f892c15a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.842+0000 7f8934222700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f892c0cd770 0x7f892c15a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.842+0000 7f8934222700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c0b2bd0 con 0x7f892c0b0ba0 2023-11-26T07:15:38.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.842+0000 7f8934222700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892c0b1fc0 con 0x7f892c0cd770 2023-11-26T07:15:38.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.842+0000 7f8934222700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892c0b22a0 con 0x7f892c0b0f80 2023-11-26T07:15:38.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.842+0000 7f89327bf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f892c0cd770 0x7f892c15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.842+0000 7f8931fbe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f892c0b0ba0 0x7f892c0af5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59400/0 (socket says 172.21.15.83:59400) 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.842+0000 7f8931fbe700 1 -- 172.21.15.83:0/2858766611 learned_addr learned my addr 172.21.15.83:0/2858766611 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f89317bd700 1 --2- 172.21.15.83:0/2858766611 >> v2:172.21.15.203:3300/0 conn(0x7f892c0b0f80 0x7f892c15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f89327bf700 1 -- 172.21.15.83:0/2858766611 >> v2:172.21.15.203:3300/0 conn(0x7f892c0b0f80 msgr2=0x7f892c15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f89327bf700 1 --2- 172.21.15.83:0/2858766611 >> v2:172.21.15.203:3300/0 conn(0x7f892c0b0f80 0x7f892c15a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f89327bf700 1 -- 172.21.15.83:0/2858766611 >> v1:172.21.15.83:6789/0 conn(0x7f892c0b0ba0 legacy=0x7f892c0af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f8930fbc700 1 -- 172.21.15.83:0/2858766611 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751442096 0 0) 0x7f892c0b2bd0 con 0x7f892c0b0ba0 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f89327bf700 1 -- 172.21.15.83:0/2858766611 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c0b2580 con 0x7f892c0cd770 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f89327bf700 1 --2- 172.21.15.83:0/2858766611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f892c0cd770 0x7f892c15a980 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f892c072270 tx=0x7f892400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d360787d77a8dfe server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f8930fbc700 1 -- 172.21.15.83:0/2858766611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892400e070 con 0x7f892c0cd770 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f8930fbc700 1 -- 172.21.15.83:0/2858766611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892400bbd0 con 0x7f892c0cd770 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f8930fbc700 1 -- 172.21.15.83:0/2858766611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924012430 con 0x7f892c0cd770 2023-11-26T07:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f8934222700 1 -- 172.21.15.83:0/2858766611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f892c0cd770 msgr2=0x7f892c15a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.843+0000 7f8934222700 1 --2- 172.21.15.83:0/2858766611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f892c0cd770 0x7f892c15a980 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f892c072270 tx=0x7f892400b330 comp rx=0 tx=0).stop 2023-11-26T07:15:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.844+0000 7f8934222700 1 -- 172.21.15.83:0/2858766611 shutdown_connections 2023-11-26T07:15:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.844+0000 7f8934222700 1 --2- 172.21.15.83:0/2858766611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f892c0cd770 0x7f892c15a980 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.844+0000 7f8934222700 1 --2- 172.21.15.83:0/2858766611 >> v2:172.21.15.203:3300/0 conn(0x7f892c0b0f80 0x7f892c15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.844+0000 7f8934222700 1 -- 172.21.15.83:0/2858766611 >> 172.21.15.83:0/2858766611 conn(0x7f892c0b57b0 msgr2=0x7f892c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.844+0000 7f8934222700 1 -- 172.21.15.83:0/2858766611 shutdown_connections 2023-11-26T07:15:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.844+0000 7f8934222700 1 -- 172.21.15.83:0/2858766611 wait complete. 2023-11-26T07:15:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.845+0000 7f8934222700 1 Processor -- start 2023-11-26T07:15:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.845+0000 7f8934222700 1 -- start start 2023-11-26T07:15:38.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.845+0000 7f8934222700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0b0ba0 0x7f892c06ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f8934222700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f892c0b0f80 0x7f892c099590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f8934222700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f892c0cd770 0x7f892c151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f8934222700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c151ee0 con 0x7f892c0cd770 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f8934222700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c099ad0 con 0x7f892c0b0f80 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f8934222700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c06dfb0 con 0x7f892c0b0ba0 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89327bf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f892c0cd770 0x7f892c151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89327bf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f892c0cd770 0x7f892c151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39370/0 (socket says 172.21.15.83:39370) 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89327bf700 1 -- 172.21.15.83:0/1350777949 learned_addr learned my addr 172.21.15.83:0/1350777949 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89317bd700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f892c0b0f80 0x7f892c099590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f8931fbe700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0b0ba0 0x7f892c06ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89327bf700 1 -- 172.21.15.83:0/1350777949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0b0ba0 msgr2=0x7f892c06ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89327bf700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0b0ba0 0x7f892c06ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89327bf700 1 -- 172.21.15.83:0/1350777949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f892c0b0f80 msgr2=0x7f892c099590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89327bf700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f892c0b0f80 0x7f892c099590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89327bf700 1 -- 172.21.15.83:0/1350777949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8924009a70 con 0x7f892c0cd770 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89327bf700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f892c0cd770 0x7f892c151520 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f892c0722a0 tx=0x7f8924004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89227fc700 1 -- 172.21.15.83:0/1350777949 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924012430 con 0x7f892c0cd770 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89227fc700 1 -- 172.21.15.83:0/1350777949 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8924020070 con 0x7f892c0cd770 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f8934222700 1 -- 172.21.15.83:0/1350777949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892c06e290 con 0x7f892c0cd770 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f89227fc700 1 -- 172.21.15.83:0/1350777949 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924004280 con 0x7f892c0cd770 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.846+0000 7f8934222700 1 -- 172.21.15.83:0/1350777949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892c06e850 con 0x7f892c0cd770 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.847+0000 7f8934222700 1 -- 172.21.15.83:0/1350777949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892c06e9e0 con 0x7f892c0cd770 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.848+0000 7f89227fc700 1 -- 172.21.15.83:0/1350777949 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f89240043e0 con 0x7f892c0cd770 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.848+0000 7f89227fc700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f891006b230 0x7f891006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.848+0000 7f89227fc700 1 -- 172.21.15.83:0/1350777949 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8924022070 con 0x7f892c0cd770 2023-11-26T07:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.848+0000 7f8931fbe700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f891006b230 0x7f891006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:38.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.848+0000 7f8931fbe700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f891006b230 0x7f891006d6f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f891c000f60 tx=0x7f891c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:38.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:38.853+0000 7f89227fc700 1 -- 172.21.15.83:0/1350777949 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8924056ac0 con 0x7f892c0cd770 2023-11-26T07:15:39.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.122+0000 7f8934222700 1 -- 172.21.15.83:0/1350777949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) v1 -- 0x7f892c067f60 con 0x7f892c0cd770 2023-11-26T07:15:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.126+0000 7f89227fc700 1 -- 172.21.15.83:0/1350777949 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v19) v1 ==== 82+0+0 (secure 0 0 0) 0x7f892401ae60 con 0x7f892c0cd770 2023-11-26T07:15:39.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 -- 172.21.15.83:0/1350777949 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f891006b230 msgr2=0x7f891006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f891006b230 0x7f891006d6f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f891c000f60 tx=0x7f891c009040 comp rx=0 tx=0).stop 2023-11-26T07:15:39.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 -- 172.21.15.83:0/1350777949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f892c0cd770 msgr2=0x7f892c151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f892c0cd770 0x7f892c151520 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f892c0722a0 tx=0x7f8924004ad0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 -- 172.21.15.83:0/1350777949 shutdown_connections 2023-11-26T07:15:39.131 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f892c0cd770 0x7f892c151520 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f891006b230 0x7f891006d6f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f892c0b0ba0 0x7f892c06ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 --2- 172.21.15.83:0/1350777949 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f892c0b0f80 0x7f892c099590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 -- 172.21.15.83:0/1350777949 >> 172.21.15.83:0/1350777949 conn(0x7f892c0b57b0 msgr2=0x7f892c0cac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:39.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 -- 172.21.15.83:0/1350777949 shutdown_connections 2023-11-26T07:15:39.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.128+0000 7f8934222700 1 -- 172.21.15.83:0/1350777949 wait complete. 2023-11-26T07:15:39.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:39.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196fa2c700 1 Processor -- start 2023-11-26T07:15:39.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196fa2c700 1 -- start start 2023-11-26T07:15:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196fa2c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f196815e2a0 0x7f196806ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196fa2c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1968073f90 0x7f1968074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196fa2c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196806e7f0 con 0x7f19680ce090 2023-11-26T07:15:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196fa2c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1968074960 con 0x7f1968073f90 2023-11-26T07:15:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196fa2c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196806dec0 con 0x7f196815e2a0 2023-11-26T07:15:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196d7c8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f196815e2a0 0x7f196806ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196dfc9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1968073f90 0x7f1968074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196cfc7700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f19680ce090 0x7f196806ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:44032/0 (socket says 172.21.15.83:44032) 2023-11-26T07:15:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196cfc7700 1 -- 172.21.15.83:0/3592401855 learned_addr learned my addr 172.21.15.83:0/3592401855 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196d7c8700 1 -- 172.21.15.83:0/3592401855 >> v2:172.21.15.203:3300/0 conn(0x7f1968073f90 msgr2=0x7f1968074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196d7c8700 1 --2- 172.21.15.83:0/3592401855 >> v2:172.21.15.203:3300/0 conn(0x7f1968073f90 0x7f1968074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196d7c8700 1 -- 172.21.15.83:0/3592401855 >> v1:172.21.15.203:6789/0 conn(0x7f19680ce090 legacy=0x7f196806ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196d7c8700 1 -- 172.21.15.83:0/3592401855 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196806e1a0 con 0x7f196815e2a0 2023-11-26T07:15:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f195ffff700 1 -- 172.21.15.83:0/3592401855 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435594228 0 0) 0x7f196806e7f0 con 0x7f19680ce090 2023-11-26T07:15:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.261+0000 7f196dfc9700 1 --2- 172.21.15.83:0/3592401855 >> v2:172.21.15.203:3300/0 conn(0x7f1968073f90 0x7f1968074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:15:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f196d7c8700 1 --2- 172.21.15.83:0/3592401855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f196815e2a0 0x7f196806ba70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1964009840 tx=0x7f1964009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0e630f8ad9cb819 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f195ffff700 1 -- 172.21.15.83:0/3592401855 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964011830 con 0x7f196815e2a0 2023-11-26T07:15:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f195ffff700 1 -- 172.21.15.83:0/3592401855 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1964011990 con 0x7f196815e2a0 2023-11-26T07:15:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f195ffff700 1 -- 172.21.15.83:0/3592401855 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964011d20 con 0x7f196815e2a0 2023-11-26T07:15:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f196fa2c700 1 -- 172.21.15.83:0/3592401855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f196815e2a0 msgr2=0x7f196806ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f196fa2c700 1 --2- 172.21.15.83:0/3592401855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f196815e2a0 0x7f196806ba70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1964009840 tx=0x7f1964009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f196fa2c700 1 -- 172.21.15.83:0/3592401855 shutdown_connections 2023-11-26T07:15:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f196fa2c700 1 --2- 172.21.15.83:0/3592401855 >> v2:172.21.15.203:3300/0 conn(0x7f1968073f90 0x7f1968074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f196fa2c700 1 --2- 172.21.15.83:0/3592401855 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f196815e2a0 0x7f196806ba70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f196fa2c700 1 -- 172.21.15.83:0/3592401855 >> 172.21.15.83:0/3592401855 conn(0x7f19680cd570 msgr2=0x7f19680cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:39.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f196fa2c700 1 -- 172.21.15.83:0/3592401855 shutdown_connections 2023-11-26T07:15:39.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.262+0000 7f196fa2c700 1 -- 172.21.15.83:0/3592401855 wait complete. 2023-11-26T07:15:39.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.264+0000 7f196fa2c700 1 Processor -- start 2023-11-26T07:15:39.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.264+0000 7f196fa2c700 1 -- start start 2023-11-26T07:15:39.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.264+0000 7f196fa2c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1968073f90 0x7f196815e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.264+0000 7f196fa2c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19680ce090 0x7f1968144650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.264+0000 7f196fa2c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196815e2a0 0x7f1968160030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.264+0000 7f196fa2c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196815ef90 con 0x7f196815e2a0 2023-11-26T07:15:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.264+0000 7f196fa2c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196815f270 con 0x7f19680ce090 2023-11-26T07:15:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.264+0000 7f196fa2c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196815f550 con 0x7f1968073f90 2023-11-26T07:15:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196dfc9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196815e2a0 0x7f1968160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196dfc9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196815e2a0 0x7f1968160030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39382/0 (socket says 172.21.15.83:39382) 2023-11-26T07:15:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196dfc9700 1 -- 172.21.15.83:0/2172447901 learned_addr learned my addr 172.21.15.83:0/2172447901 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196d7c8700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1968073f90 0x7f196815e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196cfc7700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19680ce090 0x7f1968144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196dfc9700 1 -- 172.21.15.83:0/2172447901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1968073f90 msgr2=0x7f196815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196dfc9700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1968073f90 0x7f196815e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196dfc9700 1 -- 172.21.15.83:0/2172447901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19680ce090 msgr2=0x7f1968144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196dfc9700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19680ce090 0x7f1968144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196dfc9700 1 -- 172.21.15.83:0/2172447901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960009a70 con 0x7f196815e2a0 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196dfc9700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196815e2a0 0x7f1968160030 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f196000efc0 tx=0x7f196000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f195dffb700 1 -- 172.21.15.83:0/2172447901 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000ca90 con 0x7f196815e2a0 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f195dffb700 1 -- 172.21.15.83:0/2172447901 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196000f040 con 0x7f196815e2a0 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196fa2c700 1 -- 172.21.15.83:0/2172447901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19640094a0 con 0x7f196815e2a0 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f195dffb700 1 -- 172.21.15.83:0/2172447901 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000ce30 con 0x7f196815e2a0 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.265+0000 7f196fa2c700 1 -- 172.21.15.83:0/2172447901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196815fb10 con 0x7f196815e2a0 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.266+0000 7f196fa2c700 1 -- 172.21.15.83:0/2172447901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1968067f60 con 0x7f196815e2a0 2023-11-26T07:15:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.267+0000 7f195dffb700 1 -- 172.21.15.83:0/2172447901 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f1960014480 con 0x7f196815e2a0 2023-11-26T07:15:39.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.267+0000 7f195dffb700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f194c06b230 0x7f194c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.267+0000 7f195dffb700 1 -- 172.21.15.83:0/2172447901 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1960089ed0 con 0x7f196815e2a0 2023-11-26T07:15:39.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.267+0000 7f196d7c8700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f194c06b230 0x7f194c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.268+0000 7f196d7c8700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f194c06b230 0x7f194c06d6f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f19640090b0 tx=0x7f1964008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:39.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.272+0000 7f195dffb700 1 -- 172.21.15.83:0/2172447901 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1960054e40 con 0x7f196815e2a0 2023-11-26T07:15:39.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.462+0000 7f196fa2c700 1 -- 172.21.15.83:0/2172447901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19680d0870 con 0x7f196815e2a0 2023-11-26T07:15:39.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.463+0000 7f195dffb700 1 -- 172.21.15.83:0/2172447901 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+12845 (secure 0 0 0) 0x7f1960058460 con 0x7f196815e2a0 2023-11-26T07:15:39.465 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:39.465 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":66,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:15:38.276092+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:39.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 -- 172.21.15.83:0/2172447901 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f194c06b230 msgr2=0x7f194c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f194c06b230 0x7f194c06d6f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f19640090b0 tx=0x7f1964008030 comp rx=0 tx=0).stop 2023-11-26T07:15:39.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 -- 172.21.15.83:0/2172447901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196815e2a0 msgr2=0x7f1968160030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196815e2a0 0x7f1968160030 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f196000efc0 tx=0x7f196000c3a0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 -- 172.21.15.83:0/2172447901 shutdown_connections 2023-11-26T07:15:39.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f196815e2a0 0x7f1968160030 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f194c06b230 0x7f194c06d6f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1968073f90 0x7f196815e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 --2- 172.21.15.83:0/2172447901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19680ce090 0x7f1968144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 -- 172.21.15.83:0/2172447901 >> 172.21.15.83:0/2172447901 conn(0x7f19680cd570 msgr2=0x7f19680cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:39.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 -- 172.21.15.83:0/2172447901 shutdown_connections 2023-11-26T07:15:39.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.465+0000 7f196fa2c700 1 -- 172.21.15.83:0/2172447901 wait complete. 2023-11-26T07:15:39.480 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:15:39.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:15:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.601+0000 7fc670772700 1 Processor -- start 2023-11-26T07:15:39.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.601+0000 7fc670772700 1 -- start start 2023-11-26T07:15:39.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc670772700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc6680746a0 0x7fc6680b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc670772700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680b3c60 con 0x7fc66815e160 2023-11-26T07:15:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc670772700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680b58c0 con 0x7fc6680afc90 2023-11-26T07:15:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc670772700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6680b4500 con 0x7fc6680746a0 2023-11-26T07:15:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66e50e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc66815e160 0x7fc668073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59426/0 (socket says 172.21.15.83:59426) 2023-11-26T07:15:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66e50e700 1 -- 172.21.15.83:0/2412982060 learned_addr learned my addr 172.21.15.83:0/2412982060 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66dd0d700 1 --2- 172.21.15.83:0/2412982060 >> v2:172.21.15.203:3300/0 conn(0x7fc6680746a0 0x7fc6680b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57248907 0 0) 0x7fc6680b3c60 con 0x7fc66815e160 2023-11-26T07:15:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6540035b0 con 0x7fc66815e160 2023-11-26T07:15:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1155331468 0 0) 0x7fc6540035b0 con 0x7fc66815e160 2023-11-26T07:15:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 >> v2:172.21.15.203:3300/0 conn(0x7fc6680746a0 msgr2=0x7fc6680b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:39.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 --2- 172.21.15.83:0/2412982060 >> v2:172.21.15.203:3300/0 conn(0x7fc6680746a0 0x7fc6680b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 >> v1:172.21.15.203:6789/0 conn(0x7fc6680afc90 legacy=0x7fc66806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66806bdc0 con 0x7fc66815e160 2023-11-26T07:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fc658001670 con 0x7fc66815e160 2023-11-26T07:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 >> v1:172.21.15.83:6789/0 conn(0x7fc66815e160 legacy=0x7fc668073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc654003a00 0x7fc654003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc654004850 0x7fc654004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc654005210 0x7fc65400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680b4500 con 0x7fc654005210 2023-11-26T07:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65400c0b0 con 0x7fc654003a00 2023-11-26T07:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65400c270 con 0x7fc654004850 2023-11-26T07:15:39.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66ed0f700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc654005210 0x7fc65400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66e50e700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc654003a00 0x7fc654003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66dd0d700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc654004850 0x7fc654004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66ed0f700 1 -- 172.21.15.83:0/2412982060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc654004850 msgr2=0x7fc654004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:39.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66ed0f700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc654004850 0x7fc654004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66ed0f700 1 -- 172.21.15.83:0/2412982060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc654003a00 msgr2=0x7fc654003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:39.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66ed0f700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc654003a00 0x7fc654003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66ed0f700 1 -- 172.21.15.83:0/2412982060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540035b0 con 0x7fc654005210 2023-11-26T07:15:39.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66dd0d700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc654004850 0x7fc654004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:39.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.603+0000 7fc66ed0f700 1 -- 172.21.15.83:0/2412982060 reap_dead start 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.605+0000 7fc66ed0f700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc654005210 0x7fc65400ba10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c190 tx=0x7fc654005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7dc57818681d36 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.605+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660001db0 con 0x7fc654005210 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.605+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66000ccf0 con 0x7fc654005210 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.605+0000 7fc66d50c700 1 -- 172.21.15.83:0/2412982060 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660003c40 con 0x7fc654005210 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.605+0000 7fc670772700 1 -- 172.21.15.83:0/2412982060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc654005210 msgr2=0x7fc65400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.605+0000 7fc670772700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc654005210 0x7fc65400ba10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c190 tx=0x7fc654005880 comp rx=0 tx=0).stop 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.605+0000 7fc670772700 1 -- 172.21.15.83:0/2412982060 shutdown_connections 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.605+0000 7fc670772700 1 --2- 172.21.15.83:0/2412982060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc654005210 0x7fc65400ba10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.605+0000 7fc670772700 1 -- 172.21.15.83:0/2412982060 >> 172.21.15.83:0/2412982060 conn(0x7fc6680d0bf0 msgr2=0x7fc6680d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.605+0000 7fc670772700 1 -- 172.21.15.83:0/2412982060 shutdown_connections 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.605+0000 7fc670772700 1 -- 172.21.15.83:0/2412982060 wait complete. 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.607+0000 7fc670772700 1 Processor -- start 2023-11-26T07:15:39.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.607+0000 7fc670772700 1 -- start start 2023-11-26T07:15:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.607+0000 7fc670772700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc654005210 0x7fc6681f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.607+0000 7fc670772700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6681f6d00 0x7fc66815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.607+0000 7fc670772700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc66815af70 0x7fc668155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.607+0000 7fc670772700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66815b420 con 0x7fc6681f6d00 2023-11-26T07:15:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.607+0000 7fc670772700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc668155fc0 con 0x7fc66815af70 2023-11-26T07:15:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.607+0000 7fc670772700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc668158580 con 0x7fc654005210 2023-11-26T07:15:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.607+0000 7fc66dd0d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6681f6d00 0x7fc66815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.607+0000 7fc66dd0d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6681f6d00 0x7fc66815aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39400/0 (socket says 172.21.15.83:39400) 2023-11-26T07:15:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.607+0000 7fc66dd0d700 1 -- 172.21.15.83:0/3452934112 learned_addr learned my addr 172.21.15.83:0/3452934112 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:39.615 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc66e50e700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc654005210 0x7fc6681f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc66ed0f700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc66815af70 0x7fc668155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc66dd0d700 1 -- 172.21.15.83:0/3452934112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc654005210 msgr2=0x7fc6681f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc66dd0d700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc654005210 0x7fc6681f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc66dd0d700 1 -- 172.21.15.83:0/3452934112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc66815af70 msgr2=0x7fc668155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc66dd0d700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc66815af70 0x7fc668155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc66dd0d700 1 -- 172.21.15.83:0/3452934112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc668158860 con 0x7fc6681f6d00 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc66dd0d700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6681f6d00 0x7fc66815aa30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc664010850 tx=0x7fc664010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc65effd700 1 -- 172.21.15.83:0/3452934112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664010e50 con 0x7fc6681f6d00 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc65effd700 1 -- 172.21.15.83:0/3452934112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc664004020 con 0x7fc6681f6d00 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc670772700 1 -- 172.21.15.83:0/3452934112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc668158b40 con 0x7fc6681f6d00 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc65effd700 1 -- 172.21.15.83:0/3452934112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6640043b0 con 0x7fc6681f6d00 2023-11-26T07:15:39.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.608+0000 7fc670772700 1 -- 172.21.15.83:0/3452934112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6681f7180 con 0x7fc6681f6d00 2023-11-26T07:15:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.609+0000 7fc670772700 1 -- 172.21.15.83:0/3452934112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6680b51c0 con 0x7fc6681f6d00 2023-11-26T07:15:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.609+0000 7fc65effd700 1 -- 172.21.15.83:0/3452934112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fc66400e7f0 con 0x7fc6681f6d00 2023-11-26T07:15:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.610+0000 7fc65effd700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc64c06b230 0x7fc64c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.610+0000 7fc65effd700 1 -- 172.21.15.83:0/3452934112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc664089990 con 0x7fc6681f6d00 2023-11-26T07:15:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.610+0000 7fc66e50e700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc64c06b230 0x7fc64c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.610+0000 7fc66e50e700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc64c06b230 0x7fc64c06d6f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc658000f60 tx=0x7fc658009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:39.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.614+0000 7fc65effd700 1 -- 172.21.15.83:0/3452934112 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc664054a10 con 0x7fc6681f6d00 2023-11-26T07:15:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.805+0000 7fc670772700 1 -- 172.21.15.83:0/3452934112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc668068fb0 con 0x7fc6681f6d00 2023-11-26T07:15:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.805+0000 7fc65effd700 1 -- 172.21.15.83:0/3452934112 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc664058030 con 0x7fc6681f6d00 2023-11-26T07:15:39.807 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:15:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 -- 172.21.15.83:0/3452934112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc64c06b230 msgr2=0x7fc64c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc64c06b230 0x7fc64c06d6f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc658000f60 tx=0x7fc658009040 comp rx=0 tx=0).stop 2023-11-26T07:15:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 -- 172.21.15.83:0/3452934112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6681f6d00 msgr2=0x7fc66815aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6681f6d00 0x7fc66815aa30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc664010850 tx=0x7fc664010c10 comp rx=0 tx=0).stop 2023-11-26T07:15:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 -- 172.21.15.83:0/3452934112 shutdown_connections 2023-11-26T07:15:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc66815af70 0x7fc668155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fc64c06b230 0x7fc64c06d6f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc654005210 0x7fc6681f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 --2- 172.21.15.83:0/3452934112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6681f6d00 0x7fc66815aa30 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 -- 172.21.15.83:0/3452934112 >> 172.21.15.83:0/3452934112 conn(0x7fc6680d0bf0 msgr2=0x7fc668073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:39.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 -- 172.21.15.83:0/3452934112 shutdown_connections 2023-11-26T07:15:39.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.807+0000 7fc670772700 1 -- 172.21.15.83:0/3452934112 wait complete. 2023-11-26T07:15:39.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:15:39.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.941+0000 7fb714817700 1 Processor -- start 2023-11-26T07:15:39.942 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.941+0000 7fb714817700 1 -- start start 2023-11-26T07:15:39.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb714817700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb71015bdd0 0x7fb7101604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb714817700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7100b3370 0x7fb7100b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb714817700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb7100d1c70 0x7fb7100d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.945 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb714817700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7100b2f50 con 0x7fb71015bdd0 2023-11-26T07:15:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb714817700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7100d2660 con 0x7fb7100d1c70 2023-11-26T07:15:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb714817700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100b2030 con 0x7fb7100b3370 2023-11-26T07:15:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70e59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb71015bdd0 0x7fb7101604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70e59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb71015bdd0 0x7fb7101604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39406/0 (socket says 172.21.15.83:39406) 2023-11-26T07:15:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70e59c700 1 -- 172.21.15.83:0/2181662147 learned_addr learned my addr 172.21.15.83:0/2181662147 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70dd9b700 1 --2- 172.21.15.83:0/2181662147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7100b3370 0x7fb7100b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70e59c700 1 -- 172.21.15.83:0/2181662147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7100b3370 msgr2=0x7fb7100b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70ed9d700 1 --2- 172.21.15.83:0/2181662147 >> v2:172.21.15.203:3300/0 conn(0x7fb7100d1c70 0x7fb7100d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70e59c700 1 --2- 172.21.15.83:0/2181662147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7100b3370 0x7fb7100b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70e59c700 1 -- 172.21.15.83:0/2181662147 >> v2:172.21.15.203:3300/0 conn(0x7fb7100d1c70 msgr2=0x7fb7100d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:39.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70e59c700 1 --2- 172.21.15.83:0/2181662147 >> v2:172.21.15.203:3300/0 conn(0x7fb7100d1c70 0x7fb7100d2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70e59c700 1 -- 172.21.15.83:0/2181662147 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100b22d0 con 0x7fb71015bdd0 2023-11-26T07:15:39.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70dd9b700 1 --2- 172.21.15.83:0/2181662147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7100b3370 0x7fb7100b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:39.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70ed9d700 1 --2- 172.21.15.83:0/2181662147 >> v2:172.21.15.203:3300/0 conn(0x7fb7100d1c70 0x7fb7100d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:39.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.942+0000 7fb70e59c700 1 --2- 172.21.15.83:0/2181662147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb71015bdd0 0x7fb7101604e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb710160a20 tx=0x7fb704009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bdd91e24fe8761 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:39.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb70d59a700 1 -- 172.21.15.83:0/2181662147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70400b870 con 0x7fb71015bdd0 2023-11-26T07:15:39.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb70d59a700 1 -- 172.21.15.83:0/2181662147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70400b9d0 con 0x7fb71015bdd0 2023-11-26T07:15:39.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb70d59a700 1 -- 172.21.15.83:0/2181662147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70400bd60 con 0x7fb71015bdd0 2023-11-26T07:15:39.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb714817700 1 -- 172.21.15.83:0/2181662147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb71015bdd0 msgr2=0x7fb7101604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb714817700 1 --2- 172.21.15.83:0/2181662147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb71015bdd0 0x7fb7101604e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb710160a20 tx=0x7fb704009bd0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb714817700 1 -- 172.21.15.83:0/2181662147 shutdown_connections 2023-11-26T07:15:39.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb714817700 1 --2- 172.21.15.83:0/2181662147 >> v2:172.21.15.203:3300/0 conn(0x7fb7100d1c70 0x7fb7100d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb714817700 1 --2- 172.21.15.83:0/2181662147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb71015bdd0 0x7fb7101604e0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb714817700 1 --2- 172.21.15.83:0/2181662147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7100b3370 0x7fb7100b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb714817700 1 -- 172.21.15.83:0/2181662147 >> 172.21.15.83:0/2181662147 conn(0x7fb7100731a0 msgr2=0x7fb71009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:39.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb714817700 1 -- 172.21.15.83:0/2181662147 shutdown_connections 2023-11-26T07:15:39.948 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.943+0000 7fb714817700 1 -- 172.21.15.83:0/2181662147 wait complete. 2023-11-26T07:15:39.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.945+0000 7fb714817700 1 Processor -- start 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.945+0000 7fb714817700 1 -- start start 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.945+0000 7fb714817700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7100b3370 0x7fb7101572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.945+0000 7fb714817700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7100d1c70 0x7fb71006bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.945+0000 7fb714817700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb71015bdd0 0x7fb7100ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.945+0000 7fb714817700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb710157810 con 0x7fb7100d1c70 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.945+0000 7fb714817700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100cd920 con 0x7fb71015bdd0 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.945+0000 7fb714817700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100cdc00 con 0x7fb7100b3370 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb70dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7100d1c70 0x7fb71006bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb70dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7100d1c70 0x7fb71006bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39414/0 (socket says 172.21.15.83:39414) 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb70dd9b700 1 -- 172.21.15.83:0/4184564568 learned_addr learned my addr 172.21.15.83:0/4184564568 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb70e59c700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7100b3370 0x7fb7101572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb70ed9d700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb71015bdd0 0x7fb7100ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb70dd9b700 1 -- 172.21.15.83:0/4184564568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7100b3370 msgr2=0x7fb7101572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb70dd9b700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7100b3370 0x7fb7101572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb70dd9b700 1 -- 172.21.15.83:0/4184564568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb71015bdd0 msgr2=0x7fb7100ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb70dd9b700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb71015bdd0 0x7fb7100ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb70dd9b700 1 -- 172.21.15.83:0/4184564568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb704009550 con 0x7fb7100d1c70 2023-11-26T07:15:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb70dd9b700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7100d1c70 0x7fb71006bfc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb70000c7f0 tx=0x7fb70000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb6feffd700 1 -- 172.21.15.83:0/4184564568 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700011800 con 0x7fb7100d1c70 2023-11-26T07:15:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb6feffd700 1 -- 172.21.15.83:0/4184564568 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb700011960 con 0x7fb7100d1c70 2023-11-26T07:15:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb714817700 1 -- 172.21.15.83:0/4184564568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7100cdee0 con 0x7fb7100d1c70 2023-11-26T07:15:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb6feffd700 1 -- 172.21.15.83:0/4184564568 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700011d10 con 0x7fb7100d1c70 2023-11-26T07:15:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.946+0000 7fb714817700 1 -- 172.21.15.83:0/4184564568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7100ce4a0 con 0x7fb7100d1c70 2023-11-26T07:15:39.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.947+0000 7fb714817700 1 -- 172.21.15.83:0/4184564568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb710067f60 con 0x7fb7100d1c70 2023-11-26T07:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.948+0000 7fb6feffd700 1 -- 172.21.15.83:0/4184564568 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fb700018360 con 0x7fb7100d1c70 2023-11-26T07:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.948+0000 7fb6feffd700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb6f006b230 0x7fb6f006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.948+0000 7fb6feffd700 1 -- 172.21.15.83:0/4184564568 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb700089700 con 0x7fb7100d1c70 2023-11-26T07:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.948+0000 7fb70e59c700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb6f006b230 0x7fb6f006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.948+0000 7fb70e59c700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb6f006b230 0x7fb6f006d6f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb704000c00 tx=0x7fb704008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:39.953 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:39.952+0000 7fb6feffd700 1 -- 172.21.15.83:0/4184564568 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7000545e0 con 0x7fb7100d1c70 2023-11-26T07:15:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.149+0000 7fb714817700 1 -- 172.21.15.83:0/4184564568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb7100b7c60 con 0x7fb7100d1c70 2023-11-26T07:15:40.150 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.149+0000 7fb6feffd700 1 -- 172.21.15.83:0/4184564568 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb700057c00 con 0x7fb7100d1c70 2023-11-26T07:15:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:40.151 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:15:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2023-11-26T07:15:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2023-11-26T07:15:40.151 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:40.151 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:15:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 -- 172.21.15.83:0/4184564568 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb6f006b230 msgr2=0x7fb6f006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb6f006b230 0x7fb6f006d6f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb704000c00 tx=0x7fb704008030 comp rx=0 tx=0).stop 2023-11-26T07:15:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 -- 172.21.15.83:0/4184564568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7100d1c70 msgr2=0x7fb71006bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7100d1c70 0x7fb71006bfc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb70000c7f0 tx=0x7fb70000cb00 comp rx=0 tx=0).stop 2023-11-26T07:15:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 -- 172.21.15.83:0/4184564568 shutdown_connections 2023-11-26T07:15:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb71015bdd0 0x7fb7100ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb6f006b230 0x7fb6f006d6f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7100b3370 0x7fb7101572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 --2- 172.21.15.83:0/4184564568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7100d1c70 0x7fb71006bfc0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 -- 172.21.15.83:0/4184564568 >> 172.21.15.83:0/4184564568 conn(0x7fb7100731a0 msgr2=0x7fb710159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 -- 172.21.15.83:0/4184564568 shutdown_connections 2023-11-26T07:15:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.151+0000 7fb714817700 1 -- 172.21.15.83:0/4184564568 wait complete. 2023-11-26T07:15:40.165 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:15:40.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:15:40.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.287+0000 7f4372935700 1 Processor -- start 2023-11-26T07:15:40.289 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.287+0000 7f4372935700 1 -- start start 2023-11-26T07:15:40.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.288+0000 7f4372935700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c144600 0x7f436c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:40.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.288+0000 7f4372935700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f436c06f480 0x7f436c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:40.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.288+0000 7f4372935700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436c06b200 con 0x7f436c06e830 2023-11-26T07:15:40.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f4372935700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436c06aff0 con 0x7f436c06f480 2023-11-26T07:15:40.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f4372935700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c06b4e0 con 0x7f436c144600 2023-11-26T07:15:40.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f436bfff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c144600 0x7f436c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:40.291 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f436b7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f436c06f480 0x7f436c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f4370ed2700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f436c06e830 0x7f436c0cdf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:44082/0 (socket says 172.21.15.83:44082) 2023-11-26T07:15:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f4370ed2700 1 -- 172.21.15.83:0/3739856272 learned_addr learned my addr 172.21.15.83:0/3739856272 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f436bfff700 1 -- 172.21.15.83:0/3739856272 >> v2:172.21.15.203:3300/0 conn(0x7f436c06f480 msgr2=0x7f436c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f436bfff700 1 --2- 172.21.15.83:0/3739856272 >> v2:172.21.15.203:3300/0 conn(0x7f436c06f480 0x7f436c06c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f436bfff700 1 -- 172.21.15.83:0/3739856272 >> v1:172.21.15.203:6789/0 conn(0x7f436c06e830 legacy=0x7f436c0cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f436bfff700 1 -- 172.21.15.83:0/3739856272 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c06b7c0 con 0x7f436c144600 2023-11-26T07:15:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f436b7fe700 1 --2- 172.21.15.83:0/3739856272 >> v2:172.21.15.203:3300/0 conn(0x7f436c06f480 0x7f436c06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:15:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.289+0000 7f436bfff700 1 --2- 172.21.15.83:0/3739856272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c144600 0x7f436c0b4ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f435c009840 tx=0x7f435c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45c08e7bb293b058 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.290+0000 7f436affd700 1 -- 172.21.15.83:0/3739856272 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c011830 con 0x7f436c144600 2023-11-26T07:15:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.290+0000 7f436affd700 1 -- 172.21.15.83:0/3739856272 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c011990 con 0x7f436c144600 2023-11-26T07:15:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.290+0000 7f436affd700 1 -- 172.21.15.83:0/3739856272 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c011d20 con 0x7f436c144600 2023-11-26T07:15:40.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.290+0000 7f4372935700 1 -- 172.21.15.83:0/3739856272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c144600 msgr2=0x7f436c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.290+0000 7f4372935700 1 --2- 172.21.15.83:0/3739856272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c144600 0x7f436c0b4ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f435c009840 tx=0x7f435c009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.290+0000 7f4372935700 1 -- 172.21.15.83:0/3739856272 shutdown_connections 2023-11-26T07:15:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.290+0000 7f4372935700 1 --2- 172.21.15.83:0/3739856272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c144600 0x7f436c0b4ee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.290+0000 7f4372935700 1 --2- 172.21.15.83:0/3739856272 >> v2:172.21.15.203:3300/0 conn(0x7f436c06f480 0x7f436c06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.290+0000 7f4372935700 1 -- 172.21.15.83:0/3739856272 >> 172.21.15.83:0/3739856272 conn(0x7f436c0d0b40 msgr2=0x7f436c0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.290+0000 7f4372935700 1 -- 172.21.15.83:0/3739856272 shutdown_connections 2023-11-26T07:15:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.290+0000 7f4372935700 1 -- 172.21.15.83:0/3739856272 wait complete. 2023-11-26T07:15:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.291+0000 7f4372935700 1 Processor -- start 2023-11-26T07:15:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.292+0000 7f4372935700 1 -- start start 2023-11-26T07:15:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.292+0000 7f4372935700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c06e830 0x7f436c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.292+0000 7f4372935700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c06f480 0x7f436c0c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.292+0000 7f4372935700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436c144600 0x7f436c0c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.292+0000 7f4372935700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c13eff0 con 0x7f436c06f480 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.292+0000 7f4372935700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c13e370 con 0x7f436c144600 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.292+0000 7f4372935700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c13e650 con 0x7f436c06e830 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.292+0000 7f436b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c06f480 0x7f436c0c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.292+0000 7f436b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c06f480 0x7f436c0c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39428/0 (socket says 172.21.15.83:39428) 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f436b7fe700 1 -- 172.21.15.83:0/168445548 learned_addr learned my addr 172.21.15.83:0/168445548 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f436bfff700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c06e830 0x7f436c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f4370ed2700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436c144600 0x7f436c0c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f436b7fe700 1 -- 172.21.15.83:0/168445548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c06e830 msgr2=0x7f436c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f436b7fe700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c06e830 0x7f436c15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f436b7fe700 1 -- 172.21.15.83:0/168445548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436c144600 msgr2=0x7f436c0c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f436b7fe700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436c144600 0x7f436c0c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f436b7fe700 1 -- 172.21.15.83:0/168445548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43600094a0 con 0x7f436c06f480 2023-11-26T07:15:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f436b7fe700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c06f480 0x7f436c0c69a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4360009ef0 tx=0x7f436000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f4368ff9700 1 -- 172.21.15.83:0/168445548 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000c940 con 0x7f436c06f480 2023-11-26T07:15:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f4368ff9700 1 -- 172.21.15.83:0/168445548 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436000caa0 con 0x7f436c06f480 2023-11-26T07:15:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f4372935700 1 -- 172.21.15.83:0/168445548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c0094a0 con 0x7f436c06f480 2023-11-26T07:15:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f4368ff9700 1 -- 172.21.15.83:0/168445548 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000ce30 con 0x7f436c06f480 2023-11-26T07:15:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.293+0000 7f4372935700 1 -- 172.21.15.83:0/168445548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c13ec10 con 0x7f436c06f480 2023-11-26T07:15:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.294+0000 7f4372935700 1 -- 172.21.15.83:0/168445548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c067f60 con 0x7f436c06f480 2023-11-26T07:15:40.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.295+0000 7f4368ff9700 1 -- 172.21.15.83:0/168445548 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f4360011450 con 0x7f436c06f480 2023-11-26T07:15:40.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.295+0000 7f4368ff9700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f435806b230 0x7f435806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:40.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.295+0000 7f4368ff9700 1 -- 172.21.15.83:0/168445548 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f436008b6a0 con 0x7f436c06f480 2023-11-26T07:15:40.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.295+0000 7f436bfff700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f435806b230 0x7f435806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:40.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.296+0000 7f436bfff700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f435806b230 0x7f435806d6f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f435c000c00 tx=0x7f435c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:40.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.300+0000 7f4368ff9700 1 -- 172.21.15.83:0/168445548 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4360056610 con 0x7f436c06f480 2023-11-26T07:15:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.490+0000 7f4372935700 1 -- 172.21.15.83:0/168445548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f436c0ce750 con 0x7f436c06f480 2023-11-26T07:15:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.933+0000 7f4368ff9700 1 -- 172.21.15.83:0/168445548 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4360059c30 con 0x7f436c06f480 2023-11-26T07:15:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 -- 172.21.15.83:0/168445548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f435806b230 msgr2=0x7f435806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f435806b230 0x7f435806d6f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f435c000c00 tx=0x7f435c008030 comp rx=0 tx=0).stop 2023-11-26T07:15:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 -- 172.21.15.83:0/168445548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c06f480 msgr2=0x7f436c0c69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c06f480 0x7f436c0c69a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4360009ef0 tx=0x7f436000ee90 comp rx=0 tx=0).stop 2023-11-26T07:15:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 -- 172.21.15.83:0/168445548 shutdown_connections 2023-11-26T07:15:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f436c144600 0x7f436c0c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f435806b230 0x7f435806d6f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f436c06e830 0x7f436c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 --2- 172.21.15.83:0/168445548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c06f480 0x7f436c0c69a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 -- 172.21.15.83:0/168445548 >> 172.21.15.83:0/168445548 conn(0x7f436c0d0b40 msgr2=0x7f436c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 -- 172.21.15.83:0/168445548 shutdown_connections 2023-11-26T07:15:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:40.935+0000 7f4372935700 1 -- 172.21.15.83:0/168445548 wait complete. 2023-11-26T07:15:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2023-11-26T07:15:40.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:15:41.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.079+0000 7f68aa189700 1 Processor -- start 2023-11-26T07:15:41.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.079+0000 7f68aa189700 1 -- start start 2023-11-26T07:15:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.080+0000 7f68aa189700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f68a40b5da0 0x7f68a40afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.080+0000 7f68aa189700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a40b3c60 con 0x7f68a415e160 2023-11-26T07:15:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.080+0000 7f68aa189700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a40b58c0 con 0x7f68a40746a0 2023-11-26T07:15:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.080+0000 7f68aa189700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a40b4500 con 0x7f68a40b5da0 2023-11-26T07:15:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.080+0000 7f68a37fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f68a415e160 0x7f68a4073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59474/0 (socket says 172.21.15.83:59474) 2023-11-26T07:15:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.080+0000 7f68a37fe700 1 -- 172.21.15.83:0/1294851362 learned_addr learned my addr 172.21.15.83:0/1294851362 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:41.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.080+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680207646 0 0) 0x7f68a40b3c60 con 0x7f68a415e160 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.080+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f688c0035b0 con 0x7f68a415e160 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.080+0000 7f68a3fff700 1 --2- 172.21.15.83:0/1294851362 >> v2:172.21.15.203:3300/0 conn(0x7f68a40b5da0 0x7f68a40afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3717926755 0 0) 0x7f688c0035b0 con 0x7f68a415e160 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 >> v2:172.21.15.203:3300/0 conn(0x7f68a40b5da0 msgr2=0x7f68a40afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 --2- 172.21.15.83:0/1294851362 >> v2:172.21.15.203:3300/0 conn(0x7f68a40b5da0 0x7f68a40afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 >> v1:172.21.15.203:6789/0 conn(0x7f68a40746a0 legacy=0x7f68a406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a406bdc0 con 0x7f68a415e160 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f6898001670 con 0x7f68a415e160 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 >> v1:172.21.15.83:6789/0 conn(0x7f68a415e160 legacy=0x7f68a4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f688c003a00 0x7f688c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688c004850 0x7f688c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:41.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f688c005210 0x7f688c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a40b4500 con 0x7f688c004850 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c00c0b0 con 0x7f688c005210 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c00c270 con 0x7f688c003a00 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a2ffd700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688c004850 0x7f688c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a37fe700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f688c003a00 0x7f688c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a3fff700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f688c005210 0x7f688c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a2ffd700 1 -- 172.21.15.83:0/1294851362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f688c003a00 msgr2=0x7f688c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a2ffd700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f688c003a00 0x7f688c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a2ffd700 1 -- 172.21.15.83:0/1294851362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f688c005210 msgr2=0x7f688c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a2ffd700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f688c005210 0x7f688c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a2ffd700 1 -- 172.21.15.83:0/1294851362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c0035b0 con 0x7f688c004850 2023-11-26T07:15:41.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a37fe700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f688c003a00 0x7f688c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a3fff700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f688c005210 0x7f688c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a3fff700 1 -- 172.21.15.83:0/1294851362 reap_dead start 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a2ffd700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688c004850 0x7f688c004cd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f68940017c0 tx=0x7f688c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f90febb524603eb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68940052d0 con 0x7f688c004850 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6894005430 con 0x7f688c004850 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.081+0000 7f68a27fc700 1 -- 172.21.15.83:0/1294851362 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68940038c0 con 0x7f688c004850 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.082+0000 7f68aa189700 1 -- 172.21.15.83:0/1294851362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688c004850 msgr2=0x7f688c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.082+0000 7f68aa189700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688c004850 0x7f688c004cd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f68940017c0 tx=0x7f688c005880 comp rx=0 tx=0).stop 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.082+0000 7f68aa189700 1 -- 172.21.15.83:0/1294851362 shutdown_connections 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.082+0000 7f68aa189700 1 --2- 172.21.15.83:0/1294851362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688c004850 0x7f688c004cd0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.082+0000 7f68aa189700 1 -- 172.21.15.83:0/1294851362 >> 172.21.15.83:0/1294851362 conn(0x7f68a40d0bf0 msgr2=0x7f68a40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.082+0000 7f68aa189700 1 -- 172.21.15.83:0/1294851362 shutdown_connections 2023-11-26T07:15:41.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.082+0000 7f68aa189700 1 -- 172.21.15.83:0/1294851362 wait complete. 2023-11-26T07:15:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.083+0000 7f68aa189700 1 Processor -- start 2023-11-26T07:15:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.083+0000 7f68aa189700 1 -- start start 2023-11-26T07:15:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.084+0000 7f68aa189700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f688c004850 0x7f68a414a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.084+0000 7f68aa189700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68a414c5b0 0x7f68a414aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.084+0000 7f68aa189700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68a414af90 0x7f68a40b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.084+0000 7f68aa189700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a414c120 con 0x7f68a414c5b0 2023-11-26T07:15:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.084+0000 7f68aa189700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a414a790 con 0x7f688c004850 2023-11-26T07:15:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.084+0000 7f68aa189700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a414b510 con 0x7f68a414af90 2023-11-26T07:15:41.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.084+0000 7f68a2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68a414c5b0 0x7f68a414aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.084+0000 7f68a2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68a414c5b0 0x7f68a414aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39448/0 (socket says 172.21.15.83:39448) 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.084+0000 7f68a2ffd700 1 -- 172.21.15.83:0/2769073539 learned_addr learned my addr 172.21.15.83:0/2769073539 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.084+0000 7f68a37fe700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f688c004850 0x7f68a414a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.084+0000 7f68a3fff700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68a414af90 0x7f68a40b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.085+0000 7f68a2ffd700 1 -- 172.21.15.83:0/2769073539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68a414af90 msgr2=0x7f68a40b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.085+0000 7f68a2ffd700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68a414af90 0x7f68a40b1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.085+0000 7f68a2ffd700 1 -- 172.21.15.83:0/2769073539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f688c004850 msgr2=0x7f68a414a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.085+0000 7f68a2ffd700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f688c004850 0x7f68a414a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.085+0000 7f68a2ffd700 1 -- 172.21.15.83:0/2769073539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a414b7f0 con 0x7f68a414c5b0 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.085+0000 7f68a2ffd700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68a414c5b0 0x7f68a414aa50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6894015040 tx=0x7f6894001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.085+0000 7f688bfff700 1 -- 172.21.15.83:0/2769073539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894001290 con 0x7f68a414c5b0 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.085+0000 7f688bfff700 1 -- 172.21.15.83:0/2769073539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689401d480 con 0x7f68a414c5b0 2023-11-26T07:15:41.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.085+0000 7f68aa189700 1 -- 172.21.15.83:0/2769073539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a414c120 con 0x7f68a414c5b0 2023-11-26T07:15:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.085+0000 7f688bfff700 1 -- 172.21.15.83:0/2769073539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689401d830 con 0x7f68a414c5b0 2023-11-26T07:15:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.085+0000 7f68aa189700 1 -- 172.21.15.83:0/2769073539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a414bdb0 con 0x7f68a414c5b0 2023-11-26T07:15:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.086+0000 7f68aa189700 1 -- 172.21.15.83:0/2769073539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a406e8f0 con 0x7f68a414c5b0 2023-11-26T07:15:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.086+0000 7f688bfff700 1 -- 172.21.15.83:0/2769073539 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f6894019040 con 0x7f68a414c5b0 2023-11-26T07:15:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.087+0000 7f688bfff700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f689006b230 0x7f689006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.087+0000 7f688bfff700 1 -- 172.21.15.83:0/2769073539 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f689408a540 con 0x7f68a414c5b0 2023-11-26T07:15:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.087+0000 7f68a37fe700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f689006b230 0x7f689006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:41.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.087+0000 7f68a37fe700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f689006b230 0x7f689006d6f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6898000c00 tx=0x7f6898007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:41.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.092+0000 7f688bfff700 1 -- 172.21.15.83:0/2769073539 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6894055390 con 0x7f68a414c5b0 2023-11-26T07:15:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.281+0000 7f68aa189700 1 -- 172.21.15.83:0/2769073539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f68a40b51c0 con 0x7f68a414c5b0 2023-11-26T07:15:41.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.935+0000 7f688bfff700 1 -- 172.21.15.83:0/2769073539 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v68) v1 ==== 170+0+0 (secure 0 0 0) 0x7f68940589b0 con 0x7f68a414c5b0 2023-11-26T07:15:41.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.938+0000 7f68aa189700 1 -- 172.21.15.83:0/2769073539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f689006b230 msgr2=0x7f689006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:41.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.938+0000 7f68aa189700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f689006b230 0x7f689006d6f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6898000c00 tx=0x7f6898007040 comp rx=0 tx=0).stop 2023-11-26T07:15:41.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.938+0000 7f68aa189700 1 -- 172.21.15.83:0/2769073539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68a414c5b0 msgr2=0x7f68a414aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:41.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.938+0000 7f68aa189700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68a414c5b0 0x7f68a414aa50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6894015040 tx=0x7f6894001120 comp rx=0 tx=0).stop 2023-11-26T07:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.939+0000 7f68aa189700 1 -- 172.21.15.83:0/2769073539 shutdown_connections 2023-11-26T07:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.939+0000 7f68aa189700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f689006b230 0x7f689006d6f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.939+0000 7f68aa189700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68a414af90 0x7f68a40b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.939+0000 7f68aa189700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f688c004850 0x7f68a414a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.939+0000 7f68aa189700 1 --2- 172.21.15.83:0/2769073539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68a414c5b0 0x7f68a414aa50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.939+0000 7f68aa189700 1 -- 172.21.15.83:0/2769073539 >> 172.21.15.83:0/2769073539 conn(0x7f68a40d0bf0 msgr2=0x7f68a40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.939+0000 7f68aa189700 1 -- 172.21.15.83:0/2769073539 shutdown_connections 2023-11-26T07:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:41.939+0000 7f68aa189700 1 -- 172.21.15.83:0/2769073539 wait complete. 2023-11-26T07:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2023-11-26T07:15:41.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:15:42.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.094+0000 7f0860fb2700 1 Processor -- start 2023-11-26T07:15:42.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.094+0000 7f0860fb2700 1 -- start start 2023-11-26T07:15:42.097 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f0860fb2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f085c15bf50 0x7f085c0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f0860fb2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085c0c6c30 0x7f085c0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f0860fb2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f085c0d0140 con 0x7f085c0ce830 2023-11-26T07:15:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f0860fb2700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f085c15c330 con 0x7f085c15bf50 2023-11-26T07:15:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f0860fb2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0cf3c0 con 0x7f085c0c6c30 2023-11-26T07:15:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f085a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f085c15bf50 0x7f085c0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f085a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f085c15bf50 0x7f085c0ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39454/0 (socket says 172.21.15.83:39454) 2023-11-26T07:15:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f085a59c700 1 -- 172.21.15.83:0/226437088 learned_addr learned my addr 172.21.15.83:0/226437088 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f085a59c700 1 -- 172.21.15.83:0/226437088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085c0c6c30 msgr2=0x7f085c0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f085a59c700 1 --2- 172.21.15.83:0/226437088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085c0c6c30 0x7f085c0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f085a59c700 1 -- 172.21.15.83:0/226437088 >> v1:172.21.15.203:6789/0 conn(0x7f085c0ce830 legacy=0x7f085c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:42.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f085a59c700 1 -- 172.21.15.83:0/226437088 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c0d03e0 con 0x7f085c15bf50 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f085a59c700 1 --2- 172.21.15.83:0/226437088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f085c15bf50 0x7f085c0ce2f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f084c0098f0 tx=0x7f084c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66175d04775be1c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f085959a700 1 -- 172.21.15.83:0/226437088 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c004230 con 0x7f085c15bf50 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f085959a700 1 -- 172.21.15.83:0/226437088 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c004390 con 0x7f085c15bf50 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.095+0000 7f085959a700 1 -- 172.21.15.83:0/226437088 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c0036a0 con 0x7f085c15bf50 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.096+0000 7f0860fb2700 1 -- 172.21.15.83:0/226437088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f085c15bf50 msgr2=0x7f085c0ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.096+0000 7f0860fb2700 1 --2- 172.21.15.83:0/226437088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f085c15bf50 0x7f085c0ce2f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f084c0098f0 tx=0x7f084c009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.096+0000 7f0860fb2700 1 -- 172.21.15.83:0/226437088 shutdown_connections 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.096+0000 7f0860fb2700 1 --2- 172.21.15.83:0/226437088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085c0c6c30 0x7f085c0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.096+0000 7f0860fb2700 1 --2- 172.21.15.83:0/226437088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f085c15bf50 0x7f085c0ce2f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.096+0000 7f0860fb2700 1 -- 172.21.15.83:0/226437088 >> 172.21.15.83:0/226437088 conn(0x7f085c0b3780 msgr2=0x7f085c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.096+0000 7f0860fb2700 1 -- 172.21.15.83:0/226437088 shutdown_connections 2023-11-26T07:15:42.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.096+0000 7f0860fb2700 1 -- 172.21.15.83:0/226437088 wait complete. 2023-11-26T07:15:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.098+0000 7f0860fb2700 1 Processor -- start 2023-11-26T07:15:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.098+0000 7f0860fb2700 1 -- start start 2023-11-26T07:15:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0860fb2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f085c0c6c30 0x7f085c1f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0860fb2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f085c0ce830 0x7f085c0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0860fb2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085c15bf50 0x7f085c1f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0860fb2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c1549b0 con 0x7f085c0ce830 2023-11-26T07:15:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0860fb2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c1f5300 con 0x7f085c0c6c30 2023-11-26T07:15:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0860fb2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0b6640 con 0x7f085c15bf50 2023-11-26T07:15:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0859d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f085c0ce830 0x7f085c0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0859d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f085c0ce830 0x7f085c0b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39462/0 (socket says 172.21.15.83:39462) 2023-11-26T07:15:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0859d9b700 1 -- 172.21.15.83:0/74511667 learned_addr learned my addr 172.21.15.83:0/74511667 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f085a59c700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f085c0c6c30 0x7f085c1f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f085ad9d700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085c15bf50 0x7f085c1f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0859d9b700 1 -- 172.21.15.83:0/74511667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085c15bf50 msgr2=0x7f085c1f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0859d9b700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085c15bf50 0x7f085c1f2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0859d9b700 1 -- 172.21.15.83:0/74511667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f085c0c6c30 msgr2=0x7f085c1f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0859d9b700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f085c0c6c30 0x7f085c1f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0859d9b700 1 -- 172.21.15.83:0/74511667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c009550 con 0x7f085c0ce830 2023-11-26T07:15:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0859d9b700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f085c0ce830 0x7f085c0b2e40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f085000c4a0 tx=0x7f085000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f084affd700 1 -- 172.21.15.83:0/74511667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08500054e0 con 0x7f085c0ce830 2023-11-26T07:15:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f084affd700 1 -- 172.21.15.83:0/74511667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0850003680 con 0x7f085c0ce830 2023-11-26T07:15:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0860fb2700 1 -- 172.21.15.83:0/74511667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085c0b6920 con 0x7f085c0ce830 2023-11-26T07:15:42.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f084affd700 1 -- 172.21.15.83:0/74511667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850003a30 con 0x7f085c0ce830 2023-11-26T07:15:42.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.099+0000 7f0860fb2700 1 -- 172.21.15.83:0/74511667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c0b6ee0 con 0x7f085c0ce830 2023-11-26T07:15:42.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.100+0000 7f0860fb2700 1 -- 172.21.15.83:0/74511667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085c068fb0 con 0x7f085c0ce830 2023-11-26T07:15:42.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.101+0000 7f084affd700 1 -- 172.21.15.83:0/74511667 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f0850003b90 con 0x7f085c0ce830 2023-11-26T07:15:42.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.102+0000 7f084affd700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f083c06b230 0x7f083c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:42.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.102+0000 7f084affd700 1 -- 172.21.15.83:0/74511667 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f0850089350 con 0x7f085c0ce830 2023-11-26T07:15:42.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.102+0000 7f085a59c700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f083c06b230 0x7f083c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:42.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.102+0000 7f085a59c700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f083c06b230 0x7f083c06d6f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f084c009130 tx=0x7f084c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:42.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.106+0000 7f084affd700 1 -- 172.21.15.83:0/74511667 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0850054120 con 0x7f085c0ce830 2023-11-26T07:15:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.338+0000 7f0860fb2700 1 -- 172.21.15.83:0/74511667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f085c15b6e0 con 0x7f085c0ce830 2023-11-26T07:15:42.954 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.953+0000 7f084affd700 1 -- 172.21.15.83:0/74511667 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0850057740 con 0x7f085c0ce830 2023-11-26T07:15:42.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 -- 172.21.15.83:0/74511667 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f083c06b230 msgr2=0x7f083c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:42.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f083c06b230 0x7f083c06d6f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f084c009130 tx=0x7f084c008030 comp rx=0 tx=0).stop 2023-11-26T07:15:42.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 -- 172.21.15.83:0/74511667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f085c0ce830 msgr2=0x7f085c0b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:42.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f085c0ce830 0x7f085c0b2e40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f085000c4a0 tx=0x7f085000c7b0 comp rx=0 tx=0).stop 2023-11-26T07:15:42.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 -- 172.21.15.83:0/74511667 shutdown_connections 2023-11-26T07:15:42.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f083c06b230 0x7f083c06d6f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:42.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085c15bf50 0x7f085c1f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:42.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f085c0c6c30 0x7f085c1f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:42.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 --2- 172.21.15.83:0/74511667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f085c0ce830 0x7f085c0b2e40 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:42.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 -- 172.21.15.83:0/74511667 >> 172.21.15.83:0/74511667 conn(0x7f085c0b3780 msgr2=0x7f085c0cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:42.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 -- 172.21.15.83:0/74511667 shutdown_connections 2023-11-26T07:15:42.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:42.956+0000 7f0860fb2700 1 -- 172.21.15.83:0/74511667 wait complete. 2023-11-26T07:15:42.959 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 10 and data pool 11 2023-11-26T07:15:42.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.096+0000 7fb06cc8a700 1 Processor -- start 2023-11-26T07:15:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.096+0000 7fb06cc8a700 1 -- start start 2023-11-26T07:15:43.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb06cc8a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0680b3370 0x7fb0681604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb06cc8a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb0680d1c70 0x7fb0680d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb06cc8a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0680b22d0 con 0x7fb06815bdd0 2023-11-26T07:15:43.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb06cc8a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0680b2f50 con 0x7fb0680d1c70 2023-11-26T07:15:43.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb06cc8a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680d2640 con 0x7fb0680b3370 2023-11-26T07:15:43.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb066d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb0680d1c70 0x7fb0680d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb06659c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb06815bdd0 0x7fb0680b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59520/0 (socket says 172.21.15.83:59520) 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb066d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb0680d1c70 0x7fb0680d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39484/0 (socket says 172.21.15.83:39484) 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb06659c700 1 -- 172.21.15.83:0/2611987726 learned_addr learned my addr 172.21.15.83:0/2611987726 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb065d9b700 1 --2- 172.21.15.83:0/2611987726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0680b3370 0x7fb0681604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb066d9d700 1 -- 172.21.15.83:0/2611987726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0680b3370 msgr2=0x7fb0681604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb066d9d700 1 --2- 172.21.15.83:0/2611987726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0680b3370 0x7fb0681604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.097+0000 7fb066d9d700 1 -- 172.21.15.83:0/2611987726 >> v1:172.21.15.83:6789/0 conn(0x7fb06815bdd0 legacy=0x7fb0680b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb066d9d700 1 -- 172.21.15.83:0/2611987726 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0680b25b0 con 0x7fb0680d1c70 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06559a700 1 -- 172.21.15.83:0/2611987726 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494945620 0 0) 0x7fb0680b22d0 con 0x7fb06815bdd0 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb066d9d700 1 --2- 172.21.15.83:0/2611987726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0680d1c70 0x7fb0680d2100 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb060009d90 tx=0x7fb06000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99e851ec5f52a25 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06559a700 1 -- 172.21.15.83:0/2611987726 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06000e070 con 0x7fb0680d1c70 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06559a700 1 -- 172.21.15.83:0/2611987726 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06000bbd0 con 0x7fb0680d1c70 2023-11-26T07:15:43.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06559a700 1 -- 172.21.15.83:0/2611987726 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060012430 con 0x7fb0680d1c70 2023-11-26T07:15:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06cc8a700 1 -- 172.21.15.83:0/2611987726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0680d1c70 msgr2=0x7fb0680d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06cc8a700 1 --2- 172.21.15.83:0/2611987726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0680d1c70 0x7fb0680d2100 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb060009d90 tx=0x7fb06000b330 comp rx=0 tx=0).stop 2023-11-26T07:15:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06cc8a700 1 -- 172.21.15.83:0/2611987726 shutdown_connections 2023-11-26T07:15:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06cc8a700 1 --2- 172.21.15.83:0/2611987726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0680d1c70 0x7fb0680d2100 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06cc8a700 1 --2- 172.21.15.83:0/2611987726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0680b3370 0x7fb0681604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06cc8a700 1 -- 172.21.15.83:0/2611987726 >> 172.21.15.83:0/2611987726 conn(0x7fb0680731a0 msgr2=0x7fb06809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06cc8a700 1 -- 172.21.15.83:0/2611987726 shutdown_connections 2023-11-26T07:15:43.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.098+0000 7fb06cc8a700 1 -- 172.21.15.83:0/2611987726 wait complete. 2023-11-26T07:15:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.100+0000 7fb06cc8a700 1 Processor -- start 2023-11-26T07:15:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.100+0000 7fb06cc8a700 1 -- start start 2023-11-26T07:15:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.100+0000 7fb06cc8a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0680b3370 0x7fb068153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.100+0000 7fb06cc8a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0680d1c70 0x7fb0681e6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.100+0000 7fb06cc8a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb06815bdd0 0x7fb0681e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.100+0000 7fb06cc8a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06815e4a0 con 0x7fb0680d1c70 2023-11-26T07:15:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.100+0000 7fb06cc8a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06815e780 con 0x7fb0680b3370 2023-11-26T07:15:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.100+0000 7fb06cc8a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06815ea60 con 0x7fb06815bdd0 2023-11-26T07:15:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb065d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0680d1c70 0x7fb0681e6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb065d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0680d1c70 0x7fb0681e6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39486/0 (socket says 172.21.15.83:39486) 2023-11-26T07:15:43.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb065d9b700 1 -- 172.21.15.83:0/3452813628 learned_addr learned my addr 172.21.15.83:0/3452813628 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb065d9b700 1 -- 172.21.15.83:0/3452813628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb06815bdd0 msgr2=0x7fb0681e66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb065d9b700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb06815bdd0 0x7fb0681e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb065d9b700 1 -- 172.21.15.83:0/3452813628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0680b3370 msgr2=0x7fb068153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb065d9b700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0680b3370 0x7fb068153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb065d9b700 1 -- 172.21.15.83:0/3452813628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb060009a70 con 0x7fb0680d1c70 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb065d9b700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0680d1c70 0x7fb0681e6190 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb05800e810 tx=0x7fb05800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb05effd700 1 -- 172.21.15.83:0/3452813628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058013610 con 0x7fb0680d1c70 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb05effd700 1 -- 172.21.15.83:0/3452813628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb058013770 con 0x7fb0680d1c70 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb06cc8a700 1 -- 172.21.15.83:0/3452813628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06815ed40 con 0x7fb0680d1c70 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb05effd700 1 -- 172.21.15.83:0/3452813628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058013b20 con 0x7fb0680d1c70 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.101+0000 7fb06cc8a700 1 -- 172.21.15.83:0/3452813628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb068154060 con 0x7fb0680d1c70 2023-11-26T07:15:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.102+0000 7fb06cc8a700 1 -- 172.21.15.83:0/3452813628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb068067f60 con 0x7fb0680d1c70 2023-11-26T07:15:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.103+0000 7fb05effd700 1 -- 172.21.15.83:0/3452813628 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fb058013c80 con 0x7fb0680d1c70 2023-11-26T07:15:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.103+0000 7fb05effd700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb04806b230 0x7fb04806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.103+0000 7fb05effd700 1 -- 172.21.15.83:0/3452813628 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0580898e0 con 0x7fb0680d1c70 2023-11-26T07:15:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.103+0000 7fb06659c700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb04806b230 0x7fb04806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.103+0000 7fb06659c700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb04806b230 0x7fb04806d6f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb050001610 tx=0x7fb0500079f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.107+0000 7fb05effd700 1 -- 172.21.15.83:0/3452813628 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb058054480 con 0x7fb0680d1c70 2023-11-26T07:15:43.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.297+0000 7fb06cc8a700 1 -- 172.21.15.83:0/3452813628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb06815f260 con 0x7fb0680d1c70 2023-11-26T07:15:43.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.299+0000 7fb05effd700 1 -- 172.21.15.83:0/3452813628 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fb06815f260 con 0x7fb0680d1c70 2023-11-26T07:15:43.300 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:43.301 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:15:42.339345+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:15:40.508908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:15:41.299931+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:43.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 -- 172.21.15.83:0/3452813628 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb04806b230 msgr2=0x7fb04806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb04806b230 0x7fb04806d6f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb050001610 tx=0x7fb0500079f0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 -- 172.21.15.83:0/3452813628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0680d1c70 msgr2=0x7fb0681e6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0680d1c70 0x7fb0681e6190 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb05800e810 tx=0x7fb05800ebd0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 -- 172.21.15.83:0/3452813628 shutdown_connections 2023-11-26T07:15:43.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fb04806b230 0x7fb04806d6f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb06815bdd0 0x7fb0681e66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0680b3370 0x7fb068153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 --2- 172.21.15.83:0/3452813628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0680d1c70 0x7fb0681e6190 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 -- 172.21.15.83:0/3452813628 >> 172.21.15.83:0/3452813628 conn(0x7fb0680731a0 msgr2=0x7fb06806ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:43.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 -- 172.21.15.83:0/3452813628 shutdown_connections 2023-11-26T07:15:43.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.301+0000 7fb06cc8a700 1 -- 172.21.15.83:0/3452813628 wait complete. 2023-11-26T07:15:43.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.434+0000 7ff57e939700 1 Processor -- start 2023-11-26T07:15:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.434+0000 7ff57e939700 1 -- start start 2023-11-26T07:15:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.434+0000 7ff57e939700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff57815e110 0x7ff5780b1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57e939700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff5780d2210 0x7ff5780b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57e939700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5780ce4a0 con 0x7ff5780b1570 2023-11-26T07:15:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57e939700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5780cefe0 con 0x7ff5780d2210 2023-11-26T07:15:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57e939700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5780cf470 con 0x7ff57815e110 2023-11-26T07:15:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57ced6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff5780d2210 0x7ff5780b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57ced6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff5780d2210 0x7ff5780b1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38948/0 (socket says 172.21.15.83:38948) 2023-11-26T07:15:43.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff577fff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff57815e110 0x7ff5780b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57ced6700 1 -- 172.21.15.83:0/3040905173 learned_addr learned my addr 172.21.15.83:0/3040905173 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff5777fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7ff5780b1570 0x7ff5780d1b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:44152/0 (socket says 172.21.15.83:44152) 2023-11-26T07:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57ced6700 1 -- 172.21.15.83:0/3040905173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff57815e110 msgr2=0x7ff5780b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57ced6700 1 --2- 172.21.15.83:0/3040905173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff57815e110 0x7ff5780b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57ced6700 1 -- 172.21.15.83:0/3040905173 >> v1:172.21.15.203:6789/0 conn(0x7ff5780b1570 legacy=0x7ff5780d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57ced6700 1 -- 172.21.15.83:0/3040905173 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5780ce6d0 con 0x7ff5780d2210 2023-11-26T07:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff577fff700 1 --2- 172.21.15.83:0/3040905173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff57815e110 0x7ff5780b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.435+0000 7ff57ced6700 1 --2- 172.21.15.83:0/3040905173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5780d2210 0x7ff5780b1cb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff570009e50 tx=0x7ff57000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0be2506c3ebb973 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.436+0000 7ff576ffd700 1 -- 172.21.15.83:0/3040905173 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57000e070 con 0x7ff5780d2210 2023-11-26T07:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.436+0000 7ff576ffd700 1 -- 172.21.15.83:0/3040905173 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57000b910 con 0x7ff5780d2210 2023-11-26T07:15:43.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.436+0000 7ff576ffd700 1 -- 172.21.15.83:0/3040905173 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57000bcc0 con 0x7ff5780d2210 2023-11-26T07:15:43.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.436+0000 7ff57e939700 1 -- 172.21.15.83:0/3040905173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5780d2210 msgr2=0x7ff5780b1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.436+0000 7ff57e939700 1 --2- 172.21.15.83:0/3040905173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5780d2210 0x7ff5780b1cb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff570009e50 tx=0x7ff57000b330 comp rx=0 tx=0).stop 2023-11-26T07:15:43.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.436+0000 7ff57e939700 1 -- 172.21.15.83:0/3040905173 shutdown_connections 2023-11-26T07:15:43.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.436+0000 7ff57e939700 1 --2- 172.21.15.83:0/3040905173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5780d2210 0x7ff5780b1cb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.436+0000 7ff57e939700 1 --2- 172.21.15.83:0/3040905173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff57815e110 0x7ff5780b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.436+0000 7ff57e939700 1 -- 172.21.15.83:0/3040905173 >> 172.21.15.83:0/3040905173 conn(0x7ff5780730a0 msgr2=0x7ff57809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:43.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.436+0000 7ff57e939700 1 -- 172.21.15.83:0/3040905173 shutdown_connections 2023-11-26T07:15:43.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.436+0000 7ff57e939700 1 -- 172.21.15.83:0/3040905173 wait complete. 2023-11-26T07:15:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.437+0000 7ff57e939700 1 Processor -- start 2023-11-26T07:15:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.438+0000 7ff57e939700 1 -- start start 2023-11-26T07:15:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.438+0000 7ff57e939700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5780b1570 0x7ff578151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.438+0000 7ff57e939700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5780d2210 0x7ff5780ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.438+0000 7ff57e939700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff57815e110 0x7ff5780cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.438+0000 7ff57e939700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5780b6fe0 con 0x7ff5780d2210 2023-11-26T07:15:43.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.438+0000 7ff57e939700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5781521b0 con 0x7ff57815e110 2023-11-26T07:15:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.438+0000 7ff57e939700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5780c9160 con 0x7ff5780b1570 2023-11-26T07:15:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.438+0000 7ff5777fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5780d2210 0x7ff5780ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.438+0000 7ff5777fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5780d2210 0x7ff5780ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39498/0 (socket says 172.21.15.83:39498) 2023-11-26T07:15:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.438+0000 7ff5777fe700 1 -- 172.21.15.83:0/3120626673 learned_addr learned my addr 172.21.15.83:0/3120626673 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff57ced6700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff57815e110 0x7ff5780cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff577fff700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5780b1570 0x7ff578151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff5777fe700 1 -- 172.21.15.83:0/3120626673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5780b1570 msgr2=0x7ff578151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff5777fe700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5780b1570 0x7ff578151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff5777fe700 1 -- 172.21.15.83:0/3120626673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff57815e110 msgr2=0x7ff5780cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff5777fe700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff57815e110 0x7ff5780cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff5777fe700 1 -- 172.21.15.83:0/3120626673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5680094a0 con 0x7ff5780d2210 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff57ced6700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff57815e110 0x7ff5780cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff577fff700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5780b1570 0x7ff578151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff5777fe700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5780d2210 0x7ff5780ca200 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00c4f0 tx=0x7ff56c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff574ff9700 1 -- 172.21.15.83:0/3120626673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c005cb0 con 0x7ff5780d2210 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff574ff9700 1 -- 172.21.15.83:0/3120626673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c005e10 con 0x7ff5780d2210 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff57e939700 1 -- 172.21.15.83:0/3120626673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff570009b30 con 0x7ff5780d2210 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff574ff9700 1 -- 172.21.15.83:0/3120626673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c0072d0 con 0x7ff5780d2210 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.439+0000 7ff57e939700 1 -- 172.21.15.83:0/3120626673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5780c9720 con 0x7ff5780d2210 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.440+0000 7ff57e939700 1 -- 172.21.15.83:0/3120626673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5780751e0 con 0x7ff5780d2210 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.441+0000 7ff574ff9700 1 -- 172.21.15.83:0/3120626673 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7ff56c007430 con 0x7ff5780d2210 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.441+0000 7ff574ff9700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff56406b230 0x7ff56406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.441+0000 7ff574ff9700 1 -- 172.21.15.83:0/3120626673 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff56c088ff0 con 0x7ff5780d2210 2023-11-26T07:15:43.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.441+0000 7ff577fff700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff56406b230 0x7ff56406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.441+0000 7ff577fff700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff56406b230 0x7ff56406d6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff568009c50 tx=0x7ff568008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:43.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.446+0000 7ff574ff9700 1 -- 172.21.15.83:0/3120626673 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff56c0bbe90 con 0x7ff5780d2210 2023-11-26T07:15:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.639+0000 7ff57e939700 1 -- 172.21.15.83:0/3120626673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5780b1a90 con 0x7ff5780d2210 2023-11-26T07:15:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.640+0000 7ff574ff9700 1 -- 172.21.15.83:0/3120626673 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16556 (secure 0 0 0) 0x7ff56c017020 con 0x7ff5780d2210 2023-11-26T07:15:43.642 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:43.642 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:15:43.538242+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:15:40.508908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:15:41.299931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:43.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 -- 172.21.15.83:0/3120626673 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff56406b230 msgr2=0x7ff56406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff56406b230 0x7ff56406d6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff568009c50 tx=0x7ff568008040 comp rx=0 tx=0).stop 2023-11-26T07:15:43.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 -- 172.21.15.83:0/3120626673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5780d2210 msgr2=0x7ff5780ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5780d2210 0x7ff5780ca200 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00c4f0 tx=0x7ff56c00c8b0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 -- 172.21.15.83:0/3120626673 shutdown_connections 2023-11-26T07:15:43.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff57815e110 0x7ff5780cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7ff56406b230 0x7ff56406d6f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff5780b1570 0x7ff578151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 --2- 172.21.15.83:0/3120626673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5780d2210 0x7ff5780ca200 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 -- 172.21.15.83:0/3120626673 >> 172.21.15.83:0/3120626673 conn(0x7ff5780730a0 msgr2=0x7ff578161860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:43.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 -- 172.21.15.83:0/3120626673 shutdown_connections 2023-11-26T07:15:43.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.642+0000 7ff57e939700 1 -- 172.21.15.83:0/3120626673 wait complete. 2023-11-26T07:15:43.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:15:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.778+0000 7f332407f700 1 Processor -- start 2023-11-26T07:15:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.778+0000 7f332407f700 1 -- start start 2023-11-26T07:15:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f332407f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f331c0ceb40 0x7f331c06bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f332407f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331c06ba50 con 0x7f331c15e080 2023-11-26T07:15:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f332407f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331c06b120 con 0x7f331c0b4310 2023-11-26T07:15:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f332407f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c0d1200 con 0x7f331c0ceb40 2023-11-26T07:15:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f332261c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f331c0ceb40 0x7f331c06bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f3321e1b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f331c15e080 0x7f331c0d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59546/0 (socket says 172.21.15.83:59546) 2023-11-26T07:15:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f3321e1b700 1 -- 172.21.15.83:0/2705013643 learned_addr learned my addr 172.21.15.83:0/2705013643 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f3320e19700 1 -- 172.21.15.83:0/2705013643 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577463729 0 0) 0x7f331c06ba50 con 0x7f331c15e080 2023-11-26T07:15:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f3320e19700 1 -- 172.21.15.83:0/2705013643 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33080035b0 con 0x7f331c15e080 2023-11-26T07:15:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f332261c700 1 -- 172.21.15.83:0/2705013643 >> v1:172.21.15.203:6789/0 conn(0x7f331c0b4310 legacy=0x7f331c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f332261c700 1 -- 172.21.15.83:0/2705013643 >> v1:172.21.15.83:6789/0 conn(0x7f331c15e080 legacy=0x7f331c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f332261c700 1 -- 172.21.15.83:0/2705013643 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c06b400 con 0x7f331c0ceb40 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f332261c700 1 --2- 172.21.15.83:0/2705013643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c0ceb40 0x7f331c06bee0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3314009d90 tx=0x7f331400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b06faa5624781b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.779+0000 7f3320e19700 1 -- 172.21.15.83:0/2705013643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331400e070 con 0x7f331c0ceb40 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.780+0000 7f3320e19700 1 -- 172.21.15.83:0/2705013643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331400bb20 con 0x7f331c0ceb40 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.780+0000 7f3320e19700 1 -- 172.21.15.83:0/2705013643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3314012430 con 0x7f331c0ceb40 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.780+0000 7f332407f700 1 -- 172.21.15.83:0/2705013643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c0ceb40 msgr2=0x7f331c06bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.780+0000 7f332407f700 1 --2- 172.21.15.83:0/2705013643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c0ceb40 0x7f331c06bee0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3314009d90 tx=0x7f331400b330 comp rx=0 tx=0).stop 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.780+0000 7f332407f700 1 -- 172.21.15.83:0/2705013643 shutdown_connections 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.780+0000 7f332407f700 1 --2- 172.21.15.83:0/2705013643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c0ceb40 0x7f331c06bee0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.780+0000 7f332407f700 1 -- 172.21.15.83:0/2705013643 >> 172.21.15.83:0/2705013643 conn(0x7f331c0b3780 msgr2=0x7f331c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.780+0000 7f332407f700 1 -- 172.21.15.83:0/2705013643 shutdown_connections 2023-11-26T07:15:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.780+0000 7f332407f700 1 -- 172.21.15.83:0/2705013643 wait complete. 2023-11-26T07:15:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.781+0000 7f332407f700 1 Processor -- start 2023-11-26T07:15:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.781+0000 7f332407f700 1 -- start start 2023-11-26T07:15:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f332407f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c0b4310 0x7f331c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f332407f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f331c0ceb40 0x7f331c15a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f332407f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f331c15e080 0x7f331c160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f332407f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c15a6f0 con 0x7f331c0b4310 2023-11-26T07:15:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f332407f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c15f5a0 con 0x7f331c0ceb40 2023-11-26T07:15:43.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f332407f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331c1612b0 con 0x7f331c15e080 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f3321e1b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c0b4310 0x7f331c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f3321e1b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c0b4310 0x7f331c15ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39512/0 (socket says 172.21.15.83:39512) 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f3321e1b700 1 -- 172.21.15.83:0/4186126479 learned_addr learned my addr 172.21.15.83:0/4186126479 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f3321e1b700 1 -- 172.21.15.83:0/4186126479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f331c15e080 msgr2=0x7f331c160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f3321e1b700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f331c15e080 0x7f331c160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f3321e1b700 1 -- 172.21.15.83:0/4186126479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f331c0ceb40 msgr2=0x7f331c15a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f3321e1b700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f331c0ceb40 0x7f331c15a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.782+0000 7f3321e1b700 1 -- 172.21.15.83:0/4186126479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3314009a70 con 0x7f331c0b4310 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.783+0000 7f3321e1b700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c0b4310 0x7f331c15ece0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3318001eb0 tx=0x7f331800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.783+0000 7f33127fc700 1 -- 172.21.15.83:0/4186126479 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318005d70 con 0x7f331c0b4310 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.783+0000 7f33127fc700 1 -- 172.21.15.83:0/4186126479 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33180070d0 con 0x7f331c0b4310 2023-11-26T07:15:43.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.783+0000 7f332407f700 1 -- 172.21.15.83:0/4186126479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331c15f2c0 con 0x7f331c0b4310 2023-11-26T07:15:43.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.783+0000 7f33127fc700 1 -- 172.21.15.83:0/4186126479 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318007460 con 0x7f331c0b4310 2023-11-26T07:15:43.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.783+0000 7f332407f700 1 -- 172.21.15.83:0/4186126479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331c0c7a20 con 0x7f331c0b4310 2023-11-26T07:15:43.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.784+0000 7f332407f700 1 -- 172.21.15.83:0/4186126479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331c068fb0 con 0x7f331c0b4310 2023-11-26T07:15:43.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.784+0000 7f33127fc700 1 -- 172.21.15.83:0/4186126479 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f33180075c0 con 0x7f331c0b4310 2023-11-26T07:15:43.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.784+0000 7f33127fc700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f330006b230 0x7f330006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:43.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.785+0000 7f33127fc700 1 -- 172.21.15.83:0/4186126479 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3318088fb0 con 0x7f331c0b4310 2023-11-26T07:15:43.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.785+0000 7f332161a700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f330006b230 0x7f330006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:43.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.785+0000 7f332161a700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f330006b230 0x7f330006d6f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f330c000f60 tx=0x7f330c007a40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:43.789+0000 7f33127fc700 1 -- 172.21.15.83:0/4186126479 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3318053ce0 con 0x7f331c0b4310 2023-11-26T07:15:44.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.028+0000 7f332407f700 1 -- 172.21.15.83:0/4186126479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f331c0b7ab0 con 0x7f331c0b4310 2023-11-26T07:15:44.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.966+0000 7f33127fc700 1 -- 172.21.15.83:0/4186126479 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7f33180170b0 con 0x7f331c0b4310 2023-11-26T07:15:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 -- 172.21.15.83:0/4186126479 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f330006b230 msgr2=0x7f330006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f330006b230 0x7f330006d6f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f330c000f60 tx=0x7f330c007a40 comp rx=0 tx=0).stop 2023-11-26T07:15:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 -- 172.21.15.83:0/4186126479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c0b4310 msgr2=0x7f331c15ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c0b4310 0x7f331c15ece0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3318001eb0 tx=0x7f331800c860 comp rx=0 tx=0).stop 2023-11-26T07:15:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 -- 172.21.15.83:0/4186126479 shutdown_connections 2023-11-26T07:15:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f330006b230 0x7f330006d6f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:44.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f331c15e080 0x7f331c160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c0b4310 0x7f331c15ece0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 --2- 172.21.15.83:0/4186126479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f331c0ceb40 0x7f331c15a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 -- 172.21.15.83:0/4186126479 >> 172.21.15.83:0/4186126479 conn(0x7f331c0b3780 msgr2=0x7f331c0cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 -- 172.21.15.83:0/4186126479 shutdown_connections 2023-11-26T07:15:44.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:44.969+0000 7f332407f700 1 -- 172.21.15.83:0/4186126479 wait complete. 2023-11-26T07:15:44.986 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:15:44.986 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:15:45.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.115+0000 7feab5fc4700 1 Processor -- start 2023-11-26T07:15:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.115+0000 7feab5fc4700 1 -- start start 2023-11-26T07:15:45.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.116+0000 7feab5fc4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feab015a620 0x7feab00ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:45.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.116+0000 7feab5fc4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab006ecf0 0x7feab00b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:45.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.116+0000 7feab5fc4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab00b2f20 con 0x7feab00cc540 2023-11-26T07:15:45.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.116+0000 7feab5fc4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab00ce690 con 0x7feab015a620 2023-11-26T07:15:45.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.116+0000 7feab5fc4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab015aa00 con 0x7feab006ecf0 2023-11-26T07:15:45.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.116+0000 7feaaffff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab006ecf0 0x7feab00b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.116+0000 7feaaf7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feab015a620 0x7feab00ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.116+0000 7feaaeffd700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7feab00cc540 0x7feab006e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:44180/0 (socket says 172.21.15.83:44180) 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.116+0000 7feaaeffd700 1 -- 172.21.15.83:0/2420886763 learned_addr learned my addr 172.21.15.83:0/2420886763 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.116+0000 7feaaf7fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feab015a620 0x7feab00ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38972/0 (socket says 172.21.15.83:38972) 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.116+0000 7feaaffff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab006ecf0 0x7feab00b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:49982/0 (socket says 172.21.15.83:49982) 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feaaffff700 1 -- 172.21.15.83:0/2420886763 >> v2:172.21.15.203:3300/0 conn(0x7feab015a620 msgr2=0x7feab00ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feaaffff700 1 --2- 172.21.15.83:0/2420886763 >> v2:172.21.15.203:3300/0 conn(0x7feab015a620 0x7feab00ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feaaffff700 1 -- 172.21.15.83:0/2420886763 >> v1:172.21.15.203:6789/0 conn(0x7feab00cc540 legacy=0x7feab006e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feaaffff700 1 -- 172.21.15.83:0/2420886763 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab00ce960 con 0x7feab006ecf0 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feaaf7fe700 1 --2- 172.21.15.83:0/2420886763 >> v2:172.21.15.203:3300/0 conn(0x7feab015a620 0x7feab00ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feaaffff700 1 --2- 172.21.15.83:0/2420886763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab006ecf0 0x7feab00b3350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feaa800aea0 tx=0x7feaa800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acbd12552f0b89f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:45.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feaae7fc700 1 -- 172.21.15.83:0/2420886763 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa800f070 con 0x7feab006ecf0 2023-11-26T07:15:45.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feaae7fc700 1 -- 172.21.15.83:0/2420886763 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa800a2e0 con 0x7feab006ecf0 2023-11-26T07:15:45.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feaae7fc700 1 -- 172.21.15.83:0/2420886763 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa800cb50 con 0x7feab006ecf0 2023-11-26T07:15:45.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feab5fc4700 1 -- 172.21.15.83:0/2420886763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab006ecf0 msgr2=0x7feab00b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:45.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feab5fc4700 1 --2- 172.21.15.83:0/2420886763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab006ecf0 0x7feab00b3350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feaa800aea0 tx=0x7feaa800c330 comp rx=0 tx=0).stop 2023-11-26T07:15:45.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feab5fc4700 1 -- 172.21.15.83:0/2420886763 shutdown_connections 2023-11-26T07:15:45.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feab5fc4700 1 --2- 172.21.15.83:0/2420886763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab006ecf0 0x7feab00b3350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:45.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feab5fc4700 1 --2- 172.21.15.83:0/2420886763 >> v2:172.21.15.203:3300/0 conn(0x7feab015a620 0x7feab00ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:45.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feab5fc4700 1 -- 172.21.15.83:0/2420886763 >> 172.21.15.83:0/2420886763 conn(0x7feab00b2050 msgr2=0x7feab00b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:45.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.117+0000 7feab5fc4700 1 -- 172.21.15.83:0/2420886763 shutdown_connections 2023-11-26T07:15:45.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.118+0000 7feab5fc4700 1 -- 172.21.15.83:0/2420886763 wait complete. 2023-11-26T07:15:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.119+0000 7feab5fc4700 1 Processor -- start 2023-11-26T07:15:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.119+0000 7feab5fc4700 1 -- start start 2023-11-26T07:15:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feab5fc4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feab006ecf0 0x7feab0142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:45.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feab5fc4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab00cc540 0x7feab00aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feab5fc4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab015a620 0x7feab01556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feab5fc4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0143400 con 0x7feab00cc540 2023-11-26T07:15:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feab5fc4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0152d20 con 0x7feab006ecf0 2023-11-26T07:15:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feab5fc4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab00af2e0 con 0x7feab015a620 2023-11-26T07:15:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab00cc540 0x7feab00aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab00cc540 0x7feab00aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39528/0 (socket says 172.21.15.83:39528) 2023-11-26T07:15:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaeffd700 1 -- 172.21.15.83:0/3608051483 learned_addr learned my addr 172.21.15.83:0/3608051483 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaf7fe700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feab006ecf0 0x7feab0142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaffff700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab015a620 0x7feab01556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaeffd700 1 -- 172.21.15.83:0/3608051483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab015a620 msgr2=0x7feab01556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:45.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaeffd700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab015a620 0x7feab01556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaeffd700 1 -- 172.21.15.83:0/3608051483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feab006ecf0 msgr2=0x7feab0142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaeffd700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feab006ecf0 0x7feab0142ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaeffd700 1 -- 172.21.15.83:0/3608051483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980094a0 con 0x7feab00cc540 2023-11-26T07:15:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaffff700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab015a620 0x7feab01556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaaeffd700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab00cc540 0x7feab00aeda0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7feaa000c490 tx=0x7feaa000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaa7fff700 1 -- 172.21.15.83:0/3608051483 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0005d70 con 0x7feab00cc540 2023-11-26T07:15:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaa7fff700 1 -- 172.21.15.83:0/3608051483 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa00070d0 con 0x7feab00cc540 2023-11-26T07:15:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feab5fc4700 1 -- 172.21.15.83:0/3608051483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa800ab80 con 0x7feab00cc540 2023-11-26T07:15:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feaa7fff700 1 -- 172.21.15.83:0/3608051483 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0007460 con 0x7feab00cc540 2023-11-26T07:15:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.120+0000 7feab5fc4700 1 -- 172.21.15.83:0/3608051483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab014a440 con 0x7feab00cc540 2023-11-26T07:15:45.126 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.121+0000 7feab5fc4700 1 -- 172.21.15.83:0/3608051483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab0068d90 con 0x7feab00cc540 2023-11-26T07:15:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.122+0000 7feaa7fff700 1 -- 172.21.15.83:0/3608051483 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7feaa00075c0 con 0x7feab00cc540 2023-11-26T07:15:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.122+0000 7feaa7fff700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea9006b230 0x7fea9006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.122+0000 7feaa7fff700 1 -- 172.21.15.83:0/3608051483 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feaa0088f30 con 0x7feab00cc540 2023-11-26T07:15:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.122+0000 7feaaf7fe700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea9006b230 0x7fea9006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:45.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.123+0000 7feaaf7fe700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea9006b230 0x7fea9006d6f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fea98009cd0 tx=0x7fea98009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:45.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.127+0000 7feaa7fff700 1 -- 172.21.15.83:0/3608051483 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feaa0053bd0 con 0x7feab00cc540 2023-11-26T07:15:45.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.361+0000 7feab5fc4700 1 -- 172.21.15.83:0/3608051483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7feab0068fb0 con 0x7feab00cc540 2023-11-26T07:15:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.967+0000 7feaa7fff700 1 -- 172.21.15.83:0/3608051483 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7feaa0017020 con 0x7feab00cc540 2023-11-26T07:15:45.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 -- 172.21.15.83:0/3608051483 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea9006b230 msgr2=0x7fea9006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:45.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea9006b230 0x7fea9006d6f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fea98009cd0 tx=0x7fea98009210 comp rx=0 tx=0).stop 2023-11-26T07:15:45.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 -- 172.21.15.83:0/3608051483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab00cc540 msgr2=0x7feab00aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:45.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab00cc540 0x7feab00aeda0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7feaa000c490 tx=0x7feaa000c7a0 comp rx=0 tx=0).stop 2023-11-26T07:15:45.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 -- 172.21.15.83:0/3608051483 shutdown_connections 2023-11-26T07:15:45.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fea9006b230 0x7fea9006d6f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:45.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feab015a620 0x7feab01556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:45.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feab006ecf0 0x7feab0142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:45.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 --2- 172.21.15.83:0/3608051483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab00cc540 0x7feab00aeda0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:45.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 -- 172.21.15.83:0/3608051483 >> 172.21.15.83:0/3608051483 conn(0x7feab00b2050 msgr2=0x7feab014db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:45.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 -- 172.21.15.83:0/3608051483 shutdown_connections 2023-11-26T07:15:45.974 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:45.971+0000 7feab5fc4700 1 -- 172.21.15.83:0/3608051483 wait complete. 2023-11-26T07:15:45.988 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:15:45.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:15:46.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.115+0000 7fdbe615c700 1 Processor -- start 2023-11-26T07:15:46.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.115+0000 7fdbe615c700 1 -- start start 2023-11-26T07:15:46.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbe615c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdbe00af4d0 0x7fdbe00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbe615c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe00ae430 con 0x7fdbe015e080 2023-11-26T07:15:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbe615c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe00ae710 con 0x7fdbe006c710 2023-11-26T07:15:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbe615c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe00af0b0 con 0x7fdbe00af4d0 2023-11-26T07:15:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbdf7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdbe015e080 0x7fdbe006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59574/0 (socket says 172.21.15.83:59574) 2023-11-26T07:15:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbdf7fe700 1 -- 172.21.15.83:0/4236143486 learned_addr learned my addr 172.21.15.83:0/4236143486 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbdffff700 1 --2- 172.21.15.83:0/4236143486 >> v2:172.21.15.203:3300/0 conn(0x7fdbe00af4d0 0x7fdbe00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885042448 0 0) 0x7fdbe00ae430 con 0x7fdbe015e080 2023-11-26T07:15:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbcc0035b0 con 0x7fdbe015e080 2023-11-26T07:15:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 738222480 0 0) 0x7fdbcc0035b0 con 0x7fdbe015e080 2023-11-26T07:15:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 >> v2:172.21.15.203:3300/0 conn(0x7fdbe00af4d0 msgr2=0x7fdbe00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.116+0000 7fdbde7fc700 1 --2- 172.21.15.83:0/4236143486 >> v2:172.21.15.203:3300/0 conn(0x7fdbe00af4d0 0x7fdbe00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 >> v1:172.21.15.203:6789/0 conn(0x7fdbe006c710 legacy=0x7fdbe006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe00ae9f0 con 0x7fdbe015e080 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fdbc8001670 con 0x7fdbe015e080 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 >> v1:172.21.15.83:6789/0 conn(0x7fdbe015e080 legacy=0x7fdbe006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc003a00 0x7fdbcc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbcc004850 0x7fdbcc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbcc005210 0x7fdbcc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00af0b0 con 0x7fdbcc003a00 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc00c0b0 con 0x7fdbcc004850 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc00c270 con 0x7fdbcc005210 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbdf7fe700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc003a00 0x7fdbcc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbdeffd700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbcc004850 0x7fdbcc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbdffff700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbcc005210 0x7fdbcc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbdf7fe700 1 -- 172.21.15.83:0/4236143486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbcc005210 msgr2=0x7fdbcc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbdf7fe700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbcc005210 0x7fdbcc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbdf7fe700 1 -- 172.21.15.83:0/4236143486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbcc004850 msgr2=0x7fdbcc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbdf7fe700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbcc004850 0x7fdbcc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbdf7fe700 1 -- 172.21.15.83:0/4236143486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc0035b0 con 0x7fdbcc003a00 2023-11-26T07:15:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbdffff700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbcc005210 0x7fdbcc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbdffff700 1 -- 172.21.15.83:0/4236143486 reap_dead start 2023-11-26T07:15:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbdf7fe700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc003a00 0x7fdbcc003e20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800c240 tx=0x7fdbcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd38624d2c9d3440 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8007c10 con 0x7fdbcc003a00 2023-11-26T07:15:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc800cde0 con 0x7fdbcc003a00 2023-11-26T07:15:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.117+0000 7fdbde7fc700 1 -- 172.21.15.83:0/4236143486 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8005ba0 con 0x7fdbcc003a00 2023-11-26T07:15:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.118+0000 7fdbe615c700 1 -- 172.21.15.83:0/4236143486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc003a00 msgr2=0x7fdbcc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.118+0000 7fdbe615c700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc003a00 0x7fdbcc003e20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800c240 tx=0x7fdbcc005880 comp rx=0 tx=0).stop 2023-11-26T07:15:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.118+0000 7fdbe615c700 1 -- 172.21.15.83:0/4236143486 shutdown_connections 2023-11-26T07:15:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.118+0000 7fdbe615c700 1 --2- 172.21.15.83:0/4236143486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbcc003a00 0x7fdbcc003e20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.118+0000 7fdbe615c700 1 -- 172.21.15.83:0/4236143486 >> 172.21.15.83:0/4236143486 conn(0x7fdbe0071b90 msgr2=0x7fdbe0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.118+0000 7fdbe615c700 1 -- 172.21.15.83:0/4236143486 shutdown_connections 2023-11-26T07:15:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.118+0000 7fdbe615c700 1 -- 172.21.15.83:0/4236143486 wait complete. 2023-11-26T07:15:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.119+0000 7fdbe615c700 1 Processor -- start 2023-11-26T07:15:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.119+0000 7fdbe615c700 1 -- start start 2023-11-26T07:15:46.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbe615c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbcc003a00 0x7fdbe01f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbe615c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbe01592b0 0x7fdbe0159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbe615c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe00ce440 0x7fdbe00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbe615c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00ce020 con 0x7fdbe00ce440 2023-11-26T07:15:46.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbe615c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00cd6c0 con 0x7fdbcc003a00 2023-11-26T07:15:46.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbe615c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00cd960 con 0x7fdbe01592b0 2023-11-26T07:15:46.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbdffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe00ce440 0x7fdbe00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbdffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe00ce440 0x7fdbe00ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39550/0 (socket says 172.21.15.83:39550) 2023-11-26T07:15:46.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbdffff700 1 -- 172.21.15.83:0/578406253 learned_addr learned my addr 172.21.15.83:0/578406253 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:46.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbdeffd700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbe01592b0 0x7fdbe0159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbdf7fe700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbcc003a00 0x7fdbe01f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbdffff700 1 -- 172.21.15.83:0/578406253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbe01592b0 msgr2=0x7fdbe0159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbdffff700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbe01592b0 0x7fdbe0159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbdffff700 1 -- 172.21.15.83:0/578406253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbcc003a00 msgr2=0x7fdbe01f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbdffff700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbcc003a00 0x7fdbe01f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.120+0000 7fdbdffff700 1 -- 172.21.15.83:0/578406253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe00cdc40 con 0x7fdbe00ce440 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.121+0000 7fdbdffff700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe00ce440 0x7fdbe00ce8f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fdbe006d350 tx=0x7fdbd800f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.121+0000 7fdbd7fff700 1 -- 172.21.15.83:0/578406253 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800fe60 con 0x7fdbe00ce440 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.121+0000 7fdbd7fff700 1 -- 172.21.15.83:0/578406253 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd8016420 con 0x7fdbe00ce440 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.121+0000 7fdbd7fff700 1 -- 172.21.15.83:0/578406253 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80167b0 con 0x7fdbe00ce440 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.121+0000 7fdbe615c700 1 -- 172.21.15.83:0/578406253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe01f3480 con 0x7fdbe00ce440 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.121+0000 7fdbe615c700 1 -- 172.21.15.83:0/578406253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe0159c70 con 0x7fdbe00ce440 2023-11-26T07:15:46.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.122+0000 7fdbe615c700 1 -- 172.21.15.83:0/578406253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe00753f0 con 0x7fdbe00ce440 2023-11-26T07:15:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.122+0000 7fdbd7fff700 1 -- 172.21.15.83:0/578406253 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fdbd8016910 con 0x7fdbe00ce440 2023-11-26T07:15:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.122+0000 7fdbd7fff700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdbc006b230 0x7fdbc006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.123+0000 7fdbd7fff700 1 -- 172.21.15.83:0/578406253 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbd808a630 con 0x7fdbe00ce440 2023-11-26T07:15:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.123+0000 7fdbdf7fe700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdbc006b230 0x7fdbc006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.123+0000 7fdbdf7fe700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdbc006b230 0x7fdbc006d6f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8000c00 tx=0x7fdbc8007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:46.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.127+0000 7fdbd7fff700 1 -- 172.21.15.83:0/578406253 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdbd8055250 con 0x7fdbe00ce440 2023-11-26T07:15:46.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.359+0000 7fdbe615c700 1 -- 172.21.15.83:0/578406253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbe00ce160 con 0x7fdbe00ce440 2023-11-26T07:15:46.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.359+0000 7fdbd7fff700 1 -- 172.21.15.83:0/578406253 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fdbe00ce160 con 0x7fdbe00ce440 2023-11-26T07:15:46.361 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:46.361 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5420,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2163151849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2163151849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2163151849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1539108478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1539108478},{"type":"v1","addr":"172.21.15.203:6835","nonce":1539108478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1387613510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1387613510},{"type":"v1","addr":"172.21.15.203:6834","nonce":1387613510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:15:42.339653+0000","modified":"2023-11-26T07:15:45.361821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1809465302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1809465302},{"type":"v1","addr":"172.21.15.83:6836","nonce":1809465302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:15:46.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 -- 172.21.15.83:0/578406253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdbc006b230 msgr2=0x7fdbc006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdbc006b230 0x7fdbc006d6f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8000c00 tx=0x7fdbc8007000 comp rx=0 tx=0).stop 2023-11-26T07:15:46.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 -- 172.21.15.83:0/578406253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe00ce440 msgr2=0x7fdbe00ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe00ce440 0x7fdbe00ce8f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fdbe006d350 tx=0x7fdbd800f430 comp rx=0 tx=0).stop 2023-11-26T07:15:46.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 -- 172.21.15.83:0/578406253 shutdown_connections 2023-11-26T07:15:46.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbe00ce440 0x7fdbe00ce8f0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdbcc003a00 0x7fdbe01f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fdbc006b230 0x7fdbc006d6f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 --2- 172.21.15.83:0/578406253 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdbe01592b0 0x7fdbe0159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 -- 172.21.15.83:0/578406253 >> 172.21.15.83:0/578406253 conn(0x7fdbe0071b90 msgr2=0x7fdbe00b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 -- 172.21.15.83:0/578406253 shutdown_connections 2023-11-26T07:15:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.361+0000 7fdbe615c700 1 -- 172.21.15.83:0/578406253 wait complete. 2023-11-26T07:15:46.364 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 46 2023-11-26T07:15:46.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.494+0000 7f3fb8617700 1 Processor -- start 2023-11-26T07:15:46.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.494+0000 7f3fb8617700 1 -- start start 2023-11-26T07:15:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb8617700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3fb00adf00 0x7f3fb006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb8617700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3fb006c540 0x7f3fb00b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb8617700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb00af950 con 0x7f3fb01446a0 2023-11-26T07:15:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb8617700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb006c9f0 con 0x7f3fb00adf00 2023-11-26T07:15:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb8617700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb0144a80 con 0x7f3fb006c540 2023-11-26T07:15:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb5bb2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3fb00adf00 0x7f3fb006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb5bb2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3fb00adf00 0x7f3fb006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39556/0 (socket says 172.21.15.83:39556) 2023-11-26T07:15:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb5bb2700 1 -- 172.21.15.83:0/2522591069 learned_addr learned my addr 172.21.15.83:0/2522591069 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb5bb2700 1 -- 172.21.15.83:0/2522591069 >> v2:172.21.15.203:3300/0 conn(0x7f3fb006c540 msgr2=0x7f3fb00b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb6bb4700 1 --2- 172.21.15.83:0/2522591069 >> v2:172.21.15.203:3300/0 conn(0x7f3fb006c540 0x7f3fb00b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb5bb2700 1 --2- 172.21.15.83:0/2522591069 >> v2:172.21.15.203:3300/0 conn(0x7f3fb006c540 0x7f3fb00b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb5bb2700 1 -- 172.21.15.83:0/2522591069 >> v1:172.21.15.203:6789/0 conn(0x7f3fb01446a0 legacy=0x7f3fb00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb5bb2700 1 -- 172.21.15.83:0/2522591069 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb00afc30 con 0x7f3fb00adf00 2023-11-26T07:15:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.495+0000 7f3fb6bb4700 1 --2- 172.21.15.83:0/2522591069 >> v2:172.21.15.203:3300/0 conn(0x7f3fb006c540 0x7f3fb00b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb5bb2700 1 --2- 172.21.15.83:0/2522591069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00adf00 0x7f3fb006c000 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00aedf0 tx=0x7f3fac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c575bfe4554e653 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb53b1700 1 -- 172.21.15.83:0/2522591069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac004230 con 0x7f3fb00adf00 2023-11-26T07:15:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb53b1700 1 -- 172.21.15.83:0/2522591069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fac004390 con 0x7f3fb00adf00 2023-11-26T07:15:46.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb53b1700 1 -- 172.21.15.83:0/2522591069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac0036a0 con 0x7f3fb00adf00 2023-11-26T07:15:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb8617700 1 -- 172.21.15.83:0/2522591069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00adf00 msgr2=0x7f3fb006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb8617700 1 --2- 172.21.15.83:0/2522591069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00adf00 0x7f3fb006c000 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00aedf0 tx=0x7f3fac009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb8617700 1 -- 172.21.15.83:0/2522591069 shutdown_connections 2023-11-26T07:15:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb8617700 1 --2- 172.21.15.83:0/2522591069 >> v2:172.21.15.203:3300/0 conn(0x7f3fb006c540 0x7f3fb00b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb8617700 1 --2- 172.21.15.83:0/2522591069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00adf00 0x7f3fb006c000 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb8617700 1 -- 172.21.15.83:0/2522591069 >> 172.21.15.83:0/2522591069 conn(0x7f3fb0071b90 msgr2=0x7f3fb0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb8617700 1 -- 172.21.15.83:0/2522591069 shutdown_connections 2023-11-26T07:15:46.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.496+0000 7f3fb8617700 1 -- 172.21.15.83:0/2522591069 wait complete. 2023-11-26T07:15:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.497+0000 7f3fb8617700 1 Processor -- start 2023-11-26T07:15:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb8617700 1 -- start start 2023-11-26T07:15:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb8617700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fb006c540 0x7f3fb0139150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb8617700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00adf00 0x7f3fb00749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb8617700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb01446a0 0x7f3fb013dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb8617700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb013e1d0 con 0x7f3fb00adf00 2023-11-26T07:15:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb8617700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb015d8c0 con 0x7f3fb01446a0 2023-11-26T07:15:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb8617700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb013a2b0 con 0x7f3fb006c540 2023-11-26T07:15:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb5bb2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00adf00 0x7f3fb00749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb5bb2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00adf00 0x7f3fb00749a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39562/0 (socket says 172.21.15.83:39562) 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb5bb2700 1 -- 172.21.15.83:0/2857953617 learned_addr learned my addr 172.21.15.83:0/2857953617 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb6bb4700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb01446a0 0x7f3fb013dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.498+0000 7f3fb63b3700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fb006c540 0x7f3fb0139150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.499+0000 7f3fb5bb2700 1 -- 172.21.15.83:0/2857953617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fb006c540 msgr2=0x7f3fb0139150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.499+0000 7f3fb5bb2700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fb006c540 0x7f3fb0139150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.499+0000 7f3fb5bb2700 1 -- 172.21.15.83:0/2857953617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb01446a0 msgr2=0x7f3fb013dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.499+0000 7f3fb5bb2700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb01446a0 0x7f3fb013dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.499+0000 7f3fb5bb2700 1 -- 172.21.15.83:0/2857953617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac009550 con 0x7f3fb00adf00 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.499+0000 7f3fb5bb2700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00adf00 0x7f3fb00749a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009870 tx=0x7f3fac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.499+0000 7f3fa6ffd700 1 -- 172.21.15.83:0/2857953617 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac011630 con 0x7f3fb00adf00 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.499+0000 7f3fa6ffd700 1 -- 172.21.15.83:0/2857953617 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fac011790 con 0x7f3fb00adf00 2023-11-26T07:15:46.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.499+0000 7f3fb8617700 1 -- 172.21.15.83:0/2857953617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb013ba30 con 0x7f3fb00adf00 2023-11-26T07:15:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.499+0000 7f3fa6ffd700 1 -- 172.21.15.83:0/2857953617 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac011b40 con 0x7f3fb00adf00 2023-11-26T07:15:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.499+0000 7f3fb8617700 1 -- 172.21.15.83:0/2857953617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb0139a10 con 0x7f3fb00adf00 2023-11-26T07:15:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.500+0000 7f3fb8617700 1 -- 172.21.15.83:0/2857953617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb00753f0 con 0x7f3fb00adf00 2023-11-26T07:15:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.500+0000 7f3fa6ffd700 1 -- 172.21.15.83:0/2857953617 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f3fac021070 con 0x7f3fb00adf00 2023-11-26T07:15:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.501+0000 7f3fa6ffd700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3f9406b230 0x7f3f9406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.501+0000 7f3fa6ffd700 1 -- 172.21.15.83:0/2857953617 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fac08b490 con 0x7f3fb00adf00 2023-11-26T07:15:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.501+0000 7f3fb63b3700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3f9406b230 0x7f3f9406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.501+0000 7f3fb63b3700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3f9406b230 0x7f3f9406d6f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3fb015b1f0 tx=0x7f3fa0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:46.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.505+0000 7f3fa6ffd700 1 -- 172.21.15.83:0/2857953617 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3fac0561c0 con 0x7f3fb00adf00 2023-11-26T07:15:46.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.694+0000 7f3fb8617700 1 -- 172.21.15.83:0/2857953617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fb0067f60 con 0x7f3fb00adf00 2023-11-26T07:15:46.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.695+0000 7f3fa6ffd700 1 -- 172.21.15.83:0/2857953617 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16556 (secure 0 0 0) 0x7f3fac02d3a0 con 0x7f3fb00adf00 2023-11-26T07:15:46.697 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:46.697 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:15:43.538242+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:15:40.508908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:15:41.299931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:46.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 -- 172.21.15.83:0/2857953617 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3f9406b230 msgr2=0x7f3f9406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3f9406b230 0x7f3f9406d6f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3fb015b1f0 tx=0x7f3fa0008040 comp rx=0 tx=0).stop 2023-11-26T07:15:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 -- 172.21.15.83:0/2857953617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00adf00 msgr2=0x7f3fb00749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00adf00 0x7f3fb00749a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009870 tx=0x7f3fac004750 comp rx=0 tx=0).stop 2023-11-26T07:15:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 -- 172.21.15.83:0/2857953617 shutdown_connections 2023-11-26T07:15:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fb01446a0 0x7f3fb013dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f3f9406b230 0x7f3f9406d6f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fb006c540 0x7f3fb0139150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 --2- 172.21.15.83:0/2857953617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fb00adf00 0x7f3fb00749a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 -- 172.21.15.83:0/2857953617 >> 172.21.15.83:0/2857953617 conn(0x7f3fb0071b90 msgr2=0x7f3fb00b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 -- 172.21.15.83:0/2857953617 shutdown_connections 2023-11-26T07:15:46.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.697+0000 7f3fb8617700 1 -- 172.21.15.83:0/2857953617 wait complete. 2023-11-26T07:15:46.711 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:15:46.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:15:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.830+0000 7f3287bf9700 1 Processor -- start 2023-11-26T07:15:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.830+0000 7f3287bf9700 1 -- start start 2023-11-26T07:15:46.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.830+0000 7f3287bf9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f328015e080 0x7f32800b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3287bf9700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32800ae430 con 0x7f32800af4d0 2023-11-26T07:15:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3287bf9700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32800ae710 con 0x7f328006c710 2023-11-26T07:15:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3287bf9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32800af0b0 con 0x7f328015e080 2023-11-26T07:15:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3285995700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f328015e080 0x7f32800b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3285194700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f32800af4d0 0x7f328006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59610/0 (socket says 172.21.15.83:59610) 2023-11-26T07:15:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3285194700 1 -- 172.21.15.83:0/656199775 learned_addr learned my addr 172.21.15.83:0/656199775 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3284993700 1 -- 172.21.15.83:0/656199775 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083468458 0 0) 0x7f32800ae430 con 0x7f32800af4d0 2023-11-26T07:15:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3284993700 1 -- 172.21.15.83:0/656199775 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f326c0035b0 con 0x7f32800af4d0 2023-11-26T07:15:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3285995700 1 -- 172.21.15.83:0/656199775 >> v1:172.21.15.203:6789/0 conn(0x7f328006c710 legacy=0x7f328006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3285995700 1 -- 172.21.15.83:0/656199775 >> v1:172.21.15.83:6789/0 conn(0x7f32800af4d0 legacy=0x7f328006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3285995700 1 -- 172.21.15.83:0/656199775 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32800ae9f0 con 0x7f328015e080 2023-11-26T07:15:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3285995700 1 --2- 172.21.15.83:0/656199775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f328015e080 0x7f32800b5050 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3270009840 tx=0x7f3270009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf4bca23b482b6d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3284993700 1 -- 172.21.15.83:0/656199775 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270004230 con 0x7f328015e080 2023-11-26T07:15:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3284993700 1 -- 172.21.15.83:0/656199775 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3270004390 con 0x7f328015e080 2023-11-26T07:15:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.831+0000 7f3284993700 1 -- 172.21.15.83:0/656199775 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32700036c0 con 0x7f328015e080 2023-11-26T07:15:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.832+0000 7f3287bf9700 1 -- 172.21.15.83:0/656199775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f328015e080 msgr2=0x7f32800b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.832+0000 7f3287bf9700 1 --2- 172.21.15.83:0/656199775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f328015e080 0x7f32800b5050 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f3270009840 tx=0x7f3270009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.832+0000 7f3287bf9700 1 -- 172.21.15.83:0/656199775 shutdown_connections 2023-11-26T07:15:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.832+0000 7f3287bf9700 1 --2- 172.21.15.83:0/656199775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f328015e080 0x7f32800b5050 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.832+0000 7f3287bf9700 1 -- 172.21.15.83:0/656199775 >> 172.21.15.83:0/656199775 conn(0x7f3280071b90 msgr2=0x7f3280071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.832+0000 7f3287bf9700 1 -- 172.21.15.83:0/656199775 shutdown_connections 2023-11-26T07:15:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.832+0000 7f3287bf9700 1 -- 172.21.15.83:0/656199775 wait complete. 2023-11-26T07:15:46.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.833+0000 7f3287bf9700 1 Processor -- start 2023-11-26T07:15:46.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.833+0000 7f3287bf9700 1 -- start start 2023-11-26T07:15:46.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3287bf9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f328006c710 0x7f3280157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3287bf9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32800af4d0 0x7f3280151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3287bf9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f328015e080 0x7f3280153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3287bf9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328015ee00 con 0x7f32800af4d0 2023-11-26T07:15:46.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3287bf9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3280157790 con 0x7f328006c710 2023-11-26T07:15:46.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3287bf9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3280154570 con 0x7f328015e080 2023-11-26T07:15:46.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3285194700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32800af4d0 0x7f3280151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3285194700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32800af4d0 0x7f3280151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39576/0 (socket says 172.21.15.83:39576) 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3285194700 1 -- 172.21.15.83:0/3559621629 learned_addr learned my addr 172.21.15.83:0/3559621629 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3285995700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f328006c710 0x7f3280157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3286196700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f328015e080 0x7f3280153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3285194700 1 -- 172.21.15.83:0/3559621629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f328015e080 msgr2=0x7f3280153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3285194700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f328015e080 0x7f3280153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3285194700 1 -- 172.21.15.83:0/3559621629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f328006c710 msgr2=0x7f3280157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3285194700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f328006c710 0x7f3280157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3285194700 1 -- 172.21.15.83:0/3559621629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32700094a0 con 0x7f32800af4d0 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3285194700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32800af4d0 0x7f3280151600 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c4d0 tx=0x7f327c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f32767fc700 1 -- 172.21.15.83:0/3559621629 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c005d70 con 0x7f32800af4d0 2023-11-26T07:15:46.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f32767fc700 1 -- 172.21.15.83:0/3559621629 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c0070d0 con 0x7f32800af4d0 2023-11-26T07:15:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.834+0000 7f3287bf9700 1 -- 172.21.15.83:0/3559621629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328015e460 con 0x7f32800af4d0 2023-11-26T07:15:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.835+0000 7f32767fc700 1 -- 172.21.15.83:0/3559621629 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c007480 con 0x7f32800af4d0 2023-11-26T07:15:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.835+0000 7f3287bf9700 1 -- 172.21.15.83:0/3559621629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328015ea20 con 0x7f32800af4d0 2023-11-26T07:15:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.835+0000 7f3287bf9700 1 -- 172.21.15.83:0/3559621629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32800753f0 con 0x7f32800af4d0 2023-11-26T07:15:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.836+0000 7f32767fc700 1 -- 172.21.15.83:0/3559621629 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f327c0075e0 con 0x7f32800af4d0 2023-11-26T07:15:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.836+0000 7f32767fc700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f326406b230 0x7f326406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.836+0000 7f32767fc700 1 -- 172.21.15.83:0/3559621629 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f327c089050 con 0x7f32800af4d0 2023-11-26T07:15:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.836+0000 7f3285995700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f326406b230 0x7f326406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.837+0000 7f3285995700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f326406b230 0x7f326406d6f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f32700090b0 tx=0x7f3270010000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:46.841+0000 7f32767fc700 1 -- 172.21.15.83:0/3559621629 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f327c053db0 con 0x7f32800af4d0 2023-11-26T07:15:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.031+0000 7f3287bf9700 1 -- 172.21.15.83:0/3559621629 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32800b05a0 con 0x7f326406b230 2023-11-26T07:15:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.039+0000 7f32767fc700 1 -- 172.21.15.83:0/3559621629 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289412 (secure 0 0 0) 0x7f32700036a0 con 0x7f326406b230 2023-11-26T07:15:47.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:47.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 -- 172.21.15.83:0/3559621629 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f326406b230 msgr2=0x7f326406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f326406b230 0x7f326406d6f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f32700090b0 tx=0x7f3270010000 comp rx=0 tx=0).stop 2023-11-26T07:15:47.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 -- 172.21.15.83:0/3559621629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32800af4d0 msgr2=0x7f3280151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32800af4d0 0x7f3280151600 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c4d0 tx=0x7f327c00c890 comp rx=0 tx=0).stop 2023-11-26T07:15:47.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 -- 172.21.15.83:0/3559621629 shutdown_connections 2023-11-26T07:15:47.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f326406b230 0x7f326406d6f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f328015e080 0x7f3280153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f328006c710 0x7f3280157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 --2- 172.21.15.83:0/3559621629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32800af4d0 0x7f3280151600 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 -- 172.21.15.83:0/3559621629 >> 172.21.15.83:0/3559621629 conn(0x7f3280071b90 msgr2=0x7f32800b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 -- 172.21.15.83:0/3559621629 shutdown_connections 2023-11-26T07:15:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.042+0000 7f3287bf9700 1 -- 172.21.15.83:0/3559621629 wait complete. 2023-11-26T07:15:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:15:47.055 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2023-11-26T07:15:46.363024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224040,"kb_used_data":7880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750680,"statfs":{"total":773094113280,"available":772864696320,"internally_reserved":0,"allocated":8069120,"data_stored":4052116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.429202"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545255+0000","last_change":"2023-11-26T07:15:42.552509+0000","last_active":"2023-11-26T07:15:43.545255+0000","last_peered":"2023-11-26T07:15:43.545255+0000","last_clean":"2023-11-26T07:15:43.545255+0000","last_became_active":"2023-11-26T07:15:42.552199+0000","last_became_peered":"2023-11-26T07:15:42.552199+0000","last_unstale":"2023-11-26T07:15:43.545255+0000","last_undegraded":"2023-11-26T07:15:43.545255+0000","last_fullsized":"2023-11-26T07:15:43.545255+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:52:22.331524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621617+0000","last_change":"2023-11-26T07:15:42.557781+0000","last_active":"2023-11-26T07:15:43.621617+0000","last_peered":"2023-11-26T07:15:43.621617+0000","last_clean":"2023-11-26T07:15:43.621617+0000","last_became_active":"2023-11-26T07:15:42.556908+0000","last_became_peered":"2023-11-26T07:15:42.556908+0000","last_unstale":"2023-11-26T07:15:43.621617+0000","last_undegraded":"2023-11-26T07:15:43.621617+0000","last_fullsized":"2023-11-26T07:15:43.621617+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:05:32.379392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620481+0000","last_change":"2023-11-26T07:15:42.557805+0000","last_active":"2023-11-26T07:15:43.620481+0000","last_peered":"2023-11-26T07:15:43.620481+0000","last_clean":"2023-11-26T07:15:43.620481+0000","last_became_active":"2023-11-26T07:15:42.557543+0000","last_became_peered":"2023-11-26T07:15:42.557543+0000","last_unstale":"2023-11-26T07:15:43.620481+0000","last_undegraded":"2023-11-26T07:15:43.620481+0000","last_fullsized":"2023-11-26T07:15:43.620481+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:28:59.498960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621299+0000","last_change":"2023-11-26T07:15:42.562855+0000","last_active":"2023-11-26T07:15:43.621299+0000","last_peered":"2023-11-26T07:15:43.621299+0000","last_clean":"2023-11-26T07:15:43.621299+0000","last_became_active":"2023-11-26T07:15:42.562588+0000","last_became_peered":"2023-11-26T07:15:42.562588+0000","last_unstale":"2023-11-26T07:15:43.621299+0000","last_undegraded":"2023-11-26T07:15:43.621299+0000","last_fullsized":"2023-11-26T07:15:43.621299+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:49:36.390962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620439+0000","last_change":"2023-11-26T07:15:42.548063+0000","last_active":"2023-11-26T07:15:43.620439+0000","last_peered":"2023-11-26T07:15:43.620439+0000","last_clean":"2023-11-26T07:15:43.620439+0000","last_became_active":"2023-11-26T07:15:42.547808+0000","last_became_peered":"2023-11-26T07:15:42.547808+0000","last_unstale":"2023-11-26T07:15:43.620439+0000","last_undegraded":"2023-11-26T07:15:43.620439+0000","last_fullsized":"2023-11-26T07:15:43.620439+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:43:01.257824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621311+0000","last_change":"2023-11-26T07:15:42.557161+0000","last_active":"2023-11-26T07:15:43.621311+0000","last_peered":"2023-11-26T07:15:43.621311+0000","last_clean":"2023-11-26T07:15:43.621311+0000","last_became_active":"2023-11-26T07:15:42.556877+0000","last_became_peered":"2023-11-26T07:15:42.556877+0000","last_unstale":"2023-11-26T07:15:43.621311+0000","last_undegraded":"2023-11-26T07:15:43.621311+0000","last_fullsized":"2023-11-26T07:15:43.621311+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:06.810126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620496+0000","last_change":"2023-11-26T07:15:42.564899+0000","last_active":"2023-11-26T07:15:43.620496+0000","last_peered":"2023-11-26T07:15:43.620496+0000","last_clean":"2023-11-26T07:15:43.620496+0000","last_became_active":"2023-11-26T07:15:42.564475+0000","last_became_peered":"2023-11-26T07:15:42.564475+0000","last_unstale":"2023-11-26T07:15:43.620496+0000","last_undegraded":"2023-11-26T07:15:43.620496+0000","last_fullsized":"2023-11-26T07:15:43.620496+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:45.536813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621041+0000","last_change":"2023-11-26T07:15:42.560477+0000","last_active":"2023-11-26T07:15:43.621041+0000","last_peered":"2023-11-26T07:15:43.621041+0000","last_clean":"2023-11-26T07:15:43.621041+0000","last_became_active":"2023-11-26T07:15:42.557362+0000","last_became_peered":"2023-11-26T07:15:42.557362+0000","last_unstale":"2023-11-26T07:15:43.621041+0000","last_undegraded":"2023-11-26T07:15:43.621041+0000","last_fullsized":"2023-11-26T07:15:43.621041+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:08:18.050397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620548+0000","last_change":"2023-11-26T07:15:42.556207+0000","last_active":"2023-11-26T07:15:43.620548+0000","last_peered":"2023-11-26T07:15:43.620548+0000","last_clean":"2023-11-26T07:15:43.620548+0000","last_became_active":"2023-11-26T07:15:42.555875+0000","last_became_peered":"2023-11-26T07:15:42.555875+0000","last_unstale":"2023-11-26T07:15:43.620548+0000","last_undegraded":"2023-11-26T07:15:43.620548+0000","last_fullsized":"2023-11-26T07:15:43.620548+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:16:06.747989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543562+0000","last_change":"2023-11-26T07:15:42.552732+0000","last_active":"2023-11-26T07:15:43.543562+0000","last_peered":"2023-11-26T07:15:43.543562+0000","last_clean":"2023-11-26T07:15:43.543562+0000","last_became_active":"2023-11-26T07:15:42.551602+0000","last_became_peered":"2023-11-26T07:15:42.551602+0000","last_unstale":"2023-11-26T07:15:43.543562+0000","last_undegraded":"2023-11-26T07:15:43.543562+0000","last_fullsized":"2023-11-26T07:15:43.543562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:57.146358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545141+0000","last_change":"2023-11-26T07:15:42.555561+0000","last_active":"2023-11-26T07:15:43.545141+0000","last_peered":"2023-11-26T07:15:43.545141+0000","last_clean":"2023-11-26T07:15:43.545141+0000","last_became_active":"2023-11-26T07:15:42.555261+0000","last_became_peered":"2023-11-26T07:15:42.555261+0000","last_unstale":"2023-11-26T07:15:43.545141+0000","last_undegraded":"2023-11-26T07:15:43.545141+0000","last_fullsized":"2023-11-26T07:15:43.545141+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:56:36.376641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621452+0000","last_change":"2023-11-26T07:15:42.558281+0000","last_active":"2023-11-26T07:15:43.621452+0000","last_peered":"2023-11-26T07:15:43.621452+0000","last_clean":"2023-11-26T07:15:43.621452+0000","last_became_active":"2023-11-26T07:15:42.557978+0000","last_became_peered":"2023-11-26T07:15:42.557978+0000","last_unstale":"2023-11-26T07:15:43.621452+0000","last_undegraded":"2023-11-26T07:15:43.621452+0000","last_fullsized":"2023-11-26T07:15:43.621452+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:02.439521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542168+0000","last_change":"2023-11-26T07:15:42.560447+0000","last_active":"2023-11-26T07:15:43.542168+0000","last_peered":"2023-11-26T07:15:43.542168+0000","last_clean":"2023-11-26T07:15:43.542168+0000","last_became_active":"2023-11-26T07:15:42.560172+0000","last_became_peered":"2023-11-26T07:15:42.560172+0000","last_unstale":"2023-11-26T07:15:43.542168+0000","last_undegraded":"2023-11-26T07:15:43.542168+0000","last_fullsized":"2023-11-26T07:15:43.542168+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:25:13.533544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620901+0000","last_change":"2023-11-26T07:15:42.563807+0000","last_active":"2023-11-26T07:15:43.620901+0000","last_peered":"2023-11-26T07:15:43.620901+0000","last_clean":"2023-11-26T07:15:43.620901+0000","last_became_active":"2023-11-26T07:15:42.563540+0000","last_became_peered":"2023-11-26T07:15:42.563540+0000","last_unstale":"2023-11-26T07:15:43.620901+0000","last_undegraded":"2023-11-26T07:15:43.620901+0000","last_fullsized":"2023-11-26T07:15:43.620901+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:01:26.363020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542015+0000","last_change":"2023-11-26T07:15:42.553786+0000","last_active":"2023-11-26T07:15:43.542015+0000","last_peered":"2023-11-26T07:15:43.542015+0000","last_clean":"2023-11-26T07:15:43.542015+0000","last_became_active":"2023-11-26T07:15:42.553509+0000","last_became_peered":"2023-11-26T07:15:42.553509+0000","last_unstale":"2023-11-26T07:15:43.542015+0000","last_undegraded":"2023-11-26T07:15:43.542015+0000","last_fullsized":"2023-11-26T07:15:43.542015+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:45.536813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543232+0000","last_change":"2023-11-26T07:15:42.554794+0000","last_active":"2023-11-26T07:15:43.543232+0000","last_peered":"2023-11-26T07:15:43.543232+0000","last_clean":"2023-11-26T07:15:43.543232+0000","last_became_active":"2023-11-26T07:15:42.554394+0000","last_became_peered":"2023-11-26T07:15:42.554394+0000","last_unstale":"2023-11-26T07:15:43.543232+0000","last_undegraded":"2023-11-26T07:15:43.543232+0000","last_fullsized":"2023-11-26T07:15:43.543232+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:06.810126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620933+0000","last_change":"2023-11-26T07:15:42.555104+0000","last_active":"2023-11-26T07:15:43.620933+0000","last_peered":"2023-11-26T07:15:43.620933+0000","last_clean":"2023-11-26T07:15:43.620933+0000","last_became_active":"2023-11-26T07:15:42.554486+0000","last_became_peered":"2023-11-26T07:15:42.554486+0000","last_unstale":"2023-11-26T07:15:43.620933+0000","last_undegraded":"2023-11-26T07:15:43.620933+0000","last_fullsized":"2023-11-26T07:15:43.620933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:56.979892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621628+0000","last_change":"2023-11-26T07:15:42.562465+0000","last_active":"2023-11-26T07:15:43.621628+0000","last_peered":"2023-11-26T07:15:43.621628+0000","last_clean":"2023-11-26T07:15:43.621628+0000","last_became_active":"2023-11-26T07:15:42.562131+0000","last_became_peered":"2023-11-26T07:15:42.562131+0000","last_unstale":"2023-11-26T07:15:43.621628+0000","last_undegraded":"2023-11-26T07:15:43.621628+0000","last_fullsized":"2023-11-26T07:15:43.621628+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:45:31.391520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621047+0000","last_change":"2023-11-26T07:15:42.557482+0000","last_active":"2023-11-26T07:15:43.621047+0000","last_peered":"2023-11-26T07:15:43.621047+0000","last_clean":"2023-11-26T07:15:43.621047+0000","last_became_active":"2023-11-26T07:15:42.557202+0000","last_became_peered":"2023-11-26T07:15:42.557202+0000","last_unstale":"2023-11-26T07:15:43.621047+0000","last_undegraded":"2023-11-26T07:15:43.621047+0000","last_fullsized":"2023-11-26T07:15:43.621047+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:21.644505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542011+0000","last_change":"2023-11-26T07:15:42.561478+0000","last_active":"2023-11-26T07:15:43.542011+0000","last_peered":"2023-11-26T07:15:43.542011+0000","last_clean":"2023-11-26T07:15:43.542011+0000","last_became_active":"2023-11-26T07:15:42.561228+0000","last_became_peered":"2023-11-26T07:15:42.561228+0000","last_unstale":"2023-11-26T07:15:43.542011+0000","last_undegraded":"2023-11-26T07:15:43.542011+0000","last_fullsized":"2023-11-26T07:15:43.542011+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:03:17.847732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543682+0000","last_change":"2023-11-26T07:15:42.552816+0000","last_active":"2023-11-26T07:15:43.543682+0000","last_peered":"2023-11-26T07:15:43.543682+0000","last_clean":"2023-11-26T07:15:43.543682+0000","last_became_active":"2023-11-26T07:15:42.552562+0000","last_became_peered":"2023-11-26T07:15:42.552562+0000","last_unstale":"2023-11-26T07:15:43.543682+0000","last_undegraded":"2023-11-26T07:15:43.543682+0000","last_fullsized":"2023-11-26T07:15:43.543682+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:05:32.379392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543694+0000","last_change":"2023-11-26T07:15:42.553637+0000","last_active":"2023-11-26T07:15:43.543694+0000","last_peered":"2023-11-26T07:15:43.543694+0000","last_clean":"2023-11-26T07:15:43.543694+0000","last_became_active":"2023-11-26T07:15:42.553362+0000","last_became_peered":"2023-11-26T07:15:42.553362+0000","last_unstale":"2023-11-26T07:15:43.543694+0000","last_undegraded":"2023-11-26T07:15:43.543694+0000","last_fullsized":"2023-11-26T07:15:43.543694+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:09:21.961069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.544910+0000","last_change":"2023-11-26T07:15:42.559524+0000","last_active":"2023-11-26T07:15:43.544910+0000","last_peered":"2023-11-26T07:15:43.544910+0000","last_clean":"2023-11-26T07:15:43.544910+0000","last_became_active":"2023-11-26T07:15:42.559241+0000","last_became_peered":"2023-11-26T07:15:42.559241+0000","last_unstale":"2023-11-26T07:15:43.544910+0000","last_undegraded":"2023-11-26T07:15:43.544910+0000","last_fullsized":"2023-11-26T07:15:43.544910+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:30:40.488087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.544718+0000","last_change":"2023-11-26T07:15:42.554290+0000","last_active":"2023-11-26T07:15:43.544718+0000","last_peered":"2023-11-26T07:15:43.544718+0000","last_clean":"2023-11-26T07:15:43.544718+0000","last_became_active":"2023-11-26T07:15:42.553994+0000","last_became_peered":"2023-11-26T07:15:42.553994+0000","last_unstale":"2023-11-26T07:15:43.544718+0000","last_undegraded":"2023-11-26T07:15:43.544718+0000","last_fullsized":"2023-11-26T07:15:43.544718+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:45:47.838341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543617+0000","last_change":"2023-11-26T07:15:42.546241+0000","last_active":"2023-11-26T07:15:43.543617+0000","last_peered":"2023-11-26T07:15:43.543617+0000","last_clean":"2023-11-26T07:15:43.543617+0000","last_became_active":"2023-11-26T07:15:42.545953+0000","last_became_peered":"2023-11-26T07:15:42.545953+0000","last_unstale":"2023-11-26T07:15:43.543617+0000","last_undegraded":"2023-11-26T07:15:43.543617+0000","last_fullsized":"2023-11-26T07:15:43.543617+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:48:24.866444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621043+0000","last_change":"2023-11-26T07:15:42.563104+0000","last_active":"2023-11-26T07:15:43.621043+0000","last_peered":"2023-11-26T07:15:43.621043+0000","last_clean":"2023-11-26T07:15:43.621043+0000","last_became_active":"2023-11-26T07:15:42.562801+0000","last_became_peered":"2023-11-26T07:15:42.562801+0000","last_unstale":"2023-11-26T07:15:43.621043+0000","last_undegraded":"2023-11-26T07:15:43.621043+0000","last_fullsized":"2023-11-26T07:15:43.621043+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:42:42.350198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620600+0000","last_change":"2023-11-26T07:15:42.558604+0000","last_active":"2023-11-26T07:15:43.620600+0000","last_peered":"2023-11-26T07:15:43.620600+0000","last_clean":"2023-11-26T07:15:43.620600+0000","last_became_active":"2023-11-26T07:15:42.558348+0000","last_became_peered":"2023-11-26T07:15:42.558348+0000","last_unstale":"2023-11-26T07:15:43.620600+0000","last_undegraded":"2023-11-26T07:15:43.620600+0000","last_fullsized":"2023-11-26T07:15:43.620600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:09:21.919185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543997+0000","last_change":"2023-11-26T07:15:42.551834+0000","last_active":"2023-11-26T07:15:43.543997+0000","last_peered":"2023-11-26T07:15:43.543997+0000","last_clean":"2023-11-26T07:15:43.543997+0000","last_became_active":"2023-11-26T07:15:42.551526+0000","last_became_peered":"2023-11-26T07:15:42.551526+0000","last_unstale":"2023-11-26T07:15:43.543997+0000","last_undegraded":"2023-11-26T07:15:43.543997+0000","last_fullsized":"2023-11-26T07:15:43.543997+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:16:25.381902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620148+0000","last_change":"2023-11-26T07:15:42.561995+0000","last_active":"2023-11-26T07:15:43.620148+0000","last_peered":"2023-11-26T07:15:43.620148+0000","last_clean":"2023-11-26T07:15:43.620148+0000","last_became_active":"2023-11-26T07:15:42.561737+0000","last_became_peered":"2023-11-26T07:15:42.561737+0000","last_unstale":"2023-11-26T07:15:43.620148+0000","last_undegraded":"2023-11-26T07:15:43.620148+0000","last_fullsized":"2023-11-26T07:15:43.620148+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:56:17.089456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621199+0000","last_change":"2023-11-26T07:15:42.563910+0000","last_active":"2023-11-26T07:15:43.621199+0000","last_peered":"2023-11-26T07:15:43.621199+0000","last_clean":"2023-11-26T07:15:43.621199+0000","last_became_active":"2023-11-26T07:15:42.563652+0000","last_became_peered":"2023-11-26T07:15:42.563652+0000","last_unstale":"2023-11-26T07:15:43.621199+0000","last_undegraded":"2023-11-26T07:15:43.621199+0000","last_fullsized":"2023-11-26T07:15:43.621199+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:56:47.443320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620559+0000","last_change":"2023-11-26T07:15:42.565019+0000","last_active":"2023-11-26T07:15:43.620559+0000","last_peered":"2023-11-26T07:15:43.620559+0000","last_clean":"2023-11-26T07:15:43.620559+0000","last_became_active":"2023-11-26T07:15:42.564700+0000","last_became_peered":"2023-11-26T07:15:42.564700+0000","last_unstale":"2023-11-26T07:15:43.620559+0000","last_undegraded":"2023-11-26T07:15:43.620559+0000","last_fullsized":"2023-11-26T07:15:43.620559+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:03.919981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620750+0000","last_change":"2023-11-26T07:15:42.557088+0000","last_active":"2023-11-26T07:15:43.620750+0000","last_peered":"2023-11-26T07:15:43.620750+0000","last_clean":"2023-11-26T07:15:43.620750+0000","last_became_active":"2023-11-26T07:15:42.556803+0000","last_became_peered":"2023-11-26T07:15:42.556803+0000","last_unstale":"2023-11-26T07:15:43.620750+0000","last_undegraded":"2023-11-26T07:15:43.620750+0000","last_fullsized":"2023-11-26T07:15:43.620750+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:13:50.560218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545235+0000","last_change":"2023-11-26T07:15:42.554231+0000","last_active":"2023-11-26T07:15:43.545235+0000","last_peered":"2023-11-26T07:15:43.545235+0000","last_clean":"2023-11-26T07:15:43.545235+0000","last_became_active":"2023-11-26T07:15:42.553934+0000","last_became_peered":"2023-11-26T07:15:42.553934+0000","last_unstale":"2023-11-26T07:15:43.545235+0000","last_undegraded":"2023-11-26T07:15:43.545235+0000","last_fullsized":"2023-11-26T07:15:43.545235+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:36:36.176509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545359+0000","last_change":"2023-11-26T07:15:41.953970+0000","last_active":"2023-11-26T07:15:43.545359+0000","last_peered":"2023-11-26T07:15:43.545359+0000","last_clean":"2023-11-26T07:15:43.545359+0000","last_became_active":"2023-11-26T07:15:41.953685+0000","last_became_peered":"2023-11-26T07:15:41.953685+0000","last_unstale":"2023-11-26T07:15:43.545359+0000","last_undegraded":"2023-11-26T07:15:43.545359+0000","last_fullsized":"2023-11-26T07:15:43.545359+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:37:47.890164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620785+0000","last_change":"2023-11-26T07:15:42.555141+0000","last_active":"2023-11-26T07:15:43.620785+0000","last_peered":"2023-11-26T07:15:43.620785+0000","last_clean":"2023-11-26T07:15:43.620785+0000","last_became_active":"2023-11-26T07:15:42.554838+0000","last_became_peered":"2023-11-26T07:15:42.554838+0000","last_unstale":"2023-11-26T07:15:43.620785+0000","last_undegraded":"2023-11-26T07:15:43.620785+0000","last_fullsized":"2023-11-26T07:15:43.620785+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:22:45.354130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542361+0000","last_change":"2023-11-26T07:15:41.950958+0000","last_active":"2023-11-26T07:15:43.542361+0000","last_peered":"2023-11-26T07:15:43.542361+0000","last_clean":"2023-11-26T07:15:43.542361+0000","last_became_active":"2023-11-26T07:15:41.950661+0000","last_became_peered":"2023-11-26T07:15:41.950661+0000","last_unstale":"2023-11-26T07:15:43.542361+0000","last_undegraded":"2023-11-26T07:15:43.542361+0000","last_fullsized":"2023-11-26T07:15:43.542361+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:28:47.202070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621361+0000","last_change":"2023-11-26T07:15:42.560738+0000","last_active":"2023-11-26T07:15:43.621361+0000","last_peered":"2023-11-26T07:15:43.621361+0000","last_clean":"2023-11-26T07:15:43.621361+0000","last_became_active":"2023-11-26T07:15:42.558226+0000","last_became_peered":"2023-11-26T07:15:42.558226+0000","last_unstale":"2023-11-26T07:15:43.621361+0000","last_undegraded":"2023-11-26T07:15:43.621361+0000","last_fullsized":"2023-11-26T07:15:43.621361+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:57.146358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543728+0000","last_change":"2023-11-26T07:15:41.953804+0000","last_active":"2023-11-26T07:15:43.543728+0000","last_peered":"2023-11-26T07:15:43.543728+0000","last_clean":"2023-11-26T07:15:43.543728+0000","last_became_active":"2023-11-26T07:15:41.953506+0000","last_became_peered":"2023-11-26T07:15:41.953506+0000","last_unstale":"2023-11-26T07:15:43.543728+0000","last_undegraded":"2023-11-26T07:15:43.543728+0000","last_fullsized":"2023-11-26T07:15:43.543728+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:59.824336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545130+0000","last_change":"2023-11-26T07:15:42.553374+0000","last_active":"2023-11-26T07:15:43.545130+0000","last_peered":"2023-11-26T07:15:43.545130+0000","last_clean":"2023-11-26T07:15:43.545130+0000","last_became_active":"2023-11-26T07:15:42.553066+0000","last_became_peered":"2023-11-26T07:15:42.553066+0000","last_unstale":"2023-11-26T07:15:43.545130+0000","last_undegraded":"2023-11-26T07:15:43.545130+0000","last_fullsized":"2023-11-26T07:15:43.545130+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:57.454676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621528+0000","last_change":"2023-11-26T07:15:41.955919+0000","last_active":"2023-11-26T07:15:43.621528+0000","last_peered":"2023-11-26T07:15:43.621528+0000","last_clean":"2023-11-26T07:15:43.621528+0000","last_became_active":"2023-11-26T07:15:41.955649+0000","last_became_peered":"2023-11-26T07:15:41.955649+0000","last_unstale":"2023-11-26T07:15:43.621528+0000","last_undegraded":"2023-11-26T07:15:43.621528+0000","last_fullsized":"2023-11-26T07:15:43.621528+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:01.435187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621147+0000","last_change":"2023-11-26T07:15:42.562446+0000","last_active":"2023-11-26T07:15:43.621147+0000","last_peered":"2023-11-26T07:15:43.621147+0000","last_clean":"2023-11-26T07:15:43.621147+0000","last_became_active":"2023-11-26T07:15:42.562164+0000","last_became_peered":"2023-11-26T07:15:42.562164+0000","last_unstale":"2023-11-26T07:15:43.621147+0000","last_undegraded":"2023-11-26T07:15:43.621147+0000","last_fullsized":"2023-11-26T07:15:43.621147+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:39.602824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620288+0000","last_change":"2023-11-26T07:15:41.956255+0000","last_active":"2023-11-26T07:15:43.620288+0000","last_peered":"2023-11-26T07:15:43.620288+0000","last_clean":"2023-11-26T07:15:43.620288+0000","last_became_active":"2023-11-26T07:15:41.956021+0000","last_became_peered":"2023-11-26T07:15:41.956021+0000","last_unstale":"2023-11-26T07:15:43.620288+0000","last_undegraded":"2023-11-26T07:15:43.620288+0000","last_fullsized":"2023-11-26T07:15:43.620288+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:36:35.172175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543847+0000","last_change":"2023-11-26T07:15:42.547264+0000","last_active":"2023-11-26T07:15:43.543847+0000","last_peered":"2023-11-26T07:15:43.543847+0000","last_clean":"2023-11-26T07:15:43.543847+0000","last_became_active":"2023-11-26T07:15:42.547004+0000","last_became_peered":"2023-11-26T07:15:42.547004+0000","last_unstale":"2023-11-26T07:15:43.543847+0000","last_undegraded":"2023-11-26T07:15:43.543847+0000","last_fullsized":"2023-11-26T07:15:43.543847+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:44.801698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621158+0000","last_change":"2023-11-26T07:15:41.956982+0000","last_active":"2023-11-26T07:15:43.621158+0000","last_peered":"2023-11-26T07:15:43.621158+0000","last_clean":"2023-11-26T07:15:43.621158+0000","last_became_active":"2023-11-26T07:15:41.956550+0000","last_became_peered":"2023-11-26T07:15:41.956550+0000","last_unstale":"2023-11-26T07:15:43.621158+0000","last_undegraded":"2023-11-26T07:15:43.621158+0000","last_fullsized":"2023-11-26T07:15:43.621158+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:44.333650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620520+0000","last_change":"2023-11-26T07:15:42.549320+0000","last_active":"2023-11-26T07:15:43.620520+0000","last_peered":"2023-11-26T07:15:43.620520+0000","last_clean":"2023-11-26T07:15:43.620520+0000","last_became_active":"2023-11-26T07:15:42.549146+0000","last_became_peered":"2023-11-26T07:15:42.549146+0000","last_unstale":"2023-11-26T07:15:43.620520+0000","last_undegraded":"2023-11-26T07:15:43.620520+0000","last_fullsized":"2023-11-26T07:15:43.620520+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:07.502524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621125+0000","last_change":"2023-11-26T07:15:41.951596+0000","last_active":"2023-11-26T07:15:43.621125+0000","last_peered":"2023-11-26T07:15:43.621125+0000","last_clean":"2023-11-26T07:15:43.621125+0000","last_became_active":"2023-11-26T07:15:41.951283+0000","last_became_peered":"2023-11-26T07:15:41.951283+0000","last_unstale":"2023-11-26T07:15:43.621125+0000","last_undegraded":"2023-11-26T07:15:43.621125+0000","last_fullsized":"2023-11-26T07:15:43.621125+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:55.975559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542160+0000","last_change":"2023-11-26T07:15:42.551403+0000","last_active":"2023-11-26T07:15:43.542160+0000","last_peered":"2023-11-26T07:15:43.542160+0000","last_clean":"2023-11-26T07:15:43.542160+0000","last_became_active":"2023-11-26T07:15:42.551153+0000","last_became_peered":"2023-11-26T07:15:42.551153+0000","last_unstale":"2023-11-26T07:15:43.542160+0000","last_undegraded":"2023-11-26T07:15:43.542160+0000","last_fullsized":"2023-11-26T07:15:43.542160+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:03.919981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620093+0000","last_change":"2023-11-26T07:15:41.952488+0000","last_active":"2023-11-26T07:15:43.620093+0000","last_peered":"2023-11-26T07:15:43.620093+0000","last_clean":"2023-11-26T07:15:43.620093+0000","last_became_active":"2023-11-26T07:15:41.952237+0000","last_became_peered":"2023-11-26T07:15:41.952237+0000","last_unstale":"2023-11-26T07:15:43.620093+0000","last_undegraded":"2023-11-26T07:15:43.620093+0000","last_fullsized":"2023-11-26T07:15:43.620093+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:15:16.108512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621030+0000","last_change":"2023-11-26T07:15:42.560224+0000","last_active":"2023-11-26T07:15:43.621030+0000","last_peered":"2023-11-26T07:15:43.621030+0000","last_clean":"2023-11-26T07:15:43.621030+0000","last_became_active":"2023-11-26T07:15:42.559941+0000","last_became_peered":"2023-11-26T07:15:42.559941+0000","last_unstale":"2023-11-26T07:15:43.621030+0000","last_undegraded":"2023-11-26T07:15:43.621030+0000","last_fullsized":"2023-11-26T07:15:43.621030+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:37:48.894498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543722+0000","last_change":"2023-11-26T07:15:41.959804+0000","last_active":"2023-11-26T07:15:43.543722+0000","last_peered":"2023-11-26T07:15:43.543722+0000","last_clean":"2023-11-26T07:15:43.543722+0000","last_became_active":"2023-11-26T07:15:41.959514+0000","last_became_peered":"2023-11-26T07:15:41.959514+0000","last_unstale":"2023-11-26T07:15:43.543722+0000","last_undegraded":"2023-11-26T07:15:43.543722+0000","last_fullsized":"2023-11-26T07:15:43.543722+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:08:17.046063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545276+0000","last_change":"2023-11-26T07:15:42.554213+0000","last_active":"2023-11-26T07:15:43.545276+0000","last_peered":"2023-11-26T07:15:43.545276+0000","last_clean":"2023-11-26T07:15:43.545276+0000","last_became_active":"2023-11-26T07:15:42.553928+0000","last_became_peered":"2023-11-26T07:15:42.553928+0000","last_unstale":"2023-11-26T07:15:43.545276+0000","last_undegraded":"2023-11-26T07:15:43.545276+0000","last_fullsized":"2023-11-26T07:15:43.545276+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:51.589701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621686+0000","last_change":"2023-11-26T07:15:41.959972+0000","last_active":"2023-11-26T07:15:43.621686+0000","last_peered":"2023-11-26T07:15:43.621686+0000","last_clean":"2023-11-26T07:15:43.621686+0000","last_became_active":"2023-11-26T07:15:41.959106+0000","last_became_peered":"2023-11-26T07:15:41.959106+0000","last_unstale":"2023-11-26T07:15:43.621686+0000","last_undegraded":"2023-11-26T07:15:43.621686+0000","last_fullsized":"2023-11-26T07:15:43.621686+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:16:33.088448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621415+0000","last_change":"2023-11-26T07:15:42.561135+0000","last_active":"2023-11-26T07:15:43.621415+0000","last_peered":"2023-11-26T07:15:43.621415+0000","last_clean":"2023-11-26T07:15:43.621415+0000","last_became_active":"2023-11-26T07:15:42.557955+0000","last_became_peered":"2023-11-26T07:15:42.557955+0000","last_unstale":"2023-11-26T07:15:43.621415+0000","last_undegraded":"2023-11-26T07:15:43.621415+0000","last_fullsized":"2023-11-26T07:15:43.621415+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:48:24.866444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620683+0000","last_change":"2023-11-26T07:15:41.959510+0000","last_active":"2023-11-26T07:15:43.620683+0000","last_peered":"2023-11-26T07:15:43.620683+0000","last_clean":"2023-11-26T07:15:43.620683+0000","last_became_active":"2023-11-26T07:15:41.959203+0000","last_became_peered":"2023-11-26T07:15:41.959203+0000","last_unstale":"2023-11-26T07:15:43.620683+0000","last_undegraded":"2023-11-26T07:15:43.620683+0000","last_fullsized":"2023-11-26T07:15:43.620683+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:09:18.105341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620501+0000","last_change":"2023-11-26T07:15:42.556469+0000","last_active":"2023-11-26T07:15:43.620501+0000","last_peered":"2023-11-26T07:15:43.620501+0000","last_clean":"2023-11-26T07:15:43.620501+0000","last_became_active":"2023-11-26T07:15:42.556200+0000","last_became_peered":"2023-11-26T07:15:42.556200+0000","last_unstale":"2023-11-26T07:15:43.620501+0000","last_undegraded":"2023-11-26T07:15:43.620501+0000","last_fullsized":"2023-11-26T07:15:43.620501+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:55:45.155216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"69'6","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621534+0000","last_change":"2023-11-26T07:15:41.952043+0000","last_active":"2023-11-26T07:15:43.621534+0000","last_peered":"2023-11-26T07:15:43.621534+0000","last_clean":"2023-11-26T07:15:43.621534+0000","last_became_active":"2023-11-26T07:15:41.951786+0000","last_became_peered":"2023-11-26T07:15:41.951786+0000","last_unstale":"2023-11-26T07:15:43.621534+0000","last_undegraded":"2023-11-26T07:15:43.621534+0000","last_fullsized":"2023-11-26T07:15:43.621534+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:50.064029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.544858+0000","last_change":"2023-11-26T07:15:42.552333+0000","last_active":"2023-11-26T07:15:43.544858+0000","last_peered":"2023-11-26T07:15:43.544858+0000","last_clean":"2023-11-26T07:15:43.544858+0000","last_became_active":"2023-11-26T07:15:42.552042+0000","last_became_peered":"2023-11-26T07:15:42.552042+0000","last_unstale":"2023-11-26T07:15:43.544858+0000","last_undegraded":"2023-11-26T07:15:43.544858+0000","last_fullsized":"2023-11-26T07:15:43.544858+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:23:21.337577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621619+0000","last_change":"2023-11-26T07:15:41.951469+0000","last_active":"2023-11-26T07:15:43.621619+0000","last_peered":"2023-11-26T07:15:43.621619+0000","last_clean":"2023-11-26T07:15:43.621619+0000","last_became_active":"2023-11-26T07:15:41.951172+0000","last_became_peered":"2023-11-26T07:15:41.951172+0000","last_unstale":"2023-11-26T07:15:43.621619+0000","last_undegraded":"2023-11-26T07:15:43.621619+0000","last_fullsized":"2023-11-26T07:15:43.621619+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:39:53.351270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621363+0000","last_change":"2023-11-26T07:15:42.560219+0000","last_active":"2023-11-26T07:15:43.621363+0000","last_peered":"2023-11-26T07:15:43.621363+0000","last_clean":"2023-11-26T07:15:43.621363+0000","last_became_active":"2023-11-26T07:15:42.559966+0000","last_became_peered":"2023-11-26T07:15:42.559966+0000","last_unstale":"2023-11-26T07:15:43.621363+0000","last_undegraded":"2023-11-26T07:15:43.621363+0000","last_fullsized":"2023-11-26T07:15:43.621363+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:24:21.592715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621249+0000","last_change":"2023-11-26T07:15:41.956792+0000","last_active":"2023-11-26T07:15:43.621249+0000","last_peered":"2023-11-26T07:15:43.621249+0000","last_clean":"2023-11-26T07:15:43.621249+0000","last_became_active":"2023-11-26T07:15:41.956267+0000","last_became_peered":"2023-11-26T07:15:41.956267+0000","last_unstale":"2023-11-26T07:15:43.621249+0000","last_undegraded":"2023-11-26T07:15:43.621249+0000","last_fullsized":"2023-11-26T07:15:43.621249+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:48.935177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.541921+0000","last_change":"2023-11-26T07:15:42.562248+0000","last_active":"2023-11-26T07:15:43.541921+0000","last_peered":"2023-11-26T07:15:43.541921+0000","last_clean":"2023-11-26T07:15:43.541921+0000","last_became_active":"2023-11-26T07:15:42.561971+0000","last_became_peered":"2023-11-26T07:15:42.561971+0000","last_unstale":"2023-11-26T07:15:43.541921+0000","last_undegraded":"2023-11-26T07:15:43.541921+0000","last_fullsized":"2023-11-26T07:15:43.541921+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:09:14.461759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620661+0000","last_change":"2023-11-26T07:15:41.959413+0000","last_active":"2023-11-26T07:15:43.620661+0000","last_peered":"2023-11-26T07:15:43.620661+0000","last_clean":"2023-11-26T07:15:43.620661+0000","last_became_active":"2023-11-26T07:15:41.959137+0000","last_became_peered":"2023-11-26T07:15:41.959137+0000","last_unstale":"2023-11-26T07:15:43.620661+0000","last_undegraded":"2023-11-26T07:15:43.620661+0000","last_fullsized":"2023-11-26T07:15:43.620661+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:09:13.457425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620285+0000","last_change":"2023-11-26T07:15:42.556041+0000","last_active":"2023-11-26T07:15:43.620285+0000","last_peered":"2023-11-26T07:15:43.620285+0000","last_clean":"2023-11-26T07:15:43.620285+0000","last_became_active":"2023-11-26T07:15:42.555799+0000","last_became_peered":"2023-11-26T07:15:42.555799+0000","last_unstale":"2023-11-26T07:15:43.620285+0000","last_undegraded":"2023-11-26T07:15:43.620285+0000","last_fullsized":"2023-11-26T07:15:43.620285+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:30.693006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543293+0000","last_change":"2023-11-26T07:15:41.949566+0000","last_active":"2023-11-26T07:15:43.543293+0000","last_peered":"2023-11-26T07:15:43.543293+0000","last_clean":"2023-11-26T07:15:43.543293+0000","last_became_active":"2023-11-26T07:15:41.949160+0000","last_became_peered":"2023-11-26T07:15:41.949160+0000","last_unstale":"2023-11-26T07:15:43.543293+0000","last_undegraded":"2023-11-26T07:15:43.543293+0000","last_fullsized":"2023-11-26T07:15:43.543293+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:36.817478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543754+0000","last_change":"2023-11-26T07:15:42.555690+0000","last_active":"2023-11-26T07:15:43.543754+0000","last_peered":"2023-11-26T07:15:43.543754+0000","last_clean":"2023-11-26T07:15:43.543754+0000","last_became_active":"2023-11-26T07:15:42.555401+0000","last_became_peered":"2023-11-26T07:15:42.555401+0000","last_unstale":"2023-11-26T07:15:43.543754+0000","last_undegraded":"2023-11-26T07:15:43.543754+0000","last_fullsized":"2023-11-26T07:15:43.543754+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:02:39.913059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621119+0000","last_change":"2023-11-26T07:15:41.959661+0000","last_active":"2023-11-26T07:15:43.621119+0000","last_peered":"2023-11-26T07:15:43.621119+0000","last_clean":"2023-11-26T07:15:43.621119+0000","last_became_active":"2023-11-26T07:15:41.959424+0000","last_became_peered":"2023-11-26T07:15:41.959424+0000","last_unstale":"2023-11-26T07:15:43.621119+0000","last_undegraded":"2023-11-26T07:15:43.621119+0000","last_fullsized":"2023-11-26T07:15:43.621119+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:02:45.769127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545371+0000","last_change":"2023-11-26T07:15:42.555994+0000","last_active":"2023-11-26T07:15:43.545371+0000","last_peered":"2023-11-26T07:15:43.545371+0000","last_clean":"2023-11-26T07:15:43.545371+0000","last_became_active":"2023-11-26T07:15:42.554965+0000","last_became_peered":"2023-11-26T07:15:42.554965+0000","last_unstale":"2023-11-26T07:15:43.545371+0000","last_undegraded":"2023-11-26T07:15:43.545371+0000","last_fullsized":"2023-11-26T07:15:43.545371+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:42:12.777618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543863+0000","last_change":"2023-11-26T07:15:41.952695+0000","last_active":"2023-11-26T07:15:43.543863+0000","last_peered":"2023-11-26T07:15:43.543863+0000","last_clean":"2023-11-26T07:15:43.543863+0000","last_became_active":"2023-11-26T07:15:41.952386+0000","last_became_peered":"2023-11-26T07:15:41.952386+0000","last_unstale":"2023-11-26T07:15:43.543863+0000","last_undegraded":"2023-11-26T07:15:43.543863+0000","last_fullsized":"2023-11-26T07:15:43.543863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:13:49.555884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621495+0000","last_change":"2023-11-26T07:15:42.557933+0000","last_active":"2023-11-26T07:15:43.621495+0000","last_peered":"2023-11-26T07:15:43.621495+0000","last_clean":"2023-11-26T07:15:43.621495+0000","last_became_active":"2023-11-26T07:15:42.557648+0000","last_became_peered":"2023-11-26T07:15:42.557648+0000","last_unstale":"2023-11-26T07:15:43.621495+0000","last_undegraded":"2023-11-26T07:15:43.621495+0000","last_fullsized":"2023-11-26T07:15:43.621495+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:02:46.773460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621172+0000","last_change":"2023-11-26T07:15:41.949029+0000","last_active":"2023-11-26T07:15:43.621172+0000","last_peered":"2023-11-26T07:15:43.621172+0000","last_clean":"2023-11-26T07:15:43.621172+0000","last_became_active":"2023-11-26T07:15:41.948743+0000","last_became_peered":"2023-11-26T07:15:41.948743+0000","last_unstale":"2023-11-26T07:15:43.621172+0000","last_undegraded":"2023-11-26T07:15:43.621172+0000","last_fullsized":"2023-11-26T07:15:43.621172+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:33:09.745619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620738+0000","last_change":"2023-11-26T07:15:42.551831+0000","last_active":"2023-11-26T07:15:43.620738+0000","last_peered":"2023-11-26T07:15:43.620738+0000","last_clean":"2023-11-26T07:15:43.620738+0000","last_became_active":"2023-11-26T07:15:42.551533+0000","last_became_peered":"2023-11-26T07:15:42.551533+0000","last_unstale":"2023-11-26T07:15:43.620738+0000","last_undegraded":"2023-11-26T07:15:43.620738+0000","last_fullsized":"2023-11-26T07:15:43.620738+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:39:54.355604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.544925+0000","last_change":"2023-11-26T07:15:41.948603+0000","last_active":"2023-11-26T07:15:43.544925+0000","last_peered":"2023-11-26T07:15:43.544925+0000","last_clean":"2023-11-26T07:15:43.544925+0000","last_became_active":"2023-11-26T07:15:41.948232+0000","last_became_peered":"2023-11-26T07:15:41.948232+0000","last_unstale":"2023-11-26T07:15:43.544925+0000","last_undegraded":"2023-11-26T07:15:43.544925+0000","last_fullsized":"2023-11-26T07:15:43.544925+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:03:16.843398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620281+0000","last_change":"2023-11-26T07:15:42.555110+0000","last_active":"2023-11-26T07:15:43.620281+0000","last_peered":"2023-11-26T07:15:43.620281+0000","last_clean":"2023-11-26T07:15:43.620281+0000","last_became_active":"2023-11-26T07:15:42.554818+0000","last_became_peered":"2023-11-26T07:15:42.554818+0000","last_unstale":"2023-11-26T07:15:43.620281+0000","last_undegraded":"2023-11-26T07:15:43.620281+0000","last_fullsized":"2023-11-26T07:15:43.620281+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:44:15.066137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621346+0000","last_change":"2023-11-26T07:15:41.954948+0000","last_active":"2023-11-26T07:15:43.621346+0000","last_peered":"2023-11-26T07:15:43.621346+0000","last_clean":"2023-11-26T07:15:43.621346+0000","last_became_active":"2023-11-26T07:15:41.954672+0000","last_became_peered":"2023-11-26T07:15:41.954672+0000","last_unstale":"2023-11-26T07:15:43.621346+0000","last_undegraded":"2023-11-26T07:15:43.621346+0000","last_fullsized":"2023-11-26T07:15:43.621346+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:50.064029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620001+0000","last_change":"2023-11-26T07:15:42.562761+0000","last_active":"2023-11-26T07:15:43.620001+0000","last_peered":"2023-11-26T07:15:43.620001+0000","last_clean":"2023-11-26T07:15:43.620001+0000","last_became_active":"2023-11-26T07:15:42.562517+0000","last_became_peered":"2023-11-26T07:15:42.562517+0000","last_unstale":"2023-11-26T07:15:43.620001+0000","last_undegraded":"2023-11-26T07:15:43.620001+0000","last_fullsized":"2023-11-26T07:15:43.620001+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:42:12.777618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620691+0000","last_change":"2023-11-26T07:15:41.956877+0000","last_active":"2023-11-26T07:15:43.620691+0000","last_peered":"2023-11-26T07:15:43.620691+0000","last_clean":"2023-11-26T07:15:43.620691+0000","last_became_active":"2023-11-26T07:15:41.956568+0000","last_became_peered":"2023-11-26T07:15:41.956568+0000","last_unstale":"2023-11-26T07:15:43.620691+0000","last_undegraded":"2023-11-26T07:15:43.620691+0000","last_fullsized":"2023-11-26T07:15:43.620691+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:03:16.843398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620082+0000","last_change":"2023-11-26T07:15:42.562379+0000","last_active":"2023-11-26T07:15:43.620082+0000","last_peered":"2023-11-26T07:15:43.620082+0000","last_clean":"2023-11-26T07:15:43.620082+0000","last_became_active":"2023-11-26T07:15:42.562074+0000","last_became_peered":"2023-11-26T07:15:42.562074+0000","last_unstale":"2023-11-26T07:15:43.620082+0000","last_undegraded":"2023-11-26T07:15:43.620082+0000","last_fullsized":"2023-11-26T07:15:43.620082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:43:31.617493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.541856+0000","last_change":"2023-11-26T07:15:41.950613+0000","last_active":"2023-11-26T07:15:43.541856+0000","last_peered":"2023-11-26T07:15:43.541856+0000","last_clean":"2023-11-26T07:15:43.541856+0000","last_became_active":"2023-11-26T07:15:41.950293+0000","last_became_peered":"2023-11-26T07:15:41.950293+0000","last_unstale":"2023-11-26T07:15:43.541856+0000","last_undegraded":"2023-11-26T07:15:43.541856+0000","last_fullsized":"2023-11-26T07:15:43.541856+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:39:32.383802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542312+0000","last_change":"2023-11-26T07:15:42.552746+0000","last_active":"2023-11-26T07:15:43.542312+0000","last_peered":"2023-11-26T07:15:43.542312+0000","last_clean":"2023-11-26T07:15:43.542312+0000","last_became_active":"2023-11-26T07:15:42.552494+0000","last_became_peered":"2023-11-26T07:15:42.552494+0000","last_unstale":"2023-11-26T07:15:43.542312+0000","last_undegraded":"2023-11-26T07:15:43.542312+0000","last_fullsized":"2023-11-26T07:15:43.542312+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:56:47.443320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621412+0000","last_change":"2023-11-26T07:15:41.959643+0000","last_active":"2023-11-26T07:15:43.621412+0000","last_peered":"2023-11-26T07:15:43.621412+0000","last_clean":"2023-11-26T07:15:43.621412+0000","last_became_active":"2023-11-26T07:15:41.959374+0000","last_became_peered":"2023-11-26T07:15:41.959374+0000","last_unstale":"2023-11-26T07:15:43.621412+0000","last_undegraded":"2023-11-26T07:15:43.621412+0000","last_fullsized":"2023-11-26T07:15:43.621412+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:46:42.835450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620752+0000","last_change":"2023-11-26T07:15:42.561349+0000","last_active":"2023-11-26T07:15:43.620752+0000","last_peered":"2023-11-26T07:15:43.620752+0000","last_clean":"2023-11-26T07:15:43.620752+0000","last_became_active":"2023-11-26T07:15:42.561083+0000","last_became_peered":"2023-11-26T07:15:42.561083+0000","last_unstale":"2023-11-26T07:15:43.620752+0000","last_undegraded":"2023-11-26T07:15:43.620752+0000","last_fullsized":"2023-11-26T07:15:43.620752+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:25:13.533544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621151+0000","last_change":"2023-11-26T07:15:41.959653+0000","last_active":"2023-11-26T07:15:43.621151+0000","last_peered":"2023-11-26T07:15:43.621151+0000","last_clean":"2023-11-26T07:15:43.621151+0000","last_became_active":"2023-11-26T07:15:41.959232+0000","last_became_peered":"2023-11-26T07:15:41.959232+0000","last_unstale":"2023-11-26T07:15:43.621151+0000","last_undegraded":"2023-11-26T07:15:43.621151+0000","last_fullsized":"2023-11-26T07:15:43.621151+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:39:32.383802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545063+0000","last_change":"2023-11-26T07:15:42.558530+0000","last_active":"2023-11-26T07:15:43.545063+0000","last_peered":"2023-11-26T07:15:43.545063+0000","last_clean":"2023-11-26T07:15:43.545063+0000","last_became_active":"2023-11-26T07:15:42.558220+0000","last_became_peered":"2023-11-26T07:15:42.558220+0000","last_unstale":"2023-11-26T07:15:43.545063+0000","last_undegraded":"2023-11-26T07:15:43.545063+0000","last_fullsized":"2023-11-26T07:15:43.545063+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:43:56.465281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620593+0000","last_change":"2023-11-26T07:15:41.954074+0000","last_active":"2023-11-26T07:15:43.620593+0000","last_peered":"2023-11-26T07:15:43.620593+0000","last_clean":"2023-11-26T07:15:43.620593+0000","last_became_active":"2023-11-26T07:15:41.953867+0000","last_became_peered":"2023-11-26T07:15:41.953867+0000","last_unstale":"2023-11-26T07:15:43.620593+0000","last_undegraded":"2023-11-26T07:15:43.620593+0000","last_fullsized":"2023-11-26T07:15:43.620593+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:30.504173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545308+0000","last_change":"2023-11-26T07:15:42.562446+0000","last_active":"2023-11-26T07:15:43.545308+0000","last_peered":"2023-11-26T07:15:43.545308+0000","last_clean":"2023-11-26T07:15:43.545308+0000","last_became_active":"2023-11-26T07:15:42.562126+0000","last_became_peered":"2023-11-26T07:15:42.562126+0000","last_unstale":"2023-11-26T07:15:43.545308+0000","last_undegraded":"2023-11-26T07:15:43.545308+0000","last_fullsized":"2023-11-26T07:15:43.545308+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:09:19.109675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542143+0000","last_change":"2023-11-26T07:15:41.953823+0000","last_active":"2023-11-26T07:15:43.542143+0000","last_peered":"2023-11-26T07:15:43.542143+0000","last_clean":"2023-11-26T07:15:43.542143+0000","last_became_active":"2023-11-26T07:15:41.953500+0000","last_became_peered":"2023-11-26T07:15:41.953500+0000","last_unstale":"2023-11-26T07:15:43.542143+0000","last_undegraded":"2023-11-26T07:15:43.542143+0000","last_fullsized":"2023-11-26T07:15:43.542143+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:40:56.337155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620950+0000","last_change":"2023-11-26T07:15:42.555545+0000","last_active":"2023-11-26T07:15:43.620950+0000","last_peered":"2023-11-26T07:15:43.620950+0000","last_clean":"2023-11-26T07:15:43.620950+0000","last_became_active":"2023-11-26T07:15:42.555264+0000","last_became_peered":"2023-11-26T07:15:42.555264+0000","last_unstale":"2023-11-26T07:15:43.620950+0000","last_undegraded":"2023-11-26T07:15:43.620950+0000","last_fullsized":"2023-11-26T07:15:43.620950+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:45.337983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620523+0000","last_change":"2023-11-26T07:15:41.959310+0000","last_active":"2023-11-26T07:15:43.620523+0000","last_peered":"2023-11-26T07:15:43.620523+0000","last_clean":"2023-11-26T07:15:43.620523+0000","last_became_active":"2023-11-26T07:15:41.959005+0000","last_became_peered":"2023-11-26T07:15:41.959005+0000","last_unstale":"2023-11-26T07:15:43.620523+0000","last_undegraded":"2023-11-26T07:15:43.620523+0000","last_fullsized":"2023-11-26T07:15:43.620523+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:01:17.600412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620951+0000","last_change":"2023-11-26T07:15:42.560116+0000","last_active":"2023-11-26T07:15:43.620951+0000","last_peered":"2023-11-26T07:15:43.620951+0000","last_clean":"2023-11-26T07:15:43.620951+0000","last_became_active":"2023-11-26T07:15:42.559877+0000","last_became_peered":"2023-11-26T07:15:42.559877+0000","last_unstale":"2023-11-26T07:15:43.620951+0000","last_undegraded":"2023-11-26T07:15:43.620951+0000","last_fullsized":"2023-11-26T07:15:43.620951+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:37.821812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621124+0000","last_change":"2023-11-26T07:15:41.951472+0000","last_active":"2023-11-26T07:15:43.621124+0000","last_peered":"2023-11-26T07:15:43.621124+0000","last_clean":"2023-11-26T07:15:43.621124+0000","last_became_active":"2023-11-26T07:15:41.951225+0000","last_became_peered":"2023-11-26T07:15:41.951225+0000","last_unstale":"2023-11-26T07:15:43.621124+0000","last_undegraded":"2023-11-26T07:15:43.621124+0000","last_fullsized":"2023-11-26T07:15:43.621124+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:22:44.349797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621235+0000","last_change":"2023-11-26T07:15:42.554835+0000","last_active":"2023-11-26T07:15:43.621235+0000","last_peered":"2023-11-26T07:15:43.621235+0000","last_clean":"2023-11-26T07:15:43.621235+0000","last_became_active":"2023-11-26T07:15:42.554583+0000","last_became_peered":"2023-11-26T07:15:42.554583+0000","last_unstale":"2023-11-26T07:15:43.621235+0000","last_undegraded":"2023-11-26T07:15:43.621235+0000","last_fullsized":"2023-11-26T07:15:43.621235+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:49.241773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543310+0000","last_change":"2023-11-26T07:15:41.948401+0000","last_active":"2023-11-26T07:15:43.543310+0000","last_peered":"2023-11-26T07:15:43.543310+0000","last_clean":"2023-11-26T07:15:43.543310+0000","last_became_active":"2023-11-26T07:15:41.948121+0000","last_became_peered":"2023-11-26T07:15:41.948121+0000","last_unstale":"2023-11-26T07:15:43.543310+0000","last_undegraded":"2023-11-26T07:15:43.543310+0000","last_fullsized":"2023-11-26T07:15:43.543310+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:20.640171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620533+0000","last_change":"2023-11-26T07:15:42.556121+0000","last_active":"2023-11-26T07:15:43.620533+0000","last_peered":"2023-11-26T07:15:43.620533+0000","last_clean":"2023-11-26T07:15:43.620533+0000","last_became_active":"2023-11-26T07:15:42.555836+0000","last_became_peered":"2023-11-26T07:15:42.555836+0000","last_unstale":"2023-11-26T07:15:43.620533+0000","last_undegraded":"2023-11-26T07:15:43.620533+0000","last_fullsized":"2023-11-26T07:15:43.620533+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:22:00.828670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620498+0000","last_change":"2023-11-26T07:15:41.956106+0000","last_active":"2023-11-26T07:15:43.620498+0000","last_peered":"2023-11-26T07:15:43.620498+0000","last_clean":"2023-11-26T07:15:43.620498+0000","last_became_active":"2023-11-26T07:15:41.955837+0000","last_became_peered":"2023-11-26T07:15:41.955837+0000","last_unstale":"2023-11-26T07:15:43.620498+0000","last_undegraded":"2023-11-26T07:15:43.620498+0000","last_fullsized":"2023-11-26T07:15:43.620498+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:13.627141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620145+0000","last_change":"2023-11-26T07:15:42.562538+0000","last_active":"2023-11-26T07:15:43.620145+0000","last_peered":"2023-11-26T07:15:43.620145+0000","last_clean":"2023-11-26T07:15:43.620145+0000","last_became_active":"2023-11-26T07:15:42.562179+0000","last_became_peered":"2023-11-26T07:15:42.562179+0000","last_unstale":"2023-11-26T07:15:43.620145+0000","last_undegraded":"2023-11-26T07:15:43.620145+0000","last_fullsized":"2023-11-26T07:15:43.620145+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:56:15.646039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620820+0000","last_change":"2023-11-26T07:15:41.954452+0000","last_active":"2023-11-26T07:15:43.620820+0000","last_peered":"2023-11-26T07:15:43.620820+0000","last_clean":"2023-11-26T07:15:43.620820+0000","last_became_active":"2023-11-26T07:15:41.954172+0000","last_became_peered":"2023-11-26T07:15:41.954172+0000","last_unstale":"2023-11-26T07:15:43.620820+0000","last_undegraded":"2023-11-26T07:15:43.620820+0000","last_fullsized":"2023-11-26T07:15:43.620820+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:22.663677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":216,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-26T07:15:42.541564+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:15:42.541564+0000","last_peered":"2023-11-26T07:15:42.541564+0000","last_clean":"2023-11-26T07:15:42.541564+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:15:42.541564+0000","last_undegraded":"2023-11-26T07:15:42.541564+0000","last_fullsized":"2023-11-26T07:15:42.541564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28440,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343400,"statfs":{"total":96636764160,"available":96607641600,"internally_reserved":0,"allocated":1454080,"data_stored":949304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82099999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.307,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90000000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.58899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.81299999999999994,"5min":1.048,"15min":1.048},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.67900000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.37,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.624,"15min":0.623},"min":{"1min":0.44500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.877,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.5}]},{"osd":3,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.499,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71199999999999997,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.48299999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91400000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79600000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.746}]},{"osd":5,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.747,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82999999999999996,"5min":1.171,"15min":1.171},"last":0.76300000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44700000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.006,"5min":1.361,"15min":1.361},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.46300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85499999999999998,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.70499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":358412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75900000000000001,"5min":1.159,"15min":1.159},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55900000000000005,"15min":0.57699999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73699999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.55100000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67700000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80000000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.46700000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71799999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80800000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58399999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.433,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77600000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.62},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.70299999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.5,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.747,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71299999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42699999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50700000000000001},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72099999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.5},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.747,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":360548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.33300000000000002},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.47699999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.33500000000000002},"min":{"1min":0.24199999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.441},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53300000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.55900000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.28799999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.57199999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83399999999999996,"5min":0.877,"15min":0.877},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86499999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.79200000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79400000000000004,"5min":0.995,"15min":0.995},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.62,"15min":0.629},"min":{"1min":0.45600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69099999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.42399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":1.129,"15min":1.129},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83799999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91400000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.879,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28432,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343408,"statfs":{"total":96636764160,"available":96607649792,"internally_reserved":0,"allocated":1445888,"data_stored":948688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26900000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.379,"15min":0.37},"min":{"1min":0.245,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.38200000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.29499999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64300000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.34899999999999998},"min":{"1min":0.23200000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.34899999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40000000000000002,"15min":0.40300000000000002},"min":{"1min":0.29799999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.438,"15min":0.44},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60699999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41899999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77800000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67000000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.621,"15min":0.63500000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.49299999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89000000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.625},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73699999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.502}]},{"osd":7,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81899999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.65000000000000002,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.65300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":359932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.36099999999999999,"15min":0.36899999999999999},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.61199999999999999,"5min":0.627,"15min":0.627},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.26400000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.51100000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.42699999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.39000000000000001},"min":{"1min":0.26600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.53400000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.379,"15min":0.36799999999999999},"min":{"1min":0.28799999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.372}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"min":{"1min":0.28599999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.58799999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41799999999999998,"15min":0.42599999999999999},"min":{"1min":0.29599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68100000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.67100000000000004},"min":{"1min":0.41499999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81299999999999994,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.59799999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.622,"15min":0.624},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90000000000000002,"5min":1.075,"15min":1.075},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0309999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.51300000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.004,"5min":1.03,"15min":1.03},"last":0.67900000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64700000000000002,"15min":0.65900000000000003},"min":{"1min":0.504,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.502,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96199999999999997,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":358432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35399999999999998,"15min":0.35999999999999999},"min":{"1min":0.23999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.629,"5min":1.284,"15min":1.284},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.42799999999999999,"15min":0.47299999999999998},"min":{"1min":0.26800000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.437,"15min":0.51500000000000001},"min":{"1min":0.25,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58799999999999997,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.44700000000000001,"15min":0.49199999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.69399999999999995,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.437}]},{"osd":2,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46800000000000003,"15min":0.502},"min":{"1min":0.27900000000000003,"5min":0.247,"15min":0.247},"max":{"1min":0.61199999999999999,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.504},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41199999999999998,"15min":0.42699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.59799999999999998,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.51900000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.64600000000000002,"15min":0.67400000000000004},"min":{"1min":0.41499999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79000000000000004,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.63800000000000001,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81899999999999995,"5min":1.405,"15min":1.405},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.625,"15min":0.63700000000000001},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78100000000000003,"5min":1.377,"15min":1.377},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.71499999999999997,"15min":0.79500000000000004},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89200000000000002,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.72199999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.67100000000000004},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79800000000000004,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.71599999999999997,"15min":0.78900000000000003},"min":{"1min":0.434,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82599999999999996,"5min":4.883,"15min":4.883},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.80500000000000005},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.68899999999999995,"15min":0.73299999999999998},"min":{"1min":0.437,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.876,"5min":2.923,"15min":2.923},"last":0.748}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":358390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.60999999999999999},"min":{"1min":0.34300000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.79400000000000004,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.498},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68500000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.66700000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.375,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.65700000000000003},"min":{"1min":0.376,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91100000000000003,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.53300000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56000000000000005,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.67300000000000004},"min":{"1min":0.438,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76000000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.57999999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56200000000000006,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56000000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70799999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58299999999999996,"15min":0.60299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73899999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.746},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.63600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.68600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":358410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.32000000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69199999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.54000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":1.1259999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.45800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68500000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":1.091},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.79800000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.49099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.70499999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":1.012},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.53900000000000003},"min":{"1min":0.40200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.72399999999999998,"5min":0.75,"15min":0.75},"last":0.76900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.45200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76200000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.72599999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.495,"15min":0.499},"min":{"1min":0.36599999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54500000000000004,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":1.161}]},{"osd":6,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55900000000000005,"15min":0.57699999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76100000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67000000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.93100000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.747,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":1.196},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.63,"15min":0.64600000000000002},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80500000000000005,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.89700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:15:47.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:15:47.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.216+0000 7fe47822e700 1 Processor -- start 2023-11-26T07:15:47.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.216+0000 7fe47822e700 1 -- start start 2023-11-26T07:15:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe47822e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe47015e080 0x7fe4700b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe47822e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe4700af4d0 0x7fe4700afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe47822e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe47006c000 0x7fe47006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe47822e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4700af0b0 con 0x7fe47006c000 2023-11-26T07:15:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe47822e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47006c9f0 con 0x7fe4700af4d0 2023-11-26T07:15:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe47822e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700b05a0 con 0x7fe47015e080 2023-11-26T07:15:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe4767cb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe47006c000 0x7fe47006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe4767cb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe47006c000 0x7fe47006c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39588/0 (socket says 172.21.15.83:39588) 2023-11-26T07:15:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe4767cb700 1 -- 172.21.15.83:0/4058347915 learned_addr learned my addr 172.21.15.83:0/4058347915 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe4757c9700 1 --2- 172.21.15.83:0/4058347915 >> v2:172.21.15.203:3300/0 conn(0x7fe4700af4d0 0x7fe4700afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe475fca700 1 --2- 172.21.15.83:0/4058347915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe47015e080 0x7fe4700b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe4767cb700 1 -- 172.21.15.83:0/4058347915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe47015e080 msgr2=0x7fe4700b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:47.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe4767cb700 1 --2- 172.21.15.83:0/4058347915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe47015e080 0x7fe4700b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe4767cb700 1 -- 172.21.15.83:0/4058347915 >> v2:172.21.15.203:3300/0 conn(0x7fe4700af4d0 msgr2=0x7fe4700afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:47.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe4767cb700 1 --2- 172.21.15.83:0/4058347915 >> v2:172.21.15.203:3300/0 conn(0x7fe4700af4d0 0x7fe4700afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe4767cb700 1 -- 172.21.15.83:0/4058347915 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4700ae430 con 0x7fe47006c000 2023-11-26T07:15:47.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe475fca700 1 --2- 172.21.15.83:0/4058347915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe47015e080 0x7fe4700b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:47.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe4757c9700 1 --2- 172.21.15.83:0/4058347915 >> v2:172.21.15.203:3300/0 conn(0x7fe4700af4d0 0x7fe4700afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:47.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe4767cb700 1 --2- 172.21.15.83:0/4058347915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47006c000 0x7fe47006c4b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe47006d380 tx=0x7fe46800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c8777f391a23d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:47.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe474fc8700 1 -- 172.21.15.83:0/4058347915 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800e070 con 0x7fe47006c000 2023-11-26T07:15:47.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe474fc8700 1 -- 172.21.15.83:0/4058347915 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46800b980 con 0x7fe47006c000 2023-11-26T07:15:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.217+0000 7fe474fc8700 1 -- 172.21.15.83:0/4058347915 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800bd30 con 0x7fe47006c000 2023-11-26T07:15:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.218+0000 7fe47822e700 1 -- 172.21.15.83:0/4058347915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47006c000 msgr2=0x7fe47006c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.218+0000 7fe47822e700 1 --2- 172.21.15.83:0/4058347915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47006c000 0x7fe47006c4b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe47006d380 tx=0x7fe46800b330 comp rx=0 tx=0).stop 2023-11-26T07:15:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.218+0000 7fe47822e700 1 -- 172.21.15.83:0/4058347915 shutdown_connections 2023-11-26T07:15:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.218+0000 7fe47822e700 1 --2- 172.21.15.83:0/4058347915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47006c000 0x7fe47006c4b0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.218+0000 7fe47822e700 1 --2- 172.21.15.83:0/4058347915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe47015e080 0x7fe4700b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.218+0000 7fe47822e700 1 --2- 172.21.15.83:0/4058347915 >> v2:172.21.15.203:3300/0 conn(0x7fe4700af4d0 0x7fe4700afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.218+0000 7fe47822e700 1 -- 172.21.15.83:0/4058347915 >> 172.21.15.83:0/4058347915 conn(0x7fe470071b90 msgr2=0x7fe470071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.218+0000 7fe47822e700 1 -- 172.21.15.83:0/4058347915 shutdown_connections 2023-11-26T07:15:47.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.218+0000 7fe47822e700 1 -- 172.21.15.83:0/4058347915 wait complete. 2023-11-26T07:15:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.219+0000 7fe47822e700 1 Processor -- start 2023-11-26T07:15:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.219+0000 7fe47822e700 1 -- start start 2023-11-26T07:15:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe47822e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47006c000 0x7fe4701f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe47822e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4700af4d0 0x7fe470144b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe47822e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe47015e080 0x7fe470159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe47822e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4701f72a0 con 0x7fe47006c000 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe47822e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4701450b0 con 0x7fe47015e080 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe47822e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700d02a0 con 0x7fe4700af4d0 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe475fca700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47006c000 0x7fe4701f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe475fca700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47006c000 0x7fe4701f7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39590/0 (socket says 172.21.15.83:39590) 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe475fca700 1 -- 172.21.15.83:0/1869803357 learned_addr learned my addr 172.21.15.83:0/1869803357 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe475fca700 1 -- 172.21.15.83:0/1869803357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4700af4d0 msgr2=0x7fe470144b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe475fca700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4700af4d0 0x7fe470144b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe475fca700 1 -- 172.21.15.83:0/1869803357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe47015e080 msgr2=0x7fe470159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe4767cb700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe47015e080 0x7fe470159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe475fca700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe47015e080 0x7fe470159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe475fca700 1 -- 172.21.15.83:0/1869803357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe468009b30 con 0x7fe47006c000 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe475fca700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47006c000 0x7fe4701f7730 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c7f0 tx=0x7fe46c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:47.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.220+0000 7fe4627fc700 1 -- 172.21.15.83:0/1869803357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c011800 con 0x7fe47006c000 2023-11-26T07:15:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.221+0000 7fe4627fc700 1 -- 172.21.15.83:0/1869803357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c011960 con 0x7fe47006c000 2023-11-26T07:15:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.221+0000 7fe47822e700 1 -- 172.21.15.83:0/1869803357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4700d0580 con 0x7fe47006c000 2023-11-26T07:15:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.221+0000 7fe4627fc700 1 -- 172.21.15.83:0/1869803357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c011d10 con 0x7fe47006c000 2023-11-26T07:15:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.221+0000 7fe47822e700 1 -- 172.21.15.83:0/1869803357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4701f6f30 con 0x7fe47006c000 2023-11-26T07:15:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.221+0000 7fe47822e700 1 -- 172.21.15.83:0/1869803357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4700753f0 con 0x7fe47006c000 2023-11-26T07:15:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.222+0000 7fe4627fc700 1 -- 172.21.15.83:0/1869803357 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fe46c018360 con 0x7fe47006c000 2023-11-26T07:15:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.222+0000 7fe4627fc700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe45406b230 0x7fe45406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.222+0000 7fe4627fc700 1 -- 172.21.15.83:0/1869803357 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe46c089c50 con 0x7fe47006c000 2023-11-26T07:15:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.222+0000 7fe4757c9700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe45406b230 0x7fe45406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.223+0000 7fe4757c9700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe45406b230 0x7fe45406d6f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe464006ba0 tx=0x7fe464008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:47.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.227+0000 7fe4627fc700 1 -- 172.21.15.83:0/1869803357 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe46c0547e0 con 0x7fe47006c000 2023-11-26T07:15:47.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.410+0000 7fe47822e700 1 -- 172.21.15.83:0/1869803357 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4700ae9f0 con 0x7fe45406b230 2023-11-26T07:15:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.418+0000 7fe4627fc700 1 -- 172.21.15.83:0/1869803357 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289412 (secure 0 0 0) 0x7fe464007d60 con 0x7fe45406b230 2023-11-26T07:15:47.423 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:47.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 -- 172.21.15.83:0/1869803357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe45406b230 msgr2=0x7fe45406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.425 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe45406b230 0x7fe45406d6f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe464006ba0 tx=0x7fe464008040 comp rx=0 tx=0).stop 2023-11-26T07:15:47.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 -- 172.21.15.83:0/1869803357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47006c000 msgr2=0x7fe4701f7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47006c000 0x7fe4701f7730 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c7f0 tx=0x7fe46c00cbb0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 -- 172.21.15.83:0/1869803357 shutdown_connections 2023-11-26T07:15:47.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe47015e080 0x7fe470159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe47006c000 0x7fe4701f7730 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fe45406b230 0x7fe45406d6f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 --2- 172.21.15.83:0/1869803357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4700af4d0 0x7fe470144b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 -- 172.21.15.83:0/1869803357 >> 172.21.15.83:0/1869803357 conn(0x7fe470071b90 msgr2=0x7fe47015d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:47.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 -- 172.21.15.83:0/1869803357 shutdown_connections 2023-11-26T07:15:47.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.421+0000 7fe47822e700 1 -- 172.21.15.83:0/1869803357 wait complete. 2023-11-26T07:15:47.426 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:15:47.435 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2023-11-26T07:15:46.363024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224040,"kb_used_data":7880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750680,"statfs":{"total":773094113280,"available":772864696320,"internally_reserved":0,"allocated":8069120,"data_stored":4052116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.429202"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545255+0000","last_change":"2023-11-26T07:15:42.552509+0000","last_active":"2023-11-26T07:15:43.545255+0000","last_peered":"2023-11-26T07:15:43.545255+0000","last_clean":"2023-11-26T07:15:43.545255+0000","last_became_active":"2023-11-26T07:15:42.552199+0000","last_became_peered":"2023-11-26T07:15:42.552199+0000","last_unstale":"2023-11-26T07:15:43.545255+0000","last_undegraded":"2023-11-26T07:15:43.545255+0000","last_fullsized":"2023-11-26T07:15:43.545255+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:52:22.331524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621617+0000","last_change":"2023-11-26T07:15:42.557781+0000","last_active":"2023-11-26T07:15:43.621617+0000","last_peered":"2023-11-26T07:15:43.621617+0000","last_clean":"2023-11-26T07:15:43.621617+0000","last_became_active":"2023-11-26T07:15:42.556908+0000","last_became_peered":"2023-11-26T07:15:42.556908+0000","last_unstale":"2023-11-26T07:15:43.621617+0000","last_undegraded":"2023-11-26T07:15:43.621617+0000","last_fullsized":"2023-11-26T07:15:43.621617+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:05:32.379392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620481+0000","last_change":"2023-11-26T07:15:42.557805+0000","last_active":"2023-11-26T07:15:43.620481+0000","last_peered":"2023-11-26T07:15:43.620481+0000","last_clean":"2023-11-26T07:15:43.620481+0000","last_became_active":"2023-11-26T07:15:42.557543+0000","last_became_peered":"2023-11-26T07:15:42.557543+0000","last_unstale":"2023-11-26T07:15:43.620481+0000","last_undegraded":"2023-11-26T07:15:43.620481+0000","last_fullsized":"2023-11-26T07:15:43.620481+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:28:59.498960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621299+0000","last_change":"2023-11-26T07:15:42.562855+0000","last_active":"2023-11-26T07:15:43.621299+0000","last_peered":"2023-11-26T07:15:43.621299+0000","last_clean":"2023-11-26T07:15:43.621299+0000","last_became_active":"2023-11-26T07:15:42.562588+0000","last_became_peered":"2023-11-26T07:15:42.562588+0000","last_unstale":"2023-11-26T07:15:43.621299+0000","last_undegraded":"2023-11-26T07:15:43.621299+0000","last_fullsized":"2023-11-26T07:15:43.621299+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:49:36.390962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620439+0000","last_change":"2023-11-26T07:15:42.548063+0000","last_active":"2023-11-26T07:15:43.620439+0000","last_peered":"2023-11-26T07:15:43.620439+0000","last_clean":"2023-11-26T07:15:43.620439+0000","last_became_active":"2023-11-26T07:15:42.547808+0000","last_became_peered":"2023-11-26T07:15:42.547808+0000","last_unstale":"2023-11-26T07:15:43.620439+0000","last_undegraded":"2023-11-26T07:15:43.620439+0000","last_fullsized":"2023-11-26T07:15:43.620439+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:43:01.257824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621311+0000","last_change":"2023-11-26T07:15:42.557161+0000","last_active":"2023-11-26T07:15:43.621311+0000","last_peered":"2023-11-26T07:15:43.621311+0000","last_clean":"2023-11-26T07:15:43.621311+0000","last_became_active":"2023-11-26T07:15:42.556877+0000","last_became_peered":"2023-11-26T07:15:42.556877+0000","last_unstale":"2023-11-26T07:15:43.621311+0000","last_undegraded":"2023-11-26T07:15:43.621311+0000","last_fullsized":"2023-11-26T07:15:43.621311+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:06.810126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620496+0000","last_change":"2023-11-26T07:15:42.564899+0000","last_active":"2023-11-26T07:15:43.620496+0000","last_peered":"2023-11-26T07:15:43.620496+0000","last_clean":"2023-11-26T07:15:43.620496+0000","last_became_active":"2023-11-26T07:15:42.564475+0000","last_became_peered":"2023-11-26T07:15:42.564475+0000","last_unstale":"2023-11-26T07:15:43.620496+0000","last_undegraded":"2023-11-26T07:15:43.620496+0000","last_fullsized":"2023-11-26T07:15:43.620496+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:45.536813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621041+0000","last_change":"2023-11-26T07:15:42.560477+0000","last_active":"2023-11-26T07:15:43.621041+0000","last_peered":"2023-11-26T07:15:43.621041+0000","last_clean":"2023-11-26T07:15:43.621041+0000","last_became_active":"2023-11-26T07:15:42.557362+0000","last_became_peered":"2023-11-26T07:15:42.557362+0000","last_unstale":"2023-11-26T07:15:43.621041+0000","last_undegraded":"2023-11-26T07:15:43.621041+0000","last_fullsized":"2023-11-26T07:15:43.621041+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:08:18.050397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620548+0000","last_change":"2023-11-26T07:15:42.556207+0000","last_active":"2023-11-26T07:15:43.620548+0000","last_peered":"2023-11-26T07:15:43.620548+0000","last_clean":"2023-11-26T07:15:43.620548+0000","last_became_active":"2023-11-26T07:15:42.555875+0000","last_became_peered":"2023-11-26T07:15:42.555875+0000","last_unstale":"2023-11-26T07:15:43.620548+0000","last_undegraded":"2023-11-26T07:15:43.620548+0000","last_fullsized":"2023-11-26T07:15:43.620548+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:16:06.747989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543562+0000","last_change":"2023-11-26T07:15:42.552732+0000","last_active":"2023-11-26T07:15:43.543562+0000","last_peered":"2023-11-26T07:15:43.543562+0000","last_clean":"2023-11-26T07:15:43.543562+0000","last_became_active":"2023-11-26T07:15:42.551602+0000","last_became_peered":"2023-11-26T07:15:42.551602+0000","last_unstale":"2023-11-26T07:15:43.543562+0000","last_undegraded":"2023-11-26T07:15:43.543562+0000","last_fullsized":"2023-11-26T07:15:43.543562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:57.146358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545141+0000","last_change":"2023-11-26T07:15:42.555561+0000","last_active":"2023-11-26T07:15:43.545141+0000","last_peered":"2023-11-26T07:15:43.545141+0000","last_clean":"2023-11-26T07:15:43.545141+0000","last_became_active":"2023-11-26T07:15:42.555261+0000","last_became_peered":"2023-11-26T07:15:42.555261+0000","last_unstale":"2023-11-26T07:15:43.545141+0000","last_undegraded":"2023-11-26T07:15:43.545141+0000","last_fullsized":"2023-11-26T07:15:43.545141+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:56:36.376641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621452+0000","last_change":"2023-11-26T07:15:42.558281+0000","last_active":"2023-11-26T07:15:43.621452+0000","last_peered":"2023-11-26T07:15:43.621452+0000","last_clean":"2023-11-26T07:15:43.621452+0000","last_became_active":"2023-11-26T07:15:42.557978+0000","last_became_peered":"2023-11-26T07:15:42.557978+0000","last_unstale":"2023-11-26T07:15:43.621452+0000","last_undegraded":"2023-11-26T07:15:43.621452+0000","last_fullsized":"2023-11-26T07:15:43.621452+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:02.439521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542168+0000","last_change":"2023-11-26T07:15:42.560447+0000","last_active":"2023-11-26T07:15:43.542168+0000","last_peered":"2023-11-26T07:15:43.542168+0000","last_clean":"2023-11-26T07:15:43.542168+0000","last_became_active":"2023-11-26T07:15:42.560172+0000","last_became_peered":"2023-11-26T07:15:42.560172+0000","last_unstale":"2023-11-26T07:15:43.542168+0000","last_undegraded":"2023-11-26T07:15:43.542168+0000","last_fullsized":"2023-11-26T07:15:43.542168+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:25:13.533544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620901+0000","last_change":"2023-11-26T07:15:42.563807+0000","last_active":"2023-11-26T07:15:43.620901+0000","last_peered":"2023-11-26T07:15:43.620901+0000","last_clean":"2023-11-26T07:15:43.620901+0000","last_became_active":"2023-11-26T07:15:42.563540+0000","last_became_peered":"2023-11-26T07:15:42.563540+0000","last_unstale":"2023-11-26T07:15:43.620901+0000","last_undegraded":"2023-11-26T07:15:43.620901+0000","last_fullsized":"2023-11-26T07:15:43.620901+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:01:26.363020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542015+0000","last_change":"2023-11-26T07:15:42.553786+0000","last_active":"2023-11-26T07:15:43.542015+0000","last_peered":"2023-11-26T07:15:43.542015+0000","last_clean":"2023-11-26T07:15:43.542015+0000","last_became_active":"2023-11-26T07:15:42.553509+0000","last_became_peered":"2023-11-26T07:15:42.553509+0000","last_unstale":"2023-11-26T07:15:43.542015+0000","last_undegraded":"2023-11-26T07:15:43.542015+0000","last_fullsized":"2023-11-26T07:15:43.542015+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:45.536813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543232+0000","last_change":"2023-11-26T07:15:42.554794+0000","last_active":"2023-11-26T07:15:43.543232+0000","last_peered":"2023-11-26T07:15:43.543232+0000","last_clean":"2023-11-26T07:15:43.543232+0000","last_became_active":"2023-11-26T07:15:42.554394+0000","last_became_peered":"2023-11-26T07:15:42.554394+0000","last_unstale":"2023-11-26T07:15:43.543232+0000","last_undegraded":"2023-11-26T07:15:43.543232+0000","last_fullsized":"2023-11-26T07:15:43.543232+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:06.810126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620933+0000","last_change":"2023-11-26T07:15:42.555104+0000","last_active":"2023-11-26T07:15:43.620933+0000","last_peered":"2023-11-26T07:15:43.620933+0000","last_clean":"2023-11-26T07:15:43.620933+0000","last_became_active":"2023-11-26T07:15:42.554486+0000","last_became_peered":"2023-11-26T07:15:42.554486+0000","last_unstale":"2023-11-26T07:15:43.620933+0000","last_undegraded":"2023-11-26T07:15:43.620933+0000","last_fullsized":"2023-11-26T07:15:43.620933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:56.979892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621628+0000","last_change":"2023-11-26T07:15:42.562465+0000","last_active":"2023-11-26T07:15:43.621628+0000","last_peered":"2023-11-26T07:15:43.621628+0000","last_clean":"2023-11-26T07:15:43.621628+0000","last_became_active":"2023-11-26T07:15:42.562131+0000","last_became_peered":"2023-11-26T07:15:42.562131+0000","last_unstale":"2023-11-26T07:15:43.621628+0000","last_undegraded":"2023-11-26T07:15:43.621628+0000","last_fullsized":"2023-11-26T07:15:43.621628+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:45:31.391520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621047+0000","last_change":"2023-11-26T07:15:42.557482+0000","last_active":"2023-11-26T07:15:43.621047+0000","last_peered":"2023-11-26T07:15:43.621047+0000","last_clean":"2023-11-26T07:15:43.621047+0000","last_became_active":"2023-11-26T07:15:42.557202+0000","last_became_peered":"2023-11-26T07:15:42.557202+0000","last_unstale":"2023-11-26T07:15:43.621047+0000","last_undegraded":"2023-11-26T07:15:43.621047+0000","last_fullsized":"2023-11-26T07:15:43.621047+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:21.644505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542011+0000","last_change":"2023-11-26T07:15:42.561478+0000","last_active":"2023-11-26T07:15:43.542011+0000","last_peered":"2023-11-26T07:15:43.542011+0000","last_clean":"2023-11-26T07:15:43.542011+0000","last_became_active":"2023-11-26T07:15:42.561228+0000","last_became_peered":"2023-11-26T07:15:42.561228+0000","last_unstale":"2023-11-26T07:15:43.542011+0000","last_undegraded":"2023-11-26T07:15:43.542011+0000","last_fullsized":"2023-11-26T07:15:43.542011+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:03:17.847732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543682+0000","last_change":"2023-11-26T07:15:42.552816+0000","last_active":"2023-11-26T07:15:43.543682+0000","last_peered":"2023-11-26T07:15:43.543682+0000","last_clean":"2023-11-26T07:15:43.543682+0000","last_became_active":"2023-11-26T07:15:42.552562+0000","last_became_peered":"2023-11-26T07:15:42.552562+0000","last_unstale":"2023-11-26T07:15:43.543682+0000","last_undegraded":"2023-11-26T07:15:43.543682+0000","last_fullsized":"2023-11-26T07:15:43.543682+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:05:32.379392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543694+0000","last_change":"2023-11-26T07:15:42.553637+0000","last_active":"2023-11-26T07:15:43.543694+0000","last_peered":"2023-11-26T07:15:43.543694+0000","last_clean":"2023-11-26T07:15:43.543694+0000","last_became_active":"2023-11-26T07:15:42.553362+0000","last_became_peered":"2023-11-26T07:15:42.553362+0000","last_unstale":"2023-11-26T07:15:43.543694+0000","last_undegraded":"2023-11-26T07:15:43.543694+0000","last_fullsized":"2023-11-26T07:15:43.543694+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:09:21.961069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.544910+0000","last_change":"2023-11-26T07:15:42.559524+0000","last_active":"2023-11-26T07:15:43.544910+0000","last_peered":"2023-11-26T07:15:43.544910+0000","last_clean":"2023-11-26T07:15:43.544910+0000","last_became_active":"2023-11-26T07:15:42.559241+0000","last_became_peered":"2023-11-26T07:15:42.559241+0000","last_unstale":"2023-11-26T07:15:43.544910+0000","last_undegraded":"2023-11-26T07:15:43.544910+0000","last_fullsized":"2023-11-26T07:15:43.544910+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:30:40.488087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.544718+0000","last_change":"2023-11-26T07:15:42.554290+0000","last_active":"2023-11-26T07:15:43.544718+0000","last_peered":"2023-11-26T07:15:43.544718+0000","last_clean":"2023-11-26T07:15:43.544718+0000","last_became_active":"2023-11-26T07:15:42.553994+0000","last_became_peered":"2023-11-26T07:15:42.553994+0000","last_unstale":"2023-11-26T07:15:43.544718+0000","last_undegraded":"2023-11-26T07:15:43.544718+0000","last_fullsized":"2023-11-26T07:15:43.544718+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:45:47.838341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543617+0000","last_change":"2023-11-26T07:15:42.546241+0000","last_active":"2023-11-26T07:15:43.543617+0000","last_peered":"2023-11-26T07:15:43.543617+0000","last_clean":"2023-11-26T07:15:43.543617+0000","last_became_active":"2023-11-26T07:15:42.545953+0000","last_became_peered":"2023-11-26T07:15:42.545953+0000","last_unstale":"2023-11-26T07:15:43.543617+0000","last_undegraded":"2023-11-26T07:15:43.543617+0000","last_fullsized":"2023-11-26T07:15:43.543617+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:48:24.866444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621043+0000","last_change":"2023-11-26T07:15:42.563104+0000","last_active":"2023-11-26T07:15:43.621043+0000","last_peered":"2023-11-26T07:15:43.621043+0000","last_clean":"2023-11-26T07:15:43.621043+0000","last_became_active":"2023-11-26T07:15:42.562801+0000","last_became_peered":"2023-11-26T07:15:42.562801+0000","last_unstale":"2023-11-26T07:15:43.621043+0000","last_undegraded":"2023-11-26T07:15:43.621043+0000","last_fullsized":"2023-11-26T07:15:43.621043+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:42:42.350198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620600+0000","last_change":"2023-11-26T07:15:42.558604+0000","last_active":"2023-11-26T07:15:43.620600+0000","last_peered":"2023-11-26T07:15:43.620600+0000","last_clean":"2023-11-26T07:15:43.620600+0000","last_became_active":"2023-11-26T07:15:42.558348+0000","last_became_peered":"2023-11-26T07:15:42.558348+0000","last_unstale":"2023-11-26T07:15:43.620600+0000","last_undegraded":"2023-11-26T07:15:43.620600+0000","last_fullsized":"2023-11-26T07:15:43.620600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:09:21.919185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543997+0000","last_change":"2023-11-26T07:15:42.551834+0000","last_active":"2023-11-26T07:15:43.543997+0000","last_peered":"2023-11-26T07:15:43.543997+0000","last_clean":"2023-11-26T07:15:43.543997+0000","last_became_active":"2023-11-26T07:15:42.551526+0000","last_became_peered":"2023-11-26T07:15:42.551526+0000","last_unstale":"2023-11-26T07:15:43.543997+0000","last_undegraded":"2023-11-26T07:15:43.543997+0000","last_fullsized":"2023-11-26T07:15:43.543997+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:16:25.381902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620148+0000","last_change":"2023-11-26T07:15:42.561995+0000","last_active":"2023-11-26T07:15:43.620148+0000","last_peered":"2023-11-26T07:15:43.620148+0000","last_clean":"2023-11-26T07:15:43.620148+0000","last_became_active":"2023-11-26T07:15:42.561737+0000","last_became_peered":"2023-11-26T07:15:42.561737+0000","last_unstale":"2023-11-26T07:15:43.620148+0000","last_undegraded":"2023-11-26T07:15:43.620148+0000","last_fullsized":"2023-11-26T07:15:43.620148+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:56:17.089456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621199+0000","last_change":"2023-11-26T07:15:42.563910+0000","last_active":"2023-11-26T07:15:43.621199+0000","last_peered":"2023-11-26T07:15:43.621199+0000","last_clean":"2023-11-26T07:15:43.621199+0000","last_became_active":"2023-11-26T07:15:42.563652+0000","last_became_peered":"2023-11-26T07:15:42.563652+0000","last_unstale":"2023-11-26T07:15:43.621199+0000","last_undegraded":"2023-11-26T07:15:43.621199+0000","last_fullsized":"2023-11-26T07:15:43.621199+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:56:47.443320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620559+0000","last_change":"2023-11-26T07:15:42.565019+0000","last_active":"2023-11-26T07:15:43.620559+0000","last_peered":"2023-11-26T07:15:43.620559+0000","last_clean":"2023-11-26T07:15:43.620559+0000","last_became_active":"2023-11-26T07:15:42.564700+0000","last_became_peered":"2023-11-26T07:15:42.564700+0000","last_unstale":"2023-11-26T07:15:43.620559+0000","last_undegraded":"2023-11-26T07:15:43.620559+0000","last_fullsized":"2023-11-26T07:15:43.620559+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:03.919981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620750+0000","last_change":"2023-11-26T07:15:42.557088+0000","last_active":"2023-11-26T07:15:43.620750+0000","last_peered":"2023-11-26T07:15:43.620750+0000","last_clean":"2023-11-26T07:15:43.620750+0000","last_became_active":"2023-11-26T07:15:42.556803+0000","last_became_peered":"2023-11-26T07:15:42.556803+0000","last_unstale":"2023-11-26T07:15:43.620750+0000","last_undegraded":"2023-11-26T07:15:43.620750+0000","last_fullsized":"2023-11-26T07:15:43.620750+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:13:50.560218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545235+0000","last_change":"2023-11-26T07:15:42.554231+0000","last_active":"2023-11-26T07:15:43.545235+0000","last_peered":"2023-11-26T07:15:43.545235+0000","last_clean":"2023-11-26T07:15:43.545235+0000","last_became_active":"2023-11-26T07:15:42.553934+0000","last_became_peered":"2023-11-26T07:15:42.553934+0000","last_unstale":"2023-11-26T07:15:43.545235+0000","last_undegraded":"2023-11-26T07:15:43.545235+0000","last_fullsized":"2023-11-26T07:15:43.545235+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:36:36.176509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545359+0000","last_change":"2023-11-26T07:15:41.953970+0000","last_active":"2023-11-26T07:15:43.545359+0000","last_peered":"2023-11-26T07:15:43.545359+0000","last_clean":"2023-11-26T07:15:43.545359+0000","last_became_active":"2023-11-26T07:15:41.953685+0000","last_became_peered":"2023-11-26T07:15:41.953685+0000","last_unstale":"2023-11-26T07:15:43.545359+0000","last_undegraded":"2023-11-26T07:15:43.545359+0000","last_fullsized":"2023-11-26T07:15:43.545359+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:37:47.890164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620785+0000","last_change":"2023-11-26T07:15:42.555141+0000","last_active":"2023-11-26T07:15:43.620785+0000","last_peered":"2023-11-26T07:15:43.620785+0000","last_clean":"2023-11-26T07:15:43.620785+0000","last_became_active":"2023-11-26T07:15:42.554838+0000","last_became_peered":"2023-11-26T07:15:42.554838+0000","last_unstale":"2023-11-26T07:15:43.620785+0000","last_undegraded":"2023-11-26T07:15:43.620785+0000","last_fullsized":"2023-11-26T07:15:43.620785+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:22:45.354130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542361+0000","last_change":"2023-11-26T07:15:41.950958+0000","last_active":"2023-11-26T07:15:43.542361+0000","last_peered":"2023-11-26T07:15:43.542361+0000","last_clean":"2023-11-26T07:15:43.542361+0000","last_became_active":"2023-11-26T07:15:41.950661+0000","last_became_peered":"2023-11-26T07:15:41.950661+0000","last_unstale":"2023-11-26T07:15:43.542361+0000","last_undegraded":"2023-11-26T07:15:43.542361+0000","last_fullsized":"2023-11-26T07:15:43.542361+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:28:47.202070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621361+0000","last_change":"2023-11-26T07:15:42.560738+0000","last_active":"2023-11-26T07:15:43.621361+0000","last_peered":"2023-11-26T07:15:43.621361+0000","last_clean":"2023-11-26T07:15:43.621361+0000","last_became_active":"2023-11-26T07:15:42.558226+0000","last_became_peered":"2023-11-26T07:15:42.558226+0000","last_unstale":"2023-11-26T07:15:43.621361+0000","last_undegraded":"2023-11-26T07:15:43.621361+0000","last_fullsized":"2023-11-26T07:15:43.621361+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:08:57.146358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543728+0000","last_change":"2023-11-26T07:15:41.953804+0000","last_active":"2023-11-26T07:15:43.543728+0000","last_peered":"2023-11-26T07:15:43.543728+0000","last_clean":"2023-11-26T07:15:43.543728+0000","last_became_active":"2023-11-26T07:15:41.953506+0000","last_became_peered":"2023-11-26T07:15:41.953506+0000","last_unstale":"2023-11-26T07:15:43.543728+0000","last_undegraded":"2023-11-26T07:15:43.543728+0000","last_fullsized":"2023-11-26T07:15:43.543728+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:21:59.824336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545130+0000","last_change":"2023-11-26T07:15:42.553374+0000","last_active":"2023-11-26T07:15:43.545130+0000","last_peered":"2023-11-26T07:15:43.545130+0000","last_clean":"2023-11-26T07:15:43.545130+0000","last_became_active":"2023-11-26T07:15:42.553066+0000","last_became_peered":"2023-11-26T07:15:42.553066+0000","last_unstale":"2023-11-26T07:15:43.545130+0000","last_undegraded":"2023-11-26T07:15:43.545130+0000","last_fullsized":"2023-11-26T07:15:43.545130+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:57.454676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621528+0000","last_change":"2023-11-26T07:15:41.955919+0000","last_active":"2023-11-26T07:15:43.621528+0000","last_peered":"2023-11-26T07:15:43.621528+0000","last_clean":"2023-11-26T07:15:43.621528+0000","last_became_active":"2023-11-26T07:15:41.955649+0000","last_became_peered":"2023-11-26T07:15:41.955649+0000","last_unstale":"2023-11-26T07:15:43.621528+0000","last_undegraded":"2023-11-26T07:15:43.621528+0000","last_fullsized":"2023-11-26T07:15:43.621528+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:01.435187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621147+0000","last_change":"2023-11-26T07:15:42.562446+0000","last_active":"2023-11-26T07:15:43.621147+0000","last_peered":"2023-11-26T07:15:43.621147+0000","last_clean":"2023-11-26T07:15:43.621147+0000","last_became_active":"2023-11-26T07:15:42.562164+0000","last_became_peered":"2023-11-26T07:15:42.562164+0000","last_unstale":"2023-11-26T07:15:43.621147+0000","last_undegraded":"2023-11-26T07:15:43.621147+0000","last_fullsized":"2023-11-26T07:15:43.621147+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:39.602824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620288+0000","last_change":"2023-11-26T07:15:41.956255+0000","last_active":"2023-11-26T07:15:43.620288+0000","last_peered":"2023-11-26T07:15:43.620288+0000","last_clean":"2023-11-26T07:15:43.620288+0000","last_became_active":"2023-11-26T07:15:41.956021+0000","last_became_peered":"2023-11-26T07:15:41.956021+0000","last_unstale":"2023-11-26T07:15:43.620288+0000","last_undegraded":"2023-11-26T07:15:43.620288+0000","last_fullsized":"2023-11-26T07:15:43.620288+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:36:35.172175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543847+0000","last_change":"2023-11-26T07:15:42.547264+0000","last_active":"2023-11-26T07:15:43.543847+0000","last_peered":"2023-11-26T07:15:43.543847+0000","last_clean":"2023-11-26T07:15:43.543847+0000","last_became_active":"2023-11-26T07:15:42.547004+0000","last_became_peered":"2023-11-26T07:15:42.547004+0000","last_unstale":"2023-11-26T07:15:43.543847+0000","last_undegraded":"2023-11-26T07:15:43.543847+0000","last_fullsized":"2023-11-26T07:15:43.543847+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:44.801698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621158+0000","last_change":"2023-11-26T07:15:41.956982+0000","last_active":"2023-11-26T07:15:43.621158+0000","last_peered":"2023-11-26T07:15:43.621158+0000","last_clean":"2023-11-26T07:15:43.621158+0000","last_became_active":"2023-11-26T07:15:41.956550+0000","last_became_peered":"2023-11-26T07:15:41.956550+0000","last_unstale":"2023-11-26T07:15:43.621158+0000","last_undegraded":"2023-11-26T07:15:43.621158+0000","last_fullsized":"2023-11-26T07:15:43.621158+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:44.333650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620520+0000","last_change":"2023-11-26T07:15:42.549320+0000","last_active":"2023-11-26T07:15:43.620520+0000","last_peered":"2023-11-26T07:15:43.620520+0000","last_clean":"2023-11-26T07:15:43.620520+0000","last_became_active":"2023-11-26T07:15:42.549146+0000","last_became_peered":"2023-11-26T07:15:42.549146+0000","last_unstale":"2023-11-26T07:15:43.620520+0000","last_undegraded":"2023-11-26T07:15:43.620520+0000","last_fullsized":"2023-11-26T07:15:43.620520+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:07.502524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621125+0000","last_change":"2023-11-26T07:15:41.951596+0000","last_active":"2023-11-26T07:15:43.621125+0000","last_peered":"2023-11-26T07:15:43.621125+0000","last_clean":"2023-11-26T07:15:43.621125+0000","last_became_active":"2023-11-26T07:15:41.951283+0000","last_became_peered":"2023-11-26T07:15:41.951283+0000","last_unstale":"2023-11-26T07:15:43.621125+0000","last_undegraded":"2023-11-26T07:15:43.621125+0000","last_fullsized":"2023-11-26T07:15:43.621125+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:55.975559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542160+0000","last_change":"2023-11-26T07:15:42.551403+0000","last_active":"2023-11-26T07:15:43.542160+0000","last_peered":"2023-11-26T07:15:43.542160+0000","last_clean":"2023-11-26T07:15:43.542160+0000","last_became_active":"2023-11-26T07:15:42.551153+0000","last_became_peered":"2023-11-26T07:15:42.551153+0000","last_unstale":"2023-11-26T07:15:43.542160+0000","last_undegraded":"2023-11-26T07:15:43.542160+0000","last_fullsized":"2023-11-26T07:15:43.542160+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:03.919981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620093+0000","last_change":"2023-11-26T07:15:41.952488+0000","last_active":"2023-11-26T07:15:43.620093+0000","last_peered":"2023-11-26T07:15:43.620093+0000","last_clean":"2023-11-26T07:15:43.620093+0000","last_became_active":"2023-11-26T07:15:41.952237+0000","last_became_peered":"2023-11-26T07:15:41.952237+0000","last_unstale":"2023-11-26T07:15:43.620093+0000","last_undegraded":"2023-11-26T07:15:43.620093+0000","last_fullsized":"2023-11-26T07:15:43.620093+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:15:16.108512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621030+0000","last_change":"2023-11-26T07:15:42.560224+0000","last_active":"2023-11-26T07:15:43.621030+0000","last_peered":"2023-11-26T07:15:43.621030+0000","last_clean":"2023-11-26T07:15:43.621030+0000","last_became_active":"2023-11-26T07:15:42.559941+0000","last_became_peered":"2023-11-26T07:15:42.559941+0000","last_unstale":"2023-11-26T07:15:43.621030+0000","last_undegraded":"2023-11-26T07:15:43.621030+0000","last_fullsized":"2023-11-26T07:15:43.621030+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:37:48.894498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543722+0000","last_change":"2023-11-26T07:15:41.959804+0000","last_active":"2023-11-26T07:15:43.543722+0000","last_peered":"2023-11-26T07:15:43.543722+0000","last_clean":"2023-11-26T07:15:43.543722+0000","last_became_active":"2023-11-26T07:15:41.959514+0000","last_became_peered":"2023-11-26T07:15:41.959514+0000","last_unstale":"2023-11-26T07:15:43.543722+0000","last_undegraded":"2023-11-26T07:15:43.543722+0000","last_fullsized":"2023-11-26T07:15:43.543722+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:08:17.046063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545276+0000","last_change":"2023-11-26T07:15:42.554213+0000","last_active":"2023-11-26T07:15:43.545276+0000","last_peered":"2023-11-26T07:15:43.545276+0000","last_clean":"2023-11-26T07:15:43.545276+0000","last_became_active":"2023-11-26T07:15:42.553928+0000","last_became_peered":"2023-11-26T07:15:42.553928+0000","last_unstale":"2023-11-26T07:15:43.545276+0000","last_undegraded":"2023-11-26T07:15:43.545276+0000","last_fullsized":"2023-11-26T07:15:43.545276+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:51.589701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621686+0000","last_change":"2023-11-26T07:15:41.959972+0000","last_active":"2023-11-26T07:15:43.621686+0000","last_peered":"2023-11-26T07:15:43.621686+0000","last_clean":"2023-11-26T07:15:43.621686+0000","last_became_active":"2023-11-26T07:15:41.959106+0000","last_became_peered":"2023-11-26T07:15:41.959106+0000","last_unstale":"2023-11-26T07:15:43.621686+0000","last_undegraded":"2023-11-26T07:15:43.621686+0000","last_fullsized":"2023-11-26T07:15:43.621686+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:16:33.088448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621415+0000","last_change":"2023-11-26T07:15:42.561135+0000","last_active":"2023-11-26T07:15:43.621415+0000","last_peered":"2023-11-26T07:15:43.621415+0000","last_clean":"2023-11-26T07:15:43.621415+0000","last_became_active":"2023-11-26T07:15:42.557955+0000","last_became_peered":"2023-11-26T07:15:42.557955+0000","last_unstale":"2023-11-26T07:15:43.621415+0000","last_undegraded":"2023-11-26T07:15:43.621415+0000","last_fullsized":"2023-11-26T07:15:43.621415+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:48:24.866444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620683+0000","last_change":"2023-11-26T07:15:41.959510+0000","last_active":"2023-11-26T07:15:43.620683+0000","last_peered":"2023-11-26T07:15:43.620683+0000","last_clean":"2023-11-26T07:15:43.620683+0000","last_became_active":"2023-11-26T07:15:41.959203+0000","last_became_peered":"2023-11-26T07:15:41.959203+0000","last_unstale":"2023-11-26T07:15:43.620683+0000","last_undegraded":"2023-11-26T07:15:43.620683+0000","last_fullsized":"2023-11-26T07:15:43.620683+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:09:18.105341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620501+0000","last_change":"2023-11-26T07:15:42.556469+0000","last_active":"2023-11-26T07:15:43.620501+0000","last_peered":"2023-11-26T07:15:43.620501+0000","last_clean":"2023-11-26T07:15:43.620501+0000","last_became_active":"2023-11-26T07:15:42.556200+0000","last_became_peered":"2023-11-26T07:15:42.556200+0000","last_unstale":"2023-11-26T07:15:43.620501+0000","last_undegraded":"2023-11-26T07:15:43.620501+0000","last_fullsized":"2023-11-26T07:15:43.620501+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:55:45.155216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"69'6","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621534+0000","last_change":"2023-11-26T07:15:41.952043+0000","last_active":"2023-11-26T07:15:43.621534+0000","last_peered":"2023-11-26T07:15:43.621534+0000","last_clean":"2023-11-26T07:15:43.621534+0000","last_became_active":"2023-11-26T07:15:41.951786+0000","last_became_peered":"2023-11-26T07:15:41.951786+0000","last_unstale":"2023-11-26T07:15:43.621534+0000","last_undegraded":"2023-11-26T07:15:43.621534+0000","last_fullsized":"2023-11-26T07:15:43.621534+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:50.064029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.544858+0000","last_change":"2023-11-26T07:15:42.552333+0000","last_active":"2023-11-26T07:15:43.544858+0000","last_peered":"2023-11-26T07:15:43.544858+0000","last_clean":"2023-11-26T07:15:43.544858+0000","last_became_active":"2023-11-26T07:15:42.552042+0000","last_became_peered":"2023-11-26T07:15:42.552042+0000","last_unstale":"2023-11-26T07:15:43.544858+0000","last_undegraded":"2023-11-26T07:15:43.544858+0000","last_fullsized":"2023-11-26T07:15:43.544858+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:23:21.337577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621619+0000","last_change":"2023-11-26T07:15:41.951469+0000","last_active":"2023-11-26T07:15:43.621619+0000","last_peered":"2023-11-26T07:15:43.621619+0000","last_clean":"2023-11-26T07:15:43.621619+0000","last_became_active":"2023-11-26T07:15:41.951172+0000","last_became_peered":"2023-11-26T07:15:41.951172+0000","last_unstale":"2023-11-26T07:15:43.621619+0000","last_undegraded":"2023-11-26T07:15:43.621619+0000","last_fullsized":"2023-11-26T07:15:43.621619+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:39:53.351270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621363+0000","last_change":"2023-11-26T07:15:42.560219+0000","last_active":"2023-11-26T07:15:43.621363+0000","last_peered":"2023-11-26T07:15:43.621363+0000","last_clean":"2023-11-26T07:15:43.621363+0000","last_became_active":"2023-11-26T07:15:42.559966+0000","last_became_peered":"2023-11-26T07:15:42.559966+0000","last_unstale":"2023-11-26T07:15:43.621363+0000","last_undegraded":"2023-11-26T07:15:43.621363+0000","last_fullsized":"2023-11-26T07:15:43.621363+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:24:21.592715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621249+0000","last_change":"2023-11-26T07:15:41.956792+0000","last_active":"2023-11-26T07:15:43.621249+0000","last_peered":"2023-11-26T07:15:43.621249+0000","last_clean":"2023-11-26T07:15:43.621249+0000","last_became_active":"2023-11-26T07:15:41.956267+0000","last_became_peered":"2023-11-26T07:15:41.956267+0000","last_unstale":"2023-11-26T07:15:43.621249+0000","last_undegraded":"2023-11-26T07:15:43.621249+0000","last_fullsized":"2023-11-26T07:15:43.621249+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:48.935177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.541921+0000","last_change":"2023-11-26T07:15:42.562248+0000","last_active":"2023-11-26T07:15:43.541921+0000","last_peered":"2023-11-26T07:15:43.541921+0000","last_clean":"2023-11-26T07:15:43.541921+0000","last_became_active":"2023-11-26T07:15:42.561971+0000","last_became_peered":"2023-11-26T07:15:42.561971+0000","last_unstale":"2023-11-26T07:15:43.541921+0000","last_undegraded":"2023-11-26T07:15:43.541921+0000","last_fullsized":"2023-11-26T07:15:43.541921+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:09:14.461759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620661+0000","last_change":"2023-11-26T07:15:41.959413+0000","last_active":"2023-11-26T07:15:43.620661+0000","last_peered":"2023-11-26T07:15:43.620661+0000","last_clean":"2023-11-26T07:15:43.620661+0000","last_became_active":"2023-11-26T07:15:41.959137+0000","last_became_peered":"2023-11-26T07:15:41.959137+0000","last_unstale":"2023-11-26T07:15:43.620661+0000","last_undegraded":"2023-11-26T07:15:43.620661+0000","last_fullsized":"2023-11-26T07:15:43.620661+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:09:13.457425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620285+0000","last_change":"2023-11-26T07:15:42.556041+0000","last_active":"2023-11-26T07:15:43.620285+0000","last_peered":"2023-11-26T07:15:43.620285+0000","last_clean":"2023-11-26T07:15:43.620285+0000","last_became_active":"2023-11-26T07:15:42.555799+0000","last_became_peered":"2023-11-26T07:15:42.555799+0000","last_unstale":"2023-11-26T07:15:43.620285+0000","last_undegraded":"2023-11-26T07:15:43.620285+0000","last_fullsized":"2023-11-26T07:15:43.620285+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:30.693006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543293+0000","last_change":"2023-11-26T07:15:41.949566+0000","last_active":"2023-11-26T07:15:43.543293+0000","last_peered":"2023-11-26T07:15:43.543293+0000","last_clean":"2023-11-26T07:15:43.543293+0000","last_became_active":"2023-11-26T07:15:41.949160+0000","last_became_peered":"2023-11-26T07:15:41.949160+0000","last_unstale":"2023-11-26T07:15:43.543293+0000","last_undegraded":"2023-11-26T07:15:43.543293+0000","last_fullsized":"2023-11-26T07:15:43.543293+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:36.817478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543754+0000","last_change":"2023-11-26T07:15:42.555690+0000","last_active":"2023-11-26T07:15:43.543754+0000","last_peered":"2023-11-26T07:15:43.543754+0000","last_clean":"2023-11-26T07:15:43.543754+0000","last_became_active":"2023-11-26T07:15:42.555401+0000","last_became_peered":"2023-11-26T07:15:42.555401+0000","last_unstale":"2023-11-26T07:15:43.543754+0000","last_undegraded":"2023-11-26T07:15:43.543754+0000","last_fullsized":"2023-11-26T07:15:43.543754+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:02:39.913059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621119+0000","last_change":"2023-11-26T07:15:41.959661+0000","last_active":"2023-11-26T07:15:43.621119+0000","last_peered":"2023-11-26T07:15:43.621119+0000","last_clean":"2023-11-26T07:15:43.621119+0000","last_became_active":"2023-11-26T07:15:41.959424+0000","last_became_peered":"2023-11-26T07:15:41.959424+0000","last_unstale":"2023-11-26T07:15:43.621119+0000","last_undegraded":"2023-11-26T07:15:43.621119+0000","last_fullsized":"2023-11-26T07:15:43.621119+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:02:45.769127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545371+0000","last_change":"2023-11-26T07:15:42.555994+0000","last_active":"2023-11-26T07:15:43.545371+0000","last_peered":"2023-11-26T07:15:43.545371+0000","last_clean":"2023-11-26T07:15:43.545371+0000","last_became_active":"2023-11-26T07:15:42.554965+0000","last_became_peered":"2023-11-26T07:15:42.554965+0000","last_unstale":"2023-11-26T07:15:43.545371+0000","last_undegraded":"2023-11-26T07:15:43.545371+0000","last_fullsized":"2023-11-26T07:15:43.545371+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:42:12.777618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543863+0000","last_change":"2023-11-26T07:15:41.952695+0000","last_active":"2023-11-26T07:15:43.543863+0000","last_peered":"2023-11-26T07:15:43.543863+0000","last_clean":"2023-11-26T07:15:43.543863+0000","last_became_active":"2023-11-26T07:15:41.952386+0000","last_became_peered":"2023-11-26T07:15:41.952386+0000","last_unstale":"2023-11-26T07:15:43.543863+0000","last_undegraded":"2023-11-26T07:15:43.543863+0000","last_fullsized":"2023-11-26T07:15:43.543863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:13:49.555884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621495+0000","last_change":"2023-11-26T07:15:42.557933+0000","last_active":"2023-11-26T07:15:43.621495+0000","last_peered":"2023-11-26T07:15:43.621495+0000","last_clean":"2023-11-26T07:15:43.621495+0000","last_became_active":"2023-11-26T07:15:42.557648+0000","last_became_peered":"2023-11-26T07:15:42.557648+0000","last_unstale":"2023-11-26T07:15:43.621495+0000","last_undegraded":"2023-11-26T07:15:43.621495+0000","last_fullsized":"2023-11-26T07:15:43.621495+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:02:46.773460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621172+0000","last_change":"2023-11-26T07:15:41.949029+0000","last_active":"2023-11-26T07:15:43.621172+0000","last_peered":"2023-11-26T07:15:43.621172+0000","last_clean":"2023-11-26T07:15:43.621172+0000","last_became_active":"2023-11-26T07:15:41.948743+0000","last_became_peered":"2023-11-26T07:15:41.948743+0000","last_unstale":"2023-11-26T07:15:43.621172+0000","last_undegraded":"2023-11-26T07:15:43.621172+0000","last_fullsized":"2023-11-26T07:15:43.621172+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:33:09.745619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620738+0000","last_change":"2023-11-26T07:15:42.551831+0000","last_active":"2023-11-26T07:15:43.620738+0000","last_peered":"2023-11-26T07:15:43.620738+0000","last_clean":"2023-11-26T07:15:43.620738+0000","last_became_active":"2023-11-26T07:15:42.551533+0000","last_became_peered":"2023-11-26T07:15:42.551533+0000","last_unstale":"2023-11-26T07:15:43.620738+0000","last_undegraded":"2023-11-26T07:15:43.620738+0000","last_fullsized":"2023-11-26T07:15:43.620738+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:39:54.355604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.544925+0000","last_change":"2023-11-26T07:15:41.948603+0000","last_active":"2023-11-26T07:15:43.544925+0000","last_peered":"2023-11-26T07:15:43.544925+0000","last_clean":"2023-11-26T07:15:43.544925+0000","last_became_active":"2023-11-26T07:15:41.948232+0000","last_became_peered":"2023-11-26T07:15:41.948232+0000","last_unstale":"2023-11-26T07:15:43.544925+0000","last_undegraded":"2023-11-26T07:15:43.544925+0000","last_fullsized":"2023-11-26T07:15:43.544925+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:03:16.843398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620281+0000","last_change":"2023-11-26T07:15:42.555110+0000","last_active":"2023-11-26T07:15:43.620281+0000","last_peered":"2023-11-26T07:15:43.620281+0000","last_clean":"2023-11-26T07:15:43.620281+0000","last_became_active":"2023-11-26T07:15:42.554818+0000","last_became_peered":"2023-11-26T07:15:42.554818+0000","last_unstale":"2023-11-26T07:15:43.620281+0000","last_undegraded":"2023-11-26T07:15:43.620281+0000","last_fullsized":"2023-11-26T07:15:43.620281+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:44:15.066137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621346+0000","last_change":"2023-11-26T07:15:41.954948+0000","last_active":"2023-11-26T07:15:43.621346+0000","last_peered":"2023-11-26T07:15:43.621346+0000","last_clean":"2023-11-26T07:15:43.621346+0000","last_became_active":"2023-11-26T07:15:41.954672+0000","last_became_peered":"2023-11-26T07:15:41.954672+0000","last_unstale":"2023-11-26T07:15:43.621346+0000","last_undegraded":"2023-11-26T07:15:43.621346+0000","last_fullsized":"2023-11-26T07:15:43.621346+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:27:50.064029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620001+0000","last_change":"2023-11-26T07:15:42.562761+0000","last_active":"2023-11-26T07:15:43.620001+0000","last_peered":"2023-11-26T07:15:43.620001+0000","last_clean":"2023-11-26T07:15:43.620001+0000","last_became_active":"2023-11-26T07:15:42.562517+0000","last_became_peered":"2023-11-26T07:15:42.562517+0000","last_unstale":"2023-11-26T07:15:43.620001+0000","last_undegraded":"2023-11-26T07:15:43.620001+0000","last_fullsized":"2023-11-26T07:15:43.620001+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:42:12.777618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620691+0000","last_change":"2023-11-26T07:15:41.956877+0000","last_active":"2023-11-26T07:15:43.620691+0000","last_peered":"2023-11-26T07:15:43.620691+0000","last_clean":"2023-11-26T07:15:43.620691+0000","last_became_active":"2023-11-26T07:15:41.956568+0000","last_became_peered":"2023-11-26T07:15:41.956568+0000","last_unstale":"2023-11-26T07:15:43.620691+0000","last_undegraded":"2023-11-26T07:15:43.620691+0000","last_fullsized":"2023-11-26T07:15:43.620691+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:03:16.843398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620082+0000","last_change":"2023-11-26T07:15:42.562379+0000","last_active":"2023-11-26T07:15:43.620082+0000","last_peered":"2023-11-26T07:15:43.620082+0000","last_clean":"2023-11-26T07:15:43.620082+0000","last_became_active":"2023-11-26T07:15:42.562074+0000","last_became_peered":"2023-11-26T07:15:42.562074+0000","last_unstale":"2023-11-26T07:15:43.620082+0000","last_undegraded":"2023-11-26T07:15:43.620082+0000","last_fullsized":"2023-11-26T07:15:43.620082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:43:31.617493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.541856+0000","last_change":"2023-11-26T07:15:41.950613+0000","last_active":"2023-11-26T07:15:43.541856+0000","last_peered":"2023-11-26T07:15:43.541856+0000","last_clean":"2023-11-26T07:15:43.541856+0000","last_became_active":"2023-11-26T07:15:41.950293+0000","last_became_peered":"2023-11-26T07:15:41.950293+0000","last_unstale":"2023-11-26T07:15:43.541856+0000","last_undegraded":"2023-11-26T07:15:43.541856+0000","last_fullsized":"2023-11-26T07:15:43.541856+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:39:32.383802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542312+0000","last_change":"2023-11-26T07:15:42.552746+0000","last_active":"2023-11-26T07:15:43.542312+0000","last_peered":"2023-11-26T07:15:43.542312+0000","last_clean":"2023-11-26T07:15:43.542312+0000","last_became_active":"2023-11-26T07:15:42.552494+0000","last_became_peered":"2023-11-26T07:15:42.552494+0000","last_unstale":"2023-11-26T07:15:43.542312+0000","last_undegraded":"2023-11-26T07:15:43.542312+0000","last_fullsized":"2023-11-26T07:15:43.542312+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:56:47.443320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621412+0000","last_change":"2023-11-26T07:15:41.959643+0000","last_active":"2023-11-26T07:15:43.621412+0000","last_peered":"2023-11-26T07:15:43.621412+0000","last_clean":"2023-11-26T07:15:43.621412+0000","last_became_active":"2023-11-26T07:15:41.959374+0000","last_became_peered":"2023-11-26T07:15:41.959374+0000","last_unstale":"2023-11-26T07:15:43.621412+0000","last_undegraded":"2023-11-26T07:15:43.621412+0000","last_fullsized":"2023-11-26T07:15:43.621412+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:46:42.835450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620752+0000","last_change":"2023-11-26T07:15:42.561349+0000","last_active":"2023-11-26T07:15:43.620752+0000","last_peered":"2023-11-26T07:15:43.620752+0000","last_clean":"2023-11-26T07:15:43.620752+0000","last_became_active":"2023-11-26T07:15:42.561083+0000","last_became_peered":"2023-11-26T07:15:42.561083+0000","last_unstale":"2023-11-26T07:15:43.620752+0000","last_undegraded":"2023-11-26T07:15:43.620752+0000","last_fullsized":"2023-11-26T07:15:43.620752+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:25:13.533544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621151+0000","last_change":"2023-11-26T07:15:41.959653+0000","last_active":"2023-11-26T07:15:43.621151+0000","last_peered":"2023-11-26T07:15:43.621151+0000","last_clean":"2023-11-26T07:15:43.621151+0000","last_became_active":"2023-11-26T07:15:41.959232+0000","last_became_peered":"2023-11-26T07:15:41.959232+0000","last_unstale":"2023-11-26T07:15:43.621151+0000","last_undegraded":"2023-11-26T07:15:43.621151+0000","last_fullsized":"2023-11-26T07:15:43.621151+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:39:32.383802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545063+0000","last_change":"2023-11-26T07:15:42.558530+0000","last_active":"2023-11-26T07:15:43.545063+0000","last_peered":"2023-11-26T07:15:43.545063+0000","last_clean":"2023-11-26T07:15:43.545063+0000","last_became_active":"2023-11-26T07:15:42.558220+0000","last_became_peered":"2023-11-26T07:15:42.558220+0000","last_unstale":"2023-11-26T07:15:43.545063+0000","last_undegraded":"2023-11-26T07:15:43.545063+0000","last_fullsized":"2023-11-26T07:15:43.545063+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:43:56.465281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620593+0000","last_change":"2023-11-26T07:15:41.954074+0000","last_active":"2023-11-26T07:15:43.620593+0000","last_peered":"2023-11-26T07:15:43.620593+0000","last_clean":"2023-11-26T07:15:43.620593+0000","last_became_active":"2023-11-26T07:15:41.953867+0000","last_became_peered":"2023-11-26T07:15:41.953867+0000","last_unstale":"2023-11-26T07:15:43.620593+0000","last_undegraded":"2023-11-26T07:15:43.620593+0000","last_fullsized":"2023-11-26T07:15:43.620593+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:30.504173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.545308+0000","last_change":"2023-11-26T07:15:42.562446+0000","last_active":"2023-11-26T07:15:43.545308+0000","last_peered":"2023-11-26T07:15:43.545308+0000","last_clean":"2023-11-26T07:15:43.545308+0000","last_became_active":"2023-11-26T07:15:42.562126+0000","last_became_peered":"2023-11-26T07:15:42.562126+0000","last_unstale":"2023-11-26T07:15:43.545308+0000","last_undegraded":"2023-11-26T07:15:43.545308+0000","last_fullsized":"2023-11-26T07:15:43.545308+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:09:19.109675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.542143+0000","last_change":"2023-11-26T07:15:41.953823+0000","last_active":"2023-11-26T07:15:43.542143+0000","last_peered":"2023-11-26T07:15:43.542143+0000","last_clean":"2023-11-26T07:15:43.542143+0000","last_became_active":"2023-11-26T07:15:41.953500+0000","last_became_peered":"2023-11-26T07:15:41.953500+0000","last_unstale":"2023-11-26T07:15:43.542143+0000","last_undegraded":"2023-11-26T07:15:43.542143+0000","last_fullsized":"2023-11-26T07:15:43.542143+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:40:56.337155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620950+0000","last_change":"2023-11-26T07:15:42.555545+0000","last_active":"2023-11-26T07:15:43.620950+0000","last_peered":"2023-11-26T07:15:43.620950+0000","last_clean":"2023-11-26T07:15:43.620950+0000","last_became_active":"2023-11-26T07:15:42.555264+0000","last_became_peered":"2023-11-26T07:15:42.555264+0000","last_unstale":"2023-11-26T07:15:43.620950+0000","last_undegraded":"2023-11-26T07:15:43.620950+0000","last_fullsized":"2023-11-26T07:15:43.620950+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:14:45.337983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620523+0000","last_change":"2023-11-26T07:15:41.959310+0000","last_active":"2023-11-26T07:15:43.620523+0000","last_peered":"2023-11-26T07:15:43.620523+0000","last_clean":"2023-11-26T07:15:43.620523+0000","last_became_active":"2023-11-26T07:15:41.959005+0000","last_became_peered":"2023-11-26T07:15:41.959005+0000","last_unstale":"2023-11-26T07:15:43.620523+0000","last_undegraded":"2023-11-26T07:15:43.620523+0000","last_fullsized":"2023-11-26T07:15:43.620523+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:01:17.600412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620951+0000","last_change":"2023-11-26T07:15:42.560116+0000","last_active":"2023-11-26T07:15:43.620951+0000","last_peered":"2023-11-26T07:15:43.620951+0000","last_clean":"2023-11-26T07:15:43.620951+0000","last_became_active":"2023-11-26T07:15:42.559877+0000","last_became_peered":"2023-11-26T07:15:42.559877+0000","last_unstale":"2023-11-26T07:15:43.620951+0000","last_undegraded":"2023-11-26T07:15:43.620951+0000","last_fullsized":"2023-11-26T07:15:43.620951+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:37.821812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621124+0000","last_change":"2023-11-26T07:15:41.951472+0000","last_active":"2023-11-26T07:15:43.621124+0000","last_peered":"2023-11-26T07:15:43.621124+0000","last_clean":"2023-11-26T07:15:43.621124+0000","last_became_active":"2023-11-26T07:15:41.951225+0000","last_became_peered":"2023-11-26T07:15:41.951225+0000","last_unstale":"2023-11-26T07:15:43.621124+0000","last_undegraded":"2023-11-26T07:15:43.621124+0000","last_fullsized":"2023-11-26T07:15:43.621124+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:22:44.349797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.621235+0000","last_change":"2023-11-26T07:15:42.554835+0000","last_active":"2023-11-26T07:15:43.621235+0000","last_peered":"2023-11-26T07:15:43.621235+0000","last_clean":"2023-11-26T07:15:43.621235+0000","last_became_active":"2023-11-26T07:15:42.554583+0000","last_became_peered":"2023-11-26T07:15:42.554583+0000","last_unstale":"2023-11-26T07:15:43.621235+0000","last_undegraded":"2023-11-26T07:15:43.621235+0000","last_fullsized":"2023-11-26T07:15:43.621235+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:49.241773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.543310+0000","last_change":"2023-11-26T07:15:41.948401+0000","last_active":"2023-11-26T07:15:43.543310+0000","last_peered":"2023-11-26T07:15:43.543310+0000","last_clean":"2023-11-26T07:15:43.543310+0000","last_became_active":"2023-11-26T07:15:41.948121+0000","last_became_peered":"2023-11-26T07:15:41.948121+0000","last_unstale":"2023-11-26T07:15:43.543310+0000","last_undegraded":"2023-11-26T07:15:43.543310+0000","last_fullsized":"2023-11-26T07:15:43.543310+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:20.640171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620533+0000","last_change":"2023-11-26T07:15:42.556121+0000","last_active":"2023-11-26T07:15:43.620533+0000","last_peered":"2023-11-26T07:15:43.620533+0000","last_clean":"2023-11-26T07:15:43.620533+0000","last_became_active":"2023-11-26T07:15:42.555836+0000","last_became_peered":"2023-11-26T07:15:42.555836+0000","last_unstale":"2023-11-26T07:15:43.620533+0000","last_undegraded":"2023-11-26T07:15:43.620533+0000","last_fullsized":"2023-11-26T07:15:43.620533+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:22:00.828670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620498+0000","last_change":"2023-11-26T07:15:41.956106+0000","last_active":"2023-11-26T07:15:43.620498+0000","last_peered":"2023-11-26T07:15:43.620498+0000","last_clean":"2023-11-26T07:15:43.620498+0000","last_became_active":"2023-11-26T07:15:41.955837+0000","last_became_peered":"2023-11-26T07:15:41.955837+0000","last_unstale":"2023-11-26T07:15:43.620498+0000","last_undegraded":"2023-11-26T07:15:43.620498+0000","last_fullsized":"2023-11-26T07:15:43.620498+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:13.627141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620145+0000","last_change":"2023-11-26T07:15:42.562538+0000","last_active":"2023-11-26T07:15:43.620145+0000","last_peered":"2023-11-26T07:15:43.620145+0000","last_clean":"2023-11-26T07:15:43.620145+0000","last_became_active":"2023-11-26T07:15:42.562179+0000","last_became_peered":"2023-11-26T07:15:42.562179+0000","last_unstale":"2023-11-26T07:15:43.620145+0000","last_undegraded":"2023-11-26T07:15:43.620145+0000","last_fullsized":"2023-11-26T07:15:43.620145+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:41.933278+0000","last_clean_scrub_stamp":"2023-11-26T07:15:41.933278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:56:15.646039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-26T07:15:43.620820+0000","last_change":"2023-11-26T07:15:41.954452+0000","last_active":"2023-11-26T07:15:43.620820+0000","last_peered":"2023-11-26T07:15:43.620820+0000","last_clean":"2023-11-26T07:15:43.620820+0000","last_became_active":"2023-11-26T07:15:41.954172+0000","last_became_peered":"2023-11-26T07:15:41.954172+0000","last_unstale":"2023-11-26T07:15:43.620820+0000","last_undegraded":"2023-11-26T07:15:43.620820+0000","last_fullsized":"2023-11-26T07:15:43.620820+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:15:40.928944+0000","last_clean_scrub_stamp":"2023-11-26T07:15:40.928944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:22.663677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":216,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-26T07:15:42.541564+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:15:42.541564+0000","last_peered":"2023-11-26T07:15:42.541564+0000","last_clean":"2023-11-26T07:15:42.541564+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:15:42.541564+0000","last_undegraded":"2023-11-26T07:15:42.541564+0000","last_fullsized":"2023-11-26T07:15:42.541564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28440,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343400,"statfs":{"total":96636764160,"available":96607641600,"internally_reserved":0,"allocated":1454080,"data_stored":949304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82099999999999995,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.307,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90000000000000002,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58099999999999996,"15min":0.58899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.81299999999999994,"5min":1.048,"15min":1.048},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.67900000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.37,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.624,"15min":0.623},"min":{"1min":0.44500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.877,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.5}]},{"osd":3,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.499,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71199999999999997,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.48299999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91400000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79600000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.746}]},{"osd":5,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.379,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.747,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82999999999999996,"5min":1.171,"15min":1.171},"last":0.76300000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44700000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.006,"5min":1.361,"15min":1.361},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.46300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85499999999999998,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.70499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":358412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75900000000000001,"5min":1.159,"15min":1.159},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55900000000000005,"15min":0.57699999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73699999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.55100000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67700000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80000000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.46700000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71799999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80800000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58399999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.433,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77600000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.62},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.70299999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.5,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.747,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71299999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42699999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50700000000000001},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72099999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.5},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.747,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":360548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.33300000000000002},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.47699999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.33500000000000002},"min":{"1min":0.24199999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.441},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53300000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.55900000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.28799999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.57199999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83399999999999996,"5min":0.877,"15min":0.877},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86499999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.79200000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79400000000000004,"5min":0.995,"15min":0.995},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.62,"15min":0.629},"min":{"1min":0.45600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69099999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.42399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":1.129,"15min":1.129},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83799999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91400000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.879,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28432,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343408,"statfs":{"total":96636764160,"available":96607649792,"internally_reserved":0,"allocated":1445888,"data_stored":948688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26900000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.379,"15min":0.37},"min":{"1min":0.245,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.38200000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.29499999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64300000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.34899999999999998},"min":{"1min":0.23200000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.34899999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40000000000000002,"15min":0.40300000000000002},"min":{"1min":0.29799999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.438,"15min":0.44},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60699999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41899999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77800000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67000000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.621,"15min":0.63500000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.49299999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89000000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.625},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73699999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.502}]},{"osd":7,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81899999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.65000000000000002,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.65300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":359932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.36099999999999999,"15min":0.36899999999999999},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.61199999999999999,"5min":0.627,"15min":0.627},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.26400000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.51100000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.42699999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.39000000000000001},"min":{"1min":0.26600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.53400000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.379,"15min":0.36799999999999999},"min":{"1min":0.28799999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.372}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"min":{"1min":0.28599999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.58799999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41799999999999998,"15min":0.42599999999999999},"min":{"1min":0.29599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68100000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.67100000000000004},"min":{"1min":0.41499999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81299999999999994,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.59799999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.622,"15min":0.624},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90000000000000002,"5min":1.075,"15min":1.075},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0309999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.51300000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.004,"5min":1.03,"15min":1.03},"last":0.67900000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64700000000000002,"15min":0.65900000000000003},"min":{"1min":0.504,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.502,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96199999999999997,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":358432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35399999999999998,"15min":0.35999999999999999},"min":{"1min":0.23999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.629,"5min":1.284,"15min":1.284},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.42799999999999999,"15min":0.47299999999999998},"min":{"1min":0.26800000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.437,"15min":0.51500000000000001},"min":{"1min":0.25,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58799999999999997,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.44700000000000001,"15min":0.49199999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.69399999999999995,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.437}]},{"osd":2,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46800000000000003,"15min":0.502},"min":{"1min":0.27900000000000003,"5min":0.247,"15min":0.247},"max":{"1min":0.61199999999999999,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.504},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41199999999999998,"15min":0.42699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.59799999999999998,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.51900000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.64600000000000002,"15min":0.67400000000000004},"min":{"1min":0.41499999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79000000000000004,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.63800000000000001,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81899999999999995,"5min":1.405,"15min":1.405},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.625,"15min":0.63700000000000001},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78100000000000003,"5min":1.377,"15min":1.377},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.71499999999999997,"15min":0.79500000000000004},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89200000000000002,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.72199999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.67100000000000004},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79800000000000004,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.71599999999999997,"15min":0.78900000000000003},"min":{"1min":0.434,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82599999999999996,"5min":4.883,"15min":4.883},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.80500000000000005},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.68899999999999995,"15min":0.73299999999999998},"min":{"1min":0.437,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.876,"5min":2.923,"15min":2.923},"last":0.748}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":358390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.60999999999999999},"min":{"1min":0.34300000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.79400000000000004,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.498},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68500000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.66700000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.375,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.65700000000000003},"min":{"1min":0.376,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91100000000000003,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.53300000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56000000000000005,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.67300000000000004},"min":{"1min":0.438,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76000000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.57999999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56200000000000006,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.56000000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70799999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58299999999999996,"15min":0.60299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73899999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.746},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.63600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.68600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":358410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.32000000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69199999999999995,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.54000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":1.1259999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.45800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68500000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":1.091},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.79800000000000004,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.49099999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.70499999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":1.012},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55200000000000005,"15min":0.53900000000000003},"min":{"1min":0.40200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.72399999999999998,"5min":0.75,"15min":0.75},"last":0.76900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.45200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.76200000000000001,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.72599999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.495,"15min":0.499},"min":{"1min":0.36599999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":1.5149999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54500000000000004,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":1.161}]},{"osd":6,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55900000000000005,"15min":0.57699999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76100000000000001,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55700000000000005,"15min":0.55800000000000005},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67000000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.93100000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:15:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.747,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":1.196},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.63,"15min":0.64600000000000002},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80500000000000005,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.89700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:15:47.439 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:15:47.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:15:47.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.601+0000 7f8bb3220700 1 Processor -- start 2023-11-26T07:15:47.603 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.601+0000 7f8bb3220700 1 -- start start 2023-11-26T07:15:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8bb3220700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bac0cfdf0 0x7f8bac073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8bb3220700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac0744d0 0x7f8bac0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8bb3220700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bac0b5ee0 con 0x7f8bac15be70 2023-11-26T07:15:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8bb3220700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bac06bfc0 con 0x7f8bac0cfdf0 2023-11-26T07:15:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8bb3220700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac074960 con 0x7f8bac0744d0 2023-11-26T07:15:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8babfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bac0cfdf0 0x7f8bac073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8babfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8bac0cfdf0 0x7f8bac073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39600/0 (socket says 172.21.15.83:39600) 2023-11-26T07:15:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8babfff700 1 -- 172.21.15.83:0/2239717558 learned_addr learned my addr 172.21.15.83:0/2239717558 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8bb17bd700 1 --2- 172.21.15.83:0/2239717558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac0744d0 0x7f8bac0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8babfff700 1 -- 172.21.15.83:0/2239717558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac0744d0 msgr2=0x7f8bac0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8babfff700 1 --2- 172.21.15.83:0/2239717558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac0744d0 0x7f8bac0b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8babfff700 1 -- 172.21.15.83:0/2239717558 >> v1:172.21.15.203:6789/0 conn(0x7f8bac15be70 legacy=0x7f8bac0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.602+0000 7f8babfff700 1 -- 172.21.15.83:0/2239717558 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac06ba00 con 0x7f8bac0cfdf0 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8babfff700 1 --2- 172.21.15.83:0/2239717558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac0cfdf0 0x7f8bac073f90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8b980098f0 tx=0x7f8b98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c258c26c992910b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8bab7fe700 1 -- 172.21.15.83:0/2239717558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9800b8a0 con 0x7f8bac0cfdf0 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8bab7fe700 1 -- 172.21.15.83:0/2239717558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9800ba00 con 0x7f8bac0cfdf0 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8bab7fe700 1 -- 172.21.15.83:0/2239717558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9800bd90 con 0x7f8bac0cfdf0 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8bb3220700 1 -- 172.21.15.83:0/2239717558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac0cfdf0 msgr2=0x7f8bac073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8bb3220700 1 --2- 172.21.15.83:0/2239717558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac0cfdf0 0x7f8bac073f90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8b980098f0 tx=0x7f8b98009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8bb3220700 1 -- 172.21.15.83:0/2239717558 shutdown_connections 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8bb3220700 1 --2- 172.21.15.83:0/2239717558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac0744d0 0x7f8bac0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8bb3220700 1 --2- 172.21.15.83:0/2239717558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac0cfdf0 0x7f8bac073f90 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8bb3220700 1 -- 172.21.15.83:0/2239717558 >> 172.21.15.83:0/2239717558 conn(0x7f8bac0d0bf0 msgr2=0x7f8bac0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8bb3220700 1 -- 172.21.15.83:0/2239717558 shutdown_connections 2023-11-26T07:15:47.607 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.603+0000 7f8bb3220700 1 -- 172.21.15.83:0/2239717558 wait complete. 2023-11-26T07:15:47.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.604+0000 7f8bb3220700 1 Processor -- start 2023-11-26T07:15:47.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.605+0000 7f8bb3220700 1 -- start start 2023-11-26T07:15:47.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.605+0000 7f8bb3220700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac0744d0 0x7f8bac155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.605+0000 7f8bb3220700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bac0cfdf0 0x7f8bac15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.605+0000 7f8bb3220700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15be70 0x7f8bac09dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.605+0000 7f8bb3220700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac1558d0 con 0x7f8bac0744d0 2023-11-26T07:15:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.605+0000 7f8bb3220700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac06d9d0 con 0x7f8bac0cfdf0 2023-11-26T07:15:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.605+0000 7f8bb3220700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac06dcb0 con 0x7f8bac15be70 2023-11-26T07:15:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.605+0000 7f8bb0fbc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac0744d0 0x7f8bac155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8bb0fbc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac0744d0 0x7f8bac155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39604/0 (socket says 172.21.15.83:39604) 2023-11-26T07:15:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8bb0fbc700 1 -- 172.21.15.83:0/1608950082 learned_addr learned my addr 172.21.15.83:0/1608950082 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8bb17bd700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15be70 0x7f8bac09dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8babfff700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bac0cfdf0 0x7f8bac15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8bb0fbc700 1 -- 172.21.15.83:0/1608950082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15be70 msgr2=0x7f8bac09dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8bb0fbc700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15be70 0x7f8bac09dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8bb0fbc700 1 -- 172.21.15.83:0/1608950082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bac0cfdf0 msgr2=0x7f8bac15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8bb0fbc700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bac0cfdf0 0x7f8bac15d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8bb0fbc700 1 -- 172.21.15.83:0/1608950082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b98009550 con 0x7f8bac0744d0 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8bb0fbc700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac0744d0 0x7f8bac155390 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000c450 tx=0x7f8ba000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8ba97fa700 1 -- 172.21.15.83:0/1608950082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00052d0 con 0x7f8bac0744d0 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8ba97fa700 1 -- 172.21.15.83:0/1608950082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba0005430 con 0x7f8bac0744d0 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8ba97fa700 1 -- 172.21.15.83:0/1608950082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00038c0 con 0x7f8bac0744d0 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8bb3220700 1 -- 172.21.15.83:0/1608950082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac06df90 con 0x7f8bac0744d0 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.606+0000 7f8bb3220700 1 -- 172.21.15.83:0/1608950082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac15da40 con 0x7f8bac0744d0 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.607+0000 7f8bb3220700 1 -- 172.21.15.83:0/1608950082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bac068fb0 con 0x7f8bac0744d0 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.608+0000 7f8ba97fa700 1 -- 172.21.15.83:0/1608950082 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f8ba0003a20 con 0x7f8bac0744d0 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.608+0000 7f8ba97fa700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b9006b230 0x7f8b9006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:47.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.608+0000 7f8ba97fa700 1 -- 172.21.15.83:0/1608950082 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ba0088d80 con 0x7f8bac0744d0 2023-11-26T07:15:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.608+0000 7f8babfff700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b9006b230 0x7f8b9006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:47.612 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.608+0000 7f8babfff700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b9006b230 0x7f8b9006d6f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009c40 tx=0x7f8b98008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:47.614 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.612+0000 7f8ba97fa700 1 -- 172.21.15.83:0/1608950082 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ba0053ab0 con 0x7f8bac0744d0 2023-11-26T07:15:47.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.883+0000 7f8bb3220700 1 -- 172.21.15.83:0/1608950082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8bac0d1510 con 0x7f8bac0744d0 2023-11-26T07:15:47.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.885+0000 7f8ba97fa700 1 -- 172.21.15.83:0/1608950082 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8ba00170e0 con 0x7f8bac0744d0 2023-11-26T07:15:47.887 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2023-11-26T07:15:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ== 2023-11-26T07:15:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2023-11-26T07:15:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2023-11-26T07:15:47.887 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:15:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 -- 172.21.15.83:0/1608950082 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b9006b230 msgr2=0x7f8b9006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b9006b230 0x7f8b9006d6f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009c40 tx=0x7f8b98008030 comp rx=0 tx=0).stop 2023-11-26T07:15:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 -- 172.21.15.83:0/1608950082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac0744d0 msgr2=0x7f8bac155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac0744d0 0x7f8bac155390 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000c450 tx=0x7f8ba000c760 comp rx=0 tx=0).stop 2023-11-26T07:15:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 -- 172.21.15.83:0/1608950082 shutdown_connections 2023-11-26T07:15:47.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f8b9006b230 0x7f8b9006d6f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15be70 0x7f8bac09dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac0744d0 0x7f8bac155390 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 --2- 172.21.15.83:0/1608950082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bac0cfdf0 0x7f8bac15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 -- 172.21.15.83:0/1608950082 >> 172.21.15.83:0/1608950082 conn(0x7f8bac0d0bf0 msgr2=0x7f8bac0afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 -- 172.21.15.83:0/1608950082 shutdown_connections 2023-11-26T07:15:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:47.887+0000 7f8bb3220700 1 -- 172.21.15.83:0/1608950082 wait complete. 2023-11-26T07:15:47.890 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2023-11-26T07:15:47.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:15:47.933 INFO:tasks.ceph.mds.c.smithi083.stderr:2023-11-26T07:15:47.932+0000 7ff41af97700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:15:48.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.020+0000 7f0928163700 1 Processor -- start 2023-11-26T07:15:48.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.021+0000 7f0928163700 1 -- start start 2023-11-26T07:15:48.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.021+0000 7f0928163700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f092015e2a0 0x7f092006ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0928163700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0920073f90 0x7f0920074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0928163700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092006e7f0 con 0x7f09200ce090 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0928163700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0920074960 con 0x7f0920073f90 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0928163700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092006dec0 con 0x7f092015e2a0 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0926700700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0920073f90 0x7f0920074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f09256fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f09200ce090 0x7f092006ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59652/0 (socket says 172.21.15.83:59652) 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f09256fe700 1 -- 172.21.15.83:0/3534957151 learned_addr learned my addr 172.21.15.83:0/3534957151 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0925eff700 1 --2- 172.21.15.83:0/3534957151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f092015e2a0 0x7f092006ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0926700700 1 -- 172.21.15.83:0/3534957151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f092015e2a0 msgr2=0x7f092006ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0926700700 1 --2- 172.21.15.83:0/3534957151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f092015e2a0 0x7f092006ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0926700700 1 -- 172.21.15.83:0/3534957151 >> v1:172.21.15.83:6789/0 conn(0x7f09200ce090 legacy=0x7f092006ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0926700700 1 -- 172.21.15.83:0/3534957151 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092006e1a0 con 0x7f0920073f90 2023-11-26T07:15:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0924efd700 1 -- 172.21.15.83:0/3534957151 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066575657 0 0) 0x7f092006e7f0 con 0x7f09200ce090 2023-11-26T07:15:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0926700700 1 --2- 172.21.15.83:0/3534957151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0920073f90 0x7f0920074420 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0918009d90 tx=0x7f091800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32788dbb4760a1c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0924efd700 1 -- 172.21.15.83:0/3534957151 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091800e070 con 0x7f0920073f90 2023-11-26T07:15:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0924efd700 1 -- 172.21.15.83:0/3534957151 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091800bbd0 con 0x7f0920073f90 2023-11-26T07:15:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.022+0000 7f0924efd700 1 -- 172.21.15.83:0/3534957151 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918012430 con 0x7f0920073f90 2023-11-26T07:15:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.023+0000 7f0928163700 1 -- 172.21.15.83:0/3534957151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0920073f90 msgr2=0x7f0920074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.023+0000 7f0928163700 1 --2- 172.21.15.83:0/3534957151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0920073f90 0x7f0920074420 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0918009d90 tx=0x7f091800b330 comp rx=0 tx=0).stop 2023-11-26T07:15:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.023+0000 7f0928163700 1 -- 172.21.15.83:0/3534957151 shutdown_connections 2023-11-26T07:15:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.023+0000 7f0928163700 1 --2- 172.21.15.83:0/3534957151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0920073f90 0x7f0920074420 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.023+0000 7f0928163700 1 --2- 172.21.15.83:0/3534957151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f092015e2a0 0x7f092006ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.023+0000 7f0928163700 1 -- 172.21.15.83:0/3534957151 >> 172.21.15.83:0/3534957151 conn(0x7f09200cd570 msgr2=0x7f09200cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.023+0000 7f0928163700 1 -- 172.21.15.83:0/3534957151 shutdown_connections 2023-11-26T07:15:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.023+0000 7f0928163700 1 -- 172.21.15.83:0/3534957151 wait complete. 2023-11-26T07:15:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.024+0000 7f0928163700 1 Processor -- start 2023-11-26T07:15:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.024+0000 7f0928163700 1 -- start start 2023-11-26T07:15:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0928163700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0920073f90 0x7f09200ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0928163700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09200ce090 0x7f09200cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0928163700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092015e2a0 0x7f09200c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0928163700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092014d090 con 0x7f092015e2a0 2023-11-26T07:15:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0928163700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092014d2e0 con 0x7f09200ce090 2023-11-26T07:15:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0928163700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092014d5c0 con 0x7f0920073f90 2023-11-26T07:15:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0926700700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092015e2a0 0x7f09200c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0926700700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092015e2a0 0x7f09200c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39622/0 (socket says 172.21.15.83:39622) 2023-11-26T07:15:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0926700700 1 -- 172.21.15.83:0/2157298586 learned_addr learned my addr 172.21.15.83:0/2157298586 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0925eff700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0920073f90 0x7f09200ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f09256fe700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09200ce090 0x7f09200cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0926700700 1 -- 172.21.15.83:0/2157298586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0920073f90 msgr2=0x7f09200ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0926700700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0920073f90 0x7f09200ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0926700700 1 -- 172.21.15.83:0/2157298586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09200ce090 msgr2=0x7f09200cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0926700700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09200ce090 0x7f09200cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0926700700 1 -- 172.21.15.83:0/2157298586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0918009a70 con 0x7f092015e2a0 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.025+0000 7f0926700700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092015e2a0 0x7f09200c8640 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0920073510 tx=0x7f0918004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.026+0000 7f09167fc700 1 -- 172.21.15.83:0/2157298586 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918012430 con 0x7f092015e2a0 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.026+0000 7f09167fc700 1 -- 172.21.15.83:0/2157298586 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0918020070 con 0x7f092015e2a0 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.026+0000 7f0928163700 1 -- 172.21.15.83:0/2157298586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092014d8a0 con 0x7f092015e2a0 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.026+0000 7f09167fc700 1 -- 172.21.15.83:0/2157298586 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918004280 con 0x7f092015e2a0 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.026+0000 7f0928163700 1 -- 172.21.15.83:0/2157298586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09200c8b80 con 0x7f092015e2a0 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.027+0000 7f0928163700 1 -- 172.21.15.83:0/2157298586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0920069080 con 0x7f092015e2a0 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.027+0000 7f09167fc700 1 -- 172.21.15.83:0/2157298586 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f09180043e0 con 0x7f092015e2a0 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.027+0000 7f09167fc700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f090406b230 0x7f090406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.028+0000 7f0925eff700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f090406b230 0x7f090406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.028+0000 7f09167fc700 1 -- 172.21.15.83:0/2157298586 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0918022070 con 0x7f092015e2a0 2023-11-26T07:15:48.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.028+0000 7f0925eff700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f090406b230 0x7f090406d6f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0910006b20 tx=0x7f0910008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.032+0000 7f09167fc700 1 -- 172.21.15.83:0/2157298586 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f09180568e0 con 0x7f092015e2a0 2023-11-26T07:15:48.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.265+0000 7f0928163700 1 -- 172.21.15.83:0/2157298586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0920067f60 con 0x7f092015e2a0 2023-11-26T07:15:48.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.266+0000 7f09167fc700 1 -- 172.21.15.83:0/2157298586 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f0918059f00 con 0x7f092015e2a0 2023-11-26T07:15:48.267 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:48.268 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5420,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2163151849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2163151849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2163151849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1539108478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1539108478},{"type":"v1","addr":"172.21.15.203:6835","nonce":1539108478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1387613510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1387613510},{"type":"v1","addr":"172.21.15.203:6834","nonce":1387613510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:15:42.339653+0000","modified":"2023-11-26T07:15:45.361821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1809465302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1809465302},{"type":"v1","addr":"172.21.15.83:6836","nonce":1809465302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:15:48.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 -- 172.21.15.83:0/2157298586 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f090406b230 msgr2=0x7f090406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f090406b230 0x7f090406d6f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0910006b20 tx=0x7f0910008040 comp rx=0 tx=0).stop 2023-11-26T07:15:48.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 -- 172.21.15.83:0/2157298586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092015e2a0 msgr2=0x7f09200c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092015e2a0 0x7f09200c8640 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0920073510 tx=0x7f0918004ad0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 -- 172.21.15.83:0/2157298586 shutdown_connections 2023-11-26T07:15:48.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f092015e2a0 0x7f09200c8640 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f090406b230 0x7f090406d6f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.270 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0920073f90 0x7f09200ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 --2- 172.21.15.83:0/2157298586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09200ce090 0x7f09200cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 -- 172.21.15.83:0/2157298586 >> 172.21.15.83:0/2157298586 conn(0x7f09200cd570 msgr2=0x7f092015d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:48.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 -- 172.21.15.83:0/2157298586 shutdown_connections 2023-11-26T07:15:48.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.268+0000 7f0928163700 1 -- 172.21.15.83:0/2157298586 wait complete. 2023-11-26T07:15:48.271 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 46 2023-11-26T07:15:48.684 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:15:42.339653+0000', 'modified': '2023-11-26T07:15:45.361821+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6609}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6609': {'gid': 6609, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6836/1809465302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 1809465302}, {'type': 'v1', 'addr': '172.21.15.83:6836', 'nonce': 1809465302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:15:48.684 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:15:48.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-26T07:15:48.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.806+0000 7f09f0f07700 1 Processor -- start 2023-11-26T07:15:48.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.806+0000 7f09f0f07700 1 -- start start 2023-11-26T07:15:48.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09f0f07700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f09ec15e080 0x7f09ec0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09f0f07700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f09ec0af4d0 0x7f09ec0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09f0f07700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ec0ae430 con 0x7f09ec06c000 2023-11-26T07:15:48.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09f0f07700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ec0af0b0 con 0x7f09ec0af4d0 2023-11-26T07:15:48.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09f0f07700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ec0b05a0 con 0x7f09ec15e080 2023-11-26T07:15:48.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09e9d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f09ec0af4d0 0x7f09ec0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09e9d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f09ec0af4d0 0x7f09ec0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39628/0 (socket says 172.21.15.83:39628) 2023-11-26T07:15:48.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09ead9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f09ec06c000 0x7f09ec073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59670/0 (socket says 172.21.15.83:59670) 2023-11-26T07:15:48.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09e9d9b700 1 -- 172.21.15.83:0/2931308524 learned_addr learned my addr 172.21.15.83:0/2931308524 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:48.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09e9d9b700 1 -- 172.21.15.83:0/2931308524 >> v2:172.21.15.203:3300/0 conn(0x7f09ec15e080 msgr2=0x7f09ec0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:48.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09ea59c700 1 --2- 172.21.15.83:0/2931308524 >> v2:172.21.15.203:3300/0 conn(0x7f09ec15e080 0x7f09ec0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/2931308524 >> v2:172.21.15.203:3300/0 conn(0x7f09ec15e080 0x7f09ec0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09e9d9b700 1 -- 172.21.15.83:0/2931308524 >> v1:172.21.15.83:6789/0 conn(0x7f09ec06c000 legacy=0x7f09ec073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09e959a700 1 -- 172.21.15.83:0/2931308524 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726032454 0 0) 0x7f09ec0ae430 con 0x7f09ec06c000 2023-11-26T07:15:48.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09e9d9b700 1 -- 172.21.15.83:0/2931308524 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec0ae710 con 0x7f09ec0af4d0 2023-11-26T07:15:48.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09ea59c700 1 --2- 172.21.15.83:0/2931308524 >> v2:172.21.15.203:3300/0 conn(0x7f09ec15e080 0x7f09ec0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:48.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.807+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/2931308524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ec0af4d0 0x7f09ec0afb90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f09ec072a90 tx=0x7f09e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f738d9e9905d42 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.808+0000 7f09e959a700 1 -- 172.21.15.83:0/2931308524 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e000b860 con 0x7f09ec0af4d0 2023-11-26T07:15:48.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.808+0000 7f09e959a700 1 -- 172.21.15.83:0/2931308524 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e000b9c0 con 0x7f09ec0af4d0 2023-11-26T07:15:48.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.808+0000 7f09e959a700 1 -- 172.21.15.83:0/2931308524 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e000bd50 con 0x7f09ec0af4d0 2023-11-26T07:15:48.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.808+0000 7f09f0f07700 1 -- 172.21.15.83:0/2931308524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ec0af4d0 msgr2=0x7f09ec0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.808+0000 7f09f0f07700 1 --2- 172.21.15.83:0/2931308524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ec0af4d0 0x7f09ec0afb90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f09ec072a90 tx=0x7f09e0009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:48.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.808+0000 7f09f0f07700 1 -- 172.21.15.83:0/2931308524 shutdown_connections 2023-11-26T07:15:48.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.808+0000 7f09f0f07700 1 --2- 172.21.15.83:0/2931308524 >> v2:172.21.15.203:3300/0 conn(0x7f09ec15e080 0x7f09ec0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.808+0000 7f09f0f07700 1 --2- 172.21.15.83:0/2931308524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ec0af4d0 0x7f09ec0afb90 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.808+0000 7f09f0f07700 1 -- 172.21.15.83:0/2931308524 >> 172.21.15.83:0/2931308524 conn(0x7f09ec071b90 msgr2=0x7f09ec071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:48.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.808+0000 7f09f0f07700 1 -- 172.21.15.83:0/2931308524 shutdown_connections 2023-11-26T07:15:48.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.808+0000 7f09f0f07700 1 -- 172.21.15.83:0/2931308524 wait complete. 2023-11-26T07:15:48.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.810+0000 7f09f0f07700 1 Processor -- start 2023-11-26T07:15:48.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.810+0000 7f09f0f07700 1 -- start start 2023-11-26T07:15:48.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09f0f07700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ec06c000 0x7f09ec15fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09f0f07700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09ec0af4d0 0x7f09ec160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09f0f07700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09ec15e080 0x7f09ec13e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09f0f07700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec144c90 con 0x7f09ec06c000 2023-11-26T07:15:48.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09f0f07700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec13f160 con 0x7f09ec0af4d0 2023-11-26T07:15:48.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09f0f07700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec13e550 con 0x7f09ec15e080 2023-11-26T07:15:48.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09ea59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ec06c000 0x7f09ec15fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09ea59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ec06c000 0x7f09ec15fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39632/0 (socket says 172.21.15.83:39632) 2023-11-26T07:15:48.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09ea59c700 1 -- 172.21.15.83:0/2617474720 learned_addr learned my addr 172.21.15.83:0/2617474720 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09ead9d700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09ec15e080 0x7f09ec13e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09ec0af4d0 0x7f09ec160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09ea59c700 1 -- 172.21.15.83:0/2617474720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09ec15e080 msgr2=0x7f09ec13e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09ea59c700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09ec15e080 0x7f09ec13e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09ea59c700 1 -- 172.21.15.83:0/2617474720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09ec0af4d0 msgr2=0x7f09ec160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09ea59c700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09ec0af4d0 0x7f09ec160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09ea59c700 1 -- 172.21.15.83:0/2617474720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e0009550 con 0x7f09ec06c000 2023-11-26T07:15:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09ec0af4d0 0x7f09ec160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09ead9d700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09ec15e080 0x7f09ec13e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09ea59c700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ec06c000 0x7f09ec15fe80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c820 tx=0x7f09dc00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09daffd700 1 -- 172.21.15.83:0/2617474720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc011800 con 0x7f09ec06c000 2023-11-26T07:15:48.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.811+0000 7f09daffd700 1 -- 172.21.15.83:0/2617474720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc011960 con 0x7f09ec06c000 2023-11-26T07:15:48.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.812+0000 7f09daffd700 1 -- 172.21.15.83:0/2617474720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc011d10 con 0x7f09ec06c000 2023-11-26T07:15:48.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.812+0000 7f09f0f07700 1 -- 172.21.15.83:0/2617474720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ec1611f0 con 0x7f09ec06c000 2023-11-26T07:15:48.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.812+0000 7f09f0f07700 1 -- 172.21.15.83:0/2617474720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ec13eb10 con 0x7f09ec06c000 2023-11-26T07:15:48.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.813+0000 7f09daffd700 1 -- 172.21.15.83:0/2617474720 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f09dc018360 con 0x7f09ec06c000 2023-11-26T07:15:48.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.813+0000 7f09daffd700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f09cc06b230 0x7f09cc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.813+0000 7f09daffd700 1 -- 172.21.15.83:0/2617474720 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09dc089c20 con 0x7f09ec06c000 2023-11-26T07:15:48.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.813+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f09cc06b230 0x7f09cc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.814+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f09cc06b230 0x7f09cc06d6f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f09ec072a90 tx=0x7f09e0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.815+0000 7f09fd100b80 1 Processor -- start 2023-11-26T07:15:48.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.815+0000 7f09fd100b80 1 -- start start 2023-11-26T07:15:48.820 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762c6ed0 0x5601762f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09fd100b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560176224770 con 0x5601762ae490 2023-11-26T07:15:48.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09fd100b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017620b750 con 0x5601762b1080 2023-11-26T07:15:48.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09fd100b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x560176209070 con 0x5601762c6ed0 2023-11-26T07:15:48.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09ead9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09ead9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762dae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39644/0 (socket says 172.21.15.83:39644) 2023-11-26T07:15:48.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09ead9d700 1 -- 172.21.15.83:0/884865063 learned_addr learned my addr 172.21.15.83:0/884865063 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:48.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/884865063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762c6ed0 0x5601762f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09ea59c700 1 --2- 172.21.15.83:0/884865063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09ead9d700 1 -- 172.21.15.83:0/884865063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762c6ed0 msgr2=0x5601762f7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09ead9d700 1 --2- 172.21.15.83:0/884865063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762c6ed0 0x5601762f7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.816+0000 7f09ead9d700 1 -- 172.21.15.83:0/884865063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 msgr2=0x5601762f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.817+0000 7f09ead9d700 1 --2- 172.21.15.83:0/884865063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.817+0000 7f09ead9d700 1 -- 172.21.15.83:0/884865063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601760815e0 con 0x5601762ae490 2023-11-26T07:15:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.817+0000 7f09ead9d700 1 --2- 172.21.15.83:0/884865063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762dae70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f09e400d200 tx=0x7f09e400d510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68332ccbceb8c873 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.817+0000 7f09cbfff700 1 -- 172.21.15.83:0/884865063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e40072d0 con 0x5601762ae490 2023-11-26T07:15:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.817+0000 7f09cbfff700 1 -- 172.21.15.83:0/884865063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e400de90 con 0x5601762ae490 2023-11-26T07:15:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.817+0000 7f09cbfff700 1 -- 172.21.15.83:0/884865063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4007630 con 0x5601762ae490 2023-11-26T07:15:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.817+0000 7f09fd100b80 1 -- 172.21.15.83:0/884865063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 msgr2=0x5601762dae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.817+0000 7f09fd100b80 1 --2- 172.21.15.83:0/884865063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762dae70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f09e400d200 tx=0x7f09e400d510 comp rx=0 tx=0).stop 2023-11-26T07:15:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.818+0000 7f09fd100b80 1 -- 172.21.15.83:0/884865063 shutdown_connections 2023-11-26T07:15:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.818+0000 7f09fd100b80 1 --2- 172.21.15.83:0/884865063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762dae70 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.818+0000 7f09fd100b80 1 --2- 172.21.15.83:0/884865063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.818+0000 7f09fd100b80 1 --2- 172.21.15.83:0/884865063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762c6ed0 0x5601762f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.818+0000 7f09fd100b80 1 -- 172.21.15.83:0/884865063 >> 172.21.15.83:0/884865063 conn(0x5601761df310 msgr2=0x5601762f5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.818+0000 7f09fd100b80 1 -- 172.21.15.83:0/884865063 shutdown_connections 2023-11-26T07:15:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.818+0000 7f09fd100b80 1 -- 172.21.15.83:0/884865063 wait complete. 2023-11-26T07:15:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.818+0000 7f09fd100b80 1 -- start start 2023-11-26T07:15:48.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.818+0000 7f09fd100b80 1 Processor -- start 2023-11-26T07:15:48.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762ae490 0x5601762dafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762c6ed0 0x5601762a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09fd100b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560176204960 con 0x5601762c6ed0 2023-11-26T07:15:48.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09fd100b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601761be510 con 0x5601762b1080 2023-11-26T07:15:48.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09fd100b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601761be1b0 con 0x5601762ae490 2023-11-26T07:15:48.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09fd100b80 10 client.0 _ll_register_callbacks cb 0x5601760e9a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:15:48.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09ead9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762c6ed0 0x5601762a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09ead9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762c6ed0 0x5601762a5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39650/0 (socket says 172.21.15.83:39650) 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09ead9d700 1 -- 172.21.15.83:0/1536706725 learned_addr learned my addr 172.21.15.83:0/1536706725 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.819+0000 7f09ea59c700 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762ae490 0x5601762dafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09ead9d700 1 -- 172.21.15.83:0/1536706725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762ae490 msgr2=0x5601762dafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09ead9d700 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762ae490 0x5601762dafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09ead9d700 1 -- 172.21.15.83:0/1536706725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 msgr2=0x5601762f0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09ead9d700 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09ead9d700 1 -- 172.21.15.83:0/1536706725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e4001ef0 con 0x5601762c6ed0 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09ea59c700 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762ae490 0x5601762dafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09ead9d700 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762c6ed0 0x5601762a5330 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f09e4002230 tx=0x7f09e4001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c7333beecd2f72 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09c97fa700 1 -- 172.21.15.83:0/1536706725 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e40079f0 con 0x5601762c6ed0 2023-11-26T07:15:48.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09c97fa700 1 -- 172.21.15.83:0/1536706725 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09e401f470 con 0x5601762c6ed0 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09fd100b80 1 -- 172.21.15.83:0/1536706725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601762a3a40 con 0x5601762c6ed0 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09c97fa700 1 -- 172.21.15.83:0/1536706725 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09e401f630 con 0x5601762c6ed0 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09fd100b80 10 client.6756 fetch_fsmap learned FSMap version 46 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09fd100b80 1 -- 172.21.15.83:0/1536706725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x56017602dd90 con 0x5601762c6ed0 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09c97fa700 1 -- 172.21.15.83:0/1536706725 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601762a3a40 con 0x5601762c6ed0 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09c97fa700 1 -- 172.21.15.83:0/1536706725 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f09e401f790 con 0x5601762c6ed0 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09fd100b80 10 client.6756 fetch_fsmap finished waiting for FSMap version 46 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09fd100b80 10 client.6756 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09fd100b80 20 client.6756 populate_metadata read hostname 'smithi083' 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.83:6834/1809465302,v1:172.21.15.83:6836/1809465302] conn(0x5601762f3230 0x5601762de330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09fd100b80 4 client.6756 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09fd100b80 1 -- 172.21.15.83:0/1536706725 --> [v2:172.21.15.83:6834/1809465302,v1:172.21.15.83:6836/1809465302] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56017627f840 con 0x5601762f3230 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.820+0000 7f09ea59c700 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.83:6834/1809465302,v1:172.21.15.83:6836/1809465302] conn(0x5601762f3230 0x5601762de330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.821+0000 7f09c97fa700 10 client.6756 ms_handle_connect on v2:172.21.15.83:6834/1809465302 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.821+0000 7f09ea59c700 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.83:6834/1809465302,v1:172.21.15.83:6836/1809465302] conn(0x5601762f3230 0x5601762de330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ee1031371d27f6 server_cookie=494c29d5a96da9ff in_seq=0 out_seq=0 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.821+0000 7f09c97fa700 1 -- 172.21.15.83:0/1536706725 <== mds.0 v2:172.21.15.83:6834/1809465302 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56017627f840 con 0x5601762f3230 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.821+0000 7f09c97fa700 10 client.6756 handle_command_reply: tid=0 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.821+0000 7f09fd100b80 1 client.6756 shutdown 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09fd100b80 1 -- 172.21.15.83:0/1536706725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762c6ed0 msgr2=0x5601762a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762c6ed0 0x5601762a5330 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f09e4002230 tx=0x7f09e4001630 comp rx=0 tx=0).stop 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09fd100b80 1 -- 172.21.15.83:0/1536706725 shutdown_connections 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.83:6834/1809465302,v1:172.21.15.83:6836/1809465302] conn(0x5601762f3230 0x5601762de330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762c6ed0 0x5601762a5330 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762ae490 0x5601762dafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1536706725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09c97fa700 0 client.6756 ms_handle_reset on v2:172.21.15.83:6834/1809465302 2023-11-26T07:15:48.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09fd100b80 1 -- 172.21.15.83:0/1536706725 >> 172.21.15.83:0/1536706725 conn(0x5601761df310 msgr2=0x5601762db870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:48.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09fd100b80 1 -- 172.21.15.83:0/1536706725 shutdown_connections 2023-11-26T07:15:48.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09fd100b80 1 -- 172.21.15.83:0/1536706725 wait complete. 2023-11-26T07:15:48.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.822+0000 7f09fd100b80 20 client.6756 trim_cache size 0 max 16384 2023-11-26T07:15:48.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.840+0000 7f09fd100b80 1 Processor -- start 2023-11-26T07:15:48.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.840+0000 7f09fd100b80 1 -- start start 2023-11-26T07:15:48.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762ae490 0x5601762a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762b1080 0x5601762a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762c6ed0 0x560176307460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09fd100b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601760838d0 con 0x5601762b1080 2023-11-26T07:15:48.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09fd100b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601762a3840 con 0x5601762c6ed0 2023-11-26T07:15:48.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09fd100b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601762f8ce0 con 0x5601762ae490 2023-11-26T07:15:48.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09e9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762b1080 0x5601762a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09e9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762b1080 0x5601762a51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39656/0 (socket says 172.21.15.83:39656) 2023-11-26T07:15:48.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09e9d9b700 1 -- 172.21.15.83:0/1709575994 learned_addr learned my addr 172.21.15.83:0/1709575994 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:48.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09e9d9b700 1 -- 172.21.15.83:0/1709575994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762ae490 msgr2=0x5601762a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/1709575994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762ae490 0x5601762a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09e9d9b700 1 -- 172.21.15.83:0/1709575994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762c6ed0 msgr2=0x560176307460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/1709575994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762c6ed0 0x560176307460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.841+0000 7f09e9d9b700 1 -- 172.21.15.83:0/1709575994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560176016c10 con 0x5601762b1080 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/1709575994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762b1080 0x5601762a51f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f09e000b0e0 tx=0x7f09e0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9de9183ded00c20 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09d8ff9700 1 -- 172.21.15.83:0/1709575994 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e002d070 con 0x5601762b1080 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09d8ff9700 1 -- 172.21.15.83:0/1709575994 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e000bcf0 con 0x5601762b1080 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09d8ff9700 1 -- 172.21.15.83:0/1709575994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e001bdd0 con 0x5601762b1080 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09fd100b80 1 -- 172.21.15.83:0/1709575994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762b1080 msgr2=0x5601762a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1709575994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762b1080 0x5601762a51f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f09e000b0e0 tx=0x7f09e0010000 comp rx=0 tx=0).stop 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09fd100b80 1 -- 172.21.15.83:0/1709575994 shutdown_connections 2023-11-26T07:15:48.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1709575994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762c6ed0 0x560176307460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1709575994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762ae490 0x5601762a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1709575994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762b1080 0x5601762a51f0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09fd100b80 1 -- 172.21.15.83:0/1709575994 >> 172.21.15.83:0/1709575994 conn(0x5601761df310 msgr2=0x5601762dadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:48.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09fd100b80 1 -- 172.21.15.83:0/1709575994 shutdown_connections 2023-11-26T07:15:48.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.842+0000 7f09fd100b80 1 -- 172.21.15.83:0/1709575994 wait complete. 2023-11-26T07:15:48.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.843+0000 7f09fd100b80 1 -- start start 2023-11-26T07:15:48.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.843+0000 7f09fd100b80 1 Processor -- start 2023-11-26T07:15:48.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762eb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09fd100b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762c6ed0 0x5601762f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09fd100b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560176360680 con 0x5601762ae490 2023-11-26T07:15:48.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09fd100b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601762ebda0 con 0x5601762b1080 2023-11-26T07:15:48.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09fd100b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601763084e0 con 0x5601762c6ed0 2023-11-26T07:15:48.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09fd100b80 10 client.0 _ll_register_callbacks cb 0x7f09e4002110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:15:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09ea59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09ea59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762eb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39660/0 (socket says 172.21.15.83:39660) 2023-11-26T07:15:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09ea59c700 1 -- 172.21.15.83:0/1017965094 learned_addr learned my addr 172.21.15.83:0/1017965094 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09ead9d700 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762c6ed0 0x5601762f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09ea59c700 1 -- 172.21.15.83:0/1017965094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762c6ed0 msgr2=0x5601762f09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09ea59c700 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762c6ed0 0x5601762f09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09ea59c700 1 -- 172.21.15.83:0/1017965094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 msgr2=0x5601762f04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09ea59c700 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09ea59c700 1 -- 172.21.15.83:0/1017965094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e0021070 con 0x5601762ae490 2023-11-26T07:15:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09ea59c700 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762eb800 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f09dc058180 tx=0x7f09dc056820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacde9f9be44119b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.844+0000 7f09c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09c97fa700 1 -- 172.21.15.83:0/1017965094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc081be0 con 0x5601762ae490 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09c97fa700 1 -- 172.21.15.83:0/1017965094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09dc057970 con 0x5601762ae490 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09fd100b80 1 -- 172.21.15.83:0/1017965094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09e400de90 con 0x5601762ae490 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09c97fa700 1 -- 172.21.15.83:0/1017965094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09dc057b50 con 0x5601762ae490 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09fd100b80 10 client.6762 fetch_fsmap learned FSMap version 46 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09fd100b80 1 -- 172.21.15.83:0/1017965094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x560176054200 con 0x5601762ae490 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09c97fa700 1 -- 172.21.15.83:0/1017965094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc073070 con 0x5601762ae490 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09c97fa700 1 -- 172.21.15.83:0/1017965094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f09dc0602f0 con 0x5601762ae490 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09fd100b80 10 client.6762 fetch_fsmap finished waiting for FSMap version 46 2023-11-26T07:15:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09fd100b80 10 client.6762 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09fd100b80 20 client.6762 populate_metadata read hostname 'smithi083' 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.83:6834/1809465302,v1:172.21.15.83:6836/1809465302] conn(0x5601762f7090 0x5601762f7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09fd100b80 4 client.6762 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09fd100b80 1 -- 172.21.15.83:0/1017965094 --> [v2:172.21.15.83:6834/1809465302,v1:172.21.15.83:6836/1809465302] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5601761caf80 con 0x5601762f7090 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.83:6834/1809465302,v1:172.21.15.83:6836/1809465302] conn(0x5601762f7090 0x5601762f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09c97fa700 10 client.6762 ms_handle_connect on v2:172.21.15.83:6834/1809465302 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.845+0000 7f09e9d9b700 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.83:6834/1809465302,v1:172.21.15.83:6836/1809465302] conn(0x5601762f7090 0x5601762f7470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84fff8acee40bcfa server_cookie=91e2ba62b0eca928 in_seq=0 out_seq=0 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09c97fa700 1 -- 172.21.15.83:0/1017965094 <== mds.0 v2:172.21.15.83:6834/1809465302 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5601761caf80 con 0x5601762f7090 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09c97fa700 10 client.6762 handle_command_reply: tid=0 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 1 client.6762 shutdown 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 1 -- 172.21.15.83:0/1017965094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 msgr2=0x5601762eb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762eb800 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f09dc058180 tx=0x7f09dc056820 comp rx=0 tx=0).stop 2023-11-26T07:15:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 1 -- 172.21.15.83:0/1017965094 shutdown_connections 2023-11-26T07:15:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.83:6834/1809465302,v1:172.21.15.83:6836/1809465302] conn(0x5601762f7090 0x5601762f7470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601762c6ed0 0x5601762f09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601762ae490 0x5601762eb800 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09c97fa700 0 client.6762 ms_handle_reset on v2:172.21.15.83:6834/1809465302 2023-11-26T07:15:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 1 --2- 172.21.15.83:0/1017965094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601762b1080 0x5601762f04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 1 -- 172.21.15.83:0/1017965094 >> 172.21.15.83:0/1017965094 conn(0x5601761df310 msgr2=0x5601762f7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 1 -- 172.21.15.83:0/1017965094 shutdown_connections 2023-11-26T07:15:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 1 -- 172.21.15.83:0/1017965094 wait complete. 2023-11-26T07:15:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09fd100b80 20 client.6762 trim_cache size 0 max 16384 2023-11-26T07:15:48.854 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "0aae1049-cc8c-4011-a01f-378673e57a96", 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 6609, 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.8917665120000002, 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 46, 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 70, 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 70, 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 11.842757601000001 2023-11-26T07:15:48.855 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:15:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09f0f07700 1 -- 172.21.15.83:0/2617474720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f09cc06b230 msgr2=0x7f09cc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09f0f07700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f09cc06b230 0x7f09cc06d6f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f09ec072a90 tx=0x7f09e0008030 comp rx=0 tx=0).stop 2023-11-26T07:15:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09f0f07700 1 -- 172.21.15.83:0/2617474720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ec06c000 msgr2=0x7f09ec15fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09f0f07700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ec06c000 0x7f09ec15fe80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c820 tx=0x7f09dc00cbe0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09f0f07700 1 -- 172.21.15.83:0/2617474720 shutdown_connections 2023-11-26T07:15:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09f0f07700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f09cc06b230 0x7f09cc06d6f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09f0f07700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09ec15e080 0x7f09ec13e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09f0f07700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ec06c000 0x7f09ec15fe80 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09f0f07700 1 --2- 172.21.15.83:0/2617474720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09ec0af4d0 0x7f09ec160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.846+0000 7f09f0f07700 1 -- 172.21.15.83:0/2617474720 >> 172.21.15.83:0/2617474720 conn(0x7f09ec071b90 msgr2=0x7f09ec073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.847+0000 7f09f0f07700 1 -- 172.21.15.83:0/2617474720 shutdown_connections 2023-11-26T07:15:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.847+0000 7f09f0f07700 1 -- 172.21.15.83:0/2617474720 wait complete. 2023-11-26T07:15:48.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:15:48.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.980+0000 7f29e9a75700 1 Processor -- start 2023-11-26T07:15:48.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.980+0000 7f29e9a75700 1 -- start start 2023-11-26T07:15:48.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.981+0000 7f29e9a75700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29e40d2380 0x7f29e41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.981+0000 7f29e9a75700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e40b22d0 con 0x7f29e40b3370 2023-11-26T07:15:48.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.981+0000 7f29e9a75700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e40b25b0 con 0x7f29e415bdd0 2023-11-26T07:15:48.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.981+0000 7f29e9a75700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e40b2f50 con 0x7f29e40d2380 2023-11-26T07:15:48.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.981+0000 7f29e3fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f29e40d2380 0x7f29e41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.981+0000 7f29e2ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f29e40b3370 0x7f29e40d1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59710/0 (socket says 172.21.15.83:59710) 2023-11-26T07:15:48.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.981+0000 7f29e2ffd700 1 -- 172.21.15.83:0/4191226022 learned_addr learned my addr 172.21.15.83:0/4191226022 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:48.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.981+0000 7f29e3fff700 1 -- 172.21.15.83:0/4191226022 >> v1:172.21.15.203:6789/0 conn(0x7f29e415bdd0 legacy=0x7f29e40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.981+0000 7f29e3fff700 1 -- 172.21.15.83:0/4191226022 >> v1:172.21.15.83:6789/0 conn(0x7f29e40b3370 legacy=0x7f29e40d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.981+0000 7f29e27fc700 1 -- 172.21.15.83:0/4191226022 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807755972 0 0) 0x7f29e40b22d0 con 0x7f29e40b3370 2023-11-26T07:15:48.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.981+0000 7f29e3fff700 1 -- 172.21.15.83:0/4191226022 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e40b2890 con 0x7f29e40d2380 2023-11-26T07:15:48.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.982+0000 7f29e3fff700 1 --2- 172.21.15.83:0/4191226022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e40d2380 0x7f29e41604e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009d90 tx=0x7f29dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc58c869a03b9f27 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.982+0000 7f29e27fc700 1 -- 172.21.15.83:0/4191226022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc00e070 con 0x7f29e40d2380 2023-11-26T07:15:48.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.982+0000 7f29e27fc700 1 -- 172.21.15.83:0/4191226022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc00bbd0 con 0x7f29e40d2380 2023-11-26T07:15:48.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.982+0000 7f29e27fc700 1 -- 172.21.15.83:0/4191226022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc012430 con 0x7f29e40d2380 2023-11-26T07:15:48.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.982+0000 7f29e9a75700 1 -- 172.21.15.83:0/4191226022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e40d2380 msgr2=0x7f29e41604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.982+0000 7f29e9a75700 1 --2- 172.21.15.83:0/4191226022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e40d2380 0x7f29e41604e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009d90 tx=0x7f29dc00b330 comp rx=0 tx=0).stop 2023-11-26T07:15:48.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.982+0000 7f29e9a75700 1 -- 172.21.15.83:0/4191226022 shutdown_connections 2023-11-26T07:15:48.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.982+0000 7f29e9a75700 1 --2- 172.21.15.83:0/4191226022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e40d2380 0x7f29e41604e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.982+0000 7f29e9a75700 1 -- 172.21.15.83:0/4191226022 >> 172.21.15.83:0/4191226022 conn(0x7f29e40731a0 msgr2=0x7f29e409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.982+0000 7f29e9a75700 1 -- 172.21.15.83:0/4191226022 shutdown_connections 2023-11-26T07:15:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.982+0000 7f29e9a75700 1 -- 172.21.15.83:0/4191226022 wait complete. 2023-11-26T07:15:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.983+0000 7f29e9a75700 1 Processor -- start 2023-11-26T07:15:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.984+0000 7f29e9a75700 1 -- start start 2023-11-26T07:15:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.984+0000 7f29e9a75700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e40b3370 0x7f29e414ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.984+0000 7f29e9a75700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e40d2380 0x7f29e40cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.984+0000 7f29e9a75700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e415bdd0 0x7f29e40ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.984+0000 7f29e9a75700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40c8db0 con 0x7f29e40b3370 2023-11-26T07:15:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.984+0000 7f29e9a75700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40c9000 con 0x7f29e40d2380 2023-11-26T07:15:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.984+0000 7f29e9a75700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40c92e0 con 0x7f29e415bdd0 2023-11-26T07:15:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.984+0000 7f29e37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e40b3370 0x7f29e414ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.984+0000 7f29e37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e40b3370 0x7f29e414ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39674/0 (socket says 172.21.15.83:39674) 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e37fe700 1 -- 172.21.15.83:0/3970262787 learned_addr learned my addr 172.21.15.83:0/3970262787 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e2ffd700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e40d2380 0x7f29e40cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e3fff700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e415bdd0 0x7f29e40ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e37fe700 1 -- 172.21.15.83:0/3970262787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e415bdd0 msgr2=0x7f29e40ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e37fe700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e415bdd0 0x7f29e40ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e37fe700 1 -- 172.21.15.83:0/3970262787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e40d2380 msgr2=0x7f29e40cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e37fe700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e40d2380 0x7f29e40cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e37fe700 1 -- 172.21.15.83:0/3970262787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc009a70 con 0x7f29e40b3370 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e3fff700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e415bdd0 0x7f29e40ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e37fe700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e40b3370 0x7f29e414ffb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c450 tx=0x7f29d800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29cbfff700 1 -- 172.21.15.83:0/3970262787 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80052d0 con 0x7f29e40b3370 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29cbfff700 1 -- 172.21.15.83:0/3970262787 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d8005430 con 0x7f29e40b3370 2023-11-26T07:15:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e9a75700 1 -- 172.21.15.83:0/3970262787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e40c95c0 con 0x7f29e40b3370 2023-11-26T07:15:48.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29cbfff700 1 -- 172.21.15.83:0/3970262787 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80038c0 con 0x7f29e40b3370 2023-11-26T07:15:48.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.985+0000 7f29e9a75700 1 -- 172.21.15.83:0/3970262787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e40c9b80 con 0x7f29e40b3370 2023-11-26T07:15:48.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.986+0000 7f29e9a75700 1 -- 172.21.15.83:0/3970262787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e40c9d40 con 0x7f29e40b3370 2023-11-26T07:15:48.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.986+0000 7f29cbfff700 1 -- 172.21.15.83:0/3970262787 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f29d8003a20 con 0x7f29e40b3370 2023-11-26T07:15:48.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.987+0000 7f29cbfff700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f29d006b230 0x7f29d006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:48.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.987+0000 7f29cbfff700 1 -- 172.21.15.83:0/3970262787 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29d8088ed0 con 0x7f29e40b3370 2023-11-26T07:15:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.987+0000 7f29e2ffd700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f29d006b230 0x7f29d006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.987+0000 7f29e2ffd700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f29d006b230 0x7f29d006d6f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f29d4007b10 tx=0x7f29d4007a60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:48.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:48.991+0000 7f29cbfff700 1 -- 172.21.15.83:0/3970262787 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29d8053b70 con 0x7f29e40b3370 2023-11-26T07:15:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.228+0000 7f29e9a75700 1 -- 172.21.15.83:0/3970262787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f29e40c9ef0 con 0x7f29e40b3370 2023-11-26T07:15:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.229+0000 7f29cbfff700 1 -- 172.21.15.83:0/3970262787 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f29d8017070 con 0x7f29e40b3370 2023-11-26T07:15:49.230 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:15:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 -- 172.21.15.83:0/3970262787 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f29d006b230 msgr2=0x7f29d006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f29d006b230 0x7f29d006d6f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f29d4007b10 tx=0x7f29d4007a60 comp rx=0 tx=0).stop 2023-11-26T07:15:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 -- 172.21.15.83:0/3970262787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e40b3370 msgr2=0x7f29e414ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e40b3370 0x7f29e414ffb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c450 tx=0x7f29d800c760 comp rx=0 tx=0).stop 2023-11-26T07:15:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 -- 172.21.15.83:0/3970262787 shutdown_connections 2023-11-26T07:15:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f29d006b230 0x7f29d006d6f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e415bdd0 0x7f29e40ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e40b3370 0x7f29e414ffb0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 --2- 172.21.15.83:0/3970262787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e40d2380 0x7f29e40cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:49.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 -- 172.21.15.83:0/3970262787 >> 172.21.15.83:0/3970262787 conn(0x7f29e40731a0 msgr2=0x7f29e40afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:49.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 -- 172.21.15.83:0/3970262787 shutdown_connections 2023-11-26T07:15:49.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:49.231+0000 7f29e9a75700 1 -- 172.21.15.83:0/3970262787 wait complete. 2023-11-26T07:15:49.245 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:15:49.245 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:15:49.245 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-26T07:15:49.245 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:15:49.245 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:15:49.245 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:15:49.245 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:15:49.245 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:15:49.245 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:15:49.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:49.246 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-26T07:15:49.262 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:15:49.262 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:15:49.263 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42137sec preferred_lft 42137sec 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::3a41:fab:adc5:5d6c/64 scope link noprefixroute 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::54b0:6eff:fe33:bb15/64 scope link 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:15:49.264 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether fe:46:b0:72:7b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:15:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::fc46:b0ff:fe72:7b3f/64 scope link 2023-11-26T07:15:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:49.265 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:15:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:15:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::7808:fdff:fe1b:770c/64 scope link 2023-11-26T07:15:49.265 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:49.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:49.265 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:15:49.328 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-26T07:15:49.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:49.329 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-26T07:15:49.388 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-26T07:15:49.388 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:15:49.388 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:15:49.388 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:15:49.388 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:15:49.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:49.388 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:15:49.389 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:15:49.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:15:49.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:15:49.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:15:49.389 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:15:49.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:49.492 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:15:49.508 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:15:49.508 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:15:49.509 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:15:49.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:49.509 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-26T07:15:49.576 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:15:49.576 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:15:49.633 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:15:49.633 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:15:49.633 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:15:49.689 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:15:49.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:49.690 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:49.774 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:49.774 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:49.799 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:49.799 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:49.799 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:15:49.799 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:15:49.799 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:15:49.800 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:15:49.800 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:15:49.800 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:15:49.873 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:49.873 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:49.873 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:15:49.873 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:15:49.873 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:15:49.873 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:15:49.873 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:15:49.873 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:15:49.874 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:15:49.874 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:15:49.874 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:15:49.876 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:15:49.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:49.876 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:15:49.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:49.902 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:15:49.983 INFO:teuthology.orchestra.run.smithi083.stdout:client6115 2023-11-26T07:15:49.984 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:15:49.984 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:50.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:15:50.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.200+0000 7f89a007f700 1 Processor -- start 2023-11-26T07:15:50.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.200+0000 7f89a007f700 1 -- start start 2023-11-26T07:15:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.201+0000 7f89a007f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 0x7f899806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.201+0000 7f89a007f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f899806c540 0x7f89980b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.201+0000 7f89a007f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89980af950 con 0x7f89981446a0 2023-11-26T07:15:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.201+0000 7f89a007f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8998144a80 con 0x7f899806c540 2023-11-26T07:15:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.201+0000 7f89a007f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899806c9b0 con 0x7f89980adf00 2023-11-26T07:15:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.201+0000 7f899d61a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 0x7f899806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.201+0000 7f899e61c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f899806c540 0x7f89980b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.201+0000 7f899d61a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 0x7f899806c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:50146/0 (socket says 172.21.15.83:50146) 2023-11-26T07:15:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.201+0000 7f899d61a700 1 -- 172.21.15.83:0/420797510 learned_addr learned my addr 172.21.15.83:0/420797510 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.201+0000 7f899de1b700 1 --1- 172.21.15.83:0/420797510 >> v1:172.21.15.203:6789/0 conn(0x7f89981446a0 0x7f89980b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:44344/0 (socket says 172.21.15.83:44344) 2023-11-26T07:15:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.202+0000 7f899d61a700 1 -- 172.21.15.83:0/420797510 >> v2:172.21.15.203:3300/0 conn(0x7f899806c540 msgr2=0x7f89980b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.202+0000 7f899d61a700 1 --2- 172.21.15.83:0/420797510 >> v2:172.21.15.203:3300/0 conn(0x7f899806c540 0x7f89980b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.202+0000 7f899d61a700 1 -- 172.21.15.83:0/420797510 >> v1:172.21.15.203:6789/0 conn(0x7f89981446a0 legacy=0x7f89980b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.202+0000 7f899d61a700 1 -- 172.21.15.83:0/420797510 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89980afc30 con 0x7f89980adf00 2023-11-26T07:15:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.202+0000 7f899d61a700 1 --2- 172.21.15.83:0/420797510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 0x7f899806c000 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f899400bb90 tx=0x7f899400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a6fe064242b09cb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.202+0000 7f899ce19700 1 -- 172.21.15.83:0/420797510 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899400dd30 con 0x7f89980adf00 2023-11-26T07:15:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.202+0000 7f899ce19700 1 -- 172.21.15.83:0/420797510 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899400de90 con 0x7f89980adf00 2023-11-26T07:15:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.202+0000 7f899ce19700 1 -- 172.21.15.83:0/420797510 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994010690 con 0x7f89980adf00 2023-11-26T07:15:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.202+0000 7f89a007f700 1 -- 172.21.15.83:0/420797510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 msgr2=0x7f899806c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.202+0000 7f89a007f700 1 --2- 172.21.15.83:0/420797510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 0x7f899806c000 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f899400bb90 tx=0x7f899400bea0 comp rx=0 tx=0).stop 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.203+0000 7f89a007f700 1 -- 172.21.15.83:0/420797510 shutdown_connections 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.203+0000 7f89a007f700 1 --2- 172.21.15.83:0/420797510 >> v2:172.21.15.203:3300/0 conn(0x7f899806c540 0x7f89980b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.203+0000 7f89a007f700 1 --2- 172.21.15.83:0/420797510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 0x7f899806c000 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.203+0000 7f89a007f700 1 -- 172.21.15.83:0/420797510 >> 172.21.15.83:0/420797510 conn(0x7f8998071b90 msgr2=0x7f8998071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.203+0000 7f89a007f700 1 -- 172.21.15.83:0/420797510 shutdown_connections 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.203+0000 7f89a007f700 1 -- 172.21.15.83:0/420797510 wait complete. 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.204+0000 7f89a007f700 1 Processor -- start 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.204+0000 7f89a007f700 1 -- start start 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.204+0000 7f89a007f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f899806c540 0x7f8998136d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.204+0000 7f89a007f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 0x7f8998139640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f89a007f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89981446a0 0x7f8998140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f89a007f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89980cdc60 con 0x7f89981446a0 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f89a007f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89980ce1e0 con 0x7f899806c540 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f89a007f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89981372b0 con 0x7f89980adf00 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f899e61c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89981446a0 0x7f8998140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f899e61c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89981446a0 0x7f8998140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39696/0 (socket says 172.21.15.83:39696) 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f899de1b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f899806c540 0x7f8998136d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f899e61c700 1 -- 172.21.15.83:0/2887057916 learned_addr learned my addr 172.21.15.83:0/2887057916 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f899d61a700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 0x7f8998139640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f899e61c700 1 -- 172.21.15.83:0/2887057916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 msgr2=0x7f8998139640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f899e61c700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 0x7f8998139640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f899e61c700 1 -- 172.21.15.83:0/2887057916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f899806c540 msgr2=0x7f8998136d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f899e61c700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f899806c540 0x7f8998136d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f899e61c700 1 -- 172.21.15.83:0/2887057916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899400b7f0 con 0x7f89981446a0 2023-11-26T07:15:50.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f899e61c700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89981446a0 0x7f8998140af0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8990009dd0 tx=0x7f899000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:50.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f898e7fc700 1 -- 172.21.15.83:0/2887057916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990009c40 con 0x7f89981446a0 2023-11-26T07:15:50.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f898e7fc700 1 -- 172.21.15.83:0/2887057916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899000f040 con 0x7f89981446a0 2023-11-26T07:15:50.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f89a007f700 1 -- 172.21.15.83:0/2887057916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899813a130 con 0x7f89981446a0 2023-11-26T07:15:50.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f898e7fc700 1 -- 172.21.15.83:0/2887057916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899000ceb0 con 0x7f89981446a0 2023-11-26T07:15:50.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.205+0000 7f89a007f700 1 -- 172.21.15.83:0/2887057916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89980cde70 con 0x7f89981446a0 2023-11-26T07:15:50.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.206+0000 7f89a007f700 1 -- 172.21.15.83:0/2887057916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89980753f0 con 0x7f89981446a0 2023-11-26T07:15:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.207+0000 7f898e7fc700 1 -- 172.21.15.83:0/2887057916 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f8990014480 con 0x7f89981446a0 2023-11-26T07:15:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.207+0000 7f898e7fc700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f897c06b230 0x7f897c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.207+0000 7f898e7fc700 1 -- 172.21.15.83:0/2887057916 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f899008a460 con 0x7f89981446a0 2023-11-26T07:15:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.207+0000 7f899de1b700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f897c06b230 0x7f897c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.208+0000 7f899de1b700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f897c06b230 0x7f897c06d6f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8988000f60 tx=0x7f8988007f10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:50.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.212+0000 7f898e7fc700 1 -- 172.21.15.83:0/2887057916 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8990054ff0 con 0x7f89981446a0 2023-11-26T07:15:50.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.446+0000 7f89a007f700 1 -- 172.21.15.83:0/2887057916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f899813a650 con 0x7f89981446a0 2023-11-26T07:15:50.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.447+0000 7f898e7fc700 1 -- 172.21.15.83:0/2887057916 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8990058610 con 0x7f89981446a0 2023-11-26T07:15:50.448 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:15:50.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 -- 172.21.15.83:0/2887057916 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f897c06b230 msgr2=0x7f897c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f897c06b230 0x7f897c06d6f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8988000f60 tx=0x7f8988007f10 comp rx=0 tx=0).stop 2023-11-26T07:15:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 -- 172.21.15.83:0/2887057916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89981446a0 msgr2=0x7f8998140af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89981446a0 0x7f8998140af0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8990009dd0 tx=0x7f899000c3a0 comp rx=0 tx=0).stop 2023-11-26T07:15:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 -- 172.21.15.83:0/2887057916 shutdown_connections 2023-11-26T07:15:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89981446a0 0x7f8998140af0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f899806c540 0x7f8998136d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f897c06b230 0x7f897c06d6f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 --2- 172.21.15.83:0/2887057916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89980adf00 0x7f8998139640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 -- 172.21.15.83:0/2887057916 >> 172.21.15.83:0/2887057916 conn(0x7f8998071b90 msgr2=0x7f89980b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 -- 172.21.15.83:0/2887057916 shutdown_connections 2023-11-26T07:15:50.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:50.449+0000 7f89a007f700 1 -- 172.21.15.83:0/2887057916 wait complete. 2023-11-26T07:15:50.462 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:15:50.462 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:15:50.463 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2023-11-26T07:15:50.463 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:15:50.463 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:15:50.463 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:15:50.463 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:15:50.463 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:15:50.463 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:15:50.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:50.463 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2023-11-26T07:15:50.484 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:50.485 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42139sec preferred_lft 42139sec 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::64b0:e04:3aa8:a78c/64 scope link noprefixroute 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1092:a6ff:fe63:3da7/64 scope link 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:50.486 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:15:50.487 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:15:50.487 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::b88a:cff:fe60:2b6f/64 scope link 2023-11-26T07:15:50.487 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:50.487 INFO:teuthology.orchestra.run.smithi203.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:15:50.487 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether de:b3:89:ce:09:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:15:50.487 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::dcb3:89ff:fece:9cd/64 scope link 2023-11-26T07:15:50.487 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:15:50.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:50.487 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:15:50.557 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-11-26T07:15:50.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:50.559 DEBUG:teuthology.orchestra.run.smithi203:> route 2023-11-26T07:15:50.619 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2023-11-26T07:15:50.619 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:15:50.619 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:15:50.619 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:15:50.619 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:15:50.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:50.620 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:15:50.620 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:15:50.620 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:15:50.620 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:15:50.620 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:15:50.620 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:15:50.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:50.727 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:15:50.744 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:15:50.744 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:15:50.744 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:15:50.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:50.744 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2023-11-26T07:15:50.810 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:15:50.810 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:15:50.866 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:15:50.867 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:15:50.867 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:15:50.923 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:15:50.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:50.923 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:51.008 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:51.008 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:51.032 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:51.032 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:51.032 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:15:51.032 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:15:51.032 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:15:51.032 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:15:51.032 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:15:51.032 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:15:51.063 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:15:51.063 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:15:51.063 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:15:51.063 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:15:51.063 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:15:51.063 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:15:51.063 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:15:51.063 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:15:51.063 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:15:51.063 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:15:51.063 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:15:51.067 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:15:51.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:51.067 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:15:51.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:51.096 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:15:51.167 INFO:teuthology.orchestra.run.smithi203.stdout:client6127 2023-11-26T07:15:51.167 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:15:51.168 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:51.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:15:51.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.360+0000 7f660660a700 1 Processor -- start 2023-11-26T07:15:51.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.360+0000 7f660660a700 1 -- start start 2023-11-26T07:15:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f660660a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000cfd80 0x7f66000d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f660660a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66000d0ee0 con 0x7f66000d2080 2023-11-26T07:15:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f660660a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66000d02d0 con 0x7f660015e090 2023-11-26T07:15:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f660660a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66000d2510 con 0x7f66000cfd80 2023-11-26T07:15:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f6604ba7700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f66000d2080 0x7f66000b15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59744/0 (socket says 172.21.15.83:59744) 2023-11-26T07:15:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f6604ba7700 1 -- 172.21.15.83:0/1690803958 learned_addr learned my addr 172.21.15.83:0/1690803958 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f65ff7fe700 1 --2- 172.21.15.83:0/1690803958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000cfd80 0x7f66000d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f65feffd700 1 -- 172.21.15.83:0/1690803958 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986855541 0 0) 0x7f66000d0ee0 con 0x7f66000d2080 2023-11-26T07:15:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f65feffd700 1 -- 172.21.15.83:0/1690803958 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e80035b0 con 0x7f66000d2080 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f65feffd700 1 -- 172.21.15.83:0/1690803958 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270857590 0 0) 0x7f66000d02d0 con 0x7f660015e090 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f65feffd700 1 -- 172.21.15.83:0/1690803958 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66000d0ee0 con 0x7f660015e090 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f65ff7fe700 1 -- 172.21.15.83:0/1690803958 >> v1:172.21.15.203:6789/0 conn(0x7f660015e090 legacy=0x7f660006bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f65ff7fe700 1 -- 172.21.15.83:0/1690803958 >> v1:172.21.15.83:6789/0 conn(0x7f66000d2080 legacy=0x7f66000b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f65feffd700 1 -- 172.21.15.83:0/1690803958 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1905514549 0 0) 0x7f65e80035b0 con 0x7f66000d2080 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.361+0000 7f65ff7fe700 1 -- 172.21.15.83:0/1690803958 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66000d05b0 con 0x7f66000cfd80 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.362+0000 7f65ff7fe700 1 --2- 172.21.15.83:0/1690803958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000cfd80 0x7f66000d1b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009870 tx=0x7f65f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a14f92d7d9276b72 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.362+0000 7f65feffd700 1 -- 172.21.15.83:0/1690803958 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80019f0 con 0x7f66000cfd80 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.362+0000 7f65feffd700 1 -- 172.21.15.83:0/1690803958 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f0004030 con 0x7f66000cfd80 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.362+0000 7f65feffd700 1 -- 172.21.15.83:0/1690803958 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f00043c0 con 0x7f66000cfd80 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.362+0000 7f660660a700 1 -- 172.21.15.83:0/1690803958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000cfd80 msgr2=0x7f66000d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.362+0000 7f660660a700 1 --2- 172.21.15.83:0/1690803958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000cfd80 0x7f66000d1b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009870 tx=0x7f65f0009b80 comp rx=0 tx=0).stop 2023-11-26T07:15:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.362+0000 7f660660a700 1 -- 172.21.15.83:0/1690803958 shutdown_connections 2023-11-26T07:15:51.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.362+0000 7f660660a700 1 --2- 172.21.15.83:0/1690803958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000cfd80 0x7f66000d1b40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:51.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.362+0000 7f660660a700 1 -- 172.21.15.83:0/1690803958 >> 172.21.15.83:0/1690803958 conn(0x7f66000731a0 msgr2=0x7f660009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:51.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.362+0000 7f660660a700 1 -- 172.21.15.83:0/1690803958 shutdown_connections 2023-11-26T07:15:51.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.362+0000 7f660660a700 1 -- 172.21.15.83:0/1690803958 wait complete. 2023-11-26T07:15:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.364+0000 7f660660a700 1 Processor -- start 2023-11-26T07:15:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.364+0000 7f660660a700 1 -- start start 2023-11-26T07:15:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f660660a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000cfd80 0x7f660015e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f660660a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000d2080 0x7f66000b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f660660a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660015e090 0x7f6600150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f660660a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000b1860 con 0x7f660015e090 2023-11-26T07:15:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f660660a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000b4570 con 0x7f66000cfd80 2023-11-26T07:15:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f660660a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660015ecf0 con 0x7f66000d2080 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f6604ba7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660015e090 0x7f6600150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f65fffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000cfd80 0x7f660015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f6604ba7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660015e090 0x7f6600150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39710/0 (socket says 172.21.15.83:39710) 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f6604ba7700 1 -- 172.21.15.83:0/3248322369 learned_addr learned my addr 172.21.15.83:0/3248322369 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f65ff7fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000d2080 0x7f66000b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f6604ba7700 1 -- 172.21.15.83:0/3248322369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000d2080 msgr2=0x7f66000b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f6604ba7700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000d2080 0x7f66000b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f6604ba7700 1 -- 172.21.15.83:0/3248322369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000cfd80 msgr2=0x7f660015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f6604ba7700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000cfd80 0x7f660015e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f6604ba7700 1 -- 172.21.15.83:0/3248322369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f00094a0 con 0x7f660015e090 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f65fffff700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000cfd80 0x7f660015e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f6604ba7700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660015e090 0x7f6600150a30 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f65f8001830 tx=0x7f65f800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f65fcff9700 1 -- 172.21.15.83:0/3248322369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f800ab30 con 0x7f660015e090 2023-11-26T07:15:51.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f65fcff9700 1 -- 172.21.15.83:0/3248322369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f800ac90 con 0x7f660015e090 2023-11-26T07:15:51.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f65fcff9700 1 -- 172.21.15.83:0/3248322369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80070d0 con 0x7f660015e090 2023-11-26T07:15:51.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f660660a700 1 -- 172.21.15.83:0/3248322369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66000ce150 con 0x7f660015e090 2023-11-26T07:15:51.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.365+0000 7f660660a700 1 -- 172.21.15.83:0/3248322369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66000ce710 con 0x7f660015e090 2023-11-26T07:15:51.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.366+0000 7f660660a700 1 -- 172.21.15.83:0/3248322369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66000751e0 con 0x7f660015e090 2023-11-26T07:15:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.367+0000 7f65fcff9700 1 -- 172.21.15.83:0/3248322369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f65f8007230 con 0x7f660015e090 2023-11-26T07:15:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.367+0000 7f65fcff9700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f65ec06b260 0x7f65ec06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.367+0000 7f65fcff9700 1 -- 172.21.15.83:0/3248322369 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65f8089660 con 0x7f660015e090 2023-11-26T07:15:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.367+0000 7f65fffff700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f65ec06b260 0x7f65ec06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.368+0000 7f65fffff700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f65ec06b260 0x7f65ec06d720 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f65f40011e0 tx=0x7f65f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:51.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.372+0000 7f65fcff9700 1 -- 172.21.15.83:0/3248322369 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65f8054250 con 0x7f660015e090 2023-11-26T07:15:51.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:51.608+0000 7f660660a700 1 -- 172.21.15.83:0/3248322369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f66000625c0 con 0x7f660015e090 2023-11-26T07:15:52.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.553+0000 7f65fcff9700 1 -- 172.21.15.83:0/3248322369 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 136+0+0 (secure 0 0 0) 0x7f65f8057870 con 0x7f660015e090 2023-11-26T07:15:52.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 -- 172.21.15.83:0/3248322369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f65ec06b260 msgr2=0x7f65ec06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:52.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f65ec06b260 0x7f65ec06d720 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f65f40011e0 tx=0x7f65f4009040 comp rx=0 tx=0).stop 2023-11-26T07:15:52.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 -- 172.21.15.83:0/3248322369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660015e090 msgr2=0x7f6600150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:52.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660015e090 0x7f6600150a30 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f65f8001830 tx=0x7f65f800a060 comp rx=0 tx=0).stop 2023-11-26T07:15:52.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 -- 172.21.15.83:0/3248322369 shutdown_connections 2023-11-26T07:15:52.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660015e090 0x7f6600150a30 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66000cfd80 0x7f660015e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f65ec06b260 0x7f65ec06d720 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 --2- 172.21.15.83:0/3248322369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66000d2080 0x7f66000b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 -- 172.21.15.83:0/3248322369 >> 172.21.15.83:0/3248322369 conn(0x7f66000731a0 msgr2=0x7f66000b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:52.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 -- 172.21.15.83:0/3248322369 shutdown_connections 2023-11-26T07:15:52.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.556+0000 7f660660a700 1 -- 172.21.15.83:0/3248322369 wait complete. 2023-11-26T07:15:52.573 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:15:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.698+0000 7fcdf4a08700 1 Processor -- start 2023-11-26T07:15:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.698+0000 7fcdf4a08700 1 -- start start 2023-11-26T07:15:52.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.699+0000 7fcdf4a08700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcdf0147780 0x7fcdf015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.699+0000 7fcdf4a08700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcdf006dc20 0x7fcdf0073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.699+0000 7fcdf4a08700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf015eb00 con 0x7fcdf01473a0 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.699+0000 7fcdf4a08700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf015ede0 con 0x7fcdf0147780 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.699+0000 7fcdf4a08700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf015f0c0 con 0x7fcdf006dc20 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.699+0000 7fcdedd9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcdf0147780 0x7fcdf015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.699+0000 7fcdedd9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcdf0147780 0x7fcdf015e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39716/0 (socket says 172.21.15.83:39716) 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.699+0000 7fcdedd9b700 1 -- 172.21.15.83:0/3639616203 learned_addr learned my addr 172.21.15.83:0/3639616203 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.700+0000 7fcdeed9d700 1 --2- 172.21.15.83:0/3639616203 >> v2:172.21.15.203:3300/0 conn(0x7fcdf006dc20 0x7fcdf0073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.700+0000 7fcdedd9b700 1 -- 172.21.15.83:0/3639616203 >> v2:172.21.15.203:3300/0 conn(0x7fcdf006dc20 msgr2=0x7fcdf0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.700+0000 7fcdedd9b700 1 --2- 172.21.15.83:0/3639616203 >> v2:172.21.15.203:3300/0 conn(0x7fcdf006dc20 0x7fcdf0073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.700+0000 7fcdedd9b700 1 -- 172.21.15.83:0/3639616203 >> v1:172.21.15.203:6789/0 conn(0x7fcdf01473a0 legacy=0x7fcdf006c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.700+0000 7fcdedd9b700 1 -- 172.21.15.83:0/3639616203 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf015f3a0 con 0x7fcdf0147780 2023-11-26T07:15:52.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.700+0000 7fcdedd9b700 1 --2- 172.21.15.83:0/3639616203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf0147780 0x7fcdf015e4b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00afee0 tx=0x7fcde0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673d887854a08cd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.700+0000 7fcded59a700 1 -- 172.21.15.83:0/3639616203 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000b870 con 0x7fcdf0147780 2023-11-26T07:15:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.700+0000 7fcded59a700 1 -- 172.21.15.83:0/3639616203 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde000b9d0 con 0x7fcdf0147780 2023-11-26T07:15:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.700+0000 7fcded59a700 1 -- 172.21.15.83:0/3639616203 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000bd60 con 0x7fcdf0147780 2023-11-26T07:15:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.701+0000 7fcdf4a08700 1 -- 172.21.15.83:0/3639616203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf0147780 msgr2=0x7fcdf015e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.701+0000 7fcdf4a08700 1 --2- 172.21.15.83:0/3639616203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf0147780 0x7fcdf015e4b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00afee0 tx=0x7fcde0009bd0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.701+0000 7fcdf4a08700 1 -- 172.21.15.83:0/3639616203 shutdown_connections 2023-11-26T07:15:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.701+0000 7fcdf4a08700 1 --2- 172.21.15.83:0/3639616203 >> v2:172.21.15.203:3300/0 conn(0x7fcdf006dc20 0x7fcdf0073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.701+0000 7fcdf4a08700 1 --2- 172.21.15.83:0/3639616203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf0147780 0x7fcdf015e4b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.701+0000 7fcdf4a08700 1 -- 172.21.15.83:0/3639616203 >> 172.21.15.83:0/3639616203 conn(0x7fcdf00b3130 msgr2=0x7fcdf00d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.701+0000 7fcdf4a08700 1 -- 172.21.15.83:0/3639616203 shutdown_connections 2023-11-26T07:15:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.701+0000 7fcdf4a08700 1 -- 172.21.15.83:0/3639616203 wait complete. 2023-11-26T07:15:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.702+0000 7fcdf4a08700 1 Processor -- start 2023-11-26T07:15:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.702+0000 7fcdf4a08700 1 -- start start 2023-11-26T07:15:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdf4a08700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf006dc20 0x7fcdf00cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:52.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdf4a08700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf01473a0 0x7fcdf01443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdf4a08700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf0147780 0x7fcdf0144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdf4a08700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0141320 con 0x7fcdf01473a0 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdf4a08700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00cde70 con 0x7fcdf006dc20 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdf4a08700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00ce150 con 0x7fcdf0147780 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdedd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf01473a0 0x7fcdf01443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdedd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf01473a0 0x7fcdf01443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39722/0 (socket says 172.21.15.83:39722) 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdedd9b700 1 -- 172.21.15.83:0/907139864 learned_addr learned my addr 172.21.15.83:0/907139864 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdeed9d700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf0147780 0x7fcdf0144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdee59c700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf006dc20 0x7fcdf00cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdedd9b700 1 -- 172.21.15.83:0/907139864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf0147780 msgr2=0x7fcdf0144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdedd9b700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf0147780 0x7fcdf0144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdedd9b700 1 -- 172.21.15.83:0/907139864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf006dc20 msgr2=0x7fcdf00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdedd9b700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf006dc20 0x7fcdf00cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdedd9b700 1 -- 172.21.15.83:0/907139864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde0009550 con 0x7fcdf01473a0 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcdedd9b700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf01473a0 0x7fcdf01443c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009c10 tx=0x7fcde0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.703+0000 7fcddeffd700 1 -- 172.21.15.83:0/907139864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0004540 con 0x7fcdf01473a0 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.704+0000 7fcddeffd700 1 -- 172.21.15.83:0/907139864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde00036a0 con 0x7fcdf01473a0 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.704+0000 7fcdf4a08700 1 -- 172.21.15.83:0/907139864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf00ce430 con 0x7fcdf01473a0 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.704+0000 7fcddeffd700 1 -- 172.21.15.83:0/907139864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0003a50 con 0x7fcdf01473a0 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.704+0000 7fcdf4a08700 1 -- 172.21.15.83:0/907139864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf00cdab0 con 0x7fcdf01473a0 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.704+0000 7fcdf4a08700 1 -- 172.21.15.83:0/907139864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf00cdc40 con 0x7fcdf01473a0 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.705+0000 7fcddeffd700 1 -- 172.21.15.83:0/907139864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fcde0021070 con 0x7fcdf01473a0 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.705+0000 7fcddeffd700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcdd006b230 0x7fcdd006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.705+0000 7fcddeffd700 1 -- 172.21.15.83:0/907139864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde008b300 con 0x7fcdf01473a0 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.705+0000 7fcdee59c700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcdd006b230 0x7fcdd006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.706+0000 7fcdee59c700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcdd006b230 0x7fcdd006d6f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcde4000f60 tx=0x7fcde4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:52.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.710+0000 7fcddeffd700 1 -- 172.21.15.83:0/907139864 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcde0056150 con 0x7fcdf01473a0 2023-11-26T07:15:52.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.928+0000 7fcdf4a08700 1 -- 172.21.15.83:0/907139864 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdf00744d0 con 0x7fcdd006b230 2023-11-26T07:15:52.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.930+0000 7fcddeffd700 1 -- 172.21.15.83:0/907139864 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fcde4001480 con 0x7fcdd006b230 2023-11-26T07:15:52.931 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:15:52.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephfs" 2023-11-26T07:15:52.932 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:52.932 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:15:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 -- 172.21.15.83:0/907139864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcdd006b230 msgr2=0x7fcdd006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcdd006b230 0x7fcdd006d6f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcde4000f60 tx=0x7fcde4009040 comp rx=0 tx=0).stop 2023-11-26T07:15:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 -- 172.21.15.83:0/907139864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf01473a0 msgr2=0x7fcdf01443c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf01473a0 0x7fcdf01443c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009c10 tx=0x7fcde0004750 comp rx=0 tx=0).stop 2023-11-26T07:15:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 -- 172.21.15.83:0/907139864 shutdown_connections 2023-11-26T07:15:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fcdd006b230 0x7fcdd006d6f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcdf0147780 0x7fcdf0144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcdf006dc20 0x7fcdf00cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 --2- 172.21.15.83:0/907139864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcdf01473a0 0x7fcdf01443c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 -- 172.21.15.83:0/907139864 >> 172.21.15.83:0/907139864 conn(0x7fcdf00b3130 msgr2=0x7fcdf01618a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 -- 172.21.15.83:0/907139864 shutdown_connections 2023-11-26T07:15:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:52.932+0000 7fcdf4a08700 1 -- 172.21.15.83:0/907139864 wait complete. 2023-11-26T07:15:52.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:15:53.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.067+0000 7fbed30c4700 1 Processor -- start 2023-11-26T07:15:53.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.067+0000 7fbed30c4700 1 -- start start 2023-11-26T07:15:53.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbed30c4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbecc0704b0 0x7fbecc0d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbed30c4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbecc0d2000 0x7fbecc0af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbed30c4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc0cc4a0 con 0x7fbecc15bec0 2023-11-26T07:15:53.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbed30c4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbecc0d2470 con 0x7fbecc0704b0 2023-11-26T07:15:53.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbed30c4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0d03e0 con 0x7fbecc0d2000 2023-11-26T07:15:53.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbecbfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbecc0704b0 0x7fbecc0d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbecbfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbecc0704b0 0x7fbecc0d1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39730/0 (socket says 172.21.15.83:39730) 2023-11-26T07:15:53.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbecbfff700 1 -- 172.21.15.83:0/2739710522 learned_addr learned my addr 172.21.15.83:0/2739710522 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:53.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbed1661700 1 --2- 172.21.15.83:0/2739710522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbecc0d2000 0x7fbecc0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbecbfff700 1 -- 172.21.15.83:0/2739710522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbecc0d2000 msgr2=0x7fbecc0af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:53.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbecbfff700 1 --2- 172.21.15.83:0/2739710522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbecc0d2000 0x7fbecc0af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbecbfff700 1 -- 172.21.15.83:0/2739710522 >> v1:172.21.15.203:6789/0 conn(0x7fbecc15bec0 legacy=0x7fbecc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbecbfff700 1 -- 172.21.15.83:0/2739710522 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc15c2a0 con 0x7fbecc0704b0 2023-11-26T07:15:53.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.068+0000 7fbecbfff700 1 --2- 172.21.15.83:0/2739710522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbecc0704b0 0x7fbecc0d1ac0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbeb80098f0 tx=0x7fbeb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a361e85bd436b1ba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:53.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.069+0000 7fbecb7fe700 1 -- 172.21.15.83:0/2739710522 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800b8a0 con 0x7fbecc0704b0 2023-11-26T07:15:53.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.069+0000 7fbecb7fe700 1 -- 172.21.15.83:0/2739710522 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb800ba00 con 0x7fbecc0704b0 2023-11-26T07:15:53.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.069+0000 7fbecb7fe700 1 -- 172.21.15.83:0/2739710522 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800bd90 con 0x7fbecc0704b0 2023-11-26T07:15:53.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.069+0000 7fbed30c4700 1 -- 172.21.15.83:0/2739710522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbecc0704b0 msgr2=0x7fbecc0d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.069+0000 7fbed30c4700 1 --2- 172.21.15.83:0/2739710522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbecc0704b0 0x7fbecc0d1ac0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fbeb80098f0 tx=0x7fbeb8009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:53.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.069+0000 7fbed30c4700 1 -- 172.21.15.83:0/2739710522 shutdown_connections 2023-11-26T07:15:53.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.069+0000 7fbed30c4700 1 --2- 172.21.15.83:0/2739710522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbecc0d2000 0x7fbecc0af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.069+0000 7fbed30c4700 1 --2- 172.21.15.83:0/2739710522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbecc0704b0 0x7fbecc0d1ac0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.069+0000 7fbed30c4700 1 -- 172.21.15.83:0/2739710522 >> 172.21.15.83:0/2739710522 conn(0x7fbecc0730a0 msgr2=0x7fbecc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:53.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.069+0000 7fbed30c4700 1 -- 172.21.15.83:0/2739710522 shutdown_connections 2023-11-26T07:15:53.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.069+0000 7fbed30c4700 1 -- 172.21.15.83:0/2739710522 wait complete. 2023-11-26T07:15:53.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.070+0000 7fbed30c4700 1 Processor -- start 2023-11-26T07:15:53.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.070+0000 7fbed30c4700 1 -- start start 2023-11-26T07:15:53.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.071+0000 7fbed30c4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbecc0704b0 0x7fbecc0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.071+0000 7fbed30c4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbecc0d2000 0x7fbecc0c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.071+0000 7fbed30c4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbecc15bec0 0x7fbecc0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.071+0000 7fbed30c4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0b3af0 con 0x7fbecc0d2000 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.071+0000 7fbed30c4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0ca2b0 con 0x7fbecc15bec0 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.071+0000 7fbed30c4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0b34a0 con 0x7fbecc0704b0 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.071+0000 7fbecbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbecc0d2000 0x7fbecc0c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.071+0000 7fbecbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbecc0d2000 0x7fbecc0c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39736/0 (socket says 172.21.15.83:39736) 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.071+0000 7fbecbfff700 1 -- 172.21.15.83:0/254386959 learned_addr learned my addr 172.21.15.83:0/254386959 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbed0e60700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbecc0704b0 0x7fbecc0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbed1661700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbecc15bec0 0x7fbecc0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbecbfff700 1 -- 172.21.15.83:0/254386959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbecc0704b0 msgr2=0x7fbecc0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbecbfff700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbecc0704b0 0x7fbecc0b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbecbfff700 1 -- 172.21.15.83:0/254386959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbecc15bec0 msgr2=0x7fbecc0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbecbfff700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbecc15bec0 0x7fbecc0b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbecbfff700 1 -- 172.21.15.83:0/254386959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb8009550 con 0x7fbecc0d2000 2023-11-26T07:15:53.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbed1661700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbecc15bec0 0x7fbecc0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:53.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbecbfff700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbecc0d2000 0x7fbecc0c9a60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8000c00 tx=0x7fbeb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:53.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbec97fa700 1 -- 172.21.15.83:0/254386959 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8004380 con 0x7fbecc0d2000 2023-11-26T07:15:53.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbec97fa700 1 -- 172.21.15.83:0/254386959 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb801f070 con 0x7fbecc0d2000 2023-11-26T07:15:53.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbed30c4700 1 -- 172.21.15.83:0/254386959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc0b3780 con 0x7fbecc0d2000 2023-11-26T07:15:53.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbec97fa700 1 -- 172.21.15.83:0/254386959 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb80036a0 con 0x7fbecc0d2000 2023-11-26T07:15:53.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.072+0000 7fbed30c4700 1 -- 172.21.15.83:0/254386959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc0c9fa0 con 0x7fbecc0d2000 2023-11-26T07:15:53.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.073+0000 7fbed30c4700 1 -- 172.21.15.83:0/254386959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc06f9e0 con 0x7fbecc0d2000 2023-11-26T07:15:53.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.073+0000 7fbec97fa700 1 -- 172.21.15.83:0/254386959 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fbeb8021030 con 0x7fbecc0d2000 2023-11-26T07:15:53.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.074+0000 7fbec97fa700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbeb006b260 0x7fbeb006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.074+0000 7fbec97fa700 1 -- 172.21.15.83:0/254386959 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbeb808ade0 con 0x7fbecc0d2000 2023-11-26T07:15:53.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.074+0000 7fbed0e60700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbeb006b260 0x7fbeb006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.074+0000 7fbed0e60700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbeb006b260 0x7fbeb006d720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbec0007fd0 tx=0x7fbec0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:53.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.078+0000 7fbec97fa700 1 -- 172.21.15.83:0/254386959 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbeb8055b10 con 0x7fbecc0d2000 2023-11-26T07:15:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.270+0000 7fbed30c4700 1 -- 172.21.15.83:0/254386959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbecc068fb0 con 0x7fbecc0d2000 2023-11-26T07:15:53.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.270+0000 7fbec97fa700 1 -- 172.21.15.83:0/254386959 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbeb80150c0 con 0x7fbecc0d2000 2023-11-26T07:15:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 -- 172.21.15.83:0/254386959 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbeb006b260 msgr2=0x7fbeb006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbeb006b260 0x7fbeb006d720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbec0007fd0 tx=0x7fbec0009040 comp rx=0 tx=0).stop 2023-11-26T07:15:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 -- 172.21.15.83:0/254386959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbecc0d2000 msgr2=0x7fbecc0c9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbecc0d2000 0x7fbecc0c9a60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8000c00 tx=0x7fbeb8004750 comp rx=0 tx=0).stop 2023-11-26T07:15:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 -- 172.21.15.83:0/254386959 shutdown_connections 2023-11-26T07:15:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbecc15bec0 0x7fbecc0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fbeb006b260 0x7fbeb006d720 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbecc0704b0 0x7fbecc0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 --2- 172.21.15.83:0/254386959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbecc0d2000 0x7fbecc0c9a60 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 -- 172.21.15.83:0/254386959 >> 172.21.15.83:0/254386959 conn(0x7fbecc0730a0 msgr2=0x7fbecc06fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:53.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 -- 172.21.15.83:0/254386959 shutdown_connections 2023-11-26T07:15:53.275 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.272+0000 7fbed30c4700 1 -- 172.21.15.83:0/254386959 wait complete. 2023-11-26T07:15:53.285 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:15:53.285 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:53.302 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:15:53.302 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:53.359 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.518+0000 7f2e928bf700 1 Processor -- start 2023-11-26T07:15:53.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.518+0000 7f2e928bf700 1 -- start start 2023-11-26T07:15:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e928bf700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2e8c15e080 0x7f2e8c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e928bf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e8c06c710 0x7f2e8c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e928bf700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8c0ae430 con 0x7f2e8c0af4d0 2023-11-26T07:15:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e928bf700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8c0af0b0 con 0x7f2e8c06c710 2023-11-26T07:15:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e928bf700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e8c0b05a0 con 0x7f2e8c15e080 2023-11-26T07:15:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e90e5c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e8c06c710 0x7f2e8c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e90e5c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e8c06c710 0x7f2e8c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39746/0 (socket says 172.21.15.83:39746) 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e8b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2e8c0af4d0 0x7f2e8c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59782/0 (socket says 172.21.15.83:59782) 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e90e5c700 1 -- 172.21.15.83:0/1546636496 learned_addr learned my addr 172.21.15.83:0/1546636496 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e8bfff700 1 --2- 172.21.15.83:0/1546636496 >> v2:172.21.15.203:3300/0 conn(0x7f2e8c15e080 0x7f2e8c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e90e5c700 1 -- 172.21.15.83:0/1546636496 >> v2:172.21.15.203:3300/0 conn(0x7f2e8c15e080 msgr2=0x7f2e8c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e90e5c700 1 --2- 172.21.15.83:0/1546636496 >> v2:172.21.15.203:3300/0 conn(0x7f2e8c15e080 0x7f2e8c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e90e5c700 1 -- 172.21.15.83:0/1546636496 >> v1:172.21.15.83:6789/0 conn(0x7f2e8c0af4d0 legacy=0x7f2e8c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e90e5c700 1 -- 172.21.15.83:0/1546636496 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0ae710 con 0x7f2e8c06c710 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e8affd700 1 -- 172.21.15.83:0/1546636496 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662008855 0 0) 0x7f2e8c0ae430 con 0x7f2e8c0af4d0 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e8bfff700 1 --2- 172.21.15.83:0/1546636496 >> v2:172.21.15.203:3300/0 conn(0x7f2e8c15e080 0x7f2e8c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.519+0000 7f2e90e5c700 1 --2- 172.21.15.83:0/1546636496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c06c710 0x7f2e8c0afb90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c072a90 tx=0x7f2e8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f4980b7a848924 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.520+0000 7f2e8affd700 1 -- 172.21.15.83:0/1546636496 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8400e070 con 0x7f2e8c06c710 2023-11-26T07:15:53.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.520+0000 7f2e8affd700 1 -- 172.21.15.83:0/1546636496 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8400b980 con 0x7f2e8c06c710 2023-11-26T07:15:53.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.520+0000 7f2e8affd700 1 -- 172.21.15.83:0/1546636496 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8400bd30 con 0x7f2e8c06c710 2023-11-26T07:15:53.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.520+0000 7f2e928bf700 1 -- 172.21.15.83:0/1546636496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c06c710 msgr2=0x7f2e8c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.520+0000 7f2e928bf700 1 --2- 172.21.15.83:0/1546636496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c06c710 0x7f2e8c0afb90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c072a90 tx=0x7f2e8400b330 comp rx=0 tx=0).stop 2023-11-26T07:15:53.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.520+0000 7f2e928bf700 1 -- 172.21.15.83:0/1546636496 shutdown_connections 2023-11-26T07:15:53.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.520+0000 7f2e928bf700 1 --2- 172.21.15.83:0/1546636496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c06c710 0x7f2e8c0afb90 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.520+0000 7f2e928bf700 1 --2- 172.21.15.83:0/1546636496 >> v2:172.21.15.203:3300/0 conn(0x7f2e8c15e080 0x7f2e8c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.520+0000 7f2e928bf700 1 -- 172.21.15.83:0/1546636496 >> 172.21.15.83:0/1546636496 conn(0x7f2e8c071b90 msgr2=0x7f2e8c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:53.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.520+0000 7f2e928bf700 1 -- 172.21.15.83:0/1546636496 shutdown_connections 2023-11-26T07:15:53.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.520+0000 7f2e928bf700 1 -- 172.21.15.83:0/1546636496 wait complete. 2023-11-26T07:15:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.521+0000 7f2e928bf700 1 Processor -- start 2023-11-26T07:15:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.522+0000 7f2e928bf700 1 -- start start 2023-11-26T07:15:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.522+0000 7f2e928bf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c06c710 0x7f2e8c14a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.522+0000 7f2e928bf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e8c0af4d0 0x7f2e8c1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.522+0000 7f2e928bf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e8c15e080 0x7f2e8c14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.522+0000 7f2e928bf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c14ad60 con 0x7f2e8c06c710 2023-11-26T07:15:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.522+0000 7f2e928bf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c15e7d0 con 0x7f2e8c15e080 2023-11-26T07:15:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.522+0000 7f2e928bf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8c15eab0 con 0x7f2e8c0af4d0 2023-11-26T07:15:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.522+0000 7f2e8bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c06c710 0x7f2e8c14a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.522+0000 7f2e8bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c06c710 0x7f2e8c14a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39748/0 (socket says 172.21.15.83:39748) 2023-11-26T07:15:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.522+0000 7f2e8bfff700 1 -- 172.21.15.83:0/789128470 learned_addr learned my addr 172.21.15.83:0/789128470 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e90e5c700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e8c15e080 0x7f2e8c14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e8b7fe700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e8c0af4d0 0x7f2e8c1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e8bfff700 1 -- 172.21.15.83:0/789128470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e8c0af4d0 msgr2=0x7f2e8c1504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e8bfff700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e8c0af4d0 0x7f2e8c1504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e8bfff700 1 -- 172.21.15.83:0/789128470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e8c15e080 msgr2=0x7f2e8c14f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e8bfff700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e8c15e080 0x7f2e8c14f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e8bfff700 1 -- 172.21.15.83:0/789128470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e84009b30 con 0x7f2e8c06c710 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e8bfff700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c06c710 0x7f2e8c14a820 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000c7f0 tx=0x7f2e8000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e88ff9700 1 -- 172.21.15.83:0/789128470 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80011800 con 0x7f2e8c06c710 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e88ff9700 1 -- 172.21.15.83:0/789128470 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e80011960 con 0x7f2e8c06c710 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e928bf700 1 -- 172.21.15.83:0/789128470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8c15ed90 con 0x7f2e8c06c710 2023-11-26T07:15:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e88ff9700 1 -- 172.21.15.83:0/789128470 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80011d10 con 0x7f2e8c06c710 2023-11-26T07:15:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.523+0000 7f2e928bf700 1 -- 172.21.15.83:0/789128470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8c15f350 con 0x7f2e8c06c710 2023-11-26T07:15:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.524+0000 7f2e928bf700 1 -- 172.21.15.83:0/789128470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8c15f510 con 0x7f2e8c06c710 2023-11-26T07:15:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.524+0000 7f2e88ff9700 1 -- 172.21.15.83:0/789128470 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f2e80018360 con 0x7f2e8c06c710 2023-11-26T07:15:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.525+0000 7f2e88ff9700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e7806b230 0x7f2e7806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.525+0000 7f2e88ff9700 1 -- 172.21.15.83:0/789128470 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e80089bd0 con 0x7f2e8c06c710 2023-11-26T07:15:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.525+0000 7f2e8b7fe700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e7806b230 0x7f2e7806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.525+0000 7f2e8b7fe700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e7806b230 0x7f2e7806d6f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c152ba0 tx=0x7f2e7c007be0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:53.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.529+0000 7f2e88ff9700 1 -- 172.21.15.83:0/789128470 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e80054760 con 0x7f2e8c06c710 2023-11-26T07:15:53.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.722+0000 7f2e928bf700 1 -- 172.21.15.83:0/789128470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e8c072cf0 con 0x7f2e8c06c710 2023-11-26T07:15:53.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.723+0000 7f2e88ff9700 1 -- 172.21.15.83:0/789128470 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16556 (secure 0 0 0) 0x7f2e8001f090 con 0x7f2e8c06c710 2023-11-26T07:15:53.726 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:53.726 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:15:43.538242+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:15:40.508908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:15:41.299931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 -- 172.21.15.83:0/789128470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e7806b230 msgr2=0x7f2e7806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e7806b230 0x7f2e7806d6f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c152ba0 tx=0x7f2e7c007be0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 -- 172.21.15.83:0/789128470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c06c710 msgr2=0x7f2e8c14a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c06c710 0x7f2e8c14a820 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000c7f0 tx=0x7f2e8000cbb0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 -- 172.21.15.83:0/789128470 shutdown_connections 2023-11-26T07:15:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e8c15e080 0x7f2e8c14f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c06c710 0x7f2e8c14a820 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2e7806b230 0x7f2e7806d6f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 --2- 172.21.15.83:0/789128470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e8c0af4d0 0x7f2e8c1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 -- 172.21.15.83:0/789128470 >> 172.21.15.83:0/789128470 conn(0x7f2e8c071b90 msgr2=0x7f2e8c15d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 -- 172.21.15.83:0/789128470 shutdown_connections 2023-11-26T07:15:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.726+0000 7f2e928bf700 1 -- 172.21.15.83:0/789128470 wait complete. 2023-11-26T07:15:53.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:15:40.508908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:15:41.299931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:15:53.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:15:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.861+0000 7f211598c700 1 Processor -- start 2023-11-26T07:15:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.861+0000 7f211598c700 1 -- start start 2023-11-26T07:15:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.862+0000 7f211598c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f21100ce4c0 0x7f211006ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.862+0000 7f211598c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f211015e2a0 0x7f21100d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.862+0000 7f211598c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21100cfc60 con 0x7f211015dec0 2023-11-26T07:15:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.862+0000 7f211598c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21100cf4d0 con 0x7f211015e2a0 2023-11-26T07:15:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.862+0000 7f211598c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21100ce950 con 0x7f21100ce4c0 2023-11-26T07:15:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.862+0000 7f210f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f211015e2a0 0x7f21100d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.862+0000 7f210f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f211015e2a0 0x7f21100d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39760/0 (socket says 172.21.15.83:39760) 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.862+0000 7f210f7fe700 1 -- 172.21.15.83:0/2716297329 learned_addr learned my addr 172.21.15.83:0/2716297329 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f210e7fc700 1 --2- 172.21.15.83:0/2716297329 >> v2:172.21.15.203:3300/0 conn(0x7f21100ce4c0 0x7f211006ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f210f7fe700 1 -- 172.21.15.83:0/2716297329 >> v2:172.21.15.203:3300/0 conn(0x7f21100ce4c0 msgr2=0x7f211006ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f210f7fe700 1 --2- 172.21.15.83:0/2716297329 >> v2:172.21.15.203:3300/0 conn(0x7f21100ce4c0 0x7f211006ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f210f7fe700 1 -- 172.21.15.83:0/2716297329 >> v1:172.21.15.203:6789/0 conn(0x7f211015dec0 legacy=0x7f211006f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f210f7fe700 1 -- 172.21.15.83:0/2716297329 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21100cec30 con 0x7f211015e2a0 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f210f7fe700 1 --2- 172.21.15.83:0/2716297329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f211015e2a0 0x7f21100d00d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2108009e50 tx=0x7f210800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688be7f950272c75 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f210dffb700 1 -- 172.21.15.83:0/2716297329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210800e070 con 0x7f211015e2a0 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f210dffb700 1 -- 172.21.15.83:0/2716297329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2108003a20 con 0x7f211015e2a0 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f210dffb700 1 -- 172.21.15.83:0/2716297329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108003dd0 con 0x7f211015e2a0 2023-11-26T07:15:53.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f211598c700 1 -- 172.21.15.83:0/2716297329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f211015e2a0 msgr2=0x7f21100d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f211598c700 1 --2- 172.21.15.83:0/2716297329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f211015e2a0 0x7f21100d00d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2108009e50 tx=0x7f210800b330 comp rx=0 tx=0).stop 2023-11-26T07:15:53.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f211598c700 1 -- 172.21.15.83:0/2716297329 shutdown_connections 2023-11-26T07:15:53.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f211598c700 1 --2- 172.21.15.83:0/2716297329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f211015e2a0 0x7f21100d00d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f211598c700 1 --2- 172.21.15.83:0/2716297329 >> v2:172.21.15.203:3300/0 conn(0x7f21100ce4c0 0x7f211006ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.863+0000 7f211598c700 1 -- 172.21.15.83:0/2716297329 >> 172.21.15.83:0/2716297329 conn(0x7f21100730a0 msgr2=0x7f211009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:53.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.864+0000 7f211598c700 1 -- 172.21.15.83:0/2716297329 shutdown_connections 2023-11-26T07:15:53.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.864+0000 7f211598c700 1 -- 172.21.15.83:0/2716297329 wait complete. 2023-11-26T07:15:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.865+0000 7f211598c700 1 Processor -- start 2023-11-26T07:15:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.865+0000 7f211598c700 1 -- start start 2023-11-26T07:15:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.866+0000 7f211598c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f211015dec0 0x7f21100c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.866+0000 7f211598c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f211015e2a0 0x7f21101f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.866+0000 7f211598c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2110149fb0 0x7f21100d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.866+0000 7f211598c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21100c6250 con 0x7f211015e2a0 2023-11-26T07:15:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.866+0000 7f211598c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21101f7e90 con 0x7f2110149fb0 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.866+0000 7f211598c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211015ead0 con 0x7f211015dec0 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.866+0000 7f210e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f211015e2a0 0x7f21101f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.866+0000 7f210e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f211015e2a0 0x7f21101f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39764/0 (socket says 172.21.15.83:39764) 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.866+0000 7f210e7fc700 1 -- 172.21.15.83:0/311158845 learned_addr learned my addr 172.21.15.83:0/311158845 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.866+0000 7f210f7fe700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2110149fb0 0x7f21100d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.866+0000 7f210effd700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f211015dec0 0x7f21100c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.867+0000 7f210e7fc700 1 -- 172.21.15.83:0/311158845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f211015dec0 msgr2=0x7f21100c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.867+0000 7f210e7fc700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f211015dec0 0x7f21100c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.867+0000 7f210e7fc700 1 -- 172.21.15.83:0/311158845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2110149fb0 msgr2=0x7f21100d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.867+0000 7f210e7fc700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2110149fb0 0x7f21100d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.867+0000 7f210e7fc700 1 -- 172.21.15.83:0/311158845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2108009b30 con 0x7f211015e2a0 2023-11-26T07:15:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.867+0000 7f210e7fc700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f211015e2a0 0x7f21101f7950 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f210000e810 tx=0x7f210000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.867+0000 7f2107fff700 1 -- 172.21.15.83:0/311158845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100013610 con 0x7f211015e2a0 2023-11-26T07:15:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.867+0000 7f2107fff700 1 -- 172.21.15.83:0/311158845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2100013770 con 0x7f211015e2a0 2023-11-26T07:15:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.867+0000 7f211598c700 1 -- 172.21.15.83:0/311158845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211015edb0 con 0x7f211015e2a0 2023-11-26T07:15:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.867+0000 7f2107fff700 1 -- 172.21.15.83:0/311158845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100013b20 con 0x7f211015e2a0 2023-11-26T07:15:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.867+0000 7f211598c700 1 -- 172.21.15.83:0/311158845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211015f370 con 0x7f211015e2a0 2023-11-26T07:15:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.868+0000 7f211598c700 1 -- 172.21.15.83:0/311158845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2110068fb0 con 0x7f211015e2a0 2023-11-26T07:15:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.868+0000 7f2107fff700 1 -- 172.21.15.83:0/311158845 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f2100013c80 con 0x7f211015e2a0 2023-11-26T07:15:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.869+0000 7f2107fff700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f20f006b230 0x7f20f006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.869+0000 7f2107fff700 1 -- 172.21.15.83:0/311158845 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21000898e0 con 0x7f211015e2a0 2023-11-26T07:15:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.869+0000 7f210effd700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f20f006b230 0x7f20f006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.869+0000 7f210effd700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f20f006b230 0x7f20f006d6f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f20f8004f30 tx=0x7f20f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:53.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:53.873+0000 7f2107fff700 1 -- 172.21.15.83:0/311158845 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2100054480 con 0x7f211015e2a0 2023-11-26T07:15:54.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.065+0000 7f211598c700 1 -- 172.21.15.83:0/311158845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f211006b050 con 0x7f211015e2a0 2023-11-26T07:15:54.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.065+0000 7f2107fff700 1 -- 172.21.15.83:0/311158845 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2100057aa0 con 0x7f211015e2a0 2023-11-26T07:15:54.068 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:15:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.067+0000 7f211598c700 1 -- 172.21.15.83:0/311158845 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f20f006b230 msgr2=0x7f20f006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.067+0000 7f211598c700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f20f006b230 0x7f20f006d6f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f20f8004f30 tx=0x7f20f8009040 comp rx=0 tx=0).stop 2023-11-26T07:15:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.067+0000 7f211598c700 1 -- 172.21.15.83:0/311158845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f211015e2a0 msgr2=0x7f21101f7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.067+0000 7f211598c700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f211015e2a0 0x7f21101f7950 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f210000e810 tx=0x7f210000ebd0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.067+0000 7f211598c700 1 -- 172.21.15.83:0/311158845 shutdown_connections 2023-11-26T07:15:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.067+0000 7f211598c700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2110149fb0 0x7f21100d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.067+0000 7f211598c700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f20f006b230 0x7f20f006d6f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.067+0000 7f211598c700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f211015dec0 0x7f21100c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.067+0000 7f211598c700 1 --2- 172.21.15.83:0/311158845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f211015e2a0 0x7f21101f7950 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.067+0000 7f211598c700 1 -- 172.21.15.83:0/311158845 >> 172.21.15.83:0/311158845 conn(0x7f21100730a0 msgr2=0x7f21100b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.068+0000 7f211598c700 1 -- 172.21.15.83:0/311158845 shutdown_connections 2023-11-26T07:15:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.068+0000 7f211598c700 1 -- 172.21.15.83:0/311158845 wait complete. 2023-11-26T07:15:54.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:15:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.202+0000 7fda45366700 1 Processor -- start 2023-11-26T07:15:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.202+0000 7fda45366700 1 -- start start 2023-11-26T07:15:54.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.203+0000 7fda45366700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fda4015bf50 0x7fda400ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.203+0000 7fda45366700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda400d0140 con 0x7fda400c6c30 2023-11-26T07:15:54.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.203+0000 7fda45366700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda400cf3c0 con 0x7fda400ce830 2023-11-26T07:15:54.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.203+0000 7fda45366700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4015c330 con 0x7fda4015bf50 2023-11-26T07:15:54.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.203+0000 7fda3f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fda400c6c30 0x7fda400cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59814/0 (socket says 172.21.15.83:59814) 2023-11-26T07:15:54.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.203+0000 7fda3f7fe700 1 -- 172.21.15.83:0/2472661516 learned_addr learned my addr 172.21.15.83:0/2472661516 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3effd700 1 --2- 172.21.15.83:0/2472661516 >> v2:172.21.15.203:3300/0 conn(0x7fda4015bf50 0x7fda400ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173458658 0 0) 0x7fda400d0140 con 0x7fda400c6c30 2023-11-26T07:15:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda280035b0 con 0x7fda400c6c30 2023-11-26T07:15:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4052398880 0 0) 0x7fda280035b0 con 0x7fda400c6c30 2023-11-26T07:15:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 >> v2:172.21.15.203:3300/0 conn(0x7fda4015bf50 msgr2=0x7fda400ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 --2- 172.21.15.83:0/2472661516 >> v2:172.21.15.203:3300/0 conn(0x7fda4015bf50 0x7fda400ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 >> v1:172.21.15.203:6789/0 conn(0x7fda400ce830 legacy=0x7fda400cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda400d03e0 con 0x7fda400c6c30 2023-11-26T07:15:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fda380019f0 con 0x7fda400c6c30 2023-11-26T07:15:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 >> v1:172.21.15.83:6789/0 conn(0x7fda400c6c30 legacy=0x7fda400cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 0x7fda28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda28004850 0x7fda28004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda28005210 0x7fda2800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4015c330 con 0x7fda28003a00 2023-11-26T07:15:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2800c0b0 con 0x7fda28004850 2023-11-26T07:15:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2800c270 con 0x7fda28005210 2023-11-26T07:15:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3effd700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 0x7fda28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3e7fc700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda28004850 0x7fda28004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3f7fe700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda28005210 0x7fda2800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3effd700 1 -- 172.21.15.83:0/2472661516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda28005210 msgr2=0x7fda2800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:54.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3effd700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda28005210 0x7fda2800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3effd700 1 -- 172.21.15.83:0/2472661516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda28004850 msgr2=0x7fda28004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:54.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3effd700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda28004850 0x7fda28004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3effd700 1 -- 172.21.15.83:0/2472661516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda280035b0 con 0x7fda28003a00 2023-11-26T07:15:54.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3f7fe700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda28005210 0x7fda2800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:54.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.204+0000 7fda3f7fe700 1 -- 172.21.15.83:0/2472661516 reap_dead start 2023-11-26T07:15:54.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.205+0000 7fda3effd700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 0x7fda28003e20 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fda34009b60 tx=0x7fda28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b846d486033ccaf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:54.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.205+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3400e6c0 con 0x7fda28003a00 2023-11-26T07:15:54.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.205+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3400e820 con 0x7fda28003a00 2023-11-26T07:15:54.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.205+0000 7fda3dffb700 1 -- 172.21.15.83:0/2472661516 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3400eb90 con 0x7fda28003a00 2023-11-26T07:15:54.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.205+0000 7fda45366700 1 -- 172.21.15.83:0/2472661516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 msgr2=0x7fda28003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.205+0000 7fda45366700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 0x7fda28003e20 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fda34009b60 tx=0x7fda28005880 comp rx=0 tx=0).stop 2023-11-26T07:15:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.205+0000 7fda45366700 1 -- 172.21.15.83:0/2472661516 shutdown_connections 2023-11-26T07:15:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.205+0000 7fda45366700 1 --2- 172.21.15.83:0/2472661516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 0x7fda28003e20 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.205+0000 7fda45366700 1 -- 172.21.15.83:0/2472661516 >> 172.21.15.83:0/2472661516 conn(0x7fda400b3780 msgr2=0x7fda400b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.205+0000 7fda45366700 1 -- 172.21.15.83:0/2472661516 shutdown_connections 2023-11-26T07:15:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.205+0000 7fda45366700 1 -- 172.21.15.83:0/2472661516 wait complete. 2023-11-26T07:15:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.207+0000 7fda45366700 1 Processor -- start 2023-11-26T07:15:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.207+0000 7fda45366700 1 -- start start 2023-11-26T07:15:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.207+0000 7fda45366700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 0x7fda40150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.207+0000 7fda45366700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda400ca810 0x7fda401f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.207+0000 7fda45366700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda400cac90 0x7fda401f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.207+0000 7fda45366700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda400cb7b0 con 0x7fda28003a00 2023-11-26T07:15:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.207+0000 7fda45366700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4006b980 con 0x7fda400ca810 2023-11-26T07:15:54.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.207+0000 7fda45366700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda40151250 con 0x7fda400cac90 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.207+0000 7fda3effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 0x7fda40150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.207+0000 7fda3effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 0x7fda40150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39784/0 (socket says 172.21.15.83:39784) 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.207+0000 7fda3effd700 1 -- 172.21.15.83:0/353864003 learned_addr learned my addr 172.21.15.83:0/353864003 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda3e7fc700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda400ca810 0x7fda401f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda3f7fe700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda400cac90 0x7fda401f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda3effd700 1 -- 172.21.15.83:0/353864003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda400cac90 msgr2=0x7fda401f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda3effd700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda400cac90 0x7fda401f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda3effd700 1 -- 172.21.15.83:0/353864003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda400ca810 msgr2=0x7fda401f6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda3effd700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda400ca810 0x7fda401f6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda3effd700 1 -- 172.21.15.83:0/353864003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda400cb1f0 con 0x7fda28003a00 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda3effd700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 0x7fda40150d10 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fda3400e4b0 tx=0x7fda340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda277fe700 1 -- 172.21.15.83:0/353864003 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3400e6c0 con 0x7fda28003a00 2023-11-26T07:15:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda277fe700 1 -- 172.21.15.83:0/353864003 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda34003ad0 con 0x7fda28003a00 2023-11-26T07:15:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda45366700 1 -- 172.21.15.83:0/353864003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda400cb4d0 con 0x7fda28003a00 2023-11-26T07:15:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda277fe700 1 -- 172.21.15.83:0/353864003 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34003e80 con 0x7fda28003a00 2023-11-26T07:15:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.208+0000 7fda45366700 1 -- 172.21.15.83:0/353864003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4015c470 con 0x7fda28003a00 2023-11-26T07:15:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.209+0000 7fda45366700 1 -- 172.21.15.83:0/353864003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4006bbf0 con 0x7fda28003a00 2023-11-26T07:15:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.209+0000 7fda277fe700 1 -- 172.21.15.83:0/353864003 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fda34013470 con 0x7fda28003a00 2023-11-26T07:15:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.210+0000 7fda277fe700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda2c06b230 0x7fda2c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.210+0000 7fda277fe700 1 -- 172.21.15.83:0/353864003 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda3408b370 con 0x7fda28003a00 2023-11-26T07:15:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.210+0000 7fda3e7fc700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda2c06b230 0x7fda2c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.210+0000 7fda3e7fc700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda2c06b230 0x7fda2c06d6f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda400b7990 tx=0x7fda30009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.214+0000 7fda277fe700 1 -- 172.21.15.83:0/353864003 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda340561c0 con 0x7fda28003a00 2023-11-26T07:15:54.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.405+0000 7fda45366700 1 -- 172.21.15.83:0/353864003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda4006b400 con 0x7fda28003a00 2023-11-26T07:15:54.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.406+0000 7fda277fe700 1 -- 172.21.15.83:0/353864003 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda4006b400 con 0x7fda28003a00 2023-11-26T07:15:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:15:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 -- 172.21.15.83:0/353864003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda2c06b230 msgr2=0x7fda2c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda2c06b230 0x7fda2c06d6f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda400b7990 tx=0x7fda30009040 comp rx=0 tx=0).stop 2023-11-26T07:15:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 -- 172.21.15.83:0/353864003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 msgr2=0x7fda40150d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 0x7fda40150d10 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fda3400e4b0 tx=0x7fda340047c0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 -- 172.21.15.83:0/353864003 shutdown_connections 2023-11-26T07:15:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fda2c06b230 0x7fda2c06d6f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda400cac90 0x7fda401f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda28003a00 0x7fda40150d10 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 --2- 172.21.15.83:0/353864003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda400ca810 0x7fda401f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 -- 172.21.15.83:0/353864003 >> 172.21.15.83:0/353864003 conn(0x7fda400b3780 msgr2=0x7fda4006ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 -- 172.21.15.83:0/353864003 shutdown_connections 2023-11-26T07:15:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.408+0000 7fda45366700 1 -- 172.21.15.83:0/353864003 wait complete. 2023-11-26T07:15:54.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:15:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.544+0000 7fde40a9d700 1 Processor -- start 2023-11-26T07:15:54.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.544+0000 7fde40a9d700 1 -- start start 2023-11-26T07:15:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde40a9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde3c15bf50 0x7fde3c0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde40a9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde3c0ce830 0x7fde3c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde40a9d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3c0d0140 con 0x7fde3c0cf1f0 2023-11-26T07:15:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde40a9d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3c15c330 con 0x7fde3c15bf50 2023-11-26T07:15:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde40a9d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0d03e0 con 0x7fde3c0ce830 2023-11-26T07:15:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde3a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde3c15bf50 0x7fde3c0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde3a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fde3c15bf50 0x7fde3c0ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39790/0 (socket says 172.21.15.83:39790) 2023-11-26T07:15:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde3a59c700 1 -- 172.21.15.83:0/2619649110 learned_addr learned my addr 172.21.15.83:0/2619649110 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde3ad9d700 1 --1- 172.21.15.83:0/2619649110 >> v1:172.21.15.83:6789/0 conn(0x7fde3c0cf1f0 0x7fde3c0c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59834/0 (socket says 172.21.15.83:59834) 2023-11-26T07:15:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde39d9b700 1 --2- 172.21.15.83:0/2619649110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde3c0ce830 0x7fde3c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde3a59c700 1 -- 172.21.15.83:0/2619649110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde3c0ce830 msgr2=0x7fde3c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde3a59c700 1 --2- 172.21.15.83:0/2619649110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde3c0ce830 0x7fde3c0cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde3a59c700 1 -- 172.21.15.83:0/2619649110 >> v1:172.21.15.83:6789/0 conn(0x7fde3c0cf1f0 legacy=0x7fde3c0c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde3a59c700 1 -- 172.21.15.83:0/2619649110 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c0d06c0 con 0x7fde3c15bf50 2023-11-26T07:15:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.545+0000 7fde3959a700 1 -- 172.21.15.83:0/2619649110 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691895541 0 0) 0x7fde3c0d0140 con 0x7fde3c0cf1f0 2023-11-26T07:15:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde3a59c700 1 --2- 172.21.15.83:0/2619649110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde3c15bf50 0x7fde3c0ce2f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0098f0 tx=0x7fde2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bc71250261fcf6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde3959a700 1 -- 172.21.15.83:0/2619649110 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00b8a0 con 0x7fde3c15bf50 2023-11-26T07:15:54.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde3959a700 1 -- 172.21.15.83:0/2619649110 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c00ba00 con 0x7fde3c15bf50 2023-11-26T07:15:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde3959a700 1 -- 172.21.15.83:0/2619649110 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c00bd90 con 0x7fde3c15bf50 2023-11-26T07:15:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde40a9d700 1 -- 172.21.15.83:0/2619649110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde3c15bf50 msgr2=0x7fde3c0ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde40a9d700 1 --2- 172.21.15.83:0/2619649110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde3c15bf50 0x7fde3c0ce2f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0098f0 tx=0x7fde2c009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde40a9d700 1 -- 172.21.15.83:0/2619649110 shutdown_connections 2023-11-26T07:15:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde40a9d700 1 --2- 172.21.15.83:0/2619649110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde3c15bf50 0x7fde3c0ce2f0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde40a9d700 1 --2- 172.21.15.83:0/2619649110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde3c0ce830 0x7fde3c0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde40a9d700 1 -- 172.21.15.83:0/2619649110 >> 172.21.15.83:0/2619649110 conn(0x7fde3c0b3780 msgr2=0x7fde3c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde40a9d700 1 -- 172.21.15.83:0/2619649110 shutdown_connections 2023-11-26T07:15:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.546+0000 7fde40a9d700 1 -- 172.21.15.83:0/2619649110 wait complete. 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.548+0000 7fde40a9d700 1 Processor -- start 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.548+0000 7fde40a9d700 1 -- start start 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.548+0000 7fde40a9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde3c0ce830 0x7fde3c0b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.548+0000 7fde40a9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde3c0cf1f0 0x7fde3c157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.548+0000 7fde40a9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde3c15bf50 0x7fde3c157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.548+0000 7fde40a9d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c157a90 con 0x7fde3c0ce830 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.548+0000 7fde40a9d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0c9cd0 con 0x7fde3c0cf1f0 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.548+0000 7fde40a9d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c157d40 con 0x7fde3c15bf50 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde3a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde3c0ce830 0x7fde3c0b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde3a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde3c0ce830 0x7fde3c0b6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39798/0 (socket says 172.21.15.83:39798) 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde3a59c700 1 -- 172.21.15.83:0/2622093708 learned_addr learned my addr 172.21.15.83:0/2622093708 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde39d9b700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde3c0cf1f0 0x7fde3c157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde3ad9d700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde3c15bf50 0x7fde3c157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde3a59c700 1 -- 172.21.15.83:0/2622093708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde3c15bf50 msgr2=0x7fde3c157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde3a59c700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde3c15bf50 0x7fde3c157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde3a59c700 1 -- 172.21.15.83:0/2622093708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde3c0cf1f0 msgr2=0x7fde3c157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde3a59c700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde3c0cf1f0 0x7fde3c157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde3a59c700 1 -- 172.21.15.83:0/2622093708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c009550 con 0x7fde3c0ce830 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde3a59c700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde3c0ce830 0x7fde3c0b6fd0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009c40 tx=0x7fde2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde2affd700 1 -- 172.21.15.83:0/2622093708 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c004510 con 0x7fde3c0ce830 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde2affd700 1 -- 172.21.15.83:0/2622093708 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c01e070 con 0x7fde3c0ce830 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde40a9d700 1 -- 172.21.15.83:0/2622093708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3c158020 con 0x7fde3c0ce830 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde2affd700 1 -- 172.21.15.83:0/2622093708 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c0038c0 con 0x7fde3c0ce830 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.549+0000 7fde40a9d700 1 -- 172.21.15.83:0/2622093708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c0c9f90 con 0x7fde3c0ce830 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.550+0000 7fde40a9d700 1 -- 172.21.15.83:0/2622093708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3c0ca300 con 0x7fde3c0ce830 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.551+0000 7fde2affd700 1 -- 172.21.15.83:0/2622093708 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fde2c003a20 con 0x7fde3c0ce830 2023-11-26T07:15:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.551+0000 7fde2affd700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde1c06b230 0x7fde1c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.551+0000 7fde39d9b700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde1c06b230 0x7fde1c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.551+0000 7fde2affd700 1 -- 172.21.15.83:0/2622093708 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde2c020070 con 0x7fde3c0ce830 2023-11-26T07:15:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.552+0000 7fde39d9b700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde1c06b230 0x7fde1c06d6f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0b2cc0 tx=0x7fde30008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.556+0000 7fde2affd700 1 -- 172.21.15.83:0/2622093708 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde2c055bb0 con 0x7fde3c0ce830 2023-11-26T07:15:54.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.747+0000 7fde40a9d700 1 -- 172.21.15.83:0/2622093708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde3c0ca120 con 0x7fde3c0ce830 2023-11-26T07:15:54.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.747+0000 7fde2affd700 1 -- 172.21.15.83:0/2622093708 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde2c0591d0 con 0x7fde3c0ce830 2023-11-26T07:15:54.750 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:15:54.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 -- 172.21.15.83:0/2622093708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde1c06b230 msgr2=0x7fde1c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde1c06b230 0x7fde1c06d6f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0b2cc0 tx=0x7fde30008040 comp rx=0 tx=0).stop 2023-11-26T07:15:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 -- 172.21.15.83:0/2622093708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde3c0ce830 msgr2=0x7fde3c0b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde3c0ce830 0x7fde3c0b6fd0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009c40 tx=0x7fde2c004750 comp rx=0 tx=0).stop 2023-11-26T07:15:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 -- 172.21.15.83:0/2622093708 shutdown_connections 2023-11-26T07:15:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fde1c06b230 0x7fde1c06d6f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde3c15bf50 0x7fde3c157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde3c0ce830 0x7fde3c0b6fd0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 --2- 172.21.15.83:0/2622093708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde3c0cf1f0 0x7fde3c157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 -- 172.21.15.83:0/2622093708 >> 172.21.15.83:0/2622093708 conn(0x7fde3c0b3780 msgr2=0x7fde3c0d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 -- 172.21.15.83:0/2622093708 shutdown_connections 2023-11-26T07:15:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.749+0000 7fde40a9d700 1 -- 172.21.15.83:0/2622093708 wait complete. 2023-11-26T07:15:54.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:15:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.881+0000 7f753474b700 1 Processor -- start 2023-11-26T07:15:54.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.881+0000 7f753474b700 1 -- start start 2023-11-26T07:15:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.882+0000 7f753474b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f752c0b49a0 0x7f752c0d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.882+0000 7f753474b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752c15c4b0 con 0x7f752c06c710 2023-11-26T07:15:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.882+0000 7f753474b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752c15c790 con 0x7f752c152cc0 2023-11-26T07:15:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.882+0000 7f753474b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752c15ca70 con 0x7f752c0b49a0 2023-11-26T07:15:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.882+0000 7f7531ce6700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f752c06c710 0x7f752c0cff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59846/0 (socket says 172.21.15.83:59846) 2023-11-26T07:15:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.882+0000 7f7531ce6700 1 -- 172.21.15.83:0/1906650894 learned_addr learned my addr 172.21.15.83:0/1906650894 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.882+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232304674 0 0) 0x7f752c15c4b0 con 0x7f752c06c710 2023-11-26T07:15:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.882+0000 7f7532ce8700 1 --2- 172.21.15.83:0/1906650894 >> v2:172.21.15.203:3300/0 conn(0x7f752c0b49a0 0x7f752c0d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.882+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75180035b0 con 0x7f752c06c710 2023-11-26T07:15:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3956684081 0 0) 0x7f75180035b0 con 0x7f752c06c710 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 >> v2:172.21.15.203:3300/0 conn(0x7f752c0b49a0 msgr2=0x7f752c0d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 --2- 172.21.15.83:0/1906650894 >> v2:172.21.15.203:3300/0 conn(0x7f752c0b49a0 0x7f752c0d0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 >> v1:172.21.15.203:6789/0 conn(0x7f752c152cc0 legacy=0x7f752c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c15cd50 con 0x7f752c06c710 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f751c001670 con 0x7f752c06c710 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 >> v1:172.21.15.83:6789/0 conn(0x7f752c06c710 legacy=0x7f752c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 --2- 172.21.15.83:0/1906650894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7518003a00 0x7f7518003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 --2- 172.21.15.83:0/1906650894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7518004850 0x7f7518004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 --2- 172.21.15.83:0/1906650894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7518005210 0x7f751800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c15ca70 con 0x7f7518004850 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751800c1b0 con 0x7f7518003a00 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751800c370 con 0x7f7518005210 2023-11-26T07:15:54.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f7531ce6700 1 --2- 172.21.15.83:0/1906650894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7518004850 0x7f7518004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f75324e7700 1 --2- 172.21.15.83:0/1906650894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7518003a00 0x7f7518003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f7532ce8700 1 --2- 172.21.15.83:0/1906650894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7518005210 0x7f751800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f7531ce6700 1 -- 172.21.15.83:0/1906650894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7518005210 msgr2=0x7f751800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f7531ce6700 1 --2- 172.21.15.83:0/1906650894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7518005210 0x7f751800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f7531ce6700 1 -- 172.21.15.83:0/1906650894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7518003a00 msgr2=0x7f7518003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f7531ce6700 1 --2- 172.21.15.83:0/1906650894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7518003a00 0x7f7518003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f7532ce8700 1 -- 172.21.15.83:0/1906650894 reap_dead start 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.883+0000 7f7531ce6700 1 -- 172.21.15.83:0/1906650894 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75180035b0 con 0x7f7518004850 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.884+0000 7f7531ce6700 1 --2- 172.21.15.83:0/1906650894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7518004850 0x7f7518004cd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c240 tx=0x7f7518005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2513868b357eae82 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.884+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c007c10 con 0x7f7518004850 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.884+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751c00cd60 con 0x7f7518004850 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.884+0000 7f75314e5700 1 -- 172.21.15.83:0/1906650894 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c005ba0 con 0x7f7518004850 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.884+0000 7f753474b700 1 -- 172.21.15.83:0/1906650894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7518004850 msgr2=0x7f7518004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.884+0000 7f753474b700 1 --2- 172.21.15.83:0/1906650894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7518004850 0x7f7518004cd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c240 tx=0x7f7518005880 comp rx=0 tx=0).stop 2023-11-26T07:15:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.884+0000 7f753474b700 1 -- 172.21.15.83:0/1906650894 shutdown_connections 2023-11-26T07:15:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.884+0000 7f753474b700 1 --2- 172.21.15.83:0/1906650894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7518004850 0x7f7518004cd0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.884+0000 7f753474b700 1 -- 172.21.15.83:0/1906650894 >> 172.21.15.83:0/1906650894 conn(0x7f752c071b90 msgr2=0x7f752c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.884+0000 7f753474b700 1 -- 172.21.15.83:0/1906650894 shutdown_connections 2023-11-26T07:15:54.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.884+0000 7f753474b700 1 -- 172.21.15.83:0/1906650894 wait complete. 2023-11-26T07:15:54.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.886+0000 7f753474b700 1 Processor -- start 2023-11-26T07:15:54.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.886+0000 7f753474b700 1 -- start start 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f753474b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7518004850 0x7f752c155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f753474b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752c0b1080 0x7f752c0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f753474b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f752c158ae0 0x7f752c158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f753474b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c0b1a20 con 0x7f752c0b1080 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f753474b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c13dfc0 con 0x7f7518004850 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f753474b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c14e670 con 0x7f752c158ae0 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f7531ce6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752c0b1080 0x7f752c0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f7531ce6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752c0b1080 0x7f752c0b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39818/0 (socket says 172.21.15.83:39818) 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f7531ce6700 1 -- 172.21.15.83:0/2783764874 learned_addr learned my addr 172.21.15.83:0/2783764874 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f7531ce6700 1 -- 172.21.15.83:0/2783764874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f752c158ae0 msgr2=0x7f752c158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f7531ce6700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f752c158ae0 0x7f752c158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f7531ce6700 1 -- 172.21.15.83:0/2783764874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7518004850 msgr2=0x7f752c155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f75324e7700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7518004850 0x7f752c155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f7531ce6700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7518004850 0x7f752c155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f7531ce6700 1 -- 172.21.15.83:0/2783764874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c13e250 con 0x7f752c0b1080 2023-11-26T07:15:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f7531ce6700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752c0b1080 0x7f752c0b14e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f751c001850 tx=0x7f751c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.887+0000 7f7522ffd700 1 -- 172.21.15.83:0/2783764874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c00ced0 con 0x7f752c0b1080 2023-11-26T07:15:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.888+0000 7f7522ffd700 1 -- 172.21.15.83:0/2783764874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751c001640 con 0x7f752c0b1080 2023-11-26T07:15:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.888+0000 7f753474b700 1 -- 172.21.15.83:0/2783764874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c0b1c40 con 0x7f752c0b1080 2023-11-26T07:15:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.888+0000 7f7522ffd700 1 -- 172.21.15.83:0/2783764874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c0072d0 con 0x7f752c0b1080 2023-11-26T07:15:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.888+0000 7f753474b700 1 -- 172.21.15.83:0/2783764874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c1582e0 con 0x7f752c0b1080 2023-11-26T07:15:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.889+0000 7f753474b700 1 -- 172.21.15.83:0/2783764874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f752c0753f0 con 0x7f752c0b1080 2023-11-26T07:15:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.889+0000 7f7522ffd700 1 -- 172.21.15.83:0/2783764874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f751c007430 con 0x7f752c0b1080 2023-11-26T07:15:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.889+0000 7f7522ffd700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f751006b260 0x7f751006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:54.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.889+0000 7f7522ffd700 1 -- 172.21.15.83:0/2783764874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f751c08b060 con 0x7f752c0b1080 2023-11-26T07:15:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.889+0000 7f75324e7700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f751006b260 0x7f751006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:54.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.890+0000 7f75324e7700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f751006b260 0x7f751006d720 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f752c0505a0 tx=0x7f7528009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:54.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:54.894+0000 7f7522ffd700 1 -- 172.21.15.83:0/2783764874 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f751c055eb0 con 0x7f752c0b1080 2023-11-26T07:15:55.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.098+0000 7f753474b700 1 -- 172.21.15.83:0/2783764874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f752c067f60 con 0x7f752c0b1080 2023-11-26T07:15:55.101 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.099+0000 7f7522ffd700 1 -- 172.21.15.83:0/2783764874 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) v1 ==== 80+0+0 (secure 0 0 0) 0x7f751c0076e0 con 0x7f752c0b1080 2023-11-26T07:15:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 -- 172.21.15.83:0/2783764874 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f751006b260 msgr2=0x7f751006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f751006b260 0x7f751006d720 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f752c0505a0 tx=0x7f7528009040 comp rx=0 tx=0).stop 2023-11-26T07:15:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 -- 172.21.15.83:0/2783764874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752c0b1080 msgr2=0x7f752c0b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752c0b1080 0x7f752c0b14e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f751c001850 tx=0x7f751c001a00 comp rx=0 tx=0).stop 2023-11-26T07:15:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 -- 172.21.15.83:0/2783764874 shutdown_connections 2023-11-26T07:15:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f751006b260 0x7f751006d720 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f752c158ae0 0x7f752c158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7518004850 0x7f752c155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 --2- 172.21.15.83:0/2783764874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f752c0b1080 0x7f752c0b14e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 -- 172.21.15.83:0/2783764874 >> 172.21.15.83:0/2783764874 conn(0x7f752c071b90 msgr2=0x7f752c15a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 -- 172.21.15.83:0/2783764874 shutdown_connections 2023-11-26T07:15:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.101+0000 7f753474b700 1 -- 172.21.15.83:0/2783764874 wait complete. 2023-11-26T07:15:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:15:55.116 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:15:55.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:55.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:15:55.156 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:15:55.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:55.157 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:15:55.213 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:15:55.213 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:55.234 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:15:55.235 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:15:55.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:15:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.416+0000 7feb8d2e5700 1 Processor -- start 2023-11-26T07:15:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.416+0000 7feb8d2e5700 1 -- start start 2023-11-26T07:15:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb8d2e5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb8815c030 0x7feb880d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb8d2e5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb880ae910 0x7feb880d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb8d2e5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feb880cc8c0 0x7feb880ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb8d2e5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb880cd5d0 con 0x7feb880ae910 2023-11-26T07:15:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb8d2e5700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb880cd870 con 0x7feb880cc8c0 2023-11-26T07:15:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb8d2e5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb880cdb50 con 0x7feb8815c030 2023-11-26T07:15:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb867fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb880ae910 0x7feb880d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb867fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb880ae910 0x7feb880d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39826/0 (socket says 172.21.15.83:39826) 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb867fc700 1 -- 172.21.15.83:0/3282327499 learned_addr learned my addr 172.21.15.83:0/3282327499 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb877fe700 1 --2- 172.21.15.83:0/3282327499 >> v2:172.21.15.203:3300/0 conn(0x7feb880cc8c0 0x7feb880ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb867fc700 1 -- 172.21.15.83:0/3282327499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb8815c030 msgr2=0x7feb880d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb86ffd700 1 --2- 172.21.15.83:0/3282327499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb8815c030 0x7feb880d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb867fc700 1 --2- 172.21.15.83:0/3282327499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb8815c030 0x7feb880d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb867fc700 1 -- 172.21.15.83:0/3282327499 >> v2:172.21.15.203:3300/0 conn(0x7feb880cc8c0 msgr2=0x7feb880ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb867fc700 1 --2- 172.21.15.83:0/3282327499 >> v2:172.21.15.203:3300/0 conn(0x7feb880cc8c0 0x7feb880ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb867fc700 1 -- 172.21.15.83:0/3282327499 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb880cde30 con 0x7feb880ae910 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb86ffd700 1 --2- 172.21.15.83:0/3282327499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb8815c030 0x7feb880d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb867fc700 1 --2- 172.21.15.83:0/3282327499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb880ae910 0x7feb880d05b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7feb8809b7c0 tx=0x7feb70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1514484ef8ddab1c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.417+0000 7feb85ffb700 1 -- 172.21.15.83:0/3282327499 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7000b870 con 0x7feb880ae910 2023-11-26T07:15:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.418+0000 7feb85ffb700 1 -- 172.21.15.83:0/3282327499 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7000b9d0 con 0x7feb880ae910 2023-11-26T07:15:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.418+0000 7feb85ffb700 1 -- 172.21.15.83:0/3282327499 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7000bd60 con 0x7feb880ae910 2023-11-26T07:15:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.418+0000 7feb8d2e5700 1 -- 172.21.15.83:0/3282327499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb880ae910 msgr2=0x7feb880d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.418+0000 7feb8d2e5700 1 --2- 172.21.15.83:0/3282327499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb880ae910 0x7feb880d05b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7feb8809b7c0 tx=0x7feb70009bd0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.418+0000 7feb8d2e5700 1 -- 172.21.15.83:0/3282327499 shutdown_connections 2023-11-26T07:15:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.418+0000 7feb8d2e5700 1 --2- 172.21.15.83:0/3282327499 >> v2:172.21.15.203:3300/0 conn(0x7feb880cc8c0 0x7feb880ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.418+0000 7feb8d2e5700 1 --2- 172.21.15.83:0/3282327499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb8815c030 0x7feb880d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.418+0000 7feb8d2e5700 1 --2- 172.21.15.83:0/3282327499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb880ae910 0x7feb880d05b0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.418+0000 7feb8d2e5700 1 -- 172.21.15.83:0/3282327499 >> 172.21.15.83:0/3282327499 conn(0x7feb880730a0 msgr2=0x7feb8809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.418+0000 7feb8d2e5700 1 -- 172.21.15.83:0/3282327499 shutdown_connections 2023-11-26T07:15:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.418+0000 7feb8d2e5700 1 -- 172.21.15.83:0/3282327499 wait complete. 2023-11-26T07:15:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.420+0000 7feb8d2e5700 1 Processor -- start 2023-11-26T07:15:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.420+0000 7feb8d2e5700 1 -- start start 2023-11-26T07:15:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb8d2e5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb880ae910 0x7feb880ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb8d2e5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb880cc8c0 0x7feb880c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb8d2e5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb881f2640 0x7feb881f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb8d2e5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb880cb340 con 0x7feb880cc8c0 2023-11-26T07:15:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb8d2e5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8815cf60 con 0x7feb880ae910 2023-11-26T07:15:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb8d2e5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8815c5c0 con 0x7feb881f2640 2023-11-26T07:15:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb867fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb880cc8c0 0x7feb880c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb867fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb880cc8c0 0x7feb880c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39832/0 (socket says 172.21.15.83:39832) 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb867fc700 1 -- 172.21.15.83:0/2453017631 learned_addr learned my addr 172.21.15.83:0/2453017631 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb86ffd700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb880ae910 0x7feb880ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb877fe700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb881f2640 0x7feb881f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb867fc700 1 -- 172.21.15.83:0/2453017631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb881f2640 msgr2=0x7feb881f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb867fc700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb881f2640 0x7feb881f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb867fc700 1 -- 172.21.15.83:0/2453017631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb880ae910 msgr2=0x7feb880ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb867fc700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb880ae910 0x7feb880ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb867fc700 1 -- 172.21.15.83:0/2453017631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb70009550 con 0x7feb880cc8c0 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb867fc700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb880cc8c0 0x7feb880c7ec0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7feb70000c00 tx=0x7feb70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb7f7fe700 1 -- 172.21.15.83:0/2453017631 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70004540 con 0x7feb880cc8c0 2023-11-26T07:15:55.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb7f7fe700 1 -- 172.21.15.83:0/2453017631 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb700036a0 con 0x7feb880cc8c0 2023-11-26T07:15:55.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb7f7fe700 1 -- 172.21.15.83:0/2453017631 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70003a50 con 0x7feb880cc8c0 2023-11-26T07:15:55.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb8d2e5700 1 -- 172.21.15.83:0/2453017631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8815c8a0 con 0x7feb880cc8c0 2023-11-26T07:15:55.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.421+0000 7feb8d2e5700 1 -- 172.21.15.83:0/2453017631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb880caf30 con 0x7feb880cc8c0 2023-11-26T07:15:55.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.422+0000 7feb8d2e5700 1 -- 172.21.15.83:0/2453017631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8815d1d0 con 0x7feb880cc8c0 2023-11-26T07:15:55.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.423+0000 7feb7f7fe700 1 -- 172.21.15.83:0/2453017631 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7feb70021070 con 0x7feb880cc8c0 2023-11-26T07:15:55.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.423+0000 7feb7f7fe700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feb6806b230 0x7feb6806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.423+0000 7feb7f7fe700 1 -- 172.21.15.83:0/2453017631 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb7008b220 con 0x7feb880cc8c0 2023-11-26T07:15:55.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.423+0000 7feb86ffd700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feb6806b230 0x7feb6806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.424+0000 7feb86ffd700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feb6806b230 0x7feb6806d6f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feb80008ee0 tx=0x7feb80008430 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:55.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.428+0000 7feb7f7fe700 1 -- 172.21.15.83:0/2453017631 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb70055f10 con 0x7feb880cc8c0 2023-11-26T07:15:55.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.620+0000 7feb8d2e5700 1 -- 172.21.15.83:0/2453017631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb880cb0c0 con 0x7feb880cc8c0 2023-11-26T07:15:55.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.622+0000 7feb7f7fe700 1 -- 172.21.15.83:0/2453017631 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16556 (secure 0 0 0) 0x7feb70059530 con 0x7feb880cc8c0 2023-11-26T07:15:55.624 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:55.624 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:15:43.538242+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:15:40.508908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:15:41.299931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:15:55.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 -- 172.21.15.83:0/2453017631 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feb6806b230 msgr2=0x7feb6806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feb6806b230 0x7feb6806d6f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feb80008ee0 tx=0x7feb80008430 comp rx=0 tx=0).stop 2023-11-26T07:15:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 -- 172.21.15.83:0/2453017631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb880cc8c0 msgr2=0x7feb880c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb880cc8c0 0x7feb880c7ec0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7feb70000c00 tx=0x7feb70004750 comp rx=0 tx=0).stop 2023-11-26T07:15:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 -- 172.21.15.83:0/2453017631 shutdown_connections 2023-11-26T07:15:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7feb6806b230 0x7feb6806d6f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb881f2640 0x7feb881f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb880ae910 0x7feb880ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 --2- 172.21.15.83:0/2453017631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb880cc8c0 0x7feb880c7ec0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 -- 172.21.15.83:0/2453017631 >> 172.21.15.83:0/2453017631 conn(0x7feb880730a0 msgr2=0x7feb880cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 -- 172.21.15.83:0/2453017631 shutdown_connections 2023-11-26T07:15:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.624+0000 7feb8d2e5700 1 -- 172.21.15.83:0/2453017631 wait complete. 2023-11-26T07:15:55.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:15:40.508908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:15:41.299931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:15:55.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:15:55.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.759+0000 7fca4692e700 1 Processor -- start 2023-11-26T07:15:55.762 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.759+0000 7fca4692e700 1 -- start start 2023-11-26T07:15:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.760+0000 7fca4692e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fca4015e160 0x7fca400b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.760+0000 7fca4692e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fca400afc90 0x7fca40073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.760+0000 7fca4692e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca400b3c60 con 0x7fca400744d0 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.760+0000 7fca4692e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca400b4500 con 0x7fca4015e160 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.760+0000 7fca4692e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca40074940 con 0x7fca400afc90 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.760+0000 7fca3ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fca4015e160 0x7fca400b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.760+0000 7fca3ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fca4015e160 0x7fca400b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39838/0 (socket says 172.21.15.83:39838) 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.760+0000 7fca3ffff700 1 -- 172.21.15.83:0/3216721981 learned_addr learned my addr 172.21.15.83:0/3216721981 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.761+0000 7fca3f7fe700 1 --2- 172.21.15.83:0/3216721981 >> v2:172.21.15.203:3300/0 conn(0x7fca400afc90 0x7fca40073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.761+0000 7fca3ffff700 1 -- 172.21.15.83:0/3216721981 >> v2:172.21.15.203:3300/0 conn(0x7fca400afc90 msgr2=0x7fca40073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.761+0000 7fca3ffff700 1 --2- 172.21.15.83:0/3216721981 >> v2:172.21.15.203:3300/0 conn(0x7fca400afc90 0x7fca40073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.761+0000 7fca3ffff700 1 -- 172.21.15.83:0/3216721981 >> v1:172.21.15.203:6789/0 conn(0x7fca400744d0 legacy=0x7fca4006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.761+0000 7fca3ffff700 1 -- 172.21.15.83:0/3216721981 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca400b58c0 con 0x7fca4015e160 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.761+0000 7fca3f7fe700 1 --2- 172.21.15.83:0/3216721981 >> v2:172.21.15.203:3300/0 conn(0x7fca400afc90 0x7fca40073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.761+0000 7fca3ffff700 1 --2- 172.21.15.83:0/3216721981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca4015e160 0x7fca400b5da0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fca400d0880 tx=0x7fca34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c426b30c80e84cf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.761+0000 7fca3effd700 1 -- 172.21.15.83:0/3216721981 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34004230 con 0x7fca4015e160 2023-11-26T07:15:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.761+0000 7fca3effd700 1 -- 172.21.15.83:0/3216721981 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca34004390 con 0x7fca4015e160 2023-11-26T07:15:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.761+0000 7fca3effd700 1 -- 172.21.15.83:0/3216721981 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340036a0 con 0x7fca4015e160 2023-11-26T07:15:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.761+0000 7fca4692e700 1 -- 172.21.15.83:0/3216721981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca4015e160 msgr2=0x7fca400b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.762+0000 7fca4692e700 1 --2- 172.21.15.83:0/3216721981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca4015e160 0x7fca400b5da0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fca400d0880 tx=0x7fca34009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.762+0000 7fca4692e700 1 -- 172.21.15.83:0/3216721981 shutdown_connections 2023-11-26T07:15:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.762+0000 7fca4692e700 1 --2- 172.21.15.83:0/3216721981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca4015e160 0x7fca400b5da0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.762+0000 7fca4692e700 1 --2- 172.21.15.83:0/3216721981 >> v2:172.21.15.203:3300/0 conn(0x7fca400afc90 0x7fca40073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.762+0000 7fca4692e700 1 -- 172.21.15.83:0/3216721981 >> 172.21.15.83:0/3216721981 conn(0x7fca400d0bf0 msgr2=0x7fca400d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.762+0000 7fca4692e700 1 -- 172.21.15.83:0/3216721981 shutdown_connections 2023-11-26T07:15:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.762+0000 7fca4692e700 1 -- 172.21.15.83:0/3216721981 wait complete. 2023-11-26T07:15:55.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.763+0000 7fca4692e700 1 Processor -- start 2023-11-26T07:15:55.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.763+0000 7fca4692e700 1 -- start start 2023-11-26T07:15:55.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca4692e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca400744d0 0x7fca401f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca4692e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca400afc90 0x7fca4015e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca4692e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4015e160 0x7fca401567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca4692e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4015f8f0 con 0x7fca400744d0 2023-11-26T07:15:55.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca4692e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4015ea80 con 0x7fca4015e160 2023-11-26T07:15:55.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca4692e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4015ece0 con 0x7fca400afc90 2023-11-26T07:15:55.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca400744d0 0x7fca401f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca400744d0 0x7fca401f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39844/0 (socket says 172.21.15.83:39844) 2023-11-26T07:15:55.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3ffff700 1 -- 172.21.15.83:0/3384677544 learned_addr learned my addr 172.21.15.83:0/3384677544 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca44ecb700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4015e160 0x7fca401567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3f7fe700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca400afc90 0x7fca4015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3ffff700 1 -- 172.21.15.83:0/3384677544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca400afc90 msgr2=0x7fca4015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3ffff700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca400afc90 0x7fca4015e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3ffff700 1 -- 172.21.15.83:0/3384677544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4015e160 msgr2=0x7fca401567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3ffff700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4015e160 0x7fca401567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3ffff700 1 -- 172.21.15.83:0/3384677544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34009550 con 0x7fca400744d0 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3f7fe700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca400afc90 0x7fca4015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3ffff700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca400744d0 0x7fca401f4820 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fca34009870 tx=0x7fca34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3cff9700 1 -- 172.21.15.83:0/3384677544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34011830 con 0x7fca400744d0 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3cff9700 1 -- 172.21.15.83:0/3384677544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca340119b0 con 0x7fca400744d0 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca4692e700 1 -- 172.21.15.83:0/3384677544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4015efc0 con 0x7fca400744d0 2023-11-26T07:15:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca3cff9700 1 -- 172.21.15.83:0/3384677544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34011d60 con 0x7fca400744d0 2023-11-26T07:15:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.764+0000 7fca4692e700 1 -- 172.21.15.83:0/3384677544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4015f580 con 0x7fca400744d0 2023-11-26T07:15:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.765+0000 7fca4692e700 1 -- 172.21.15.83:0/3384677544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4015fbd0 con 0x7fca400744d0 2023-11-26T07:15:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.766+0000 7fca3cff9700 1 -- 172.21.15.83:0/3384677544 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fca3401b440 con 0x7fca400744d0 2023-11-26T07:15:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.766+0000 7fca3cff9700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fca2c06b230 0x7fca2c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.766+0000 7fca3cff9700 1 -- 172.21.15.83:0/3384677544 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca3408b490 con 0x7fca400744d0 2023-11-26T07:15:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.766+0000 7fca3f7fe700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fca2c06b230 0x7fca2c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.767+0000 7fca3f7fe700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fca2c06b230 0x7fca2c06d6f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fca400b2420 tx=0x7fca30008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.771+0000 7fca3cff9700 1 -- 172.21.15.83:0/3384677544 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fca340561c0 con 0x7fca400744d0 2023-11-26T07:15:55.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.958+0000 7fca4692e700 1 -- 172.21.15.83:0/3384677544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca40068fb0 con 0x7fca400744d0 2023-11-26T07:15:55.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.958+0000 7fca3cff9700 1 -- 172.21.15.83:0/3384677544 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca340597e0 con 0x7fca400744d0 2023-11-26T07:15:55.961 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:15:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.960+0000 7fca4692e700 1 -- 172.21.15.83:0/3384677544 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fca2c06b230 msgr2=0x7fca2c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.960+0000 7fca4692e700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fca2c06b230 0x7fca2c06d6f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fca400b2420 tx=0x7fca30008040 comp rx=0 tx=0).stop 2023-11-26T07:15:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.960+0000 7fca4692e700 1 -- 172.21.15.83:0/3384677544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca400744d0 msgr2=0x7fca401f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:55.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.960+0000 7fca4692e700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca400744d0 0x7fca401f4820 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fca34009870 tx=0x7fca34004750 comp rx=0 tx=0).stop 2023-11-26T07:15:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.960+0000 7fca4692e700 1 -- 172.21.15.83:0/3384677544 shutdown_connections 2023-11-26T07:15:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.960+0000 7fca4692e700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca4015e160 0x7fca401567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.960+0000 7fca4692e700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca400744d0 0x7fca401f4820 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.960+0000 7fca4692e700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fca2c06b230 0x7fca2c06d6f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.960+0000 7fca4692e700 1 --2- 172.21.15.83:0/3384677544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca400afc90 0x7fca4015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.960+0000 7fca4692e700 1 -- 172.21.15.83:0/3384677544 >> 172.21.15.83:0/3384677544 conn(0x7fca400d0bf0 msgr2=0x7fca4006ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.961+0000 7fca4692e700 1 -- 172.21.15.83:0/3384677544 shutdown_connections 2023-11-26T07:15:55.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:55.961+0000 7fca4692e700 1 -- 172.21.15.83:0/3384677544 wait complete. 2023-11-26T07:15:55.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:15:56.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.096+0000 7f2aecc46700 1 Processor -- start 2023-11-26T07:15:56.098 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.096+0000 7f2aecc46700 1 -- start start 2023-11-26T07:15:56.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.097+0000 7f2aecc46700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2ae815be70 0x7f2ae80cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.097+0000 7f2aecc46700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae80b5ee0 con 0x7f2ae80add40 2023-11-26T07:15:56.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.097+0000 7f2aecc46700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae806ba00 con 0x7f2ae80746a0 2023-11-26T07:15:56.099 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.097+0000 7f2aecc46700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae806bfc0 con 0x7f2ae815be70 2023-11-26T07:15:56.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.097+0000 7f2ae5d9b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2ae80add40 0x7f2ae8073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59892/0 (socket says 172.21.15.83:59892) 2023-11-26T07:15:56.100 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.097+0000 7f2ae5d9b700 1 -- 172.21.15.83:0/1662737544 learned_addr learned my addr 172.21.15.83:0/1662737544 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:56.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.097+0000 7f2ae659c700 1 --2- 172.21.15.83:0/1662737544 >> v2:172.21.15.203:3300/0 conn(0x7f2ae815be70 0x7f2ae80cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.097+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289431197 0 0) 0x7f2ae80b5ee0 con 0x7f2ae80add40 2023-11-26T07:15:56.102 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.097+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad00035b0 con 0x7f2ae80add40 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1762534865 0 0) 0x7f2ad00035b0 con 0x7f2ae80add40 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 >> v2:172.21.15.203:3300/0 conn(0x7f2ae815be70 msgr2=0x7f2ae80cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 --2- 172.21.15.83:0/1662737544 >> v2:172.21.15.203:3300/0 conn(0x7f2ae815be70 0x7f2ae80cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 >> v1:172.21.15.203:6789/0 conn(0x7f2ae80746a0 legacy=0x7f2ae80b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae806bce0 con 0x7f2ae80add40 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f2ad8001670 con 0x7f2ae80add40 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 >> v1:172.21.15.83:6789/0 conn(0x7f2ae80add40 legacy=0x7f2ae8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 --2- 172.21.15.83:0/1662737544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ad0003a00 0x7f2ad0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 --2- 172.21.15.83:0/1662737544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad0004850 0x7f2ad0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 --2- 172.21.15.83:0/1662737544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ad0005210 0x7f2ad000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae806bfc0 con 0x7f2ad0004850 2023-11-26T07:15:56.103 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad000c1b0 con 0x7f2ad0005210 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad000c370 con 0x7f2ad0003a00 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae5d9b700 1 --2- 172.21.15.83:0/1662737544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad0004850 0x7f2ad0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae659c700 1 --2- 172.21.15.83:0/1662737544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ad0003a00 0x7f2ad0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae6d9d700 1 --2- 172.21.15.83:0/1662737544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ad0005210 0x7f2ad000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae5d9b700 1 -- 172.21.15.83:0/1662737544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ad0003a00 msgr2=0x7f2ad0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae5d9b700 1 --2- 172.21.15.83:0/1662737544 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ad0003a00 0x7f2ad0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae5d9b700 1 -- 172.21.15.83:0/1662737544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ad0005210 msgr2=0x7f2ad000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae5d9b700 1 --2- 172.21.15.83:0/1662737544 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ad0005210 0x7f2ad000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae6d9d700 1 -- 172.21.15.83:0/1662737544 reap_dead start 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae5d9b700 1 -- 172.21.15.83:0/1662737544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad00035b0 con 0x7f2ad0004850 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae5d9b700 1 --2- 172.21.15.83:0/1662737544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad0004850 0x7f2ad0004cd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800c240 tx=0x7f2ad0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d972283045abb86 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:56.104 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8007c10 con 0x7f2ad0004850 2023-11-26T07:15:56.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad800cde0 con 0x7f2ad0004850 2023-11-26T07:15:56.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.098+0000 7f2ae559a700 1 -- 172.21.15.83:0/1662737544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8005ba0 con 0x7f2ad0004850 2023-11-26T07:15:56.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.099+0000 7f2aecc46700 1 -- 172.21.15.83:0/1662737544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad0004850 msgr2=0x7f2ad0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.099+0000 7f2aecc46700 1 --2- 172.21.15.83:0/1662737544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad0004850 0x7f2ad0004cd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800c240 tx=0x7f2ad0005880 comp rx=0 tx=0).stop 2023-11-26T07:15:56.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.099+0000 7f2aecc46700 1 -- 172.21.15.83:0/1662737544 shutdown_connections 2023-11-26T07:15:56.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.099+0000 7f2aecc46700 1 --2- 172.21.15.83:0/1662737544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad0004850 0x7f2ad0004cd0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.099+0000 7f2aecc46700 1 -- 172.21.15.83:0/1662737544 >> 172.21.15.83:0/1662737544 conn(0x7f2ae80d0bf0 msgr2=0x7f2ae80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:56.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.099+0000 7f2aecc46700 1 -- 172.21.15.83:0/1662737544 shutdown_connections 2023-11-26T07:15:56.105 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.099+0000 7f2aecc46700 1 -- 172.21.15.83:0/1662737544 wait complete. 2023-11-26T07:15:56.107 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.100+0000 7f2aecc46700 1 Processor -- start 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.100+0000 7f2aecc46700 1 -- start start 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.101+0000 7f2aecc46700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ad0004850 0x7f2ae815d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.101+0000 7f2aecc46700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae81455c0 0x7f2ae815dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.101+0000 7f2aecc46700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae815e2c0 0x7f2ae80b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.101+0000 7f2aecc46700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae815eb20 con 0x7f2ae815e2c0 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.101+0000 7f2aecc46700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae815d590 con 0x7f2ad0004850 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.101+0000 7f2aecc46700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae815e840 con 0x7f2ae81455c0 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.101+0000 7f2ae6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae815e2c0 0x7f2ae80b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.101+0000 7f2ae6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae815e2c0 0x7f2ae80b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39866/0 (socket says 172.21.15.83:39866) 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.101+0000 7f2ae6d9d700 1 -- 172.21.15.83:0/1702612432 learned_addr learned my addr 172.21.15.83:0/1702612432 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2ae5d9b700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae81455c0 0x7f2ae815dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2ae6d9d700 1 -- 172.21.15.83:0/1702612432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae81455c0 msgr2=0x7f2ae815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.108 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2ae6d9d700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae81455c0 0x7f2ae815dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2ae6d9d700 1 -- 172.21.15.83:0/1702612432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ad0004850 msgr2=0x7f2ae815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2ae659c700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ad0004850 0x7f2ae815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2ae6d9d700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ad0004850 0x7f2ae815d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2ae6d9d700 1 -- 172.21.15.83:0/1702612432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae815c570 con 0x7f2ae815e2c0 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2ae6d9d700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae815e2c0 0x7f2ae80b4120 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8072cd0 tx=0x7f2ae000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2ad6ffd700 1 -- 172.21.15.83:0/1702612432 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0003c10 con 0x7f2ae815e2c0 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2ad6ffd700 1 -- 172.21.15.83:0/1702612432 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae0003d90 con 0x7f2ae815e2c0 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2aecc46700 1 -- 172.21.15.83:0/1702612432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae815c850 con 0x7f2ae815e2c0 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2ad6ffd700 1 -- 172.21.15.83:0/1702612432 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0005310 con 0x7f2ae815e2c0 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.102+0000 7f2aecc46700 1 -- 172.21.15.83:0/1702612432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae8150ef0 con 0x7f2ae815e2c0 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.103+0000 7f2aecc46700 1 -- 172.21.15.83:0/1702612432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae8068fb0 con 0x7f2ae815e2c0 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.103+0000 7f2ad6ffd700 1 -- 172.21.15.83:0/1702612432 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f2ae0005470 con 0x7f2ae815e2c0 2023-11-26T07:15:56.109 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.104+0000 7f2ad6ffd700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2ac806b230 0x7f2ac806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.104+0000 7f2ad6ffd700 1 -- 172.21.15.83:0/1702612432 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ae0089a10 con 0x7f2ae815e2c0 2023-11-26T07:15:56.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.104+0000 7f2ae659c700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2ac806b230 0x7f2ac806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.104+0000 7f2ae659c700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2ac806b230 0x7f2ac806d6f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ae81f6fa0 tx=0x7f2adc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:56.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.108+0000 7f2ad6ffd700 1 -- 172.21.15.83:0/1702612432 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ae0054800 con 0x7f2ae815e2c0 2023-11-26T07:15:56.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.298+0000 7f2aecc46700 1 -- 172.21.15.83:0/1702612432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ae80d0830 con 0x7f2ae815e2c0 2023-11-26T07:15:56.301 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.298+0000 7f2ad6ffd700 1 -- 172.21.15.83:0/1702612432 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ae001a310 con 0x7f2ae815e2c0 2023-11-26T07:15:56.301 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:15:56.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.300+0000 7f2aecc46700 1 -- 172.21.15.83:0/1702612432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2ac806b230 msgr2=0x7f2ac806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.300+0000 7f2aecc46700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2ac806b230 0x7f2ac806d6f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ae81f6fa0 tx=0x7f2adc008040 comp rx=0 tx=0).stop 2023-11-26T07:15:56.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.300+0000 7f2aecc46700 1 -- 172.21.15.83:0/1702612432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae815e2c0 msgr2=0x7f2ae80b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.300+0000 7f2aecc46700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae815e2c0 0x7f2ae80b4120 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8072cd0 tx=0x7f2ae000ee00 comp rx=0 tx=0).stop 2023-11-26T07:15:56.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.300+0000 7f2aecc46700 1 -- 172.21.15.83:0/1702612432 shutdown_connections 2023-11-26T07:15:56.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.300+0000 7f2aecc46700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae815e2c0 0x7f2ae80b4120 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.300+0000 7f2aecc46700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ad0004850 0x7f2ae815d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.301+0000 7f2aecc46700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f2ac806b230 0x7f2ac806d6f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.301+0000 7f2aecc46700 1 --2- 172.21.15.83:0/1702612432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae81455c0 0x7f2ae815dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.301+0000 7f2aecc46700 1 -- 172.21.15.83:0/1702612432 >> 172.21.15.83:0/1702612432 conn(0x7f2ae80d0bf0 msgr2=0x7f2ae806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:56.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.301+0000 7f2aecc46700 1 -- 172.21.15.83:0/1702612432 shutdown_connections 2023-11-26T07:15:56.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.301+0000 7f2aecc46700 1 -- 172.21.15.83:0/1702612432 wait complete. 2023-11-26T07:15:56.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:15:56.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.435+0000 7f89d879b700 1 Processor -- start 2023-11-26T07:15:56.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.435+0000 7f89d879b700 1 -- start start 2023-11-26T07:15:56.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.436+0000 7f89d879b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89d00c6c30 0x7f89d00cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.436+0000 7f89d879b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f89d00cef40 0x7f89d00cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.436+0000 7f89d879b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d00d0140 con 0x7f89d015bf50 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.436+0000 7f89d879b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d015c330 con 0x7f89d00c6c30 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.436+0000 7f89d879b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d00cf3d0 con 0x7f89d00cef40 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.436+0000 7f89d5d36700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89d00c6c30 0x7f89d00cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.436+0000 7f89d6537700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f89d015bf50 0x7f89d00ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59910/0 (socket says 172.21.15.83:59910) 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.436+0000 7f89d6537700 1 -- 172.21.15.83:0/2488833273 learned_addr learned my addr 172.21.15.83:0/2488833273 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.437+0000 7f89d6d38700 1 --2- 172.21.15.83:0/2488833273 >> v2:172.21.15.203:3300/0 conn(0x7f89d00cef40 0x7f89d00cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.437+0000 7f89d5d36700 1 -- 172.21.15.83:0/2488833273 >> v2:172.21.15.203:3300/0 conn(0x7f89d00cef40 msgr2=0x7f89d00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.437+0000 7f89d5d36700 1 --2- 172.21.15.83:0/2488833273 >> v2:172.21.15.203:3300/0 conn(0x7f89d00cef40 0x7f89d00cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.437+0000 7f89d5d36700 1 -- 172.21.15.83:0/2488833273 >> v1:172.21.15.83:6789/0 conn(0x7f89d015bf50 legacy=0x7f89d00ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.437+0000 7f89d5d36700 1 -- 172.21.15.83:0/2488833273 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d00d03e0 con 0x7f89d00c6c30 2023-11-26T07:15:56.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.437+0000 7f89d5d36700 1 --2- 172.21.15.83:0/2488833273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d00c6c30 0x7f89d00cea00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f89d00c8c00 tx=0x7f89cc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158df59b5fb7e936 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:56.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.437+0000 7f89d5535700 1 -- 172.21.15.83:0/2488833273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc011830 con 0x7f89d00c6c30 2023-11-26T07:15:56.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.437+0000 7f89d5535700 1 -- 172.21.15.83:0/2488833273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc011990 con 0x7f89d00c6c30 2023-11-26T07:15:56.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.437+0000 7f89d5535700 1 -- 172.21.15.83:0/2488833273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc011d20 con 0x7f89d00c6c30 2023-11-26T07:15:56.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.437+0000 7f89d879b700 1 -- 172.21.15.83:0/2488833273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d00c6c30 msgr2=0x7f89d00cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.437+0000 7f89d879b700 1 --2- 172.21.15.83:0/2488833273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d00c6c30 0x7f89d00cea00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f89d00c8c00 tx=0x7f89cc009b20 comp rx=0 tx=0).stop 2023-11-26T07:15:56.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.438+0000 7f89d879b700 1 -- 172.21.15.83:0/2488833273 shutdown_connections 2023-11-26T07:15:56.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.438+0000 7f89d879b700 1 --2- 172.21.15.83:0/2488833273 >> v2:172.21.15.203:3300/0 conn(0x7f89d00cef40 0x7f89d00cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.438+0000 7f89d879b700 1 --2- 172.21.15.83:0/2488833273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d00c6c30 0x7f89d00cea00 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.438+0000 7f89d879b700 1 -- 172.21.15.83:0/2488833273 >> 172.21.15.83:0/2488833273 conn(0x7f89d00b3780 msgr2=0x7f89d00b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.438+0000 7f89d879b700 1 -- 172.21.15.83:0/2488833273 shutdown_connections 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.438+0000 7f89d879b700 1 -- 172.21.15.83:0/2488833273 wait complete. 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.439+0000 7f89d879b700 1 Processor -- start 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.439+0000 7f89d879b700 1 -- start start 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.439+0000 7f89d879b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89d00c6c30 0x7f89d01572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.439+0000 7f89d879b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d00cef40 0x7f89d01550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d879b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89d015bf50 0x7f89d01f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d879b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01f26f0 con 0x7f89d00cef40 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d879b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01f29d0 con 0x7f89d00c6c30 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d879b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01f2cb0 con 0x7f89d015bf50 2023-11-26T07:15:56.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d5d36700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d00cef40 0x7f89d01550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d5d36700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d00cef40 0x7f89d01550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39878/0 (socket says 172.21.15.83:39878) 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d5d36700 1 -- 172.21.15.83:0/2388995691 learned_addr learned my addr 172.21.15.83:0/2388995691 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d6d38700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89d015bf50 0x7f89d01f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d6537700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89d00c6c30 0x7f89d01572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d5d36700 1 -- 172.21.15.83:0/2388995691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89d015bf50 msgr2=0x7f89d01f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d5d36700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89d015bf50 0x7f89d01f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d5d36700 1 -- 172.21.15.83:0/2388995691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89d00c6c30 msgr2=0x7f89d01572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d5d36700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89d00c6c30 0x7f89d01572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d5d36700 1 -- 172.21.15.83:0/2388995691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc0094a0 con 0x7f89d00cef40 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d6d38700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89d015bf50 0x7f89d01f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d5d36700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d00cef40 0x7f89d01550f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f89cc000c00 tx=0x7f89cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:56.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89c6ffd700 1 -- 172.21.15.83:0/2388995691 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc0038a0 con 0x7f89d00cef40 2023-11-26T07:15:56.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89c6ffd700 1 -- 172.21.15.83:0/2388995691 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc01f070 con 0x7f89d00cef40 2023-11-26T07:15:56.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d879b700 1 -- 172.21.15.83:0/2388995691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d00c8c80 con 0x7f89d00cef40 2023-11-26T07:15:56.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89c6ffd700 1 -- 172.21.15.83:0/2388995691 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc003c20 con 0x7f89d00cef40 2023-11-26T07:15:56.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.440+0000 7f89d879b700 1 -- 172.21.15.83:0/2388995691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d01f2360 con 0x7f89d00cef40 2023-11-26T07:15:56.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.441+0000 7f89d879b700 1 -- 172.21.15.83:0/2388995691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d00cdc90 con 0x7f89d00cef40 2023-11-26T07:15:56.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.442+0000 7f89c6ffd700 1 -- 172.21.15.83:0/2388995691 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f89cc01a430 con 0x7f89d00cef40 2023-11-26T07:15:56.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.442+0000 7f89c6ffd700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f89b406b230 0x7f89b406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.442+0000 7f89d6537700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f89b406b230 0x7f89b406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.442+0000 7f89c6ffd700 1 -- 172.21.15.83:0/2388995691 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89cc021070 con 0x7f89d00cef40 2023-11-26T07:15:56.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.443+0000 7f89d6537700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f89b406b230 0x7f89b406d6f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f89c0005910 tx=0x7f89c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:56.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.447+0000 7f89c6ffd700 1 -- 172.21.15.83:0/2388995691 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89cc056340 con 0x7f89d00cef40 2023-11-26T07:15:56.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.639+0000 7f89d879b700 1 -- 172.21.15.83:0/2388995691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89d0068fb0 con 0x7f89d00cef40 2023-11-26T07:15:56.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.639+0000 7f89c6ffd700 1 -- 172.21.15.83:0/2388995691 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89cc059960 con 0x7f89d00cef40 2023-11-26T07:15:56.642 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:15:56.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.641+0000 7f89d879b700 1 -- 172.21.15.83:0/2388995691 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f89b406b230 msgr2=0x7f89b406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.641+0000 7f89d879b700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f89b406b230 0x7f89b406d6f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f89c0005910 tx=0x7f89c0009040 comp rx=0 tx=0).stop 2023-11-26T07:15:56.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.641+0000 7f89d879b700 1 -- 172.21.15.83:0/2388995691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d00cef40 msgr2=0x7f89d01550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.641+0000 7f89d879b700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d00cef40 0x7f89d01550f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f89cc000c00 tx=0x7f89cc004750 comp rx=0 tx=0).stop 2023-11-26T07:15:56.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.641+0000 7f89d879b700 1 -- 172.21.15.83:0/2388995691 shutdown_connections 2023-11-26T07:15:56.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.641+0000 7f89d879b700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f89b406b230 0x7f89b406d6f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.641+0000 7f89d879b700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89d015bf50 0x7f89d01f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.641+0000 7f89d879b700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89d00c6c30 0x7f89d01572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.641+0000 7f89d879b700 1 --2- 172.21.15.83:0/2388995691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89d00cef40 0x7f89d01550f0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.641+0000 7f89d879b700 1 -- 172.21.15.83:0/2388995691 >> 172.21.15.83:0/2388995691 conn(0x7f89d00b3780 msgr2=0x7f89d006ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.642+0000 7f89d879b700 1 -- 172.21.15.83:0/2388995691 shutdown_connections 2023-11-26T07:15:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.642+0000 7f89d879b700 1 -- 172.21.15.83:0/2388995691 wait complete. 2023-11-26T07:15:56.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:15:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.774+0000 7f68ce520700 1 Processor -- start 2023-11-26T07:15:56.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.774+0000 7f68ce520700 1 -- start start 2023-11-26T07:15:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.775+0000 7f68ce520700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68c80b3370 0x7f68c81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.775+0000 7f68ce520700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68c80d1c70 0x7f68c80d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.775+0000 7f68ce520700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c80b22d0 con 0x7f68c815bdd0 2023-11-26T07:15:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.775+0000 7f68ce520700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c80b2f50 con 0x7f68c80b3370 2023-11-26T07:15:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.775+0000 7f68ce520700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80d2640 con 0x7f68c80d1c70 2023-11-26T07:15:56.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.775+0000 7f68c77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68c80b3370 0x7f68c81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.775+0000 7f68c77fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f68c80b3370 0x7f68c81604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39886/0 (socket says 172.21.15.83:39886) 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.775+0000 7f68c77fe700 1 -- 172.21.15.83:0/3180340274 learned_addr learned my addr 172.21.15.83:0/3180340274 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.776+0000 7f68c77fe700 1 -- 172.21.15.83:0/3180340274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68c80d1c70 msgr2=0x7f68c80d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.776+0000 7f68ccabd700 1 --2- 172.21.15.83:0/3180340274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68c80d1c70 0x7f68c80d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.776+0000 7f68c77fe700 1 --2- 172.21.15.83:0/3180340274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68c80d1c70 0x7f68c80d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.776+0000 7f68c77fe700 1 -- 172.21.15.83:0/3180340274 >> v1:172.21.15.203:6789/0 conn(0x7f68c815bdd0 legacy=0x7f68c80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.776+0000 7f68c77fe700 1 -- 172.21.15.83:0/3180340274 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c80b25b0 con 0x7f68c80b3370 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.776+0000 7f68c77fe700 1 --2- 172.21.15.83:0/3180340274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80b3370 0x7f68c81604e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0098f0 tx=0x7f68bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fdc83310d15fa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.776+0000 7f68c6ffd700 1 -- 172.21.15.83:0/3180340274 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc004230 con 0x7f68c80b3370 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.776+0000 7f68c6ffd700 1 -- 172.21.15.83:0/3180340274 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc004390 con 0x7f68c80b3370 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.776+0000 7f68c6ffd700 1 -- 172.21.15.83:0/3180340274 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0036a0 con 0x7f68c80b3370 2023-11-26T07:15:56.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.777+0000 7f68ce520700 1 -- 172.21.15.83:0/3180340274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80b3370 msgr2=0x7f68c81604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.777+0000 7f68ce520700 1 --2- 172.21.15.83:0/3180340274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80b3370 0x7f68c81604e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0098f0 tx=0x7f68bc009c00 comp rx=0 tx=0).stop 2023-11-26T07:15:56.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.777+0000 7f68ce520700 1 -- 172.21.15.83:0/3180340274 shutdown_connections 2023-11-26T07:15:56.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.777+0000 7f68ce520700 1 --2- 172.21.15.83:0/3180340274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68c80d1c70 0x7f68c80d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.777+0000 7f68ce520700 1 --2- 172.21.15.83:0/3180340274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80b3370 0x7f68c81604e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.777+0000 7f68ce520700 1 -- 172.21.15.83:0/3180340274 >> 172.21.15.83:0/3180340274 conn(0x7f68c80731a0 msgr2=0x7f68c809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:56.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.777+0000 7f68ce520700 1 -- 172.21.15.83:0/3180340274 shutdown_connections 2023-11-26T07:15:56.781 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.777+0000 7f68ce520700 1 -- 172.21.15.83:0/3180340274 wait complete. 2023-11-26T07:15:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.778+0000 7f68ce520700 1 Processor -- start 2023-11-26T07:15:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.778+0000 7f68ce520700 1 -- start start 2023-11-26T07:15:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68ce520700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80b3370 0x7f68c80c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68ce520700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68c80d1c70 0x7f68c8158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.783 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68ce520700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68c815bdd0 0x7f68c81592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68ce520700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8157a10 con 0x7f68c80b3370 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68ce520700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80b48a0 con 0x7f68c815bdd0 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68ce520700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c81f2770 con 0x7f68c80d1c70 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80b3370 0x7f68c80c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80b3370 0x7f68c80c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39890/0 (socket says 172.21.15.83:39890) 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c7fff700 1 -- 172.21.15.83:0/2392193094 learned_addr learned my addr 172.21.15.83:0/2392193094 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c7fff700 1 -- 172.21.15.83:0/2392193094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68c80d1c70 msgr2=0x7f68c8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c77fe700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68c80d1c70 0x7f68c8158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c7fff700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68c80d1c70 0x7f68c8158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68ccabd700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68c815bdd0 0x7f68c81592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c7fff700 1 -- 172.21.15.83:0/2392193094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68c815bdd0 msgr2=0x7f68c81592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c7fff700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68c815bdd0 0x7f68c81592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c7fff700 1 -- 172.21.15.83:0/2392193094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc009550 con 0x7f68c80b3370 2023-11-26T07:15:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c7fff700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80b3370 0x7f68c80c6ba0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f68b800b4a0 tx=0x7f68b800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c4ff9700 1 -- 172.21.15.83:0/2392193094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8011830 con 0x7f68c80b3370 2023-11-26T07:15:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c4ff9700 1 -- 172.21.15.83:0/2392193094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b8011990 con 0x7f68c80b3370 2023-11-26T07:15:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68c4ff9700 1 -- 172.21.15.83:0/2392193094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8011d40 con 0x7f68c80b3370 2023-11-26T07:15:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68ce520700 1 -- 172.21.15.83:0/2392193094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c80b4b40 con 0x7f68c80b3370 2023-11-26T07:15:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.779+0000 7f68ce520700 1 -- 172.21.15.83:0/2392193094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c80b5100 con 0x7f68c80b3370 2023-11-26T07:15:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.780+0000 7f68ce520700 1 -- 172.21.15.83:0/2392193094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c80b5470 con 0x7f68c80b3370 2023-11-26T07:15:56.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.781+0000 7f68c4ff9700 1 -- 172.21.15.83:0/2392193094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f68b801a400 con 0x7f68c80b3370 2023-11-26T07:15:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.781+0000 7f68c4ff9700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68b406b230 0x7f68b406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.781+0000 7f68c77fe700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68b406b230 0x7f68b406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.782+0000 7f68c4ff9700 1 -- 172.21.15.83:0/2392193094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68b8089830 con 0x7f68c80b3370 2023-11-26T07:15:56.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.782+0000 7f68c77fe700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68b406b230 0x7f68b406d6f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f68c809b510 tx=0x7f68bc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:56.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.786+0000 7f68c4ff9700 1 -- 172.21.15.83:0/2392193094 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68b80544d0 con 0x7f68c80b3370 2023-11-26T07:15:56.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.994+0000 7f68ce520700 1 -- 172.21.15.83:0/2392193094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f68c8067f60 con 0x7f68c80b3370 2023-11-26T07:15:56.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.994+0000 7f68c4ff9700 1 -- 172.21.15.83:0/2392193094 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) v1 ==== 80+0+0 (secure 0 0 0) 0x7f68b8057af0 con 0x7f68c80b3370 2023-11-26T07:15:56.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.996+0000 7f68ce520700 1 -- 172.21.15.83:0/2392193094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68b406b230 msgr2=0x7f68b406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:56.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.997+0000 7f68ce520700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68b406b230 0x7f68b406d6f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f68c809b510 tx=0x7f68bc008030 comp rx=0 tx=0).stop 2023-11-26T07:15:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.997+0000 7f68ce520700 1 -- 172.21.15.83:0/2392193094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80b3370 msgr2=0x7f68c80c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.997+0000 7f68ce520700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80b3370 0x7f68c80c6ba0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f68b800b4a0 tx=0x7f68b800b860 comp rx=0 tx=0).stop 2023-11-26T07:15:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.997+0000 7f68ce520700 1 -- 172.21.15.83:0/2392193094 shutdown_connections 2023-11-26T07:15:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.997+0000 7f68ce520700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68c815bdd0 0x7f68c81592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.997+0000 7f68ce520700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c80b3370 0x7f68c80c6ba0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.997+0000 7f68ce520700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f68b406b230 0x7f68b406d6f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.997+0000 7f68ce520700 1 --2- 172.21.15.83:0/2392193094 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68c80d1c70 0x7f68c8158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.997+0000 7f68ce520700 1 -- 172.21.15.83:0/2392193094 >> 172.21.15.83:0/2392193094 conn(0x7f68c80731a0 msgr2=0x7f68c815a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.997+0000 7f68ce520700 1 -- 172.21.15.83:0/2392193094 shutdown_connections 2023-11-26T07:15:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:56.997+0000 7f68ce520700 1 -- 172.21.15.83:0/2392193094 wait complete. 2023-11-26T07:15:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:15:57.011 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:15:57.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:57.011 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:15:57.062 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:15:57.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:15:57.063 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:15:57.081 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:15:57.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.199+0000 7fa3527c1700 1 Processor -- start 2023-11-26T07:15:57.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.199+0000 7fa3527c1700 1 -- start start 2023-11-26T07:15:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa3527c1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 0x7fa34c15e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa3527c1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa34c15f630 0x7fa34c06b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa3527c1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa34c1414c0 con 0x7fa34c0ce140 2023-11-26T07:15:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa3527c1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa34c15c300 con 0x7fa34c15f630 2023-11-26T07:15:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa3527c1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c0ce9f0 con 0x7fa34c15af10 2023-11-26T07:15:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa350d5e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa34c15f630 0x7fa34c06b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa34b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa34c0ce140 0x7fa34c15ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59942/0 (socket says 172.21.15.83:59942) 2023-11-26T07:15:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa34b7fe700 1 -- 172.21.15.83:0/997496087 learned_addr learned my addr 172.21.15.83:0/997496087 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:57.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa350d5e700 1 -- 172.21.15.83:0/997496087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 msgr2=0x7fa34c15e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa34bfff700 1 --2- 172.21.15.83:0/997496087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 0x7fa34c15e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa350d5e700 1 --2- 172.21.15.83:0/997496087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 0x7fa34c15e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa350d5e700 1 -- 172.21.15.83:0/997496087 >> v1:172.21.15.83:6789/0 conn(0x7fa34c0ce140 legacy=0x7fa34c15ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa34affd700 1 -- 172.21.15.83:0/997496087 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036705420 0 0) 0x7fa34c1414c0 con 0x7fa34c0ce140 2023-11-26T07:15:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa350d5e700 1 -- 172.21.15.83:0/997496087 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34c0cecd0 con 0x7fa34c15f630 2023-11-26T07:15:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa34bfff700 1 --2- 172.21.15.83:0/997496087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 0x7fa34c15e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:15:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa350d5e700 1 --2- 172.21.15.83:0/997496087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa34c15f630 0x7fa34c06b640 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa344009d90 tx=0x7fa34400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c94ef6a39927c52 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.200+0000 7fa34affd700 1 -- 172.21.15.83:0/997496087 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34400e070 con 0x7fa34c15f630 2023-11-26T07:15:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.201+0000 7fa34affd700 1 -- 172.21.15.83:0/997496087 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34400bbd0 con 0x7fa34c15f630 2023-11-26T07:15:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.201+0000 7fa34affd700 1 -- 172.21.15.83:0/997496087 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344012430 con 0x7fa34c15f630 2023-11-26T07:15:57.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.201+0000 7fa3527c1700 1 -- 172.21.15.83:0/997496087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa34c15f630 msgr2=0x7fa34c06b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:57.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.201+0000 7fa3527c1700 1 --2- 172.21.15.83:0/997496087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa34c15f630 0x7fa34c06b640 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa344009d90 tx=0x7fa34400b330 comp rx=0 tx=0).stop 2023-11-26T07:15:57.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.201+0000 7fa3527c1700 1 -- 172.21.15.83:0/997496087 shutdown_connections 2023-11-26T07:15:57.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.201+0000 7fa3527c1700 1 --2- 172.21.15.83:0/997496087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa34c15f630 0x7fa34c06b640 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.201+0000 7fa3527c1700 1 --2- 172.21.15.83:0/997496087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 0x7fa34c15e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.201+0000 7fa3527c1700 1 -- 172.21.15.83:0/997496087 >> 172.21.15.83:0/997496087 conn(0x7fa34c0cd570 msgr2=0x7fa34c0cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:57.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.201+0000 7fa3527c1700 1 -- 172.21.15.83:0/997496087 shutdown_connections 2023-11-26T07:15:57.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.201+0000 7fa3527c1700 1 -- 172.21.15.83:0/997496087 wait complete. 2023-11-26T07:15:57.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.202+0000 7fa3527c1700 1 Processor -- start 2023-11-26T07:15:57.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.202+0000 7fa3527c1700 1 -- start start 2023-11-26T07:15:57.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.203+0000 7fa3527c1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa34c0ce140 0x7fa34c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.203+0000 7fa3527c1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 0x7fa34c13ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.203+0000 7fa3527c1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa34c13cda0 0x7fa34c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.203+0000 7fa3527c1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c1590f0 con 0x7fa34c0ce140 2023-11-26T07:15:57.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.203+0000 7fa3527c1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c159510 con 0x7fa34c13cda0 2023-11-26T07:15:57.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.203+0000 7fa3527c1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c141800 con 0x7fa34c15af10 2023-11-26T07:15:57.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.203+0000 7fa34bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa34c0ce140 0x7fa34c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.203+0000 7fa34bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa34c0ce140 0x7fa34c157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39904/0 (socket says 172.21.15.83:39904) 2023-11-26T07:15:57.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.203+0000 7fa34bfff700 1 -- 172.21.15.83:0/1443727411 learned_addr learned my addr 172.21.15.83:0/1443727411 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:57.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.203+0000 7fa34b7fe700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 0x7fa34c13ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.203+0000 7fa350d5e700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa34c13cda0 0x7fa34c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa34bfff700 1 -- 172.21.15.83:0/1443727411 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 msgr2=0x7fa34c13ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa34bfff700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 0x7fa34c13ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa34bfff700 1 -- 172.21.15.83:0/1443727411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa34c13cda0 msgr2=0x7fa34c13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa34bfff700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa34c13cda0 0x7fa34c13f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa34bfff700 1 -- 172.21.15.83:0/1443727411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa344009a70 con 0x7fa34c0ce140 2023-11-26T07:15:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa34b7fe700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 0x7fa34c13ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa34bfff700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa34c0ce140 0x7fa34c157cf0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00ed40 tx=0x7fa33c007380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa348ff9700 1 -- 172.21.15.83:0/1443727411 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c007de0 con 0x7fa34c0ce140 2023-11-26T07:15:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa348ff9700 1 -- 172.21.15.83:0/1443727411 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33c00ee20 con 0x7fa34c0ce140 2023-11-26T07:15:57.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa3527c1700 1 -- 172.21.15.83:0/1443727411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34c158500 con 0x7fa34c0ce140 2023-11-26T07:15:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa348ff9700 1 -- 172.21.15.83:0/1443727411 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c010640 con 0x7fa34c0ce140 2023-11-26T07:15:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.204+0000 7fa3527c1700 1 -- 172.21.15.83:0/1443727411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34c158230 con 0x7fa34c0ce140 2023-11-26T07:15:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.205+0000 7fa3527c1700 1 -- 172.21.15.83:0/1443727411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34c067f60 con 0x7fa34c0ce140 2023-11-26T07:15:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.205+0000 7fa348ff9700 1 -- 172.21.15.83:0/1443727411 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7fa33c0107a0 con 0x7fa34c0ce140 2023-11-26T07:15:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.206+0000 7fa348ff9700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa32c06b230 0x7fa32c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.206+0000 7fa348ff9700 1 -- 172.21.15.83:0/1443727411 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa33c014070 con 0x7fa34c0ce140 2023-11-26T07:15:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.206+0000 7fa34b7fe700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa32c06b230 0x7fa32c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.206+0000 7fa34b7fe700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa32c06b230 0x7fa32c06d6f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa34c1f4d60 tx=0x7fa334007cc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:57.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.210+0000 7fa348ff9700 1 -- 172.21.15.83:0/1443727411 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa33c057010 con 0x7fa34c0ce140 2023-11-26T07:15:57.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:15:57.249 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:15:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.447+0000 7fa3527c1700 1 -- 172.21.15.83:0/1443727411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa34c072e00 con 0x7fa34c0ce140 2023-11-26T07:15:57.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.447+0000 7fa348ff9700 1 -- 172.21.15.83:0/1443727411 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fa33c05a630 con 0x7fa34c0ce140 2023-11-26T07:15:57.450 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:57.450 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5420,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2163151849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2163151849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2163151849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1539108478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1539108478},{"type":"v1","addr":"172.21.15.203:6835","nonce":1539108478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1387613510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1387613510},{"type":"v1","addr":"172.21.15.203:6834","nonce":1387613510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:15:42.339653+0000","modified":"2023-11-26T07:15:45.361821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1809465302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1809465302},{"type":"v1","addr":"172.21.15.83:6836","nonce":1809465302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:15:57.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.449+0000 7fa3527c1700 1 -- 172.21.15.83:0/1443727411 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa32c06b230 msgr2=0x7fa32c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:57.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.449+0000 7fa3527c1700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa32c06b230 0x7fa32c06d6f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa34c1f4d60 tx=0x7fa334007cc0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.450+0000 7fa3527c1700 1 -- 172.21.15.83:0/1443727411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa34c0ce140 msgr2=0x7fa34c157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.450+0000 7fa3527c1700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa34c0ce140 0x7fa34c157cf0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00ed40 tx=0x7fa33c007380 comp rx=0 tx=0).stop 2023-11-26T07:15:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.450+0000 7fa3527c1700 1 -- 172.21.15.83:0/1443727411 shutdown_connections 2023-11-26T07:15:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.450+0000 7fa3527c1700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa34c13cda0 0x7fa34c13f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.450+0000 7fa3527c1700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa34c0ce140 0x7fa34c157cf0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.450+0000 7fa3527c1700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fa32c06b230 0x7fa32c06d6f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.450+0000 7fa3527c1700 1 --2- 172.21.15.83:0/1443727411 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa34c15af10 0x7fa34c13ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.450+0000 7fa3527c1700 1 -- 172.21.15.83:0/1443727411 >> 172.21.15.83:0/1443727411 conn(0x7fa34c0cd570 msgr2=0x7fa34c15b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.450+0000 7fa3527c1700 1 -- 172.21.15.83:0/1443727411 shutdown_connections 2023-11-26T07:15:57.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.450+0000 7fa3527c1700 1 -- 172.21.15.83:0/1443727411 wait complete. 2023-11-26T07:15:57.454 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 47 2023-11-26T07:15:57.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:15:57.606 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2023-11-26T07:15:57.606 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-11-26T07:15:57.606 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol19700 2023-11-26T07:15:57.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.727+0000 7f02b990a700 1 Processor -- start 2023-11-26T07:15:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.727+0000 7f02b990a700 1 -- start start 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b990a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02b415e140 0x7f02b40d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b990a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f02b40b98c0 0x7f02b40d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b990a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b40b80e0 con 0x7f02b40d39d0 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b990a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b415e520 con 0x7f02b40b98c0 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b990a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b40d2930 con 0x7f02b415e140 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b37fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f02b40b98c0 0x7f02b40d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b27fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f02b40d39d0 0x7f02b40b59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:59952/0 (socket says 172.21.15.83:59952) 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b27fc700 1 -- 172.21.15.83:0/4083498510 learned_addr learned my addr 172.21.15.83:0/4083498510 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b2ffd700 1 --2- 172.21.15.83:0/4083498510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02b415e140 0x7f02b40d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b37fe700 1 -- 172.21.15.83:0/4083498510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02b415e140 msgr2=0x7f02b40d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b37fe700 1 --2- 172.21.15.83:0/4083498510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02b415e140 0x7f02b40d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b37fe700 1 -- 172.21.15.83:0/4083498510 >> v1:172.21.15.83:6789/0 conn(0x7f02b40d39d0 legacy=0x7f02b40b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b37fe700 1 -- 172.21.15.83:0/4083498510 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b40d2c10 con 0x7f02b40b98c0 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.728+0000 7f02b37fe700 1 --2- 172.21.15.83:0/4083498510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02b40b98c0 0x7f02b40d4400 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009d90 tx=0x7f02ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7682b70725733ed4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.729+0000 7f02b1ffb700 1 -- 172.21.15.83:0/4083498510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00e070 con 0x7f02b40b98c0 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.729+0000 7f02b1ffb700 1 -- 172.21.15.83:0/4083498510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ac00bbd0 con 0x7f02b40b98c0 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.729+0000 7f02b1ffb700 1 -- 172.21.15.83:0/4083498510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac012430 con 0x7f02b40b98c0 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.729+0000 7f02b990a700 1 -- 172.21.15.83:0/4083498510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02b40b98c0 msgr2=0x7f02b40d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.729+0000 7f02b990a700 1 --2- 172.21.15.83:0/4083498510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02b40b98c0 0x7f02b40d4400 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009d90 tx=0x7f02ac00b330 comp rx=0 tx=0).stop 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.729+0000 7f02b990a700 1 -- 172.21.15.83:0/4083498510 shutdown_connections 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.729+0000 7f02b990a700 1 --2- 172.21.15.83:0/4083498510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02b40b98c0 0x7f02b40d4400 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.729+0000 7f02b990a700 1 --2- 172.21.15.83:0/4083498510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02b415e140 0x7f02b40d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.729+0000 7f02b990a700 1 -- 172.21.15.83:0/4083498510 >> 172.21.15.83:0/4083498510 conn(0x7f02b406b410 msgr2=0x7f02b406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.729+0000 7f02b990a700 1 -- 172.21.15.83:0/4083498510 shutdown_connections 2023-11-26T07:15:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.729+0000 7f02b990a700 1 -- 172.21.15.83:0/4083498510 wait complete. 2023-11-26T07:15:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.731+0000 7f02b990a700 1 Processor -- start 2023-11-26T07:15:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.731+0000 7f02b990a700 1 -- start start 2023-11-26T07:15:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b990a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02b40b98c0 0x7f02b40b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b990a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02b40d39d0 0x7f02b41f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b990a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02b415e140 0x7f02b41f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b990a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b40c8810 con 0x7f02b415e140 2023-11-26T07:15:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b990a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b40b47e0 con 0x7f02b40d39d0 2023-11-26T07:15:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b990a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b41f46a0 con 0x7f02b40b98c0 2023-11-26T07:15:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02b415e140 0x7f02b41f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02b415e140 0x7f02b41f4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39922/0 (socket says 172.21.15.83:39922) 2023-11-26T07:15:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b37fe700 1 -- 172.21.15.83:0/2763050250 learned_addr learned my addr 172.21.15.83:0/2763050250 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b2ffd700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02b40b98c0 0x7f02b40b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b27fc700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02b40d39d0 0x7f02b41f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b37fe700 1 -- 172.21.15.83:0/2763050250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02b40b98c0 msgr2=0x7f02b40b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b37fe700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02b40b98c0 0x7f02b40b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b37fe700 1 -- 172.21.15.83:0/2763050250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02b40d39d0 msgr2=0x7f02b41f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b37fe700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02b40d39d0 0x7f02b41f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b37fe700 1 -- 172.21.15.83:0/2763050250 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac009a70 con 0x7f02b415e140 2023-11-26T07:15:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b37fe700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02b415e140 0x7f02b41f4890 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f02ac013000 tx=0x7f02ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f029bfff700 1 -- 172.21.15.83:0/2763050250 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac003ea0 con 0x7f02b415e140 2023-11-26T07:15:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f029bfff700 1 -- 172.21.15.83:0/2763050250 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ac020070 con 0x7f02b415e140 2023-11-26T07:15:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b990a700 1 -- 172.21.15.83:0/2763050250 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b40c78b0 con 0x7f02b415e140 2023-11-26T07:15:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f029bfff700 1 -- 172.21.15.83:0/2763050250 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac004280 con 0x7f02b415e140 2023-11-26T07:15:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.732+0000 7f02b990a700 1 -- 172.21.15.83:0/2763050250 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b40c7e70 con 0x7f02b415e140 2023-11-26T07:15:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.733+0000 7f02b990a700 1 -- 172.21.15.83:0/2763050250 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b40c8a80 con 0x7f02b415e140 2023-11-26T07:15:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.734+0000 7f029bfff700 1 -- 172.21.15.83:0/2763050250 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f02ac0043e0 con 0x7f02b415e140 2023-11-26T07:15:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.734+0000 7f029bfff700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f02a006b230 0x7f02a006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.734+0000 7f029bfff700 1 -- 172.21.15.83:0/2763050250 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02ac022070 con 0x7f02b415e140 2023-11-26T07:15:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.734+0000 7f02b2ffd700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f02a006b230 0x7f02a006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.735+0000 7f02b2ffd700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f02a006b230 0x7f02a006d6f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b9b0 tx=0x7f02a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.739+0000 7f029bfff700 1 -- 172.21.15.83:0/2763050250 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02ac0569c0 con 0x7f02b415e140 2023-11-26T07:15:57.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.962+0000 7f02b990a700 1 -- 172.21.15.83:0/2763050250 --> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol19700", "target": ["mon-mgr", ""]}) v1 -- 0x7f02b40d34b0 con 0x7f02a006b230 2023-11-26T07:15:57.970 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:57.967+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:57.970 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:57.967+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:57.970 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:57.967+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:57.970 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:57.967+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:57.970 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:15:57.967+0000 7f377eb66700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:15:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.997+0000 7f029bfff700 1 -- 172.21.15.83:0/2763050250 <== mgr.4110 v2:172.21.15.83:6832/1483230064 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f02a800be80 con 0x7f02a006b230 2023-11-26T07:15:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 -- 172.21.15.83:0/2763050250 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f02a006b230 msgr2=0x7f02a006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f02a006b230 0x7f02a006d6f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b9b0 tx=0x7f02a8008040 comp rx=0 tx=0).stop 2023-11-26T07:15:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 -- 172.21.15.83:0/2763050250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02b415e140 msgr2=0x7f02b41f4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02b415e140 0x7f02b41f4890 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f02ac013000 tx=0x7f02ac004ad0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 -- 172.21.15.83:0/2763050250 shutdown_connections 2023-11-26T07:15:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02b415e140 0x7f02b41f4890 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f02a006b230 0x7f02a006d6f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02b40b98c0 0x7f02b40b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 --2- 172.21.15.83:0/2763050250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02b40d39d0 0x7f02b41f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 -- 172.21.15.83:0/2763050250 >> 172.21.15.83:0/2763050250 conn(0x7f02b406b410 msgr2=0x7f02b40b70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 -- 172.21.15.83:0/2763050250 shutdown_connections 2023-11-26T07:15:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:57.999+0000 7f02b990a700 1 -- 172.21.15.83:0/2763050250 wait complete. 2023-11-26T07:15:58.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:15:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.131+0000 7f03e0aa4700 1 Processor -- start 2023-11-26T07:15:58.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.132+0000 7f03e0aa4700 1 -- start start 2023-11-26T07:15:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.132+0000 7f03e0aa4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03dc15e080 0x7f03dc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.132+0000 7f03e0aa4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03dc0af4d0 0x7f03dc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.132+0000 7f03e0aa4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03dc0ae430 con 0x7f03dc06c000 2023-11-26T07:15:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.132+0000 7f03e0aa4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03dc0af0b0 con 0x7f03dc0af4d0 2023-11-26T07:15:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.132+0000 7f03e0aa4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0b05a0 con 0x7f03dc15e080 2023-11-26T07:15:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.133+0000 7f03d9d9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03dc0af4d0 0x7f03dc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.133+0000 7f03da59c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03dc15e080 0x7f03dc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.133+0000 7f03d9d9b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03dc0af4d0 0x7f03dc0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:39394/0 (socket says 172.21.15.83:39394) 2023-11-26T07:15:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.133+0000 7f03d9d9b700 1 -- 172.21.15.83:0/2785781424 learned_addr learned my addr 172.21.15.83:0/2785781424 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:58.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.133+0000 7f03da59c700 1 -- 172.21.15.83:0/2785781424 >> v2:172.21.15.203:3300/0 conn(0x7f03dc0af4d0 msgr2=0x7f03dc0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:58.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.133+0000 7f03da59c700 1 --2- 172.21.15.83:0/2785781424 >> v2:172.21.15.203:3300/0 conn(0x7f03dc0af4d0 0x7f03dc0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.133+0000 7f03da59c700 1 -- 172.21.15.83:0/2785781424 >> v1:172.21.15.203:6789/0 conn(0x7f03dc06c000 legacy=0x7f03dc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.133+0000 7f03da59c700 1 -- 172.21.15.83:0/2785781424 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc0ae710 con 0x7f03dc15e080 2023-11-26T07:15:58.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.133+0000 7f03da59c700 1 --2- 172.21.15.83:0/2785781424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03dc15e080 0x7f03dc0b5050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009840 tx=0x7f03d0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ff5cc10047ad74d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:58.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.134+0000 7f03d959a700 1 -- 172.21.15.83:0/2785781424 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0011830 con 0x7f03dc15e080 2023-11-26T07:15:58.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.134+0000 7f03d959a700 1 -- 172.21.15.83:0/2785781424 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d0011990 con 0x7f03dc15e080 2023-11-26T07:15:58.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.134+0000 7f03d959a700 1 -- 172.21.15.83:0/2785781424 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0011d20 con 0x7f03dc15e080 2023-11-26T07:15:58.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.134+0000 7f03e0aa4700 1 -- 172.21.15.83:0/2785781424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03dc15e080 msgr2=0x7f03dc0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.134+0000 7f03e0aa4700 1 --2- 172.21.15.83:0/2785781424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03dc15e080 0x7f03dc0b5050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009840 tx=0x7f03d0009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:58.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.134+0000 7f03e0aa4700 1 -- 172.21.15.83:0/2785781424 shutdown_connections 2023-11-26T07:15:58.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.134+0000 7f03e0aa4700 1 --2- 172.21.15.83:0/2785781424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03dc15e080 0x7f03dc0b5050 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.134+0000 7f03e0aa4700 1 --2- 172.21.15.83:0/2785781424 >> v2:172.21.15.203:3300/0 conn(0x7f03dc0af4d0 0x7f03dc0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.134+0000 7f03e0aa4700 1 -- 172.21.15.83:0/2785781424 >> 172.21.15.83:0/2785781424 conn(0x7f03dc071b90 msgr2=0x7f03dc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:58.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.134+0000 7f03e0aa4700 1 -- 172.21.15.83:0/2785781424 shutdown_connections 2023-11-26T07:15:58.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.134+0000 7f03e0aa4700 1 -- 172.21.15.83:0/2785781424 wait complete. 2023-11-26T07:15:58.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.136+0000 7f03e0aa4700 1 Processor -- start 2023-11-26T07:15:58.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.136+0000 7f03e0aa4700 1 -- start start 2023-11-26T07:15:58.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03e0aa4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03dc06c000 0x7f03dc158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:58.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03e0aa4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc0af4d0 0x7f03dc15c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:58.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03e0aa4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03dc15e080 0x7f03dc149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:58.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03e0aa4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc1575a0 con 0x7f03dc0af4d0 2023-11-26T07:15:58.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03e0aa4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc157880 con 0x7f03dc06c000 2023-11-26T07:15:58.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03e0aa4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc157b60 con 0x7f03dc15e080 2023-11-26T07:15:58.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03d9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc0af4d0 0x7f03dc15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03d9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc0af4d0 0x7f03dc15c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39948/0 (socket says 172.21.15.83:39948) 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03d9d9b700 1 -- 172.21.15.83:0/4155913064 learned_addr learned my addr 172.21.15.83:0/4155913064 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03dad9d700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03dc15e080 0x7f03dc149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03da59c700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03dc06c000 0x7f03dc158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03d9d9b700 1 -- 172.21.15.83:0/4155913064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03dc15e080 msgr2=0x7f03dc149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03d9d9b700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03dc15e080 0x7f03dc149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03d9d9b700 1 -- 172.21.15.83:0/4155913064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03dc06c000 msgr2=0x7f03dc158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03d9d9b700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03dc06c000 0x7f03dc158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.137+0000 7f03d9d9b700 1 -- 172.21.15.83:0/4155913064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d00094a0 con 0x7f03dc0af4d0 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.138+0000 7f03d9d9b700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc0af4d0 0x7f03dc15c160 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e8e0 tx=0x7f03cc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.138+0000 7f03caffd700 1 -- 172.21.15.83:0/4155913064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00eee0 con 0x7f03dc0af4d0 2023-11-26T07:15:58.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.138+0000 7f03caffd700 1 -- 172.21.15.83:0/4155913064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03cc00c850 con 0x7f03dc0af4d0 2023-11-26T07:15:58.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.138+0000 7f03e0aa4700 1 -- 172.21.15.83:0/4155913064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc157e40 con 0x7f03dc0af4d0 2023-11-26T07:15:58.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.138+0000 7f03caffd700 1 -- 172.21.15.83:0/4155913064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00cbc0 con 0x7f03dc0af4d0 2023-11-26T07:15:58.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.138+0000 7f03e0aa4700 1 -- 172.21.15.83:0/4155913064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc15c6a0 con 0x7f03dc0af4d0 2023-11-26T07:15:58.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.139+0000 7f03e0aa4700 1 -- 172.21.15.83:0/4155913064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03dc0753f0 con 0x7f03dc0af4d0 2023-11-26T07:15:58.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.139+0000 7f03caffd700 1 -- 172.21.15.83:0/4155913064 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 89537+0+0 (secure 0 0 0) 0x7f03cc00cd20 con 0x7f03dc0af4d0 2023-11-26T07:15:58.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.140+0000 7f03caffd700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03bc06b230 0x7f03bc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:58.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.140+0000 7f03da59c700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03bc06b230 0x7f03bc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:58.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.140+0000 7f03caffd700 1 -- 172.21.15.83:0/4155913064 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03cc089ac0 con 0x7f03dc0af4d0 2023-11-26T07:15:58.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.140+0000 7f03da59c700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03bc06b230 0x7f03bc06d6f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009fd0 tx=0x7f03d001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:58.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.144+0000 7f03caffd700 1 -- 172.21.15.83:0/4155913064 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03cc08be60 con 0x7f03dc0af4d0 2023-11-26T07:15:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.381+0000 7f03e0aa4700 1 -- 172.21.15.83:0/4155913064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03dc067f60 con 0x7f03dc0af4d0 2023-11-26T07:15:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.381+0000 7f03caffd700 1 -- 172.21.15.83:0/4155913064 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f03cc021090 con 0x7f03dc0af4d0 2023-11-26T07:15:58.384 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:58.384 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5420,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2163151849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2163151849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2163151849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1539108478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1539108478},{"type":"v1","addr":"172.21.15.203:6835","nonce":1539108478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1387613510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1387613510},{"type":"v1","addr":"172.21.15.203:6834","nonce":1387613510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:15:42.339653+0000","modified":"2023-11-26T07:15:45.361821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1809465302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1809465302},{"type":"v1","addr":"172.21.15.83:6836","nonce":1809465302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:15:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 -- 172.21.15.83:0/4155913064 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03bc06b230 msgr2=0x7f03bc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03bc06b230 0x7f03bc06d6f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009fd0 tx=0x7f03d001c040 comp rx=0 tx=0).stop 2023-11-26T07:15:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 -- 172.21.15.83:0/4155913064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc0af4d0 msgr2=0x7f03dc15c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc0af4d0 0x7f03dc15c160 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00e8e0 tx=0x7f03cc00eca0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 -- 172.21.15.83:0/4155913064 shutdown_connections 2023-11-26T07:15:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03bc06b230 0x7f03bc06d6f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03dc15e080 0x7f03dc149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03dc06c000 0x7f03dc158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 --2- 172.21.15.83:0/4155913064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc0af4d0 0x7f03dc15c160 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 -- 172.21.15.83:0/4155913064 >> 172.21.15.83:0/4155913064 conn(0x7f03dc071b90 msgr2=0x7f03dc06f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 -- 172.21.15.83:0/4155913064 shutdown_connections 2023-11-26T07:15:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.383+0000 7f03e0aa4700 1 -- 172.21.15.83:0/4155913064 wait complete. 2023-11-26T07:15:58.387 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 47 2023-11-26T07:15:58.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:15:58.538 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":6885,"entity":{"name":{"type":"client","num":6885},"addr":{"type":"any","addr":"172.21.15.83:0","nonce":2325877059}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.55598676800000002,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.972299161479619,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6885 172.21.15.83:0/2325877059","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi083","pid":"103268","root":"/"}}] 2023-11-26T07:15:58.538 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi083", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103268", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.83:0", "nonce": 2325877059, "type": "any" }, "name": { "num": 6885, "type": "client" } }, "id": 6885, "inst": "client.6885 172.21.15.83:0/2325877059", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.97229916147962 }, "state": "open", "uptime": 0.555986768 } ] 2023-11-26T07:15:58.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-26T07:15:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.660+0000 7f0400ae4700 1 Processor -- start 2023-11-26T07:15:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.660+0000 7f0400ae4700 1 -- start start 2023-11-26T07:15:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.661+0000 7f0400ae4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f03fc0adf00 0x7f03fc06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.661+0000 7f0400ae4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc0af950 con 0x7f03fc06c540 2023-11-26T07:15:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.661+0000 7f0400ae4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03fc0afc30 con 0x7f03fc1446a0 2023-11-26T07:15:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.661+0000 7f0400ae4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03fc06c9f0 con 0x7f03fc0adf00 2023-11-26T07:15:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.661+0000 7f03f9d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f03fc0adf00 0x7f03fc06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.661+0000 7f03fad9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f03fc06c540 0x7f03fc0b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60000/0 (socket says 172.21.15.83:60000) 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.661+0000 7f03fad9d700 1 -- 172.21.15.83:0/161733677 learned_addr learned my addr 172.21.15.83:0/161733677 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.662+0000 7f03f9d9b700 1 -- 172.21.15.83:0/161733677 >> v1:172.21.15.203:6789/0 conn(0x7f03fc1446a0 legacy=0x7f03fc0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.662+0000 7f03f959a700 1 -- 172.21.15.83:0/161733677 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376050478 0 0) 0x7f03fc0af950 con 0x7f03fc06c540 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.662+0000 7f03f9d9b700 1 -- 172.21.15.83:0/161733677 >> v1:172.21.15.83:6789/0 conn(0x7f03fc06c540 legacy=0x7f03fc0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.662+0000 7f03f9d9b700 1 -- 172.21.15.83:0/161733677 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc0aff10 con 0x7f03fc0adf00 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.662+0000 7f03f9d9b700 1 --2- 172.21.15.83:0/161733677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03fc0adf00 0x7f03fc06c000 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009840 tx=0x7f03f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d89b2df00583e99 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.662+0000 7f03f959a700 1 -- 172.21.15.83:0/161733677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0011830 con 0x7f03fc0adf00 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.662+0000 7f03f959a700 1 -- 172.21.15.83:0/161733677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f0011990 con 0x7f03fc0adf00 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.662+0000 7f03f959a700 1 -- 172.21.15.83:0/161733677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0011d20 con 0x7f03fc0adf00 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.662+0000 7f0400ae4700 1 -- 172.21.15.83:0/161733677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03fc0adf00 msgr2=0x7f03fc06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.662+0000 7f0400ae4700 1 --2- 172.21.15.83:0/161733677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03fc0adf00 0x7f03fc06c000 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009840 tx=0x7f03f0009b50 comp rx=0 tx=0).stop 2023-11-26T07:15:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.663+0000 7f0400ae4700 1 -- 172.21.15.83:0/161733677 shutdown_connections 2023-11-26T07:15:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.663+0000 7f0400ae4700 1 --2- 172.21.15.83:0/161733677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03fc0adf00 0x7f03fc06c000 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.663+0000 7f0400ae4700 1 -- 172.21.15.83:0/161733677 >> 172.21.15.83:0/161733677 conn(0x7f03fc071b90 msgr2=0x7f03fc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.663+0000 7f0400ae4700 1 -- 172.21.15.83:0/161733677 shutdown_connections 2023-11-26T07:15:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.663+0000 7f0400ae4700 1 -- 172.21.15.83:0/161733677 wait complete. 2023-11-26T07:15:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.664+0000 7f0400ae4700 1 Processor -- start 2023-11-26T07:15:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.664+0000 7f0400ae4700 1 -- start start 2023-11-26T07:15:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f0400ae4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03fc06c540 0x7f03fc140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f0400ae4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03fc0adf00 0x7f03fc15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f0400ae4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03fc1446a0 0x7f03fc13f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f0400ae4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc141a70 con 0x7f03fc06c540 2023-11-26T07:15:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f0400ae4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc15a940 con 0x7f03fc0adf00 2023-11-26T07:15:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f0400ae4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1414b0 con 0x7f03fc1446a0 2023-11-26T07:15:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03fa59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03fc06c540 0x7f03fc140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03fa59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03fc06c540 0x7f03fc140e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39962/0 (socket says 172.21.15.83:39962) 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03fa59c700 1 -- 172.21.15.83:0/684885235 learned_addr learned my addr 172.21.15.83:0/684885235 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03f9d9b700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03fc0adf00 0x7f03fc15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03fad9d700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03fc1446a0 0x7f03fc13f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03fa59c700 1 -- 172.21.15.83:0/684885235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03fc1446a0 msgr2=0x7f03fc13f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03fa59c700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03fc1446a0 0x7f03fc13f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03fa59c700 1 -- 172.21.15.83:0/684885235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03fc0adf00 msgr2=0x7f03fc15a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03fa59c700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03fc0adf00 0x7f03fc15a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03fa59c700 1 -- 172.21.15.83:0/684885235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f00094a0 con 0x7f03fc06c540 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03fad9d700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03fc1446a0 0x7f03fc13f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.665+0000 7f03fa59c700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03fc06c540 0x7f03fc140e00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c430 tx=0x7f03ec00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.666+0000 7f03eaffd700 1 -- 172.21.15.83:0/684885235 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0052d0 con 0x7f03fc06c540 2023-11-26T07:15:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.666+0000 7f03eaffd700 1 -- 172.21.15.83:0/684885235 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ec005450 con 0x7f03fc06c540 2023-11-26T07:15:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.666+0000 7f0400ae4700 1 -- 172.21.15.83:0/684885235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc141790 con 0x7f03fc06c540 2023-11-26T07:15:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.666+0000 7f03eaffd700 1 -- 172.21.15.83:0/684885235 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec006aa0 con 0x7f03fc06c540 2023-11-26T07:15:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.666+0000 7f0400ae4700 1 -- 172.21.15.83:0/684885235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc0cda80 con 0x7f03fc06c540 2023-11-26T07:15:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.667+0000 7f0400ae4700 1 -- 172.21.15.83:0/684885235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc0cdc40 con 0x7f03fc06c540 2023-11-26T07:15:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.667+0000 7f03eaffd700 1 -- 172.21.15.83:0/684885235 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f03ec003680 con 0x7f03fc06c540 2023-11-26T07:15:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.668+0000 7f03eaffd700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03dc06b2c0 0x7f03dc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:15:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.668+0000 7f03f9d9b700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03dc06b2c0 0x7f03dc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:15:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.668+0000 7f03eaffd700 1 -- 172.21.15.83:0/684885235 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03ec088e00 con 0x7f03fc06c540 2023-11-26T07:15:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.668+0000 7f03f9d9b700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03dc06b2c0 0x7f03dc06d780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009840 tx=0x7f03f0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:15:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.672+0000 7f03eaffd700 1 -- 172.21.15.83:0/684885235 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03ec08d070 con 0x7f03fc06c540 2023-11-26T07:15:58.922 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.920+0000 7f0400ae4700 1 -- 172.21.15.83:0/684885235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f03fc0753f0 con 0x7f03fc06c540 2023-11-26T07:15:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.925+0000 7f03eaffd700 1 -- 172.21.15.83:0/684885235 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v12) v1 ==== 81+0+467875 (secure 0 0 0) 0x7f03ec053920 con 0x7f03fc06c540 2023-11-26T07:15:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:15:58.934 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:15:58.934 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 12, 2023-11-26T07:15:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "active_gid": 4110, 2023-11-26T07:15:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "active_name": "y", 2023-11-26T07:15:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addrs": { 2023-11-26T07:15:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6832", 2023-11-26T07:15:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1483230064 2023-11-26T07:15:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2023-11-26T07:15:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6833", 2023-11-26T07:15:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1483230064 2023-11-26T07:15:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addr": "172.21.15.83:6833/1483230064", 2023-11-26T07:15:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "active_change": "2023-11-26T07:11:08.299320+0000", 2023-11-26T07:15:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "active_mgr_features": 4540138322906710015, 2023-11-26T07:15:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "available": true, 2023-11-26T07:15:58.936 INFO:teuthology.orchestra.run.smithi083.stdout: "standbys": [ 2023-11-26T07:15:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 4105, 2023-11-26T07:15:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "x", 2023-11-26T07:15:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540138322906710015, 2023-11-26T07:15:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2023-11-26T07:15:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2023-11-26T07:15:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:58.937 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.938 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:58.939 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:58.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.941 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:58.942 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.943 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2023-11-26T07:15:58.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2023-11-26T07:15:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2023-11-26T07:15:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2023-11-26T07:15:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2023-11-26T07:15:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.947 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2023-11-26T07:15:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2023-11-26T07:15:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2023-11-26T07:15:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.948 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2023-11-26T07:15:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.949 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.951 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-26T07:15:58.952 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2023-11-26T07:15:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2023-11-26T07:15:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.953 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2023-11-26T07:15:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2023-11-26T07:15:58.954 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2023-11-26T07:15:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2023-11-26T07:15:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.955 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2023-11-26T07:15:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2023-11-26T07:15:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2023-11-26T07:15:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2023-11-26T07:15:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2023-11-26T07:15:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:15:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.957 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.958 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-26T07:15:58.959 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2023-11-26T07:15:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2023-11-26T07:15:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2023-11-26T07:15:58.960 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.961 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2023-11-26T07:15:58.963 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:58.964 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:58.965 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:58.967 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:58.968 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.969 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:58.970 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2023-11-26T07:15:58.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2023-11-26T07:15:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2023-11-26T07:15:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2023-11-26T07:15:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap" 2023-11-26T07:15:58.972 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2023-11-26T07:15:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2023-11-26T07:15:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2023-11-26T07:15:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.973 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-26T07:15:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2023-11-26T07:15:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2023-11-26T07:15:58.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2023-11-26T07:15:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-26T07:15:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2023-11-26T07:15:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2023-11-26T07:15:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2023-11-26T07:15:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2023-11-26T07:15:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-26T07:15:58.976 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.977 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:58.978 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.980 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.981 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2023-11-26T07:15:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2023-11-26T07:15:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.982 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.983 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.984 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:58.985 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.986 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.988 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2023-11-26T07:15:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2023-11-26T07:15:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.989 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-26T07:15:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2023-11-26T07:15:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2023-11-26T07:15:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.990 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2023-11-26T07:15:58.992 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2023-11-26T07:15:58.993 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2023-11-26T07:15:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-26T07:15:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2023-11-26T07:15:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.995 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2023-11-26T07:15:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2023-11-26T07:15:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2023-11-26T07:15:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-26T07:15:58.996 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.997 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:58.998 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2023-11-26T07:15:58.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.000 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.002 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.003 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.004 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.005 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-26T07:15:59.006 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.007 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.008 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.009 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2023-11-26T07:15:59.010 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2023-11-26T07:15:59.011 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2023-11-26T07:15:59.012 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.013 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.014 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2023-11-26T07:15:59.015 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.016 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.017 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.018 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.020 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.021 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.022 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-26T07:15:59.023 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.025 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2023-11-26T07:15:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2023-11-26T07:15:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.026 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.027 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.028 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2023-11-26T07:15:59.029 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.030 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.031 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.032 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.033 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.034 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2023-11-26T07:15:59.035 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.036 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2023-11-26T07:15:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2023-11-26T07:15:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-26T07:15:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.037 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2023-11-26T07:15:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2023-11-26T07:15:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2023-11-26T07:15:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.038 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.039 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.040 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.041 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.042 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.042 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.042 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.042 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-26T07:15:59.042 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.042 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.042 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.042 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2023-11-26T07:15:59.042 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2023-11-26T07:15:59.042 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2023-11-26T07:15:59.043 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2023-11-26T07:15:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-26T07:15:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.046 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.047 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.048 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.049 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.051 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-26T07:15:59.052 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.054 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.055 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.057 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.058 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.059 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.060 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.061 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.062 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.063 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-26T07:15:59.064 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.066 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-26T07:15:59.066 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-26T07:15:59.066 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.066 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.066 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.066 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.066 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.066 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.066 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.066 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.066 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.067 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.068 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.069 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2023-11-26T07:15:59.070 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.071 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-26T07:15:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-26T07:15:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.072 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.073 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.074 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-26T07:15:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-26T07:15:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.075 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-26T07:15:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-26T07:15:59.076 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.077 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.079 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.080 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.081 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.082 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.083 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.084 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-26T07:15:59.086 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-26T07:15:59.087 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.089 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.090 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2023-11-26T07:15:59.090 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-26T07:15:59.090 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.090 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.090 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.090 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.090 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.090 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.090 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.090 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.092 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.093 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.094 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.095 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.096 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.096 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.096 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.096 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.096 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.096 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.096 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.096 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.096 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.096 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-26T07:15:59.097 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-26T07:15:59.097 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.097 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.097 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.097 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2023-11-26T07:15:59.097 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.097 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.097 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.097 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.097 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.098 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.098 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2023-11-26T07:15:59.098 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-26T07:15:59.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.098 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.098 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.098 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.098 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.099 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.099 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.099 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.099 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.099 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.099 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2023-11-26T07:15:59.099 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-26T07:15:59.099 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.099 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.099 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.100 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-26T07:15:59.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-26T07:15:59.102 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.103 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.104 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-26T07:15:59.104 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-26T07:15:59.104 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.104 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.104 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.104 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2023-11-26T07:15:59.104 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.104 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.104 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.105 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.106 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.107 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2023-11-26T07:15:59.108 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.110 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.111 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.113 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.114 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.116 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.117 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.118 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.119 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.120 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2023-11-26T07:15:59.121 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.122 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2023-11-26T07:15:59.123 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2023-11-26T07:15:59.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.123 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.123 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2023-11-26T07:15:59.123 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.123 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.123 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.123 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.123 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2023-11-26T07:15:59.123 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2023-11-26T07:15:59.124 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2023-11-26T07:15:59.125 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.125 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.125 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.125 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.125 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.125 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2023-11-26T07:15:59.125 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2023-11-26T07:15:59.125 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.125 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.125 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.126 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.126 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.126 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.126 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.126 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.126 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.126 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.126 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.126 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.126 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.127 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.128 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.129 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.130 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.131 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.132 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.134 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.135 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.136 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2023-11-26T07:15:59.137 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.138 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.139 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.140 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2023-11-26T07:15:59.141 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.142 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.143 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.144 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.145 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.146 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.147 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.148 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2023-11-26T07:15:59.149 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.150 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.151 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.152 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.153 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2023-11-26T07:15:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.155 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.156 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.158 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.159 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2023-11-26T07:15:59.160 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2023-11-26T07:15:59.161 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.162 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.165 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:15:59.166 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.167 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.168 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.169 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.170 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.171 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.172 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.173 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.174 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.175 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.176 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.177 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.178 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.179 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.180 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.181 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.183 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.184 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.185 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.186 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.187 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.188 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.189 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.190 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2023-11-26T07:15:59.190 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.190 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.190 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.190 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-26T07:15:59.190 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-26T07:15:59.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.190 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.190 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.191 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.192 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.193 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.194 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.195 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.196 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.197 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.198 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.199 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.200 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.201 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.202 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.203 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.204 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.206 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.209 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.210 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.211 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.212 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.213 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.214 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.216 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.217 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.219 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.220 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.221 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.222 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.223 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.224 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.226 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.227 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.229 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.230 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.230 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.230 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.230 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.230 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.230 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.230 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.230 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.230 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.230 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.230 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.231 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.232 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.233 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2023-11-26T07:15:59.234 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2023-11-26T07:15:59.235 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.235 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.235 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.235 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2023-11-26T07:15:59.235 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.235 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.235 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.235 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.235 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.235 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2023-11-26T07:15:59.236 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2023-11-26T07:15:59.237 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.238 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.239 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.240 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.241 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.242 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.243 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.244 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.245 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.246 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2023-11-26T07:15:59.247 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.248 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.249 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.250 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.252 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.253 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.254 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.255 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.256 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.257 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.258 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.260 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.261 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2023-11-26T07:15:59.262 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.262 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.262 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.262 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.262 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.262 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.262 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2023-11-26T07:15:59.262 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2023-11-26T07:15:59.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.263 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.263 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2023-11-26T07:15:59.263 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.263 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.263 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.263 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2023-11-26T07:15:59.263 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2023-11-26T07:15:59.263 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.263 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.263 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.264 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.265 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.266 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.267 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.268 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.269 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.273 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.274 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.274 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.274 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2023-11-26T07:15:59.274 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.274 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.274 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.274 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2023-11-26T07:15:59.274 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2023-11-26T07:15:59.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2023-11-26T07:15:59.275 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.276 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.277 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.278 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.278 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.278 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.278 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.278 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.278 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.278 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.278 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.278 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.279 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.280 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.281 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.282 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.283 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.284 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.284 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.284 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.284 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.284 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.284 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.284 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.284 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2023-11-26T07:15:59.284 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.285 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.286 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.286 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.286 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.286 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.286 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.286 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.286 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.286 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.287 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.288 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.289 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.289 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.289 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.289 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.289 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.289 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.289 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.289 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.289 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: "drive_group_interval": { 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "drive_group_interval", 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.291 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300.0", 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.292 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.293 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.294 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.295 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.297 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.297 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.297 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.297 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.297 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.297 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.297 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.297 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.298 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.301 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.302 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.304 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.304 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.304 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.304 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.304 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.304 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.304 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.304 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.304 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.305 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.306 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.306 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2023-11-26T07:15:59.306 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2023-11-26T07:15:59.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.306 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.306 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.306 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2023-11-26T07:15:59.306 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.306 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.307 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.308 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.308 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.308 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.308 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.308 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.308 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.308 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2023-11-26T07:15:59.308 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2023-11-26T07:15:59.308 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.308 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2023-11-26T07:15:59.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.310 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.310 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.310 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.310 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.310 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.310 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.310 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.310 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.310 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.310 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.311 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.312 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.313 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.313 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.313 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2023-11-26T07:15:59.313 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2023-11-26T07:15:59.313 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.313 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.313 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.313 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.313 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.313 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.314 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.314 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.314 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.314 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.314 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.314 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2023-11-26T07:15:59.314 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2023-11-26T07:15:59.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.314 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.315 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2023-11-26T07:15:59.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2023-11-26T07:15:59.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.319 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.320 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.322 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.323 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.323 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.323 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.323 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.323 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.323 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.323 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.324 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.325 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.325 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.325 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.325 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.325 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.325 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.325 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.325 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.325 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.325 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.325 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.326 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.326 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.326 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.326 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.326 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.326 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.326 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.326 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.326 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.327 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.331 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.332 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.333 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.334 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.336 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.337 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.338 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2023-11-26T07:15:59.339 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.340 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.341 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.342 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.343 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.344 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.345 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.346 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2023-11-26T07:15:59.347 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.349 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-26T07:15:59.350 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.351 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.352 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.353 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.354 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.355 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2023-11-26T07:15:59.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.357 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.358 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.359 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.361 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.362 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.363 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.365 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.366 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.367 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.367 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.367 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.367 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.367 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.367 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.367 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.367 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2023-11-26T07:15:59.367 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2023-11-26T07:15:59.368 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2023-11-26T07:15:59.369 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2023-11-26T07:15:59.370 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.370 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.370 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-26T07:15:59.370 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.370 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.370 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.370 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.370 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.370 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.370 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.371 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.373 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.374 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.375 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.376 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.377 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.378 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.378 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.378 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.378 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.378 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.378 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.378 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.378 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.378 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.378 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.378 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.379 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.380 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.381 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.382 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-26T07:15:59.383 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.384 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2023-11-26T07:15:59.385 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.386 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.386 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.386 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-26T07:15:59.386 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.386 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.386 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.386 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.386 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.386 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.387 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.388 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.388 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.388 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2023-11-26T07:15:59.388 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2023-11-26T07:15:59.388 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.388 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.388 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.388 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.388 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.388 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.389 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.390 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.391 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.392 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.393 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.394 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.396 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.398 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: "modules": [ 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: "iostat", 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: "nfs", 2023-11-26T07:15:59.399 INFO:teuthology.orchestra.run.smithi083.stdout: "restful" 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2023-11-26T07:15:59.400 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.401 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.402 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.403 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.404 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.406 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2023-11-26T07:15:59.407 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.408 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.409 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.410 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.411 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.412 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2023-11-26T07:15:59.414 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.415 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.416 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.417 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.418 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2023-11-26T07:15:59.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2023-11-26T07:15:59.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.422 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:15:59.424 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.425 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.426 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.427 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.428 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.429 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.430 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.431 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap" 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.433 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.434 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.435 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.436 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.437 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.438 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.439 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.440 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-26T07:15:59.441 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-26T07:15:59.442 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.443 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.444 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.445 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.447 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.448 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2023-11-26T07:15:59.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.450 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2023-11-26T07:15:59.451 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2023-11-26T07:15:59.451 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.451 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.451 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-26T07:15:59.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.451 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.451 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.451 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2023-11-26T07:15:59.451 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.452 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.452 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.452 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2023-11-26T07:15:59.452 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2023-11-26T07:15:59.452 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.452 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.452 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.452 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-26T07:15:59.452 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.453 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.453 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.453 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2023-11-26T07:15:59.453 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.453 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.453 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.453 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2023-11-26T07:15:59.453 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2023-11-26T07:15:59.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.453 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2023-11-26T07:15:59.454 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2023-11-26T07:15:59.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.455 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.455 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-26T07:15:59.455 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.455 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.455 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.455 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2023-11-26T07:15:59.455 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.456 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.456 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2023-11-26T07:15:59.456 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2023-11-26T07:15:59.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.456 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.456 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-26T07:15:59.456 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.456 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.457 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.457 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2023-11-26T07:15:59.457 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.457 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.457 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.457 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2023-11-26T07:15:59.457 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2023-11-26T07:15:59.457 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.457 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.457 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.460 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.461 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2023-11-26T07:15:59.461 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2023-11-26T07:15:59.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.461 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.461 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-26T07:15:59.461 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.461 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.461 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.461 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2023-11-26T07:15:59.462 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.462 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.462 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.462 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2023-11-26T07:15:59.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2023-11-26T07:15:59.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.462 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.462 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.462 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-11-26T07:15:59.462 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.463 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-26T07:15:59.464 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.464 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.464 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.464 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2023-11-26T07:15:59.464 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.464 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.464 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.464 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2023-11-26T07:15:59.464 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2023-11-26T07:15:59.464 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2023-11-26T07:15:59.465 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2023-11-26T07:15:59.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.466 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.466 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.466 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-26T07:15:59.466 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.466 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.466 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.466 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2023-11-26T07:15:59.466 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.466 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.466 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.467 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2023-11-26T07:15:59.467 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2023-11-26T07:15:59.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.467 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.467 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.467 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.467 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.467 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.467 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.467 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.468 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.469 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.470 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.472 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2023-11-26T07:15:59.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.474 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.475 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.476 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.477 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2023-11-26T07:15:59.478 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.480 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.481 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-26T07:15:59.483 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-26T07:15:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.485 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.486 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.487 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.488 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2023-11-26T07:15:59.489 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-26T07:15:59.490 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.491 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.492 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.493 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.494 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.494 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.494 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.494 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.494 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.494 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-26T07:15:59.494 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.494 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.494 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.494 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.495 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-26T07:15:59.496 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.497 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.497 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.497 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2023-11-26T07:15:59.497 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2023-11-26T07:15:59.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.497 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.497 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2023-11-26T07:15:59.497 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.497 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.498 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.499 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.500 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.501 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2023-11-26T07:15:59.502 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2023-11-26T07:15:59.503 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.504 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.505 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.506 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.507 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.509 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.510 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.510 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.510 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.510 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.510 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.510 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.510 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.510 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.510 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.510 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.510 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.511 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.512 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-26T07:15:59.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.514 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.514 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.514 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.514 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.514 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.514 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.514 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.514 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.514 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.515 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.516 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.516 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-26T07:15:59.516 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-26T07:15:59.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.516 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.516 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.516 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.516 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.517 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-26T07:15:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-26T07:15:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.518 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-26T07:15:59.519 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.520 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-26T07:15:59.521 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-26T07:15:59.521 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.521 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.521 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.521 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.521 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.521 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.521 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.521 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.521 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.521 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.522 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.523 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.525 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.526 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-26T07:15:59.527 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-26T07:15:59.528 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2023-11-26T07:15:59.529 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.530 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.531 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.533 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.534 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.535 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.536 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.537 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.538 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.539 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-26T07:15:59.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-26T07:15:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.543 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.544 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.545 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.546 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.547 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.548 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.549 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.550 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.552 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2023-11-26T07:15:59.553 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.555 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.556 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.557 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.558 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.558 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.558 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.558 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2023-11-26T07:15:59.558 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-26T07:15:59.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.558 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.558 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.558 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.558 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.560 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.561 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2023-11-26T07:15:59.562 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.563 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.565 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.566 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.567 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.567 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.567 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2023-11-26T07:15:59.567 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2023-11-26T07:15:59.567 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.567 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.567 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.567 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.567 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.567 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.568 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.569 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.570 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.571 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.572 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.573 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2023-11-26T07:15:59.574 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.575 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.577 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.578 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.579 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.580 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2023-11-26T07:15:59.581 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.582 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.583 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.584 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.585 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.586 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.588 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.589 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.590 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2023-11-26T07:15:59.593 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.594 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.595 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-26T07:15:59.597 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.598 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.599 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.600 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.601 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.602 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.603 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.604 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.605 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.606 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2023-11-26T07:15:59.607 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-26T07:15:59.608 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.609 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.610 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2023-11-26T07:15:59.611 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.612 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.613 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.614 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.615 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.616 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.618 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.619 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.620 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2023-11-26T07:15:59.621 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2023-11-26T07:15:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-26T07:15:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2023-11-26T07:15:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2023-11-26T07:15:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.623 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.626 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.627 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.628 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.630 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.632 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.633 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.634 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.635 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2023-11-26T07:15:59.637 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.638 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.639 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.640 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.641 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.642 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.643 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2023-11-26T07:15:59.644 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2023-11-26T07:15:59.645 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.646 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2023-11-26T07:15:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2023-11-26T07:15:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2023-11-26T07:15:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.647 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2023-11-26T07:15:59.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.649 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.650 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.652 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.653 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.655 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.656 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.658 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.659 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.662 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.663 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.664 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.665 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.666 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.667 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.668 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.669 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.670 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.671 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.673 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.674 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.675 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.676 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.677 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.679 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.680 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.681 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.683 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.684 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.685 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.686 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.687 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.688 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.690 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.691 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.692 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.693 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.695 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2023-11-26T07:15:59.696 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.697 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2023-11-26T07:15:59.698 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.700 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.701 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.701 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.701 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.701 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.701 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.701 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.701 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.701 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.701 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.701 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.703 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.703 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.703 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.703 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.703 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.703 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.703 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.703 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.703 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.704 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.705 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2023-11-26T07:15:59.706 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2023-11-26T07:15:59.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.706 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.706 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.706 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2023-11-26T07:15:59.706 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.706 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.706 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.706 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2023-11-26T07:15:59.706 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2023-11-26T07:15:59.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2023-11-26T07:15:59.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2023-11-26T07:15:59.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.710 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.710 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.710 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.710 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.710 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.710 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-26T07:15:59.710 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-26T07:15:59.710 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.710 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.711 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.711 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.711 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.711 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.711 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.711 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.711 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.711 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.711 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.712 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.713 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.713 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.713 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.713 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.713 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.713 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.713 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.713 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.713 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.714 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.714 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.714 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.714 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.714 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.714 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.714 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.714 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.714 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.714 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2023-11-26T07:15:59.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-26T07:15:59.717 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.718 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.719 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.723 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.724 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2023-11-26T07:15:59.724 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2023-11-26T07:15:59.724 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.724 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.724 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.724 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.724 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.724 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2023-11-26T07:15:59.724 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2023-11-26T07:15:59.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2023-11-26T07:15:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2023-11-26T07:15:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2023-11-26T07:15:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.726 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.726 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2023-11-26T07:15:59.726 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.726 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.726 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.726 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.726 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.726 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.726 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.727 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.727 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.727 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.727 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.727 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.727 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.727 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.727 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.727 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.727 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.728 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.729 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.730 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.731 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.732 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.733 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.734 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.735 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.735 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.735 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.735 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.735 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2023-11-26T07:15:59.735 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.735 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.735 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.736 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.737 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.738 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.741 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.742 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2023-11-26T07:15:59.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.744 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.746 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.747 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.748 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.749 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "drive_group_interval": { 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "drive_group_interval", 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300.0", 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.752 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.753 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.754 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.755 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.756 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.757 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.759 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.760 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.761 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.762 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.763 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2023-11-26T07:15:59.765 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.767 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.768 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.769 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.770 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.771 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.772 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2023-11-26T07:15:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2023-11-26T07:15:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.774 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2023-11-26T07:15:59.776 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.777 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.778 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.779 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.780 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.781 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.782 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.783 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.784 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.785 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.786 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.787 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.788 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.789 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.790 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.791 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.793 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.794 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.796 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.796 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.796 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.796 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.796 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.796 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.796 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.796 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.796 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.796 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.796 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.797 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.798 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-26T07:15:59.799 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.800 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.801 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.802 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.804 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.805 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.806 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.806 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.806 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.806 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.806 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.806 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.806 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.806 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.806 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.806 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.806 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.808 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.809 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2023-11-26T07:15:59.810 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2023-11-26T07:15:59.811 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2023-11-26T07:15:59.812 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.813 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.814 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.815 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.816 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.817 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2023-11-26T07:15:59.818 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.819 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.820 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.821 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.822 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.823 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.824 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.825 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.827 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.828 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.829 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.830 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.831 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.832 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.833 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.833 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.833 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.833 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.833 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.833 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.833 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.833 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.833 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.833 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.834 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.834 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.834 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.834 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.834 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.834 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.834 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.834 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.834 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.834 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.834 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.835 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.836 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.836 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.836 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.836 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.836 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.836 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.836 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.836 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.836 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.839 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.840 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.842 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.843 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.844 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.845 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2023-11-26T07:15:59.846 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.847 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.848 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.849 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.850 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.850 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.850 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.850 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.850 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.850 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2023-11-26T07:15:59.850 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2023-11-26T07:15:59.850 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.850 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.850 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.851 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.851 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.851 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.851 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.851 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.851 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.851 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.851 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.851 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.851 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.851 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.852 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.852 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.852 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.852 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.852 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2023-11-26T07:15:59.852 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2023-11-26T07:15:59.852 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.852 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.852 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.852 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.853 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.853 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.853 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.853 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.853 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.853 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.853 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.853 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2023-11-26T07:15:59.853 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2023-11-26T07:15:59.853 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2023-11-26T07:15:59.854 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2023-11-26T07:15:59.855 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.855 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.855 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.855 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.855 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.855 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2023-11-26T07:15:59.855 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2023-11-26T07:15:59.855 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2023-11-26T07:15:59.855 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2023-11-26T07:15:59.856 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2023-11-26T07:15:59.857 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.857 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.857 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.857 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2023-11-26T07:15:59.857 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.857 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.857 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.857 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.857 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.857 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.857 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.858 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2023-11-26T07:15:59.858 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2023-11-26T07:15:59.858 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2023-11-26T07:15:59.858 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.858 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.858 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2023-11-26T07:15:59.858 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.858 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.858 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.858 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2023-11-26T07:15:59.859 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: "services": {}, 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: "always_on_modules": { 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: "octopus": [ 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2023-11-26T07:15:59.860 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "pacific": [ 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2023-11-26T07:15:59.861 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: "quincy": [ 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2023-11-26T07:15:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "reef": [ 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2023-11-26T07:15:59.863 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: "last_failure_osd_epoch": 0, 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: "active_clients": [ 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2023-11-26T07:15:59.864 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1792598028 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "libcephsqlite", 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:59.865 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2023-11-26T07:15:59.866 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1164473704 2023-11-26T07:15:59.866 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.866 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.866 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2023-11-26T07:15:59.866 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:59.866 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:59.866 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2023-11-26T07:15:59.866 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 304865544 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2981312946 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1953425269 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2023-11-26T07:15:59.868 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 367450219 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:59.869 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4012256052 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3840607975 2023-11-26T07:15:59.870 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4194383148 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1574556410 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2023-11-26T07:15:59.872 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2023-11-26T07:15:59.873 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:15:59.873 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2023-11-26T07:15:59.873 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2023-11-26T07:15:59.873 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2325877059 2023-11-26T07:15:59.873 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.873 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.873 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:15:59.873 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:15:59.873 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 -- 172.21.15.83:0/684885235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03dc06b2c0 msgr2=0x7f03dc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03dc06b2c0 0x7f03dc06d780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009840 tx=0x7f03f0008030 comp rx=0 tx=0).stop 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 -- 172.21.15.83:0/684885235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03fc06c540 msgr2=0x7f03fc140e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03fc06c540 0x7f03fc140e00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c430 tx=0x7f03ec00c740 comp rx=0 tx=0).stop 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 -- 172.21.15.83:0/684885235 shutdown_connections 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7f03dc06b2c0 0x7f03dc06d780 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03fc1446a0 0x7f03fc13f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03fc06c540 0x7f03fc140e00 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 --2- 172.21.15.83:0/684885235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03fc0adf00 0x7f03fc15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 -- 172.21.15.83:0/684885235 >> 172.21.15.83:0/684885235 conn(0x7f03fc071b90 msgr2=0x7f03fc15dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 -- 172.21.15.83:0/684885235 shutdown_connections 2023-11-26T07:15:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:15:58.930+0000 7f0400ae4700 1 -- 172.21.15.83:0/684885235 wait complete. 2023-11-26T07:15:59.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail y 2023-11-26T07:16:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.011+0000 7fab57b5f700 1 Processor -- start 2023-11-26T07:16:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.011+0000 7fab57b5f700 1 -- start start 2023-11-26T07:16:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab57b5f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5015e850 0x7fab5006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab57b5f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5006dd80 con 0x7fab500afe70 2023-11-26T07:16:00.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab57b5f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5015f880 con 0x7fab5015bef0 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab57b5f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5015c2d0 con 0x7fab5015e850 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab550fa700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fab500afe70 0x7fab501624d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60012/0 (socket says 172.21.15.83:60012) 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab550fa700 1 -- 172.21.15.83:0/2249844989 learned_addr learned my addr 172.21.15.83:0/2249844989 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab560fc700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5015e850 0x7fab5006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964159618 0 0) 0x7fab5006dd80 con 0x7fab500afe70 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab3c0035b0 con 0x7fab500afe70 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4080981184 0 0) 0x7fab3c0035b0 con 0x7fab500afe70 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5015e850 msgr2=0x7fab5006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab548f9700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5015e850 0x7fab5006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 >> v1:172.21.15.203:6789/0 conn(0x7fab5015bef0 legacy=0x7fab500b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5015efe0 con 0x7fab500afe70 2023-11-26T07:16:00.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.012+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fab4c001670 con 0x7fab500afe70 2023-11-26T07:16:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 >> v1:172.21.15.83:6789/0 conn(0x7fab500afe70 legacy=0x7fab501624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab548f9700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5015e850 0x7fab3c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab548f9700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3c003fb0 0x7fab3c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab548f9700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab3c004d90 0x7fab3c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5015c2d0 con 0x7fab3c003fb0 2023-11-26T07:16:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c00bba0 con 0x7fab3c004d90 2023-11-26T07:16:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c00bd60 con 0x7fab5015e850 2023-11-26T07:16:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab550fa700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3c003fb0 0x7fab3c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab558fb700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5015e850 0x7fab3c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab560fc700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab3c004d90 0x7fab3c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab550fa700 1 -- 172.21.15.83:0/2249844989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5015e850 msgr2=0x7fab3c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab550fa700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab5015e850 0x7fab3c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab550fa700 1 -- 172.21.15.83:0/2249844989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab3c004d90 msgr2=0x7fab3c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab550fa700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab3c004d90 0x7fab3c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab550fa700 1 -- 172.21.15.83:0/2249844989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c0035b0 con 0x7fab3c003fb0 2023-11-26T07:16:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab550fa700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3c003fb0 0x7fab3c004850 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00c240 tx=0x7fab3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135537bb32272289 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c007c10 con 0x7fab3c003fb0 2023-11-26T07:16:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c00cde0 con 0x7fab3c003fb0 2023-11-26T07:16:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.013+0000 7fab548f9700 1 -- 172.21.15.83:0/2249844989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c005ba0 con 0x7fab3c003fb0 2023-11-26T07:16:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.014+0000 7fab57b5f700 1 -- 172.21.15.83:0/2249844989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3c003fb0 msgr2=0x7fab3c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.014+0000 7fab57b5f700 1 --2- 172.21.15.83:0/2249844989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3c003fb0 0x7fab3c004850 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00c240 tx=0x7fab3c005880 comp rx=0 tx=0).stop 2023-11-26T07:16:00.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.014+0000 7fab560fc700 1 -- 172.21.15.83:0/2249844989 reap_dead start 2023-11-26T07:16:00.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.014+0000 7fab57b5f700 1 -- 172.21.15.83:0/2249844989 shutdown_connections 2023-11-26T07:16:00.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.014+0000 7fab57b5f700 1 -- 172.21.15.83:0/2249844989 >> 172.21.15.83:0/2249844989 conn(0x7fab50071ac0 msgr2=0x7fab50071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:00.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.014+0000 7fab57b5f700 1 -- 172.21.15.83:0/2249844989 shutdown_connections 2023-11-26T07:16:00.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.014+0000 7fab57b5f700 1 -- 172.21.15.83:0/2249844989 wait complete. 2023-11-26T07:16:00.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.015+0000 7fab57b5f700 1 Processor -- start 2023-11-26T07:16:00.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.015+0000 7fab57b5f700 1 -- start start 2023-11-26T07:16:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab57b5f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab3c003fb0 0x7fab5015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab57b5f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab501464e0 0x7fab500b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab57b5f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab50146960 0x7fab500b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab57b5f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5015b6d0 con 0x7fab501464e0 2023-11-26T07:16:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab57b5f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5015a160 con 0x7fab3c003fb0 2023-11-26T07:16:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab57b5f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5015a3d0 con 0x7fab50146960 2023-11-26T07:16:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab550fa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab501464e0 0x7fab500b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab550fa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab501464e0 0x7fab500b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39984/0 (socket says 172.21.15.83:39984) 2023-11-26T07:16:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab550fa700 1 -- 172.21.15.83:0/3200969220 learned_addr learned my addr 172.21.15.83:0/3200969220 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab560fc700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab50146960 0x7fab500b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab558fb700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab3c003fb0 0x7fab5015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab550fa700 1 -- 172.21.15.83:0/3200969220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab50146960 msgr2=0x7fab500b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab550fa700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab50146960 0x7fab500b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab550fa700 1 -- 172.21.15.83:0/3200969220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab3c003fb0 msgr2=0x7fab5015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab550fa700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab3c003fb0 0x7fab5015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab550fa700 1 -- 172.21.15.83:0/3200969220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5015a6b0 con 0x7fab501464e0 2023-11-26T07:16:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab558fb700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab3c003fb0 0x7fab5015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.016+0000 7fab550fa700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab501464e0 0x7fab500b08b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fab4c005ab0 tx=0x7fab4c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.017+0000 7fab467fc700 1 -- 172.21.15.83:0/3200969220 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c000ed0 con 0x7fab501464e0 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.017+0000 7fab467fc700 1 -- 172.21.15.83:0/3200969220 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c0070d0 con 0x7fab501464e0 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.017+0000 7fab57b5f700 1 -- 172.21.15.83:0/3200969220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5015a990 con 0x7fab501464e0 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.017+0000 7fab467fc700 1 -- 172.21.15.83:0/3200969220 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c007480 con 0x7fab501464e0 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.017+0000 7fab57b5f700 1 -- 172.21.15.83:0/3200969220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5014c710 con 0x7fab501464e0 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.018+0000 7fab57b5f700 1 -- 172.21.15.83:0/3200969220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab50067f60 con 0x7fab501464e0 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.018+0000 7fab467fc700 1 -- 172.21.15.83:0/3200969220 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fab4c019030 con 0x7fab501464e0 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.018+0000 7fab467fc700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fab3406b2c0 0x7fab3406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.019+0000 7fab467fc700 1 -- 172.21.15.83:0/3200969220 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab4c08b040 con 0x7fab501464e0 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.019+0000 7fab558fb700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fab3406b2c0 0x7fab3406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.019+0000 7fab558fb700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fab3406b2c0 0x7fab3406d780 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fab40000c00 tx=0x7fab40009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:00.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.023+0000 7fab467fc700 1 -- 172.21.15.83:0/3200969220 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab4c0bde90 con 0x7fab501464e0 2023-11-26T07:16:00.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.269+0000 7fab57b5f700 1 -- 172.21.15.83:0/3200969220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr fail", "who": "y"} v 0) v1 -- 0x7fab50161c40 con 0x7fab501464e0 2023-11-26T07:16:00.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.272+0000 7fab467fc700 1 -- 172.21.15.83:0/3200969220 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "y"}]=0 v13) v1 ==== 68+0+0 (secure 0 0 0) 0x7fab4c055cc0 con 0x7fab501464e0 2023-11-26T07:16:00.276 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:00.273+0000 7f37fb7b4700 -1 mgr handle_mgr_map I was active but no longer am 2023-11-26T07:16:00.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.276+0000 7fab467fc700 1 -- 172.21.15.83:0/3200969220 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 13) v1 ==== 44567+0+0 (secure 0 0 0) 0x7fab4c056090 con 0x7fab501464e0 2023-11-26T07:16:00.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.276+0000 7fab467fc700 1 -- 172.21.15.83:0/3200969220 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fab3406b2c0 msgr2=0x7fab3406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.276+0000 7fab467fc700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fab3406b2c0 0x7fab3406d780 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fab40000c00 tx=0x7fab40009040 comp rx=0 tx=0).stop 2023-11-26T07:16:00.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.276+0000 7fab57b5f700 1 -- 172.21.15.83:0/3200969220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab501464e0 msgr2=0x7fab500b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.276+0000 7fab57b5f700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab501464e0 0x7fab500b08b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fab4c005ab0 tx=0x7fab4c001a00 comp rx=0 tx=0).stop 2023-11-26T07:16:00.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.277+0000 7fab57b5f700 1 -- 172.21.15.83:0/3200969220 shutdown_connections 2023-11-26T07:16:00.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.277+0000 7fab57b5f700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.83:6832/1483230064,v1:172.21.15.83:6833/1483230064] conn(0x7fab3406b2c0 0x7fab3406d780 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.277+0000 7fab57b5f700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab50146960 0x7fab500b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.277+0000 7fab57b5f700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab3c003fb0 0x7fab5015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.277+0000 7fab57b5f700 1 --2- 172.21.15.83:0/3200969220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab501464e0 0x7fab500b08b0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.277+0000 7fab57b5f700 1 -- 172.21.15.83:0/3200969220 >> 172.21.15.83:0/3200969220 conn(0x7fab50071ac0 msgr2=0x7fab501575c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:00.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.277+0000 7fab57b5f700 1 -- 172.21.15.83:0/3200969220 shutdown_connections 2023-11-26T07:16:00.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.277+0000 7fab57b5f700 1 -- 172.21.15.83:0/3200969220 wait complete. 2023-11-26T07:16:00.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:00.376 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:16:00.374+0000 7f30f9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:16:00.376 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:16:00.374+0000 7f30f9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:16:00.376 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:16:00.374+0000 7f30f9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:16:00.376 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:16:00.374+0000 7f30f9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:16:00.377 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:16:00.374+0000 7f30f9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:16:00.377 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:16:00.374+0000 7f30f5f2c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:16:00.377 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:16:00.374+0000 7f30f5f2c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:16:00.377 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:16:00.374+0000 7f30f5f2c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:16:00.377 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:16:00.374+0000 7f30f5f2c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:16:00.377 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:16:00.374+0000 7f30f5f2c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:16:00.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.440+0000 7f72a67cf700 1 Processor -- start 2023-11-26T07:16:00.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.440+0000 7f72a67cf700 1 -- start start 2023-11-26T07:16:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.441+0000 7f72a67cf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce4d0 0x7f72a00d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.441+0000 7f72a67cf700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f72a00ce910 0x7f72a00d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.441+0000 7f72a67cf700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a00d0150 con 0x7f72a015e080 2023-11-26T07:16:00.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.441+0000 7f72a67cf700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a015e460 con 0x7f72a00ce910 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.441+0000 7f72a67cf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00cc9b0 con 0x7f72a00ce4d0 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.441+0000 7f729f7fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce4d0 0x7f72a00d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.441+0000 7f729f7fe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce4d0 0x7f72a00d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:50462/0 (socket says 172.21.15.83:50462) 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.441+0000 7f72a4d6c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f72a00ce910 0x7f72a00d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.441+0000 7f729f7fe700 1 -- 172.21.15.83:0/1353325721 learned_addr learned my addr 172.21.15.83:0/1353325721 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.441+0000 7f729ffff700 1 --1- 172.21.15.83:0/1353325721 >> v1:172.21.15.203:6789/0 conn(0x7f72a015e080 0x7f72a00cc2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:44660/0 (socket says 172.21.15.83:44660) 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.442+0000 7f729effd700 1 -- 172.21.15.83:0/1353325721 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1776331446 0 0) 0x7f72a00d0150 con 0x7f72a015e080 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.442+0000 7f729f7fe700 1 -- 172.21.15.83:0/1353325721 >> v2:172.21.15.203:3300/0 conn(0x7f72a00ce910 msgr2=0x7f72a00d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.442+0000 7f729f7fe700 1 --2- 172.21.15.83:0/1353325721 >> v2:172.21.15.203:3300/0 conn(0x7f72a00ce910 0x7f72a00d2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.442+0000 7f729f7fe700 1 -- 172.21.15.83:0/1353325721 >> v1:172.21.15.203:6789/0 conn(0x7f72a015e080 legacy=0x7f72a00cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.442+0000 7f729f7fe700 1 -- 172.21.15.83:0/1353325721 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a00cee50 con 0x7f72a00ce4d0 2023-11-26T07:16:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.442+0000 7f72a4d6c700 1 --2- 172.21.15.83:0/1353325721 >> v2:172.21.15.203:3300/0 conn(0x7f72a00ce910 0x7f72a00d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.442+0000 7f729f7fe700 1 --2- 172.21.15.83:0/1353325721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce4d0 0x7f72a00d1be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72940098f0 tx=0x7f7294009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4acfe8af842c6491 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.442+0000 7f729effd700 1 -- 172.21.15.83:0/1353325721 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294011830 con 0x7f72a00ce4d0 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.442+0000 7f729effd700 1 -- 172.21.15.83:0/1353325721 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7294011990 con 0x7f72a00ce4d0 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.442+0000 7f729effd700 1 -- 172.21.15.83:0/1353325721 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294011d20 con 0x7f72a00ce4d0 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.443+0000 7f72a67cf700 1 -- 172.21.15.83:0/1353325721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce4d0 msgr2=0x7f72a00d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.443+0000 7f72a67cf700 1 --2- 172.21.15.83:0/1353325721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce4d0 0x7f72a00d1be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72940098f0 tx=0x7f7294009c00 comp rx=0 tx=0).stop 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.443+0000 7f72a67cf700 1 -- 172.21.15.83:0/1353325721 shutdown_connections 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.443+0000 7f72a67cf700 1 --2- 172.21.15.83:0/1353325721 >> v2:172.21.15.203:3300/0 conn(0x7f72a00ce910 0x7f72a00d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.443+0000 7f72a67cf700 1 --2- 172.21.15.83:0/1353325721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce4d0 0x7f72a00d1be0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.443+0000 7f72a67cf700 1 -- 172.21.15.83:0/1353325721 >> 172.21.15.83:0/1353325721 conn(0x7f72a0073130 msgr2=0x7f72a009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.443+0000 7f72a67cf700 1 -- 172.21.15.83:0/1353325721 shutdown_connections 2023-11-26T07:16:00.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.443+0000 7f72a67cf700 1 -- 172.21.15.83:0/1353325721 wait complete. 2023-11-26T07:16:00.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.444+0000 7f72a67cf700 1 Processor -- start 2023-11-26T07:16:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.444+0000 7f72a67cf700 1 -- start start 2023-11-26T07:16:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.445+0000 7f72a67cf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a00ce4d0 0x7f72a00ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.445+0000 7f72a67cf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce910 0x7f72a015c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.445+0000 7f72a67cf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72a015e080 0x7f72a00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.445+0000 7f72a67cf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00ca3a0 con 0x7f72a00ce4d0 2023-11-26T07:16:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.445+0000 7f72a67cf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00c97d0 con 0x7f72a015e080 2023-11-26T07:16:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.445+0000 7f72a67cf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00c9a70 con 0x7f72a00ce910 2023-11-26T07:16:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.445+0000 7f729ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a00ce4d0 0x7f72a00ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.445+0000 7f729ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a00ce4d0 0x7f72a00ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40008/0 (socket says 172.21.15.83:40008) 2023-11-26T07:16:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f729ffff700 1 -- 172.21.15.83:0/2446782968 learned_addr learned my addr 172.21.15.83:0/2446782968 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:00.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f729f7fe700 1 --2- 172.21.15.83:0/2446782968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce910 0x7f72a015c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f72a4d6c700 1 --2- 172.21.15.83:0/2446782968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72a015e080 0x7f72a00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f729ffff700 1 -- 172.21.15.83:0/2446782968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce910 msgr2=0x7f72a015c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f729ffff700 1 --2- 172.21.15.83:0/2446782968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce910 0x7f72a015c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f729ffff700 1 -- 172.21.15.83:0/2446782968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72a015e080 msgr2=0x7f72a00b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f729ffff700 1 --2- 172.21.15.83:0/2446782968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72a015e080 0x7f72a00b6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f729ffff700 1 -- 172.21.15.83:0/2446782968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7298009a70 con 0x7f72a00ce4d0 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f729ffff700 1 --2- 172.21.15.83:0/2446782968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a00ce4d0 0x7f72a00ca830 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f729000c550 tx=0x7f729000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f729cff9700 1 -- 172.21.15.83:0/2446782968 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290004c80 con 0x7f72a00ce4d0 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f729cff9700 1 -- 172.21.15.83:0/2446782968 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7290004e00 con 0x7f72a00ce4d0 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f72a67cf700 1 -- 172.21.15.83:0/2446782968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7294009550 con 0x7f72a00ce4d0 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f729cff9700 1 -- 172.21.15.83:0/2446782968 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7290007310 con 0x7f72a00ce4d0 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.446+0000 7f72a67cf700 1 -- 172.21.15.83:0/2446782968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a00ca030 con 0x7f72a00ce4d0 2023-11-26T07:16:00.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.447+0000 7f729cff9700 1 -- 172.21.15.83:0/2446782968 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 44567+0+0 (secure 0 0 0) 0x7f7290007470 con 0x7f72a00ce4d0 2023-11-26T07:16:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.447+0000 7f72a67cf700 1 -- 172.21.15.83:0/2446782968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a0067f60 con 0x7f72a00ce4d0 2023-11-26T07:16:00.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.447+0000 7f729cff9700 1 -- 172.21.15.83:0/2446782968 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f729004a320 con 0x7f72a00ce4d0 2023-11-26T07:16:00.452 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:00.449+0000 7fcd9b9ee200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-26T07:16:00.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.452+0000 7f729cff9700 1 -- 172.21.15.83:0/2446782968 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f72900133c0 con 0x7f72a00ce4d0 2023-11-26T07:16:00.520 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:00.518+0000 7fcd9b9ee200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-26T07:16:00.592 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:00.589+0000 7fcd9b9ee200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-26T07:16:00.670 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:00.668+0000 7fcd9b9ee200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-26T07:16:00.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.694+0000 7f72a67cf700 1 -- 172.21.15.83:0/2446782968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72a00b7bd0 con 0x7f72a00ce4d0 2023-11-26T07:16:00.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.695+0000 7f729cff9700 1 -- 172.21.15.83:0/2446782968 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f7290011360 con 0x7f72a00ce4d0 2023-11-26T07:16:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:00.697 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5420,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2163151849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2163151849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2163151849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1539108478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1539108478},{"type":"v1","addr":"172.21.15.203:6835","nonce":1539108478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1387613510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1387613510},{"type":"v1","addr":"172.21.15.203:6834","nonce":1387613510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:15:42.339653+0000","modified":"2023-11-26T07:15:45.361821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1809465302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1809465302},{"type":"v1","addr":"172.21.15.83:6836","nonce":1809465302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:16:00.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.697+0000 7f72a67cf700 1 -- 172.21.15.83:0/2446782968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a00ce4d0 msgr2=0x7f72a00ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:00.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.697+0000 7f72a67cf700 1 --2- 172.21.15.83:0/2446782968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a00ce4d0 0x7f72a00ca830 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f729000c550 tx=0x7f729000c860 comp rx=0 tx=0).stop 2023-11-26T07:16:00.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.697+0000 7f72a67cf700 1 -- 172.21.15.83:0/2446782968 shutdown_connections 2023-11-26T07:16:00.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.697+0000 7f72a67cf700 1 --2- 172.21.15.83:0/2446782968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72a015e080 0x7f72a00b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.697+0000 7f72a67cf700 1 --2- 172.21.15.83:0/2446782968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a00ce4d0 0x7f72a00ca830 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.697+0000 7f72a67cf700 1 --2- 172.21.15.83:0/2446782968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72a00ce910 0x7f72a015c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.697+0000 7f72a67cf700 1 -- 172.21.15.83:0/2446782968 >> 172.21.15.83:0/2446782968 conn(0x7f72a0073130 msgr2=0x7f72a006fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.697+0000 7f72a67cf700 1 -- 172.21.15.83:0/2446782968 shutdown_connections 2023-11-26T07:16:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:00.697+0000 7f72a67cf700 1 -- 172.21.15.83:0/2446782968 wait complete. 2023-11-26T07:16:00.701 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 47 2023-11-26T07:16:00.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:16:00.820 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:00.818+0000 7fcd9b9ee200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-26T07:16:00.857 INFO:teuthology.orchestra.run.smithi083.stdout:[{"id":6936,"entity":{"name":{"type":"client","num":6936},"addr":{"type":"any","addr":"172.21.15.203:0","nonce":3842876259}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.46698888599999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978686044473282,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6936 172.21.15.203:0/3842876259","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi203","pid":"103138","root":"/"}},{"id":6244,"entity":{"name":{"type":"client","num":6244},"addr":{"type":"any","addr":"172.21.15.203:0","nonce":2702588363}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.46698888599999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978686044473282,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6244 172.21.15.203:0/2702588363","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi203","pid":"103138","root":"/"}}] 2023-11-26T07:16:00.857 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi203", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103138", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.203:0", "nonce": 3842876259, "type": "any" }, "name": { "num": 6936, "type": "client" } }, "id": 6936, "inst": "client.6936 172.21.15.203:0/3842876259", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978686044473282 }, "state": "open", "uptime": 0.466988886 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi203", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "103138", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.203:0", "nonce": 2702588363, "type": "any" }, "name": { "num": 6244, "type": "client" } }, "id": 6244, "inst": "client.6244 172.21.15.203:0/2702588363", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978686044473282 }, "state": "open", "uptime": 0.466988886 } ] 2023-11-26T07:16:00.858 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-26T07:16:00.858 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:16:00.858 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:16:00.897 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:16:00.946 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:16:00.946 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:16:00.946 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:00.963 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:16:00.963 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:16:00.963 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:00.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:16:00.983 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:16:00.983 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:01.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:16:01.019 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:16:01.019 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:01.042 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:16:01.042 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:01.160 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:01.157+0000 7fcd9b9ee200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-26T07:16:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.179+0000 7f2343bc5700 1 Processor -- start 2023-11-26T07:16:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.180+0000 7f2343bc5700 1 -- start start 2023-11-26T07:16:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.180+0000 7f2343bc5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f233c15c140 0x7f233c0cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.180+0000 7f2343bc5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233c0cc780 con 0x7f233c0cf2b0 2023-11-26T07:16:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.180+0000 7f2343bc5700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233c0cc9b0 con 0x7f233c15bd60 2023-11-26T07:16:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.180+0000 7f2343bc5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f233c0ccc90 con 0x7f233c15c140 2023-11-26T07:16:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.180+0000 7f2341160700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f233c15c140 0x7f233c0cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.180+0000 7f2342162700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f233c0cf2b0 0x7f233c0c57c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60058/0 (socket says 172.21.15.83:60058) 2023-11-26T07:16:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.180+0000 7f2342162700 1 -- 172.21.15.83:0/1877794244 learned_addr learned my addr 172.21.15.83:0/1877794244 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.181+0000 7f234095f700 1 -- 172.21.15.83:0/1877794244 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988152805 0 0) 0x7f233c0cc780 con 0x7f233c0cf2b0 2023-11-26T07:16:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.181+0000 7f234095f700 1 -- 172.21.15.83:0/1877794244 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23280035b0 con 0x7f233c0cf2b0 2023-11-26T07:16:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.181+0000 7f2341160700 1 -- 172.21.15.83:0/1877794244 >> v1:172.21.15.203:6789/0 conn(0x7f233c15bd60 legacy=0x7f233c0d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.181+0000 7f2341160700 1 -- 172.21.15.83:0/1877794244 >> v1:172.21.15.83:6789/0 conn(0x7f233c0cf2b0 legacy=0x7f233c0c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.181+0000 7f2341160700 1 -- 172.21.15.83:0/1877794244 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c0ccf70 con 0x7f233c15c140 2023-11-26T07:16:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.181+0000 7f2341160700 1 --2- 172.21.15.83:0/1877794244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233c15c140 0x7f233c0cda50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f232c009840 tx=0x7f232c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edeb3dc3a69eab31 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.181+0000 7f234095f700 1 -- 172.21.15.83:0/1877794244 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c004230 con 0x7f233c15c140 2023-11-26T07:16:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.181+0000 7f234095f700 1 -- 172.21.15.83:0/1877794244 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c004390 con 0x7f233c15c140 2023-11-26T07:16:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.181+0000 7f234095f700 1 -- 172.21.15.83:0/1877794244 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c0036a0 con 0x7f233c15c140 2023-11-26T07:16:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.182+0000 7f2343bc5700 1 -- 172.21.15.83:0/1877794244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233c15c140 msgr2=0x7f233c0cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.182+0000 7f2343bc5700 1 --2- 172.21.15.83:0/1877794244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233c15c140 0x7f233c0cda50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f232c009840 tx=0x7f232c009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.182+0000 7f2343bc5700 1 -- 172.21.15.83:0/1877794244 shutdown_connections 2023-11-26T07:16:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.182+0000 7f2343bc5700 1 --2- 172.21.15.83:0/1877794244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233c15c140 0x7f233c0cda50 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.182+0000 7f2343bc5700 1 -- 172.21.15.83:0/1877794244 >> 172.21.15.83:0/1877794244 conn(0x7f233c073130 msgr2=0x7f233c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.182+0000 7f2343bc5700 1 -- 172.21.15.83:0/1877794244 shutdown_connections 2023-11-26T07:16:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.182+0000 7f2343bc5700 1 -- 172.21.15.83:0/1877794244 wait complete. 2023-11-26T07:16:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.183+0000 7f2343bc5700 1 Processor -- start 2023-11-26T07:16:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.183+0000 7f2343bc5700 1 -- start start 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2343bc5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233c0cf2b0 0x7f233c0c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2343bc5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c15bd60 0x7f233c0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2343bc5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233c15c140 0x7f233c06fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2343bc5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c146f10 con 0x7f233c15c140 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2343bc5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c1471f0 con 0x7f233c15bd60 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2343bc5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c1474d0 con 0x7f233c0cf2b0 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2342162700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233c15c140 0x7f233c06fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2342162700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233c15c140 0x7f233c06fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40024/0 (socket says 172.21.15.83:40024) 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2342162700 1 -- 172.21.15.83:0/1890057255 learned_addr learned my addr 172.21.15.83:0/1890057255 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2341160700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c15bd60 0x7f233c0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2341961700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233c0cf2b0 0x7f233c0c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2342162700 1 -- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233c0cf2b0 msgr2=0x7f233c0c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2342162700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233c0cf2b0 0x7f233c0c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2342162700 1 -- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c15bd60 msgr2=0x7f233c0cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2342162700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c15bd60 0x7f233c0cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2342162700 1 -- 172.21.15.83:0/1890057255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c0094a0 con 0x7f233c15c140 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2341160700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c15bd60 0x7f233c0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2342162700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233c15c140 0x7f233c06fe10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f233400cb40 tx=0x7f233400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f23327fc700 1 -- 172.21.15.83:0/1890057255 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233400ab50 con 0x7f233c15c140 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f23327fc700 1 -- 172.21.15.83:0/1890057255 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233400acb0 con 0x7f233c15c140 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f23327fc700 1 -- 172.21.15.83:0/1890057255 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334012620 con 0x7f233c15c140 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2343bc5700 1 -- 172.21.15.83:0/1890057255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233c1477b0 con 0x7f233c15c140 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.184+0000 7f2343bc5700 1 -- 172.21.15.83:0/1890057255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233c0cc010 con 0x7f233c15c140 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.185+0000 7f2343bc5700 1 -- 172.21.15.83:0/1890057255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233c067f60 con 0x7f233c15c140 2023-11-26T07:16:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.185+0000 7f23327fc700 1 -- 172.21.15.83:0/1890057255 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 44567+0+0 (secure 0 0 0) 0x7f233401b3b0 con 0x7f233c15c140 2023-11-26T07:16:01.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.186+0000 7f23327fc700 1 -- 172.21.15.83:0/1890057255 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f233404a500 con 0x7f233c15c140 2023-11-26T07:16:01.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.190+0000 7f23327fc700 1 -- 172.21.15.83:0/1890057255 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23340192c0 con 0x7f233c15c140 2023-11-26T07:16:01.279 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.277+0000 7f23327fc700 1 -- 172.21.15.83:0/1890057255 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 14) v1 ==== 44694+0+0 (secure 0 0 0) 0x7f233401b660 con 0x7f233c15c140 2023-11-26T07:16:01.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.277+0000 7f23327fc700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f232003d4e0 0x7f232003f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.278+0000 7f2341961700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f232003d4e0 0x7f232003f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.279+0000 7f2341961700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f232003d4e0 0x7f232003f8d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2338003e60 tx=0x7f2338009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:01.282 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:01.280+0000 7fcd9b9ee200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-26T07:16:01.358 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:01.356+0000 7fcd9b9ee200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-26T07:16:01.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.431+0000 7f2343bc5700 1 -- 172.21.15.83:0/1890057255 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f233c0b7bf0 con 0x7f233c15c140 2023-11-26T07:16:01.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.432+0000 7f23327fc700 1 -- 172.21.15.83:0/1890057255 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f23340192c0 con 0x7f233c15c140 2023-11-26T07:16:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:01.435 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5420,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2163151849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2163151849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2163151849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1539108478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1539108478},{"type":"v1","addr":"172.21.15.203:6835","nonce":1539108478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1387613510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1387613510},{"type":"v1","addr":"172.21.15.203:6834","nonce":1387613510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:15:42.339653+0000","modified":"2023-11-26T07:15:45.361821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1809465302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1809465302},{"type":"v1","addr":"172.21.15.83:6836","nonce":1809465302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:16:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.434+0000 7f2343bc5700 1 -- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f232003d4e0 msgr2=0x7f232003f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.434+0000 7f2343bc5700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f232003d4e0 0x7f232003f8d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2338003e60 tx=0x7f2338009040 comp rx=0 tx=0).stop 2023-11-26T07:16:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.434+0000 7f2343bc5700 1 -- 172.21.15.83:0/1890057255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233c15c140 msgr2=0x7f233c06fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.434+0000 7f2343bc5700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233c15c140 0x7f233c06fe10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f233400cb40 tx=0x7f233400cf00 comp rx=0 tx=0).stop 2023-11-26T07:16:01.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.434+0000 7f2343bc5700 1 -- 172.21.15.83:0/1890057255 shutdown_connections 2023-11-26T07:16:01.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.434+0000 7f2343bc5700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f232003d4e0 0x7f232003f8d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.434+0000 7f2343bc5700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f233c15c140 0x7f233c06fe10 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.434+0000 7f2343bc5700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f233c0cf2b0 0x7f233c0c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.434+0000 7f2343bc5700 1 --2- 172.21.15.83:0/1890057255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f233c15bd60 0x7f233c0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.434+0000 7f2343bc5700 1 -- 172.21.15.83:0/1890057255 >> 172.21.15.83:0/1890057255 conn(0x7f233c073130 msgr2=0x7f233c0c6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:01.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.435+0000 7f2343bc5700 1 -- 172.21.15.83:0/1890057255 shutdown_connections 2023-11-26T07:16:01.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.435+0000 7f2343bc5700 1 -- 172.21.15.83:0/1890057255 wait complete. 2023-11-26T07:16:01.438 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 47 2023-11-26T07:16:01.441 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:01.439+0000 7fcd9b9ee200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-26T07:16:01.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:01.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.572+0000 7f2d3c721700 1 Processor -- start 2023-11-26T07:16:01.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.572+0000 7f2d3c721700 1 -- start start 2023-11-26T07:16:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d3c721700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d34145760 0x7f2d3406e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d3c721700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3406ee70 con 0x7f2d3406df20 2023-11-26T07:16:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d3c721700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3406d4d0 con 0x7f2d3415ffb0 2023-11-26T07:16:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d3c721700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d340cc5e0 con 0x7f2d34145760 2023-11-26T07:16:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d3a4bd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d34145760 0x7f2d3406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d3a4bd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d34145760 0x7f2d3406e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40042/0 (socket says 172.21.15.83:40042) 2023-11-26T07:16:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d3a4bd700 1 -- 172.21.15.83:0/3583437845 learned_addr learned my addr 172.21.15.83:0/3583437845 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d3a4bd700 1 -- 172.21.15.83:0/3583437845 >> v1:172.21.15.203:6789/0 conn(0x7f2d3415ffb0 legacy=0x7f2d340cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d3a4bd700 1 -- 172.21.15.83:0/3583437845 >> v1:172.21.15.83:6789/0 conn(0x7f2d3406df20 legacy=0x7f2d3415f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d3a4bd700 1 -- 172.21.15.83:0/3583437845 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d340cc8c0 con 0x7f2d34145760 2023-11-26T07:16:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d3a4bd700 1 --2- 172.21.15.83:0/3583437845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d34145760 0x7f2d3406e930 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2d300098f0 tx=0x7f2d30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77cd982cd6866ce server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.573+0000 7f2d394bb700 1 -- 172.21.15.83:0/3583437845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000b8a0 con 0x7f2d34145760 2023-11-26T07:16:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.574+0000 7f2d394bb700 1 -- 172.21.15.83:0/3583437845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3000ba00 con 0x7f2d34145760 2023-11-26T07:16:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.574+0000 7f2d394bb700 1 -- 172.21.15.83:0/3583437845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3000bd90 con 0x7f2d34145760 2023-11-26T07:16:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.574+0000 7f2d3c721700 1 -- 172.21.15.83:0/3583437845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d34145760 msgr2=0x7f2d3406e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.574+0000 7f2d3c721700 1 --2- 172.21.15.83:0/3583437845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d34145760 0x7f2d3406e930 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2d300098f0 tx=0x7f2d30009c00 comp rx=0 tx=0).stop 2023-11-26T07:16:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.574+0000 7f2d3c721700 1 -- 172.21.15.83:0/3583437845 shutdown_connections 2023-11-26T07:16:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.574+0000 7f2d3c721700 1 --2- 172.21.15.83:0/3583437845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d34145760 0x7f2d3406e930 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.574+0000 7f2d3c721700 1 -- 172.21.15.83:0/3583437845 >> 172.21.15.83:0/3583437845 conn(0x7f2d340b1740 msgr2=0x7f2d340b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.574+0000 7f2d3c721700 1 -- 172.21.15.83:0/3583437845 shutdown_connections 2023-11-26T07:16:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.574+0000 7f2d3c721700 1 -- 172.21.15.83:0/3583437845 wait complete. 2023-11-26T07:16:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.575+0000 7f2d3c721700 1 Processor -- start 2023-11-26T07:16:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.575+0000 7f2d3c721700 1 -- start start 2023-11-26T07:16:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.576+0000 7f2d3c721700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d3406df20 0x7f2d341371a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.576+0000 7f2d3c721700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d34145760 0x7f2d340b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.576+0000 7f2d3c721700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d3415ffb0 0x7f2d340b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.576+0000 7f2d3c721700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340af2e0 con 0x7f2d3406df20 2023-11-26T07:16:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.576+0000 7f2d3c721700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d34141fa0 con 0x7f2d34145760 2023-11-26T07:16:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.576+0000 7f2d3c721700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340af000 con 0x7f2d3415ffb0 2023-11-26T07:16:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.576+0000 7f2d3a4bd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d3406df20 0x7f2d341371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.576+0000 7f2d3a4bd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d3406df20 0x7f2d341371a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40048/0 (socket says 172.21.15.83:40048) 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.576+0000 7f2d3a4bd700 1 -- 172.21.15.83:0/3038000566 learned_addr learned my addr 172.21.15.83:0/3038000566 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d39cbc700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d34145760 0x7f2d340b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d3acbe700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d3415ffb0 0x7f2d340b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d3a4bd700 1 -- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d3415ffb0 msgr2=0x7f2d340b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d3a4bd700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d3415ffb0 0x7f2d340b41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d3a4bd700 1 -- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d34145760 msgr2=0x7f2d340b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d3a4bd700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d34145760 0x7f2d340b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d3a4bd700 1 -- 172.21.15.83:0/3038000566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d30009550 con 0x7f2d3406df20 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d3a4bd700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d3406df20 0x7f2d341371a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2d30000c00 tx=0x7f2d30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d2affd700 1 -- 172.21.15.83:0/3038000566 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30004570 con 0x7f2d3406df20 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d2affd700 1 -- 172.21.15.83:0/3038000566 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3001f070 con 0x7f2d3406df20 2023-11-26T07:16:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d3c721700 1 -- 172.21.15.83:0/3038000566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d34142240 con 0x7f2d3406df20 2023-11-26T07:16:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d2affd700 1 -- 172.21.15.83:0/3038000566 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300038e0 con 0x7f2d3406df20 2023-11-26T07:16:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.577+0000 7f2d3c721700 1 -- 172.21.15.83:0/3038000566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d341376e0 con 0x7f2d3406df20 2023-11-26T07:16:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.578+0000 7f2d3c721700 1 -- 172.21.15.83:0/3038000566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d340751e0 con 0x7f2d3406df20 2023-11-26T07:16:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.579+0000 7f2d2affd700 1 -- 172.21.15.83:0/3038000566 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 14) v1 ==== 44694+0+0 (secure 0 0 0) 0x7f2d30021030 con 0x7f2d3406df20 2023-11-26T07:16:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.579+0000 7f2d2affd700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2d18037810 0x7f2d18039cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.579+0000 7f2d2affd700 1 -- 172.21.15.83:0/3038000566 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f2d3004c250 con 0x7f2d3406df20 2023-11-26T07:16:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.579+0000 7f2d39cbc700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2d18037810 0x7f2d18039cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.580+0000 7f2d39cbc700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2d18037810 0x7f2d18039cd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2d340b3c40 tx=0x7f2d24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:01.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.583+0000 7f2d2affd700 1 -- 172.21.15.83:0/3038000566 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d3001a450 con 0x7f2d3406df20 2023-11-26T07:16:01.784 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.782+0000 7f2d3c721700 1 -- 172.21.15.83:0/3038000566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d340af700 con 0x7f2d3406df20 2023-11-26T07:16:01.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.783+0000 7f2d2affd700 1 -- 172.21.15.83:0/3038000566 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17365 (secure 0 0 0) 0x7f2d30015020 con 0x7f2d3406df20 2023-11-26T07:16:01.786 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:01.787 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":71,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:00.270920+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:15:40.508908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:15:41.299931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3840607975":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1574556410":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/2981312946":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1953425269":"2023-11-27T07:16:00.270871+0000","172.21.15.83:6833/1483230064":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/367450219":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1792598028":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/2325877059":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/304865544":"2023-11-27T07:16:00.270871+0000","172.21.15.83:6832/1483230064":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/4194383148":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1164473704":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/4012256052":"2023-11-27T07:16:00.270871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 -- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2d18037810 msgr2=0x7f2d18039cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2d18037810 0x7f2d18039cd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2d340b3c40 tx=0x7f2d24009040 comp rx=0 tx=0).stop 2023-11-26T07:16:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 -- 172.21.15.83:0/3038000566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d3406df20 msgr2=0x7f2d341371a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d3406df20 0x7f2d341371a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2d30000c00 tx=0x7f2d30004750 comp rx=0 tx=0).stop 2023-11-26T07:16:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 -- 172.21.15.83:0/3038000566 shutdown_connections 2023-11-26T07:16:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2d18037810 0x7f2d18039cd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d3415ffb0 0x7f2d340b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d3406df20 0x7f2d341371a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 --2- 172.21.15.83:0/3038000566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d34145760 0x7f2d340b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 -- 172.21.15.83:0/3038000566 >> 172.21.15.83:0/3038000566 conn(0x7f2d340b1740 msgr2=0x7f2d3406cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 -- 172.21.15.83:0/3038000566 shutdown_connections 2023-11-26T07:16:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.786+0000 7f2d3c721700 1 -- 172.21.15.83:0/3038000566 wait complete. 2023-11-26T07:16:01.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:15:40.508908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:15:41.299931+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:16:01.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:16:01.829 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:01.826+0000 7fcd9b9ee200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-26T07:16:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.926+0000 7f925091e700 1 Processor -- start 2023-11-26T07:16:01.929 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.927+0000 7f925091e700 1 -- start start 2023-11-26T07:16:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.927+0000 7f925091e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f924c15bec0 0x7f924c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.927+0000 7f925091e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f924c0702e0 0x7f924c0d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.927+0000 7f925091e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f924c0cc4a0 con 0x7f924c0d2000 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f925091e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924c070770 con 0x7f924c15bec0 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f925091e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924c0d03e0 con 0x7f924c0702e0 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f924a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f924c15bec0 0x7f924c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f924a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f924c15bec0 0x7f924c06fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40056/0 (socket says 172.21.15.83:40056) 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f924a59c700 1 -- 172.21.15.83:0/1896317868 learned_addr learned my addr 172.21.15.83:0/1896317868 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f9249d9b700 1 --2- 172.21.15.83:0/1896317868 >> v2:172.21.15.203:3300/0 conn(0x7f924c0702e0 0x7f924c0d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f924a59c700 1 -- 172.21.15.83:0/1896317868 >> v2:172.21.15.203:3300/0 conn(0x7f924c0702e0 msgr2=0x7f924c0d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f924a59c700 1 --2- 172.21.15.83:0/1896317868 >> v2:172.21.15.203:3300/0 conn(0x7f924c0702e0 0x7f924c0d1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f924a59c700 1 -- 172.21.15.83:0/1896317868 >> v1:172.21.15.83:6789/0 conn(0x7f924c0d2000 legacy=0x7f924c15f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f924a59c700 1 -- 172.21.15.83:0/1896317868 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c0d2430 con 0x7f924c15bec0 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f924a59c700 1 --2- 172.21.15.83:0/1896317868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c15bec0 0x7f924c06fda0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f924c073030 tx=0x7f923c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16ece339db3f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f924959a700 1 -- 172.21.15.83:0/1896317868 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c00b870 con 0x7f924c15bec0 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f924959a700 1 -- 172.21.15.83:0/1896317868 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c00b9d0 con 0x7f924c15bec0 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.928+0000 7f924959a700 1 -- 172.21.15.83:0/1896317868 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c00bd60 con 0x7f924c15bec0 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.929+0000 7f925091e700 1 -- 172.21.15.83:0/1896317868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c15bec0 msgr2=0x7f924c06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.929+0000 7f925091e700 1 --2- 172.21.15.83:0/1896317868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c15bec0 0x7f924c06fda0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f924c073030 tx=0x7f923c009bd0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.929+0000 7f925091e700 1 -- 172.21.15.83:0/1896317868 shutdown_connections 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.929+0000 7f925091e700 1 --2- 172.21.15.83:0/1896317868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c15bec0 0x7f924c06fda0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.929+0000 7f925091e700 1 --2- 172.21.15.83:0/1896317868 >> v2:172.21.15.203:3300/0 conn(0x7f924c0702e0 0x7f924c0d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.929+0000 7f925091e700 1 -- 172.21.15.83:0/1896317868 >> 172.21.15.83:0/1896317868 conn(0x7f924c0730a0 msgr2=0x7f924c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.929+0000 7f925091e700 1 -- 172.21.15.83:0/1896317868 shutdown_connections 2023-11-26T07:16:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.929+0000 7f925091e700 1 -- 172.21.15.83:0/1896317868 wait complete. 2023-11-26T07:16:01.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.930+0000 7f925091e700 1 Processor -- start 2023-11-26T07:16:01.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.930+0000 7f925091e700 1 -- start start 2023-11-26T07:16:01.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.931+0000 7f925091e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0702e0 0x7f924c1f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.931+0000 7f925091e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f924c0d2000 0x7f924c0c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.931+0000 7f925091e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f924c15bec0 0x7f924c15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.931+0000 7f925091e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0c6b50 con 0x7f924c0702e0 2023-11-26T07:16:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.931+0000 7f925091e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0c6d40 con 0x7f924c0d2000 2023-11-26T07:16:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.931+0000 7f925091e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0c7020 con 0x7f924c15bec0 2023-11-26T07:16:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.931+0000 7f924a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0702e0 0x7f924c1f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.931+0000 7f924a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0702e0 0x7f924c1f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40062/0 (socket says 172.21.15.83:40062) 2023-11-26T07:16:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.931+0000 7f924a59c700 1 -- 172.21.15.83:0/119966994 learned_addr learned my addr 172.21.15.83:0/119966994 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f9249d9b700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f924c0d2000 0x7f924c0c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f924a59c700 1 -- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f924c15bec0 msgr2=0x7f924c15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f924ad9d700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f924c15bec0 0x7f924c15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f924a59c700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f924c15bec0 0x7f924c15eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f924a59c700 1 -- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f924c0d2000 msgr2=0x7f924c0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f924a59c700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f924c0d2000 0x7f924c0c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f924a59c700 1 -- 172.21.15.83:0/119966994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c009550 con 0x7f924c0702e0 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f9249d9b700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f924c0d2000 0x7f924c0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f924a59c700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0702e0 0x7f924c1f5c00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f924c0cf7c0 tx=0x7f923c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f923affd700 1 -- 172.21.15.83:0/119966994 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c004310 con 0x7f924c0702e0 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f923affd700 1 -- 172.21.15.83:0/119966994 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c01f070 con 0x7f924c0702e0 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f925091e700 1 -- 172.21.15.83:0/119966994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924c0c7300 con 0x7f924c0702e0 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f923affd700 1 -- 172.21.15.83:0/119966994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c0036a0 con 0x7f924c0702e0 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.932+0000 7f925091e700 1 -- 172.21.15.83:0/119966994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c0c78c0 con 0x7f924c0702e0 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.933+0000 7f923affd700 1 -- 172.21.15.83:0/119966994 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 14) v1 ==== 44694+0+0 (secure 0 0 0) 0x7f923c021030 con 0x7f924c0702e0 2023-11-26T07:16:01.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.933+0000 7f925091e700 1 -- 172.21.15.83:0/119966994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924c0c7a50 con 0x7f924c0702e0 2023-11-26T07:16:01.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.933+0000 7f923affd700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f922c037810 0x7f922c039cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:01.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.933+0000 7f923affd700 1 -- 172.21.15.83:0/119966994 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f923c04be00 con 0x7f924c0702e0 2023-11-26T07:16:01.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.933+0000 7f9249d9b700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f922c037810 0x7f922c039cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:01.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.934+0000 7f9249d9b700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f922c037810 0x7f922c039cd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9240006fd0 tx=0x7f9240008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:01.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:01.938+0000 7f923affd700 1 -- 172.21.15.83:0/119966994 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f923c023c00 con 0x7f924c0702e0 2023-11-26T07:16:02.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.133+0000 7f925091e700 1 -- 172.21.15.83:0/119966994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f924c068fb0 con 0x7f924c0702e0 2023-11-26T07:16:02.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.134+0000 7f923affd700 1 -- 172.21.15.83:0/119966994 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f923c04d020 con 0x7f924c0702e0 2023-11-26T07:16:02.136 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:16:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 -- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f922c037810 msgr2=0x7f922c039cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f922c037810 0x7f922c039cd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9240006fd0 tx=0x7f9240008040 comp rx=0 tx=0).stop 2023-11-26T07:16:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 -- 172.21.15.83:0/119966994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0702e0 msgr2=0x7f924c1f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0702e0 0x7f924c1f5c00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f924c0cf7c0 tx=0x7f923c004750 comp rx=0 tx=0).stop 2023-11-26T07:16:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 -- 172.21.15.83:0/119966994 shutdown_connections 2023-11-26T07:16:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f922c037810 0x7f922c039cd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f924c15bec0 0x7f924c15eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0702e0 0x7f924c1f5c00 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 --2- 172.21.15.83:0/119966994 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f924c0d2000 0x7f924c0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 -- 172.21.15.83:0/119966994 >> 172.21.15.83:0/119966994 conn(0x7f924c0730a0 msgr2=0x7f924c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 -- 172.21.15.83:0/119966994 shutdown_connections 2023-11-26T07:16:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.136+0000 7f925091e700 1 -- 172.21.15.83:0/119966994 wait complete. 2023-11-26T07:16:02.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:16:02.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.338+0000 7f9932836700 1 Processor -- start 2023-11-26T07:16:02.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.338+0000 7f9932836700 1 -- start start 2023-11-26T07:16:02.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.339+0000 7f9932836700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f992c06fab0 0x7f992c0ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.339+0000 7f9932836700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c0b55a0 con 0x7f992c151c30 2023-11-26T07:16:02.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.339+0000 7f9932836700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c062900 con 0x7f992c0cf0b0 2023-11-26T07:16:02.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.339+0000 7f9932836700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c0b52c0 con 0x7f992c06fab0 2023-11-26T07:16:02.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.339+0000 7f992bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f992c151c30 0x7f992c0ce9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60110/0 (socket says 172.21.15.83:60110) 2023-11-26T07:16:02.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.339+0000 7f992bfff700 1 -- 172.21.15.83:0/2501869882 learned_addr learned my addr 172.21.15.83:0/2501869882 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:02.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992b7fe700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f992c06fab0 0x7f992c0ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779821644 0 0) 0x7f992c0b55a0 con 0x7f992c151c30 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99140035b0 con 0x7f992c151c30 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3662942010 0 0) 0x7f99140035b0 con 0x7f992c151c30 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f992c06fab0 msgr2=0x7f992c0ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f992c06fab0 0x7f992c0ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 >> v1:172.21.15.203:6789/0 conn(0x7f992c0cf0b0 legacy=0x7f992c15e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c062340 con 0x7f992c151c30 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f991c001670 con 0x7f992c151c30 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 >> v1:172.21.15.83:6789/0 conn(0x7f992c151c30 legacy=0x7f992c0ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f992c06fab0 0x7f9914003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914003f70 0x7f9914004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9914004d90 0x7f9914005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c0b52c0 con 0x7f9914003f70 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991400ba70 con 0x7f9914004d90 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991400bc30 con 0x7f992c06fab0 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992b7fe700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914003f70 0x7f9914004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992bfff700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f992c06fab0 0x7f9914003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f9930dd3700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9914004d90 0x7f9914005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992b7fe700 1 -- 172.21.15.83:0/2501869882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f992c06fab0 msgr2=0x7f9914003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992b7fe700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f992c06fab0 0x7f9914003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992b7fe700 1 -- 172.21.15.83:0/2501869882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9914004d90 msgr2=0x7f9914005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992b7fe700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9914004d90 0x7f9914005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.340+0000 7f992b7fe700 1 -- 172.21.15.83:0/2501869882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99140035b0 con 0x7f9914003f70 2023-11-26T07:16:02.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.341+0000 7f992b7fe700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914003f70 0x7f9914004850 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9920009b30 tx=0x7f9914005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ff33c781446141 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:02.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.341+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920009ea0 con 0x7f9914003f70 2023-11-26T07:16:02.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.341+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992000e6c0 con 0x7f9914003f70 2023-11-26T07:16:02.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.341+0000 7f992affd700 1 -- 172.21.15.83:0/2501869882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992000ea30 con 0x7f9914003f70 2023-11-26T07:16:02.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.341+0000 7f9932836700 1 -- 172.21.15.83:0/2501869882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914003f70 msgr2=0x7f9914004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.341+0000 7f9932836700 1 --2- 172.21.15.83:0/2501869882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9914003f70 0x7f9914004850 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9920009b30 tx=0x7f9914005880 comp rx=0 tx=0).stop 2023-11-26T07:16:02.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.341+0000 7f9930dd3700 1 -- 172.21.15.83:0/2501869882 reap_dead start 2023-11-26T07:16:02.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.341+0000 7f9932836700 1 -- 172.21.15.83:0/2501869882 shutdown_connections 2023-11-26T07:16:02.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.341+0000 7f9932836700 1 -- 172.21.15.83:0/2501869882 >> 172.21.15.83:0/2501869882 conn(0x7f992c0b2050 msgr2=0x7f992c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:02.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.341+0000 7f9932836700 1 -- 172.21.15.83:0/2501869882 shutdown_connections 2023-11-26T07:16:02.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.341+0000 7f9932836700 1 -- 172.21.15.83:0/2501869882 wait complete. 2023-11-26T07:16:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.343+0000 7f9932836700 1 Processor -- start 2023-11-26T07:16:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.343+0000 7f9932836700 1 -- start start 2023-11-26T07:16:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9932836700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9914003f70 0x7f992c1423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9932836700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992c0ab630 0x7f992c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9932836700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f992c15b590 0x7f992c13fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9932836700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c15ba40 con 0x7f992c0ab630 2023-11-26T07:16:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9932836700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c13f6e0 con 0x7f992c15b590 2023-11-26T07:16:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9932836700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c13f970 con 0x7f9914003f70 2023-11-26T07:16:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f992b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992c0ab630 0x7f992c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f992b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992c0ab630 0x7f992c15b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40084/0 (socket says 172.21.15.83:40084) 2023-11-26T07:16:02.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f992b7fe700 1 -- 172.21.15.83:0/1648084437 learned_addr learned my addr 172.21.15.83:0/1648084437 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9930dd3700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f992c15b590 0x7f992c13fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f992bfff700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9914003f70 0x7f992c1423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f992b7fe700 1 -- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9914003f70 msgr2=0x7f992c1423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f992b7fe700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9914003f70 0x7f992c1423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f992b7fe700 1 -- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f992c15b590 msgr2=0x7f992c13fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f992b7fe700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f992c15b590 0x7f992c13fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f992b7fe700 1 -- 172.21.15.83:0/1648084437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c142900 con 0x7f992c0ab630 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f992b7fe700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992c0ab630 0x7f992c15b050 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f992000e4b0 tx=0x7f99200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9928ff9700 1 -- 172.21.15.83:0/1648084437 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920023070 con 0x7f992c0ab630 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9928ff9700 1 -- 172.21.15.83:0/1648084437 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9920009ea0 con 0x7f992c0ab630 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9928ff9700 1 -- 172.21.15.83:0/1648084437 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992000fe80 con 0x7f992c0ab630 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9932836700 1 -- 172.21.15.83:0/1648084437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992c0abbd0 con 0x7f992c0ab630 2023-11-26T07:16:02.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.344+0000 7f9932836700 1 -- 172.21.15.83:0/1648084437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992c0ac190 con 0x7f992c0ab630 2023-11-26T07:16:02.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.345+0000 7f9928ff9700 1 -- 172.21.15.83:0/1648084437 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 14) v1 ==== 44694+0+0 (secure 0 0 0) 0x7f9920027070 con 0x7f992c0ab630 2023-11-26T07:16:02.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.345+0000 7f9928ff9700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9918037810 0x7f9918039cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.345+0000 7f9928ff9700 1 -- 172.21.15.83:0/1648084437 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f992004da70 con 0x7f992c0ab630 2023-11-26T07:16:02.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.346+0000 7f9932836700 1 -- 172.21.15.83:0/1648084437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992c15ba40 con 0x7f992c0ab630 2023-11-26T07:16:02.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.346+0000 7f992bfff700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9918037810 0x7f9918039cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.346+0000 7f992bfff700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9918037810 0x7f9918039cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f991c001050 tx=0x7f991c000f80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:02.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.351+0000 7f9928ff9700 1 -- 172.21.15.83:0/1648084437 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f992002e080 con 0x7f992c0ab630 2023-11-26T07:16:02.422 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:02.420+0000 7fcd9b9ee200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-26T07:16:02.493 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:02.491+0000 7fcd9b9ee200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-26T07:16:02.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.549+0000 7f9932836700 1 -- 172.21.15.83:0/1648084437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f992c068d90 con 0x7f992c0ab630 2023-11-26T07:16:02.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.550+0000 7f9928ff9700 1 -- 172.21.15.83:0/1648084437 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9920017030 con 0x7f992c0ab630 2023-11-26T07:16:02.552 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:16:02.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 -- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9918037810 msgr2=0x7f9918039cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9918037810 0x7f9918039cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f991c001050 tx=0x7f991c000f80 comp rx=0 tx=0).stop 2023-11-26T07:16:02.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 -- 172.21.15.83:0/1648084437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992c0ab630 msgr2=0x7f992c15b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992c0ab630 0x7f992c15b050 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f992000e4b0 tx=0x7f99200047c0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 -- 172.21.15.83:0/1648084437 shutdown_connections 2023-11-26T07:16:02.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9918037810 0x7f9918039cd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f992c15b590 0x7f992c13fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9914003f70 0x7f992c1423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 --2- 172.21.15.83:0/1648084437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f992c0ab630 0x7f992c15b050 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 -- 172.21.15.83:0/1648084437 >> 172.21.15.83:0/1648084437 conn(0x7f992c0b2050 msgr2=0x7f992c0b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:02.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 -- 172.21.15.83:0/1648084437 shutdown_connections 2023-11-26T07:16:02.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.552+0000 7f9932836700 1 -- 172.21.15.83:0/1648084437 wait complete. 2023-11-26T07:16:02.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:16:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.691+0000 7f852d7a2700 1 Processor -- start 2023-11-26T07:16:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.691+0000 7f852d7a2700 1 -- start start 2023-11-26T07:16:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.692+0000 7f852d7a2700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8528158390 0x7f85280afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.692+0000 7f852d7a2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85280ae610 con 0x7f8528073f90 2023-11-26T07:16:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.692+0000 7f852d7a2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85280ae050 con 0x7f85280ad870 2023-11-26T07:16:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.692+0000 7f852d7a2700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85280ae330 con 0x7f8528158390 2023-11-26T07:16:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.692+0000 7f8527fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8528158390 0x7f85280afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.692+0000 7f8527fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8528158390 0x7f85280afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40090/0 (socket says 172.21.15.83:40090) 2023-11-26T07:16:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.692+0000 7f8527fff700 1 -- 172.21.15.83:0/3328145414 learned_addr learned my addr 172.21.15.83:0/3328145414 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.692+0000 7f8527fff700 1 -- 172.21.15.83:0/3328145414 >> v1:172.21.15.203:6789/0 conn(0x7f85280ad870 legacy=0x7f852806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.692+0000 7f8527fff700 1 -- 172.21.15.83:0/3328145414 >> v1:172.21.15.83:6789/0 conn(0x7f8528073f90 legacy=0x7f8528074370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.692+0000 7f8527fff700 1 -- 172.21.15.83:0/3328145414 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852806b170 con 0x7f8528158390 2023-11-26T07:16:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.692+0000 7f8527fff700 1 --2- 172.21.15.83:0/3328145414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8528158390 0x7f85280afe50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f85180098f0 tx=0x7f8518009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599d3f31decc3cf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.693+0000 7f8526ffd700 1 -- 172.21.15.83:0/3328145414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8518004230 con 0x7f8528158390 2023-11-26T07:16:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.693+0000 7f8526ffd700 1 -- 172.21.15.83:0/3328145414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8518004390 con 0x7f8528158390 2023-11-26T07:16:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.693+0000 7f8526ffd700 1 -- 172.21.15.83:0/3328145414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85180036a0 con 0x7f8528158390 2023-11-26T07:16:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.693+0000 7f852d7a2700 1 -- 172.21.15.83:0/3328145414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8528158390 msgr2=0x7f85280afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.693+0000 7f852d7a2700 1 --2- 172.21.15.83:0/3328145414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8528158390 0x7f85280afe50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f85180098f0 tx=0x7f8518009c00 comp rx=0 tx=0).stop 2023-11-26T07:16:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.693+0000 7f852d7a2700 1 -- 172.21.15.83:0/3328145414 shutdown_connections 2023-11-26T07:16:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.693+0000 7f852d7a2700 1 --2- 172.21.15.83:0/3328145414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8528158390 0x7f85280afe50 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.693+0000 7f852d7a2700 1 -- 172.21.15.83:0/3328145414 >> 172.21.15.83:0/3328145414 conn(0x7f85280d0d40 msgr2=0x7f85280d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:02.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.693+0000 7f852d7a2700 1 -- 172.21.15.83:0/3328145414 shutdown_connections 2023-11-26T07:16:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.693+0000 7f852d7a2700 1 -- 172.21.15.83:0/3328145414 wait complete. 2023-11-26T07:16:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.695+0000 7f852d7a2700 1 Processor -- start 2023-11-26T07:16:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.695+0000 7f852d7a2700 1 -- start start 2023-11-26T07:16:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.695+0000 7f852d7a2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8528073f90 0x7f85281474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.695+0000 7f852d7a2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85280ad870 0x7f85281e8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.695+0000 7f852d7a2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8528158390 0x7f8528160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.695+0000 7f852d7a2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85281e7520 con 0x7f8528158390 2023-11-26T07:16:02.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.695+0000 7f852d7a2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85281e6910 con 0x7f8528073f90 2023-11-26T07:16:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.695+0000 7f852d7a2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85281e6bf0 con 0x7f85280ad870 2023-11-26T07:16:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f852cfa1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8528158390 0x7f8528160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f852cfa1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8528158390 0x7f8528160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40098/0 (socket says 172.21.15.83:40098) 2023-11-26T07:16:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f852cfa1700 1 -- 172.21.15.83:0/4147331362 learned_addr learned my addr 172.21.15.83:0/4147331362 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f8527fff700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8528073f90 0x7f85281474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f85277fe700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85280ad870 0x7f85281e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f852cfa1700 1 -- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85280ad870 msgr2=0x7f85281e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f852cfa1700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85280ad870 0x7f85281e8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f852cfa1700 1 -- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8528073f90 msgr2=0x7f85281474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f852cfa1700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8528073f90 0x7f85281474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f852cfa1700 1 -- 172.21.15.83:0/4147331362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8518009550 con 0x7f8528158390 2023-11-26T07:16:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f85277fe700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85280ad870 0x7f85281e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f8527fff700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8528073f90 0x7f85281474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:02.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f852cfa1700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8528158390 0x7f8528160120 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f852000cc50 tx=0x7f852000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f8524ff9700 1 -- 172.21.15.83:0/4147331362 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000ad80 con 0x7f8528158390 2023-11-26T07:16:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f8524ff9700 1 -- 172.21.15.83:0/4147331362 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852000ce50 con 0x7f8528158390 2023-11-26T07:16:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f852d7a2700 1 -- 172.21.15.83:0/4147331362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85281e6ed0 con 0x7f8528158390 2023-11-26T07:16:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f8524ff9700 1 -- 172.21.15.83:0/4147331362 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8520003c40 con 0x7f8528158390 2023-11-26T07:16:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.696+0000 7f852d7a2700 1 -- 172.21.15.83:0/4147331362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85281e7e60 con 0x7f8528158390 2023-11-26T07:16:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.697+0000 7f8524ff9700 1 -- 172.21.15.83:0/4147331362 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 15) v1 ==== 44902+0+0 (secure 0 0 0) 0x7f85200050d0 con 0x7f8528158390 2023-11-26T07:16:02.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.697+0000 7f852d7a2700 1 -- 172.21.15.83:0/4147331362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8528068480 con 0x7f8528158390 2023-11-26T07:16:02.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.697+0000 7f8524ff9700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8514037a50 0x7f8514039f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:02.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.698+0000 7f8524ff9700 1 -- 172.21.15.83:0/4147331362 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f852004ad70 con 0x7f8528158390 2023-11-26T07:16:02.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.698+0000 7f8527fff700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8514037a50 0x7f8514039f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:02.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.698+0000 7f8527fff700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8514037a50 0x7f8514039f10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8518009130 tx=0x7f8518008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:02.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.702+0000 7f8524ff9700 1 -- 172.21.15.83:0/4147331362 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8520003da0 con 0x7f8528158390 2023-11-26T07:16:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.897+0000 7f852d7a2700 1 -- 172.21.15.83:0/4147331362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f85280adce0 con 0x7f8528158390 2023-11-26T07:16:02.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.897+0000 7f8524ff9700 1 -- 172.21.15.83:0/4147331362 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f85280adce0 con 0x7f8528158390 2023-11-26T07:16:02.900 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:16:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.899+0000 7f852d7a2700 1 -- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8514037a50 msgr2=0x7f8514039f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.899+0000 7f852d7a2700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8514037a50 0x7f8514039f10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8518009130 tx=0x7f8518008030 comp rx=0 tx=0).stop 2023-11-26T07:16:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.899+0000 7f852d7a2700 1 -- 172.21.15.83:0/4147331362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8528158390 msgr2=0x7f8528160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.899+0000 7f852d7a2700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8528158390 0x7f8528160120 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f852000cc50 tx=0x7f852000a340 comp rx=0 tx=0).stop 2023-11-26T07:16:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.899+0000 7f852d7a2700 1 -- 172.21.15.83:0/4147331362 shutdown_connections 2023-11-26T07:16:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.900+0000 7f852d7a2700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8514037a50 0x7f8514039f10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.900+0000 7f852d7a2700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8528158390 0x7f8528160120 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.900+0000 7f852d7a2700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8528073f90 0x7f85281474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.900+0000 7f852d7a2700 1 --2- 172.21.15.83:0/4147331362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85280ad870 0x7f85281e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.900+0000 7f852d7a2700 1 -- 172.21.15.83:0/4147331362 >> 172.21.15.83:0/4147331362 conn(0x7f85280d0d40 msgr2=0x7f85280b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.900+0000 7f852d7a2700 1 -- 172.21.15.83:0/4147331362 shutdown_connections 2023-11-26T07:16:02.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:02.900+0000 7f852d7a2700 1 -- 172.21.15.83:0/4147331362 wait complete. 2023-11-26T07:16:02.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:03.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.038+0000 7f0ef9c15700 1 Processor -- start 2023-11-26T07:16:03.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.038+0000 7f0ef9c15700 1 -- start start 2023-11-26T07:16:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.039+0000 7f0ef9c15700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ef415f180 0x7f0ef40cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.039+0000 7f0ef9c15700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0ef40cfd40 0x7f0ef40b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.039+0000 7f0ef9c15700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef40b4290 con 0x7f0ef413e000 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.039+0000 7f0ef9c15700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef413e570 con 0x7f0ef40cfd40 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.039+0000 7f0ef9c15700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef40d05d0 con 0x7f0ef415f180 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.039+0000 7f0ef37fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0ef413e000 0x7f0ef415ea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60142/0 (socket says 172.21.15.83:60142) 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.039+0000 7f0ef37fe700 1 -- 172.21.15.83:0/1985985433 learned_addr learned my addr 172.21.15.83:0/1985985433 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.039+0000 7f0ef3fff700 1 --2- 172.21.15.83:0/1985985433 >> v2:172.21.15.83:3300/0 conn(0x7f0ef40cfd40 0x7f0ef40b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.039+0000 7f0ef2ffd700 1 --2- 172.21.15.83:0/1985985433 >> v2:172.21.15.203:3300/0 conn(0x7f0ef415f180 0x7f0ef40cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.040+0000 7f0ef3fff700 1 -- 172.21.15.83:0/1985985433 >> v2:172.21.15.203:3300/0 conn(0x7f0ef415f180 msgr2=0x7f0ef40cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.040+0000 7f0ef3fff700 1 --2- 172.21.15.83:0/1985985433 >> v2:172.21.15.203:3300/0 conn(0x7f0ef415f180 0x7f0ef40cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.040+0000 7f0ef3fff700 1 -- 172.21.15.83:0/1985985433 >> v1:172.21.15.83:6789/0 conn(0x7f0ef413e000 legacy=0x7f0ef415ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.040+0000 7f0ef3fff700 1 -- 172.21.15.83:0/1985985433 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef40d08b0 con 0x7f0ef40cfd40 2023-11-26T07:16:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.040+0000 7f0ef3fff700 1 --2- 172.21.15.83:0/1985985433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef40cfd40 0x7f0ef40b4720 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0ef415e9d0 tx=0x7f0eec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385efe516799f52e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.040+0000 7f0ef27fc700 1 -- 172.21.15.83:0/1985985433 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00e070 con 0x7f0ef40cfd40 2023-11-26T07:16:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.040+0000 7f0ef27fc700 1 -- 172.21.15.83:0/1985985433 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec00bb20 con 0x7f0ef40cfd40 2023-11-26T07:16:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.040+0000 7f0ef27fc700 1 -- 172.21.15.83:0/1985985433 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec012430 con 0x7f0ef40cfd40 2023-11-26T07:16:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.041+0000 7f0ef9c15700 1 -- 172.21.15.83:0/1985985433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef40cfd40 msgr2=0x7f0ef40b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.041+0000 7f0ef9c15700 1 --2- 172.21.15.83:0/1985985433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef40cfd40 0x7f0ef40b4720 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0ef415e9d0 tx=0x7f0eec00b330 comp rx=0 tx=0).stop 2023-11-26T07:16:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.041+0000 7f0ef9c15700 1 -- 172.21.15.83:0/1985985433 shutdown_connections 2023-11-26T07:16:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.041+0000 7f0ef9c15700 1 --2- 172.21.15.83:0/1985985433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef40cfd40 0x7f0ef40b4720 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.041+0000 7f0ef9c15700 1 --2- 172.21.15.83:0/1985985433 >> v2:172.21.15.203:3300/0 conn(0x7f0ef415f180 0x7f0ef40cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.041+0000 7f0ef9c15700 1 -- 172.21.15.83:0/1985985433 >> 172.21.15.83:0/1985985433 conn(0x7f0ef4072990 msgr2=0x7f0ef406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.041+0000 7f0ef9c15700 1 -- 172.21.15.83:0/1985985433 shutdown_connections 2023-11-26T07:16:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.041+0000 7f0ef9c15700 1 -- 172.21.15.83:0/1985985433 wait complete. 2023-11-26T07:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.043+0000 7f0ef9c15700 1 Processor -- start 2023-11-26T07:16:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.043+0000 7f0ef9c15700 1 -- start start 2023-11-26T07:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.044+0000 7f0ef9c15700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef40cfd40 0x7f0ef41415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.044+0000 7f0ef9c15700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ef413e000 0x7f0ef4144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.044+0000 7f0ef9c15700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef415f180 0x7f0ef41f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.044+0000 7f0ef9c15700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4141b10 con 0x7f0ef415f180 2023-11-26T07:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.044+0000 7f0ef9c15700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef41446b0 con 0x7f0ef40cfd40 2023-11-26T07:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.044+0000 7f0ef9c15700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4141d20 con 0x7f0ef413e000 2023-11-26T07:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.044+0000 7f0ef3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef415f180 0x7f0ef41f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.044+0000 7f0ef3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef415f180 0x7f0ef41f3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40112/0 (socket says 172.21.15.83:40112) 2023-11-26T07:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.044+0000 7f0ef3fff700 1 -- 172.21.15.83:0/3575450827 learned_addr learned my addr 172.21.15.83:0/3575450827 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0ef37fe700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef40cfd40 0x7f0ef41415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0ef2ffd700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ef413e000 0x7f0ef4144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0ef3fff700 1 -- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ef413e000 msgr2=0x7f0ef4144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0ef3fff700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ef413e000 0x7f0ef4144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0ef3fff700 1 -- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef40cfd40 msgr2=0x7f0ef41415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0ef3fff700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef40cfd40 0x7f0ef41415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0ef3fff700 1 -- 172.21.15.83:0/3575450827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec009a70 con 0x7f0ef415f180 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0ef37fe700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef40cfd40 0x7f0ef41415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0ef3fff700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef415f180 0x7f0ef41f3dd0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0ef406c770 tx=0x7f0eec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0edbfff700 1 -- 172.21.15.83:0/3575450827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec012c20 con 0x7f0ef415f180 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0edbfff700 1 -- 172.21.15.83:0/3575450827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec022070 con 0x7f0ef415f180 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0ef9c15700 1 -- 172.21.15.83:0/3575450827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef4142000 con 0x7f0ef415f180 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0edbfff700 1 -- 172.21.15.83:0/3575450827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0098f0 con 0x7f0ef415f180 2023-11-26T07:16:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.045+0000 7f0ef9c15700 1 -- 172.21.15.83:0/3575450827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef41425c0 con 0x7f0ef415f180 2023-11-26T07:16:03.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.046+0000 7f0edbfff700 1 -- 172.21.15.83:0/3575450827 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 15) v1 ==== 44902+0+0 (secure 0 0 0) 0x7f0eec01e610 con 0x7f0ef415f180 2023-11-26T07:16:03.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.046+0000 7f0ef9c15700 1 -- 172.21.15.83:0/3575450827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef4067f60 con 0x7f0ef415f180 2023-11-26T07:16:03.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.046+0000 7f0edbfff700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0ee0037a50 0x7f0ee0039f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.046+0000 7f0edbfff700 1 -- 172.21.15.83:0/3575450827 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f0eec024070 con 0x7f0ef415f180 2023-11-26T07:16:03.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.046+0000 7f0ef37fe700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0ee0037a50 0x7f0ee0039f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.047+0000 7f0ef37fe700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0ee0037a50 0x7f0ee0039f10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4007fd0 tx=0x7f0ee4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:03.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.051+0000 7f0edbfff700 1 -- 172.21.15.83:0/3575450827 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0eec01a380 con 0x7f0ef415f180 2023-11-26T07:16:03.149 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:03.147+0000 7fcd9b9ee200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:16:03.228 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:03.226+0000 7fcd9b9ee200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-26T07:16:03.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.294+0000 7f0ef9c15700 1 -- 172.21.15.83:0/3575450827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ef40b0890 con 0x7f0ef415f180 2023-11-26T07:16:03.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.294+0000 7f0edbfff700 1 -- 172.21.15.83:0/3575450827 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f0eec01ae60 con 0x7f0ef415f180 2023-11-26T07:16:03.297 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:03.297 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5420,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2163151849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2163151849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2163151849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1539108478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1539108478},{"type":"v1","addr":"172.21.15.203:6835","nonce":1539108478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1387613510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1387613510},{"type":"v1","addr":"172.21.15.203:6834","nonce":1387613510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:15:42.339653+0000","modified":"2023-11-26T07:15:45.361821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1809465302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1809465302},{"type":"v1","addr":"172.21.15.83:6836","nonce":1809465302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:16:03.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.296+0000 7f0ef9c15700 1 -- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0ee0037a50 msgr2=0x7f0ee0039f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.296+0000 7f0ef9c15700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0ee0037a50 0x7f0ee0039f10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4007fd0 tx=0x7f0ee4009040 comp rx=0 tx=0).stop 2023-11-26T07:16:03.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.296+0000 7f0ef9c15700 1 -- 172.21.15.83:0/3575450827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef415f180 msgr2=0x7f0ef41f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.296+0000 7f0ef9c15700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef415f180 0x7f0ef41f3dd0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0ef406c770 tx=0x7f0eec004ad0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.296+0000 7f0ef9c15700 1 -- 172.21.15.83:0/3575450827 shutdown_connections 2023-11-26T07:16:03.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.296+0000 7f0ef9c15700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0ee0037a50 0x7f0ee0039f10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.296+0000 7f0ef9c15700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef415f180 0x7f0ef41f3dd0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.296+0000 7f0ef9c15700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ef40cfd40 0x7f0ef41415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.296+0000 7f0ef9c15700 1 --2- 172.21.15.83:0/3575450827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ef413e000 0x7f0ef4144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.296+0000 7f0ef9c15700 1 -- 172.21.15.83:0/3575450827 >> 172.21.15.83:0/3575450827 conn(0x7f0ef4072990 msgr2=0x7f0ef406fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:03.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.297+0000 7f0ef9c15700 1 -- 172.21.15.83:0/3575450827 shutdown_connections 2023-11-26T07:16:03.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.297+0000 7f0ef9c15700 1 -- 172.21.15.83:0/3575450827 wait complete. 2023-11-26T07:16:03.301 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 47 2023-11-26T07:16:03.311 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:03.379 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:03.377+0000 7fcd9b9ee200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:16:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.431+0000 7fcf46559700 1 Processor -- start 2023-11-26T07:16:03.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.431+0000 7fcf46559700 1 -- start start 2023-11-26T07:16:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf46559700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcf400b0fb0 0x7fcf400cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf46559700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf400b1390 0x7fcf4006b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf46559700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf400b1c60 con 0x7fcf400d0280 2023-11-26T07:16:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf46559700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf400d0720 con 0x7fcf400b0fb0 2023-11-26T07:16:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf46559700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400d0a00 con 0x7fcf400b1390 2023-11-26T07:16:03.436 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf3ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcf400b0fb0 0x7fcf400cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf3ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcf400b0fb0 0x7fcf400cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40116/0 (socket says 172.21.15.83:40116) 2023-11-26T07:16:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf3ffff700 1 -- 172.21.15.83:0/2181618186 learned_addr learned my addr 172.21.15.83:0/2181618186 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf3ffff700 1 -- 172.21.15.83:0/2181618186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf400b1390 msgr2=0x7fcf4006b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf3f7fe700 1 --2- 172.21.15.83:0/2181618186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf400b1390 0x7fcf4006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf3ffff700 1 --2- 172.21.15.83:0/2181618186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf400b1390 0x7fcf4006b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf3ffff700 1 -- 172.21.15.83:0/2181618186 >> v1:172.21.15.203:6789/0 conn(0x7fcf400d0280 legacy=0x7fcf4015d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf3ffff700 1 -- 172.21.15.83:0/2181618186 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf400d0ce0 con 0x7fcf400b0fb0 2023-11-26T07:16:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf3ffff700 1 --2- 172.21.15.83:0/2181618186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf400b0fb0 0x7fcf400cfd40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcf300098f0 tx=0x7fcf30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d54a46c8ad9de0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.432+0000 7fcf3effd700 1 -- 172.21.15.83:0/2181618186 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30004230 con 0x7fcf400b0fb0 2023-11-26T07:16:03.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.433+0000 7fcf3effd700 1 -- 172.21.15.83:0/2181618186 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf30004390 con 0x7fcf400b0fb0 2023-11-26T07:16:03.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.433+0000 7fcf3effd700 1 -- 172.21.15.83:0/2181618186 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf300036a0 con 0x7fcf400b0fb0 2023-11-26T07:16:03.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.433+0000 7fcf46559700 1 -- 172.21.15.83:0/2181618186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf400b0fb0 msgr2=0x7fcf400cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.433+0000 7fcf46559700 1 --2- 172.21.15.83:0/2181618186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf400b0fb0 0x7fcf400cfd40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcf300098f0 tx=0x7fcf30009c00 comp rx=0 tx=0).stop 2023-11-26T07:16:03.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.433+0000 7fcf46559700 1 -- 172.21.15.83:0/2181618186 shutdown_connections 2023-11-26T07:16:03.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.433+0000 7fcf46559700 1 --2- 172.21.15.83:0/2181618186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf400b0fb0 0x7fcf400cfd40 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.433+0000 7fcf46559700 1 --2- 172.21.15.83:0/2181618186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf400b1390 0x7fcf4006b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.433+0000 7fcf46559700 1 -- 172.21.15.83:0/2181618186 >> 172.21.15.83:0/2181618186 conn(0x7fcf4009aa00 msgr2=0x7fcf4009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:03.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.433+0000 7fcf46559700 1 -- 172.21.15.83:0/2181618186 shutdown_connections 2023-11-26T07:16:03.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.434+0000 7fcf46559700 1 -- 172.21.15.83:0/2181618186 wait complete. 2023-11-26T07:16:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.435+0000 7fcf46559700 1 Processor -- start 2023-11-26T07:16:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.435+0000 7fcf46559700 1 -- start start 2023-11-26T07:16:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf46559700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf400b0fb0 0x7fcf400c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf46559700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf400b1390 0x7fcf400c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf46559700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf400d0280 0x7fcf400cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf46559700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400c7e00 con 0x7fcf400b1390 2023-11-26T07:16:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf46559700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400c7180 con 0x7fcf400b0fb0 2023-11-26T07:16:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf46559700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400c7460 con 0x7fcf400d0280 2023-11-26T07:16:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf400b1390 0x7fcf400c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf400b1390 0x7fcf400c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40124/0 (socket says 172.21.15.83:40124) 2023-11-26T07:16:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf3f7fe700 1 -- 172.21.15.83:0/1777822877 learned_addr learned my addr 172.21.15.83:0/1777822877 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf44af6700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf400d0280 0x7fcf400cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf3ffff700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf400b0fb0 0x7fcf400c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf3f7fe700 1 -- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf400d0280 msgr2=0x7fcf400cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf3f7fe700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf400d0280 0x7fcf400cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf3f7fe700 1 -- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf400b0fb0 msgr2=0x7fcf400c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf3f7fe700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf400b0fb0 0x7fcf400c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf3f7fe700 1 -- 172.21.15.83:0/1777822877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf30009550 con 0x7fcf400b1390 2023-11-26T07:16:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf3ffff700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf400b0fb0 0x7fcf400c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.436+0000 7fcf3f7fe700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf400b1390 0x7fcf400c6830 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400c7f0 tx=0x7fcf3400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.437+0000 7fcf3cff9700 1 -- 172.21.15.83:0/1777822877 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011800 con 0x7fcf400b1390 2023-11-26T07:16:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.437+0000 7fcf3cff9700 1 -- 172.21.15.83:0/1777822877 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf34011960 con 0x7fcf400b1390 2023-11-26T07:16:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.437+0000 7fcf46559700 1 -- 172.21.15.83:0/1777822877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf400c7740 con 0x7fcf400b1390 2023-11-26T07:16:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.437+0000 7fcf3cff9700 1 -- 172.21.15.83:0/1777822877 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011cf0 con 0x7fcf400b1390 2023-11-26T07:16:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.437+0000 7fcf46559700 1 -- 172.21.15.83:0/1777822877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf400c6d70 con 0x7fcf400b1390 2023-11-26T07:16:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.437+0000 7fcf3cff9700 1 -- 172.21.15.83:0/1777822877 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 15) v1 ==== 44902+0+0 (secure 0 0 0) 0x7fcf34018360 con 0x7fcf400b1390 2023-11-26T07:16:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.438+0000 7fcf3cff9700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcf2c037a50 0x7fcf2c039f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.438+0000 7fcf3cff9700 1 -- 172.21.15.83:0/1777822877 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fcf3404ad70 con 0x7fcf400b1390 2023-11-26T07:16:03.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.438+0000 7fcf46559700 1 -- 172.21.15.83:0/1777822877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf40068480 con 0x7fcf400b1390 2023-11-26T07:16:03.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.438+0000 7fcf3ffff700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcf2c037a50 0x7fcf2c039f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.438+0000 7fcf3ffff700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcf2c037a50 0x7fcf2c039f10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009c40 tx=0x7fcf30008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:03.445 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.443+0000 7fcf3cff9700 1 -- 172.21.15.83:0/1777822877 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf34018610 con 0x7fcf400b1390 2023-11-26T07:16:03.613 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:03.611+0000 7fcd9b9ee200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-26T07:16:03.645 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.643+0000 7fcf46559700 1 -- 172.21.15.83:0/1777822877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf400c7f40 con 0x7fcf400b1390 2023-11-26T07:16:03.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.645+0000 7fcf3cff9700 1 -- 172.21.15.83:0/1777822877 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17365 (secure 0 0 0) 0x7fcf400c7f40 con 0x7fcf400b1390 2023-11-26T07:16:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:03.648 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":71,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:00.270920+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:15:40.508908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:15:41.299931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3840607975":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1574556410":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/2981312946":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1953425269":"2023-11-27T07:16:00.270871+0000","172.21.15.83:6833/1483230064":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/367450219":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1792598028":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/2325877059":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/304865544":"2023-11-27T07:16:00.270871+0000","172.21.15.83:6832/1483230064":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/4194383148":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1164473704":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/4012256052":"2023-11-27T07:16:00.270871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:03.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 -- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcf2c037a50 msgr2=0x7fcf2c039f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcf2c037a50 0x7fcf2c039f10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009c40 tx=0x7fcf30008030 comp rx=0 tx=0).stop 2023-11-26T07:16:03.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 -- 172.21.15.83:0/1777822877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf400b1390 msgr2=0x7fcf400c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf400b1390 0x7fcf400c6830 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400c7f0 tx=0x7fcf3400cbb0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 -- 172.21.15.83:0/1777822877 shutdown_connections 2023-11-26T07:16:03.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcf2c037a50 0x7fcf2c039f10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf400d0280 0x7fcf400cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf400b0fb0 0x7fcf400c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 --2- 172.21.15.83:0/1777822877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf400b1390 0x7fcf400c6830 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 -- 172.21.15.83:0/1777822877 >> 172.21.15.83:0/1777822877 conn(0x7fcf4009aa00 msgr2=0x7fcf4006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:03.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 -- 172.21.15.83:0/1777822877 shutdown_connections 2023-11-26T07:16:03.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.647+0000 7fcf46559700 1 -- 172.21.15.83:0/1777822877 wait complete. 2023-11-26T07:16:03.662 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:16:03.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:03.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.787+0000 7fdb92e32700 1 Processor -- start 2023-11-26T07:16:03.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.787+0000 7fdb92e32700 1 -- start start 2023-11-26T07:16:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.788+0000 7fdb92e32700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c0d0870 0x7fdb8c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.791 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.788+0000 7fdb92e32700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdb8c0b6ef0 0x7fdb8c0b7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.788+0000 7fdb92e32700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8c0d1210 con 0x7fdb8c15e1e0 2023-11-26T07:16:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.788+0000 7fdb92e32700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8c0d3150 con 0x7fdb8c0b6ef0 2023-11-26T07:16:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.788+0000 7fdb92e32700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c15e5c0 con 0x7fdb8c0d0870 2023-11-26T07:16:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.788+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c0d0870 0x7fdb8c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.788+0000 7fdb913cf700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdb8c0b6ef0 0x7fdb8c0b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.788+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c0d0870 0x7fdb8c0d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:50590/0 (socket says 172.21.15.83:50590) 2023-11-26T07:16:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.788+0000 7fdb913cf700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdb8c0b6ef0 0x7fdb8c0b7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:39586/0 (socket says 172.21.15.83:39586) 2023-11-26T07:16:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.788+0000 7fdb8bfff700 1 -- 172.21.15.83:0/3049263901 learned_addr learned my addr 172.21.15.83:0/3049263901 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.788+0000 7fdb8b7fe700 1 -- 172.21.15.83:0/3049263901 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529684144 0 0) 0x7fdb8c0d1210 con 0x7fdb8c15e1e0 2023-11-26T07:16:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.789+0000 7fdb8bfff700 1 -- 172.21.15.83:0/3049263901 >> v2:172.21.15.203:3300/0 conn(0x7fdb8c0b6ef0 msgr2=0x7fdb8c0b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:03.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.789+0000 7fdb8bfff700 1 --2- 172.21.15.83:0/3049263901 >> v2:172.21.15.203:3300/0 conn(0x7fdb8c0b6ef0 0x7fdb8c0b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.789+0000 7fdb8bfff700 1 -- 172.21.15.83:0/3049263901 >> v1:172.21.15.203:6789/0 conn(0x7fdb8c15e1e0 legacy=0x7fdb8c0d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.789+0000 7fdb8bfff700 1 -- 172.21.15.83:0/3049263901 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c0d20b0 con 0x7fdb8c0d0870 2023-11-26T07:16:03.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.789+0000 7fdb8bfff700 1 --2- 172.21.15.83:0/3049263901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c0d0870 0x7fdb8c0d0cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb780098f0 tx=0x7fdb78009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52f6433cff990254 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:03.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.789+0000 7fdb8b7fe700 1 -- 172.21.15.83:0/3049263901 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78011830 con 0x7fdb8c0d0870 2023-11-26T07:16:03.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.789+0000 7fdb8b7fe700 1 -- 172.21.15.83:0/3049263901 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb78011990 con 0x7fdb8c0d0870 2023-11-26T07:16:03.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.789+0000 7fdb8b7fe700 1 -- 172.21.15.83:0/3049263901 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78011d20 con 0x7fdb8c0d0870 2023-11-26T07:16:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.789+0000 7fdb92e32700 1 -- 172.21.15.83:0/3049263901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c0d0870 msgr2=0x7fdb8c0d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.789+0000 7fdb92e32700 1 --2- 172.21.15.83:0/3049263901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c0d0870 0x7fdb8c0d0cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb780098f0 tx=0x7fdb78009c00 comp rx=0 tx=0).stop 2023-11-26T07:16:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.790+0000 7fdb92e32700 1 -- 172.21.15.83:0/3049263901 shutdown_connections 2023-11-26T07:16:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.790+0000 7fdb92e32700 1 --2- 172.21.15.83:0/3049263901 >> v2:172.21.15.203:3300/0 conn(0x7fdb8c0b6ef0 0x7fdb8c0b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.790+0000 7fdb92e32700 1 --2- 172.21.15.83:0/3049263901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c0d0870 0x7fdb8c0d0cd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.790+0000 7fdb92e32700 1 -- 172.21.15.83:0/3049263901 >> 172.21.15.83:0/3049263901 conn(0x7fdb8c0b5b50 msgr2=0x7fdb8c0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.790+0000 7fdb92e32700 1 -- 172.21.15.83:0/3049263901 shutdown_connections 2023-11-26T07:16:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.790+0000 7fdb92e32700 1 -- 172.21.15.83:0/3049263901 wait complete. 2023-11-26T07:16:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.791+0000 7fdb92e32700 1 Processor -- start 2023-11-26T07:16:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.791+0000 7fdb92e32700 1 -- start start 2023-11-26T07:16:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb92e32700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb8c0b6ef0 0x7fdb8c1510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb92e32700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb8c0d0870 0x7fdb8c160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb92e32700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c15e1e0 0x7fdb8c0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb92e32700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c15c370 con 0x7fdb8c0d0870 2023-11-26T07:16:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb92e32700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c15bad0 con 0x7fdb8c0b6ef0 2023-11-26T07:16:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb92e32700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c15bdb0 con 0x7fdb8c15e1e0 2023-11-26T07:16:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb8c0d0870 0x7fdb8c160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb8bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb8c0d0870 0x7fdb8c160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40138/0 (socket says 172.21.15.83:40138) 2023-11-26T07:16:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb8bfff700 1 -- 172.21.15.83:0/2326909442 learned_addr learned my addr 172.21.15.83:0/2326909442 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb90bce700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb8c0b6ef0 0x7fdb8c1510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb913cf700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c15e1e0 0x7fdb8c0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb8bfff700 1 -- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c15e1e0 msgr2=0x7fdb8c0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb8bfff700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c15e1e0 0x7fdb8c0caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb8bfff700 1 -- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb8c0b6ef0 msgr2=0x7fdb8c1510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb8bfff700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb8c0b6ef0 0x7fdb8c1510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb8bfff700 1 -- 172.21.15.83:0/2326909442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb78009550 con 0x7fdb8c0d0870 2023-11-26T07:16:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.792+0000 7fdb8bfff700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb8c0d0870 0x7fdb8c160690 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009c40 tx=0x7fdb78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.793+0000 7fdb897fa700 1 -- 172.21.15.83:0/2326909442 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78004550 con 0x7fdb8c0d0870 2023-11-26T07:16:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.793+0000 7fdb897fa700 1 -- 172.21.15.83:0/2326909442 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7801f070 con 0x7fdb8c0d0870 2023-11-26T07:16:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.793+0000 7fdb92e32700 1 -- 172.21.15.83:0/2326909442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c15c090 con 0x7fdb8c0d0870 2023-11-26T07:16:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.793+0000 7fdb897fa700 1 -- 172.21.15.83:0/2326909442 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb780038e0 con 0x7fdb8c0d0870 2023-11-26T07:16:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.793+0000 7fdb92e32700 1 -- 172.21.15.83:0/2326909442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c151950 con 0x7fdb8c0d0870 2023-11-26T07:16:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.793+0000 7fdb897fa700 1 -- 172.21.15.83:0/2326909442 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 15) v1 ==== 44902+0+0 (secure 0 0 0) 0x7fdb78003a40 con 0x7fdb8c0d0870 2023-11-26T07:16:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.794+0000 7fdb897fa700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdb70037a50 0x7fdb70039f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.794+0000 7fdb897fa700 1 -- 172.21.15.83:0/2326909442 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fdb78021070 con 0x7fdb8c0d0870 2023-11-26T07:16:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.794+0000 7fdb90bce700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdb70037a50 0x7fdb70039f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.794+0000 7fdb76ffd700 1 -- 172.21.15.83:0/2326909442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb640052e0 con 0x7fdb8c0d0870 2023-11-26T07:16:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.794+0000 7fdb90bce700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdb70037a50 0x7fdb70039f10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdb80000f60 tx=0x7fdb80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:03.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:03.799+0000 7fdb897fa700 1 -- 172.21.15.83:0/2326909442 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb7801a7b0 con 0x7fdb8c0d0870 2023-11-26T07:16:03.831 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:03.829+0000 7fcd9b9ee200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-26T07:16:03.960 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:03.957+0000 7fcd9b9ee200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-26T07:16:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.039+0000 7fdb76ffd700 1 -- 172.21.15.83:0/2326909442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb64005860 con 0x7fdb8c0d0870 2023-11-26T07:16:04.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.040+0000 7fdb897fa700 1 -- 172.21.15.83:0/2326909442 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fdb7802b030 con 0x7fdb8c0d0870 2023-11-26T07:16:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:04.043 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5420,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2163151849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2163151849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2163151849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1539108478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1539108478},{"type":"v1","addr":"172.21.15.203:6835","nonce":1539108478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1387613510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1387613510},{"type":"v1","addr":"172.21.15.203:6834","nonce":1387613510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:15:42.339653+0000","modified":"2023-11-26T07:15:45.361821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1809465302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1809465302},{"type":"v1","addr":"172.21.15.83:6836","nonce":1809465302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:16:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 -- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdb70037a50 msgr2=0x7fdb70039f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdb70037a50 0x7fdb70039f10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdb80000f60 tx=0x7fdb80009040 comp rx=0 tx=0).stop 2023-11-26T07:16:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 -- 172.21.15.83:0/2326909442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb8c0d0870 msgr2=0x7fdb8c160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb8c0d0870 0x7fdb8c160690 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009c40 tx=0x7fdb78004750 comp rx=0 tx=0).stop 2023-11-26T07:16:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 -- 172.21.15.83:0/2326909442 shutdown_connections 2023-11-26T07:16:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdb70037a50 0x7fdb70039f10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb8c15e1e0 0x7fdb8c0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb8c0b6ef0 0x7fdb8c1510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 --2- 172.21.15.83:0/2326909442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb8c0d0870 0x7fdb8c160690 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 -- 172.21.15.83:0/2326909442 >> 172.21.15.83:0/2326909442 conn(0x7fdb8c0b5b50 msgr2=0x7fdb8c15d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 -- 172.21.15.83:0/2326909442 shutdown_connections 2023-11-26T07:16:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.042+0000 7fdb76ffd700 1 -- 172.21.15.83:0/2326909442 wait complete. 2023-11-26T07:16:04.046 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 47 2023-11-26T07:16:04.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:04.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.178+0000 7f77422e5700 1 Processor -- start 2023-11-26T07:16:04.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.178+0000 7f77422e5700 1 -- start start 2023-11-26T07:16:04.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.179+0000 7f77422e5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f773c15bd60 0x7f773c0d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.179+0000 7f77422e5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f773c0cf2b0 0x7f773c0c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.179+0000 7f77422e5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773c0cc780 con 0x7f773c15c140 2023-11-26T07:16:04.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.179+0000 7f77422e5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773c0cc9b0 con 0x7f773c15bd60 2023-11-26T07:16:04.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.179+0000 7f77422e5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0ccc90 con 0x7f773c0cf2b0 2023-11-26T07:16:04.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.179+0000 7f773bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f773c15bd60 0x7f773c0d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.179+0000 7f773bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f773c15bd60 0x7f773c0d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40144/0 (socket says 172.21.15.83:40144) 2023-11-26T07:16:04.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.179+0000 7f773bfff700 1 -- 172.21.15.83:0/2517829462 learned_addr learned my addr 172.21.15.83:0/2517829462 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:04.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.179+0000 7f7740882700 1 --2- 172.21.15.83:0/2517829462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f773c0cf2b0 0x7f773c0c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.179+0000 7f773bfff700 1 -- 172.21.15.83:0/2517829462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f773c0cf2b0 msgr2=0x7f773c0c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:04.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.179+0000 7f773bfff700 1 --2- 172.21.15.83:0/2517829462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f773c0cf2b0 0x7f773c0c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f773bfff700 1 -- 172.21.15.83:0/2517829462 >> v1:172.21.15.83:6789/0 conn(0x7f773c15c140 legacy=0x7f773c0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f773bfff700 1 -- 172.21.15.83:0/2517829462 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c0ccf70 con 0x7f773c15bd60 2023-11-26T07:16:04.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f773bfff700 1 --2- 172.21.15.83:0/2517829462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f773c15bd60 0x7f773c0d1ff0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f772c0098f0 tx=0x7f772c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e7286d8882635b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:04.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f773affd700 1 -- 172.21.15.83:0/2517829462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c00b8a0 con 0x7f773c15bd60 2023-11-26T07:16:04.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f773affd700 1 -- 172.21.15.83:0/2517829462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c00ba00 con 0x7f773c15bd60 2023-11-26T07:16:04.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f773affd700 1 -- 172.21.15.83:0/2517829462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c00bd90 con 0x7f773c15bd60 2023-11-26T07:16:04.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f77422e5700 1 -- 172.21.15.83:0/2517829462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f773c15bd60 msgr2=0x7f773c0d1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f77422e5700 1 --2- 172.21.15.83:0/2517829462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f773c15bd60 0x7f773c0d1ff0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f772c0098f0 tx=0x7f772c009c00 comp rx=0 tx=0).stop 2023-11-26T07:16:04.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f77422e5700 1 -- 172.21.15.83:0/2517829462 shutdown_connections 2023-11-26T07:16:04.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f77422e5700 1 --2- 172.21.15.83:0/2517829462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f773c0cf2b0 0x7f773c0c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f77422e5700 1 --2- 172.21.15.83:0/2517829462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f773c15bd60 0x7f773c0d1ff0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f77422e5700 1 -- 172.21.15.83:0/2517829462 >> 172.21.15.83:0/2517829462 conn(0x7f773c073130 msgr2=0x7f773c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:04.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.180+0000 7f77422e5700 1 -- 172.21.15.83:0/2517829462 shutdown_connections 2023-11-26T07:16:04.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.181+0000 7f77422e5700 1 -- 172.21.15.83:0/2517829462 wait complete. 2023-11-26T07:16:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.182+0000 7f77422e5700 1 Processor -- start 2023-11-26T07:16:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.182+0000 7f77422e5700 1 -- start start 2023-11-26T07:16:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.182+0000 7f77422e5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f773c0cf2b0 0x7f773c0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f77422e5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f773c15bd60 0x7f773c155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f77422e5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f773c15c140 0x7f773c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f77422e5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0caf10 con 0x7f773c15bd60 2023-11-26T07:16:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f77422e5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0ca300 con 0x7f773c15c140 2023-11-26T07:16:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f77422e5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0ca5e0 con 0x7f773c0cf2b0 2023-11-26T07:16:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f773b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f773c15bd60 0x7f773c155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f773b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f773c15bd60 0x7f773c155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40152/0 (socket says 172.21.15.83:40152) 2023-11-26T07:16:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f773b7fe700 1 -- 172.21.15.83:0/2910799262 learned_addr learned my addr 172.21.15.83:0/2910799262 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f773bfff700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f773c0cf2b0 0x7f773c0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f7740882700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f773c15c140 0x7f773c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f773b7fe700 1 -- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f773c0cf2b0 msgr2=0x7f773c0cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f773b7fe700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f773c0cf2b0 0x7f773c0cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f773b7fe700 1 -- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f773c15c140 msgr2=0x7f773c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f773b7fe700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f773c15c140 0x7f773c1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f773b7fe700 1 -- 172.21.15.83:0/2910799262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c009550 con 0x7f773c15bd60 2023-11-26T07:16:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f773b7fe700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f773c15bd60 0x7f773c155f90 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f773000c450 tx=0x7f773000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f7738ff9700 1 -- 172.21.15.83:0/2910799262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77300052d0 con 0x7f773c15bd60 2023-11-26T07:16:04.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.183+0000 7f7738ff9700 1 -- 172.21.15.83:0/2910799262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7730005430 con 0x7f773c15bd60 2023-11-26T07:16:04.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.184+0000 7f77422e5700 1 -- 172.21.15.83:0/2910799262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c0ca8c0 con 0x7f773c15bd60 2023-11-26T07:16:04.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.184+0000 7f7738ff9700 1 -- 172.21.15.83:0/2910799262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77300038c0 con 0x7f773c15bd60 2023-11-26T07:16:04.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.184+0000 7f77422e5700 1 -- 172.21.15.83:0/2910799262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c1564d0 con 0x7f773c15bd60 2023-11-26T07:16:04.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.184+0000 7f77422e5700 1 -- 172.21.15.83:0/2910799262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773c0cb0c0 con 0x7f773c15bd60 2023-11-26T07:16:04.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.185+0000 7f7738ff9700 1 -- 172.21.15.83:0/2910799262 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 15) v1 ==== 44902+0+0 (secure 0 0 0) 0x7f7730003a20 con 0x7f773c15bd60 2023-11-26T07:16:04.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.185+0000 7f7738ff9700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7728037a50 0x7f7728039f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.186+0000 7f7738ff9700 1 -- 172.21.15.83:0/2910799262 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f773004a200 con 0x7f773c15bd60 2023-11-26T07:16:04.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.186+0000 7f773bfff700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7728037a50 0x7f7728039f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.186+0000 7f773bfff700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7728037a50 0x7f7728039f10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f772c009c40 tx=0x7f772c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:04.191 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:04.189+0000 7fcd9b9ee200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-26T07:16:04.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.193+0000 7f7738ff9700 1 -- 172.21.15.83:0/2910799262 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f773000ce60 con 0x7f773c15bd60 2023-11-26T07:16:04.344 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:04.342+0000 7fcd9b9ee200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-26T07:16:04.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.437+0000 7f77422e5700 1 -- 172.21.15.83:0/2910799262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f773c0cb0c0 con 0x7f773c15bd60 2023-11-26T07:16:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.437+0000 7f7738ff9700 1 -- 172.21.15.83:0/2910799262 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f773c0cb0c0 con 0x7f773c15bd60 2023-11-26T07:16:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:04.440 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5420,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2163151849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2163151849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2163151849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1539108478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1539108478},{"type":"v1","addr":"172.21.15.203:6835","nonce":1539108478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1387613510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1387613510},{"type":"v1","addr":"172.21.15.203:6834","nonce":1387613510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:15:42.339653+0000","modified":"2023-11-26T07:15:45.361821+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6836/1809465302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":1809465302},{"type":"v1","addr":"172.21.15.83:6836","nonce":1809465302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:16:04.442 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 -- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7728037a50 msgr2=0x7f7728039f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7728037a50 0x7f7728039f10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f772c009c40 tx=0x7f772c008030 comp rx=0 tx=0).stop 2023-11-26T07:16:04.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 -- 172.21.15.83:0/2910799262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f773c15bd60 msgr2=0x7f773c155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f773c15bd60 0x7f773c155f90 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f773000c450 tx=0x7f773000c760 comp rx=0 tx=0).stop 2023-11-26T07:16:04.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 -- 172.21.15.83:0/2910799262 shutdown_connections 2023-11-26T07:16:04.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7728037a50 0x7f7728039f10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f773c15c140 0x7f773c1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f773c0cf2b0 0x7f773c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 --2- 172.21.15.83:0/2910799262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f773c15bd60 0x7f773c155f90 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 -- 172.21.15.83:0/2910799262 >> 172.21.15.83:0/2910799262 conn(0x7f773c073130 msgr2=0x7f773c0c7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:04.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 -- 172.21.15.83:0/2910799262 shutdown_connections 2023-11-26T07:16:04.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.440+0000 7f77422e5700 1 -- 172.21.15.83:0/2910799262 wait complete. 2023-11-26T07:16:04.444 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 47 2023-11-26T07:16:04.455 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:04.463 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:04.460+0000 7fcd9b9ee200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-26T07:16:04.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.579+0000 7f83fc403700 1 Processor -- start 2023-11-26T07:16:04.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.579+0000 7f83fc403700 1 -- start start 2023-11-26T07:16:04.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.580+0000 7f83fc403700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83f40af4d0 0x7f83f40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.580+0000 7f83fc403700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f40ae430 con 0x7f83f406c710 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.580+0000 7f83fc403700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f40ae710 con 0x7f83f415e080 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.580+0000 7f83fc403700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f40af0b0 con 0x7f83f40af4d0 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.580+0000 7f83f999e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f83f406c710 0x7f83f406e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60198/0 (socket says 172.21.15.83:60198) 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.580+0000 7f83f999e700 1 -- 172.21.15.83:0/2773853187 learned_addr learned my addr 172.21.15.83:0/2773853187 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.581+0000 7f83fa9a0700 1 --2- 172.21.15.83:0/2773853187 >> v2:172.21.15.203:3300/0 conn(0x7f83f40af4d0 0x7f83f40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.581+0000 7f83f919d700 1 -- 172.21.15.83:0/2773853187 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366715210 0 0) 0x7f83f40ae430 con 0x7f83f406c710 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.581+0000 7f83f919d700 1 -- 172.21.15.83:0/2773853187 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e00035b0 con 0x7f83f406c710 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.581+0000 7f83fa9a0700 1 -- 172.21.15.83:0/2773853187 >> v1:172.21.15.203:6789/0 conn(0x7f83f415e080 legacy=0x7f83f406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.581+0000 7f83fa9a0700 1 -- 172.21.15.83:0/2773853187 >> v1:172.21.15.83:6789/0 conn(0x7f83f406c710 legacy=0x7f83f406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.581+0000 7f83fa9a0700 1 -- 172.21.15.83:0/2773853187 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f40ae9f0 con 0x7f83f40af4d0 2023-11-26T07:16:04.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.581+0000 7f83fa9a0700 1 --2- 172.21.15.83:0/2773853187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f40af4d0 0x7f83f40b5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009d90 tx=0x7f83ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da3fc577387eab67 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:04.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.581+0000 7f83f919d700 1 -- 172.21.15.83:0/2773853187 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00e070 con 0x7f83f40af4d0 2023-11-26T07:16:04.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.581+0000 7f83f919d700 1 -- 172.21.15.83:0/2773853187 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec00bb20 con 0x7f83f40af4d0 2023-11-26T07:16:04.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.581+0000 7f83f919d700 1 -- 172.21.15.83:0/2773853187 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec012430 con 0x7f83f40af4d0 2023-11-26T07:16:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.582+0000 7f83fc403700 1 -- 172.21.15.83:0/2773853187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f40af4d0 msgr2=0x7f83f40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.582+0000 7f83fc403700 1 --2- 172.21.15.83:0/2773853187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f40af4d0 0x7f83f40b5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009d90 tx=0x7f83ec00b330 comp rx=0 tx=0).stop 2023-11-26T07:16:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.582+0000 7f83fc403700 1 -- 172.21.15.83:0/2773853187 shutdown_connections 2023-11-26T07:16:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.582+0000 7f83fc403700 1 --2- 172.21.15.83:0/2773853187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f40af4d0 0x7f83f40b5050 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.582+0000 7f83fc403700 1 -- 172.21.15.83:0/2773853187 >> 172.21.15.83:0/2773853187 conn(0x7f83f4071b90 msgr2=0x7f83f4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.582+0000 7f83fc403700 1 -- 172.21.15.83:0/2773853187 shutdown_connections 2023-11-26T07:16:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.582+0000 7f83fc403700 1 -- 172.21.15.83:0/2773853187 wait complete. 2023-11-26T07:16:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.583+0000 7f83fc403700 1 Processor -- start 2023-11-26T07:16:04.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.583+0000 7f83fc403700 1 -- start start 2023-11-26T07:16:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.584+0000 7f83fc403700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f406c710 0x7f83f4147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.584+0000 7f83fc403700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83f40af4d0 0x7f83f41578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.584+0000 7f83fc403700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f415e080 0x7f83f40b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.584+0000 7f83fc403700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41582d0 con 0x7f83f406c710 2023-11-26T07:16:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.584+0000 7f83fc403700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41585b0 con 0x7f83f415e080 2023-11-26T07:16:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.584+0000 7f83fc403700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f4158890 con 0x7f83f40af4d0 2023-11-26T07:16:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.584+0000 7f83fa19f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f406c710 0x7f83f4147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.584+0000 7f83fa19f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f406c710 0x7f83f4147480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40164/0 (socket says 172.21.15.83:40164) 2023-11-26T07:16:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.584+0000 7f83fa19f700 1 -- 172.21.15.83:0/1612276877 learned_addr learned my addr 172.21.15.83:0/1612276877 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:04.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83f999e700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83f40af4d0 0x7f83f41578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83fa9a0700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f415e080 0x7f83f40b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83fa19f700 1 -- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83f40af4d0 msgr2=0x7f83f41578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83fa19f700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83f40af4d0 0x7f83f41578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83fa19f700 1 -- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f415e080 msgr2=0x7f83f40b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83fa19f700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f415e080 0x7f83f40b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83fa19f700 1 -- 172.21.15.83:0/1612276877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec009a70 con 0x7f83f406c710 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83f999e700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83f40af4d0 0x7f83f41578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83fa19f700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f406c710 0x7f83f4147480 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c450 tx=0x7f83e800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83e6ffd700 1 -- 172.21.15.83:0/1612276877 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80052d0 con 0x7f83f406c710 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83fc403700 1 -- 172.21.15.83:0/1612276877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f4158b70 con 0x7f83f406c710 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83e6ffd700 1 -- 172.21.15.83:0/1612276877 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e8005430 con 0x7f83f406c710 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83e6ffd700 1 -- 172.21.15.83:0/1612276877 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80038c0 con 0x7f83f406c710 2023-11-26T07:16:04.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.585+0000 7f83fc403700 1 -- 172.21.15.83:0/1612276877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f41479c0 con 0x7f83f406c710 2023-11-26T07:16:04.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.586+0000 7f83fc403700 1 -- 172.21.15.83:0/1612276877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f40753f0 con 0x7f83f406c710 2023-11-26T07:16:04.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.586+0000 7f83e6ffd700 1 -- 172.21.15.83:0/1612276877 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 15) v1 ==== 44902+0+0 (secure 0 0 0) 0x7f83e8003a20 con 0x7f83f406c710 2023-11-26T07:16:04.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.586+0000 7f83e6ffd700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f83d8037a50 0x7f83d8039f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.587+0000 7f83e6ffd700 1 -- 172.21.15.83:0/1612276877 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f83e804a0b0 con 0x7f83f406c710 2023-11-26T07:16:04.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.587+0000 7f83f999e700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f83d8037a50 0x7f83d8039f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.587+0000 7f83f999e700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f83d8037a50 0x7f83d8039f10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f83f4160ee0 tx=0x7f83f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:04.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.594+0000 7f83e6ffd700 1 -- 172.21.15.83:0/1612276877 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83e8018260 con 0x7f83f406c710 2023-11-26T07:16:04.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.606+0000 7f83e6ffd700 1 -- 172.21.15.83:0/1612276877 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 16) v1 ==== 44958+0+0 (secure 0 0 0) 0x7f83e801bda0 con 0x7f83f406c710 2023-11-26T07:16:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.796+0000 7f83fc403700 1 -- 172.21.15.83:0/1612276877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83f4067f60 con 0x7f83f406c710 2023-11-26T07:16:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.797+0000 7f83e6ffd700 1 -- 172.21.15.83:0/1612276877 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17365 (secure 0 0 0) 0x7f83e801add0 con 0x7f83f406c710 2023-11-26T07:16:04.799 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:04.800 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":71,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:00.270920+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:15:40.508908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:15:41.299931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3840607975":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1574556410":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/2981312946":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1953425269":"2023-11-27T07:16:00.270871+0000","172.21.15.83:6833/1483230064":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/367450219":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1792598028":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/2325877059":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/304865544":"2023-11-27T07:16:00.270871+0000","172.21.15.83:6832/1483230064":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/4194383148":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1164473704":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/4012256052":"2023-11-27T07:16:00.270871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:04.801 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:04.799+0000 7fcd9b9ee200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-26T07:16:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 -- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f83d8037a50 msgr2=0x7f83d8039f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f83d8037a50 0x7f83d8039f10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f83f4160ee0 tx=0x7f83f0009040 comp rx=0 tx=0).stop 2023-11-26T07:16:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 -- 172.21.15.83:0/1612276877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f406c710 msgr2=0x7f83f4147480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f406c710 0x7f83f4147480 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c450 tx=0x7f83e800c810 comp rx=0 tx=0).stop 2023-11-26T07:16:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 -- 172.21.15.83:0/1612276877 shutdown_connections 2023-11-26T07:16:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f83d8037a50 0x7f83d8039f10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83f415e080 0x7f83f40b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f406c710 0x7f83f4147480 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 --2- 172.21.15.83:0/1612276877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83f40af4d0 0x7f83f41578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 -- 172.21.15.83:0/1612276877 >> 172.21.15.83:0/1612276877 conn(0x7f83f4071b90 msgr2=0x7f83f406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 -- 172.21.15.83:0/1612276877 shutdown_connections 2023-11-26T07:16:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.799+0000 7f83fc403700 1 -- 172.21.15.83:0/1612276877 wait complete. 2023-11-26T07:16:04.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:16:04.893 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:04.891+0000 7fcd9b9ee200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-26T07:16:04.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.977+0000 7fc5ea474700 1 Processor -- start 2023-11-26T07:16:04.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.977+0000 7fc5ea474700 1 -- start start 2023-11-26T07:16:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.977+0000 7fc5ea474700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e415e1e0 0x7fc5e40d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.977+0000 7fc5ea474700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5e40d0f80 0x7fc5e40b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.977+0000 7fc5ea474700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e40d20b0 con 0x7fc5e40d3a00 2023-11-26T07:16:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.977+0000 7fc5ea474700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e40d3150 con 0x7fc5e40d0f80 2023-11-26T07:16:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.977+0000 7fc5ea474700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e415e5c0 con 0x7fc5e415e1e0 2023-11-26T07:16:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e8a11700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5e40d0f80 0x7fc5e40b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e8a11700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5e40d0f80 0x7fc5e40b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40176/0 (socket says 172.21.15.83:40176) 2023-11-26T07:16:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e8a11700 1 -- 172.21.15.83:0/831499703 learned_addr learned my addr 172.21.15.83:0/831499703 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e37fe700 1 --1- 172.21.15.83:0/831499703 >> v1:172.21.15.83:6789/0 conn(0x7fc5e40d3a00 0x7fc5e40d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60212/0 (socket says 172.21.15.83:60212) 2023-11-26T07:16:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e3fff700 1 --2- 172.21.15.83:0/831499703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e415e1e0 0x7fc5e40d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e8a11700 1 -- 172.21.15.83:0/831499703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e415e1e0 msgr2=0x7fc5e40d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e8a11700 1 --2- 172.21.15.83:0/831499703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e415e1e0 0x7fc5e40d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e8a11700 1 -- 172.21.15.83:0/831499703 >> v1:172.21.15.83:6789/0 conn(0x7fc5e40d3a00 legacy=0x7fc5e40d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e8a11700 1 -- 172.21.15.83:0/831499703 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e40d2390 con 0x7fc5e40d0f80 2023-11-26T07:16:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e3fff700 1 --2- 172.21.15.83:0/831499703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e415e1e0 0x7fc5e40d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e8a11700 1 --2- 172.21.15.83:0/831499703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e40d0f80 0x7fc5e40b6ef0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009e80 tx=0x7fc5dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8b5301c9c15cc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e2ffd700 1 -- 172.21.15.83:0/831499703 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00e070 con 0x7fc5e40d0f80 2023-11-26T07:16:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e2ffd700 1 -- 172.21.15.83:0/831499703 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5dc00b980 con 0x7fc5e40d0f80 2023-11-26T07:16:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.978+0000 7fc5e2ffd700 1 -- 172.21.15.83:0/831499703 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00bd30 con 0x7fc5e40d0f80 2023-11-26T07:16:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.979+0000 7fc5ea474700 1 -- 172.21.15.83:0/831499703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e40d0f80 msgr2=0x7fc5e40b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.979+0000 7fc5ea474700 1 --2- 172.21.15.83:0/831499703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e40d0f80 0x7fc5e40b6ef0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009e80 tx=0x7fc5dc00b330 comp rx=0 tx=0).stop 2023-11-26T07:16:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.979+0000 7fc5ea474700 1 -- 172.21.15.83:0/831499703 shutdown_connections 2023-11-26T07:16:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.979+0000 7fc5ea474700 1 --2- 172.21.15.83:0/831499703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e40d0f80 0x7fc5e40b6ef0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.979+0000 7fc5ea474700 1 --2- 172.21.15.83:0/831499703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e415e1e0 0x7fc5e40d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.979+0000 7fc5ea474700 1 -- 172.21.15.83:0/831499703 >> 172.21.15.83:0/831499703 conn(0x7fc5e40b5b50 msgr2=0x7fc5e40b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.979+0000 7fc5ea474700 1 -- 172.21.15.83:0/831499703 shutdown_connections 2023-11-26T07:16:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.979+0000 7fc5ea474700 1 -- 172.21.15.83:0/831499703 wait complete. 2023-11-26T07:16:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.980+0000 7fc5ea474700 1 Processor -- start 2023-11-26T07:16:04.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.981+0000 7fc5ea474700 1 -- start start 2023-11-26T07:16:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.981+0000 7fc5ea474700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e40d0f80 0x7fc5e4151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.981+0000 7fc5ea474700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e40d3a00 0x7fc5e40b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.981+0000 7fc5ea474700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e415e1e0 0x7fc5e40b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.981+0000 7fc5ea474700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e41518a0 con 0x7fc5e415e1e0 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.981+0000 7fc5ea474700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40cdf10 con 0x7fc5e40d3a00 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.981+0000 7fc5ea474700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40cd390 con 0x7fc5e40d0f80 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.981+0000 7fc5e8a11700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e415e1e0 0x7fc5e40b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.981+0000 7fc5e8a11700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e415e1e0 0x7fc5e40b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40182/0 (socket says 172.21.15.83:40182) 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e8a11700 1 -- 172.21.15.83:0/1611418291 learned_addr learned my addr 172.21.15.83:0/1611418291 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e3fff700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e40d0f80 0x7fc5e4151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e37fe700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e40d3a00 0x7fc5e40b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e8a11700 1 -- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e40d0f80 msgr2=0x7fc5e4151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e8a11700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e40d0f80 0x7fc5e4151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e8a11700 1 -- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e40d3a00 msgr2=0x7fc5e40b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e8a11700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e40d3a00 0x7fc5e40b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e8a11700 1 -- 172.21.15.83:0/1611418291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc009b30 con 0x7fc5e415e1e0 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e37fe700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e40d3a00 0x7fc5e40b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e8a11700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e415e1e0 0x7fc5e40b4d50 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc012000 tx=0x7fc5dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e0ff9700 1 -- 172.21.15.83:0/1611418291 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc004960 con 0x7fc5e415e1e0 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e0ff9700 1 -- 172.21.15.83:0/1611418291 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5dc0099b0 con 0x7fc5e415e1e0 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5ea474700 1 -- 172.21.15.83:0/1611418291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e40cd670 con 0x7fc5e415e1e0 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5e0ff9700 1 -- 172.21.15.83:0/1611418291 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc01c680 con 0x7fc5e415e1e0 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.982+0000 7fc5ea474700 1 -- 172.21.15.83:0/1611418291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e40cdc30 con 0x7fc5e415e1e0 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.983+0000 7fc5e0ff9700 1 -- 172.21.15.83:0/1611418291 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 44958+0+0 (secure 0 0 0) 0x7fc5dc01c7e0 con 0x7fc5e415e1e0 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.983+0000 7fc5e0ff9700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc5d0037ae0 0x7fc5d0039fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:04.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.983+0000 7fc5e0ff9700 1 -- 172.21.15.83:0/1611418291 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fc5dc04c560 con 0x7fc5e415e1e0 2023-11-26T07:16:04.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.983+0000 7fc5ea474700 1 -- 172.21.15.83:0/1611418291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e40751e0 con 0x7fc5e415e1e0 2023-11-26T07:16:04.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.983+0000 7fc5e3fff700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc5d0037ae0 0x7fc5d0039fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:04.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.984+0000 7fc5e3fff700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc5d0037ae0 0x7fc5d0039fa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40098f0 tx=0x7fc5d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:04.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:04.988+0000 7fc5e0ff9700 1 -- 172.21.15.83:0/1611418291 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc5dc0123f0 con 0x7fc5e415e1e0 2023-11-26T07:16:05.062 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:05.059+0000 7fcd9b9ee200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-26T07:16:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.229+0000 7fc5ea474700 1 -- 172.21.15.83:0/1611418291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc5e40d0a10 con 0x7fc5e415e1e0 2023-11-26T07:16:05.232 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-26T07:16:05.230+0000 7fcd25823700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:16:05.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.231+0000 7fc5e0ff9700 1 -- 172.21.15.83:0/1611418291 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc5dc024820 con 0x7fc5e415e1e0 2023-11-26T07:16:05.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.234+0000 7fc5ea474700 1 -- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc5d0037ae0 msgr2=0x7fc5d0039fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:05.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.234+0000 7fc5ea474700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc5d0037ae0 0x7fc5d0039fa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40098f0 tx=0x7fc5d4008040 comp rx=0 tx=0).stop 2023-11-26T07:16:05.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.234+0000 7fc5ea474700 1 -- 172.21.15.83:0/1611418291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e415e1e0 msgr2=0x7fc5e40b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:05.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.234+0000 7fc5ea474700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e415e1e0 0x7fc5e40b4d50 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc012000 tx=0x7fc5dc004ad0 comp rx=0 tx=0).stop 2023-11-26T07:16:05.240 INFO:tasks.ceph.mds.c.smithi083.stderr: -8282> 2023-11-26T07:15:47.932+0000 7ff41af97700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:16:05.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.239+0000 7fc5ea474700 1 -- 172.21.15.83:0/1611418291 shutdown_connections 2023-11-26T07:16:05.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.239+0000 7fc5ea474700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc5d0037ae0 0x7fc5d0039fa0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:05.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.239+0000 7fc5ea474700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e415e1e0 0x7fc5e40b4d50 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:05.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.239+0000 7fc5ea474700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5e40d0f80 0x7fc5e4151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:05.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.239+0000 7fc5ea474700 1 --2- 172.21.15.83:0/1611418291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5e40d3a00 0x7fc5e40b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:05.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.239+0000 7fc5ea474700 1 -- 172.21.15.83:0/1611418291 >> 172.21.15.83:0/1611418291 conn(0x7fc5e40b5b50 msgr2=0x7fc5e415d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:05.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.239+0000 7fc5ea474700 1 -- 172.21.15.83:0/1611418291 shutdown_connections 2023-11-26T07:16:05.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.239+0000 7fc5ea474700 1 -- 172.21.15.83:0/1611418291 wait complete. 2023-11-26T07:16:05.242 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:16:05.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:16:05.274 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-26T07:16:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.396+0000 7fae31e3a700 1 Processor -- start 2023-11-26T07:16:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.396+0000 7fae31e3a700 1 -- start start 2023-11-26T07:16:05.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.397+0000 7fae31e3a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae2c0b4370 0x7fae2c15b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:05.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.397+0000 7fae31e3a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c0ae5e0 con 0x7fae2c15b370 2023-11-26T07:16:05.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.397+0000 7fae31e3a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c15be30 con 0x7fae2c1587f0 2023-11-26T07:16:05.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.397+0000 7fae31e3a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2c15c110 con 0x7fae2c0b4370 2023-11-26T07:16:05.400 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.397+0000 7fae2bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae2c0b4370 0x7fae2c15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.397+0000 7fae2bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fae2c0b4370 0x7fae2c15b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40208/0 (socket says 172.21.15.83:40208) 2023-11-26T07:16:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.397+0000 7fae2bfff700 1 -- 172.21.15.83:0/1920991065 learned_addr learned my addr 172.21.15.83:0/1920991065 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae2bfff700 1 -- 172.21.15.83:0/1920991065 >> v1:172.21.15.203:6789/0 conn(0x7fae2c1587f0 legacy=0x7fae2c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae2bfff700 1 -- 172.21.15.83:0/1920991065 >> v1:172.21.15.83:6789/0 conn(0x7fae2c15b370 legacy=0x7fae2c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae2bfff700 1 -- 172.21.15.83:0/1920991065 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c0ae860 con 0x7fae2c0b4370 2023-11-26T07:16:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae2bfff700 1 --2- 172.21.15.83:0/1920991065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2c0b4370 0x7fae2c15b800 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fae24009e80 tx=0x7fae2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab53f991fefe9722 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae2a7fc700 1 -- 172.21.15.83:0/1920991065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24003c20 con 0x7fae2c0b4370 2023-11-26T07:16:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae2a7fc700 1 -- 172.21.15.83:0/1920991065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae24003d80 con 0x7fae2c0b4370 2023-11-26T07:16:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae2a7fc700 1 -- 172.21.15.83:0/1920991065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2400bb80 con 0x7fae2c0b4370 2023-11-26T07:16:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae31e3a700 1 -- 172.21.15.83:0/1920991065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2c0b4370 msgr2=0x7fae2c15b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae31e3a700 1 --2- 172.21.15.83:0/1920991065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2c0b4370 0x7fae2c15b800 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fae24009e80 tx=0x7fae2400b330 comp rx=0 tx=0).stop 2023-11-26T07:16:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae31e3a700 1 -- 172.21.15.83:0/1920991065 shutdown_connections 2023-11-26T07:16:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae31e3a700 1 --2- 172.21.15.83:0/1920991065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2c0b4370 0x7fae2c15b800 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae31e3a700 1 -- 172.21.15.83:0/1920991065 >> 172.21.15.83:0/1920991065 conn(0x7fae2c0d0bf0 msgr2=0x7fae2c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae31e3a700 1 -- 172.21.15.83:0/1920991065 shutdown_connections 2023-11-26T07:16:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.398+0000 7fae31e3a700 1 -- 172.21.15.83:0/1920991065 wait complete. 2023-11-26T07:16:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.400+0000 7fae31e3a700 1 Processor -- start 2023-11-26T07:16:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.400+0000 7fae31e3a700 1 -- start start 2023-11-26T07:16:05.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.400+0000 7fae31e3a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae2c0b4370 0x7fae2c0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:05.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae31e3a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2c1587f0 0x7fae2c0afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae31e3a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae2c15b370 0x7fae2c148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae31e3a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0b0380 con 0x7fae2c1587f0 2023-11-26T07:16:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae31e3a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0b05b0 con 0x7fae2c15b370 2023-11-26T07:16:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae31e3a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0b0890 con 0x7fae2c0b4370 2023-11-26T07:16:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2c1587f0 0x7fae2c0afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2c1587f0 0x7fae2c0afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40212/0 (socket says 172.21.15.83:40212) 2023-11-26T07:16:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2affd700 1 -- 172.21.15.83:0/2344825324 learned_addr learned my addr 172.21.15.83:0/2344825324 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2b7fe700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae2c0b4370 0x7fae2c0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2bfff700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae2c15b370 0x7fae2c148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2affd700 1 -- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae2c0b4370 msgr2=0x7fae2c0b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2affd700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae2c0b4370 0x7fae2c0b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2affd700 1 -- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae2c15b370 msgr2=0x7fae2c148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2affd700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae2c15b370 0x7fae2c148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2affd700 1 -- 172.21.15.83:0/2344825324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae24009b30 con 0x7fae2c1587f0 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2bfff700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae2c15b370 0x7fae2c148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae2affd700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2c1587f0 0x7fae2c0afe40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c430 tx=0x7fae2000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae13fff700 1 -- 172.21.15.83:0/2344825324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae200052d0 con 0x7fae2c1587f0 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae13fff700 1 -- 172.21.15.83:0/2344825324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae20005450 con 0x7fae2c1587f0 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae31e3a700 1 -- 172.21.15.83:0/2344825324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2c0b0b70 con 0x7fae2c1587f0 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae13fff700 1 -- 172.21.15.83:0/2344825324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20006aa0 con 0x7fae2c1587f0 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.401+0000 7fae31e3a700 1 -- 172.21.15.83:0/2344825324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2c0b1130 con 0x7fae2c1587f0 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.402+0000 7fae13fff700 1 -- 172.21.15.83:0/2344825324 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 44958+0+0 (secure 0 0 0) 0x7fae20003680 con 0x7fae2c1587f0 2023-11-26T07:16:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.402+0000 7fae31e3a700 1 -- 172.21.15.83:0/2344825324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2c0b12c0 con 0x7fae2c1587f0 2023-11-26T07:16:05.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.402+0000 7fae13fff700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae18037ae0 0x7fae18039fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:05.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.402+0000 7fae13fff700 1 -- 172.21.15.83:0/2344825324 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fae2004a060 con 0x7fae2c1587f0 2023-11-26T07:16:05.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.403+0000 7fae2b7fe700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae18037ae0 0x7fae18039fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:05.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.403+0000 7fae2b7fe700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae18037ae0 0x7fae18039fa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fae1c003e60 tx=0x7fae1c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:05.409 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.407+0000 7fae13fff700 1 -- 172.21.15.83:0/2344825324 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae20019b60 con 0x7fae2c1587f0 2023-11-26T07:16:05.610 INFO:tasks.ceph.mgr.y.smithi083.stderr:2023-11-26T07:16:05.608+0000 7fcd9b9ee200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-26T07:16:05.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:05.656+0000 7fae31e3a700 1 -- 172.21.15.83:0/2344825324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fae2c09c3d0 con 0x7fae2c1587f0 2023-11-26T07:16:06.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.233+0000 7fae13fff700 1 -- 172.21.15.83:0/2344825324 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7fae20019780 con 0x7fae2c1587f0 2023-11-26T07:16:06.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 -- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae18037ae0 msgr2=0x7fae18039fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae18037ae0 0x7fae18039fa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fae1c003e60 tx=0x7fae1c009040 comp rx=0 tx=0).stop 2023-11-26T07:16:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 -- 172.21.15.83:0/2344825324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2c1587f0 msgr2=0x7fae2c0afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2c1587f0 0x7fae2c0afe40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c430 tx=0x7fae2000c7f0 comp rx=0 tx=0).stop 2023-11-26T07:16:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 -- 172.21.15.83:0/2344825324 shutdown_connections 2023-11-26T07:16:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae18037ae0 0x7fae18039fa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae2c15b370 0x7fae2c148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae2c0b4370 0x7fae2c0b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 --2- 172.21.15.83:0/2344825324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae2c1587f0 0x7fae2c0afe40 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 -- 172.21.15.83:0/2344825324 >> 172.21.15.83:0/2344825324 conn(0x7fae2c0d0bf0 msgr2=0x7fae2c06b990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:06.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 -- 172.21.15.83:0/2344825324 shutdown_connections 2023-11-26T07:16:06.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.236+0000 7fae31e3a700 1 -- 172.21.15.83:0/2344825324 wait complete. 2023-11-26T07:16:06.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:16:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.376+0000 7f77deb5e700 1 Processor -- start 2023-11-26T07:16:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.376+0000 7f77deb5e700 1 -- start start 2023-11-26T07:16:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.377+0000 7f77deb5e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77d806f480 0x7f77d806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.377+0000 7f77deb5e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d806b200 con 0x7f77d806e830 2023-11-26T07:16:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.377+0000 7f77deb5e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d806b4e0 con 0x7f77d8144600 2023-11-26T07:16:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.377+0000 7f77deb5e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d806aff0 con 0x7f77d806f480 2023-11-26T07:16:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.377+0000 7f77d7fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77d806f480 0x7f77d806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.377+0000 7f77d7fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77d806f480 0x7f77d806c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40222/0 (socket says 172.21.15.83:40222) 2023-11-26T07:16:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.377+0000 7f77d7fff700 1 -- 172.21.15.83:0/2925536817 learned_addr learned my addr 172.21.15.83:0/2925536817 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77d7fff700 1 -- 172.21.15.83:0/2925536817 >> v1:172.21.15.203:6789/0 conn(0x7f77d8144600 legacy=0x7f77d80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77d7fff700 1 -- 172.21.15.83:0/2925536817 >> v1:172.21.15.83:6789/0 conn(0x7f77d806e830 legacy=0x7f77d80cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77d7fff700 1 -- 172.21.15.83:0/2925536817 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d806b7c0 con 0x7f77d806f480 2023-11-26T07:16:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77d7fff700 1 --2- 172.21.15.83:0/2925536817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d806f480 0x7f77d806c2a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f77c40098f0 tx=0x7f77c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1063602c96f0e4d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77d77fe700 1 -- 172.21.15.83:0/2925536817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4004230 con 0x7f77d806f480 2023-11-26T07:16:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77d77fe700 1 -- 172.21.15.83:0/2925536817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c4004390 con 0x7f77d806f480 2023-11-26T07:16:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77d77fe700 1 -- 172.21.15.83:0/2925536817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c40036a0 con 0x7f77d806f480 2023-11-26T07:16:06.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77deb5e700 1 -- 172.21.15.83:0/2925536817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d806f480 msgr2=0x7f77d806c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:06.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77deb5e700 1 --2- 172.21.15.83:0/2925536817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d806f480 0x7f77d806c2a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f77c40098f0 tx=0x7f77c4009c00 comp rx=0 tx=0).stop 2023-11-26T07:16:06.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77deb5e700 1 -- 172.21.15.83:0/2925536817 shutdown_connections 2023-11-26T07:16:06.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77deb5e700 1 --2- 172.21.15.83:0/2925536817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d806f480 0x7f77d806c2a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:06.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77deb5e700 1 -- 172.21.15.83:0/2925536817 >> 172.21.15.83:0/2925536817 conn(0x7f77d80d0b40 msgr2=0x7f77d80d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:06.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.378+0000 7f77deb5e700 1 -- 172.21.15.83:0/2925536817 shutdown_connections 2023-11-26T07:16:06.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.379+0000 7f77deb5e700 1 -- 172.21.15.83:0/2925536817 wait complete. 2023-11-26T07:16:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.380+0000 7f77deb5e700 1 Processor -- start 2023-11-26T07:16:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.380+0000 7f77deb5e700 1 -- start start 2023-11-26T07:16:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77deb5e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d806e830 0x7f77d81f7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77deb5e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d806f480 0x7f77d815a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77deb5e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d8144600 0x7f77d8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77deb5e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81f7400 con 0x7f77d8144600 2023-11-26T07:16:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77deb5e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81f6e10 con 0x7f77d806e830 2023-11-26T07:16:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77deb5e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d81f7d60 con 0x7f77d806f480 2023-11-26T07:16:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77dd0fb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d8144600 0x7f77d8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77dd0fb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d8144600 0x7f77d8073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40232/0 (socket says 172.21.15.83:40232) 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77dd0fb700 1 -- 172.21.15.83:0/1436849170 learned_addr learned my addr 172.21.15.83:0/1436849170 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77dd0fb700 1 -- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d806f480 msgr2=0x7f77d815a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77dd0fb700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d806f480 0x7f77d815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77dd0fb700 1 -- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d806e830 msgr2=0x7f77d81f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77dc8fa700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d806e830 0x7f77d81f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77dd0fb700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d806e830 0x7f77d81f7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77dd0fb700 1 -- 172.21.15.83:0/1436849170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c4009550 con 0x7f77d8144600 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77dc8fa700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d806e830 0x7f77d81f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77dd0fb700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d8144600 0x7f77d8073f90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f77d000cc50 tx=0x7f77d000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77d57fa700 1 -- 172.21.15.83:0/1436849170 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d000ad80 con 0x7f77d8144600 2023-11-26T07:16:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77d57fa700 1 -- 172.21.15.83:0/1436849170 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d000ce50 con 0x7f77d8144600 2023-11-26T07:16:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77deb5e700 1 -- 172.21.15.83:0/1436849170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d8074a10 con 0x7f77d8144600 2023-11-26T07:16:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77d57fa700 1 -- 172.21.15.83:0/1436849170 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0003c40 con 0x7f77d8144600 2023-11-26T07:16:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.381+0000 7f77deb5e700 1 -- 172.21.15.83:0/1436849170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d81f7020 con 0x7f77d8144600 2023-11-26T07:16:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.382+0000 7f77d57fa700 1 -- 172.21.15.83:0/1436849170 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 44958+0+0 (secure 0 0 0) 0x7f77d00050d0 con 0x7f77d8144600 2023-11-26T07:16:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.382+0000 7f77deb5e700 1 -- 172.21.15.83:0/1436849170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d8067f60 con 0x7f77d8144600 2023-11-26T07:16:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.383+0000 7f77d57fa700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f77bc037ae0 0x7f77bc039fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.383+0000 7f77d57fa700 1 -- 172.21.15.83:0/1436849170 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f77d004af10 con 0x7f77d8144600 2023-11-26T07:16:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.383+0000 7f77dc8fa700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f77bc037ae0 0x7f77bc039fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.383+0000 7f77dc8fa700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f77bc037ae0 0x7f77bc039fa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f77cc006fd0 tx=0x7f77cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:06.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.387+0000 7f77d57fa700 1 -- 172.21.15.83:0/1436849170 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77d0003da0 con 0x7f77d8144600 2023-11-26T07:16:06.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.580+0000 7f77deb5e700 1 -- 172.21.15.83:0/1436849170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f77d806e0e0 con 0x7f77d8144600 2023-11-26T07:16:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:06.614+0000 7f77d57fa700 1 -- 172.21.15.83:0/1436849170 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f77d004c020 con 0x7f77d8144600 2023-11-26T07:16:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.237+0000 7f77d57fa700 1 -- 172.21.15.83:0/1436849170 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7f77d0058550 con 0x7f77d8144600 2023-11-26T07:16:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 -- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f77bc037ae0 msgr2=0x7f77bc039fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f77bc037ae0 0x7f77bc039fa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f77cc006fd0 tx=0x7f77cc008040 comp rx=0 tx=0).stop 2023-11-26T07:16:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 -- 172.21.15.83:0/1436849170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d8144600 msgr2=0x7f77d8073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d8144600 0x7f77d8073f90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f77d000cc50 tx=0x7f77d000a340 comp rx=0 tx=0).stop 2023-11-26T07:16:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 -- 172.21.15.83:0/1436849170 shutdown_connections 2023-11-26T07:16:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f77bc037ae0 0x7f77bc039fa0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d8144600 0x7f77d8073f90 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77d806e830 0x7f77d81f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 --2- 172.21.15.83:0/1436849170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77d806f480 0x7f77d815a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 -- 172.21.15.83:0/1436849170 >> 172.21.15.83:0/1436849170 conn(0x7f77d80d0b40 msgr2=0x7f77d80b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 -- 172.21.15.83:0/1436849170 shutdown_connections 2023-11-26T07:16:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.241+0000 7f77deb5e700 1 -- 172.21.15.83:0/1436849170 wait complete. 2023-11-26T07:16:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:16:07.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:16:07.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.382+0000 7f144e286700 1 Processor -- start 2023-11-26T07:16:07.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.382+0000 7f144e286700 1 -- start start 2023-11-26T07:16:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.383+0000 7f144e286700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f14480cf040 0x7f144806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.383+0000 7f144e286700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f14480d21c0 0x7f14480af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.383+0000 7f144e286700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14480d01c0 con 0x7f14480702e0 2023-11-26T07:16:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.383+0000 7f144e286700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1448070910 con 0x7f14480d21c0 2023-11-26T07:16:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.383+0000 7f144e286700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1448070bf0 con 0x7f14480cf040 2023-11-26T07:16:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.383+0000 7f14477fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f14480702e0 0x7f14480d1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60278/0 (socket says 172.21.15.83:60278) 2023-11-26T07:16:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.383+0000 7f14477fe700 1 -- 172.21.15.83:0/3716258830 learned_addr learned my addr 172.21.15.83:0/3716258830 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.383+0000 7f144c823700 1 --2- 172.21.15.83:0/3716258830 >> v2:172.21.15.83:3300/0 conn(0x7f14480d21c0 0x7f14480af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f1447fff700 1 --2- 172.21.15.83:0/3716258830 >> v2:172.21.15.203:3300/0 conn(0x7f14480cf040 0x7f144806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f144c823700 1 -- 172.21.15.83:0/3716258830 >> v2:172.21.15.203:3300/0 conn(0x7f14480cf040 msgr2=0x7f144806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f144c823700 1 --2- 172.21.15.83:0/3716258830 >> v2:172.21.15.203:3300/0 conn(0x7f14480cf040 0x7f144806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f144c823700 1 -- 172.21.15.83:0/3716258830 >> v1:172.21.15.83:6789/0 conn(0x7f14480702e0 legacy=0x7f14480d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f144c823700 1 -- 172.21.15.83:0/3716258830 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14480aebb0 con 0x7f14480d21c0 2023-11-26T07:16:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f1446ffd700 1 -- 172.21.15.83:0/3716258830 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924693477 0 0) 0x7f14480d01c0 con 0x7f14480702e0 2023-11-26T07:16:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f144c823700 1 --2- 172.21.15.83:0/3716258830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14480d21c0 0x7f14480af3b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f14480cf690 tx=0x7f144000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbb270c7d1667ea server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f1446ffd700 1 -- 172.21.15.83:0/3716258830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000e070 con 0x7f14480d21c0 2023-11-26T07:16:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f1446ffd700 1 -- 172.21.15.83:0/3716258830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144000bbd0 con 0x7f14480d21c0 2023-11-26T07:16:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f1446ffd700 1 -- 172.21.15.83:0/3716258830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440012430 con 0x7f14480d21c0 2023-11-26T07:16:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f144e286700 1 -- 172.21.15.83:0/3716258830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14480d21c0 msgr2=0x7f14480af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.384+0000 7f144e286700 1 --2- 172.21.15.83:0/3716258830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14480d21c0 0x7f14480af3b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f14480cf690 tx=0x7f144000b330 comp rx=0 tx=0).stop 2023-11-26T07:16:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.385+0000 7f144e286700 1 -- 172.21.15.83:0/3716258830 shutdown_connections 2023-11-26T07:16:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.385+0000 7f144e286700 1 --2- 172.21.15.83:0/3716258830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14480d21c0 0x7f14480af3b0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.385+0000 7f144e286700 1 --2- 172.21.15.83:0/3716258830 >> v2:172.21.15.203:3300/0 conn(0x7f14480cf040 0x7f144806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.385+0000 7f144e286700 1 -- 172.21.15.83:0/3716258830 >> 172.21.15.83:0/3716258830 conn(0x7f144809aa00 msgr2=0x7f144809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.385+0000 7f144e286700 1 -- 172.21.15.83:0/3716258830 shutdown_connections 2023-11-26T07:16:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.385+0000 7f144e286700 1 -- 172.21.15.83:0/3716258830 wait complete. 2023-11-26T07:16:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.386+0000 7f144e286700 1 Processor -- start 2023-11-26T07:16:07.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.386+0000 7f144e286700 1 -- start start 2023-11-26T07:16:07.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f144e286700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14480702e0 0x7f14480c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:07.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f144e286700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14480cf040 0x7f14480c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:07.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f144e286700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14480d21c0 0x7f144815d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:07.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f144e286700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14480c59b0 con 0x7f14480cf040 2023-11-26T07:16:07.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f144e286700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14480c82d0 con 0x7f14480702e0 2023-11-26T07:16:07.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f144e286700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14480c85b0 con 0x7f14480d21c0 2023-11-26T07:16:07.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f14477fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14480cf040 0x7f14480c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:07.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f14477fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14480cf040 0x7f14480c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40246/0 (socket says 172.21.15.83:40246) 2023-11-26T07:16:07.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f14477fe700 1 -- 172.21.15.83:0/2198621046 learned_addr learned my addr 172.21.15.83:0/2198621046 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:07.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f1447fff700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14480702e0 0x7f14480c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:07.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f144c823700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14480d21c0 0x7f144815d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:07.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f14477fe700 1 -- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14480d21c0 msgr2=0x7f144815d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:07.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f14477fe700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14480d21c0 0x7f144815d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:07.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f14477fe700 1 -- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14480702e0 msgr2=0x7f14480c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:07.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f14477fe700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14480702e0 0x7f14480c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:07.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f14477fe700 1 -- 172.21.15.83:0/2198621046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440009a70 con 0x7f14480cf040 2023-11-26T07:16:07.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f1447fff700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14480702e0 0x7f14480c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:07.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f144c823700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14480d21c0 0x7f144815d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f14477fe700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14480cf040 0x7f14480c5cd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f14380011e0 tx=0x7f143800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f1444ff9700 1 -- 172.21.15.83:0/2198621046 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438004c80 con 0x7f14480cf040 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f1444ff9700 1 -- 172.21.15.83:0/2198621046 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1438004e00 con 0x7f14480cf040 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f144e286700 1 -- 172.21.15.83:0/2198621046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14480c8890 con 0x7f14480cf040 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f1444ff9700 1 -- 172.21.15.83:0/2198621046 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438007310 con 0x7f14480cf040 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.387+0000 7f144e286700 1 -- 172.21.15.83:0/2198621046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144815d620 con 0x7f14480cf040 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.388+0000 7f144e286700 1 -- 172.21.15.83:0/2198621046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14480cc0e0 con 0x7f14480cf040 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.389+0000 7f1444ff9700 1 -- 172.21.15.83:0/2198621046 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f1438007470 con 0x7f14480cf040 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.389+0000 7f1444ff9700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f142806aff0 0x7f142806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.389+0000 7f1444ff9700 1 -- 172.21.15.83:0/2198621046 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5916+0+0 (secure 0 0 0) 0x7f14380895b0 con 0x7f14480cf040 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.389+0000 7f1447fff700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f142806aff0 0x7f142806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:07.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.390+0000 7f1447fff700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f142806aff0 0x7f142806d4b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14300099f0 tx=0x7f1430008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:07.396 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.394+0000 7f1444ff9700 1 -- 172.21.15.83:0/2198621046 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1438054510 con 0x7f14480cf040 2023-11-26T07:16:07.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:07.588+0000 7f144e286700 1 -- 172.21.15.83:0/2198621046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f14480ca000 con 0x7f14480cf040 2023-11-26T07:16:08.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.543+0000 7f1444ff9700 1 -- 172.21.15.83:0/2198621046 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7f14480ca000 con 0x7f14480cf040 2023-11-26T07:16:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.546+0000 7f144e286700 1 -- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f142806aff0 msgr2=0x7f142806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.546+0000 7f144e286700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f142806aff0 0x7f142806d4b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14300099f0 tx=0x7f1430008040 comp rx=0 tx=0).stop 2023-11-26T07:16:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.546+0000 7f144e286700 1 -- 172.21.15.83:0/2198621046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14480cf040 msgr2=0x7f14480c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.546+0000 7f144e286700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14480cf040 0x7f14480c5cd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f14380011e0 tx=0x7f143800cae0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.546+0000 7f144e286700 1 -- 172.21.15.83:0/2198621046 shutdown_connections 2023-11-26T07:16:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.546+0000 7f144e286700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f142806aff0 0x7f142806d4b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.546+0000 7f144e286700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14480d21c0 0x7f144815d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.546+0000 7f144e286700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14480702e0 0x7f14480c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.546+0000 7f144e286700 1 --2- 172.21.15.83:0/2198621046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14480cf040 0x7f14480c5cd0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.546+0000 7f144e286700 1 -- 172.21.15.83:0/2198621046 >> 172.21.15.83:0/2198621046 conn(0x7f144809aa00 msgr2=0x7f144815fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.547+0000 7f144e286700 1 -- 172.21.15.83:0/2198621046 shutdown_connections 2023-11-26T07:16:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.547+0000 7f144e286700 1 -- 172.21.15.83:0/2198621046 wait complete. 2023-11-26T07:16:08.551 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2023-11-26T07:16:08.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:16:08.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.692+0000 7f353b2ae700 1 Processor -- start 2023-11-26T07:16:08.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.692+0000 7f353b2ae700 1 -- start start 2023-11-26T07:16:08.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353b2ae700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f35340cf610 0x7f35340add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353b2ae700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3534073f90 0x7f3534074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353b2ae700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f353406b080 con 0x7f353415f9e0 2023-11-26T07:16:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353b2ae700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3534074960 con 0x7f3534073f90 2023-11-26T07:16:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353b2ae700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f353415c290 con 0x7f35340cf610 2023-11-26T07:16:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353984b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3534073f90 0x7f3534074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353984b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3534073f90 0x7f3534074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40256/0 (socket says 172.21.15.83:40256) 2023-11-26T07:16:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353984b700 1 -- 172.21.15.83:0/1666957702 learned_addr learned my addr 172.21.15.83:0/1666957702 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353984b700 1 -- 172.21.15.83:0/1666957702 >> v2:172.21.15.203:3300/0 conn(0x7f35340cf610 msgr2=0x7f35340add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353904a700 1 --2- 172.21.15.83:0/1666957702 >> v2:172.21.15.203:3300/0 conn(0x7f35340cf610 0x7f35340add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353984b700 1 --2- 172.21.15.83:0/1666957702 >> v2:172.21.15.203:3300/0 conn(0x7f35340cf610 0x7f35340add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353984b700 1 -- 172.21.15.83:0/1666957702 >> v1:172.21.15.203:6789/0 conn(0x7f353415f9e0 legacy=0x7f35340b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353984b700 1 -- 172.21.15.83:0/1666957702 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35340cf9f0 con 0x7f3534073f90 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353904a700 1 --2- 172.21.15.83:0/1666957702 >> v2:172.21.15.203:3300/0 conn(0x7f35340cf610 0x7f35340add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.693+0000 7f353984b700 1 --2- 172.21.15.83:0/1666957702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3534073f90 0x7f3534074420 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f352c009e50 tx=0x7f352c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52975659512ad73 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.694+0000 7f352bfff700 1 -- 172.21.15.83:0/1666957702 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c00e070 con 0x7f3534073f90 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.694+0000 7f352bfff700 1 -- 172.21.15.83:0/1666957702 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352c003a20 con 0x7f3534073f90 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.694+0000 7f352bfff700 1 -- 172.21.15.83:0/1666957702 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c003dd0 con 0x7f3534073f90 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.694+0000 7f353b2ae700 1 -- 172.21.15.83:0/1666957702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3534073f90 msgr2=0x7f3534074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.694+0000 7f353b2ae700 1 --2- 172.21.15.83:0/1666957702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3534073f90 0x7f3534074420 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f352c009e50 tx=0x7f352c00b330 comp rx=0 tx=0).stop 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.694+0000 7f353b2ae700 1 -- 172.21.15.83:0/1666957702 shutdown_connections 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.694+0000 7f353b2ae700 1 --2- 172.21.15.83:0/1666957702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3534073f90 0x7f3534074420 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.694+0000 7f353b2ae700 1 --2- 172.21.15.83:0/1666957702 >> v2:172.21.15.203:3300/0 conn(0x7f35340cf610 0x7f35340add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.694+0000 7f353b2ae700 1 -- 172.21.15.83:0/1666957702 >> 172.21.15.83:0/1666957702 conn(0x7f35340d0bf0 msgr2=0x7f35340d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:08.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.694+0000 7f353b2ae700 1 -- 172.21.15.83:0/1666957702 shutdown_connections 2023-11-26T07:16:08.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.694+0000 7f353b2ae700 1 -- 172.21.15.83:0/1666957702 wait complete. 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.695+0000 7f353b2ae700 1 Processor -- start 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.695+0000 7f353b2ae700 1 -- start start 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353b2ae700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3534073f90 0x7f3534142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353b2ae700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35340cf610 0x7f3534157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353b2ae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f353415f9e0 0x7f3534155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353b2ae700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3534157350 con 0x7f3534073f90 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353b2ae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3534156410 con 0x7f353415f9e0 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353b2ae700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35341566d0 con 0x7f35340cf610 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353904a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3534073f90 0x7f3534142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353904a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3534073f90 0x7f3534142b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40258/0 (socket says 172.21.15.83:40258) 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353904a700 1 -- 172.21.15.83:0/367482460 learned_addr learned my addr 172.21.15.83:0/367482460 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:08.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353904a700 1 -- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35340cf610 msgr2=0x7f3534157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353904a700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35340cf610 0x7f3534157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353904a700 1 -- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f353415f9e0 msgr2=0x7f3534155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.696+0000 7f353904a700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f353415f9e0 0x7f3534155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.697+0000 7f353904a700 1 -- 172.21.15.83:0/367482460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c009b30 con 0x7f3534073f90 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.697+0000 7f353904a700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3534073f90 0x7f3534142b80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f353000c750 tx=0x7f353000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.697+0000 7f3529ffb700 1 -- 172.21.15.83:0/367482460 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530011800 con 0x7f3534073f90 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.697+0000 7f3529ffb700 1 -- 172.21.15.83:0/367482460 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3530011960 con 0x7f3534073f90 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.697+0000 7f353b2ae700 1 -- 172.21.15.83:0/367482460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35341569b0 con 0x7f3534073f90 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.697+0000 7f3529ffb700 1 -- 172.21.15.83:0/367482460 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530011cd0 con 0x7f3534073f90 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.697+0000 7f353b2ae700 1 -- 172.21.15.83:0/367482460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3534156f70 con 0x7f3534073f90 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.698+0000 7f353b2ae700 1 -- 172.21.15.83:0/367482460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3534068fb0 con 0x7f3534073f90 2023-11-26T07:16:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.698+0000 7f3529ffb700 1 -- 172.21.15.83:0/367482460 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f3530021350 con 0x7f3534073f90 2023-11-26T07:16:08.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.699+0000 7f3529ffb700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f351806b020 0x7f351806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:08.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.699+0000 7f3529ffb700 1 -- 172.21.15.83:0/367482460 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f3530089730 con 0x7f3534073f90 2023-11-26T07:16:08.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.699+0000 7f3538849700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f351806b020 0x7f351806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:08.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.699+0000 7f3538849700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f351806b020 0x7f351806d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f35341512b0 tx=0x7f3524007a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:08.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.703+0000 7f3529ffb700 1 -- 172.21.15.83:0/367482460 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3530054600 con 0x7f3534073f90 2023-11-26T07:16:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.901+0000 7f353b2ae700 1 -- 172.21.15.83:0/367482460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3534157100 con 0x7f3534073f90 2023-11-26T07:16:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.904+0000 7f3529ffb700 1 -- 172.21.15.83:0/367482460 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3530057c20 con 0x7f3534073f90 2023-11-26T07:16:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 -- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f351806b020 msgr2=0x7f351806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f351806b020 0x7f351806d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f35341512b0 tx=0x7f3524007a40 comp rx=0 tx=0).stop 2023-11-26T07:16:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 -- 172.21.15.83:0/367482460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3534073f90 msgr2=0x7f3534142b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3534073f90 0x7f3534142b80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f353000c750 tx=0x7f353000cb10 comp rx=0 tx=0).stop 2023-11-26T07:16:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 -- 172.21.15.83:0/367482460 shutdown_connections 2023-11-26T07:16:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f351806b020 0x7f351806d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f353415f9e0 0x7f3534155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3534073f90 0x7f3534142b80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 --2- 172.21.15.83:0/367482460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35340cf610 0x7f3534157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 -- 172.21.15.83:0/367482460 >> 172.21.15.83:0/367482460 conn(0x7f35340d0bf0 msgr2=0x7f35340b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 -- 172.21.15.83:0/367482460 shutdown_connections 2023-11-26T07:16:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:08.907+0000 7f353b2ae700 1 -- 172.21.15.83:0/367482460 wait complete. 2023-11-26T07:16:08.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2023-11-26T07:16:09.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.039+0000 7efe6e81f700 1 Processor -- start 2023-11-26T07:16:09.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.039+0000 7efe6e81f700 1 -- start start 2023-11-26T07:16:09.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.040+0000 7efe6e81f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efe680afe70 0x7efe6815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.040+0000 7efe6e81f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6806dd80 con 0x7efe6815bef0 2023-11-26T07:16:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.040+0000 7efe6e81f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6815f880 con 0x7efe681624d0 2023-11-26T07:16:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.040+0000 7efe6e81f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6815ed90 con 0x7efe680afe70 2023-11-26T07:16:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.040+0000 7efe67fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efe6815bef0 0x7efe680b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60306/0 (socket says 172.21.15.83:60306) 2023-11-26T07:16:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.040+0000 7efe67fff700 1 -- 172.21.15.83:0/3641733748 learned_addr learned my addr 172.21.15.83:0/3641733748 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871894777 0 0) 0x7efe6806dd80 con 0x7efe6815bef0 2023-11-26T07:16:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe500035b0 con 0x7efe6815bef0 2023-11-26T07:16:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe677fe700 1 --2- 172.21.15.83:0/3641733748 >> v2:172.21.15.203:3300/0 conn(0x7efe680afe70 0x7efe6815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 703525377 0 0) 0x7efe500035b0 con 0x7efe6815bef0 2023-11-26T07:16:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 >> v2:172.21.15.203:3300/0 conn(0x7efe680afe70 msgr2=0x7efe6815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 --2- 172.21.15.83:0/3641733748 >> v2:172.21.15.203:3300/0 conn(0x7efe680afe70 0x7efe6815e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 >> v1:172.21.15.203:6789/0 conn(0x7efe681624d0 legacy=0x7efe6806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6815c2d0 con 0x7efe6815bef0 2023-11-26T07:16:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7efe5c001670 con 0x7efe6815bef0 2023-11-26T07:16:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 >> v1:172.21.15.83:6789/0 conn(0x7efe6815bef0 legacy=0x7efe680b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe50003a00 0x7efe50003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe50004850 0x7efe50004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe50005210 0x7efe5000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6815ed90 con 0x7efe50005210 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5000c1b0 con 0x7efe50004850 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5000c370 con 0x7efe50003a00 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe6cdbc700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe50005210 0x7efe5000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe677fe700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe50004850 0x7efe50004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe67fff700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe50003a00 0x7efe50003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe6cdbc700 1 -- 172.21.15.83:0/3641733748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe50003a00 msgr2=0x7efe50003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe6cdbc700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe50003a00 0x7efe50003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe6cdbc700 1 -- 172.21.15.83:0/3641733748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe50004850 msgr2=0x7efe50004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe6cdbc700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe50004850 0x7efe50004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe6cdbc700 1 -- 172.21.15.83:0/3641733748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe500035b0 con 0x7efe50005210 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe677fe700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe50004850 0x7efe50004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe6cdbc700 1 -- 172.21.15.83:0/3641733748 reap_dead start 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.041+0000 7efe6cdbc700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe50005210 0x7efe5000ba10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7efe6000c190 tx=0x7efe50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87de2b7776790df4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.042+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60001db0 con 0x7efe50005210 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.042+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6000ccd0 con 0x7efe50005210 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.042+0000 7efe66ffd700 1 -- 172.21.15.83:0/3641733748 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60003c40 con 0x7efe50005210 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.042+0000 7efe6e81f700 1 -- 172.21.15.83:0/3641733748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe50005210 msgr2=0x7efe5000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.042+0000 7efe6e81f700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe50005210 0x7efe5000ba10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7efe6000c190 tx=0x7efe50005880 comp rx=0 tx=0).stop 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.042+0000 7efe6e81f700 1 -- 172.21.15.83:0/3641733748 shutdown_connections 2023-11-26T07:16:09.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.042+0000 7efe6e81f700 1 --2- 172.21.15.83:0/3641733748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe50005210 0x7efe5000ba10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.042+0000 7efe6e81f700 1 -- 172.21.15.83:0/3641733748 >> 172.21.15.83:0/3641733748 conn(0x7efe68071ac0 msgr2=0x7efe68071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.042+0000 7efe6e81f700 1 -- 172.21.15.83:0/3641733748 shutdown_connections 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.042+0000 7efe6e81f700 1 -- 172.21.15.83:0/3641733748 wait complete. 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.043+0000 7efe6e81f700 1 Processor -- start 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.044+0000 7efe6e81f700 1 -- start start 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.044+0000 7efe6e81f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe50005210 0x7efe681617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.044+0000 7efe6e81f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe68160390 0x7efe681e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.044+0000 7efe6e81f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe681f3720 0x7efe681f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.044+0000 7efe6e81f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe681f4110 con 0x7efe68160390 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.044+0000 7efe6e81f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe681f78e0 con 0x7efe50005210 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.044+0000 7efe6e81f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe681f7bc0 con 0x7efe681f3720 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.044+0000 7efe677fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe68160390 0x7efe681e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.044+0000 7efe677fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe68160390 0x7efe681e88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40280/0 (socket says 172.21.15.83:40280) 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.044+0000 7efe677fe700 1 -- 172.21.15.83:0/918245607 learned_addr learned my addr 172.21.15.83:0/918245607 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe67fff700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe50005210 0x7efe681617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe6cdbc700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe681f3720 0x7efe681f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe677fe700 1 -- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe681f3720 msgr2=0x7efe681f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe677fe700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe681f3720 0x7efe681f3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe677fe700 1 -- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe50005210 msgr2=0x7efe681617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe677fe700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe50005210 0x7efe681617c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe677fe700 1 -- 172.21.15.83:0/918245607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe681f7ea0 con 0x7efe68160390 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe67fff700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe50005210 0x7efe681617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe6cdbc700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe681f3720 0x7efe681f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe677fe700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe68160390 0x7efe681e88a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7efe58010850 tx=0x7efe58010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe64ff9700 1 -- 172.21.15.83:0/918245607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58010e50 con 0x7efe68160390 2023-11-26T07:16:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe64ff9700 1 -- 172.21.15.83:0/918245607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe58004020 con 0x7efe68160390 2023-11-26T07:16:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe6e81f700 1 -- 172.21.15.83:0/918245607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe681f8180 con 0x7efe68160390 2023-11-26T07:16:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe64ff9700 1 -- 172.21.15.83:0/918245607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe580043b0 con 0x7efe68160390 2023-11-26T07:16:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.045+0000 7efe6e81f700 1 -- 172.21.15.83:0/918245607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe681e8de0 con 0x7efe68160390 2023-11-26T07:16:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.046+0000 7efe6e81f700 1 -- 172.21.15.83:0/918245607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe680b6f30 con 0x7efe68160390 2023-11-26T07:16:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.046+0000 7efe64ff9700 1 -- 172.21.15.83:0/918245607 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7efe5800e8d0 con 0x7efe68160390 2023-11-26T07:16:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.047+0000 7efe64ff9700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efe5406aff0 0x7efe5406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.047+0000 7efe64ff9700 1 -- 172.21.15.83:0/918245607 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5502+0+0 (secure 0 0 0) 0x7efe58089910 con 0x7efe68160390 2023-11-26T07:16:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.047+0000 7efe67fff700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efe5406aff0 0x7efe5406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.048+0000 7efe67fff700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efe5406aff0 0x7efe5406d4b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efe5c001050 tx=0x7efe5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.051+0000 7efe64ff9700 1 -- 172.21.15.83:0/918245607 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe58054a10 con 0x7efe68160390 2023-11-26T07:16:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.318+0000 7efe6e81f700 1 -- 172.21.15.83:0/918245607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7efe6806f650 con 0x7efe68160390 2023-11-26T07:16:09.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.544+0000 7efe64ff9700 1 -- 172.21.15.83:0/918245607 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v256) v1 ==== 131+0+0 (secure 0 0 0) 0x7efe58058030 con 0x7efe68160390 2023-11-26T07:16:09.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 -- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efe5406aff0 msgr2=0x7efe5406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efe5406aff0 0x7efe5406d4b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efe5c001050 tx=0x7efe5c009040 comp rx=0 tx=0).stop 2023-11-26T07:16:09.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 -- 172.21.15.83:0/918245607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe68160390 msgr2=0x7efe681e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe68160390 0x7efe681e88a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7efe58010850 tx=0x7efe58010c10 comp rx=0 tx=0).stop 2023-11-26T07:16:09.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 -- 172.21.15.83:0/918245607 shutdown_connections 2023-11-26T07:16:09.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efe5406aff0 0x7efe5406d4b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe681f3720 0x7efe681f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe50005210 0x7efe681617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 --2- 172.21.15.83:0/918245607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe68160390 0x7efe681e88a0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 -- 172.21.15.83:0/918245607 >> 172.21.15.83:0/918245607 conn(0x7efe68071ac0 msgr2=0x7efe680b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:09.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 -- 172.21.15.83:0/918245607 shutdown_connections 2023-11-26T07:16:09.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.546+0000 7efe6e81f700 1 -- 172.21.15.83:0/918245607 wait complete. 2023-11-26T07:16:09.561 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-11-26T07:16:09.561 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2023-11-26T07:16:09.561 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:16:09.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.679+0000 7f0f38461700 1 Processor -- start 2023-11-26T07:16:09.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.679+0000 7f0f38461700 1 -- start start 2023-11-26T07:16:09.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.680+0000 7f0f38461700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f30166d40 0x7f0f30169130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.680+0000 7f0f38461700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f3006f6c0 con 0x7f0f300b4600 2023-11-26T07:16:09.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.680+0000 7f0f38461700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f3006f900 con 0x7f0f3006c000 2023-11-26T07:16:09.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.680+0000 7f0f38461700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30074e80 con 0x7f0f30166d40 2023-11-26T07:16:09.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.680+0000 7f0f359fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0f300b4600 0x7f0f3006efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60326/0 (socket says 172.21.15.83:60326) 2023-11-26T07:16:09.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.680+0000 7f0f359fc700 1 -- 172.21.15.83:0/53546852 learned_addr learned my addr 172.21.15.83:0/53546852 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f369fe700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f30166d40 0x7f0f30169130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859524417 0 0) 0x7f0f2c001110 con 0x7f0f300b4600 2023-11-26T07:16:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f1c0035b0 con 0x7f0f300b4600 2023-11-26T07:16:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1360791499 0 0) 0x7f0f1c0035b0 con 0x7f0f300b4600 2023-11-26T07:16:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f30166d40 msgr2=0x7f0f30169130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f30166d40 0x7f0f30169130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 >> v1:172.21.15.203:6789/0 conn(0x7f0f3006c000 legacy=0x7f0f3006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f300740c0 con 0x7f0f300b4600 2023-11-26T07:16:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f0f2c001800 con 0x7f0f300b4600 2023-11-26T07:16:09.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 >> v1:172.21.15.83:6789/0 conn(0x7f0f300b4600 legacy=0x7f0f3006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f1c003a30 0x7f0f1c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f30166d40 0x7f0f1c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f1c004d90 0x7f0f1c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30074e80 con 0x7f0f1c003a30 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c00bb70 con 0x7f0f1c004d90 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1c00bd30 con 0x7f0f30166d40 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f361fd700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f1c003a30 0x7f0f1c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f359fc700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f30166d40 0x7f0f1c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f369fe700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f1c004d90 0x7f0f1c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f361fd700 1 -- 172.21.15.83:0/53546852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f30166d40 msgr2=0x7f0f1c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f361fd700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f30166d40 0x7f0f1c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f361fd700 1 -- 172.21.15.83:0/53546852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f1c004d90 msgr2=0x7f0f1c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f361fd700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f1c004d90 0x7f0f1c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.681+0000 7f0f361fd700 1 -- 172.21.15.83:0/53546852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c0035b0 con 0x7f0f1c003a30 2023-11-26T07:16:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.682+0000 7f0f361fd700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f1c003a30 0x7f0f1c003e50 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0f200017c0 tx=0x7f0f1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d881a049ea81a1f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.682+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f200052d0 con 0x7f0f1c003a30 2023-11-26T07:16:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.682+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f20005430 con 0x7f0f1c003a30 2023-11-26T07:16:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.682+0000 7f0f351fb700 1 -- 172.21.15.83:0/53546852 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f200038c0 con 0x7f0f1c003a30 2023-11-26T07:16:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.682+0000 7f0f38461700 1 -- 172.21.15.83:0/53546852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f1c003a30 msgr2=0x7f0f1c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.682+0000 7f0f38461700 1 --2- 172.21.15.83:0/53546852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f1c003a30 0x7f0f1c003e50 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0f200017c0 tx=0x7f0f1c005880 comp rx=0 tx=0).stop 2023-11-26T07:16:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.682+0000 7f0f369fe700 1 -- 172.21.15.83:0/53546852 reap_dead start 2023-11-26T07:16:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.682+0000 7f0f38461700 1 -- 172.21.15.83:0/53546852 shutdown_connections 2023-11-26T07:16:09.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.682+0000 7f0f38461700 1 -- 172.21.15.83:0/53546852 >> 172.21.15.83:0/53546852 conn(0x7f0f30071a90 msgr2=0x7f0f30071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:09.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.682+0000 7f0f38461700 1 -- 172.21.15.83:0/53546852 shutdown_connections 2023-11-26T07:16:09.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.682+0000 7f0f38461700 1 -- 172.21.15.83:0/53546852 wait complete. 2023-11-26T07:16:09.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.683+0000 7f0f38461700 1 Processor -- start 2023-11-26T07:16:09.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.684+0000 7f0f38461700 1 -- start start 2023-11-26T07:16:09.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.684+0000 7f0f38461700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1c003a30 0x7f0f3014bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.684+0000 7f0f38461700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f301555d0 0x7f0f30155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.684+0000 7f0f38461700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f30160410 0x7f0f30160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.684+0000 7f0f38461700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30155f90 con 0x7f0f30160410 2023-11-26T07:16:09.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.684+0000 7f0f38461700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30160dd0 con 0x7f0f301555d0 2023-11-26T07:16:09.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.684+0000 7f0f38461700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3014dc70 con 0x7f0f1c003a30 2023-11-26T07:16:09.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.684+0000 7f0f369fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f30160410 0x7f0f30160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.684+0000 7f0f369fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f30160410 0x7f0f30160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40302/0 (socket says 172.21.15.83:40302) 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.684+0000 7f0f369fe700 1 -- 172.21.15.83:0/3473024707 learned_addr learned my addr 172.21.15.83:0/3473024707 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.684+0000 7f0f361fd700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1c003a30 0x7f0f3014bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f359fc700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f301555d0 0x7f0f30155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f369fe700 1 -- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1c003a30 msgr2=0x7f0f3014bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f369fe700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1c003a30 0x7f0f3014bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f369fe700 1 -- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f301555d0 msgr2=0x7f0f30155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f369fe700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f301555d0 0x7f0f30155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f369fe700 1 -- 172.21.15.83:0/3473024707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f301525f0 con 0x7f0f30160410 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f369fe700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f30160410 0x7f0f30160890 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0f3006d470 tx=0x7f0f280111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f26ffd700 1 -- 172.21.15.83:0/3473024707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2800e070 con 0x7f0f30160410 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f26ffd700 1 -- 172.21.15.83:0/3473024707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f28003cf0 con 0x7f0f30160410 2023-11-26T07:16:09.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f38461700 1 -- 172.21.15.83:0/3473024707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f301528d0 con 0x7f0f30160410 2023-11-26T07:16:09.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f26ffd700 1 -- 172.21.15.83:0/3473024707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28003ea0 con 0x7f0f30160410 2023-11-26T07:16:09.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.685+0000 7f0f38461700 1 -- 172.21.15.83:0/3473024707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f30152e90 con 0x7f0f30160410 2023-11-26T07:16:09.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.686+0000 7f0f38461700 1 -- 172.21.15.83:0/3473024707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f300751e0 con 0x7f0f30160410 2023-11-26T07:16:09.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.686+0000 7f0f26ffd700 1 -- 172.21.15.83:0/3473024707 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f0f280117a0 con 0x7f0f30160410 2023-11-26T07:16:09.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.687+0000 7f0f26ffd700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f1406aff0 0x7f0f1406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:09.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.687+0000 7f0f26ffd700 1 -- 172.21.15.83:0/3473024707 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f0f280898e0 con 0x7f0f30160410 2023-11-26T07:16:09.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.687+0000 7f0f361fd700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f1406aff0 0x7f0f1406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:09.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.687+0000 7f0f361fd700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f1406aff0 0x7f0f1406d4b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0f20000c00 tx=0x7f0f20009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:09.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.691+0000 7f0f26ffd700 1 -- 172.21.15.83:0/3473024707 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f280548d0 con 0x7f0f30160410 2023-11-26T07:16:09.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.884+0000 7f0f38461700 1 -- 172.21.15.83:0/3473024707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f300b2380 con 0x7f0f30160410 2023-11-26T07:16:09.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.884+0000 7f0f26ffd700 1 -- 172.21.15.83:0/3473024707 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13785 (secure 0 0 0) 0x7f0f28057ef0 con 0x7f0f30160410 2023-11-26T07:16:09.887 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:09.887 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":74,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:08.542943+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6836/1809465302":"2023-11-27T07:16:05.231399+0000","172.21.15.83:6834/1809465302":"2023-11-27T07:16:05.231399+0000","172.21.15.83:0/3840607975":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1574556410":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/2981312946":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1953425269":"2023-11-27T07:16:00.270871+0000","172.21.15.83:6833/1483230064":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/367450219":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1792598028":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/2325877059":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/304865544":"2023-11-27T07:16:00.270871+0000","172.21.15.83:6832/1483230064":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/4194383148":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/1164473704":"2023-11-27T07:16:00.270871+0000","172.21.15.83:0/4012256052":"2023-11-27T07:16:00.270871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:09.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.886+0000 7f0f38461700 1 -- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f1406aff0 msgr2=0x7f0f1406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.886+0000 7f0f38461700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f1406aff0 0x7f0f1406d4b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0f20000c00 tx=0x7f0f20009030 comp rx=0 tx=0).stop 2023-11-26T07:16:09.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.886+0000 7f0f38461700 1 -- 172.21.15.83:0/3473024707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f30160410 msgr2=0x7f0f30160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:09.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.886+0000 7f0f38461700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f30160410 0x7f0f30160890 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0f3006d470 tx=0x7f0f280111d0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.886+0000 7f0f38461700 1 -- 172.21.15.83:0/3473024707 shutdown_connections 2023-11-26T07:16:09.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.886+0000 7f0f38461700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f1406aff0 0x7f0f1406d4b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.886+0000 7f0f38461700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f30160410 0x7f0f30160890 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.886+0000 7f0f38461700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1c003a30 0x7f0f3014bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.886+0000 7f0f38461700 1 --2- 172.21.15.83:0/3473024707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f301555d0 0x7f0f30155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:09.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.886+0000 7f0f38461700 1 -- 172.21.15.83:0/3473024707 >> 172.21.15.83:0/3473024707 conn(0x7f0f30071a90 msgr2=0x7f0f30071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:09.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.887+0000 7f0f38461700 1 -- 172.21.15.83:0/3473024707 shutdown_connections 2023-11-26T07:16:09.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:09.887+0000 7f0f38461700 1 -- 172.21.15.83:0/3473024707 wait complete. 2023-11-26T07:16:09.901 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:16:09.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:16:10.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.022+0000 7f3d014ac700 1 Processor -- start 2023-11-26T07:16:10.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.022+0000 7f3d014ac700 1 -- start start 2023-11-26T07:16:10.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.022+0000 7f3d014ac700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cfc073f90 0x7f3cfc06f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:10.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.022+0000 7f3d014ac700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc06fa50 con 0x7f3cfc074370 2023-11-26T07:16:10.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.022+0000 7f3d014ac700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc0b5630 con 0x7f3cfc15e1b0 2023-11-26T07:16:10.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.022+0000 7f3d014ac700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc15e770 con 0x7f3cfc073f90 2023-11-26T07:16:10.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cfb7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3cfc074370 0x7f3cfc06aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60350/0 (socket says 172.21.15.83:60350) 2023-11-26T07:16:10.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cfb7fe700 1 -- 172.21.15.83:0/1926665030 learned_addr learned my addr 172.21.15.83:0/1926665030 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:10.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cfaffd700 1 --2- 172.21.15.83:0/1926665030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cfc073f90 0x7f3cfc06f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:10.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cf9ffb700 1 -- 172.21.15.83:0/1926665030 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851331572 0 0) 0x7f3cf4001490 con 0x7f3cfc074370 2023-11-26T07:16:10.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cf9ffb700 1 -- 172.21.15.83:0/1926665030 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ce40035b0 con 0x7f3cfc074370 2023-11-26T07:16:10.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cf9ffb700 1 -- 172.21.15.83:0/1926665030 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758805333 0 0) 0x7f3cfc0b5630 con 0x7f3cfc15e1b0 2023-11-26T07:16:10.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cf9ffb700 1 -- 172.21.15.83:0/1926665030 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf4001490 con 0x7f3cfc15e1b0 2023-11-26T07:16:10.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cf9ffb700 1 -- 172.21.15.83:0/1926665030 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233994606 0 0) 0x7f3ce40035b0 con 0x7f3cfc074370 2023-11-26T07:16:10.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cfaffd700 1 -- 172.21.15.83:0/1926665030 >> v1:172.21.15.203:6789/0 conn(0x7f3cfc15e1b0 legacy=0x7f3cfc0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cfaffd700 1 -- 172.21.15.83:0/1926665030 >> v1:172.21.15.83:6789/0 conn(0x7f3cfc074370 legacy=0x7f3cfc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cfaffd700 1 -- 172.21.15.83:0/1926665030 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc0b58d0 con 0x7f3cfc073f90 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cfaffd700 1 --2- 172.21.15.83:0/1926665030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cfc073f90 0x7f3cfc06f380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009870 tx=0x7f3cec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd41b5e95517e151 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.023+0000 7f3cf9ffb700 1 -- 172.21.15.83:0/1926665030 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4001b80 con 0x7f3cfc073f90 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.024+0000 7f3cf9ffb700 1 -- 172.21.15.83:0/1926665030 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cec004030 con 0x7f3cfc073f90 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.024+0000 7f3cf9ffb700 1 -- 172.21.15.83:0/1926665030 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec0043c0 con 0x7f3cfc073f90 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.024+0000 7f3d014ac700 1 -- 172.21.15.83:0/1926665030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cfc073f90 msgr2=0x7f3cfc06f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.024+0000 7f3d014ac700 1 --2- 172.21.15.83:0/1926665030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cfc073f90 0x7f3cfc06f380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009870 tx=0x7f3cec009b80 comp rx=0 tx=0).stop 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.024+0000 7f3d014ac700 1 -- 172.21.15.83:0/1926665030 shutdown_connections 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.024+0000 7f3d014ac700 1 --2- 172.21.15.83:0/1926665030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cfc073f90 0x7f3cfc06f380 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.024+0000 7f3d014ac700 1 -- 172.21.15.83:0/1926665030 >> 172.21.15.83:0/1926665030 conn(0x7f3cfc0d0c40 msgr2=0x7f3cfc0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:10.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.024+0000 7f3d014ac700 1 -- 172.21.15.83:0/1926665030 shutdown_connections 2023-11-26T07:16:10.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.024+0000 7f3d014ac700 1 -- 172.21.15.83:0/1926665030 wait complete. 2023-11-26T07:16:10.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.025+0000 7f3d014ac700 1 Processor -- start 2023-11-26T07:16:10.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.025+0000 7f3d014ac700 1 -- start start 2023-11-26T07:16:10.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.026+0000 7f3d014ac700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cfc073f90 0x7f3cfc1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:10.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.026+0000 7f3d014ac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cfc074370 0x7f3cfc159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:10.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.026+0000 7f3d014ac700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cfc15e1b0 0x7f3cfc159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:10.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.026+0000 7f3d014ac700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc159dd0 con 0x7f3cfc073f90 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.026+0000 7f3d014ac700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc158590 con 0x7f3cfc074370 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.026+0000 7f3d014ac700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc158870 con 0x7f3cfc15e1b0 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.026+0000 7f3cfaffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cfc073f90 0x7f3cfc1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.026+0000 7f3cfaffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cfc073f90 0x7f3cfc1538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40312/0 (socket says 172.21.15.83:40312) 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.026+0000 7f3cfaffd700 1 -- 172.21.15.83:0/4293781977 learned_addr learned my addr 172.21.15.83:0/4293781977 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3cfb7fe700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cfc15e1b0 0x7f3cfc159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3cfaffd700 1 -- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cfc15e1b0 msgr2=0x7f3cfc159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3cfaffd700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cfc15e1b0 0x7f3cfc159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3cfa7fc700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cfc074370 0x7f3cfc159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3cfaffd700 1 -- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cfc074370 msgr2=0x7f3cfc159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3cfaffd700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cfc074370 0x7f3cfc159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3cfaffd700 1 -- 172.21.15.83:0/4293781977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec0094a0 con 0x7f3cfc073f90 2023-11-26T07:16:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3cfaffd700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cfc073f90 0x7f3cfc1538f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3cec000c00 tx=0x7f3cec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3ce37fe700 1 -- 172.21.15.83:0/4293781977 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec011630 con 0x7f3cfc073f90 2023-11-26T07:16:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3ce37fe700 1 -- 172.21.15.83:0/4293781977 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cec01f070 con 0x7f3cfc073f90 2023-11-26T07:16:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3ce37fe700 1 -- 172.21.15.83:0/4293781977 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec0119f0 con 0x7f3cfc073f90 2023-11-26T07:16:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3d014ac700 1 -- 172.21.15.83:0/4293781977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc158b50 con 0x7f3cfc073f90 2023-11-26T07:16:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.027+0000 7f3d014ac700 1 -- 172.21.15.83:0/4293781977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc158240 con 0x7f3cfc073f90 2023-11-26T07:16:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.028+0000 7f3d014ac700 1 -- 172.21.15.83:0/4293781977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc0d0090 con 0x7f3cfc073f90 2023-11-26T07:16:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.028+0000 7f3ce37fe700 1 -- 172.21.15.83:0/4293781977 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f3cec011b50 con 0x7f3cfc073f90 2023-11-26T07:16:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.029+0000 7f3ce37fe700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ce806aff0 0x7f3ce806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.029+0000 7f3ce37fe700 1 -- 172.21.15.83:0/4293781977 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f3cec021070 con 0x7f3cfc073f90 2023-11-26T07:16:10.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.029+0000 7f3cfa7fc700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ce806aff0 0x7f3ce806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:10.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.029+0000 7f3cfa7fc700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ce806aff0 0x7f3ce806d4b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00010e0 tx=0x7f3cf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:10.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.033+0000 7f3ce37fe700 1 -- 172.21.15.83:0/4293781977 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3cfc0d0090 con 0x7f3cfc073f90 2023-11-26T07:16:10.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.220+0000 7f3d014ac700 1 -- 172.21.15.83:0/4293781977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3cfc0d0090 con 0x7f3cfc073f90 2023-11-26T07:16:10.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.221+0000 7f3ce37fe700 1 -- 172.21.15.83:0/4293781977 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3cfc0d0090 con 0x7f3cfc073f90 2023-11-26T07:16:10.223 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:16:10.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 -- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ce806aff0 msgr2=0x7f3ce806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:10.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ce806aff0 0x7f3ce806d4b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00010e0 tx=0x7f3cf0009040 comp rx=0 tx=0).stop 2023-11-26T07:16:10.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 -- 172.21.15.83:0/4293781977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cfc073f90 msgr2=0x7f3cfc1538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:10.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cfc073f90 0x7f3cfc1538f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3cec000c00 tx=0x7f3cec004750 comp rx=0 tx=0).stop 2023-11-26T07:16:10.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 -- 172.21.15.83:0/4293781977 shutdown_connections 2023-11-26T07:16:10.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ce806aff0 0x7f3ce806d4b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cfc15e1b0 0x7f3cfc159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cfc073f90 0x7f3cfc1538f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 --2- 172.21.15.83:0/4293781977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cfc074370 0x7f3cfc159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 -- 172.21.15.83:0/4293781977 >> 172.21.15.83:0/4293781977 conn(0x7f3cfc0d0c40 msgr2=0x7f3cfc0adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:10.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 -- 172.21.15.83:0/4293781977 shutdown_connections 2023-11-26T07:16:10.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.223+0000 7f3d014ac700 1 -- 172.21.15.83:0/4293781977 wait complete. 2023-11-26T07:16:10.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2023-11-26T07:16:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.355+0000 7f890e672700 1 Processor -- start 2023-11-26T07:16:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.355+0000 7f890e672700 1 -- start start 2023-11-26T07:16:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.356+0000 7f890e672700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89080ce140 0x7f890815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.356+0000 7f890e672700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f890815f630 0x7f890806b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.356+0000 7f890e672700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89081414c0 con 0x7f890815af10 2023-11-26T07:16:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.356+0000 7f890e672700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f890815c300 con 0x7f89080ce140 2023-11-26T07:16:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.356+0000 7f890e672700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89080ce9f0 con 0x7f890815f630 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.356+0000 7f89077fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89080ce140 0x7f890815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.356+0000 7f89077fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f89080ce140 0x7f890815f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40322/0 (socket says 172.21.15.83:40322) 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.356+0000 7f89077fe700 1 -- 172.21.15.83:0/619706661 learned_addr learned my addr 172.21.15.83:0/619706661 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.357+0000 7f890cc0f700 1 --2- 172.21.15.83:0/619706661 >> v2:172.21.15.203:3300/0 conn(0x7f890815f630 0x7f890806b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.357+0000 7f89077fe700 1 -- 172.21.15.83:0/619706661 >> v2:172.21.15.203:3300/0 conn(0x7f890815f630 msgr2=0x7f890806b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.357+0000 7f89077fe700 1 --2- 172.21.15.83:0/619706661 >> v2:172.21.15.203:3300/0 conn(0x7f890815f630 0x7f890806b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.357+0000 7f89077fe700 1 -- 172.21.15.83:0/619706661 >> v1:172.21.15.203:6789/0 conn(0x7f890815af10 legacy=0x7f890815e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.357+0000 7f89077fe700 1 -- 172.21.15.83:0/619706661 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89080cecd0 con 0x7f89080ce140 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.357+0000 7f89077fe700 1 --2- 172.21.15.83:0/619706661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89080ce140 0x7f890815f0f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f89080cffd0 tx=0x7f88f800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d638eda50e16a0c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.357+0000 7f8906ffd700 1 -- 172.21.15.83:0/619706661 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f800dcf0 con 0x7f89080ce140 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.357+0000 7f8906ffd700 1 -- 172.21.15.83:0/619706661 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f800de50 con 0x7f89080ce140 2023-11-26T07:16:10.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.357+0000 7f8906ffd700 1 -- 172.21.15.83:0/619706661 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f8004250 con 0x7f89080ce140 2023-11-26T07:16:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.358+0000 7f890e672700 1 -- 172.21.15.83:0/619706661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89080ce140 msgr2=0x7f890815f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.358+0000 7f890e672700 1 --2- 172.21.15.83:0/619706661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89080ce140 0x7f890815f0f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f89080cffd0 tx=0x7f88f800be70 comp rx=0 tx=0).stop 2023-11-26T07:16:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.358+0000 7f890e672700 1 -- 172.21.15.83:0/619706661 shutdown_connections 2023-11-26T07:16:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.358+0000 7f890e672700 1 --2- 172.21.15.83:0/619706661 >> v2:172.21.15.203:3300/0 conn(0x7f890815f630 0x7f890806b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.358+0000 7f890e672700 1 --2- 172.21.15.83:0/619706661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89080ce140 0x7f890815f0f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.358+0000 7f890e672700 1 -- 172.21.15.83:0/619706661 >> 172.21.15.83:0/619706661 conn(0x7f89080cd570 msgr2=0x7f89080cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.358+0000 7f890e672700 1 -- 172.21.15.83:0/619706661 shutdown_connections 2023-11-26T07:16:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.358+0000 7f890e672700 1 -- 172.21.15.83:0/619706661 wait complete. 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.359+0000 7f890e672700 1 Processor -- start 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.359+0000 7f890e672700 1 -- start start 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890e672700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89080ce140 0x7f89080b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890e672700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f890815af10 0x7f890814c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890e672700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8908159550 0x7f89081599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890e672700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8908159f20 con 0x7f8908159550 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890e672700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890815d510 con 0x7f890815af10 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890e672700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890815c900 con 0x7f89080ce140 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890cc0f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8908159550 0x7f89081599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890cc0f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8908159550 0x7f89081599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40330/0 (socket says 172.21.15.83:40330) 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890cc0f700 1 -- 172.21.15.83:0/920076580 learned_addr learned my addr 172.21.15.83:0/920076580 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f8907fff700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89080ce140 0x7f89080b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890cc0f700 1 -- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89080ce140 msgr2=0x7f89080b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f89077fe700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f890815af10 0x7f890814c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890cc0f700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89080ce140 0x7f89080b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890cc0f700 1 -- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f890815af10 msgr2=0x7f890814c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890cc0f700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f890815af10 0x7f890814c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890cc0f700 1 -- 172.21.15.83:0/920076580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f800b7f0 con 0x7f8908159550 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890cc0f700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8908159550 0x7f89081599e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8900007e90 tx=0x7f890000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f8904ff9700 1 -- 172.21.15.83:0/920076580 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890000ede0 con 0x7f8908159550 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f8904ff9700 1 -- 172.21.15.83:0/920076580 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8900014420 con 0x7f8908159550 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890e672700 1 -- 172.21.15.83:0/920076580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890815cbe0 con 0x7f8908159550 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f8904ff9700 1 -- 172.21.15.83:0/920076580 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89000147d0 con 0x7f8908159550 2023-11-26T07:16:10.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.360+0000 7f890e672700 1 -- 172.21.15.83:0/920076580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890815d1a0 con 0x7f8908159550 2023-11-26T07:16:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.361+0000 7f890e672700 1 -- 172.21.15.83:0/920076580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8908067f60 con 0x7f8908159550 2023-11-26T07:16:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.362+0000 7f8904ff9700 1 -- 172.21.15.83:0/920076580 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f8900014930 con 0x7f8908159550 2023-11-26T07:16:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.362+0000 7f8904ff9700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f88f406aff0 0x7f88f406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.362+0000 7f8904ff9700 1 -- 172.21.15.83:0/920076580 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f8900089e10 con 0x7f8908159550 2023-11-26T07:16:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.362+0000 7f8907fff700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f88f406aff0 0x7f88f406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.363+0000 7f8907fff700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f88f406aff0 0x7f88f406d4b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88fc003e60 tx=0x7f88fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.367+0000 7f8904ff9700 1 -- 172.21.15.83:0/920076580 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8900054e80 con 0x7f8908159550 2023-11-26T07:16:10.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:10.640+0000 7f890e672700 1 -- 172.21.15.83:0/920076580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7f8908072e00 con 0x7f8908159550 2023-11-26T07:16:11.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.552+0000 7f8904ff9700 1 -- 172.21.15.83:0/920076580 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v258) v1 ==== 149+0+0 (secure 0 0 0) 0x7f89000584a0 con 0x7f8908159550 2023-11-26T07:16:11.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.555+0000 7f890e672700 1 -- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f88f406aff0 msgr2=0x7f88f406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.555+0000 7f890e672700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f88f406aff0 0x7f88f406d4b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88fc003e60 tx=0x7f88fc009040 comp rx=0 tx=0).stop 2023-11-26T07:16:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.555+0000 7f890e672700 1 -- 172.21.15.83:0/920076580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8908159550 msgr2=0x7f89081599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.555+0000 7f890e672700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8908159550 0x7f89081599e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8900007e90 tx=0x7f890000e3c0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.556+0000 7f890e672700 1 -- 172.21.15.83:0/920076580 shutdown_connections 2023-11-26T07:16:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.556+0000 7f890e672700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f88f406aff0 0x7f88f406d4b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.556+0000 7f890e672700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8908159550 0x7f89081599e0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.556+0000 7f890e672700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89080ce140 0x7f89080b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.556+0000 7f890e672700 1 --2- 172.21.15.83:0/920076580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f890815af10 0x7f890814c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.556+0000 7f890e672700 1 -- 172.21.15.83:0/920076580 >> 172.21.15.83:0/920076580 conn(0x7f89080cd570 msgr2=0x7f89080cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.556+0000 7f890e672700 1 -- 172.21.15.83:0/920076580 shutdown_connections 2023-11-26T07:16:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.556+0000 7f890e672700 1 -- 172.21.15.83:0/920076580 wait complete. 2023-11-26T07:16:11.573 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:16:11.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.700+0000 7fdca2963700 1 Processor -- start 2023-11-26T07:16:11.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.700+0000 7fdca2963700 1 -- start start 2023-11-26T07:16:11.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.701+0000 7fdca2963700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc9c15e160 0x7fdc9c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:11.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.701+0000 7fdca2963700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c0746a0 0x7fdc9c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:11.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.701+0000 7fdca2963700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc9c0b3c60 con 0x7fdc9c0afc90 2023-11-26T07:16:11.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.701+0000 7fdca2963700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9c0b4500 con 0x7fdc9c15e160 2023-11-26T07:16:11.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.701+0000 7fdca2963700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0b58c0 con 0x7fdc9c0746a0 2023-11-26T07:16:11.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.701+0000 7fdc9b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdc9c0afc90 0x7fdc9c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60374/0 (socket says 172.21.15.83:60374) 2023-11-26T07:16:11.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.701+0000 7fdc9b7fe700 1 -- 172.21.15.83:0/938680192 learned_addr learned my addr 172.21.15.83:0/938680192 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.701+0000 7fdc9bfff700 1 --2- 172.21.15.83:0/938680192 >> v2:172.21.15.203:3300/0 conn(0x7fdc9c15e160 0x7fdc9c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.701+0000 7fdca0f00700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c0746a0 0x7fdc9c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.701+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245756447 0 0) 0x7fdc9c0b3c60 con 0x7fdc9c0afc90 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.701+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc840035b0 con 0x7fdc9c0afc90 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 219014575 0 0) 0x7fdc840035b0 con 0x7fdc9c0afc90 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c0746a0 msgr2=0x7fdc9c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c0746a0 0x7fdc9c06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 >> v2:172.21.15.203:3300/0 conn(0x7fdc9c15e160 msgr2=0x7fdc9c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 --2- 172.21.15.83:0/938680192 >> v2:172.21.15.203:3300/0 conn(0x7fdc9c15e160 0x7fdc9c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c0b0140 con 0x7fdc9c0afc90 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fdc8c001670 con 0x7fdc9c0afc90 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9bfff700 1 --2- 172.21.15.83:0/938680192 >> v2:172.21.15.203:3300/0 conn(0x7fdc9c15e160 0x7fdc9c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 >> v1:172.21.15.83:6789/0 conn(0x7fdc9c0afc90 legacy=0x7fdc9c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:11.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c0746a0 0x7fdc84003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc84003f80 0x7fdc84004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc84004d90 0x7fdc84005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0b4500 con 0x7fdc84004d90 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0b58c0 con 0x7fdc84003f80 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8400ba10 con 0x7fdc9c0746a0 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdca0f00700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc84004d90 0x7fdc84005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9bfff700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c0746a0 0x7fdc84003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9b7fe700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc84003f80 0x7fdc84004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdca0f00700 1 -- 172.21.15.83:0/938680192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c0746a0 msgr2=0x7fdc84003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdca0f00700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c0746a0 0x7fdc84003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdca0f00700 1 -- 172.21.15.83:0/938680192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc84003f80 msgr2=0x7fdc84004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:11.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdca0f00700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc84003f80 0x7fdc84004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdca0f00700 1 -- 172.21.15.83:0/938680192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc840035b0 con 0x7fdc84004d90 2023-11-26T07:16:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdca0f00700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc84004d90 0x7fdc84005210 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009f90 tx=0x7fdc84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672284c0f9ead4d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400ebd0 con 0x7fdc84004d90 2023-11-26T07:16:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9400f040 con 0x7fdc84004d90 2023-11-26T07:16:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.702+0000 7fdc9affd700 1 -- 172.21.15.83:0/938680192 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94014480 con 0x7fdc84004d90 2023-11-26T07:16:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.703+0000 7fdca2963700 1 -- 172.21.15.83:0/938680192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc84004d90 msgr2=0x7fdc84005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.703+0000 7fdca2963700 1 --2- 172.21.15.83:0/938680192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc84004d90 0x7fdc84005210 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009f90 tx=0x7fdc84005880 comp rx=0 tx=0).stop 2023-11-26T07:16:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.703+0000 7fdca0f00700 1 -- 172.21.15.83:0/938680192 reap_dead start 2023-11-26T07:16:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.703+0000 7fdca2963700 1 -- 172.21.15.83:0/938680192 shutdown_connections 2023-11-26T07:16:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.703+0000 7fdca2963700 1 -- 172.21.15.83:0/938680192 >> 172.21.15.83:0/938680192 conn(0x7fdc9c0d0bf0 msgr2=0x7fdc9c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:11.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.703+0000 7fdca2963700 1 -- 172.21.15.83:0/938680192 shutdown_connections 2023-11-26T07:16:11.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.703+0000 7fdca2963700 1 -- 172.21.15.83:0/938680192 wait complete. 2023-11-26T07:16:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.704+0000 7fdca2963700 1 Processor -- start 2023-11-26T07:16:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.704+0000 7fdca2963700 1 -- start start 2023-11-26T07:16:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca2963700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc9c15aa10 0x7fdc9c15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:11.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca2963700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c15adf0 0x7fdc9c1f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca2963700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c0adfa0 0x7fdc9c0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca2963700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0ae990 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca2963700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c1f4fb0 con 0x7fdc9c15aa10 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca2963700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c1f7750 con 0x7fdc9c15adf0 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca0f00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c0adfa0 0x7fdc9c0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca0f00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c0adfa0 0x7fdc9c0ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40348/0 (socket says 172.21.15.83:40348) 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca0f00700 1 -- 172.21.15.83:0/2916717513 learned_addr learned my addr 172.21.15.83:0/2916717513 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdc9bfff700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc9c15aa10 0x7fdc9c15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdc9b7fe700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c15adf0 0x7fdc9c1f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca0f00700 1 -- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c15adf0 msgr2=0x7fdc9c1f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca0f00700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c15adf0 0x7fdc9c1f4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.705+0000 7fdca0f00700 1 -- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc9c15aa10 msgr2=0x7fdc9c15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:11.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.706+0000 7fdca0f00700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc9c15aa10 0x7fdc9c15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.706+0000 7fdca0f00700 1 -- 172.21.15.83:0/2916717513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c1f7a30 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.706+0000 7fdc9bfff700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc9c15aa10 0x7fdc9c15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.706+0000 7fdca0f00700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c0adfa0 0x7fdc9c0ae450 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdc840046f0 tx=0x7fdc94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.706+0000 7fdc98ff9700 1 -- 172.21.15.83:0/2916717513 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94007000 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.706+0000 7fdc98ff9700 1 -- 172.21.15.83:0/2916717513 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc94003e70 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.706+0000 7fdc98ff9700 1 -- 172.21.15.83:0/2916717513 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94015ac0 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.706+0000 7fdca2963700 1 -- 172.21.15.83:0/2916717513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c1f7d10 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.706+0000 7fdca2963700 1 -- 172.21.15.83:0/2916717513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c1f82d0 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.707+0000 7fdca2963700 1 -- 172.21.15.83:0/2916717513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9c1f8460 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.707+0000 7fdc98ff9700 1 -- 172.21.15.83:0/2916717513 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fdc94024070 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.708+0000 7fdc98ff9700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc8806aff0 0x7fdc8806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:11.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.708+0000 7fdc98ff9700 1 -- 172.21.15.83:0/2916717513 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fdc9408aab0 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.708+0000 7fdc9bfff700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc8806aff0 0x7fdc8806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.709+0000 7fdc9bfff700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc8806aff0 0x7fdc8806d4b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0d0310 tx=0x7fdc90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:11.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.712+0000 7fdc98ff9700 1 -- 172.21.15.83:0/2916717513 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc94059760 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.905 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.903+0000 7fdca2963700 1 -- 172.21.15.83:0/2916717513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdc9c068fb0 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.906+0000 7fdc98ff9700 1 -- 172.21.15.83:0/2916717513 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdc94018020 con 0x7fdc9c0adfa0 2023-11-26T07:16:11.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.908+0000 7fdca2963700 1 -- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc8806aff0 msgr2=0x7fdc8806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:11.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.908+0000 7fdca2963700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc8806aff0 0x7fdc8806d4b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0d0310 tx=0x7fdc90008040 comp rx=0 tx=0).stop 2023-11-26T07:16:11.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.908+0000 7fdca2963700 1 -- 172.21.15.83:0/2916717513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c0adfa0 msgr2=0x7fdc9c0ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:11.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.908+0000 7fdca2963700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c0adfa0 0x7fdc9c0ae450 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdc840046f0 tx=0x7fdc94009750 comp rx=0 tx=0).stop 2023-11-26T07:16:11.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.909+0000 7fdca2963700 1 -- 172.21.15.83:0/2916717513 shutdown_connections 2023-11-26T07:16:11.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.909+0000 7fdca2963700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc8806aff0 0x7fdc8806d4b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.909+0000 7fdca2963700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c0adfa0 0x7fdc9c0ae450 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.909+0000 7fdca2963700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc9c15aa10 0x7fdc9c15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.909+0000 7fdca2963700 1 --2- 172.21.15.83:0/2916717513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc9c15adf0 0x7fdc9c1f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:11.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.909+0000 7fdca2963700 1 -- 172.21.15.83:0/2916717513 >> 172.21.15.83:0/2916717513 conn(0x7fdc9c0d0bf0 msgr2=0x7fdc9c161ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:11.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.909+0000 7fdca2963700 1 -- 172.21.15.83:0/2916717513 shutdown_connections 2023-11-26T07:16:11.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:11.909+0000 7fdca2963700 1 -- 172.21.15.83:0/2916717513 wait complete. 2023-11-26T07:16:11.923 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:16:11.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:16:11.974 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:16:12.021 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:16:12.022 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:16:12.022 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:12.039 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:16:12.039 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:16:12.039 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:12.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:16:12.058 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:16:12.058 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:12.096 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:16:12.096 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:16:12.096 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:12.117 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:16:12.117 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:16:12.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:12.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.254+0000 7f7d815b1700 1 Processor -- start 2023-11-26T07:16:12.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.254+0000 7f7d815b1700 1 -- start start 2023-11-26T07:16:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d815b1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c0d2250 0x7f7d7c168c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d815b1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c0d0ee0 con 0x7f7d7c15e090 2023-11-26T07:16:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d815b1700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c0d02d0 con 0x7f7d7c0cfd80 2023-11-26T07:16:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d815b1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0d05b0 con 0x7f7d7c0d2250 2023-11-26T07:16:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d7affd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7d7c15e090 0x7f7d7c06bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60392/0 (socket says 172.21.15.83:60392) 2023-11-26T07:16:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d7affd700 1 -- 172.21.15.83:0/1288123879 learned_addr learned my addr 172.21.15.83:0/1288123879 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d7b7fe700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c0d2250 0x7f7d7c168c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426391918 0 0) 0x7f7d7c0d0ee0 con 0x7f7d7c15e090 2023-11-26T07:16:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d640035b0 con 0x7f7d7c15e090 2023-11-26T07:16:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4065872826 0 0) 0x7f7d640035b0 con 0x7f7d7c15e090 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c0d2250 msgr2=0x7f7d7c168c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c0d2250 0x7f7d7c168c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 >> v1:172.21.15.203:6789/0 conn(0x7f7d7c0cfd80 legacy=0x7f7d7c0d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c0d0890 con 0x7f7d7c15e090 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f7d6c001670 con 0x7f7d7c15e090 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 >> v1:172.21.15.83:6789/0 conn(0x7f7d7c15e090 legacy=0x7f7d7c06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 0x7f7d64003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d64004850 0x7f7d64004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c0d2250 0x7f7d64005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0d05b0 con 0x7f7d64003a30 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6400bbb0 con 0x7f7d64004850 2023-11-26T07:16:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6400bd70 con 0x7f7d7c0d2250 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.255+0000 7f7d7affd700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 0x7f7d64003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d7b7fe700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c0d2250 0x7f7d64005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d7a7fc700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d64004850 0x7f7d64004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d7affd700 1 -- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c0d2250 msgr2=0x7f7d64005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d7affd700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c0d2250 0x7f7d64005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d7affd700 1 -- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d64004850 msgr2=0x7f7d64004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d7affd700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d64004850 0x7f7d64004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d7affd700 1 -- 172.21.15.83:0/1288123879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d640035b0 con 0x7f7d64003a30 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d7b7fe700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c0d2250 0x7f7d64005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d7affd700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 0x7f7d64003e70 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00c270 tx=0x7f7d64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b9b3f182c65823 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c007c10 con 0x7f7d64003a30 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c00cdb0 con 0x7f7d64003a30 2023-11-26T07:16:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d79ffb700 1 -- 172.21.15.83:0/1288123879 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c005ba0 con 0x7f7d64003a30 2023-11-26T07:16:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d815b1700 1 -- 172.21.15.83:0/1288123879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 msgr2=0x7f7d64003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d815b1700 1 --2- 172.21.15.83:0/1288123879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 0x7f7d64003e70 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00c270 tx=0x7f7d64005880 comp rx=0 tx=0).stop 2023-11-26T07:16:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.256+0000 7f7d7b7fe700 1 -- 172.21.15.83:0/1288123879 reap_dead start 2023-11-26T07:16:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.257+0000 7f7d815b1700 1 -- 172.21.15.83:0/1288123879 shutdown_connections 2023-11-26T07:16:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.257+0000 7f7d815b1700 1 -- 172.21.15.83:0/1288123879 >> 172.21.15.83:0/1288123879 conn(0x7f7d7c0731a0 msgr2=0x7f7d7c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.257+0000 7f7d815b1700 1 -- 172.21.15.83:0/1288123879 shutdown_connections 2023-11-26T07:16:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.257+0000 7f7d815b1700 1 -- 172.21.15.83:0/1288123879 wait complete. 2023-11-26T07:16:12.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.258+0000 7f7d815b1700 1 Processor -- start 2023-11-26T07:16:12.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.258+0000 7f7d815b1700 1 -- start start 2023-11-26T07:16:12.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d815b1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 0x7f7d7c1f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d815b1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d7c0ccb90 0x7f7d7c1f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d815b1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c156940 0x7f7d7c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d815b1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0c8180 con 0x7f7d64003a30 2023-11-26T07:16:12.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d815b1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0c8460 con 0x7f7d7c0ccb90 2023-11-26T07:16:12.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d815b1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0ce320 con 0x7f7d7c156940 2023-11-26T07:16:12.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d7affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 0x7f7d7c1f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d7affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 0x7f7d7c1f7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40364/0 (socket says 172.21.15.83:40364) 2023-11-26T07:16:12.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d7affd700 1 -- 172.21.15.83:0/611670269 learned_addr learned my addr 172.21.15.83:0/611670269 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:12.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d7a7fc700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d7c0ccb90 0x7f7d7c1f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d7b7fe700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c156940 0x7f7d7c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d7affd700 1 -- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c156940 msgr2=0x7f7d7c156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d7affd700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c156940 0x7f7d7c156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d7affd700 1 -- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d7c0ccb90 msgr2=0x7f7d7c1f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d7affd700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d7c0ccb90 0x7f7d7c1f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d7affd700 1 -- 172.21.15.83:0/611670269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c0ce600 con 0x7f7d64003a30 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d7affd700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 0x7f7d7c1f7200 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400bb80 tx=0x7f7d7c0d1370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d637fe700 1 -- 172.21.15.83:0/611670269 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c001630 con 0x7f7d64003a30 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d637fe700 1 -- 172.21.15.83:0/611670269 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c0070d0 con 0x7f7d64003a30 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d815b1700 1 -- 172.21.15.83:0/611670269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c0c7da0 con 0x7f7d64003a30 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.259+0000 7f7d637fe700 1 -- 172.21.15.83:0/611670269 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c0074a0 con 0x7f7d64003a30 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.260+0000 7f7d815b1700 1 -- 172.21.15.83:0/611670269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c1f7e60 con 0x7f7d64003a30 2023-11-26T07:16:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.260+0000 7f7d815b1700 1 -- 172.21.15.83:0/611670269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c0751e0 con 0x7f7d64003a30 2023-11-26T07:16:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.261+0000 7f7d637fe700 1 -- 172.21.15.83:0/611670269 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f7d6c019030 con 0x7f7d64003a30 2023-11-26T07:16:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.261+0000 7f7d637fe700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7d6806aff0 0x7f7d6806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.262+0000 7f7d637fe700 1 -- 172.21.15.83:0/611670269 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f7d6c08a8c0 con 0x7f7d64003a30 2023-11-26T07:16:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.262+0000 7f7d7a7fc700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7d6806aff0 0x7f7d6806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.262+0000 7f7d7a7fc700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7d6806aff0 0x7f7d6806d4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d70000c00 tx=0x7f7d70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.266+0000 7f7d637fe700 1 -- 172.21.15.83:0/611670269 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d6c0559c0 con 0x7f7d64003a30 2023-11-26T07:16:12.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.499+0000 7f7d815b1700 1 -- 172.21.15.83:0/611670269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d7c0c5540 con 0x7f7d64003a30 2023-11-26T07:16:12.502 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.500+0000 7f7d637fe700 1 -- 172.21.15.83:0/611670269 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f7d6c058fe0 con 0x7f7d64003a30 2023-11-26T07:16:12.502 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:12.502 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5420,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/2163151849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6835","nonce":2163151849},{"type":"v1","addr":"172.21.15.83:6837","nonce":2163151849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1539108478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1539108478},{"type":"v1","addr":"172.21.15.203:6835","nonce":1539108478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6004,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1387613510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1387613510},{"type":"v1","addr":"172.21.15.203:6834","nonce":1387613510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7029,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/1238234927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":1238234927},{"type":"v1","addr":"172.21.15.83:6833","nonce":1238234927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[]} 2023-11-26T07:16:12.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 -- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7d6806aff0 msgr2=0x7f7d6806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7d6806aff0 0x7f7d6806d4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d70000c00 tx=0x7f7d70009040 comp rx=0 tx=0).stop 2023-11-26T07:16:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 -- 172.21.15.83:0/611670269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 msgr2=0x7f7d7c1f7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 0x7f7d7c1f7200 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400bb80 tx=0x7f7d7c0d1370 comp rx=0 tx=0).stop 2023-11-26T07:16:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 -- 172.21.15.83:0/611670269 shutdown_connections 2023-11-26T07:16:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7d6806aff0 0x7f7d6806d4b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d7c156940 0x7f7d7c156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d64003a30 0x7f7d7c1f7200 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 --2- 172.21.15.83:0/611670269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d7c0ccb90 0x7f7d7c1f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 -- 172.21.15.83:0/611670269 >> 172.21.15.83:0/611670269 conn(0x7f7d7c0731a0 msgr2=0x7f7d7c0cad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 -- 172.21.15.83:0/611670269 shutdown_connections 2023-11-26T07:16:12.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.502+0000 7f7d815b1700 1 -- 172.21.15.83:0/611670269 wait complete. 2023-11-26T07:16:12.506 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 49 2023-11-26T07:16:12.517 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:16:12.517 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:16:12.517 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:16:12.517 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:16:12.518 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:16:12.518 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:16:12.518 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:16:12.518 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:16:12.518 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:16:12.518 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:16:12.519 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:16:12.519 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:16:12.519 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:16:12.519 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:16:12.519 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:16:12.519 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:16:12.526 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:16:12.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:16:12.528 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:16:12.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:16:12.534 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:16:12.534 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:16:12.535 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:16:12.535 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:16:12.570 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:16:12.570 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:12.571 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:16:12.572 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:12.576 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:16:12.577 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:12.577 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:16:12.577 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:12.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:16:12.649 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-26T07:16:12.649 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-26T07:16:12.663 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-11-26T07:16:12.664 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-11-26T07:16:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.709+0000 7f36927a2700 1 Processor -- start 2023-11-26T07:16:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.709+0000 7f36927a2700 1 -- start start 2023-11-26T07:16:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f36927a2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f368c13e000 0x7f368c15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f36927a2700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c0d03a0 con 0x7f368c0cff10 2023-11-26T07:16:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f36927a2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c0b4290 con 0x7f368c15efb0 2023-11-26T07:16:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f36927a2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c15f430 con 0x7f368c13e000 2023-11-26T07:16:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f3690d3f700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f368c0cff10 0x7f368c0b4720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60440/0 (socket says 172.21.15.83:60440) 2023-11-26T07:16:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f3690d3f700 1 -- 172.21.15.83:0/2266427697 learned_addr learned my addr 172.21.15.83:0/2266427697 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f368bfff700 1 --2- 172.21.15.83:0/2266427697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f368c13e000 0x7f368c15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f368affd700 1 -- 172.21.15.83:0/2266427697 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957621024 0 0) 0x7f368c0d03a0 con 0x7f368c0cff10 2023-11-26T07:16:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f368affd700 1 -- 172.21.15.83:0/2266427697 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36740035b0 con 0x7f368c0cff10 2023-11-26T07:16:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f368affd700 1 -- 172.21.15.83:0/2266427697 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236890602 0 0) 0x7f368c0b4290 con 0x7f368c15efb0 2023-11-26T07:16:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f368affd700 1 -- 172.21.15.83:0/2266427697 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f368c0d03a0 con 0x7f368c15efb0 2023-11-26T07:16:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f368bfff700 1 -- 172.21.15.83:0/2266427697 >> v1:172.21.15.203:6789/0 conn(0x7f368c15efb0 legacy=0x7f368c0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f368bfff700 1 -- 172.21.15.83:0/2266427697 >> v1:172.21.15.83:6789/0 conn(0x7f368c0cff10 legacy=0x7f368c0b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f368bfff700 1 -- 172.21.15.83:0/2266427697 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c13e570 con 0x7f368c13e000 2023-11-26T07:16:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.710+0000 7f368bfff700 1 --2- 172.21.15.83:0/2266427697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f368c13e000 0x7f368c15ea70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f367c009870 tx=0x7f367c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5a01f41704a54ed server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.711+0000 7f368affd700 1 -- 172.21.15.83:0/2266427697 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c011830 con 0x7f368c13e000 2023-11-26T07:16:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.711+0000 7f368affd700 1 -- 172.21.15.83:0/2266427697 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c011990 con 0x7f368c13e000 2023-11-26T07:16:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.711+0000 7f368affd700 1 -- 172.21.15.83:0/2266427697 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c011d20 con 0x7f368c13e000 2023-11-26T07:16:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.711+0000 7f36927a2700 1 -- 172.21.15.83:0/2266427697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f368c13e000 msgr2=0x7f368c15ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.711+0000 7f36927a2700 1 --2- 172.21.15.83:0/2266427697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f368c13e000 0x7f368c15ea70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f367c009870 tx=0x7f367c009b80 comp rx=0 tx=0).stop 2023-11-26T07:16:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.711+0000 7f36927a2700 1 -- 172.21.15.83:0/2266427697 shutdown_connections 2023-11-26T07:16:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.711+0000 7f36927a2700 1 --2- 172.21.15.83:0/2266427697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f368c13e000 0x7f368c15ea70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.711+0000 7f36927a2700 1 -- 172.21.15.83:0/2266427697 >> 172.21.15.83:0/2266427697 conn(0x7f368c072990 msgr2=0x7f368c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.711+0000 7f36927a2700 1 -- 172.21.15.83:0/2266427697 shutdown_connections 2023-11-26T07:16:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.711+0000 7f36927a2700 1 -- 172.21.15.83:0/2266427697 wait complete. 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.712+0000 7f36927a2700 1 Processor -- start 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.713+0000 7f36927a2700 1 -- start start 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.713+0000 7f36927a2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f368c0cff10 0x7f368c1415d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.713+0000 7f36927a2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f368c13e000 0x7f368c142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.713+0000 7f36927a2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f368c15efb0 0x7f368c06ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.713+0000 7f36927a2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1f03e0 con 0x7f368c15efb0 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.713+0000 7f36927a2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c06fe80 con 0x7f368c13e000 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.713+0000 7f36927a2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1ef500 con 0x7f368c0cff10 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.713+0000 7f3690d3f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f368c15efb0 0x7f368c06ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.713+0000 7f3690d3f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f368c15efb0 0x7f368c06ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40402/0 (socket says 172.21.15.83:40402) 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.713+0000 7f3690d3f700 1 -- 172.21.15.83:0/4105502542 learned_addr learned my addr 172.21.15.83:0/4105502542 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f368bfff700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f368c0cff10 0x7f368c1415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f368b7fe700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f368c13e000 0x7f368c142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f3690d3f700 1 -- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f368c0cff10 msgr2=0x7f368c1415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f3690d3f700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f368c0cff10 0x7f368c1415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f3690d3f700 1 -- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f368c13e000 msgr2=0x7f368c142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f3690d3f700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f368c13e000 0x7f368c142110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f3690d3f700 1 -- 172.21.15.83:0/4105502542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c0094a0 con 0x7f368c15efb0 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f3690d3f700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f368c15efb0 0x7f368c06ed10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f368400cba0 tx=0x7f368400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f3688ff9700 1 -- 172.21.15.83:0/4105502542 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368400aa40 con 0x7f368c15efb0 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f3688ff9700 1 -- 172.21.15.83:0/4105502542 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368400aba0 con 0x7f368c15efb0 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f3688ff9700 1 -- 172.21.15.83:0/4105502542 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684012400 con 0x7f368c15efb0 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f36927a2700 1 -- 172.21.15.83:0/4105502542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c1ef7d0 con 0x7f368c15efb0 2023-11-26T07:16:12.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.714+0000 7f36927a2700 1 -- 172.21.15.83:0/4105502542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c1efd90 con 0x7f368c15efb0 2023-11-26T07:16:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.715+0000 7f36927a2700 1 -- 172.21.15.83:0/4105502542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c067f60 con 0x7f368c15efb0 2023-11-26T07:16:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.716+0000 7f3688ff9700 1 -- 172.21.15.83:0/4105502542 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f368401b3b0 con 0x7f368c15efb0 2023-11-26T07:16:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.716+0000 7f3688ff9700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f367806aff0 0x7f367806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.716+0000 7f3688ff9700 1 -- 172.21.15.83:0/4105502542 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f36840893c0 con 0x7f368c15efb0 2023-11-26T07:16:12.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.717+0000 7f368bfff700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f367806aff0 0x7f367806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:12.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.718+0000 7f368bfff700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f367806aff0 0x7f367806d4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f367c0097c0 tx=0x7f367c01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:12.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.721+0000 7f3688ff9700 1 -- 172.21.15.83:0/4105502542 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36840543a0 con 0x7f368c15efb0 2023-11-26T07:16:12.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:12.935+0000 7f36927a2700 1 -- 172.21.15.83:0/4105502542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f368c1f0520 con 0x7f368c15efb0 2023-11-26T07:16:13.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.932+0000 7f3688ff9700 1 -- 172.21.15.83:0/4105502542 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) v1 ==== 97+0+0 (secure 0 0 0) 0x7f368c1f0520 con 0x7f368c15efb0 2023-11-26T07:16:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 -- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f367806aff0 msgr2=0x7f367806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f367806aff0 0x7f367806d4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f367c0097c0 tx=0x7f367c01c040 comp rx=0 tx=0).stop 2023-11-26T07:16:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 -- 172.21.15.83:0/4105502542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f368c15efb0 msgr2=0x7f368c06ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f368c15efb0 0x7f368c06ed10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f368400cba0 tx=0x7f368400ceb0 comp rx=0 tx=0).stop 2023-11-26T07:16:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 -- 172.21.15.83:0/4105502542 shutdown_connections 2023-11-26T07:16:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f367806aff0 0x7f367806d4b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f368c15efb0 0x7f368c06ed10 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f368c0cff10 0x7f368c1415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 --2- 172.21.15.83:0/4105502542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f368c13e000 0x7f368c142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 -- 172.21.15.83:0/4105502542 >> 172.21.15.83:0/4105502542 conn(0x7f368c072990 msgr2=0x7f368c15c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:13.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 -- 172.21.15.83:0/4105502542 shutdown_connections 2023-11-26T07:16:13.939 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:13.935+0000 7f36927a2700 1 -- 172.21.15.83:0/4105502542 wait complete. 2023-11-26T07:16:13.939 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2023-11-26T07:16:13.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:16:14.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.076+0000 7fb948eb6700 1 Processor -- start 2023-11-26T07:16:14.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.076+0000 7fb948eb6700 1 -- start start 2023-11-26T07:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb948eb6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb94415e080 0x7fb9440b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb948eb6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb9440af4d0 0x7fb9440afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb948eb6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9440ae430 con 0x7fb94406c000 2023-11-26T07:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb948eb6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9440af0b0 con 0x7fb94415e080 2023-11-26T07:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb948eb6700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9440b05a0 con 0x7fb9440af4d0 2023-11-26T07:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb94259c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb94415e080 0x7fb9440b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb942d9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb94406c000 0x7fb944073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60458/0 (socket says 172.21.15.83:60458) 2023-11-26T07:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb942d9d700 1 -- 172.21.15.83:0/1930947580 learned_addr learned my addr 172.21.15.83:0/1930947580 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb94259c700 1 -- 172.21.15.83:0/1930947580 >> v2:172.21.15.203:3300/0 conn(0x7fb9440af4d0 msgr2=0x7fb9440afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb94259c700 1 --2- 172.21.15.83:0/1930947580 >> v2:172.21.15.203:3300/0 conn(0x7fb9440af4d0 0x7fb9440afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb94259c700 1 -- 172.21.15.83:0/1930947580 >> v1:172.21.15.83:6789/0 conn(0x7fb94406c000 legacy=0x7fb944073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb94259c700 1 -- 172.21.15.83:0/1930947580 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440ae710 con 0x7fb94415e080 2023-11-26T07:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb94159a700 1 -- 172.21.15.83:0/1930947580 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330826810 0 0) 0x7fb9440ae430 con 0x7fb94406c000 2023-11-26T07:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb94259c700 1 --2- 172.21.15.83:0/1930947580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb94415e080 0x7fb9440b5050 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb94406e4e0 tx=0x7fb934009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1c5f5004fad1cb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb94159a700 1 -- 172.21.15.83:0/1930947580 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011830 con 0x7fb94415e080 2023-11-26T07:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb94159a700 1 -- 172.21.15.83:0/1930947580 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb934011990 con 0x7fb94415e080 2023-11-26T07:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.077+0000 7fb94159a700 1 -- 172.21.15.83:0/1930947580 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934011d20 con 0x7fb94415e080 2023-11-26T07:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.078+0000 7fb948eb6700 1 -- 172.21.15.83:0/1930947580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb94415e080 msgr2=0x7fb9440b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.078+0000 7fb948eb6700 1 --2- 172.21.15.83:0/1930947580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb94415e080 0x7fb9440b5050 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb94406e4e0 tx=0x7fb934009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.078+0000 7fb948eb6700 1 -- 172.21.15.83:0/1930947580 shutdown_connections 2023-11-26T07:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.078+0000 7fb948eb6700 1 --2- 172.21.15.83:0/1930947580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb94415e080 0x7fb9440b5050 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.078+0000 7fb948eb6700 1 --2- 172.21.15.83:0/1930947580 >> v2:172.21.15.203:3300/0 conn(0x7fb9440af4d0 0x7fb9440afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.078+0000 7fb948eb6700 1 -- 172.21.15.83:0/1930947580 >> 172.21.15.83:0/1930947580 conn(0x7fb944071b90 msgr2=0x7fb944071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.078+0000 7fb948eb6700 1 -- 172.21.15.83:0/1930947580 shutdown_connections 2023-11-26T07:16:14.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.078+0000 7fb948eb6700 1 -- 172.21.15.83:0/1930947580 wait complete. 2023-11-26T07:16:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.080+0000 7fb948eb6700 1 Processor -- start 2023-11-26T07:16:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.080+0000 7fb948eb6700 1 -- start start 2023-11-26T07:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb948eb6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb94406c000 0x7fb944144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb948eb6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9440af4d0 0x7fb944147700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb948eb6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb94415e080 0x7fb944147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb948eb6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb944145370 con 0x7fb94415e080 2023-11-26T07:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb948eb6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94415f6c0 con 0x7fb94406c000 2023-11-26T07:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb948eb6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94415f9a0 con 0x7fb9440af4d0 2023-11-26T07:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb942d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb94415e080 0x7fb944147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb942d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb94415e080 0x7fb944147c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40424/0 (socket says 172.21.15.83:40424) 2023-11-26T07:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb942d9d700 1 -- 172.21.15.83:0/3804410048 learned_addr learned my addr 172.21.15.83:0/3804410048 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb94259c700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb94406c000 0x7fb944144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb942d9d700 1 -- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9440af4d0 msgr2=0x7fb944147700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb941d9b700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9440af4d0 0x7fb944147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb942d9d700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9440af4d0 0x7fb944147700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb942d9d700 1 -- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb94406c000 msgr2=0x7fb944144e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb942d9d700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb94406c000 0x7fb944144e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb942d9d700 1 -- 172.21.15.83:0/3804410048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9340094a0 con 0x7fb94415e080 2023-11-26T07:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb942d9d700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb94415e080 0x7fb944147c40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00cca0 tx=0x7fb93c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb932ffd700 1 -- 172.21.15.83:0/3804410048 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00cea0 con 0x7fb94415e080 2023-11-26T07:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb932ffd700 1 -- 172.21.15.83:0/3804410048 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c00ade0 con 0x7fb94415e080 2023-11-26T07:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb948eb6700 1 -- 172.21.15.83:0/3804410048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94415fc80 con 0x7fb94415e080 2023-11-26T07:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb932ffd700 1 -- 172.21.15.83:0/3804410048 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c007310 con 0x7fb94415e080 2023-11-26T07:16:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.081+0000 7fb948eb6700 1 -- 172.21.15.83:0/3804410048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb944154760 con 0x7fb94415e080 2023-11-26T07:16:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.082+0000 7fb948eb6700 1 -- 172.21.15.83:0/3804410048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9440753f0 con 0x7fb94415e080 2023-11-26T07:16:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.083+0000 7fb932ffd700 1 -- 172.21.15.83:0/3804410048 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fb93c007470 con 0x7fb94415e080 2023-11-26T07:16:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.083+0000 7fb932ffd700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb92406aff0 0x7fb92406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.083+0000 7fb932ffd700 1 -- 172.21.15.83:0/3804410048 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb93c0892f0 con 0x7fb94415e080 2023-11-26T07:16:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.083+0000 7fb94259c700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb92406aff0 0x7fb92406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.084+0000 7fb94259c700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb92406aff0 0x7fb92406d4b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9340097c0 tx=0x7fb93401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.088+0000 7fb932ffd700 1 -- 172.21.15.83:0/3804410048 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb93c0545e0 con 0x7fb94415e080 2023-11-26T07:16:14.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.353+0000 7fb948eb6700 1 -- 172.21.15.83:0/3804410048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb9441456a0 con 0x7fb94415e080 2023-11-26T07:16:14.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.354+0000 7fb932ffd700 1 -- 172.21.15.83:0/3804410048 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb93c057c00 con 0x7fb94415e080 2023-11-26T07:16:14.356 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:14.356 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:16:14.356 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2023-11-26T07:16:14.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.357 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2023-11-26T07:16:14.357 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlA8pxNxAAt5NuewAjm7yPThCguJ6sEA==", 2023-11-26T07:16:14.357 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.357 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:14.357 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:16:14.357 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:16:14.357 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.357 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2023-11-26T07:16:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlHBgDCBAAk6egig5krY7kiMiHkMKrWg==", 2023-11-26T07:16:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:16:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:16:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2023-11-26T07:16:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlbKEQAxAAM2tBBQwM+tt2d8JU37YURg==", 2023-11-26T07:16:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:16:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:16:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2023-11-26T07:16:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlI3ZHDxAArtLmBVGr/vAB/YGwEFfhGw==", 2023-11-26T07:16:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:16:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:16:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2023-11-26T07:16:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBo72JlhuSTKRAA6LRDOHBQVl5TL5tIHzyfzA==", 2023-11-26T07:16:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2023-11-26T07:16:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBr72JlteU6KRAAuuHTYosYpl+KptywVmaeHA==", 2023-11-26T07:16:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2023-11-26T07:16:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBu72JlEb6dKBAAebp4pnu9XZsSRJpojTZcvQ==", 2023-11-26T07:16:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2023-11-26T07:16:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBx72JlbTJ3KBAA1JUxxAoyCzTtfgubUxSJEA==", 2023-11-26T07:16:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB372JlufqqLhAAm6ZUt9juWj5yAIDlnCXqWg==", 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB672Jl1L3vLRAAX06GcsntcmB+7L1CsfnJzw==", 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB972Jlv41HLRAALtuyuREvGmTuDZt/RT0UcA==", 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2023-11-26T07:16:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCA72JlKCHsLBAAAXDzA7AN36wcLjhK97nlkg==", 2023-11-26T07:16:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ==", 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JleNpIGhAAKwoQj4zB9YOxpFMD28juCA==", 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2023-11-26T07:16:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JldAHlFhAAb3u9KljU9vssDl54zgno2Q==", 2023-11-26T07:16:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:16:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:16:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:16:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2023-11-26T07:16:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JluiQUIBAA06mqpeCzgpbSLo2/fulfmw==", 2023-11-26T07:16:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:16:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:16:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:16:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2023-11-26T07:16:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlD4jbGhAA05xtSS5EzQTuhBjVqS2JpA==", 2023-11-26T07:16:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlsA0VNBAAfSlnVquhI+vpLFwmgLGGVA==", 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:16:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72Jl3qkjMBAA+jcxYuDG/UHhB7W7HDwmaw==", 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:14.373 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:16:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:16:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 -- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb92406aff0 msgr2=0x7fb92406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb92406aff0 0x7fb92406d4b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9340097c0 tx=0x7fb93401c040 comp rx=0 tx=0).stop 2023-11-26T07:16:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 -- 172.21.15.83:0/3804410048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb94415e080 msgr2=0x7fb944147c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb94415e080 0x7fb944147c40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00cca0 tx=0x7fb93c00a340 comp rx=0 tx=0).stop 2023-11-26T07:16:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 -- 172.21.15.83:0/3804410048 shutdown_connections 2023-11-26T07:16:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb92406aff0 0x7fb92406d4b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb94415e080 0x7fb944147c40 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb94406c000 0x7fb944144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 --2- 172.21.15.83:0/3804410048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9440af4d0 0x7fb944147700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 -- 172.21.15.83:0/3804410048 >> 172.21.15.83:0/3804410048 conn(0x7fb944071b90 msgr2=0x7fb944073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 -- 172.21.15.83:0/3804410048 shutdown_connections 2023-11-26T07:16:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.356+0000 7fb948eb6700 1 -- 172.21.15.83:0/3804410048 wait complete. 2023-11-26T07:16:14.376 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.493+0000 7fcfc569f700 1 Processor -- start 2023-11-26T07:16:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.493+0000 7fcfc569f700 1 -- start start 2023-11-26T07:16:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.494+0000 7fcfc569f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfc015e080 0x7fcfc00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.494+0000 7fcfc569f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcfc00af4d0 0x7fcfc00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.494+0000 7fcfc569f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcfc006c000 0x7fcfc006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.494+0000 7fcfc569f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc00af0b0 con 0x7fcfc006c000 2023-11-26T07:16:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.494+0000 7fcfc569f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc006c9f0 con 0x7fcfc00af4d0 2023-11-26T07:16:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.494+0000 7fcfc569f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00b05a0 con 0x7fcfc015e080 2023-11-26T07:16:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.494+0000 7fcfbf7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcfc006c000 0x7fcfc006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.494+0000 7fcfbf7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcfc006c000 0x7fcfc006c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40432/0 (socket says 172.21.15.83:40432) 2023-11-26T07:16:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.494+0000 7fcfbf7fe700 1 -- 172.21.15.83:0/1866114992 learned_addr learned my addr 172.21.15.83:0/1866114992 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.495+0000 7fcfbeffd700 1 --2- 172.21.15.83:0/1866114992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfc015e080 0x7fcfc00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.495+0000 7fcfbe7fc700 1 --2- 172.21.15.83:0/1866114992 >> v2:172.21.15.203:3300/0 conn(0x7fcfc00af4d0 0x7fcfc00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.495+0000 7fcfbf7fe700 1 -- 172.21.15.83:0/1866114992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfc015e080 msgr2=0x7fcfc00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.495+0000 7fcfbf7fe700 1 --2- 172.21.15.83:0/1866114992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfc015e080 0x7fcfc00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.495+0000 7fcfbf7fe700 1 -- 172.21.15.83:0/1866114992 >> v2:172.21.15.203:3300/0 conn(0x7fcfc00af4d0 msgr2=0x7fcfc00afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.495+0000 7fcfbf7fe700 1 --2- 172.21.15.83:0/1866114992 >> v2:172.21.15.203:3300/0 conn(0x7fcfc00af4d0 0x7fcfc00afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.495+0000 7fcfbf7fe700 1 -- 172.21.15.83:0/1866114992 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc00ae430 con 0x7fcfc006c000 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.495+0000 7fcfbf7fe700 1 --2- 172.21.15.83:0/1866114992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc006c000 0x7fcfc006c4b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fcfc006d380 tx=0x7fcfb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0e631416ddec00 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.495+0000 7fcfbdffb700 1 -- 172.21.15.83:0/1866114992 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800e070 con 0x7fcfc006c000 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.495+0000 7fcfbdffb700 1 -- 172.21.15.83:0/1866114992 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb800b980 con 0x7fcfc006c000 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.495+0000 7fcfbdffb700 1 -- 172.21.15.83:0/1866114992 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800bd30 con 0x7fcfc006c000 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.496+0000 7fcfc569f700 1 -- 172.21.15.83:0/1866114992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc006c000 msgr2=0x7fcfc006c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.496+0000 7fcfc569f700 1 --2- 172.21.15.83:0/1866114992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc006c000 0x7fcfc006c4b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fcfc006d380 tx=0x7fcfb800b330 comp rx=0 tx=0).stop 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.496+0000 7fcfc569f700 1 -- 172.21.15.83:0/1866114992 shutdown_connections 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.496+0000 7fcfc569f700 1 --2- 172.21.15.83:0/1866114992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc006c000 0x7fcfc006c4b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.496+0000 7fcfc569f700 1 --2- 172.21.15.83:0/1866114992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfc015e080 0x7fcfc00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.496+0000 7fcfc569f700 1 --2- 172.21.15.83:0/1866114992 >> v2:172.21.15.203:3300/0 conn(0x7fcfc00af4d0 0x7fcfc00afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.496+0000 7fcfc569f700 1 -- 172.21.15.83:0/1866114992 >> 172.21.15.83:0/1866114992 conn(0x7fcfc0071b90 msgr2=0x7fcfc0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.496+0000 7fcfc569f700 1 -- 172.21.15.83:0/1866114992 shutdown_connections 2023-11-26T07:16:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.496+0000 7fcfc569f700 1 -- 172.21.15.83:0/1866114992 wait complete. 2023-11-26T07:16:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.497+0000 7fcfc569f700 1 Processor -- start 2023-11-26T07:16:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.497+0000 7fcfc569f700 1 -- start start 2023-11-26T07:16:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfc569f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc006c000 0x7fcfc01425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfc569f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfc00af4d0 0x7fcfc0141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfc569f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfc015e080 0x7fcfc00cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfc569f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc01519d0 con 0x7fcfc006c000 2023-11-26T07:16:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfc569f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00cf100 con 0x7fcfc00af4d0 2023-11-26T07:16:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfc569f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc015a4d0 con 0x7fcfc015e080 2023-11-26T07:16:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc006c000 0x7fcfc01425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc006c000 0x7fcfc01425e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40434/0 (socket says 172.21.15.83:40434) 2023-11-26T07:16:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbeffd700 1 -- 172.21.15.83:0/1122126667 learned_addr learned my addr 172.21.15.83:0/1122126667 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbe7fc700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfc00af4d0 0x7fcfc0141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbf7fe700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfc015e080 0x7fcfc00cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbeffd700 1 -- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfc015e080 msgr2=0x7fcfc00cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbeffd700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfc015e080 0x7fcfc00cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbeffd700 1 -- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfc00af4d0 msgr2=0x7fcfc0141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbeffd700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfc00af4d0 0x7fcfc0141bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbeffd700 1 -- 172.21.15.83:0/1122126667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb8009b30 con 0x7fcfc006c000 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbe7fc700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfc00af4d0 0x7fcfc0141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfbeffd700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc006c000 0x7fcfc01425e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400d800 tx=0x7fcfb400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfa77fe700 1 -- 172.21.15.83:0/1122126667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40125f0 con 0x7fcfc006c000 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfa77fe700 1 -- 172.21.15.83:0/1122126667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb4012750 con 0x7fcfc006c000 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfc569f700 1 -- 172.21.15.83:0/1122126667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc0157280 con 0x7fcfc006c000 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfa77fe700 1 -- 172.21.15.83:0/1122126667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4012b00 con 0x7fcfc006c000 2023-11-26T07:16:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.498+0000 7fcfc569f700 1 -- 172.21.15.83:0/1122126667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc0157610 con 0x7fcfc006c000 2023-11-26T07:16:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.500+0000 7fcfc569f700 1 -- 172.21.15.83:0/1122126667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc00cf430 con 0x7fcfc006c000 2023-11-26T07:16:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.500+0000 7fcfa77fe700 1 -- 172.21.15.83:0/1122126667 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fcfb4012c60 con 0x7fcfc006c000 2023-11-26T07:16:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.500+0000 7fcfa77fe700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcfac06b020 0x7fcfac06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.500+0000 7fcfa77fe700 1 -- 172.21.15.83:0/1122126667 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcfb40897c0 con 0x7fcfc006c000 2023-11-26T07:16:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.500+0000 7fcfbe7fc700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcfac06b020 0x7fcfac06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.501+0000 7fcfbe7fc700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcfac06b020 0x7fcfac06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0158b70 tx=0x7fcfb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:14.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.505+0000 7fcfa77fe700 1 -- 172.21.15.83:0/1122126667 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfb4054a20 con 0x7fcfc006c000 2023-11-26T07:16:14.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.698+0000 7fcfc569f700 1 -- 172.21.15.83:0/1122126667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfc00753f0 con 0x7fcfc006c000 2023-11-26T07:16:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.699+0000 7fcfa77fe700 1 -- 172.21.15.83:0/1122126667 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+12846 (secure 0 0 0) 0x7fcfb4058040 con 0x7fcfc006c000 2023-11-26T07:16:14.701 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:14.701 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":79,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:13.931773+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 -- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcfac06b020 msgr2=0x7fcfac06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcfac06b020 0x7fcfac06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0158b70 tx=0x7fcfb0008040 comp rx=0 tx=0).stop 2023-11-26T07:16:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 -- 172.21.15.83:0/1122126667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc006c000 msgr2=0x7fcfc01425e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc006c000 0x7fcfc01425e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400d800 tx=0x7fcfb400dbc0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 -- 172.21.15.83:0/1122126667 shutdown_connections 2023-11-26T07:16:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcfac06b020 0x7fcfac06d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfc015e080 0x7fcfc00cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc006c000 0x7fcfc01425e0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 --2- 172.21.15.83:0/1122126667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfc00af4d0 0x7fcfc0141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 -- 172.21.15.83:0/1122126667 >> 172.21.15.83:0/1122126667 conn(0x7fcfc0071b90 msgr2=0x7fcfc00b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 -- 172.21.15.83:0/1122126667 shutdown_connections 2023-11-26T07:16:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.701+0000 7fcfc569f700 1 -- 172.21.15.83:0/1122126667 wait complete. 2023-11-26T07:16:14.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:16:14.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:16:14.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.838+0000 7f8dc2706700 1 Processor -- start 2023-11-26T07:16:14.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.838+0000 7f8dc2706700 1 -- start start 2023-11-26T07:16:14.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.838+0000 7f8dc2706700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc1587f0 0x7f8dbc06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.838+0000 7f8dc2706700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8dbc0b4370 0x7f8dbc15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.838+0000 7f8dc2706700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc06b530 con 0x7f8dbc15b370 2023-11-26T07:16:14.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.838+0000 7f8dc2706700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dbc0ae5e0 con 0x7f8dbc0b4370 2023-11-26T07:16:14.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dc2706700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc15be30 con 0x7f8dbc1587f0 2023-11-26T07:16:14.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc1587f0 0x7f8dbc06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dc0ca3700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8dbc0b4370 0x7f8dbc15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dbb7fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f8dbc15b370 0x7f8dbc0b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:45102/0 (socket says 172.21.15.83:45102) 2023-11-26T07:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc1587f0 0x7f8dbc06aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:50900/0 (socket says 172.21.15.83:50900) 2023-11-26T07:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dbb7fe700 1 -- 172.21.15.83:0/2120284490 learned_addr learned my addr 172.21.15.83:0/2120284490 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dbbfff700 1 -- 172.21.15.83:0/2120284490 >> v2:172.21.15.203:3300/0 conn(0x7f8dbc0b4370 msgr2=0x7f8dbc15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dbbfff700 1 --2- 172.21.15.83:0/2120284490 >> v2:172.21.15.203:3300/0 conn(0x7f8dbc0b4370 0x7f8dbc15b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dbbfff700 1 -- 172.21.15.83:0/2120284490 >> v1:172.21.15.203:6789/0 conn(0x7f8dbc15b370 legacy=0x7f8dbc0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dbbfff700 1 -- 172.21.15.83:0/2120284490 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc15c110 con 0x7f8dbc1587f0 2023-11-26T07:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dbaffd700 1 -- 172.21.15.83:0/2120284490 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611894031 0 0) 0x7f8dbc06b530 con 0x7f8dbc15b370 2023-11-26T07:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.839+0000 7f8dc0ca3700 1 --2- 172.21.15.83:0/2120284490 >> v2:172.21.15.203:3300/0 conn(0x7f8dbc0b4370 0x7f8dbc15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dbbfff700 1 --2- 172.21.15.83:0/2120284490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc1587f0 0x7f8dbc06aff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009840 tx=0x7f8dac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a377b2af810c2a19 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dbaffd700 1 -- 172.21.15.83:0/2120284490 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac011830 con 0x7f8dbc1587f0 2023-11-26T07:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dbaffd700 1 -- 172.21.15.83:0/2120284490 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac011990 con 0x7f8dbc1587f0 2023-11-26T07:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dbaffd700 1 -- 172.21.15.83:0/2120284490 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac011d20 con 0x7f8dbc1587f0 2023-11-26T07:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dc2706700 1 -- 172.21.15.83:0/2120284490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc1587f0 msgr2=0x7f8dbc06aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dc2706700 1 --2- 172.21.15.83:0/2120284490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc1587f0 0x7f8dbc06aff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009840 tx=0x7f8dac009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dc2706700 1 -- 172.21.15.83:0/2120284490 shutdown_connections 2023-11-26T07:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dc2706700 1 --2- 172.21.15.83:0/2120284490 >> v2:172.21.15.203:3300/0 conn(0x7f8dbc0b4370 0x7f8dbc15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dc2706700 1 --2- 172.21.15.83:0/2120284490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc1587f0 0x7f8dbc06aff0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dc2706700 1 -- 172.21.15.83:0/2120284490 >> 172.21.15.83:0/2120284490 conn(0x7f8dbc0d0bf0 msgr2=0x7f8dbc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dc2706700 1 -- 172.21.15.83:0/2120284490 shutdown_connections 2023-11-26T07:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.840+0000 7f8dc2706700 1 -- 172.21.15.83:0/2120284490 wait complete. 2023-11-26T07:16:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.841+0000 7f8dc2706700 1 Processor -- start 2023-11-26T07:16:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.842+0000 7f8dc2706700 1 -- start start 2023-11-26T07:16:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.842+0000 7f8dc2706700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc0b4370 0x7f8dbc149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.842+0000 7f8dc2706700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dbc1587f0 0x7f8dbc0c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.842+0000 7f8dc2706700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dbc15b370 0x7f8dbc0c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.842+0000 7f8dc2706700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0d06e0 con 0x7f8dbc1587f0 2023-11-26T07:16:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.842+0000 7f8dc2706700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc149880 con 0x7f8dbc15b370 2023-11-26T07:16:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.842+0000 7f8dc2706700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc149550 con 0x7f8dbc0b4370 2023-11-26T07:16:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dbb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dbc1587f0 0x7f8dbc0c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dbb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dbc1587f0 0x7f8dbc0c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40450/0 (socket says 172.21.15.83:40450) 2023-11-26T07:16:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dbb7fe700 1 -- 172.21.15.83:0/469586357 learned_addr learned my addr 172.21.15.83:0/469586357 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dc0ca3700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dbc15b370 0x7f8dbc0c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dbbfff700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc0b4370 0x7f8dbc149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dbb7fe700 1 -- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc0b4370 msgr2=0x7f8dbc149010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dbb7fe700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc0b4370 0x7f8dbc149010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dbb7fe700 1 -- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dbc15b370 msgr2=0x7f8dbc0c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dbb7fe700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dbc15b370 0x7f8dbc0c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dbb7fe700 1 -- 172.21.15.83:0/469586357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db4009a70 con 0x7f8dbc1587f0 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dbb7fe700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dbc1587f0 0x7f8dbc0c8ad0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c550 tx=0x7f8db000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8db8ff9700 1 -- 172.21.15.83:0/469586357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0004c80 con 0x7f8dbc1587f0 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8db8ff9700 1 -- 172.21.15.83:0/469586357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db0004e00 con 0x7f8dbc1587f0 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8db8ff9700 1 -- 172.21.15.83:0/469586357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0007310 con 0x7f8dbc1587f0 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dc2706700 1 -- 172.21.15.83:0/469586357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac0094a0 con 0x7f8dbc1587f0 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.843+0000 7f8dc2706700 1 -- 172.21.15.83:0/469586357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dbc149b40 con 0x7f8dbc1587f0 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.844+0000 7f8dc2706700 1 -- 172.21.15.83:0/469586357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dbc068fb0 con 0x7f8dbc1587f0 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.845+0000 7f8db8ff9700 1 -- 172.21.15.83:0/469586357 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f8db0007470 con 0x7f8dbc1587f0 2023-11-26T07:16:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.845+0000 7f8db8ff9700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8da806aff0 0x7f8da806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.845+0000 7f8db8ff9700 1 -- 172.21.15.83:0/469586357 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8db0088f10 con 0x7f8dbc1587f0 2023-11-26T07:16:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.846+0000 7f8dbbfff700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8da806aff0 0x7f8da806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.846+0000 7f8dbbfff700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8da806aff0 0x7f8da806d4b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0090b0 tx=0x7f8dac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:14.850+0000 7f8db8ff9700 1 -- 172.21.15.83:0/469586357 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8db0054290 con 0x7f8dbc1587f0 2023-11-26T07:16:15.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.040+0000 7f8dc2706700 1 -- 172.21.15.83:0/469586357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8dbc06f3f0 con 0x7f8dbc1587f0 2023-11-26T07:16:15.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.041+0000 7f8db8ff9700 1 -- 172.21.15.83:0/469586357 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8db00578b0 con 0x7f8dbc1587f0 2023-11-26T07:16:15.043 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:16:15.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 -- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8da806aff0 msgr2=0x7f8da806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8da806aff0 0x7f8da806d4b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0090b0 tx=0x7f8dac008030 comp rx=0 tx=0).stop 2023-11-26T07:16:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 -- 172.21.15.83:0/469586357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dbc1587f0 msgr2=0x7f8dbc0c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dbc1587f0 0x7f8dbc0c8ad0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f8db000c550 tx=0x7f8db000c860 comp rx=0 tx=0).stop 2023-11-26T07:16:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 -- 172.21.15.83:0/469586357 shutdown_connections 2023-11-26T07:16:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f8da806aff0 0x7f8da806d4b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8dbc15b370 0x7f8dbc0c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8dbc0b4370 0x7f8dbc149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 --2- 172.21.15.83:0/469586357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dbc1587f0 0x7f8dbc0c8ad0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 -- 172.21.15.83:0/469586357 >> 172.21.15.83:0/469586357 conn(0x7f8dbc0d0bf0 msgr2=0x7f8dbc06be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 -- 172.21.15.83:0/469586357 shutdown_connections 2023-11-26T07:16:15.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.043+0000 7f8dc2706700 1 -- 172.21.15.83:0/469586357 wait complete. 2023-11-26T07:16:15.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:16:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.176+0000 7f3c090d5700 1 Processor -- start 2023-11-26T07:16:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.176+0000 7f3c090d5700 1 -- start start 2023-11-26T07:16:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c090d5700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3c040adf00 0x7f3c0406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c090d5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c040af950 con 0x7f3c0406c540 2023-11-26T07:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c090d5700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c040afc30 con 0x7f3c041446a0 2023-11-26T07:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c090d5700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c0406c9f0 con 0x7f3c040adf00 2023-11-26T07:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c0359e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3c0406c540 0x7f3c040b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60500/0 (socket says 172.21.15.83:60500) 2023-11-26T07:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c0359e700 1 -- 172.21.15.83:0/3982966618 learned_addr learned my addr 172.21.15.83:0/3982966618 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745691368 0 0) 0x7f3c040af950 con 0x7f3c0406c540 2023-11-26T07:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c0259c700 1 --2- 172.21.15.83:0/3982966618 >> v2:172.21.15.203:3300/0 conn(0x7f3c040adf00 0x7f3c0406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bec0035b0 con 0x7f3c0406c540 2023-11-26T07:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2139548746 0 0) 0x7f3bec0035b0 con 0x7f3c0406c540 2023-11-26T07:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 >> v2:172.21.15.203:3300/0 conn(0x7f3c040adf00 msgr2=0x7f3c0406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c01d9b700 1 --2- 172.21.15.83:0/3982966618 >> v2:172.21.15.203:3300/0 conn(0x7f3c040adf00 0x7f3c0406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 >> v1:172.21.15.203:6789/0 conn(0x7f3c041446a0 legacy=0x7f3c040b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c040aff10 con 0x7f3c0406c540 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f3bfc0019f0 con 0x7f3c0406c540 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c0259c700 1 --2- 172.21.15.83:0/3982966618 >> v2:172.21.15.203:3300/0 conn(0x7f3c040adf00 0x7f3c0406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 >> v1:172.21.15.83:6789/0 conn(0x7f3c0406c540 legacy=0x7f3c040b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c01d9b700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bec003a00 0x7f3bec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.177+0000 7f3c01d9b700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 0x7f3bec004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c01d9b700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bec005210 0x7f3bec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0406c9f0 con 0x7f3bec004850 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec00c160 con 0x7f3bec005210 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec00c320 con 0x7f3bec003a00 2023-11-26T07:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c0259c700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 0x7f3bec004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c02d9d700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bec003a00 0x7f3bec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c0359e700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bec005210 0x7f3bec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c0259c700 1 -- 172.21.15.83:0/3982966618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bec003a00 msgr2=0x7f3bec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c0259c700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bec003a00 0x7f3bec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c0259c700 1 -- 172.21.15.83:0/3982966618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bec005210 msgr2=0x7f3bec00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c0259c700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bec005210 0x7f3bec00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c0259c700 1 -- 172.21.15.83:0/3982966618 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0035b0 con 0x7f3bec004850 2023-11-26T07:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c0359e700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bec005210 0x7f3bec00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c0359e700 1 -- 172.21.15.83:0/3982966618 reap_dead start 2023-11-26T07:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c0259c700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 0x7f3bec004cd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3c0406eef0 tx=0x7f3bec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8a6e4f34796325 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf800c070 con 0x7f3bec004850 2023-11-26T07:16:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf8004020 con 0x7f3bec004850 2023-11-26T07:16:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.178+0000 7f3c01d9b700 1 -- 172.21.15.83:0/3982966618 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf80043d0 con 0x7f3bec004850 2023-11-26T07:16:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.179+0000 7f3c090d5700 1 -- 172.21.15.83:0/3982966618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 msgr2=0x7f3bec004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.179+0000 7f3c090d5700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 0x7f3bec004cd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3c0406eef0 tx=0x7f3bec005880 comp rx=0 tx=0).stop 2023-11-26T07:16:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.179+0000 7f3c090d5700 1 -- 172.21.15.83:0/3982966618 shutdown_connections 2023-11-26T07:16:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.179+0000 7f3c090d5700 1 --2- 172.21.15.83:0/3982966618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 0x7f3bec004cd0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.179+0000 7f3c090d5700 1 -- 172.21.15.83:0/3982966618 >> 172.21.15.83:0/3982966618 conn(0x7f3c04071b90 msgr2=0x7f3c04071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.179+0000 7f3c090d5700 1 -- 172.21.15.83:0/3982966618 shutdown_connections 2023-11-26T07:16:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.179+0000 7f3c090d5700 1 -- 172.21.15.83:0/3982966618 wait complete. 2023-11-26T07:16:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.181+0000 7f3c090d5700 1 Processor -- start 2023-11-26T07:16:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.181+0000 7f3c090d5700 1 -- start start 2023-11-26T07:16:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.181+0000 7f3c090d5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 0x7f3c0414bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.181+0000 7f3c090d5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c041490c0 0x7f3c0414a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c090d5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c040cf890 0x7f3c0414b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c090d5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c041498a0 con 0x7f3bec004850 2023-11-26T07:16:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c090d5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c04149b80 con 0x7f3c040cf890 2023-11-26T07:16:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c090d5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c04149e60 con 0x7f3c041490c0 2023-11-26T07:16:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c02d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 0x7f3c0414bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c02d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 0x7f3c0414bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40468/0 (socket says 172.21.15.83:40468) 2023-11-26T07:16:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c02d9d700 1 -- 172.21.15.83:0/3047571073 learned_addr learned my addr 172.21.15.83:0/3047571073 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c0359e700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c040cf890 0x7f3c0414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c0259c700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c041490c0 0x7f3c0414a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c02d9d700 1 -- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c041490c0 msgr2=0x7f3c0414a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c02d9d700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c041490c0 0x7f3c0414a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c02d9d700 1 -- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c040cf890 msgr2=0x7f3c0414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c02d9d700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c040cf890 0x7f3c0414b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c02d9d700 1 -- 172.21.15.83:0/3047571073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0414a140 con 0x7f3bec004850 2023-11-26T07:16:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c0359e700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c040cf890 0x7f3c0414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c02d9d700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 0x7f3c0414bcf0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400e7b0 tx=0x7f3bf400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3beb7fe700 1 -- 172.21.15.83:0/3047571073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf40052e0 con 0x7f3bec004850 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3beb7fe700 1 -- 172.21.15.83:0/3047571073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf4005440 con 0x7f3bec004850 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3beb7fe700 1 -- 172.21.15.83:0/3047571073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf40038a0 con 0x7f3bec004850 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c090d5700 1 -- 172.21.15.83:0/3047571073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0414a420 con 0x7f3bec004850 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.182+0000 7f3c090d5700 1 -- 172.21.15.83:0/3047571073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c04149540 con 0x7f3bec004850 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.183+0000 7f3c090d5700 1 -- 172.21.15.83:0/3047571073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c040753f0 con 0x7f3bec004850 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.184+0000 7f3beb7fe700 1 -- 172.21.15.83:0/3047571073 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f3bf4003a00 con 0x7f3bec004850 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.184+0000 7f3beb7fe700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3bf006afe0 0x7f3bf006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.184+0000 7f3beb7fe700 1 -- 172.21.15.83:0/3047571073 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3bf4088b50 con 0x7f3bec004850 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.184+0000 7f3c0259c700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3bf006afe0 0x7f3bf006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.185+0000 7f3c0259c700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3bf006afe0 0x7f3bf006d4a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8016040 tx=0x7f3bf800c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.188+0000 7f3beb7fe700 1 -- 172.21.15.83:0/3047571073 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3bf4053ef0 con 0x7f3bec004850 2023-11-26T07:16:15.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.385+0000 7f3c090d5700 1 -- 172.21.15.83:0/3047571073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3c04067f60 con 0x7f3bec004850 2023-11-26T07:16:15.388 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.386+0000 7f3beb7fe700 1 -- 172.21.15.83:0/3047571073 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3bf4019020 con 0x7f3bec004850 2023-11-26T07:16:15.388 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:15.389 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:16:15.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:15.389 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2023-11-26T07:16:15.389 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2023-11-26T07:16:15.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:15.389 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:16:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 -- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3bf006afe0 msgr2=0x7f3bf006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3bf006afe0 0x7f3bf006d4a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8016040 tx=0x7f3bf800c000 comp rx=0 tx=0).stop 2023-11-26T07:16:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 -- 172.21.15.83:0/3047571073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 msgr2=0x7f3c0414bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 0x7f3c0414bcf0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400e7b0 tx=0x7f3bf400eb70 comp rx=0 tx=0).stop 2023-11-26T07:16:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 -- 172.21.15.83:0/3047571073 shutdown_connections 2023-11-26T07:16:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3bf006afe0 0x7f3bf006d4a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c040cf890 0x7f3c0414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bec004850 0x7f3c0414bcf0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 --2- 172.21.15.83:0/3047571073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c041490c0 0x7f3c0414a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 -- 172.21.15.83:0/3047571073 >> 172.21.15.83:0/3047571073 conn(0x7f3c04071b90 msgr2=0x7f3c040b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 -- 172.21.15.83:0/3047571073 shutdown_connections 2023-11-26T07:16:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.388+0000 7f3c090d5700 1 -- 172.21.15.83:0/3047571073 wait complete. 2023-11-26T07:16:15.403 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:16:15.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:16:15.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.524+0000 7f94101d0700 1 Processor -- start 2023-11-26T07:16:15.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.525+0000 7f94101d0700 1 -- start start 2023-11-26T07:16:15.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.525+0000 7f94101d0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f940815e020 0x7f94080b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.525+0000 7f94101d0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94080b1ba0 con 0x7f94080b1720 2023-11-26T07:16:15.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.525+0000 7f94101d0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94080cff70 con 0x7f94080cfae0 2023-11-26T07:16:15.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.525+0000 7f94101d0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9408161c80 con 0x7f940815e020 2023-11-26T07:16:15.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940d76b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f94080b1720 0x7f94080cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60516/0 (socket says 172.21.15.83:60516) 2023-11-26T07:16:15.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940d76b700 1 -- 172.21.15.83:0/3738074953 learned_addr learned my addr 172.21.15.83:0/3738074953 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:15.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940df6c700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f940815e020 0x7f94080b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635661501 0 0) 0x7f94080b1ba0 con 0x7f94080b1720 2023-11-26T07:16:15.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93f40035b0 con 0x7f94080b1720 2023-11-26T07:16:15.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013494415 0 0) 0x7f94080cff70 con 0x7f94080cfae0 2023-11-26T07:16:15.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94080b1ba0 con 0x7f94080cfae0 2023-11-26T07:16:15.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3293946863 0 0) 0x7f93f40035b0 con 0x7f94080b1720 2023-11-26T07:16:15.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f940815e020 msgr2=0x7f94080b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:15.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f940815e020 0x7f94080b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 >> v1:172.21.15.203:6789/0 conn(0x7f94080cfae0 legacy=0x7f94080af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9408161f40 con 0x7f94080b1720 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f9404001670 con 0x7f94080b1720 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 >> v1:172.21.15.83:6789/0 conn(0x7f94080b1720 legacy=0x7f94080cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93f4003bc0 0x7f93f4003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f4004850 0x7f93f4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f940815e020 0x7f93f4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9408161c80 con 0x7f93f4004850 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f400bb20 con 0x7f93f4003bc0 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f400bce0 con 0x7f940815e020 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940d76b700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f4004850 0x7f93f4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940df6c700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93f4003bc0 0x7f93f4003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.526+0000 7f940e76d700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f940815e020 0x7f93f4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f940d76b700 1 -- 172.21.15.83:0/3738074953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f940815e020 msgr2=0x7f93f4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:15.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f940d76b700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f940815e020 0x7f93f4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f940d76b700 1 -- 172.21.15.83:0/3738074953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93f4003bc0 msgr2=0x7f93f4003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:15.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f940d76b700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93f4003bc0 0x7f93f4003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f940d76b700 1 -- 172.21.15.83:0/3738074953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f40035b0 con 0x7f93f4004850 2023-11-26T07:16:15.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f940d76b700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f4004850 0x7f93f4004cd0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f940400c210 tx=0x7f93f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b311d287e1e5382f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:15.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940400cdb0 con 0x7f93f4004850 2023-11-26T07:16:15.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9404001af0 con 0x7f93f4004850 2023-11-26T07:16:15.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f940cf6a700 1 -- 172.21.15.83:0/3738074953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404005ba0 con 0x7f93f4004850 2023-11-26T07:16:15.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f94101d0700 1 -- 172.21.15.83:0/3738074953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f4004850 msgr2=0x7f93f4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f94101d0700 1 --2- 172.21.15.83:0/3738074953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93f4004850 0x7f93f4004cd0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f940400c210 tx=0x7f93f4005880 comp rx=0 tx=0).stop 2023-11-26T07:16:15.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f940e76d700 1 -- 172.21.15.83:0/3738074953 reap_dead start 2023-11-26T07:16:15.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f94101d0700 1 -- 172.21.15.83:0/3738074953 shutdown_connections 2023-11-26T07:16:15.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f94101d0700 1 -- 172.21.15.83:0/3738074953 >> 172.21.15.83:0/3738074953 conn(0x7f9408072710 msgr2=0x7f9408072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:15.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f94101d0700 1 -- 172.21.15.83:0/3738074953 shutdown_connections 2023-11-26T07:16:15.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.527+0000 7f94101d0700 1 -- 172.21.15.83:0/3738074953 wait complete. 2023-11-26T07:16:15.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.529+0000 7f94101d0700 1 Processor -- start 2023-11-26T07:16:15.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.529+0000 7f94101d0700 1 -- start start 2023-11-26T07:16:15.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.529+0000 7f94101d0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93f4004850 0x7f94081f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.529+0000 7f94101d0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9408099880 0x7f9408153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.529+0000 7f94101d0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9408154490 0x7f9408097260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.529+0000 7f94101d0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9408152eb0 con 0x7f9408154490 2023-11-26T07:16:15.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.529+0000 7f94101d0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9408153190 con 0x7f9408099880 2023-11-26T07:16:15.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.529+0000 7f94101d0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9408153470 con 0x7f93f4004850 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.529+0000 7f940e76d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9408154490 0x7f9408097260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f940e76d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9408154490 0x7f9408097260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40492/0 (socket says 172.21.15.83:40492) 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f940e76d700 1 -- 172.21.15.83:0/2016680732 learned_addr learned my addr 172.21.15.83:0/2016680732 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f940d76b700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9408099880 0x7f9408153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f940df6c700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93f4004850 0x7f94081f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f940e76d700 1 -- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93f4004850 msgr2=0x7f94081f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f940e76d700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93f4004850 0x7f94081f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f940e76d700 1 -- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9408099880 msgr2=0x7f9408153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f940e76d700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9408099880 0x7f9408153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f940e76d700 1 -- 172.21.15.83:0/2016680732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9408153750 con 0x7f9408154490 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f940e76d700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9408154490 0x7f9408097260 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f94080cdd00 tx=0x7f940000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:15.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f93fe7fc700 1 -- 172.21.15.83:0/2016680732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940000fe70 con 0x7f9408154490 2023-11-26T07:16:15.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f93fe7fc700 1 -- 172.21.15.83:0/2016680732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9400003c10 con 0x7f9408154490 2023-11-26T07:16:15.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f94101d0700 1 -- 172.21.15.83:0/2016680732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9408153a30 con 0x7f9408154490 2023-11-26T07:16:15.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f93fe7fc700 1 -- 172.21.15.83:0/2016680732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940000d990 con 0x7f9408154490 2023-11-26T07:16:15.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.530+0000 7f94101d0700 1 -- 172.21.15.83:0/2016680732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94081f4320 con 0x7f9408154490 2023-11-26T07:16:15.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.531+0000 7f94101d0700 1 -- 172.21.15.83:0/2016680732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9408153bc0 con 0x7f9408154490 2023-11-26T07:16:15.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.532+0000 7f93fe7fc700 1 -- 172.21.15.83:0/2016680732 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f940000daf0 con 0x7f9408154490 2023-11-26T07:16:15.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.532+0000 7f93fe7fc700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f93ec06aff0 0x7f93ec06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:15.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.532+0000 7f93fe7fc700 1 -- 172.21.15.83:0/2016680732 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9400089620 con 0x7f9408154490 2023-11-26T07:16:15.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.532+0000 7f940df6c700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f93ec06aff0 0x7f93ec06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:15.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.533+0000 7f940df6c700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f93ec06aff0 0x7f93ec06d4b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009b00 tx=0x7f93f8009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:15.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.536+0000 7f93fe7fc700 1 -- 172.21.15.83:0/2016680732 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9400054910 con 0x7f9408154490 2023-11-26T07:16:15.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:15.728+0000 7f94101d0700 1 -- 172.21.15.83:0/2016680732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9408067f60 con 0x7f9408154490 2023-11-26T07:16:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.565+0000 7f93fe7fc700 1 -- 172.21.15.83:0/2016680732 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9400057f30 con 0x7f9408154490 2023-11-26T07:16:16.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 -- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f93ec06aff0 msgr2=0x7f93ec06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f93ec06aff0 0x7f93ec06d4b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009b00 tx=0x7f93f8009210 comp rx=0 tx=0).stop 2023-11-26T07:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 -- 172.21.15.83:0/2016680732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9408154490 msgr2=0x7f9408097260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9408154490 0x7f9408097260 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f94080cdd00 tx=0x7f940000d430 comp rx=0 tx=0).stop 2023-11-26T07:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 -- 172.21.15.83:0/2016680732 shutdown_connections 2023-11-26T07:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f93ec06aff0 0x7f93ec06d4b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9408154490 0x7f9408097260 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f93f4004850 0x7f94081f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 --2- 172.21.15.83:0/2016680732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9408099880 0x7f9408153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 -- 172.21.15.83:0/2016680732 >> 172.21.15.83:0/2016680732 conn(0x7f9408072710 msgr2=0x7f94081610c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 -- 172.21.15.83:0/2016680732 shutdown_connections 2023-11-26T07:16:16.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.569+0000 7f94101d0700 1 -- 172.21.15.83:0/2016680732 wait complete. 2023-11-26T07:16:16.573 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2023-11-26T07:16:16.590 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:16:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.720+0000 7f63f23c9700 1 Processor -- start 2023-11-26T07:16:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.720+0000 7f63f23c9700 1 -- start start 2023-11-26T07:16:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f23c9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f63ec0746a0 0x7f63ec0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f23c9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f63ec0afc90 0x7f63ec06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f23c9700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ec0b3c60 con 0x7f63ec15e160 2023-11-26T07:16:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f23c9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ec0b4500 con 0x7f63ec0afc90 2023-11-26T07:16:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f23c9700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ec0b58c0 con 0x7f63ec0746a0 2023-11-26T07:16:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f0966700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f63ec0afc90 0x7f63ec06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f0966700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f63ec0afc90 0x7f63ec06d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40500/0 (socket says 172.21.15.83:40500) 2023-11-26T07:16:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f0966700 1 -- 172.21.15.83:0/605827926 learned_addr learned my addr 172.21.15.83:0/605827926 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63eb7fe700 1 --2- 172.21.15.83:0/605827926 >> v2:172.21.15.203:3300/0 conn(0x7f63ec0746a0 0x7f63ec0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f0966700 1 -- 172.21.15.83:0/605827926 >> v2:172.21.15.203:3300/0 conn(0x7f63ec0746a0 msgr2=0x7f63ec0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f0966700 1 --2- 172.21.15.83:0/605827926 >> v2:172.21.15.203:3300/0 conn(0x7f63ec0746a0 0x7f63ec0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f0966700 1 -- 172.21.15.83:0/605827926 >> v1:172.21.15.203:6789/0 conn(0x7f63ec15e160 legacy=0x7f63ec073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f0966700 1 -- 172.21.15.83:0/605827926 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec06bdc0 con 0x7f63ec0afc90 2023-11-26T07:16:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.721+0000 7f63f0966700 1 --2- 172.21.15.83:0/605827926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec0afc90 0x7f63ec06d930 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0d0880 tx=0x7f63e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cea552c4552c3ad server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.722+0000 7f63eaffd700 1 -- 172.21.15.83:0/605827926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e400e070 con 0x7f63ec0afc90 2023-11-26T07:16:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.722+0000 7f63eaffd700 1 -- 172.21.15.83:0/605827926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e4003a20 con 0x7f63ec0afc90 2023-11-26T07:16:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.722+0000 7f63eaffd700 1 -- 172.21.15.83:0/605827926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4003dd0 con 0x7f63ec0afc90 2023-11-26T07:16:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.722+0000 7f63f23c9700 1 -- 172.21.15.83:0/605827926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec0afc90 msgr2=0x7f63ec06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.722+0000 7f63f23c9700 1 --2- 172.21.15.83:0/605827926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec0afc90 0x7f63ec06d930 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0d0880 tx=0x7f63e400b330 comp rx=0 tx=0).stop 2023-11-26T07:16:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.722+0000 7f63f23c9700 1 -- 172.21.15.83:0/605827926 shutdown_connections 2023-11-26T07:16:16.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.722+0000 7f63f23c9700 1 --2- 172.21.15.83:0/605827926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec0afc90 0x7f63ec06d930 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:16.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.722+0000 7f63f23c9700 1 --2- 172.21.15.83:0/605827926 >> v2:172.21.15.203:3300/0 conn(0x7f63ec0746a0 0x7f63ec0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:16.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.722+0000 7f63f23c9700 1 -- 172.21.15.83:0/605827926 >> 172.21.15.83:0/605827926 conn(0x7f63ec0d0bf0 msgr2=0x7f63ec0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:16.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.722+0000 7f63f23c9700 1 -- 172.21.15.83:0/605827926 shutdown_connections 2023-11-26T07:16:16.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.722+0000 7f63f23c9700 1 -- 172.21.15.83:0/605827926 wait complete. 2023-11-26T07:16:16.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.723+0000 7f63f23c9700 1 Processor -- start 2023-11-26T07:16:16.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.724+0000 7f63f23c9700 1 -- start start 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.724+0000 7f63f23c9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec0746a0 0x7f63ec14d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.724+0000 7f63f23c9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0afc90 0x7f63ec0c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.724+0000 7f63f23c9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec15e160 0x7f63ec14b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.724+0000 7f63f23c9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec14b180 con 0x7f63ec15e160 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.724+0000 7f63f23c9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec0c5c70 con 0x7f63ec0746a0 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.724+0000 7f63f23c9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec0c77f0 con 0x7f63ec0afc90 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.724+0000 7f63f0966700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec15e160 0x7f63ec14b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.724+0000 7f63f0966700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec15e160 0x7f63ec14b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40506/0 (socket says 172.21.15.83:40506) 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.724+0000 7f63f0966700 1 -- 172.21.15.83:0/3621402214 learned_addr learned my addr 172.21.15.83:0/3621402214 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63ebfff700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec0746a0 0x7f63ec14d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63eb7fe700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0afc90 0x7f63ec0c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:16.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63f0966700 1 -- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0afc90 msgr2=0x7f63ec0c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63f0966700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0afc90 0x7f63ec0c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63f0966700 1 -- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec0746a0 msgr2=0x7f63ec14d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63f0966700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec0746a0 0x7f63ec14d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63f0966700 1 -- 172.21.15.83:0/3621402214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e4009b30 con 0x7f63ec15e160 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63ebfff700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec0746a0 0x7f63ec14d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63f0966700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec15e160 0x7f63ec14b430 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f63ec072ed0 tx=0x7f63e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63e8ff9700 1 -- 172.21.15.83:0/3621402214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4004440 con 0x7f63ec15e160 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63e8ff9700 1 -- 172.21.15.83:0/3621402214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e40045a0 con 0x7f63ec15e160 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63f23c9700 1 -- 172.21.15.83:0/3621402214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ec14daa0 con 0x7f63ec15e160 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63e8ff9700 1 -- 172.21.15.83:0/3621402214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4004950 con 0x7f63ec15e160 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.725+0000 7f63f23c9700 1 -- 172.21.15.83:0/3621402214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ec14e060 con 0x7f63ec15e160 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.726+0000 7f63f23c9700 1 -- 172.21.15.83:0/3621402214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ec0c5f70 con 0x7f63ec15e160 2023-11-26T07:16:16.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.727+0000 7f63e8ff9700 1 -- 172.21.15.83:0/3621402214 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f63e401c440 con 0x7f63ec15e160 2023-11-26T07:16:16.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.727+0000 7f63e8ff9700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f63d806aff0 0x7f63d806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:16.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.727+0000 7f63e8ff9700 1 -- 172.21.15.83:0/3621402214 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f63e408aae0 con 0x7f63ec15e160 2023-11-26T07:16:16.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.727+0000 7f63ebfff700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f63d806aff0 0x7f63d806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:16.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.728+0000 7f63ebfff700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f63d806aff0 0x7f63d806d4b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009910 tx=0x7f63e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:16.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.732+0000 7f63e8ff9700 1 -- 172.21.15.83:0/3621402214 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63e4055b60 con 0x7f63ec15e160 2023-11-26T07:16:16.926 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:16.923+0000 7f63f23c9700 1 -- 172.21.15.83:0/3621402214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f63ec0c5db0 con 0x7f63ec15e160 2023-11-26T07:16:17.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.567+0000 7f63e8ff9700 1 -- 172.21.15.83:0/3621402214 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f63e4059180 con 0x7f63ec15e160 2023-11-26T07:16:17.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.571+0000 7f63f23c9700 1 -- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f63d806aff0 msgr2=0x7f63d806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:17.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.571+0000 7f63f23c9700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f63d806aff0 0x7f63d806d4b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009910 tx=0x7f63e0008040 comp rx=0 tx=0).stop 2023-11-26T07:16:17.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.571+0000 7f63f23c9700 1 -- 172.21.15.83:0/3621402214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec15e160 msgr2=0x7f63ec14b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:17.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.571+0000 7f63f23c9700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec15e160 0x7f63ec14b430 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f63ec072ed0 tx=0x7f63e4004ad0 comp rx=0 tx=0).stop 2023-11-26T07:16:17.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.572+0000 7f63f23c9700 1 -- 172.21.15.83:0/3621402214 shutdown_connections 2023-11-26T07:16:17.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.572+0000 7f63f23c9700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f63d806aff0 0x7f63d806d4b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:17.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.572+0000 7f63f23c9700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63ec15e160 0x7f63ec14b430 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:17.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.572+0000 7f63f23c9700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63ec0746a0 0x7f63ec14d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:17.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.572+0000 7f63f23c9700 1 --2- 172.21.15.83:0/3621402214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63ec0afc90 0x7f63ec0c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:17.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.572+0000 7f63f23c9700 1 -- 172.21.15.83:0/3621402214 >> 172.21.15.83:0/3621402214 conn(0x7f63ec0d0bf0 msgr2=0x7f63ec0ae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:17.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.572+0000 7f63f23c9700 1 -- 172.21.15.83:0/3621402214 shutdown_connections 2023-11-26T07:16:17.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.573+0000 7f63f23c9700 1 -- 172.21.15.83:0/3621402214 wait complete. 2023-11-26T07:16:17.580 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2023-11-26T07:16:17.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:16:17.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.720+0000 7f22b2fb6700 1 Processor -- start 2023-11-26T07:16:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.720+0000 7f22b2fb6700 1 -- start start 2023-11-26T07:16:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22b2fb6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22ac0746a0 0x7f22ac0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22b2fb6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22ac0afc90 0x7f22ac06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22b2fb6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac0b3c60 con 0x7f22ac15e160 2023-11-26T07:16:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22b2fb6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac0b4500 con 0x7f22ac0746a0 2023-11-26T07:16:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22b2fb6700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac0b58c0 con 0x7f22ac0afc90 2023-11-26T07:16:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22abfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22ac0746a0 0x7f22ac0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:17.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22abfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22ac0746a0 0x7f22ac0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40512/0 (socket says 172.21.15.83:40512) 2023-11-26T07:16:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22abfff700 1 -- 172.21.15.83:0/3649756900 learned_addr learned my addr 172.21.15.83:0/3649756900 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22b1553700 1 --2- 172.21.15.83:0/3649756900 >> v2:172.21.15.203:3300/0 conn(0x7f22ac0afc90 0x7f22ac06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22abfff700 1 -- 172.21.15.83:0/3649756900 >> v2:172.21.15.203:3300/0 conn(0x7f22ac0afc90 msgr2=0x7f22ac06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22abfff700 1 --2- 172.21.15.83:0/3649756900 >> v2:172.21.15.203:3300/0 conn(0x7f22ac0afc90 0x7f22ac06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22abfff700 1 -- 172.21.15.83:0/3649756900 >> v1:172.21.15.203:6789/0 conn(0x7f22ac15e160 legacy=0x7f22ac073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.721+0000 7f22abfff700 1 -- 172.21.15.83:0/3649756900 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac06bdc0 con 0x7f22ac0746a0 2023-11-26T07:16:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22b1553700 1 --2- 172.21.15.83:0/3649756900 >> v2:172.21.15.203:3300/0 conn(0x7f22ac0afc90 0x7f22ac06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22abfff700 1 --2- 172.21.15.83:0/3649756900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ac0746a0 0x7f22ac0b5da0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f22ac162db0 tx=0x7f2298009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b70ba227e0ad9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22ab7fe700 1 -- 172.21.15.83:0/3649756900 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229800b870 con 0x7f22ac0746a0 2023-11-26T07:16:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22ab7fe700 1 -- 172.21.15.83:0/3649756900 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229800b9d0 con 0x7f22ac0746a0 2023-11-26T07:16:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22ab7fe700 1 -- 172.21.15.83:0/3649756900 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229800bd60 con 0x7f22ac0746a0 2023-11-26T07:16:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22b2fb6700 1 -- 172.21.15.83:0/3649756900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ac0746a0 msgr2=0x7f22ac0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22b2fb6700 1 --2- 172.21.15.83:0/3649756900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ac0746a0 0x7f22ac0b5da0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f22ac162db0 tx=0x7f2298009bd0 comp rx=0 tx=0).stop 2023-11-26T07:16:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22b2fb6700 1 -- 172.21.15.83:0/3649756900 shutdown_connections 2023-11-26T07:16:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22b2fb6700 1 --2- 172.21.15.83:0/3649756900 >> v2:172.21.15.203:3300/0 conn(0x7f22ac0afc90 0x7f22ac06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22b2fb6700 1 --2- 172.21.15.83:0/3649756900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ac0746a0 0x7f22ac0b5da0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22b2fb6700 1 -- 172.21.15.83:0/3649756900 >> 172.21.15.83:0/3649756900 conn(0x7f22ac0d0bf0 msgr2=0x7f22ac0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22b2fb6700 1 -- 172.21.15.83:0/3649756900 shutdown_connections 2023-11-26T07:16:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.722+0000 7f22b2fb6700 1 -- 172.21.15.83:0/3649756900 wait complete. 2023-11-26T07:16:17.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.724+0000 7f22b2fb6700 1 Processor -- start 2023-11-26T07:16:17.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.724+0000 7f22b2fb6700 1 -- start start 2023-11-26T07:16:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.724+0000 7f22b2fb6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ac0746a0 0x7f22ac1590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.724+0000 7f22b2fb6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22ac0afc90 0x7f22ac1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.724+0000 7f22b2fb6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac15e160 0x7f22ac1447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.724+0000 7f22b2fb6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1596d0 con 0x7f22ac0746a0 2023-11-26T07:16:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.724+0000 7f22b2fb6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac145930 con 0x7f22ac15e160 2023-11-26T07:16:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b2fb6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac145000 con 0x7f22ac0afc90 2023-11-26T07:16:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b0d52700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ac0746a0 0x7f22ac1590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b0d52700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ac0746a0 0x7f22ac1590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40516/0 (socket says 172.21.15.83:40516) 2023-11-26T07:16:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b0d52700 1 -- 172.21.15.83:0/147764944 learned_addr learned my addr 172.21.15.83:0/147764944 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22abfff700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22ac0afc90 0x7f22ac1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b1553700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac15e160 0x7f22ac1447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b0d52700 1 -- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22ac0afc90 msgr2=0x7f22ac1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b0d52700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22ac0afc90 0x7f22ac1567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b0d52700 1 -- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac15e160 msgr2=0x7f22ac1447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b0d52700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac15e160 0x7f22ac1447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b0d52700 1 -- 172.21.15.83:0/147764944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2298009550 con 0x7f22ac0746a0 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22abfff700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22ac0afc90 0x7f22ac1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b1553700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac15e160 0x7f22ac1447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b0d52700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ac0746a0 0x7f22ac1590f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c430 tx=0x7f22a000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22a97fa700 1 -- 172.21.15.83:0/147764944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a00052d0 con 0x7f22ac0746a0 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22a97fa700 1 -- 172.21.15.83:0/147764944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a0005450 con 0x7f22ac0746a0 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b2fb6700 1 -- 172.21.15.83:0/147764944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac1452e0 con 0x7f22ac0746a0 2023-11-26T07:16:17.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22a97fa700 1 -- 172.21.15.83:0/147764944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0006aa0 con 0x7f22ac0746a0 2023-11-26T07:16:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.725+0000 7f22b2fb6700 1 -- 172.21.15.83:0/147764944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac147ba0 con 0x7f22ac0746a0 2023-11-26T07:16:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.726+0000 7f22b2fb6700 1 -- 172.21.15.83:0/147764944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac147d30 con 0x7f22ac0746a0 2023-11-26T07:16:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.727+0000 7f22a97fa700 1 -- 172.21.15.83:0/147764944 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f22a0003680 con 0x7f22ac0746a0 2023-11-26T07:16:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.727+0000 7f22a97fa700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f229006aff0 0x7f229006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.727+0000 7f22a97fa700 1 -- 172.21.15.83:0/147764944 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f22a00888b0 con 0x7f22ac0746a0 2023-11-26T07:16:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.727+0000 7f22abfff700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f229006aff0 0x7f229006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.728+0000 7f22abfff700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f229006aff0 0x7f229006d4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2298000c00 tx=0x7f2298008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.732+0000 7f22a97fa700 1 -- 172.21.15.83:0/147764944 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f22a0053940 con 0x7f22ac0746a0 2023-11-26T07:16:17.967 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:17.965+0000 7f22b2fb6700 1 -- 172.21.15.83:0/147764944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f22ac068fb0 con 0x7f22ac0746a0 2023-11-26T07:16:18.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.580+0000 7f22a97fa700 1 -- 172.21.15.83:0/147764944 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v51) v1 ==== 174+0+0 (secure 0 0 0) 0x7f22a0056f60 con 0x7f22ac0746a0 2023-11-26T07:16:18.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.583+0000 7f22b2fb6700 1 -- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f229006aff0 msgr2=0x7f229006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:18.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.583+0000 7f22b2fb6700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f229006aff0 0x7f229006d4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2298000c00 tx=0x7f2298008030 comp rx=0 tx=0).stop 2023-11-26T07:16:18.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.583+0000 7f22b2fb6700 1 -- 172.21.15.83:0/147764944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ac0746a0 msgr2=0x7f22ac1590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:18.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.583+0000 7f22b2fb6700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ac0746a0 0x7f22ac1590f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c430 tx=0x7f22a000c7f0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.583+0000 7f22b2fb6700 1 -- 172.21.15.83:0/147764944 shutdown_connections 2023-11-26T07:16:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.583+0000 7f22b2fb6700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f229006aff0 0x7f229006d4b0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2298000c00 tx=0x7f2298008030 comp rx=0 tx=0).stop 2023-11-26T07:16:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.583+0000 7f22b2fb6700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22ac15e160 0x7f22ac1447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.583+0000 7f22b2fb6700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22ac0746a0 0x7f22ac1590f0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.583+0000 7f22b2fb6700 1 --2- 172.21.15.83:0/147764944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22ac0afc90 0x7f22ac1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.583+0000 7f22b2fb6700 1 -- 172.21.15.83:0/147764944 >> 172.21.15.83:0/147764944 conn(0x7f22ac0d0bf0 msgr2=0x7f22ac06df50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.584+0000 7f22b2fb6700 1 -- 172.21.15.83:0/147764944 shutdown_connections 2023-11-26T07:16:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.584+0000 7f22b2fb6700 1 -- 172.21.15.83:0/147764944 wait complete. 2023-11-26T07:16:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 12 and data pool 13 2023-11-26T07:16:18.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.732+0000 7f2981826700 1 Processor -- start 2023-11-26T07:16:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.732+0000 7f2981826700 1 -- start start 2023-11-26T07:16:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.732+0000 7f2981826700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f297c15e1e0 0x7f297c0d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:18.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.732+0000 7f2981826700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0d3a00 0x7f297c0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.732+0000 7f2981826700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f297c0d0db0 0x7f297c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.732+0000 7f2981826700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c0d3150 con 0x7f297c15e1e0 2023-11-26T07:16:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f2981826700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f297c0d1240 con 0x7f297c0d0db0 2023-11-26T07:16:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f2981826700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c15e5c0 con 0x7f297c0d3a00 2023-11-26T07:16:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f297affd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f297c15e1e0 0x7f297c0d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f297affd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f297c15e1e0 0x7f297c0d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40524/0 (socket says 172.21.15.83:40524) 2023-11-26T07:16:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f297affd700 1 -- 172.21.15.83:0/3375537723 learned_addr learned my addr 172.21.15.83:0/3375537723 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f297affd700 1 -- 172.21.15.83:0/3375537723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0d3a00 msgr2=0x7f297c0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f297affd700 1 --2- 172.21.15.83:0/3375537723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0d3a00 0x7f297c0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f297affd700 1 -- 172.21.15.83:0/3375537723 >> v2:172.21.15.203:3300/0 conn(0x7f297c0d0db0 msgr2=0x7f297c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f297affd700 1 --2- 172.21.15.83:0/3375537723 >> v2:172.21.15.203:3300/0 conn(0x7f297c0d0db0 0x7f297c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f297affd700 1 -- 172.21.15.83:0/3375537723 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0d20b0 con 0x7f297c15e1e0 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f297affd700 1 --2- 172.21.15.83:0/3375537723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15e1e0 0x7f297c0d34c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f297c0ce440 tx=0x7f296c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4121c7ed1d3ff9c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f2979ffb700 1 -- 172.21.15.83:0/3375537723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c00b860 con 0x7f297c15e1e0 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f2979ffb700 1 -- 172.21.15.83:0/3375537723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c00b9c0 con 0x7f297c15e1e0 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.733+0000 7f2979ffb700 1 -- 172.21.15.83:0/3375537723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c00bd50 con 0x7f297c15e1e0 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.734+0000 7f2981826700 1 -- 172.21.15.83:0/3375537723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15e1e0 msgr2=0x7f297c0d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.734+0000 7f2981826700 1 --2- 172.21.15.83:0/3375537723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15e1e0 0x7f297c0d34c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f297c0ce440 tx=0x7f296c009c00 comp rx=0 tx=0).stop 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.734+0000 7f2981826700 1 -- 172.21.15.83:0/3375537723 shutdown_connections 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.734+0000 7f2981826700 1 --2- 172.21.15.83:0/3375537723 >> v2:172.21.15.203:3300/0 conn(0x7f297c0d0db0 0x7f297c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.734+0000 7f2981826700 1 --2- 172.21.15.83:0/3375537723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15e1e0 0x7f297c0d34c0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.734+0000 7f2981826700 1 --2- 172.21.15.83:0/3375537723 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0d3a00 0x7f297c0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.734+0000 7f2981826700 1 -- 172.21.15.83:0/3375537723 >> 172.21.15.83:0/3375537723 conn(0x7f297c0b5b50 msgr2=0x7f297c0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.734+0000 7f2981826700 1 -- 172.21.15.83:0/3375537723 shutdown_connections 2023-11-26T07:16:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.734+0000 7f2981826700 1 -- 172.21.15.83:0/3375537723 wait complete. 2023-11-26T07:16:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.735+0000 7f2981826700 1 Processor -- start 2023-11-26T07:16:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.736+0000 7f2981826700 1 -- start start 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.736+0000 7f2981826700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0d0db0 0x7f297c151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.736+0000 7f2981826700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c0d3a00 0x7f297c0ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.736+0000 7f2981826700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15e1e0 0x7f297c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.736+0000 7f2981826700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0aa840 con 0x7f297c15e1e0 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.736+0000 7f2981826700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c1519a0 con 0x7f297c0d3a00 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.736+0000 7f2981826700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297c0aaa90 con 0x7f297c0d0db0 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f297b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15e1e0 0x7f297c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f297b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15e1e0 0x7f297c0cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40534/0 (socket says 172.21.15.83:40534) 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f297b7fe700 1 -- 172.21.15.83:0/2700875545 learned_addr learned my addr 172.21.15.83:0/2700875545 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f297a7fc700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c0d3a00 0x7f297c0ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f297affd700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0d0db0 0x7f297c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f297b7fe700 1 -- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0d0db0 msgr2=0x7f297c151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f297b7fe700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0d0db0 0x7f297c151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f297b7fe700 1 -- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c0d3a00 msgr2=0x7f297c0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f297b7fe700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c0d3a00 0x7f297c0ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f297b7fe700 1 -- 172.21.15.83:0/2700875545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c009550 con 0x7f297c15e1e0 2023-11-26T07:16:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f297b7fe700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15e1e0 0x7f297c0cd500 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f297400b970 tx=0x7f297400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f2963fff700 1 -- 172.21.15.83:0/2700875545 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297400d860 con 0x7f297c15e1e0 2023-11-26T07:16:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f2963fff700 1 -- 172.21.15.83:0/2700875545 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297400d9e0 con 0x7f297c15e1e0 2023-11-26T07:16:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f2963fff700 1 -- 172.21.15.83:0/2700875545 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297400dd90 con 0x7f297c15e1e0 2023-11-26T07:16:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f2981826700 1 -- 172.21.15.83:0/2700875545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c0aad70 con 0x7f297c15e1e0 2023-11-26T07:16:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.737+0000 7f2981826700 1 -- 172.21.15.83:0/2700875545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c0ab330 con 0x7f297c15e1e0 2023-11-26T07:16:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.738+0000 7f2981826700 1 -- 172.21.15.83:0/2700875545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c0751e0 con 0x7f297c15e1e0 2023-11-26T07:16:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.739+0000 7f2963fff700 1 -- 172.21.15.83:0/2700875545 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f297400e430 con 0x7f297c15e1e0 2023-11-26T07:16:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.739+0000 7f2963fff700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f296806aff0 0x7f296806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.739+0000 7f2963fff700 1 -- 172.21.15.83:0/2700875545 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2974006e40 con 0x7f297c15e1e0 2023-11-26T07:16:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.739+0000 7f297affd700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f296806aff0 0x7f296806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.740+0000 7f297affd700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f296806aff0 0x7f296806d4b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f296c000c00 tx=0x7f296c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.743+0000 7f2963fff700 1 -- 172.21.15.83:0/2700875545 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29740548a0 con 0x7f297c15e1e0 2023-11-26T07:16:18.936 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.934+0000 7f2981826700 1 -- 172.21.15.83:0/2700875545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f297c0b5410 con 0x7f297c15e1e0 2023-11-26T07:16:18.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.935+0000 7f2963fff700 1 -- 172.21.15.83:0/2700875545 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16561 (secure 0 0 0) 0x7f297401c090 con 0x7f297c15e1e0 2023-11-26T07:16:18.938 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:18.938 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:17.967212+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:16:15.747889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:16:16.943249+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:18.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 -- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f296806aff0 msgr2=0x7f296806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:18.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f296806aff0 0x7f296806d4b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f296c000c00 tx=0x7f296c008030 comp rx=0 tx=0).stop 2023-11-26T07:16:18.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 -- 172.21.15.83:0/2700875545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15e1e0 msgr2=0x7f297c0cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:18.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15e1e0 0x7f297c0cd500 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f297400b970 tx=0x7f297400bc80 comp rx=0 tx=0).stop 2023-11-26T07:16:18.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 -- 172.21.15.83:0/2700875545 shutdown_connections 2023-11-26T07:16:18.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f296806aff0 0x7f296806d4b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c15e1e0 0x7f297c0cd500 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297c0d0db0 0x7f297c151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 --2- 172.21.15.83:0/2700875545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f297c0d3a00 0x7f297c0ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:18.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 -- 172.21.15.83:0/2700875545 >> 172.21.15.83:0/2700875545 conn(0x7f297c0b5b50 msgr2=0x7f297c0b9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:18.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 -- 172.21.15.83:0/2700875545 shutdown_connections 2023-11-26T07:16:18.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:18.937+0000 7f2981826700 1 -- 172.21.15.83:0/2700875545 wait complete. 2023-11-26T07:16:18.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:19.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.069+0000 7f3b383a3700 1 Processor -- start 2023-11-26T07:16:19.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.069+0000 7f3b383a3700 1 -- start start 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.070+0000 7f3b383a3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b3015e080 0x7f3b300b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.070+0000 7f3b383a3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b300af4d0 0x7f3b300afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.070+0000 7f3b383a3700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b300ae430 con 0x7f3b3006c000 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.070+0000 7f3b383a3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b300af0b0 con 0x7f3b3015e080 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.070+0000 7f3b383a3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b300b05a0 con 0x7f3b300af4d0 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3613f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b3015e080 0x7f3b300b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3613f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b3015e080 0x7f3b300b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40538/0 (socket says 172.21.15.83:40538) 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3613f700 1 -- 172.21.15.83:0/3643417117 learned_addr learned my addr 172.21.15.83:0/3643417117 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3593e700 1 --2- 172.21.15.83:0/3643417117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b300af4d0 0x7f3b300afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3613f700 1 -- 172.21.15.83:0/3643417117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b300af4d0 msgr2=0x7f3b300afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3613f700 1 --2- 172.21.15.83:0/3643417117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b300af4d0 0x7f3b300afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3613f700 1 -- 172.21.15.83:0/3643417117 >> v1:172.21.15.203:6789/0 conn(0x7f3b3006c000 legacy=0x7f3b30073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3613f700 1 -- 172.21.15.83:0/3643417117 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b300ae710 con 0x7f3b3015e080 2023-11-26T07:16:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3613f700 1 --2- 172.21.15.83:0/3643417117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b3015e080 0x7f3b300b5050 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009920 tx=0x7f3b20009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd460bc5ce492cb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3513d700 1 -- 172.21.15.83:0/3643417117 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20011830 con 0x7f3b3015e080 2023-11-26T07:16:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3513d700 1 -- 172.21.15.83:0/3643417117 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b20011990 con 0x7f3b3015e080 2023-11-26T07:16:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.071+0000 7f3b3513d700 1 -- 172.21.15.83:0/3643417117 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20011d20 con 0x7f3b3015e080 2023-11-26T07:16:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.072+0000 7f3b383a3700 1 -- 172.21.15.83:0/3643417117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b3015e080 msgr2=0x7f3b300b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.072+0000 7f3b383a3700 1 --2- 172.21.15.83:0/3643417117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b3015e080 0x7f3b300b5050 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009920 tx=0x7f3b20009c30 comp rx=0 tx=0).stop 2023-11-26T07:16:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.072+0000 7f3b383a3700 1 -- 172.21.15.83:0/3643417117 shutdown_connections 2023-11-26T07:16:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.072+0000 7f3b383a3700 1 --2- 172.21.15.83:0/3643417117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b3015e080 0x7f3b300b5050 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.072+0000 7f3b383a3700 1 --2- 172.21.15.83:0/3643417117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b300af4d0 0x7f3b300afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.072+0000 7f3b383a3700 1 -- 172.21.15.83:0/3643417117 >> 172.21.15.83:0/3643417117 conn(0x7f3b30071b90 msgr2=0x7f3b30071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:19.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.072+0000 7f3b383a3700 1 -- 172.21.15.83:0/3643417117 shutdown_connections 2023-11-26T07:16:19.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.072+0000 7f3b383a3700 1 -- 172.21.15.83:0/3643417117 wait complete. 2023-11-26T07:16:19.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.073+0000 7f3b383a3700 1 Processor -- start 2023-11-26T07:16:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.073+0000 7f3b383a3700 1 -- start start 2023-11-26T07:16:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b383a3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b3006c000 0x7f3b30156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b383a3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b300af4d0 0x7f3b30157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b383a3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b3015e080 0x7f3b301452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b383a3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b301f7d00 con 0x7f3b3006c000 2023-11-26T07:16:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b383a3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b301579b0 con 0x7f3b3015e080 2023-11-26T07:16:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b383a3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b301f7f70 con 0x7f3b300af4d0 2023-11-26T07:16:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b3613f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b3006c000 0x7f3b30156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b3613f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b3006c000 0x7f3b30156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40544/0 (socket says 172.21.15.83:40544) 2023-11-26T07:16:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b3613f700 1 -- 172.21.15.83:0/628024766 learned_addr learned my addr 172.21.15.83:0/628024766 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:19.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b3593e700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b300af4d0 0x7f3b30157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b3613f700 1 -- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b300af4d0 msgr2=0x7f3b30157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b3613f700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b300af4d0 0x7f3b30157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b3613f700 1 -- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b3015e080 msgr2=0x7f3b301452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b3613f700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b3015e080 0x7f3b301452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b3613f700 1 -- 172.21.15.83:0/628024766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b20009550 con 0x7f3b3006c000 2023-11-26T07:16:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b3613f700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b3006c000 0x7f3b30156ad0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009130 tx=0x7f3b20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b26ffd700 1 -- 172.21.15.83:0/628024766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200036a0 con 0x7f3b3006c000 2023-11-26T07:16:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b26ffd700 1 -- 172.21.15.83:0/628024766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2001e070 con 0x7f3b3006c000 2023-11-26T07:16:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b383a3700 1 -- 172.21.15.83:0/628024766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b30159190 con 0x7f3b3006c000 2023-11-26T07:16:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.074+0000 7f3b26ffd700 1 -- 172.21.15.83:0/628024766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20003a60 con 0x7f3b3006c000 2023-11-26T07:16:19.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.075+0000 7f3b383a3700 1 -- 172.21.15.83:0/628024766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b30158e40 con 0x7f3b3006c000 2023-11-26T07:16:19.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.075+0000 7f3b383a3700 1 -- 172.21.15.83:0/628024766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b300753f0 con 0x7f3b3006c000 2023-11-26T07:16:19.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.076+0000 7f3b26ffd700 1 -- 172.21.15.83:0/628024766 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f3b20003bc0 con 0x7f3b3006c000 2023-11-26T07:16:19.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.076+0000 7f3b26ffd700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3b1406aff0 0x7f3b1406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.076+0000 7f3b26ffd700 1 -- 172.21.15.83:0/628024766 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b2008a6c0 con 0x7f3b3006c000 2023-11-26T07:16:19.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.076+0000 7f3b3593e700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3b1406aff0 0x7f3b1406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:19.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.077+0000 7f3b3593e700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3b1406aff0 0x7f3b1406d4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b301560b0 tx=0x7f3b2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:19.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.080+0000 7f3b26ffd700 1 -- 172.21.15.83:0/628024766 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b20055670 con 0x7f3b3006c000 2023-11-26T07:16:19.272 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.270+0000 7f3b383a3700 1 -- 172.21.15.83:0/628024766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b30158fd0 con 0x7f3b3006c000 2023-11-26T07:16:19.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.271+0000 7f3b26ffd700 1 -- 172.21.15.83:0/628024766 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16561 (secure 0 0 0) 0x7f3b20058c90 con 0x7f3b3006c000 2023-11-26T07:16:19.274 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:19.274 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:17.967212+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:16:15.747889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:16:16.943249+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:19.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.273+0000 7f3b383a3700 1 -- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3b1406aff0 msgr2=0x7f3b1406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:19.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.273+0000 7f3b383a3700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3b1406aff0 0x7f3b1406d4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b301560b0 tx=0x7f3b2c008040 comp rx=0 tx=0).stop 2023-11-26T07:16:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.273+0000 7f3b383a3700 1 -- 172.21.15.83:0/628024766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b3006c000 msgr2=0x7f3b30156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.273+0000 7f3b383a3700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b3006c000 0x7f3b30156ad0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009130 tx=0x7f3b20004750 comp rx=0 tx=0).stop 2023-11-26T07:16:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.273+0000 7f3b383a3700 1 -- 172.21.15.83:0/628024766 shutdown_connections 2023-11-26T07:16:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.273+0000 7f3b383a3700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3b1406aff0 0x7f3b1406d4b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.273+0000 7f3b383a3700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b3015e080 0x7f3b301452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.273+0000 7f3b383a3700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b3006c000 0x7f3b30156ad0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.273+0000 7f3b383a3700 1 --2- 172.21.15.83:0/628024766 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b300af4d0 0x7f3b30157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.274+0000 7f3b383a3700 1 -- 172.21.15.83:0/628024766 >> 172.21.15.83:0/628024766 conn(0x7f3b30071b90 msgr2=0x7f3b300b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.274+0000 7f3b383a3700 1 -- 172.21.15.83:0/628024766 shutdown_connections 2023-11-26T07:16:19.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.274+0000 7f3b383a3700 1 -- 172.21.15.83:0/628024766 wait complete. 2023-11-26T07:16:19.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:16:19.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.409+0000 7f6bf0101700 1 Processor -- start 2023-11-26T07:16:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.409+0000 7f6bf0101700 1 -- start start 2023-11-26T07:16:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bf0101700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6be80c6c30 0x7f6be80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bf0101700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6be80cef40 0x7f6be80cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bf0101700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be80d0140 con 0x7f6be815bf50 2023-11-26T07:16:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bf0101700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be815c330 con 0x7f6be80c6c30 2023-11-26T07:16:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bf0101700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be80cf3d0 con 0x7f6be80cef40 2023-11-26T07:16:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bed69c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6be80c6c30 0x7f6be80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bed69c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6be80c6c30 0x7f6be80cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40554/0 (socket says 172.21.15.83:40554) 2023-11-26T07:16:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bed69c700 1 -- 172.21.15.83:0/346863946 learned_addr learned my addr 172.21.15.83:0/346863946 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bee69e700 1 --2- 172.21.15.83:0/346863946 >> v2:172.21.15.203:3300/0 conn(0x7f6be80cef40 0x7f6be80cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bed69c700 1 -- 172.21.15.83:0/346863946 >> v2:172.21.15.203:3300/0 conn(0x7f6be80cef40 msgr2=0x7f6be80cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bed69c700 1 --2- 172.21.15.83:0/346863946 >> v2:172.21.15.203:3300/0 conn(0x7f6be80cef40 0x7f6be80cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bed69c700 1 -- 172.21.15.83:0/346863946 >> v1:172.21.15.203:6789/0 conn(0x7f6be815bf50 legacy=0x7f6be80ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.410+0000 7f6bed69c700 1 -- 172.21.15.83:0/346863946 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be80d03e0 con 0x7f6be80c6c30 2023-11-26T07:16:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.411+0000 7f6bed69c700 1 --2- 172.21.15.83:0/346863946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be80c6c30 0x7f6be80cea00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6be80c8c00 tx=0x7f6be4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf373143243833f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.411+0000 7f6bece9b700 1 -- 172.21.15.83:0/346863946 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be400b870 con 0x7f6be80c6c30 2023-11-26T07:16:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.411+0000 7f6bece9b700 1 -- 172.21.15.83:0/346863946 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be400b9d0 con 0x7f6be80c6c30 2023-11-26T07:16:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.411+0000 7f6bece9b700 1 -- 172.21.15.83:0/346863946 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be400bd60 con 0x7f6be80c6c30 2023-11-26T07:16:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.411+0000 7f6bf0101700 1 -- 172.21.15.83:0/346863946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be80c6c30 msgr2=0x7f6be80cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.411+0000 7f6bf0101700 1 --2- 172.21.15.83:0/346863946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be80c6c30 0x7f6be80cea00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6be80c8c00 tx=0x7f6be4009bd0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.411+0000 7f6bf0101700 1 -- 172.21.15.83:0/346863946 shutdown_connections 2023-11-26T07:16:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.411+0000 7f6bf0101700 1 --2- 172.21.15.83:0/346863946 >> v2:172.21.15.203:3300/0 conn(0x7f6be80cef40 0x7f6be80cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.411+0000 7f6bf0101700 1 --2- 172.21.15.83:0/346863946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be80c6c30 0x7f6be80cea00 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.411+0000 7f6bf0101700 1 -- 172.21.15.83:0/346863946 >> 172.21.15.83:0/346863946 conn(0x7f6be80b3780 msgr2=0x7f6be80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.411+0000 7f6bf0101700 1 -- 172.21.15.83:0/346863946 shutdown_connections 2023-11-26T07:16:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.412+0000 7f6bf0101700 1 -- 172.21.15.83:0/346863946 wait complete. 2023-11-26T07:16:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.413+0000 7f6bf0101700 1 Processor -- start 2023-11-26T07:16:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.413+0000 7f6bf0101700 1 -- start start 2023-11-26T07:16:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.413+0000 7f6bf0101700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6be80c6c30 0x7f6be81572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.413+0000 7f6bf0101700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6be80cef40 0x7f6be81550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.413+0000 7f6bf0101700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be815bf50 0x7f6be81f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.413+0000 7f6bf0101700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be81f26f0 con 0x7f6be815bf50 2023-11-26T07:16:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.413+0000 7f6bf0101700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be81f29d0 con 0x7f6be80cef40 2023-11-26T07:16:19.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.413+0000 7f6bf0101700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be81f2cb0 con 0x7f6be80c6c30 2023-11-26T07:16:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bee69e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be815bf50 0x7f6be81f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bee69e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be815bf50 0x7f6be81f1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40560/0 (socket says 172.21.15.83:40560) 2023-11-26T07:16:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bee69e700 1 -- 172.21.15.83:0/2403286290 learned_addr learned my addr 172.21.15.83:0/2403286290 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bed69c700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6be80cef40 0x7f6be81550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bede9d700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6be80c6c30 0x7f6be81572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bee69e700 1 -- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6be80c6c30 msgr2=0x7f6be81572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bee69e700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6be80c6c30 0x7f6be81572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bee69e700 1 -- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6be80cef40 msgr2=0x7f6be81550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bee69e700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6be80cef40 0x7f6be81550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bee69e700 1 -- 172.21.15.83:0/2403286290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be4009550 con 0x7f6be815bf50 2023-11-26T07:16:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bee69e700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be815bf50 0x7f6be81f1e20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6be0007e40 tx=0x7f6be000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bde7fc700 1 -- 172.21.15.83:0/2403286290 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be000ed30 con 0x7f6be815bf50 2023-11-26T07:16:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bde7fc700 1 -- 172.21.15.83:0/2403286290 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be0014420 con 0x7f6be815bf50 2023-11-26T07:16:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bf0101700 1 -- 172.21.15.83:0/2403286290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be80c8c80 con 0x7f6be815bf50 2023-11-26T07:16:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bde7fc700 1 -- 172.21.15.83:0/2403286290 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be00147d0 con 0x7f6be815bf50 2023-11-26T07:16:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.414+0000 7f6bf0101700 1 -- 172.21.15.83:0/2403286290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be81f2360 con 0x7f6be815bf50 2023-11-26T07:16:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.415+0000 7f6bf0101700 1 -- 172.21.15.83:0/2403286290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be80cdc90 con 0x7f6be815bf50 2023-11-26T07:16:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.416+0000 7f6bde7fc700 1 -- 172.21.15.83:0/2403286290 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f6be0014930 con 0x7f6be815bf50 2023-11-26T07:16:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.416+0000 7f6bde7fc700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6bcc06aff0 0x7f6bcc06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.416+0000 7f6bde7fc700 1 -- 172.21.15.83:0/2403286290 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6be0089e00 con 0x7f6be815bf50 2023-11-26T07:16:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.416+0000 7f6bede9d700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6bcc06aff0 0x7f6bcc06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.417+0000 7f6bede9d700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6bcc06aff0 0x7f6bcc06d4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8007fd0 tx=0x7f6bd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:19.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.420+0000 7f6bde7fc700 1 -- 172.21.15.83:0/2403286290 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6be0054da0 con 0x7f6be815bf50 2023-11-26T07:16:19.658 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:19.656+0000 7f6bf0101700 1 -- 172.21.15.83:0/2403286290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6be8068fb0 con 0x7f6be815bf50 2023-11-26T07:16:20.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.618+0000 7f6bde7fc700 1 -- 172.21.15.83:0/2403286290 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6be00583c0 con 0x7f6be815bf50 2023-11-26T07:16:20.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 -- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6bcc06aff0 msgr2=0x7f6bcc06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6bcc06aff0 0x7f6bcc06d4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8007fd0 tx=0x7f6bd8009040 comp rx=0 tx=0).stop 2023-11-26T07:16:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 -- 172.21.15.83:0/2403286290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be815bf50 msgr2=0x7f6be81f1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be815bf50 0x7f6be81f1e20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6be0007e40 tx=0x7f6be000e3c0 comp rx=0 tx=0).stop 2023-11-26T07:16:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 -- 172.21.15.83:0/2403286290 shutdown_connections 2023-11-26T07:16:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6bcc06aff0 0x7f6bcc06d4b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be815bf50 0x7f6be81f1e20 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6be80c6c30 0x7f6be81572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 --2- 172.21.15.83:0/2403286290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6be80cef40 0x7f6be81550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 -- 172.21.15.83:0/2403286290 >> 172.21.15.83:0/2403286290 conn(0x7f6be80b3780 msgr2=0x7f6be806ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:20.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 -- 172.21.15.83:0/2403286290 shutdown_connections 2023-11-26T07:16:20.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.621+0000 7f6bf0101700 1 -- 172.21.15.83:0/2403286290 wait complete. 2023-11-26T07:16:20.638 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:16:20.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:16:20.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.765+0000 7f3ed97a8700 1 Processor -- start 2023-11-26T07:16:20.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.765+0000 7f3ed97a8700 1 -- start start 2023-11-26T07:16:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed97a8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3ed40d45d0 0x7f3ed40b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed97a8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40d39d0 0x7f3ed40b7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed97a8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed40d2930 con 0x7f3ed415e140 2023-11-26T07:16:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed97a8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed415e520 con 0x7f3ed40d45d0 2023-11-26T07:16:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed97a8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40d2c10 con 0x7f3ed40d39d0 2023-11-26T07:16:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed2ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3ed415e140 0x7f3ed40d3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60606/0 (socket says 172.21.15.83:60606) 2023-11-26T07:16:20.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed2ffd700 1 -- 172.21.15.83:0/290371739 learned_addr learned my addr 172.21.15.83:0/290371739 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541273778 0 0) 0x7f3ed40d2930 con 0x7f3ed415e140 2023-11-26T07:16:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ebc0035b0 con 0x7f3ed415e140 2023-11-26T07:16:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed27fc700 1 --2- 172.21.15.83:0/290371739 >> v2:172.21.15.203:3300/0 conn(0x7f3ed40d45d0 0x7f3ed40b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed37fe700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40d39d0 0x7f3ed40b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2535312754 0 0) 0x7f3ebc0035b0 con 0x7f3ed415e140 2023-11-26T07:16:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40d39d0 msgr2=0x7f3ed40b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed1ffb700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40d39d0 0x7f3ed40b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 >> v2:172.21.15.203:3300/0 conn(0x7f3ed40d45d0 msgr2=0x7f3ed40b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed1ffb700 1 --2- 172.21.15.83:0/290371739 >> v2:172.21.15.203:3300/0 conn(0x7f3ed40d45d0 0x7f3ed40b70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:20.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed40d2ef0 con 0x7f3ed415e140 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.766+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f3ec8001670 con 0x7f3ed415e140 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 >> v1:172.21.15.83:6789/0 conn(0x7f3ed415e140 legacy=0x7f3ed40d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed1ffb700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ebc003a00 0x7f3ebc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed1ffb700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ebc004850 0x7f3ebc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed1ffb700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40d39d0 0x7f3ebc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed415e520 con 0x7f3ebc003a00 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40d2c10 con 0x7f3ebc004850 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc00ba10 con 0x7f3ed40d39d0 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed2ffd700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ebc003a00 0x7f3ebc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed37fe700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40d39d0 0x7f3ebc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed27fc700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ebc004850 0x7f3ebc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:20.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed2ffd700 1 -- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40d39d0 msgr2=0x7f3ebc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed2ffd700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40d39d0 0x7f3ebc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed2ffd700 1 -- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ebc004850 msgr2=0x7f3ebc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed2ffd700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ebc004850 0x7f3ebc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed2ffd700 1 -- 172.21.15.83:0/290371739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc0035b0 con 0x7f3ebc003a00 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed37fe700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40d39d0 0x7f3ebc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed2ffd700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ebc003a00 0x7f3ebc003e40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800c370 tx=0x7f3ebc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c28feb99e353b7e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8007c60 con 0x7f3ebc003a00 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec800ce50 con 0x7f3ebc003a00 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.767+0000 7f3ed1ffb700 1 -- 172.21.15.83:0/290371739 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8005ba0 con 0x7f3ebc003a00 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.768+0000 7f3ed97a8700 1 -- 172.21.15.83:0/290371739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ebc003a00 msgr2=0x7f3ebc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.768+0000 7f3ed97a8700 1 --2- 172.21.15.83:0/290371739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ebc003a00 0x7f3ebc003e40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800c370 tx=0x7f3ebc005880 comp rx=0 tx=0).stop 2023-11-26T07:16:20.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.768+0000 7f3ed37fe700 1 -- 172.21.15.83:0/290371739 reap_dead start 2023-11-26T07:16:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.768+0000 7f3ed97a8700 1 -- 172.21.15.83:0/290371739 shutdown_connections 2023-11-26T07:16:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.768+0000 7f3ed97a8700 1 -- 172.21.15.83:0/290371739 >> 172.21.15.83:0/290371739 conn(0x7f3ed406b410 msgr2=0x7f3ed406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.768+0000 7f3ed97a8700 1 -- 172.21.15.83:0/290371739 shutdown_connections 2023-11-26T07:16:20.775 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.768+0000 7f3ed97a8700 1 -- 172.21.15.83:0/290371739 wait complete. 2023-11-26T07:16:20.777 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.769+0000 7f3ed97a8700 1 Processor -- start 2023-11-26T07:16:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.770+0000 7f3ed97a8700 1 -- start start 2023-11-26T07:16:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.770+0000 7f3ed97a8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ebc003a00 0x7f3ed41609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.770+0000 7f3ed97a8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed41511c0 0x7f3ed4151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.770+0000 7f3ed97a8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40cc0f0 0x7f3ed40cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.770+0000 7f3ed97a8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40b5aa0 con 0x7f3ed41511c0 2023-11-26T07:16:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.770+0000 7f3ed97a8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed4151b80 con 0x7f3ebc003a00 2023-11-26T07:16:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.770+0000 7f3ed97a8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40ccab0 con 0x7f3ed40cc0f0 2023-11-26T07:16:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed41511c0 0x7f3ed4151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed27fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed41511c0 0x7f3ed4151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40580/0 (socket says 172.21.15.83:40580) 2023-11-26T07:16:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed27fc700 1 -- 172.21.15.83:0/2091097853 learned_addr learned my addr 172.21.15.83:0/2091097853 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:20.778 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed2ffd700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ebc003a00 0x7f3ed41609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed37fe700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40cc0f0 0x7f3ed40cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed27fc700 1 -- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40cc0f0 msgr2=0x7f3ed40cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed27fc700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40cc0f0 0x7f3ed40cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed27fc700 1 -- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ebc003a00 msgr2=0x7f3ed41609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed27fc700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ebc003a00 0x7f3ed41609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed27fc700 1 -- 172.21.15.83:0/2091097853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed40b5cf0 con 0x7f3ed41511c0 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed2ffd700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ebc003a00 0x7f3ed41609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed27fc700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed41511c0 0x7f3ed4151640 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4011ad0 tx=0x7f3ec4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ebb7fe700 1 -- 172.21.15.83:0/2091097853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec400fa00 con 0x7f3ed41511c0 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ebb7fe700 1 -- 172.21.15.83:0/2091097853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec400fb80 con 0x7f3ed41511c0 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ebb7fe700 1 -- 172.21.15.83:0/2091097853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4014440 con 0x7f3ed41511c0 2023-11-26T07:16:20.779 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed97a8700 1 -- 172.21.15.83:0/2091097853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed40b5fd0 con 0x7f3ed41511c0 2023-11-26T07:16:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.771+0000 7f3ed97a8700 1 -- 172.21.15.83:0/2091097853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed40b6590 con 0x7f3ed41511c0 2023-11-26T07:16:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.772+0000 7f3ed97a8700 1 -- 172.21.15.83:0/2091097853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed4068fb0 con 0x7f3ed41511c0 2023-11-26T07:16:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.773+0000 7f3ebb7fe700 1 -- 172.21.15.83:0/2091097853 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f3ec40145a0 con 0x7f3ed41511c0 2023-11-26T07:16:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.773+0000 7f3ebb7fe700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ec006aff0 0x7f3ec006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.773+0000 7f3ebb7fe700 1 -- 172.21.15.83:0/2091097853 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ec4089e50 con 0x7f3ed41511c0 2023-11-26T07:16:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.773+0000 7f3ed2ffd700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ec006aff0 0x7f3ec006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.774+0000 7f3ed2ffd700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ec006aff0 0x7f3ec006d4b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8015040 tx=0x7f3ec800a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:20.780 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:20.778+0000 7f3ebb7fe700 1 -- 172.21.15.83:0/2091097853 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ec4054eb0 con 0x7f3ed41511c0 2023-11-26T07:16:21.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.012+0000 7f3ed97a8700 1 -- 172.21.15.83:0/2091097853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3ed40d1bb0 con 0x7f3ed41511c0 2023-11-26T07:16:21.623 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.621+0000 7f3ebb7fe700 1 -- 172.21.15.83:0/2091097853 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3ec4022020 con 0x7f3ed41511c0 2023-11-26T07:16:21.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.624+0000 7f3ed97a8700 1 -- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ec006aff0 msgr2=0x7f3ec006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:21.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.625+0000 7f3ed97a8700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ec006aff0 0x7f3ec006d4b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8015040 tx=0x7f3ec800a000 comp rx=0 tx=0).stop 2023-11-26T07:16:21.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.625+0000 7f3ed97a8700 1 -- 172.21.15.83:0/2091097853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed41511c0 msgr2=0x7f3ed4151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.625+0000 7f3ed97a8700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed41511c0 0x7f3ed4151640 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4011ad0 tx=0x7f3ec4011e90 comp rx=0 tx=0).stop 2023-11-26T07:16:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.625+0000 7f3ed97a8700 1 -- 172.21.15.83:0/2091097853 shutdown_connections 2023-11-26T07:16:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.625+0000 7f3ed97a8700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ec006aff0 0x7f3ec006d4b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.625+0000 7f3ed97a8700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ed40cc0f0 0x7f3ed40cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.625+0000 7f3ed97a8700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ebc003a00 0x7f3ed41609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.625+0000 7f3ed97a8700 1 --2- 172.21.15.83:0/2091097853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed41511c0 0x7f3ed4151640 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.625+0000 7f3ed97a8700 1 -- 172.21.15.83:0/2091097853 >> 172.21.15.83:0/2091097853 conn(0x7f3ed406b410 msgr2=0x7f3ed40b8990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.625+0000 7f3ed97a8700 1 -- 172.21.15.83:0/2091097853 shutdown_connections 2023-11-26T07:16:21.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.625+0000 7f3ed97a8700 1 -- 172.21.15.83:0/2091097853 wait complete. 2023-11-26T07:16:21.640 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:16:21.640 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.763+0000 7f5f7addf700 1 Processor -- start 2023-11-26T07:16:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.763+0000 7f5f7addf700 1 -- start start 2023-11-26T07:16:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7addf700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5f740b3370 0x7f5f741604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7addf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f740d1c70 0x7f5f740d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7addf700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f740b22d0 con 0x7f5f7415bdd0 2023-11-26T07:16:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7addf700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f740b2f50 con 0x7f5f740b3370 2023-11-26T07:16:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7addf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740d2640 con 0x7f5f740d1c70 2023-11-26T07:16:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7937c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f740d1c70 0x7f5f740d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f73fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5f740b3370 0x7f5f741604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7937c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f740d1c70 0x7f5f740d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:51048/0 (socket says 172.21.15.83:51048) 2023-11-26T07:16:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f73fff700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5f740b3370 0x7f5f741604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40040/0 (socket says 172.21.15.83:40040) 2023-11-26T07:16:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7937c700 1 -- 172.21.15.83:0/1212367363 learned_addr learned my addr 172.21.15.83:0/1212367363 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f78b7b700 1 --1- 172.21.15.83:0/1212367363 >> v1:172.21.15.203:6789/0 conn(0x7f5f7415bdd0 0x7f5f740b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:45244/0 (socket says 172.21.15.83:45244) 2023-11-26T07:16:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7937c700 1 -- 172.21.15.83:0/1212367363 >> v2:172.21.15.203:3300/0 conn(0x7f5f740b3370 msgr2=0x7f5f741604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7937c700 1 --2- 172.21.15.83:0/1212367363 >> v2:172.21.15.203:3300/0 conn(0x7f5f740b3370 0x7f5f741604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7937c700 1 -- 172.21.15.83:0/1212367363 >> v1:172.21.15.203:6789/0 conn(0x7f5f7415bdd0 legacy=0x7f5f740b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f7937c700 1 -- 172.21.15.83:0/1212367363 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740b25b0 con 0x7f5f740d1c70 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.764+0000 7f5f73fff700 1 --2- 172.21.15.83:0/1212367363 >> v2:172.21.15.203:3300/0 conn(0x7f5f740b3370 0x7f5f741604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f7937c700 1 --2- 172.21.15.83:0/1212367363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f740d1c70 0x7f5f740d2100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009e50 tx=0x7f5f6c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e695c190c95aa3ed server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f737fe700 1 -- 172.21.15.83:0/1212367363 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c00e070 con 0x7f5f740d1c70 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f737fe700 1 -- 172.21.15.83:0/1212367363 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c00b910 con 0x7f5f740d1c70 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f737fe700 1 -- 172.21.15.83:0/1212367363 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c00bcc0 con 0x7f5f740d1c70 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f7addf700 1 -- 172.21.15.83:0/1212367363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f740d1c70 msgr2=0x7f5f740d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f7addf700 1 --2- 172.21.15.83:0/1212367363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f740d1c70 0x7f5f740d2100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009e50 tx=0x7f5f6c00b330 comp rx=0 tx=0).stop 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f7addf700 1 -- 172.21.15.83:0/1212367363 shutdown_connections 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f7addf700 1 --2- 172.21.15.83:0/1212367363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f740d1c70 0x7f5f740d2100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f7addf700 1 --2- 172.21.15.83:0/1212367363 >> v2:172.21.15.203:3300/0 conn(0x7f5f740b3370 0x7f5f741604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f7addf700 1 -- 172.21.15.83:0/1212367363 >> 172.21.15.83:0/1212367363 conn(0x7f5f740731a0 msgr2=0x7f5f7409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:21.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f7addf700 1 -- 172.21.15.83:0/1212367363 shutdown_connections 2023-11-26T07:16:21.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.765+0000 7f5f7addf700 1 -- 172.21.15.83:0/1212367363 wait complete. 2023-11-26T07:16:21.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.767+0000 7f5f7addf700 1 Processor -- start 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.767+0000 7f5f7addf700 1 -- start start 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f7addf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f740b3370 0x7f5f740cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f7addf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f740d1c70 0x7f5f74156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f7addf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f7415bdd0 0x7f5f741568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f7addf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7406c150 con 0x7f5f740d1c70 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f7addf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7406b4d0 con 0x7f5f740b3370 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f7addf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7406b7b0 con 0x7f5f7415bdd0 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f73fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f740d1c70 0x7f5f74156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f73fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f740d1c70 0x7f5f74156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40594/0 (socket says 172.21.15.83:40594) 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f73fff700 1 -- 172.21.15.83:0/3016625842 learned_addr learned my addr 172.21.15.83:0/3016625842 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f78b7b700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f740b3370 0x7f5f740cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f7937c700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f7415bdd0 0x7f5f741568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f73fff700 1 -- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f7415bdd0 msgr2=0x7f5f741568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f73fff700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f7415bdd0 0x7f5f741568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f73fff700 1 -- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f740b3370 msgr2=0x7f5f740cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f73fff700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f740b3370 0x7f5f740cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.768+0000 7f5f73fff700 1 -- 172.21.15.83:0/3016625842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f600094a0 con 0x7f5f740d1c70 2023-11-26T07:16:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.769+0000 7f5f73fff700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f740d1c70 0x7f5f74156370 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000ebe0 tx=0x7f5f60009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.769+0000 7f5f717fa700 1 -- 172.21.15.83:0/3016625842 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000c910 con 0x7f5f740d1c70 2023-11-26T07:16:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.769+0000 7f5f717fa700 1 -- 172.21.15.83:0/3016625842 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6000ca70 con 0x7f5f740d1c70 2023-11-26T07:16:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.769+0000 7f5f717fa700 1 -- 172.21.15.83:0/3016625842 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000cde0 con 0x7f5f740d1c70 2023-11-26T07:16:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.769+0000 7f5f7addf700 1 -- 172.21.15.83:0/3016625842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c009b30 con 0x7f5f740d1c70 2023-11-26T07:16:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.769+0000 7f5f7addf700 1 -- 172.21.15.83:0/3016625842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7406bd70 con 0x7f5f740d1c70 2023-11-26T07:16:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.770+0000 7f5f7addf700 1 -- 172.21.15.83:0/3016625842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7406c290 con 0x7f5f740d1c70 2023-11-26T07:16:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.770+0000 7f5f717fa700 1 -- 172.21.15.83:0/3016625842 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f5f60013470 con 0x7f5f740d1c70 2023-11-26T07:16:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.771+0000 7f5f717fa700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5f5806aff0 0x7f5f5806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.771+0000 7f5f717fa700 1 -- 172.21.15.83:0/3016625842 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f60089900 con 0x7f5f740d1c70 2023-11-26T07:16:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.771+0000 7f5f78b7b700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5f5806aff0 0x7f5f5806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.771+0000 7f5f78b7b700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5f5806aff0 0x7f5f5806d4b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f68000f60 tx=0x7f5f68007d10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:21.775+0000 7f5f717fa700 1 -- 172.21.15.83:0/3016625842 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f7406c290 con 0x7f5f740d1c70 2023-11-26T07:16:22.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.008+0000 7f5f7addf700 1 -- 172.21.15.83:0/3016625842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f7406c290 con 0x7f5f740d1c70 2023-11-26T07:16:22.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.008+0000 7f5f717fa700 1 -- 172.21.15.83:0/3016625842 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f5f7406c290 con 0x7f5f740d1c70 2023-11-26T07:16:22.010 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:22.010 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1015690400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1015690400},{"type":"v1","addr":"172.21.15.203:6834","nonce":1015690400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2716428832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2716428832},{"type":"v1","addr":"172.21.15.83:6834","nonce":2716428832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/469139444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":469139444},{"type":"v1","addr":"172.21.15.83:6835","nonce":469139444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:16:17.967531+0000","modified":"2023-11-26T07:16:21.013928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1901175330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1901175330},{"type":"v1","addr":"172.21.15.203:6835","nonce":1901175330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:16:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.010+0000 7f5f7addf700 1 -- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5f5806aff0 msgr2=0x7f5f5806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.010+0000 7f5f7addf700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5f5806aff0 0x7f5f5806d4b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f68000f60 tx=0x7f5f68007d10 comp rx=0 tx=0).stop 2023-11-26T07:16:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.010+0000 7f5f7addf700 1 -- 172.21.15.83:0/3016625842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f740d1c70 msgr2=0x7f5f74156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.010+0000 7f5f7addf700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f740d1c70 0x7f5f74156370 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000ebe0 tx=0x7f5f60009ec0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.011+0000 7f5f7addf700 1 -- 172.21.15.83:0/3016625842 shutdown_connections 2023-11-26T07:16:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.011+0000 7f5f7addf700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5f5806aff0 0x7f5f5806d4b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.011+0000 7f5f7addf700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5f7415bdd0 0x7f5f741568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.011+0000 7f5f7addf700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5f740b3370 0x7f5f740cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.011+0000 7f5f7addf700 1 --2- 172.21.15.83:0/3016625842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f740d1c70 0x7f5f74156370 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.011+0000 7f5f7addf700 1 -- 172.21.15.83:0/3016625842 >> 172.21.15.83:0/3016625842 conn(0x7f5f740731a0 msgr2=0x7f5f740b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.011+0000 7f5f7addf700 1 -- 172.21.15.83:0/3016625842 shutdown_connections 2023-11-26T07:16:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.011+0000 7f5f7addf700 1 -- 172.21.15.83:0/3016625842 wait complete. 2023-11-26T07:16:22.015 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 55 2023-11-26T07:16:22.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:22.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.144+0000 7fe0250af700 1 Processor -- start 2023-11-26T07:16:22.146 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.144+0000 7fe0250af700 1 -- start start 2023-11-26T07:16:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe0250af700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe02015e080 0x7fe0200b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe0250af700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0200af4d0 0x7fe0200afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe0250af700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0200ae430 con 0x7fe02006c000 2023-11-26T07:16:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe0250af700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200af0b0 con 0x7fe02015e080 2023-11-26T07:16:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe0250af700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200b05a0 con 0x7fe0200af4d0 2023-11-26T07:16:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe01ed9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe02015e080 0x7fe0200b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe01ed9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe02015e080 0x7fe0200b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40600/0 (socket says 172.21.15.83:40600) 2023-11-26T07:16:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe01ed9d700 1 -- 172.21.15.83:0/2501347991 learned_addr learned my addr 172.21.15.83:0/2501347991 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe01e59c700 1 --2- 172.21.15.83:0/2501347991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0200af4d0 0x7fe0200afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe01ed9d700 1 -- 172.21.15.83:0/2501347991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0200af4d0 msgr2=0x7fe0200afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe01ed9d700 1 --2- 172.21.15.83:0/2501347991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0200af4d0 0x7fe0200afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe01ed9d700 1 -- 172.21.15.83:0/2501347991 >> v1:172.21.15.203:6789/0 conn(0x7fe02006c000 legacy=0x7fe020073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe01ed9d700 1 -- 172.21.15.83:0/2501347991 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0200ae710 con 0x7fe02015e080 2023-11-26T07:16:22.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.145+0000 7fe01ed9d700 1 --2- 172.21.15.83:0/2501347991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe02015e080 0x7fe0200b5050 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe014009920 tx=0x7fe014009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeff5679bf13764 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:22.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.146+0000 7fe01dd9b700 1 -- 172.21.15.83:0/2501347991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01400ba20 con 0x7fe02015e080 2023-11-26T07:16:22.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.146+0000 7fe01dd9b700 1 -- 172.21.15.83:0/2501347991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01400bb80 con 0x7fe02015e080 2023-11-26T07:16:22.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.146+0000 7fe01dd9b700 1 -- 172.21.15.83:0/2501347991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014009e70 con 0x7fe02015e080 2023-11-26T07:16:22.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.146+0000 7fe0250af700 1 -- 172.21.15.83:0/2501347991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe02015e080 msgr2=0x7fe0200b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.146+0000 7fe0250af700 1 --2- 172.21.15.83:0/2501347991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe02015e080 0x7fe0200b5050 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe014009920 tx=0x7fe014009c30 comp rx=0 tx=0).stop 2023-11-26T07:16:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.146+0000 7fe0250af700 1 -- 172.21.15.83:0/2501347991 shutdown_connections 2023-11-26T07:16:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.146+0000 7fe0250af700 1 --2- 172.21.15.83:0/2501347991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe02015e080 0x7fe0200b5050 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.146+0000 7fe0250af700 1 --2- 172.21.15.83:0/2501347991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0200af4d0 0x7fe0200afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.146+0000 7fe0250af700 1 -- 172.21.15.83:0/2501347991 >> 172.21.15.83:0/2501347991 conn(0x7fe020071b90 msgr2=0x7fe020071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.146+0000 7fe0250af700 1 -- 172.21.15.83:0/2501347991 shutdown_connections 2023-11-26T07:16:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.146+0000 7fe0250af700 1 -- 172.21.15.83:0/2501347991 wait complete. 2023-11-26T07:16:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.148+0000 7fe0250af700 1 Processor -- start 2023-11-26T07:16:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.148+0000 7fe0250af700 1 -- start start 2023-11-26T07:16:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.148+0000 7fe0250af700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02006c000 0x7fe020149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.148+0000 7fe0250af700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0200af4d0 0x7fe0200c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.148+0000 7fe0250af700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02015e080 0x7fe02015b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.148+0000 7fe0250af700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe020158a60 con 0x7fe0200af4d0 2023-11-26T07:16:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.148+0000 7fe0250af700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe020149a50 con 0x7fe02015e080 2023-11-26T07:16:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.148+0000 7fe0250af700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe020157e50 con 0x7fe02006c000 2023-11-26T07:16:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe01e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0200af4d0 0x7fe0200c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe01e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0200af4d0 0x7fe0200c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40610/0 (socket says 172.21.15.83:40610) 2023-11-26T07:16:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe01e59c700 1 -- 172.21.15.83:0/2667558789 learned_addr learned my addr 172.21.15.83:0/2667558789 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe01ed9d700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02006c000 0x7fe020149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe01f59e700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02015e080 0x7fe02015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe01e59c700 1 -- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02006c000 msgr2=0x7fe020149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe01e59c700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02006c000 0x7fe020149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe01e59c700 1 -- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02015e080 msgr2=0x7fe02015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe01e59c700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02015e080 0x7fe02015b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe01e59c700 1 -- 172.21.15.83:0/2667558789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe014009550 con 0x7fe0200af4d0 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe01e59c700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0200af4d0 0x7fe0200c6620 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe01000e840 tx=0x7fe01000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe0077fe700 1 -- 172.21.15.83:0/2667558789 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010004230 con 0x7fe0200af4d0 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe0077fe700 1 -- 172.21.15.83:0/2667558789 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010004390 con 0x7fe0200af4d0 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe0077fe700 1 -- 172.21.15.83:0/2667558789 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100036a0 con 0x7fe0200af4d0 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe0250af700 1 -- 172.21.15.83:0/2667558789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe020158130 con 0x7fe0200af4d0 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.149+0000 7fe0250af700 1 -- 172.21.15.83:0/2667558789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0201586f0 con 0x7fe0200af4d0 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.150+0000 7fe0250af700 1 -- 172.21.15.83:0/2667558789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0200753f0 con 0x7fe0200af4d0 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.151+0000 7fe0077fe700 1 -- 172.21.15.83:0/2667558789 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fe010003800 con 0x7fe0200af4d0 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.151+0000 7fe0077fe700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe00c06aff0 0x7fe00c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.151+0000 7fe0077fe700 1 -- 172.21.15.83:0/2667558789 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe010089270 con 0x7fe0200af4d0 2023-11-26T07:16:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.151+0000 7fe01ed9d700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe00c06aff0 0x7fe00c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.152+0000 7fe01ed9d700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe00c06aff0 0x7fe00c06d4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe014000c00 tx=0x7fe014008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.155+0000 7fe0077fe700 1 -- 172.21.15.83:0/2667558789 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe010054100 con 0x7fe0200af4d0 2023-11-26T07:16:22.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.345+0000 7fe0250af700 1 -- 172.21.15.83:0/2667558789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe020067f60 con 0x7fe0200af4d0 2023-11-26T07:16:22.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.346+0000 7fe0077fe700 1 -- 172.21.15.83:0/2667558789 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16548 (secure 0 0 0) 0x7fe010057720 con 0x7fe0200af4d0 2023-11-26T07:16:22.348 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:22.348 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":83,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:19.594710+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:16:15.747889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:16:16.943249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 -- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe00c06aff0 msgr2=0x7fe00c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe00c06aff0 0x7fe00c06d4b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe014000c00 tx=0x7fe014008030 comp rx=0 tx=0).stop 2023-11-26T07:16:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 -- 172.21.15.83:0/2667558789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0200af4d0 msgr2=0x7fe0200c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0200af4d0 0x7fe0200c6620 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe01000e840 tx=0x7fe01000eb50 comp rx=0 tx=0).stop 2023-11-26T07:16:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 -- 172.21.15.83:0/2667558789 shutdown_connections 2023-11-26T07:16:22.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe00c06aff0 0x7fe00c06d4b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe02015e080 0x7fe02015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe02006c000 0x7fe020149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 --2- 172.21.15.83:0/2667558789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0200af4d0 0x7fe0200c6620 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 -- 172.21.15.83:0/2667558789 >> 172.21.15.83:0/2667558789 conn(0x7fe020071b90 msgr2=0x7fe02015d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:22.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 -- 172.21.15.83:0/2667558789 shutdown_connections 2023-11-26T07:16:22.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.348+0000 7fe0250af700 1 -- 172.21.15.83:0/2667558789 wait complete. 2023-11-26T07:16:22.363 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:16:22.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:16:22.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.482+0000 7f42440ab700 1 Processor -- start 2023-11-26T07:16:22.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.482+0000 7f42440ab700 1 -- start start 2023-11-26T07:16:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.483+0000 7f42440ab700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f423c15e080 0x7f423c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.483+0000 7f42440ab700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f423c0af4d0 0x7f423c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.483+0000 7f42440ab700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423c0ae430 con 0x7f423c06c000 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.483+0000 7f42440ab700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c0af0b0 con 0x7f423c0af4d0 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.483+0000 7f42440ab700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0b05a0 con 0x7f423c15e080 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.483+0000 7f4242648700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f423c06c000 0x7f423c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60658/0 (socket says 172.21.15.83:60658) 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.483+0000 7f4242648700 1 -- 172.21.15.83:0/52881882 learned_addr learned my addr 172.21.15.83:0/52881882 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4241e47700 1 --2- 172.21.15.83:0/52881882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f423c15e080 0x7f423c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4241646700 1 --2- 172.21.15.83:0/52881882 >> v2:172.21.15.203:3300/0 conn(0x7f423c0af4d0 0x7f423c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4240e45700 1 -- 172.21.15.83:0/52881882 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991716087 0 0) 0x7f423c0ae430 con 0x7f423c06c000 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4240e45700 1 -- 172.21.15.83:0/52881882 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42280035b0 con 0x7f423c06c000 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4240e45700 1 -- 172.21.15.83:0/52881882 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1251843745 0 0) 0x7f42280035b0 con 0x7f423c06c000 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4241e47700 1 -- 172.21.15.83:0/52881882 >> v2:172.21.15.203:3300/0 conn(0x7f423c0af4d0 msgr2=0x7f423c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4241e47700 1 --2- 172.21.15.83:0/52881882 >> v2:172.21.15.203:3300/0 conn(0x7f423c0af4d0 0x7f423c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4241e47700 1 -- 172.21.15.83:0/52881882 >> v1:172.21.15.83:6789/0 conn(0x7f423c06c000 legacy=0x7f423c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4241e47700 1 -- 172.21.15.83:0/52881882 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c0ae710 con 0x7f423c15e080 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4241646700 1 --2- 172.21.15.83:0/52881882 >> v2:172.21.15.203:3300/0 conn(0x7f423c0af4d0 0x7f423c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4241e47700 1 --2- 172.21.15.83:0/52881882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f423c15e080 0x7f423c0b5050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f422c009840 tx=0x7f422c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b22fe5ab0461a865 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4240e45700 1 -- 172.21.15.83:0/52881882 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42340019f0 con 0x7f423c15e080 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4240e45700 1 -- 172.21.15.83:0/52881882 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422c004030 con 0x7f423c15e080 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.484+0000 7f4240e45700 1 -- 172.21.15.83:0/52881882 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c0043a0 con 0x7f423c15e080 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.485+0000 7f42440ab700 1 -- 172.21.15.83:0/52881882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f423c15e080 msgr2=0x7f423c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.485+0000 7f42440ab700 1 --2- 172.21.15.83:0/52881882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f423c15e080 0x7f423c0b5050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f422c009840 tx=0x7f422c009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.485+0000 7f42440ab700 1 -- 172.21.15.83:0/52881882 shutdown_connections 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.485+0000 7f42440ab700 1 --2- 172.21.15.83:0/52881882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f423c15e080 0x7f423c0b5050 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.485+0000 7f42440ab700 1 --2- 172.21.15.83:0/52881882 >> v2:172.21.15.203:3300/0 conn(0x7f423c0af4d0 0x7f423c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.485+0000 7f42440ab700 1 -- 172.21.15.83:0/52881882 >> 172.21.15.83:0/52881882 conn(0x7f423c071b90 msgr2=0x7f423c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.485+0000 7f42440ab700 1 -- 172.21.15.83:0/52881882 shutdown_connections 2023-11-26T07:16:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.485+0000 7f42440ab700 1 -- 172.21.15.83:0/52881882 wait complete. 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.486+0000 7f42440ab700 1 Processor -- start 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.486+0000 7f42440ab700 1 -- start start 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f42440ab700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f423c06c000 0x7f423c15e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f42440ab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0af4d0 0x7f423c155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f42440ab700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f423c15e080 0x7f423c156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f42440ab700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c15e9a0 con 0x7f423c15e080 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f42440ab700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0ce440 con 0x7f423c0af4d0 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f42440ab700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0cf4a0 con 0x7f423c06c000 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f4242648700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f423c15e080 0x7f423c156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f4242648700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f423c15e080 0x7f423c156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40622/0 (socket says 172.21.15.83:40622) 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f4242648700 1 -- 172.21.15.83:0/3669755614 learned_addr learned my addr 172.21.15.83:0/3669755614 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f4241e47700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f423c06c000 0x7f423c15e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f4241646700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0af4d0 0x7f423c155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f4242648700 1 -- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f423c06c000 msgr2=0x7f423c15e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f4242648700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f423c06c000 0x7f423c15e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f4242648700 1 -- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0af4d0 msgr2=0x7f423c155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f4242648700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0af4d0 0x7f423c155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f4242648700 1 -- 172.21.15.83:0/3669755614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42380094a0 con 0x7f423c15e080 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.487+0000 7f4241646700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0af4d0 0x7f423c155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.488+0000 7f4242648700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f423c15e080 0x7f423c156450 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4234001ba0 tx=0x7f423400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.488+0000 7f42327fc700 1 -- 172.21.15.83:0/3669755614 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423400ab70 con 0x7f423c15e080 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.488+0000 7f42327fc700 1 -- 172.21.15.83:0/3669755614 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423400acd0 con 0x7f423c15e080 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.488+0000 7f42440ab700 1 -- 172.21.15.83:0/3669755614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c0094a0 con 0x7f423c15e080 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.488+0000 7f42327fc700 1 -- 172.21.15.83:0/3669755614 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234007310 con 0x7f423c15e080 2023-11-26T07:16:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.488+0000 7f42440ab700 1 -- 172.21.15.83:0/3669755614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c0ce9c0 con 0x7f423c15e080 2023-11-26T07:16:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.489+0000 7f42440ab700 1 -- 172.21.15.83:0/3669755614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c0753f0 con 0x7f423c15e080 2023-11-26T07:16:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.489+0000 7f42327fc700 1 -- 172.21.15.83:0/3669755614 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f4234007470 con 0x7f423c15e080 2023-11-26T07:16:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.489+0000 7f42327fc700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f422006aff0 0x7f422006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.489+0000 7f42327fc700 1 -- 172.21.15.83:0/3669755614 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42340895b0 con 0x7f423c15e080 2023-11-26T07:16:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.490+0000 7f4241e47700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f422006aff0 0x7f422006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.490+0000 7f4241e47700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f422006aff0 0x7f422006d4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f422c0090b0 tx=0x7f422c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.494+0000 7f42327fc700 1 -- 172.21.15.83:0/3669755614 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42340545e0 con 0x7f423c15e080 2023-11-26T07:16:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.681+0000 7f42440ab700 1 -- 172.21.15.83:0/3669755614 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f423c0aecd0 con 0x7f422006aff0 2023-11-26T07:16:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.691+0000 7f42327fc700 1 -- 172.21.15.83:0/3669755614 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289415 (secure 0 0 0) 0x7f422c0042e0 con 0x7f422006aff0 2023-11-26T07:16:22.696 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 -- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f422006aff0 msgr2=0x7f422006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f422006aff0 0x7f422006d4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f422c0090b0 tx=0x7f422c008030 comp rx=0 tx=0).stop 2023-11-26T07:16:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 -- 172.21.15.83:0/3669755614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f423c15e080 msgr2=0x7f423c156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f423c15e080 0x7f423c156450 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4234001ba0 tx=0x7f423400cf50 comp rx=0 tx=0).stop 2023-11-26T07:16:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 -- 172.21.15.83:0/3669755614 shutdown_connections 2023-11-26T07:16:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f422006aff0 0x7f422006d4b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f423c15e080 0x7f423c156450 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f423c06c000 0x7f423c15e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 --2- 172.21.15.83:0/3669755614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f423c0af4d0 0x7f423c155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 -- 172.21.15.83:0/3669755614 >> 172.21.15.83:0/3669755614 conn(0x7f423c071b90 msgr2=0x7f423c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 -- 172.21.15.83:0/3669755614 shutdown_connections 2023-11-26T07:16:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.694+0000 7f42440ab700 1 -- 172.21.15.83:0/3669755614 wait complete. 2023-11-26T07:16:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:16:22.707 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-26T07:16:22.310325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225352,"kb_used_data":9192,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749368,"statfs":{"total":773094113280,"available":772863352832,"internally_reserved":0,"allocated":9412608,"data_stored":4691396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.742268"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171656+0000","last_change":"2023-11-26T07:16:18.603161+0000","last_active":"2023-11-26T07:16:20.171656+0000","last_peered":"2023-11-26T07:16:20.171656+0000","last_clean":"2023-11-26T07:16:20.171656+0000","last_became_active":"2023-11-26T07:16:18.602887+0000","last_became_peered":"2023-11-26T07:16:18.602887+0000","last_unstale":"2023-11-26T07:16:20.171656+0000","last_undegraded":"2023-11-26T07:16:20.171656+0000","last_fullsized":"2023-11-26T07:16:20.171656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:36:54.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171980+0000","last_change":"2023-11-26T07:16:18.602020+0000","last_active":"2023-11-26T07:16:20.171980+0000","last_peered":"2023-11-26T07:16:20.171980+0000","last_clean":"2023-11-26T07:16:20.171980+0000","last_became_active":"2023-11-26T07:16:18.601741+0000","last_became_peered":"2023-11-26T07:16:18.601741+0000","last_unstale":"2023-11-26T07:16:20.171980+0000","last_undegraded":"2023-11-26T07:16:20.171980+0000","last_fullsized":"2023-11-26T07:16:20.171980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:44:32.098990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607570+0000","last_change":"2023-11-26T07:16:18.597182+0000","last_active":"2023-11-26T07:16:19.607570+0000","last_peered":"2023-11-26T07:16:19.607570+0000","last_clean":"2023-11-26T07:16:19.607570+0000","last_became_active":"2023-11-26T07:16:18.596897+0000","last_became_peered":"2023-11-26T07:16:18.596897+0000","last_unstale":"2023-11-26T07:16:19.607570+0000","last_undegraded":"2023-11-26T07:16:19.607570+0000","last_fullsized":"2023-11-26T07:16:19.607570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:35.572964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171798+0000","last_change":"2023-11-26T07:16:18.604093+0000","last_active":"2023-11-26T07:16:20.171798+0000","last_peered":"2023-11-26T07:16:20.171798+0000","last_clean":"2023-11-26T07:16:20.171798+0000","last_became_active":"2023-11-26T07:16:18.603811+0000","last_became_peered":"2023-11-26T07:16:18.603811+0000","last_unstale":"2023-11-26T07:16:20.171798+0000","last_undegraded":"2023-11-26T07:16:20.171798+0000","last_fullsized":"2023-11-26T07:16:20.171798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:09:50.095468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608656+0000","last_change":"2023-11-26T07:16:18.612495+0000","last_active":"2023-11-26T07:16:19.608656+0000","last_peered":"2023-11-26T07:16:19.608656+0000","last_clean":"2023-11-26T07:16:19.608656+0000","last_became_active":"2023-11-26T07:16:18.612101+0000","last_became_peered":"2023-11-26T07:16:18.612101+0000","last_unstale":"2023-11-26T07:16:19.608656+0000","last_undegraded":"2023-11-26T07:16:19.608656+0000","last_fullsized":"2023-11-26T07:16:19.608656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:48.875519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170737+0000","last_change":"2023-11-26T07:16:18.599860+0000","last_active":"2023-11-26T07:16:20.170737+0000","last_peered":"2023-11-26T07:16:20.170737+0000","last_clean":"2023-11-26T07:16:20.170737+0000","last_became_active":"2023-11-26T07:16:18.599603+0000","last_became_peered":"2023-11-26T07:16:18.599603+0000","last_unstale":"2023-11-26T07:16:20.170737+0000","last_undegraded":"2023-11-26T07:16:20.170737+0000","last_fullsized":"2023-11-26T07:16:20.170737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:45.731926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170878+0000","last_change":"2023-11-26T07:16:18.600023+0000","last_active":"2023-11-26T07:16:20.170878+0000","last_peered":"2023-11-26T07:16:20.170878+0000","last_clean":"2023-11-26T07:16:20.170878+0000","last_became_active":"2023-11-26T07:16:18.599474+0000","last_became_peered":"2023-11-26T07:16:18.599474+0000","last_unstale":"2023-11-26T07:16:20.170878+0000","last_undegraded":"2023-11-26T07:16:20.170878+0000","last_fullsized":"2023-11-26T07:16:20.170878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:17:01.015611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170742+0000","last_change":"2023-11-26T07:16:18.597258+0000","last_active":"2023-11-26T07:16:20.170742+0000","last_peered":"2023-11-26T07:16:20.170742+0000","last_clean":"2023-11-26T07:16:20.170742+0000","last_became_active":"2023-11-26T07:16:18.596991+0000","last_became_peered":"2023-11-26T07:16:18.596991+0000","last_unstale":"2023-11-26T07:16:20.170742+0000","last_undegraded":"2023-11-26T07:16:20.170742+0000","last_fullsized":"2023-11-26T07:16:20.170742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:48.875519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171146+0000","last_change":"2023-11-26T07:16:18.612700+0000","last_active":"2023-11-26T07:16:20.171146+0000","last_peered":"2023-11-26T07:16:20.171146+0000","last_clean":"2023-11-26T07:16:20.171146+0000","last_became_active":"2023-11-26T07:16:18.612394+0000","last_became_peered":"2023-11-26T07:16:18.612394+0000","last_unstale":"2023-11-26T07:16:20.171146+0000","last_undegraded":"2023-11-26T07:16:20.171146+0000","last_fullsized":"2023-11-26T07:16:20.171146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:20.475765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171265+0000","last_change":"2023-11-26T07:16:18.600807+0000","last_active":"2023-11-26T07:16:20.171265+0000","last_peered":"2023-11-26T07:16:20.171265+0000","last_clean":"2023-11-26T07:16:20.171265+0000","last_became_active":"2023-11-26T07:16:18.600493+0000","last_became_peered":"2023-11-26T07:16:18.600493+0000","last_unstale":"2023-11-26T07:16:20.171265+0000","last_undegraded":"2023-11-26T07:16:20.171265+0000","last_fullsized":"2023-11-26T07:16:20.171265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:28:14.679869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170935+0000","last_change":"2023-11-26T07:16:18.596861+0000","last_active":"2023-11-26T07:16:20.170935+0000","last_peered":"2023-11-26T07:16:20.170935+0000","last_clean":"2023-11-26T07:16:20.170935+0000","last_became_active":"2023-11-26T07:16:18.596555+0000","last_became_peered":"2023-11-26T07:16:18.596555+0000","last_unstale":"2023-11-26T07:16:20.170935+0000","last_undegraded":"2023-11-26T07:16:20.170935+0000","last_fullsized":"2023-11-26T07:16:20.170935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:24:57.226424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170639+0000","last_change":"2023-11-26T07:16:18.604520+0000","last_active":"2023-11-26T07:16:20.170639+0000","last_peered":"2023-11-26T07:16:20.170639+0000","last_clean":"2023-11-26T07:16:20.170639+0000","last_became_active":"2023-11-26T07:16:18.604225+0000","last_became_peered":"2023-11-26T07:16:18.604225+0000","last_unstale":"2023-11-26T07:16:20.170639+0000","last_undegraded":"2023-11-26T07:16:20.170639+0000","last_fullsized":"2023-11-26T07:16:20.170639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:50.031595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607538+0000","last_change":"2023-11-26T07:16:18.602610+0000","last_active":"2023-11-26T07:16:19.607538+0000","last_peered":"2023-11-26T07:16:19.607538+0000","last_clean":"2023-11-26T07:16:19.607538+0000","last_became_active":"2023-11-26T07:16:18.602252+0000","last_became_peered":"2023-11-26T07:16:18.602252+0000","last_unstale":"2023-11-26T07:16:19.607538+0000","last_undegraded":"2023-11-26T07:16:19.607538+0000","last_fullsized":"2023-11-26T07:16:19.607538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:23.025046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170998+0000","last_change":"2023-11-26T07:16:18.611064+0000","last_active":"2023-11-26T07:16:20.170998+0000","last_peered":"2023-11-26T07:16:20.170998+0000","last_clean":"2023-11-26T07:16:20.170998+0000","last_became_active":"2023-11-26T07:16:18.610771+0000","last_became_peered":"2023-11-26T07:16:18.610771+0000","last_unstale":"2023-11-26T07:16:20.170998+0000","last_undegraded":"2023-11-26T07:16:20.170998+0000","last_fullsized":"2023-11-26T07:16:20.170998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:48:59.976510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606297+0000","last_change":"2023-11-26T07:16:18.601102+0000","last_active":"2023-11-26T07:16:19.606297+0000","last_peered":"2023-11-26T07:16:19.606297+0000","last_clean":"2023-11-26T07:16:19.606297+0000","last_became_active":"2023-11-26T07:16:18.600698+0000","last_became_peered":"2023-11-26T07:16:18.600698+0000","last_unstale":"2023-11-26T07:16:19.606297+0000","last_undegraded":"2023-11-26T07:16:19.606297+0000","last_fullsized":"2023-11-26T07:16:19.606297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:12:12.749341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608284+0000","last_change":"2023-11-26T07:16:18.604279+0000","last_active":"2023-11-26T07:16:19.608284+0000","last_peered":"2023-11-26T07:16:19.608284+0000","last_clean":"2023-11-26T07:16:19.608284+0000","last_became_active":"2023-11-26T07:16:18.604022+0000","last_became_peered":"2023-11-26T07:16:18.604022+0000","last_unstale":"2023-11-26T07:16:19.608284+0000","last_undegraded":"2023-11-26T07:16:19.608284+0000","last_fullsized":"2023-11-26T07:16:19.608284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:35:12.961085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.609098+0000","last_change":"2023-11-26T07:16:18.607619+0000","last_active":"2023-11-26T07:16:19.609098+0000","last_peered":"2023-11-26T07:16:19.609098+0000","last_clean":"2023-11-26T07:16:19.609098+0000","last_became_active":"2023-11-26T07:16:18.607322+0000","last_became_peered":"2023-11-26T07:16:18.607322+0000","last_unstale":"2023-11-26T07:16:19.609098+0000","last_undegraded":"2023-11-26T07:16:19.609098+0000","last_fullsized":"2023-11-26T07:16:19.609098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:59:20.892686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171115+0000","last_change":"2023-11-26T07:16:18.602760+0000","last_active":"2023-11-26T07:16:20.171115+0000","last_peered":"2023-11-26T07:16:20.171115+0000","last_clean":"2023-11-26T07:16:20.171115+0000","last_became_active":"2023-11-26T07:16:18.602105+0000","last_became_peered":"2023-11-26T07:16:18.602105+0000","last_unstale":"2023-11-26T07:16:20.171115+0000","last_undegraded":"2023-11-26T07:16:20.171115+0000","last_fullsized":"2023-11-26T07:16:20.171115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:23.840113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171550+0000","last_change":"2023-11-26T07:16:18.604756+0000","last_active":"2023-11-26T07:16:20.171550+0000","last_peered":"2023-11-26T07:16:20.171550+0000","last_clean":"2023-11-26T07:16:20.171550+0000","last_became_active":"2023-11-26T07:16:18.604507+0000","last_became_peered":"2023-11-26T07:16:18.604507+0000","last_unstale":"2023-11-26T07:16:20.171550+0000","last_undegraded":"2023-11-26T07:16:20.171550+0000","last_fullsized":"2023-11-26T07:16:20.171550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:50.031595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171900+0000","last_change":"2023-11-26T07:16:18.607794+0000","last_active":"2023-11-26T07:16:20.171900+0000","last_peered":"2023-11-26T07:16:20.171900+0000","last_clean":"2023-11-26T07:16:20.171900+0000","last_became_active":"2023-11-26T07:16:18.607450+0000","last_became_peered":"2023-11-26T07:16:18.607450+0000","last_unstale":"2023-11-26T07:16:20.171900+0000","last_undegraded":"2023-11-26T07:16:20.171900+0000","last_fullsized":"2023-11-26T07:16:20.171900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:03:53.481441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171444+0000","last_change":"2023-11-26T07:16:18.611677+0000","last_active":"2023-11-26T07:16:20.171444+0000","last_peered":"2023-11-26T07:16:20.171444+0000","last_clean":"2023-11-26T07:16:20.171444+0000","last_became_active":"2023-11-26T07:16:18.611429+0000","last_became_peered":"2023-11-26T07:16:18.611429+0000","last_unstale":"2023-11-26T07:16:20.171444+0000","last_undegraded":"2023-11-26T07:16:20.171444+0000","last_fullsized":"2023-11-26T07:16:20.171444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:35:55.637783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170763+0000","last_change":"2023-11-26T07:16:18.607278+0000","last_active":"2023-11-26T07:16:20.170763+0000","last_peered":"2023-11-26T07:16:20.170763+0000","last_clean":"2023-11-26T07:16:20.170763+0000","last_became_active":"2023-11-26T07:16:18.606775+0000","last_became_peered":"2023-11-26T07:16:18.606775+0000","last_unstale":"2023-11-26T07:16:20.170763+0000","last_undegraded":"2023-11-26T07:16:20.170763+0000","last_fullsized":"2023-11-26T07:16:20.170763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:43:17.983907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170394+0000","last_change":"2023-11-26T07:16:18.607945+0000","last_active":"2023-11-26T07:16:20.170394+0000","last_peered":"2023-11-26T07:16:20.170394+0000","last_clean":"2023-11-26T07:16:20.170394+0000","last_became_active":"2023-11-26T07:16:18.607690+0000","last_became_peered":"2023-11-26T07:16:18.607690+0000","last_unstale":"2023-11-26T07:16:20.170394+0000","last_undegraded":"2023-11-26T07:16:20.170394+0000","last_fullsized":"2023-11-26T07:16:20.170394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:45.731926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171541+0000","last_change":"2023-11-26T07:16:18.607087+0000","last_active":"2023-11-26T07:16:20.171541+0000","last_peered":"2023-11-26T07:16:20.171541+0000","last_clean":"2023-11-26T07:16:20.171541+0000","last_became_active":"2023-11-26T07:16:18.606747+0000","last_became_peered":"2023-11-26T07:16:18.606747+0000","last_unstale":"2023-11-26T07:16:20.171541+0000","last_undegraded":"2023-11-26T07:16:20.171541+0000","last_fullsized":"2023-11-26T07:16:20.171541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:41:32.975198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170738+0000","last_change":"2023-11-26T07:16:18.606444+0000","last_active":"2023-11-26T07:16:20.170738+0000","last_peered":"2023-11-26T07:16:20.170738+0000","last_clean":"2023-11-26T07:16:20.170738+0000","last_became_active":"2023-11-26T07:16:18.606140+0000","last_became_peered":"2023-11-26T07:16:18.606140+0000","last_unstale":"2023-11-26T07:16:20.170738+0000","last_undegraded":"2023-11-26T07:16:20.170738+0000","last_fullsized":"2023-11-26T07:16:20.170738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:38:24.528206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607470+0000","last_change":"2023-11-26T07:16:18.600865+0000","last_active":"2023-11-26T07:16:19.607470+0000","last_peered":"2023-11-26T07:16:19.607470+0000","last_clean":"2023-11-26T07:16:19.607470+0000","last_became_active":"2023-11-26T07:16:18.600618+0000","last_became_peered":"2023-11-26T07:16:18.600618+0000","last_unstale":"2023-11-26T07:16:19.607470+0000","last_undegraded":"2023-11-26T07:16:19.607470+0000","last_fullsized":"2023-11-26T07:16:19.607470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:06:56.993863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608629+0000","last_change":"2023-11-26T07:16:18.605000+0000","last_active":"2023-11-26T07:16:19.608629+0000","last_peered":"2023-11-26T07:16:19.608629+0000","last_clean":"2023-11-26T07:16:19.608629+0000","last_became_active":"2023-11-26T07:16:18.604733+0000","last_became_peered":"2023-11-26T07:16:18.604733+0000","last_unstale":"2023-11-26T07:16:19.608629+0000","last_undegraded":"2023-11-26T07:16:19.608629+0000","last_fullsized":"2023-11-26T07:16:19.608629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:45.731926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170435+0000","last_change":"2023-11-26T07:16:18.612041+0000","last_active":"2023-11-26T07:16:20.170435+0000","last_peered":"2023-11-26T07:16:20.170435+0000","last_clean":"2023-11-26T07:16:20.170435+0000","last_became_active":"2023-11-26T07:16:18.611757+0000","last_became_peered":"2023-11-26T07:16:18.611757+0000","last_unstale":"2023-11-26T07:16:20.170435+0000","last_undegraded":"2023-11-26T07:16:20.170435+0000","last_fullsized":"2023-11-26T07:16:20.170435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:48.875519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170861+0000","last_change":"2023-11-26T07:16:18.597641+0000","last_active":"2023-11-26T07:16:20.170861+0000","last_peered":"2023-11-26T07:16:20.170861+0000","last_clean":"2023-11-26T07:16:20.170861+0000","last_became_active":"2023-11-26T07:16:18.597280+0000","last_became_peered":"2023-11-26T07:16:18.597280+0000","last_unstale":"2023-11-26T07:16:20.170861+0000","last_undegraded":"2023-11-26T07:16:20.170861+0000","last_fullsized":"2023-11-26T07:16:20.170861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:58:15.236533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171766+0000","last_change":"2023-11-26T07:16:18.607413+0000","last_active":"2023-11-26T07:16:20.171766+0000","last_peered":"2023-11-26T07:16:20.171766+0000","last_clean":"2023-11-26T07:16:20.171766+0000","last_became_active":"2023-11-26T07:16:18.607080+0000","last_became_peered":"2023-11-26T07:16:18.607080+0000","last_unstale":"2023-11-26T07:16:20.171766+0000","last_undegraded":"2023-11-26T07:16:20.171766+0000","last_fullsized":"2023-11-26T07:16:20.171766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:31:16.121796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608440+0000","last_change":"2023-11-26T07:16:18.610537+0000","last_active":"2023-11-26T07:16:19.608440+0000","last_peered":"2023-11-26T07:16:19.608440+0000","last_clean":"2023-11-26T07:16:19.608440+0000","last_became_active":"2023-11-26T07:16:18.610137+0000","last_became_peered":"2023-11-26T07:16:18.610137+0000","last_unstale":"2023-11-26T07:16:19.608440+0000","last_undegraded":"2023-11-26T07:16:19.608440+0000","last_fullsized":"2023-11-26T07:16:19.608440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:40.918037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170336+0000","last_change":"2023-11-26T07:16:18.601464+0000","last_active":"2023-11-26T07:16:20.170336+0000","last_peered":"2023-11-26T07:16:20.170336+0000","last_clean":"2023-11-26T07:16:20.170336+0000","last_became_active":"2023-11-26T07:16:18.600122+0000","last_became_peered":"2023-11-26T07:16:18.600122+0000","last_unstale":"2023-11-26T07:16:20.170336+0000","last_undegraded":"2023-11-26T07:16:20.170336+0000","last_fullsized":"2023-11-26T07:16:20.170336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:40.918037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.609056+0000","last_change":"2023-11-26T07:16:18.604227+0000","last_active":"2023-11-26T07:16:19.609056+0000","last_peered":"2023-11-26T07:16:19.609056+0000","last_clean":"2023-11-26T07:16:19.609056+0000","last_became_active":"2023-11-26T07:16:18.603919+0000","last_became_peered":"2023-11-26T07:16:18.603919+0000","last_unstale":"2023-11-26T07:16:19.609056+0000","last_undegraded":"2023-11-26T07:16:19.609056+0000","last_fullsized":"2023-11-26T07:16:19.609056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:50.031595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171442+0000","last_change":"2023-11-26T07:16:17.593158+0000","last_active":"2023-11-26T07:16:20.171442+0000","last_peered":"2023-11-26T07:16:20.171442+0000","last_clean":"2023-11-26T07:16:20.171442+0000","last_became_active":"2023-11-26T07:16:17.592853+0000","last_became_peered":"2023-11-26T07:16:17.592853+0000","last_unstale":"2023-11-26T07:16:20.171442+0000","last_undegraded":"2023-11-26T07:16:20.171442+0000","last_fullsized":"2023-11-26T07:16:20.171442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:49:12.199077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171611+0000","last_change":"2023-11-26T07:16:18.607574+0000","last_active":"2023-11-26T07:16:20.171611+0000","last_peered":"2023-11-26T07:16:20.171611+0000","last_clean":"2023-11-26T07:16:20.171611+0000","last_became_active":"2023-11-26T07:16:18.606978+0000","last_became_peered":"2023-11-26T07:16:18.606978+0000","last_unstale":"2023-11-26T07:16:20.171611+0000","last_undegraded":"2023-11-26T07:16:20.171611+0000","last_fullsized":"2023-11-26T07:16:20.171611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:04:47.701141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.609150+0000","last_change":"2023-11-26T07:16:17.586841+0000","last_active":"2023-11-26T07:16:19.609150+0000","last_peered":"2023-11-26T07:16:19.609150+0000","last_clean":"2023-11-26T07:16:19.609150+0000","last_became_active":"2023-11-26T07:16:17.586579+0000","last_became_peered":"2023-11-26T07:16:17.586579+0000","last_unstale":"2023-11-26T07:16:19.609150+0000","last_undegraded":"2023-11-26T07:16:19.609150+0000","last_fullsized":"2023-11-26T07:16:19.609150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:41.150316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170479+0000","last_change":"2023-11-26T07:16:18.612092+0000","last_active":"2023-11-26T07:16:20.170479+0000","last_peered":"2023-11-26T07:16:20.170479+0000","last_clean":"2023-11-26T07:16:20.170479+0000","last_became_active":"2023-11-26T07:16:18.611857+0000","last_became_peered":"2023-11-26T07:16:18.611857+0000","last_unstale":"2023-11-26T07:16:20.170479+0000","last_undegraded":"2023-11-26T07:16:20.170479+0000","last_fullsized":"2023-11-26T07:16:20.170479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:41:44.521573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171749+0000","last_change":"2023-11-26T07:16:17.581813+0000","last_active":"2023-11-26T07:16:20.171749+0000","last_peered":"2023-11-26T07:16:20.171749+0000","last_clean":"2023-11-26T07:16:20.171749+0000","last_became_active":"2023-11-26T07:16:17.581530+0000","last_became_peered":"2023-11-26T07:16:17.581530+0000","last_unstale":"2023-11-26T07:16:20.171749+0000","last_undegraded":"2023-11-26T07:16:20.171749+0000","last_fullsized":"2023-11-26T07:16:20.171749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:49:12.199077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607721+0000","last_change":"2023-11-26T07:16:18.600345+0000","last_active":"2023-11-26T07:16:19.607721+0000","last_peered":"2023-11-26T07:16:19.607721+0000","last_clean":"2023-11-26T07:16:19.607721+0000","last_became_active":"2023-11-26T07:16:18.600089+0000","last_became_peered":"2023-11-26T07:16:18.600089+0000","last_unstale":"2023-11-26T07:16:19.607721+0000","last_undegraded":"2023-11-26T07:16:19.607721+0000","last_fullsized":"2023-11-26T07:16:19.607721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:46.678079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171672+0000","last_change":"2023-11-26T07:16:17.585021+0000","last_active":"2023-11-26T07:16:20.171672+0000","last_peered":"2023-11-26T07:16:20.171672+0000","last_clean":"2023-11-26T07:16:20.171672+0000","last_became_active":"2023-11-26T07:16:17.584722+0000","last_became_peered":"2023-11-26T07:16:17.584722+0000","last_unstale":"2023-11-26T07:16:20.171672+0000","last_undegraded":"2023-11-26T07:16:20.171672+0000","last_fullsized":"2023-11-26T07:16:20.171672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:47.871151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171668+0000","last_change":"2023-11-26T07:16:18.608161+0000","last_active":"2023-11-26T07:16:20.171668+0000","last_peered":"2023-11-26T07:16:20.171668+0000","last_clean":"2023-11-26T07:16:20.171668+0000","last_became_active":"2023-11-26T07:16:18.606300+0000","last_became_peered":"2023-11-26T07:16:18.606300+0000","last_unstale":"2023-11-26T07:16:20.171668+0000","last_undegraded":"2023-11-26T07:16:20.171668+0000","last_fullsized":"2023-11-26T07:16:20.171668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:23:56.971286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608761+0000","last_change":"2023-11-26T07:16:17.584391+0000","last_active":"2023-11-26T07:16:19.608761+0000","last_peered":"2023-11-26T07:16:19.608761+0000","last_clean":"2023-11-26T07:16:19.608761+0000","last_became_active":"2023-11-26T07:16:17.584013+0000","last_became_peered":"2023-11-26T07:16:17.584013+0000","last_unstale":"2023-11-26T07:16:19.608761+0000","last_undegraded":"2023-11-26T07:16:19.608761+0000","last_fullsized":"2023-11-26T07:16:19.608761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:49:12.199077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608740+0000","last_change":"2023-11-26T07:16:18.608329+0000","last_active":"2023-11-26T07:16:19.608740+0000","last_peered":"2023-11-26T07:16:19.608740+0000","last_clean":"2023-11-26T07:16:19.608740+0000","last_became_active":"2023-11-26T07:16:18.608055+0000","last_became_peered":"2023-11-26T07:16:18.608055+0000","last_unstale":"2023-11-26T07:16:19.608740+0000","last_undegraded":"2023-11-26T07:16:19.608740+0000","last_fullsized":"2023-11-26T07:16:19.608740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:41:44.521573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608423+0000","last_change":"2023-11-26T07:16:17.589142+0000","last_active":"2023-11-26T07:16:19.608423+0000","last_peered":"2023-11-26T07:16:19.608423+0000","last_clean":"2023-11-26T07:16:19.608423+0000","last_became_active":"2023-11-26T07:16:17.588821+0000","last_became_peered":"2023-11-26T07:16:17.588821+0000","last_unstale":"2023-11-26T07:16:19.608423+0000","last_undegraded":"2023-11-26T07:16:19.608423+0000","last_fullsized":"2023-11-26T07:16:19.608423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:34.128300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607241+0000","last_change":"2023-11-26T07:16:18.606595+0000","last_active":"2023-11-26T07:16:19.607241+0000","last_peered":"2023-11-26T07:16:19.607241+0000","last_clean":"2023-11-26T07:16:19.607241+0000","last_became_active":"2023-11-26T07:16:18.606153+0000","last_became_peered":"2023-11-26T07:16:18.606153+0000","last_unstale":"2023-11-26T07:16:19.607241+0000","last_undegraded":"2023-11-26T07:16:19.607241+0000","last_fullsized":"2023-11-26T07:16:19.607241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:12:42.626819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171672+0000","last_change":"2023-11-26T07:16:17.585406+0000","last_active":"2023-11-26T07:16:20.171672+0000","last_peered":"2023-11-26T07:16:20.171672+0000","last_clean":"2023-11-26T07:16:20.171672+0000","last_became_active":"2023-11-26T07:16:17.585147+0000","last_became_peered":"2023-11-26T07:16:17.585147+0000","last_unstale":"2023-11-26T07:16:20.171672+0000","last_undegraded":"2023-11-26T07:16:20.171672+0000","last_fullsized":"2023-11-26T07:16:20.171672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:56:50.275379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606380+0000","last_change":"2023-11-26T07:16:18.600937+0000","last_active":"2023-11-26T07:16:19.606380+0000","last_peered":"2023-11-26T07:16:19.606380+0000","last_clean":"2023-11-26T07:16:19.606380+0000","last_became_active":"2023-11-26T07:16:18.600372+0000","last_became_peered":"2023-11-26T07:16:18.600372+0000","last_unstale":"2023-11-26T07:16:19.606380+0000","last_undegraded":"2023-11-26T07:16:19.606380+0000","last_fullsized":"2023-11-26T07:16:19.606380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:53.869386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171725+0000","last_change":"2023-11-26T07:16:17.578953+0000","last_active":"2023-11-26T07:16:20.171725+0000","last_peered":"2023-11-26T07:16:20.171725+0000","last_clean":"2023-11-26T07:16:20.171725+0000","last_became_active":"2023-11-26T07:16:17.578650+0000","last_became_peered":"2023-11-26T07:16:17.578650+0000","last_unstale":"2023-11-26T07:16:20.171725+0000","last_undegraded":"2023-11-26T07:16:20.171725+0000","last_fullsized":"2023-11-26T07:16:20.171725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:09:49.091099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606546+0000","last_change":"2023-11-26T07:16:18.597738+0000","last_active":"2023-11-26T07:16:19.606546+0000","last_peered":"2023-11-26T07:16:19.606546+0000","last_clean":"2023-11-26T07:16:19.606546+0000","last_became_active":"2023-11-26T07:16:18.597440+0000","last_became_peered":"2023-11-26T07:16:18.597440+0000","last_unstale":"2023-11-26T07:16:19.606546+0000","last_undegraded":"2023-11-26T07:16:19.606546+0000","last_fullsized":"2023-11-26T07:16:19.606546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:46.678079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171443+0000","last_change":"2023-11-26T07:16:17.593644+0000","last_active":"2023-11-26T07:16:20.171443+0000","last_peered":"2023-11-26T07:16:20.171443+0000","last_clean":"2023-11-26T07:16:20.171443+0000","last_became_active":"2023-11-26T07:16:17.593333+0000","last_became_peered":"2023-11-26T07:16:17.593333+0000","last_unstale":"2023-11-26T07:16:20.171443+0000","last_undegraded":"2023-11-26T07:16:20.171443+0000","last_fullsized":"2023-11-26T07:16:20.171443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:41.150316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171177+0000","last_change":"2023-11-26T07:16:18.599341+0000","last_active":"2023-11-26T07:16:20.171177+0000","last_peered":"2023-11-26T07:16:20.171177+0000","last_clean":"2023-11-26T07:16:20.171177+0000","last_became_active":"2023-11-26T07:16:18.599052+0000","last_became_peered":"2023-11-26T07:16:18.599052+0000","last_unstale":"2023-11-26T07:16:20.171177+0000","last_undegraded":"2023-11-26T07:16:20.171177+0000","last_fullsized":"2023-11-26T07:16:20.171177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:50.031595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171337+0000","last_change":"2023-11-26T07:16:17.585949+0000","last_active":"2023-11-26T07:16:20.171337+0000","last_peered":"2023-11-26T07:16:20.171337+0000","last_clean":"2023-11-26T07:16:20.171337+0000","last_became_active":"2023-11-26T07:16:17.585688+0000","last_became_peered":"2023-11-26T07:16:17.585688+0000","last_unstale":"2023-11-26T07:16:20.171337+0000","last_undegraded":"2023-11-26T07:16:20.171337+0000","last_fullsized":"2023-11-26T07:16:20.171337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:42:47.406959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.172089+0000","last_change":"2023-11-26T07:16:18.605464+0000","last_active":"2023-11-26T07:16:20.172089+0000","last_peered":"2023-11-26T07:16:20.172089+0000","last_clean":"2023-11-26T07:16:20.172089+0000","last_became_active":"2023-11-26T07:16:18.605180+0000","last_became_peered":"2023-11-26T07:16:18.605180+0000","last_unstale":"2023-11-26T07:16:20.172089+0000","last_undegraded":"2023-11-26T07:16:20.172089+0000","last_fullsized":"2023-11-26T07:16:20.172089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:59:20.892686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170939+0000","last_change":"2023-11-26T07:16:17.585612+0000","last_active":"2023-11-26T07:16:20.170939+0000","last_peered":"2023-11-26T07:16:20.170939+0000","last_clean":"2023-11-26T07:16:20.170939+0000","last_became_active":"2023-11-26T07:16:17.585332+0000","last_became_peered":"2023-11-26T07:16:17.585332+0000","last_unstale":"2023-11-26T07:16:20.170939+0000","last_undegraded":"2023-11-26T07:16:20.170939+0000","last_fullsized":"2023-11-26T07:16:20.170939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:09:53.739016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608938+0000","last_change":"2023-11-26T07:16:18.610149+0000","last_active":"2023-11-26T07:16:19.608938+0000","last_peered":"2023-11-26T07:16:19.608938+0000","last_clean":"2023-11-26T07:16:19.608938+0000","last_became_active":"2023-11-26T07:16:18.609849+0000","last_became_peered":"2023-11-26T07:16:18.609849+0000","last_unstale":"2023-11-26T07:16:19.608938+0000","last_undegraded":"2023-11-26T07:16:19.608938+0000","last_fullsized":"2023-11-26T07:16:19.608938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:07.718629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171008+0000","last_change":"2023-11-26T07:16:17.591559+0000","last_active":"2023-11-26T07:16:20.171008+0000","last_peered":"2023-11-26T07:16:20.171008+0000","last_clean":"2023-11-26T07:16:20.171008+0000","last_became_active":"2023-11-26T07:16:17.591285+0000","last_became_peered":"2023-11-26T07:16:17.591285+0000","last_unstale":"2023-11-26T07:16:20.171008+0000","last_undegraded":"2023-11-26T07:16:20.171008+0000","last_fullsized":"2023-11-26T07:16:20.171008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:56:50.275379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171534+0000","last_change":"2023-11-26T07:16:18.603137+0000","last_active":"2023-11-26T07:16:20.171534+0000","last_peered":"2023-11-26T07:16:20.171534+0000","last_clean":"2023-11-26T07:16:20.171534+0000","last_became_active":"2023-11-26T07:16:18.602867+0000","last_became_peered":"2023-11-26T07:16:18.602867+0000","last_unstale":"2023-11-26T07:16:20.171534+0000","last_undegraded":"2023-11-26T07:16:20.171534+0000","last_fullsized":"2023-11-26T07:16:20.171534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:20.475765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170948+0000","last_change":"2023-11-26T07:16:17.586972+0000","last_active":"2023-11-26T07:16:20.170948+0000","last_peered":"2023-11-26T07:16:20.170948+0000","last_clean":"2023-11-26T07:16:20.170948+0000","last_became_active":"2023-11-26T07:16:17.583592+0000","last_became_peered":"2023-11-26T07:16:17.583592+0000","last_unstale":"2023-11-26T07:16:20.170948+0000","last_undegraded":"2023-11-26T07:16:20.170948+0000","last_fullsized":"2023-11-26T07:16:20.170948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:03:52.477073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607550+0000","last_change":"2023-11-26T07:16:18.607402+0000","last_active":"2023-11-26T07:16:19.607550+0000","last_peered":"2023-11-26T07:16:19.607550+0000","last_clean":"2023-11-26T07:16:19.607550+0000","last_became_active":"2023-11-26T07:16:18.607072+0000","last_became_peered":"2023-11-26T07:16:18.607072+0000","last_unstale":"2023-11-26T07:16:19.607550+0000","last_undegraded":"2023-11-26T07:16:19.607550+0000","last_fullsized":"2023-11-26T07:16:19.607550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:12:12.749341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606365+0000","last_change":"2023-11-26T07:16:17.589498+0000","last_active":"2023-11-26T07:16:19.606365+0000","last_peered":"2023-11-26T07:16:19.606365+0000","last_clean":"2023-11-26T07:16:19.606365+0000","last_became_active":"2023-11-26T07:16:17.588955+0000","last_became_peered":"2023-11-26T07:16:17.588955+0000","last_unstale":"2023-11-26T07:16:19.606365+0000","last_undegraded":"2023-11-26T07:16:19.606365+0000","last_fullsized":"2023-11-26T07:16:19.606365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:34.568596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170562+0000","last_change":"2023-11-26T07:16:18.607799+0000","last_active":"2023-11-26T07:16:20.170562+0000","last_peered":"2023-11-26T07:16:20.170562+0000","last_clean":"2023-11-26T07:16:20.170562+0000","last_became_active":"2023-11-26T07:16:18.606724+0000","last_became_peered":"2023-11-26T07:16:18.606724+0000","last_unstale":"2023-11-26T07:16:20.170562+0000","last_undegraded":"2023-11-26T07:16:20.170562+0000","last_fullsized":"2023-11-26T07:16:20.170562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:25:49.167252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170907+0000","last_change":"2023-11-26T07:16:17.590072+0000","last_active":"2023-11-26T07:16:20.170907+0000","last_peered":"2023-11-26T07:16:20.170907+0000","last_clean":"2023-11-26T07:16:20.170907+0000","last_became_active":"2023-11-26T07:16:17.589771+0000","last_became_peered":"2023-11-26T07:16:17.589771+0000","last_unstale":"2023-11-26T07:16:20.170907+0000","last_undegraded":"2023-11-26T07:16:20.170907+0000","last_fullsized":"2023-11-26T07:16:20.170907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:41.150316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170589+0000","last_change":"2023-11-26T07:16:18.601829+0000","last_active":"2023-11-26T07:16:20.170589+0000","last_peered":"2023-11-26T07:16:20.170589+0000","last_clean":"2023-11-26T07:16:20.170589+0000","last_became_active":"2023-11-26T07:16:18.601596+0000","last_became_peered":"2023-11-26T07:16:18.601596+0000","last_unstale":"2023-11-26T07:16:20.170589+0000","last_undegraded":"2023-11-26T07:16:20.170589+0000","last_fullsized":"2023-11-26T07:16:20.170589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:23.077029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.172356+0000","last_change":"2023-11-26T07:16:17.582108+0000","last_active":"2023-11-26T07:16:20.172356+0000","last_peered":"2023-11-26T07:16:20.172356+0000","last_clean":"2023-11-26T07:16:20.172356+0000","last_became_active":"2023-11-26T07:16:17.581840+0000","last_became_peered":"2023-11-26T07:16:17.581840+0000","last_unstale":"2023-11-26T07:16:20.172356+0000","last_undegraded":"2023-11-26T07:16:20.172356+0000","last_fullsized":"2023-11-26T07:16:20.172356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:44:49.695478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607565+0000","last_change":"2023-11-26T07:16:18.598723+0000","last_active":"2023-11-26T07:16:19.607565+0000","last_peered":"2023-11-26T07:16:19.607565+0000","last_clean":"2023-11-26T07:16:19.607565+0000","last_became_active":"2023-11-26T07:16:18.598441+0000","last_became_peered":"2023-11-26T07:16:18.598441+0000","last_unstale":"2023-11-26T07:16:19.607565+0000","last_undegraded":"2023-11-26T07:16:19.607565+0000","last_fullsized":"2023-11-26T07:16:19.607565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:53.869386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171335+0000","last_change":"2023-11-26T07:16:17.593677+0000","last_active":"2023-11-26T07:16:20.171335+0000","last_peered":"2023-11-26T07:16:20.171335+0000","last_clean":"2023-11-26T07:16:20.171335+0000","last_became_active":"2023-11-26T07:16:17.593426+0000","last_became_peered":"2023-11-26T07:16:17.593426+0000","last_unstale":"2023-11-26T07:16:20.171335+0000","last_undegraded":"2023-11-26T07:16:20.171335+0000","last_fullsized":"2023-11-26T07:16:20.171335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:09:56.548526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171554+0000","last_change":"2023-11-26T07:16:18.605993+0000","last_active":"2023-11-26T07:16:20.171554+0000","last_peered":"2023-11-26T07:16:20.171554+0000","last_clean":"2023-11-26T07:16:20.171554+0000","last_became_active":"2023-11-26T07:16:18.605669+0000","last_became_peered":"2023-11-26T07:16:18.605669+0000","last_unstale":"2023-11-26T07:16:20.171554+0000","last_undegraded":"2023-11-26T07:16:20.171554+0000","last_fullsized":"2023-11-26T07:16:20.171554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:59:20.892686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171814+0000","last_change":"2023-11-26T07:16:17.584124+0000","last_active":"2023-11-26T07:16:20.171814+0000","last_peered":"2023-11-26T07:16:20.171814+0000","last_clean":"2023-11-26T07:16:20.171814+0000","last_became_active":"2023-11-26T07:16:17.583832+0000","last_became_peered":"2023-11-26T07:16:17.583832+0000","last_unstale":"2023-11-26T07:16:20.171814+0000","last_undegraded":"2023-11-26T07:16:20.171814+0000","last_fullsized":"2023-11-26T07:16:20.171814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:34.128300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608954+0000","last_change":"2023-11-26T07:16:18.613019+0000","last_active":"2023-11-26T07:16:19.608954+0000","last_peered":"2023-11-26T07:16:19.608954+0000","last_clean":"2023-11-26T07:16:19.608954+0000","last_became_active":"2023-11-26T07:16:18.612702+0000","last_became_peered":"2023-11-26T07:16:18.612702+0000","last_unstale":"2023-11-26T07:16:19.608954+0000","last_undegraded":"2023-11-26T07:16:19.608954+0000","last_fullsized":"2023-11-26T07:16:19.608954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:20.475765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171762+0000","last_change":"2023-11-26T07:16:17.590247+0000","last_active":"2023-11-26T07:16:20.171762+0000","last_peered":"2023-11-26T07:16:20.171762+0000","last_clean":"2023-11-26T07:16:20.171762+0000","last_became_active":"2023-11-26T07:16:17.589988+0000","last_became_peered":"2023-11-26T07:16:17.589988+0000","last_unstale":"2023-11-26T07:16:20.171762+0000","last_undegraded":"2023-11-26T07:16:20.171762+0000","last_fullsized":"2023-11-26T07:16:20.171762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:41:43.517205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171153+0000","last_change":"2023-11-26T07:16:18.610674+0000","last_active":"2023-11-26T07:16:20.171153+0000","last_peered":"2023-11-26T07:16:20.171153+0000","last_clean":"2023-11-26T07:16:20.171153+0000","last_became_active":"2023-11-26T07:16:18.610404+0000","last_became_peered":"2023-11-26T07:16:18.610404+0000","last_unstale":"2023-11-26T07:16:20.171153+0000","last_undegraded":"2023-11-26T07:16:20.171153+0000","last_fullsized":"2023-11-26T07:16:20.171153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:09.021844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606535+0000","last_change":"2023-11-26T07:16:17.585931+0000","last_active":"2023-11-26T07:16:19.606535+0000","last_peered":"2023-11-26T07:16:19.606535+0000","last_clean":"2023-11-26T07:16:19.606535+0000","last_became_active":"2023-11-26T07:16:17.585681+0000","last_became_peered":"2023-11-26T07:16:17.585681+0000","last_unstale":"2023-11-26T07:16:19.606535+0000","last_undegraded":"2023-11-26T07:16:19.606535+0000","last_fullsized":"2023-11-26T07:16:19.606535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:22.020678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170989+0000","last_change":"2023-11-26T07:16:18.601095+0000","last_active":"2023-11-26T07:16:20.170989+0000","last_peered":"2023-11-26T07:16:20.170989+0000","last_clean":"2023-11-26T07:16:20.170989+0000","last_became_active":"2023-11-26T07:16:18.600757+0000","last_became_peered":"2023-11-26T07:16:18.600757+0000","last_unstale":"2023-11-26T07:16:20.170989+0000","last_undegraded":"2023-11-26T07:16:20.170989+0000","last_fullsized":"2023-11-26T07:16:20.170989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:07.718629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171326+0000","last_change":"2023-11-26T07:16:17.582741+0000","last_active":"2023-11-26T07:16:20.171326+0000","last_peered":"2023-11-26T07:16:20.171326+0000","last_clean":"2023-11-26T07:16:20.171326+0000","last_became_active":"2023-11-26T07:16:17.582472+0000","last_became_peered":"2023-11-26T07:16:17.582472+0000","last_unstale":"2023-11-26T07:16:20.171326+0000","last_undegraded":"2023-11-26T07:16:20.171326+0000","last_fullsized":"2023-11-26T07:16:20.171326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:09:56.590410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171684+0000","last_change":"2023-11-26T07:16:18.603976+0000","last_active":"2023-11-26T07:16:20.171684+0000","last_peered":"2023-11-26T07:16:20.171684+0000","last_clean":"2023-11-26T07:16:20.171684+0000","last_became_active":"2023-11-26T07:16:18.603663+0000","last_became_peered":"2023-11-26T07:16:18.603663+0000","last_unstale":"2023-11-26T07:16:20.171684+0000","last_undegraded":"2023-11-26T07:16:20.171684+0000","last_fullsized":"2023-11-26T07:16:20.171684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:16.330339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171055+0000","last_change":"2023-11-26T07:16:17.588408+0000","last_active":"2023-11-26T07:16:20.171055+0000","last_peered":"2023-11-26T07:16:20.171055+0000","last_clean":"2023-11-26T07:16:20.171055+0000","last_became_active":"2023-11-26T07:16:17.588080+0000","last_became_peered":"2023-11-26T07:16:17.588080+0000","last_unstale":"2023-11-26T07:16:20.171055+0000","last_undegraded":"2023-11-26T07:16:20.171055+0000","last_fullsized":"2023-11-26T07:16:20.171055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:08:52.679738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170959+0000","last_change":"2023-11-26T07:16:18.598247+0000","last_active":"2023-11-26T07:16:20.170959+0000","last_peered":"2023-11-26T07:16:20.170959+0000","last_clean":"2023-11-26T07:16:20.170959+0000","last_became_active":"2023-11-26T07:16:18.597923+0000","last_became_peered":"2023-11-26T07:16:18.597923+0000","last_unstale":"2023-11-26T07:16:20.170959+0000","last_undegraded":"2023-11-26T07:16:20.170959+0000","last_fullsized":"2023-11-26T07:16:20.170959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:40.918037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606244+0000","last_change":"2023-11-26T07:16:17.585908+0000","last_active":"2023-11-26T07:16:19.606244+0000","last_peered":"2023-11-26T07:16:19.606244+0000","last_clean":"2023-11-26T07:16:19.606244+0000","last_became_active":"2023-11-26T07:16:17.585624+0000","last_became_peered":"2023-11-26T07:16:17.585624+0000","last_unstale":"2023-11-26T07:16:19.606244+0000","last_undegraded":"2023-11-26T07:16:19.606244+0000","last_fullsized":"2023-11-26T07:16:19.606244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:40.325385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.609084+0000","last_change":"2023-11-26T07:16:18.609471+0000","last_active":"2023-11-26T07:16:19.609084+0000","last_peered":"2023-11-26T07:16:19.609084+0000","last_clean":"2023-11-26T07:16:19.609084+0000","last_became_active":"2023-11-26T07:16:18.609179+0000","last_became_peered":"2023-11-26T07:16:18.609179+0000","last_unstale":"2023-11-26T07:16:19.609084+0000","last_undegraded":"2023-11-26T07:16:19.609084+0000","last_fullsized":"2023-11-26T07:16:19.609084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:28:14.679869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170843+0000","last_change":"2023-11-26T07:16:17.581389+0000","last_active":"2023-11-26T07:16:20.170843+0000","last_peered":"2023-11-26T07:16:20.170843+0000","last_clean":"2023-11-26T07:16:20.170843+0000","last_became_active":"2023-11-26T07:16:17.581119+0000","last_became_peered":"2023-11-26T07:16:17.581119+0000","last_unstale":"2023-11-26T07:16:20.170843+0000","last_undegraded":"2023-11-26T07:16:20.170843+0000","last_fullsized":"2023-11-26T07:16:20.170843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:22.835744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607981+0000","last_change":"2023-11-26T07:16:18.605266+0000","last_active":"2023-11-26T07:16:19.607981+0000","last_peered":"2023-11-26T07:16:19.607981+0000","last_clean":"2023-11-26T07:16:19.607981+0000","last_became_active":"2023-11-26T07:16:18.604983+0000","last_became_peered":"2023-11-26T07:16:18.604983+0000","last_unstale":"2023-11-26T07:16:19.607981+0000","last_undegraded":"2023-11-26T07:16:19.607981+0000","last_fullsized":"2023-11-26T07:16:19.607981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:52:47.905499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171901+0000","last_change":"2023-11-26T07:16:17.582872+0000","last_active":"2023-11-26T07:16:20.171901+0000","last_peered":"2023-11-26T07:16:20.171901+0000","last_clean":"2023-11-26T07:16:20.171901+0000","last_became_active":"2023-11-26T07:16:17.582623+0000","last_became_peered":"2023-11-26T07:16:17.582623+0000","last_unstale":"2023-11-26T07:16:20.171901+0000","last_undegraded":"2023-11-26T07:16:20.171901+0000","last_fullsized":"2023-11-26T07:16:20.171901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:26:20.166154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170587+0000","last_change":"2023-11-26T07:16:18.608196+0000","last_active":"2023-11-26T07:16:20.170587+0000","last_peered":"2023-11-26T07:16:20.170587+0000","last_clean":"2023-11-26T07:16:20.170587+0000","last_became_active":"2023-11-26T07:16:18.607929+0000","last_became_peered":"2023-11-26T07:16:18.607929+0000","last_unstale":"2023-11-26T07:16:20.170587+0000","last_undegraded":"2023-11-26T07:16:20.170587+0000","last_fullsized":"2023-11-26T07:16:20.170587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:20.475765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171030+0000","last_change":"2023-11-26T07:16:17.587193+0000","last_active":"2023-11-26T07:16:20.171030+0000","last_peered":"2023-11-26T07:16:20.171030+0000","last_clean":"2023-11-26T07:16:20.171030+0000","last_became_active":"2023-11-26T07:16:17.586894+0000","last_became_peered":"2023-11-26T07:16:17.586894+0000","last_unstale":"2023-11-26T07:16:20.171030+0000","last_undegraded":"2023-11-26T07:16:20.171030+0000","last_fullsized":"2023-11-26T07:16:20.171030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:44:06.246834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.609102+0000","last_change":"2023-11-26T07:16:18.610127+0000","last_active":"2023-11-26T07:16:19.609102+0000","last_peered":"2023-11-26T07:16:19.609102+0000","last_clean":"2023-11-26T07:16:19.609102+0000","last_became_active":"2023-11-26T07:16:18.609428+0000","last_became_peered":"2023-11-26T07:16:18.609428+0000","last_unstale":"2023-11-26T07:16:19.609102+0000","last_undegraded":"2023-11-26T07:16:19.609102+0000","last_fullsized":"2023-11-26T07:16:19.609102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:04:47.701141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171070+0000","last_change":"2023-11-26T07:16:17.588399+0000","last_active":"2023-11-26T07:16:20.171070+0000","last_peered":"2023-11-26T07:16:20.171070+0000","last_clean":"2023-11-26T07:16:20.171070+0000","last_became_active":"2023-11-26T07:16:17.588135+0000","last_became_peered":"2023-11-26T07:16:17.588135+0000","last_unstale":"2023-11-26T07:16:20.171070+0000","last_undegraded":"2023-11-26T07:16:20.171070+0000","last_fullsized":"2023-11-26T07:16:20.171070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:34.128300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170831+0000","last_change":"2023-11-26T07:16:18.597411+0000","last_active":"2023-11-26T07:16:20.170831+0000","last_peered":"2023-11-26T07:16:20.170831+0000","last_clean":"2023-11-26T07:16:20.170831+0000","last_became_active":"2023-11-26T07:16:18.596501+0000","last_became_peered":"2023-11-26T07:16:18.596501+0000","last_unstale":"2023-11-26T07:16:20.170831+0000","last_undegraded":"2023-11-26T07:16:20.170831+0000","last_fullsized":"2023-11-26T07:16:20.170831+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:41:44.521573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171580+0000","last_change":"2023-11-26T07:16:17.582720+0000","last_active":"2023-11-26T07:16:20.171580+0000","last_peered":"2023-11-26T07:16:20.171580+0000","last_clean":"2023-11-26T07:16:20.171580+0000","last_became_active":"2023-11-26T07:16:17.582268+0000","last_became_peered":"2023-11-26T07:16:17.582268+0000","last_unstale":"2023-11-26T07:16:20.171580+0000","last_undegraded":"2023-11-26T07:16:20.171580+0000","last_fullsized":"2023-11-26T07:16:20.171580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:09:56.548526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171974+0000","last_change":"2023-11-26T07:16:18.603811+0000","last_active":"2023-11-26T07:16:20.171974+0000","last_peered":"2023-11-26T07:16:20.171974+0000","last_clean":"2023-11-26T07:16:20.171974+0000","last_became_active":"2023-11-26T07:16:18.603540+0000","last_became_peered":"2023-11-26T07:16:18.603540+0000","last_unstale":"2023-11-26T07:16:20.171974+0000","last_undegraded":"2023-11-26T07:16:20.171974+0000","last_fullsized":"2023-11-26T07:16:20.171974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:28:14.679869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608429+0000","last_change":"2023-11-26T07:16:17.586817+0000","last_active":"2023-11-26T07:16:19.608429+0000","last_peered":"2023-11-26T07:16:19.608429+0000","last_clean":"2023-11-26T07:16:19.608429+0000","last_became_active":"2023-11-26T07:16:17.586505+0000","last_became_peered":"2023-11-26T07:16:17.586505+0000","last_unstale":"2023-11-26T07:16:19.608429+0000","last_undegraded":"2023-11-26T07:16:19.608429+0000","last_fullsized":"2023-11-26T07:16:19.608429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:09:56.548526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607332+0000","last_change":"2023-11-26T07:16:18.599629+0000","last_active":"2023-11-26T07:16:19.607332+0000","last_peered":"2023-11-26T07:16:19.607332+0000","last_clean":"2023-11-26T07:16:19.607332+0000","last_became_active":"2023-11-26T07:16:18.599367+0000","last_became_peered":"2023-11-26T07:16:18.599367+0000","last_unstale":"2023-11-26T07:16:19.607332+0000","last_undegraded":"2023-11-26T07:16:19.607332+0000","last_fullsized":"2023-11-26T07:16:19.607332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:47.337440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170516+0000","last_change":"2023-11-26T07:16:17.589253+0000","last_active":"2023-11-26T07:16:20.170516+0000","last_peered":"2023-11-26T07:16:20.170516+0000","last_clean":"2023-11-26T07:16:20.170516+0000","last_became_active":"2023-11-26T07:16:17.588770+0000","last_became_peered":"2023-11-26T07:16:17.588770+0000","last_unstale":"2023-11-26T07:16:20.170516+0000","last_undegraded":"2023-11-26T07:16:20.170516+0000","last_fullsized":"2023-11-26T07:16:20.170516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:50:26.219042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170589+0000","last_change":"2023-11-26T07:16:18.605132+0000","last_active":"2023-11-26T07:16:20.170589+0000","last_peered":"2023-11-26T07:16:20.170589+0000","last_clean":"2023-11-26T07:16:20.170589+0000","last_became_active":"2023-11-26T07:16:18.604881+0000","last_became_peered":"2023-11-26T07:16:18.604881+0000","last_unstale":"2023-11-26T07:16:20.170589+0000","last_undegraded":"2023-11-26T07:16:20.170589+0000","last_fullsized":"2023-11-26T07:16:20.170589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:07.718629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608310+0000","last_change":"2023-11-26T07:16:17.585783+0000","last_active":"2023-11-26T07:16:19.608310+0000","last_peered":"2023-11-26T07:16:19.608310+0000","last_clean":"2023-11-26T07:16:19.608310+0000","last_became_active":"2023-11-26T07:16:17.585518+0000","last_became_peered":"2023-11-26T07:16:17.585518+0000","last_unstale":"2023-11-26T07:16:19.608310+0000","last_undegraded":"2023-11-26T07:16:19.608310+0000","last_fullsized":"2023-11-26T07:16:19.608310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:44:49.695478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171085+0000","last_change":"2023-11-26T07:16:18.606707+0000","last_active":"2023-11-26T07:16:20.171085+0000","last_peered":"2023-11-26T07:16:20.171085+0000","last_clean":"2023-11-26T07:16:20.171085+0000","last_became_active":"2023-11-26T07:16:18.606449+0000","last_became_peered":"2023-11-26T07:16:18.606449+0000","last_unstale":"2023-11-26T07:16:20.171085+0000","last_undegraded":"2023-11-26T07:16:20.171085+0000","last_fullsized":"2023-11-26T07:16:20.171085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:46:23.472050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607780+0000","last_change":"2023-11-26T07:16:17.581842+0000","last_active":"2023-11-26T07:16:19.607780+0000","last_peered":"2023-11-26T07:16:19.607780+0000","last_clean":"2023-11-26T07:16:19.607780+0000","last_became_active":"2023-11-26T07:16:17.581522+0000","last_became_peered":"2023-11-26T07:16:17.581522+0000","last_unstale":"2023-11-26T07:16:19.607780+0000","last_undegraded":"2023-11-26T07:16:19.607780+0000","last_fullsized":"2023-11-26T07:16:19.607780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:04:46.696773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":268,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-26T07:16:18.588211+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:16:18.588211+0000","last_peered":"2023-11-26T07:16:18.588211+0000","last_clean":"2023-11-26T07:16:18.588211+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:16:18.588211+0000","last_undegraded":"2023-11-26T07:16:18.588211+0000","last_fullsized":"2023-11-26T07:16:18.588211+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28600,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343240,"statfs":{"total":96636764160,"available":96607477760,"internally_reserved":0,"allocated":1617920,"data_stored":1028668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.56200000000000006},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74299999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65600000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58699999999999997},"min":{"1min":0.40799999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88100000000000001,"5min":1.048,"15min":1.048},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.315,"15min":0.315},"max":{"1min":0.71599999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55000000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82499999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.622},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79700000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.624}]},{"osd":3,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.46400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.876,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.51800000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.49199999999999999},"min":{"1min":0.34200000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.45500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.52800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":1.171,"15min":1.171},"last":0.504}]},{"osd":6,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.39500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.93899999999999995,"5min":1.361,"15min":1.361},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0409999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":438924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75900000000000001,"5min":1.159,"15min":1.159},"last":0.747},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55900000000000005,"15min":0.57699999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73699999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67700000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80000000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.46700000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71799999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80800000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72299999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.433,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77600000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.70299999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.70999999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.5,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.747,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71299999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50700000000000001},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72099999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.625}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.747,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":438424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.33300000000000002},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.51000000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.33500000000000002},"min":{"1min":0.24199999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53300000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.58299999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.28799999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.57099999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83399999999999996,"5min":0.877,"15min":0.877},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86499999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55600000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79400000000000004,"5min":0.995,"15min":0.995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.62,"15min":0.629},"min":{"1min":0.45600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.42399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":1.129,"15min":1.129},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83799999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91400000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.879,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28600,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343240,"statfs":{"total":96636764160,"available":96607477760,"internally_reserved":0,"allocated":1617920,"data_stored":1028632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26900000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.219},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.379,"15min":0.37},"min":{"1min":0.245,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.309}]},{"osd":2,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.29499999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64300000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.249},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.34899999999999998},"min":{"1min":0.23200000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.34599999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40000000000000002,"15min":0.40300000000000002},"min":{"1min":0.29799999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.438,"15min":0.44},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60699999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33500000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77800000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.621,"15min":0.63500000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53600000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89000000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73699999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72999999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81899999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.65000000000000002,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.60799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":438812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.36099999999999999,"15min":0.36899999999999999},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.61199999999999999,"5min":0.627,"15min":0.627},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.26400000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.51100000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.35699999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.39000000000000001},"min":{"1min":0.26600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.53400000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.379,"15min":0.36799999999999999},"min":{"1min":0.28799999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38900000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"min":{"1min":0.28599999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.58799999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41799999999999998,"15min":0.42599999999999999},"min":{"1min":0.29599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68100000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.377}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.67100000000000004},"min":{"1min":0.41499999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81299999999999994,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55800000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.622,"15min":0.624},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90000000000000002,"5min":1.075,"15min":1.075},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0309999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.66600000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.747},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.51300000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.004,"5min":1.03,"15min":1.03},"last":0.56799999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64700000000000002,"15min":0.65900000000000003},"min":{"1min":0.504,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.502,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96199999999999997,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.57799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":439842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35399999999999998,"15min":0.35999999999999999},"min":{"1min":0.23999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.629,"5min":1.284,"15min":1.284},"last":0.312},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.42799999999999999,"15min":0.47299999999999998},"min":{"1min":0.26800000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.26000000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.437,"15min":0.51500000000000001},"min":{"1min":0.25,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58799999999999997,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.44700000000000001,"15min":0.49199999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.69399999999999995,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.35099999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46800000000000003,"15min":0.502},"min":{"1min":0.27900000000000003,"5min":0.247,"15min":0.247},"max":{"1min":0.61199999999999999,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41199999999999998,"15min":0.42699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.59799999999999998,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.37}]},{"osd":4,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.64600000000000002,"15min":0.67400000000000004},"min":{"1min":0.41499999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79000000000000004,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.63800000000000001,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81899999999999995,"5min":1.405,"15min":1.405},"last":0.45000000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.625,"15min":0.63700000000000001},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78100000000000003,"5min":1.377,"15min":1.377},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.71499999999999997,"15min":0.79500000000000004},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89200000000000002,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.47099999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.67100000000000004},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79800000000000004,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.71599999999999997,"15min":0.78900000000000003},"min":{"1min":0.434,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82599999999999996,"5min":4.883,"15min":4.883},"last":0.50700000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.80500000000000005},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.68899999999999995,"15min":0.73299999999999998},"min":{"1min":0.437,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.876,"5min":2.923,"15min":2.923},"last":0.46100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":438230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.60999999999999999},"min":{"1min":0.34300000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.79400000000000004,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68500000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.69499999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.375,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.65700000000000003},"min":{"1min":0.376,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91100000000000003,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.78600000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56000000000000005,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.67300000000000004},"min":{"1min":0.438,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76000000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56200000000000006,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53600000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58299999999999996,"15min":0.60299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73899999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.63600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.51000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":439864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53000000000000003,"15min":0.50800000000000001},"min":{"1min":0.29799999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.54300000000000004},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.76700000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.71199999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.65400000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.55400000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.749}]},{"osd":3,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.39300000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.627},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.751,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71999999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.35599999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.67300000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84699999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69099999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.78400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:16:22.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:16:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.867+0000 7fdd4696f700 1 Processor -- start 2023-11-26T07:16:22.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.867+0000 7fdd4696f700 1 -- start start 2023-11-26T07:16:22.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd4696f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdd40158670 0x7fdd400622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd4696f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdd40158a50 0x7fdd400c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd4696f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd400cfff0 con 0x7fdd40162a30 2023-11-26T07:16:22.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd4696f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd400cfd10 con 0x7fdd40158a50 2023-11-26T07:16:22.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd4696f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd40161f50 con 0x7fdd40158670 2023-11-26T07:16:22.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd3f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdd40158a50 0x7fdd400c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd44f0c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fdd40162a30 0x7fdd400d0230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60672/0 (socket says 172.21.15.83:60672) 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd44f0c700 1 -- 172.21.15.83:0/3351054930 learned_addr learned my addr 172.21.15.83:0/3351054930 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd3ffff700 1 --2- 172.21.15.83:0/3351054930 >> v2:172.21.15.203:3300/0 conn(0x7fdd40158670 0x7fdd400622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd3f7fe700 1 -- 172.21.15.83:0/3351054930 >> v2:172.21.15.203:3300/0 conn(0x7fdd40158670 msgr2=0x7fdd400622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd3f7fe700 1 --2- 172.21.15.83:0/3351054930 >> v2:172.21.15.203:3300/0 conn(0x7fdd40158670 0x7fdd400622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd3f7fe700 1 -- 172.21.15.83:0/3351054930 >> v1:172.21.15.83:6789/0 conn(0x7fdd40162a30 legacy=0x7fdd400d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd3f7fe700 1 -- 172.21.15.83:0/3351054930 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd40162230 con 0x7fdd40158a50 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.868+0000 7fdd3effd700 1 -- 172.21.15.83:0/3351054930 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896697147 0 0) 0x7fdd400cfff0 con 0x7fdd40162a30 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd3f7fe700 1 --2- 172.21.15.83:0/3351054930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40158a50 0x7fdd400c98b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdd4009e5c0 tx=0x7fdd30009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa22bf96f622210 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd3effd700 1 -- 172.21.15.83:0/3351054930 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30011830 con 0x7fdd40158a50 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd3effd700 1 -- 172.21.15.83:0/3351054930 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd30011990 con 0x7fdd40158a50 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd3effd700 1 -- 172.21.15.83:0/3351054930 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30011d20 con 0x7fdd40158a50 2023-11-26T07:16:22.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd4696f700 1 -- 172.21.15.83:0/3351054930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40158a50 msgr2=0x7fdd400c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd4696f700 1 --2- 172.21.15.83:0/3351054930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40158a50 0x7fdd400c98b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdd4009e5c0 tx=0x7fdd30009b20 comp rx=0 tx=0).stop 2023-11-26T07:16:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd4696f700 1 -- 172.21.15.83:0/3351054930 shutdown_connections 2023-11-26T07:16:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd4696f700 1 --2- 172.21.15.83:0/3351054930 >> v2:172.21.15.203:3300/0 conn(0x7fdd40158670 0x7fdd400622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd4696f700 1 --2- 172.21.15.83:0/3351054930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40158a50 0x7fdd400c98b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd4696f700 1 -- 172.21.15.83:0/3351054930 >> 172.21.15.83:0/3351054930 conn(0x7fdd400b57b0 msgr2=0x7fdd400b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd4696f700 1 -- 172.21.15.83:0/3351054930 shutdown_connections 2023-11-26T07:16:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.869+0000 7fdd4696f700 1 -- 172.21.15.83:0/3351054930 wait complete. 2023-11-26T07:16:22.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.871+0000 7fdd4696f700 1 Processor -- start 2023-11-26T07:16:22.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.871+0000 7fdd4696f700 1 -- start start 2023-11-26T07:16:22.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.871+0000 7fdd4696f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd40158670 0x7fdd40161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.871+0000 7fdd4696f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40158a50 0x7fdd4015fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.871+0000 7fdd4696f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd40162a30 0x7fdd4014b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.871+0000 7fdd4696f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd400b48a0 con 0x7fdd40158a50 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd4696f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd40160370 con 0x7fdd40162a30 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd4696f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd401611b0 con 0x7fdd40158670 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40158a50 0x7fdd4015fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40158a50 0x7fdd4015fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40636/0 (socket says 172.21.15.83:40636) 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3f7fe700 1 -- 172.21.15.83:0/3263855811 learned_addr learned my addr 172.21.15.83:0/3263855811 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3ffff700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd40158670 0x7fdd40161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd44f0c700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd40162a30 0x7fdd4014b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3f7fe700 1 -- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd40158670 msgr2=0x7fdd40161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3f7fe700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd40158670 0x7fdd40161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3f7fe700 1 -- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd40162a30 msgr2=0x7fdd4014b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3f7fe700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd40162a30 0x7fdd4014b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3f7fe700 1 -- 172.21.15.83:0/3263855811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd300094a0 con 0x7fdd40158a50 2023-11-26T07:16:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3ffff700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd40158670 0x7fdd40161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd44f0c700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd40162a30 0x7fdd4014b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3f7fe700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40158a50 0x7fdd4015fe30 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdd300090b0 tx=0x7fdd30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3cff9700 1 -- 172.21.15.83:0/3263855811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd300038a0 con 0x7fdd40158a50 2023-11-26T07:16:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3cff9700 1 -- 172.21.15.83:0/3263855811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3001f070 con 0x7fdd40158a50 2023-11-26T07:16:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd4696f700 1 -- 172.21.15.83:0/3263855811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd401605a0 con 0x7fdd40158a50 2023-11-26T07:16:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd3cff9700 1 -- 172.21.15.83:0/3263855811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30003c20 con 0x7fdd40158a50 2023-11-26T07:16:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.872+0000 7fdd4696f700 1 -- 172.21.15.83:0/3263855811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd40160930 con 0x7fdd40158a50 2023-11-26T07:16:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.873+0000 7fdd4696f700 1 -- 172.21.15.83:0/3263855811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd40067f60 con 0x7fdd40158a50 2023-11-26T07:16:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.874+0000 7fdd3cff9700 1 -- 172.21.15.83:0/3263855811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fdd3001a430 con 0x7fdd40158a50 2023-11-26T07:16:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.874+0000 7fdd3cff9700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd2c06b020 0x7fdd2c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.874+0000 7fdd3cff9700 1 -- 172.21.15.83:0/3263855811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd30021070 con 0x7fdd40158a50 2023-11-26T07:16:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.874+0000 7fdd3ffff700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd2c06b020 0x7fdd2c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.875+0000 7fdd3ffff700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd2c06b020 0x7fdd2c06d4e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd340099f0 tx=0x7fdd34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:22.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:22.879+0000 7fdd3cff9700 1 -- 172.21.15.83:0/3263855811 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd300560e0 con 0x7fdd40158a50 2023-11-26T07:16:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.064+0000 7fdd4696f700 1 -- 172.21.15.83:0/3263855811 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd400cf8a0 con 0x7fdd2c06b020 2023-11-26T07:16:23.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.072+0000 7fdd3cff9700 1 -- 172.21.15.83:0/3263855811 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289415 (secure 0 0 0) 0x7fdd34009680 con 0x7fdd2c06b020 2023-11-26T07:16:23.077 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 -- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd2c06b020 msgr2=0x7fdd2c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd2c06b020 0x7fdd2c06d4e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd340099f0 tx=0x7fdd34008040 comp rx=0 tx=0).stop 2023-11-26T07:16:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 -- 172.21.15.83:0/3263855811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40158a50 msgr2=0x7fdd4015fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40158a50 0x7fdd4015fe30 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdd300090b0 tx=0x7fdd30004750 comp rx=0 tx=0).stop 2023-11-26T07:16:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 -- 172.21.15.83:0/3263855811 shutdown_connections 2023-11-26T07:16:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd2c06b020 0x7fdd2c06d4e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd40162a30 0x7fdd4014b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd40158670 0x7fdd40161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 --2- 172.21.15.83:0/3263855811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd40158a50 0x7fdd4015fe30 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 -- 172.21.15.83:0/3263855811 >> 172.21.15.83:0/3263855811 conn(0x7fdd400b57b0 msgr2=0x7fdd4006e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 -- 172.21.15.83:0/3263855811 shutdown_connections 2023-11-26T07:16:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.076+0000 7fdd4696f700 1 -- 172.21.15.83:0/3263855811 wait complete. 2023-11-26T07:16:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:16:23.089 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-26T07:16:22.310325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225352,"kb_used_data":9192,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749368,"statfs":{"total":773094113280,"available":772863352832,"internally_reserved":0,"allocated":9412608,"data_stored":4691396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.742268"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171656+0000","last_change":"2023-11-26T07:16:18.603161+0000","last_active":"2023-11-26T07:16:20.171656+0000","last_peered":"2023-11-26T07:16:20.171656+0000","last_clean":"2023-11-26T07:16:20.171656+0000","last_became_active":"2023-11-26T07:16:18.602887+0000","last_became_peered":"2023-11-26T07:16:18.602887+0000","last_unstale":"2023-11-26T07:16:20.171656+0000","last_undegraded":"2023-11-26T07:16:20.171656+0000","last_fullsized":"2023-11-26T07:16:20.171656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:36:54.339287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171980+0000","last_change":"2023-11-26T07:16:18.602020+0000","last_active":"2023-11-26T07:16:20.171980+0000","last_peered":"2023-11-26T07:16:20.171980+0000","last_clean":"2023-11-26T07:16:20.171980+0000","last_became_active":"2023-11-26T07:16:18.601741+0000","last_became_peered":"2023-11-26T07:16:18.601741+0000","last_unstale":"2023-11-26T07:16:20.171980+0000","last_undegraded":"2023-11-26T07:16:20.171980+0000","last_fullsized":"2023-11-26T07:16:20.171980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:44:32.098990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607570+0000","last_change":"2023-11-26T07:16:18.597182+0000","last_active":"2023-11-26T07:16:19.607570+0000","last_peered":"2023-11-26T07:16:19.607570+0000","last_clean":"2023-11-26T07:16:19.607570+0000","last_became_active":"2023-11-26T07:16:18.596897+0000","last_became_peered":"2023-11-26T07:16:18.596897+0000","last_unstale":"2023-11-26T07:16:19.607570+0000","last_undegraded":"2023-11-26T07:16:19.607570+0000","last_fullsized":"2023-11-26T07:16:19.607570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:35.572964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171798+0000","last_change":"2023-11-26T07:16:18.604093+0000","last_active":"2023-11-26T07:16:20.171798+0000","last_peered":"2023-11-26T07:16:20.171798+0000","last_clean":"2023-11-26T07:16:20.171798+0000","last_became_active":"2023-11-26T07:16:18.603811+0000","last_became_peered":"2023-11-26T07:16:18.603811+0000","last_unstale":"2023-11-26T07:16:20.171798+0000","last_undegraded":"2023-11-26T07:16:20.171798+0000","last_fullsized":"2023-11-26T07:16:20.171798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:09:50.095468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608656+0000","last_change":"2023-11-26T07:16:18.612495+0000","last_active":"2023-11-26T07:16:19.608656+0000","last_peered":"2023-11-26T07:16:19.608656+0000","last_clean":"2023-11-26T07:16:19.608656+0000","last_became_active":"2023-11-26T07:16:18.612101+0000","last_became_peered":"2023-11-26T07:16:18.612101+0000","last_unstale":"2023-11-26T07:16:19.608656+0000","last_undegraded":"2023-11-26T07:16:19.608656+0000","last_fullsized":"2023-11-26T07:16:19.608656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:48.875519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170737+0000","last_change":"2023-11-26T07:16:18.599860+0000","last_active":"2023-11-26T07:16:20.170737+0000","last_peered":"2023-11-26T07:16:20.170737+0000","last_clean":"2023-11-26T07:16:20.170737+0000","last_became_active":"2023-11-26T07:16:18.599603+0000","last_became_peered":"2023-11-26T07:16:18.599603+0000","last_unstale":"2023-11-26T07:16:20.170737+0000","last_undegraded":"2023-11-26T07:16:20.170737+0000","last_fullsized":"2023-11-26T07:16:20.170737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:45.731926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170878+0000","last_change":"2023-11-26T07:16:18.600023+0000","last_active":"2023-11-26T07:16:20.170878+0000","last_peered":"2023-11-26T07:16:20.170878+0000","last_clean":"2023-11-26T07:16:20.170878+0000","last_became_active":"2023-11-26T07:16:18.599474+0000","last_became_peered":"2023-11-26T07:16:18.599474+0000","last_unstale":"2023-11-26T07:16:20.170878+0000","last_undegraded":"2023-11-26T07:16:20.170878+0000","last_fullsized":"2023-11-26T07:16:20.170878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:17:01.015611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170742+0000","last_change":"2023-11-26T07:16:18.597258+0000","last_active":"2023-11-26T07:16:20.170742+0000","last_peered":"2023-11-26T07:16:20.170742+0000","last_clean":"2023-11-26T07:16:20.170742+0000","last_became_active":"2023-11-26T07:16:18.596991+0000","last_became_peered":"2023-11-26T07:16:18.596991+0000","last_unstale":"2023-11-26T07:16:20.170742+0000","last_undegraded":"2023-11-26T07:16:20.170742+0000","last_fullsized":"2023-11-26T07:16:20.170742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:48.875519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171146+0000","last_change":"2023-11-26T07:16:18.612700+0000","last_active":"2023-11-26T07:16:20.171146+0000","last_peered":"2023-11-26T07:16:20.171146+0000","last_clean":"2023-11-26T07:16:20.171146+0000","last_became_active":"2023-11-26T07:16:18.612394+0000","last_became_peered":"2023-11-26T07:16:18.612394+0000","last_unstale":"2023-11-26T07:16:20.171146+0000","last_undegraded":"2023-11-26T07:16:20.171146+0000","last_fullsized":"2023-11-26T07:16:20.171146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:20.475765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171265+0000","last_change":"2023-11-26T07:16:18.600807+0000","last_active":"2023-11-26T07:16:20.171265+0000","last_peered":"2023-11-26T07:16:20.171265+0000","last_clean":"2023-11-26T07:16:20.171265+0000","last_became_active":"2023-11-26T07:16:18.600493+0000","last_became_peered":"2023-11-26T07:16:18.600493+0000","last_unstale":"2023-11-26T07:16:20.171265+0000","last_undegraded":"2023-11-26T07:16:20.171265+0000","last_fullsized":"2023-11-26T07:16:20.171265+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:28:14.679869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170935+0000","last_change":"2023-11-26T07:16:18.596861+0000","last_active":"2023-11-26T07:16:20.170935+0000","last_peered":"2023-11-26T07:16:20.170935+0000","last_clean":"2023-11-26T07:16:20.170935+0000","last_became_active":"2023-11-26T07:16:18.596555+0000","last_became_peered":"2023-11-26T07:16:18.596555+0000","last_unstale":"2023-11-26T07:16:20.170935+0000","last_undegraded":"2023-11-26T07:16:20.170935+0000","last_fullsized":"2023-11-26T07:16:20.170935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:24:57.226424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170639+0000","last_change":"2023-11-26T07:16:18.604520+0000","last_active":"2023-11-26T07:16:20.170639+0000","last_peered":"2023-11-26T07:16:20.170639+0000","last_clean":"2023-11-26T07:16:20.170639+0000","last_became_active":"2023-11-26T07:16:18.604225+0000","last_became_peered":"2023-11-26T07:16:18.604225+0000","last_unstale":"2023-11-26T07:16:20.170639+0000","last_undegraded":"2023-11-26T07:16:20.170639+0000","last_fullsized":"2023-11-26T07:16:20.170639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:50.031595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607538+0000","last_change":"2023-11-26T07:16:18.602610+0000","last_active":"2023-11-26T07:16:19.607538+0000","last_peered":"2023-11-26T07:16:19.607538+0000","last_clean":"2023-11-26T07:16:19.607538+0000","last_became_active":"2023-11-26T07:16:18.602252+0000","last_became_peered":"2023-11-26T07:16:18.602252+0000","last_unstale":"2023-11-26T07:16:19.607538+0000","last_undegraded":"2023-11-26T07:16:19.607538+0000","last_fullsized":"2023-11-26T07:16:19.607538+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:23.025046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170998+0000","last_change":"2023-11-26T07:16:18.611064+0000","last_active":"2023-11-26T07:16:20.170998+0000","last_peered":"2023-11-26T07:16:20.170998+0000","last_clean":"2023-11-26T07:16:20.170998+0000","last_became_active":"2023-11-26T07:16:18.610771+0000","last_became_peered":"2023-11-26T07:16:18.610771+0000","last_unstale":"2023-11-26T07:16:20.170998+0000","last_undegraded":"2023-11-26T07:16:20.170998+0000","last_fullsized":"2023-11-26T07:16:20.170998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:48:59.976510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606297+0000","last_change":"2023-11-26T07:16:18.601102+0000","last_active":"2023-11-26T07:16:19.606297+0000","last_peered":"2023-11-26T07:16:19.606297+0000","last_clean":"2023-11-26T07:16:19.606297+0000","last_became_active":"2023-11-26T07:16:18.600698+0000","last_became_peered":"2023-11-26T07:16:18.600698+0000","last_unstale":"2023-11-26T07:16:19.606297+0000","last_undegraded":"2023-11-26T07:16:19.606297+0000","last_fullsized":"2023-11-26T07:16:19.606297+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:12:12.749341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608284+0000","last_change":"2023-11-26T07:16:18.604279+0000","last_active":"2023-11-26T07:16:19.608284+0000","last_peered":"2023-11-26T07:16:19.608284+0000","last_clean":"2023-11-26T07:16:19.608284+0000","last_became_active":"2023-11-26T07:16:18.604022+0000","last_became_peered":"2023-11-26T07:16:18.604022+0000","last_unstale":"2023-11-26T07:16:19.608284+0000","last_undegraded":"2023-11-26T07:16:19.608284+0000","last_fullsized":"2023-11-26T07:16:19.608284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:35:12.961085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.609098+0000","last_change":"2023-11-26T07:16:18.607619+0000","last_active":"2023-11-26T07:16:19.609098+0000","last_peered":"2023-11-26T07:16:19.609098+0000","last_clean":"2023-11-26T07:16:19.609098+0000","last_became_active":"2023-11-26T07:16:18.607322+0000","last_became_peered":"2023-11-26T07:16:18.607322+0000","last_unstale":"2023-11-26T07:16:19.609098+0000","last_undegraded":"2023-11-26T07:16:19.609098+0000","last_fullsized":"2023-11-26T07:16:19.609098+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:59:20.892686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171115+0000","last_change":"2023-11-26T07:16:18.602760+0000","last_active":"2023-11-26T07:16:20.171115+0000","last_peered":"2023-11-26T07:16:20.171115+0000","last_clean":"2023-11-26T07:16:20.171115+0000","last_became_active":"2023-11-26T07:16:18.602105+0000","last_became_peered":"2023-11-26T07:16:18.602105+0000","last_unstale":"2023-11-26T07:16:20.171115+0000","last_undegraded":"2023-11-26T07:16:20.171115+0000","last_fullsized":"2023-11-26T07:16:20.171115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:23.840113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171550+0000","last_change":"2023-11-26T07:16:18.604756+0000","last_active":"2023-11-26T07:16:20.171550+0000","last_peered":"2023-11-26T07:16:20.171550+0000","last_clean":"2023-11-26T07:16:20.171550+0000","last_became_active":"2023-11-26T07:16:18.604507+0000","last_became_peered":"2023-11-26T07:16:18.604507+0000","last_unstale":"2023-11-26T07:16:20.171550+0000","last_undegraded":"2023-11-26T07:16:20.171550+0000","last_fullsized":"2023-11-26T07:16:20.171550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:50.031595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171900+0000","last_change":"2023-11-26T07:16:18.607794+0000","last_active":"2023-11-26T07:16:20.171900+0000","last_peered":"2023-11-26T07:16:20.171900+0000","last_clean":"2023-11-26T07:16:20.171900+0000","last_became_active":"2023-11-26T07:16:18.607450+0000","last_became_peered":"2023-11-26T07:16:18.607450+0000","last_unstale":"2023-11-26T07:16:20.171900+0000","last_undegraded":"2023-11-26T07:16:20.171900+0000","last_fullsized":"2023-11-26T07:16:20.171900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:03:53.481441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171444+0000","last_change":"2023-11-26T07:16:18.611677+0000","last_active":"2023-11-26T07:16:20.171444+0000","last_peered":"2023-11-26T07:16:20.171444+0000","last_clean":"2023-11-26T07:16:20.171444+0000","last_became_active":"2023-11-26T07:16:18.611429+0000","last_became_peered":"2023-11-26T07:16:18.611429+0000","last_unstale":"2023-11-26T07:16:20.171444+0000","last_undegraded":"2023-11-26T07:16:20.171444+0000","last_fullsized":"2023-11-26T07:16:20.171444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:35:55.637783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170763+0000","last_change":"2023-11-26T07:16:18.607278+0000","last_active":"2023-11-26T07:16:20.170763+0000","last_peered":"2023-11-26T07:16:20.170763+0000","last_clean":"2023-11-26T07:16:20.170763+0000","last_became_active":"2023-11-26T07:16:18.606775+0000","last_became_peered":"2023-11-26T07:16:18.606775+0000","last_unstale":"2023-11-26T07:16:20.170763+0000","last_undegraded":"2023-11-26T07:16:20.170763+0000","last_fullsized":"2023-11-26T07:16:20.170763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:43:17.983907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170394+0000","last_change":"2023-11-26T07:16:18.607945+0000","last_active":"2023-11-26T07:16:20.170394+0000","last_peered":"2023-11-26T07:16:20.170394+0000","last_clean":"2023-11-26T07:16:20.170394+0000","last_became_active":"2023-11-26T07:16:18.607690+0000","last_became_peered":"2023-11-26T07:16:18.607690+0000","last_unstale":"2023-11-26T07:16:20.170394+0000","last_undegraded":"2023-11-26T07:16:20.170394+0000","last_fullsized":"2023-11-26T07:16:20.170394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:45.731926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171541+0000","last_change":"2023-11-26T07:16:18.607087+0000","last_active":"2023-11-26T07:16:20.171541+0000","last_peered":"2023-11-26T07:16:20.171541+0000","last_clean":"2023-11-26T07:16:20.171541+0000","last_became_active":"2023-11-26T07:16:18.606747+0000","last_became_peered":"2023-11-26T07:16:18.606747+0000","last_unstale":"2023-11-26T07:16:20.171541+0000","last_undegraded":"2023-11-26T07:16:20.171541+0000","last_fullsized":"2023-11-26T07:16:20.171541+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:41:32.975198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170738+0000","last_change":"2023-11-26T07:16:18.606444+0000","last_active":"2023-11-26T07:16:20.170738+0000","last_peered":"2023-11-26T07:16:20.170738+0000","last_clean":"2023-11-26T07:16:20.170738+0000","last_became_active":"2023-11-26T07:16:18.606140+0000","last_became_peered":"2023-11-26T07:16:18.606140+0000","last_unstale":"2023-11-26T07:16:20.170738+0000","last_undegraded":"2023-11-26T07:16:20.170738+0000","last_fullsized":"2023-11-26T07:16:20.170738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:38:24.528206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607470+0000","last_change":"2023-11-26T07:16:18.600865+0000","last_active":"2023-11-26T07:16:19.607470+0000","last_peered":"2023-11-26T07:16:19.607470+0000","last_clean":"2023-11-26T07:16:19.607470+0000","last_became_active":"2023-11-26T07:16:18.600618+0000","last_became_peered":"2023-11-26T07:16:18.600618+0000","last_unstale":"2023-11-26T07:16:19.607470+0000","last_undegraded":"2023-11-26T07:16:19.607470+0000","last_fullsized":"2023-11-26T07:16:19.607470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:06:56.993863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608629+0000","last_change":"2023-11-26T07:16:18.605000+0000","last_active":"2023-11-26T07:16:19.608629+0000","last_peered":"2023-11-26T07:16:19.608629+0000","last_clean":"2023-11-26T07:16:19.608629+0000","last_became_active":"2023-11-26T07:16:18.604733+0000","last_became_peered":"2023-11-26T07:16:18.604733+0000","last_unstale":"2023-11-26T07:16:19.608629+0000","last_undegraded":"2023-11-26T07:16:19.608629+0000","last_fullsized":"2023-11-26T07:16:19.608629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:42:45.731926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170435+0000","last_change":"2023-11-26T07:16:18.612041+0000","last_active":"2023-11-26T07:16:20.170435+0000","last_peered":"2023-11-26T07:16:20.170435+0000","last_clean":"2023-11-26T07:16:20.170435+0000","last_became_active":"2023-11-26T07:16:18.611757+0000","last_became_peered":"2023-11-26T07:16:18.611757+0000","last_unstale":"2023-11-26T07:16:20.170435+0000","last_undegraded":"2023-11-26T07:16:20.170435+0000","last_fullsized":"2023-11-26T07:16:20.170435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:48.875519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170861+0000","last_change":"2023-11-26T07:16:18.597641+0000","last_active":"2023-11-26T07:16:20.170861+0000","last_peered":"2023-11-26T07:16:20.170861+0000","last_clean":"2023-11-26T07:16:20.170861+0000","last_became_active":"2023-11-26T07:16:18.597280+0000","last_became_peered":"2023-11-26T07:16:18.597280+0000","last_unstale":"2023-11-26T07:16:20.170861+0000","last_undegraded":"2023-11-26T07:16:20.170861+0000","last_fullsized":"2023-11-26T07:16:20.170861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:58:15.236533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171766+0000","last_change":"2023-11-26T07:16:18.607413+0000","last_active":"2023-11-26T07:16:20.171766+0000","last_peered":"2023-11-26T07:16:20.171766+0000","last_clean":"2023-11-26T07:16:20.171766+0000","last_became_active":"2023-11-26T07:16:18.607080+0000","last_became_peered":"2023-11-26T07:16:18.607080+0000","last_unstale":"2023-11-26T07:16:20.171766+0000","last_undegraded":"2023-11-26T07:16:20.171766+0000","last_fullsized":"2023-11-26T07:16:20.171766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:31:16.121796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608440+0000","last_change":"2023-11-26T07:16:18.610537+0000","last_active":"2023-11-26T07:16:19.608440+0000","last_peered":"2023-11-26T07:16:19.608440+0000","last_clean":"2023-11-26T07:16:19.608440+0000","last_became_active":"2023-11-26T07:16:18.610137+0000","last_became_peered":"2023-11-26T07:16:18.610137+0000","last_unstale":"2023-11-26T07:16:19.608440+0000","last_undegraded":"2023-11-26T07:16:19.608440+0000","last_fullsized":"2023-11-26T07:16:19.608440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:40.918037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170336+0000","last_change":"2023-11-26T07:16:18.601464+0000","last_active":"2023-11-26T07:16:20.170336+0000","last_peered":"2023-11-26T07:16:20.170336+0000","last_clean":"2023-11-26T07:16:20.170336+0000","last_became_active":"2023-11-26T07:16:18.600122+0000","last_became_peered":"2023-11-26T07:16:18.600122+0000","last_unstale":"2023-11-26T07:16:20.170336+0000","last_undegraded":"2023-11-26T07:16:20.170336+0000","last_fullsized":"2023-11-26T07:16:20.170336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:40.918037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.609056+0000","last_change":"2023-11-26T07:16:18.604227+0000","last_active":"2023-11-26T07:16:19.609056+0000","last_peered":"2023-11-26T07:16:19.609056+0000","last_clean":"2023-11-26T07:16:19.609056+0000","last_became_active":"2023-11-26T07:16:18.603919+0000","last_became_peered":"2023-11-26T07:16:18.603919+0000","last_unstale":"2023-11-26T07:16:19.609056+0000","last_undegraded":"2023-11-26T07:16:19.609056+0000","last_fullsized":"2023-11-26T07:16:19.609056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:50.031595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171442+0000","last_change":"2023-11-26T07:16:17.593158+0000","last_active":"2023-11-26T07:16:20.171442+0000","last_peered":"2023-11-26T07:16:20.171442+0000","last_clean":"2023-11-26T07:16:20.171442+0000","last_became_active":"2023-11-26T07:16:17.592853+0000","last_became_peered":"2023-11-26T07:16:17.592853+0000","last_unstale":"2023-11-26T07:16:20.171442+0000","last_undegraded":"2023-11-26T07:16:20.171442+0000","last_fullsized":"2023-11-26T07:16:20.171442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:49:12.199077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171611+0000","last_change":"2023-11-26T07:16:18.607574+0000","last_active":"2023-11-26T07:16:20.171611+0000","last_peered":"2023-11-26T07:16:20.171611+0000","last_clean":"2023-11-26T07:16:20.171611+0000","last_became_active":"2023-11-26T07:16:18.606978+0000","last_became_peered":"2023-11-26T07:16:18.606978+0000","last_unstale":"2023-11-26T07:16:20.171611+0000","last_undegraded":"2023-11-26T07:16:20.171611+0000","last_fullsized":"2023-11-26T07:16:20.171611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:04:47.701141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.609150+0000","last_change":"2023-11-26T07:16:17.586841+0000","last_active":"2023-11-26T07:16:19.609150+0000","last_peered":"2023-11-26T07:16:19.609150+0000","last_clean":"2023-11-26T07:16:19.609150+0000","last_became_active":"2023-11-26T07:16:17.586579+0000","last_became_peered":"2023-11-26T07:16:17.586579+0000","last_unstale":"2023-11-26T07:16:19.609150+0000","last_undegraded":"2023-11-26T07:16:19.609150+0000","last_fullsized":"2023-11-26T07:16:19.609150+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:41.150316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170479+0000","last_change":"2023-11-26T07:16:18.612092+0000","last_active":"2023-11-26T07:16:20.170479+0000","last_peered":"2023-11-26T07:16:20.170479+0000","last_clean":"2023-11-26T07:16:20.170479+0000","last_became_active":"2023-11-26T07:16:18.611857+0000","last_became_peered":"2023-11-26T07:16:18.611857+0000","last_unstale":"2023-11-26T07:16:20.170479+0000","last_undegraded":"2023-11-26T07:16:20.170479+0000","last_fullsized":"2023-11-26T07:16:20.170479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:41:44.521573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171749+0000","last_change":"2023-11-26T07:16:17.581813+0000","last_active":"2023-11-26T07:16:20.171749+0000","last_peered":"2023-11-26T07:16:20.171749+0000","last_clean":"2023-11-26T07:16:20.171749+0000","last_became_active":"2023-11-26T07:16:17.581530+0000","last_became_peered":"2023-11-26T07:16:17.581530+0000","last_unstale":"2023-11-26T07:16:20.171749+0000","last_undegraded":"2023-11-26T07:16:20.171749+0000","last_fullsized":"2023-11-26T07:16:20.171749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:49:12.199077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607721+0000","last_change":"2023-11-26T07:16:18.600345+0000","last_active":"2023-11-26T07:16:19.607721+0000","last_peered":"2023-11-26T07:16:19.607721+0000","last_clean":"2023-11-26T07:16:19.607721+0000","last_became_active":"2023-11-26T07:16:18.600089+0000","last_became_peered":"2023-11-26T07:16:18.600089+0000","last_unstale":"2023-11-26T07:16:19.607721+0000","last_undegraded":"2023-11-26T07:16:19.607721+0000","last_fullsized":"2023-11-26T07:16:19.607721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:46.678079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171672+0000","last_change":"2023-11-26T07:16:17.585021+0000","last_active":"2023-11-26T07:16:20.171672+0000","last_peered":"2023-11-26T07:16:20.171672+0000","last_clean":"2023-11-26T07:16:20.171672+0000","last_became_active":"2023-11-26T07:16:17.584722+0000","last_became_peered":"2023-11-26T07:16:17.584722+0000","last_unstale":"2023-11-26T07:16:20.171672+0000","last_undegraded":"2023-11-26T07:16:20.171672+0000","last_fullsized":"2023-11-26T07:16:20.171672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:22:47.871151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171668+0000","last_change":"2023-11-26T07:16:18.608161+0000","last_active":"2023-11-26T07:16:20.171668+0000","last_peered":"2023-11-26T07:16:20.171668+0000","last_clean":"2023-11-26T07:16:20.171668+0000","last_became_active":"2023-11-26T07:16:18.606300+0000","last_became_peered":"2023-11-26T07:16:18.606300+0000","last_unstale":"2023-11-26T07:16:20.171668+0000","last_undegraded":"2023-11-26T07:16:20.171668+0000","last_fullsized":"2023-11-26T07:16:20.171668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:23:56.971286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608761+0000","last_change":"2023-11-26T07:16:17.584391+0000","last_active":"2023-11-26T07:16:19.608761+0000","last_peered":"2023-11-26T07:16:19.608761+0000","last_clean":"2023-11-26T07:16:19.608761+0000","last_became_active":"2023-11-26T07:16:17.584013+0000","last_became_peered":"2023-11-26T07:16:17.584013+0000","last_unstale":"2023-11-26T07:16:19.608761+0000","last_undegraded":"2023-11-26T07:16:19.608761+0000","last_fullsized":"2023-11-26T07:16:19.608761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:49:12.199077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608740+0000","last_change":"2023-11-26T07:16:18.608329+0000","last_active":"2023-11-26T07:16:19.608740+0000","last_peered":"2023-11-26T07:16:19.608740+0000","last_clean":"2023-11-26T07:16:19.608740+0000","last_became_active":"2023-11-26T07:16:18.608055+0000","last_became_peered":"2023-11-26T07:16:18.608055+0000","last_unstale":"2023-11-26T07:16:19.608740+0000","last_undegraded":"2023-11-26T07:16:19.608740+0000","last_fullsized":"2023-11-26T07:16:19.608740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:41:44.521573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608423+0000","last_change":"2023-11-26T07:16:17.589142+0000","last_active":"2023-11-26T07:16:19.608423+0000","last_peered":"2023-11-26T07:16:19.608423+0000","last_clean":"2023-11-26T07:16:19.608423+0000","last_became_active":"2023-11-26T07:16:17.588821+0000","last_became_peered":"2023-11-26T07:16:17.588821+0000","last_unstale":"2023-11-26T07:16:19.608423+0000","last_undegraded":"2023-11-26T07:16:19.608423+0000","last_fullsized":"2023-11-26T07:16:19.608423+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:34.128300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607241+0000","last_change":"2023-11-26T07:16:18.606595+0000","last_active":"2023-11-26T07:16:19.607241+0000","last_peered":"2023-11-26T07:16:19.607241+0000","last_clean":"2023-11-26T07:16:19.607241+0000","last_became_active":"2023-11-26T07:16:18.606153+0000","last_became_peered":"2023-11-26T07:16:18.606153+0000","last_unstale":"2023-11-26T07:16:19.607241+0000","last_undegraded":"2023-11-26T07:16:19.607241+0000","last_fullsized":"2023-11-26T07:16:19.607241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:12:42.626819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171672+0000","last_change":"2023-11-26T07:16:17.585406+0000","last_active":"2023-11-26T07:16:20.171672+0000","last_peered":"2023-11-26T07:16:20.171672+0000","last_clean":"2023-11-26T07:16:20.171672+0000","last_became_active":"2023-11-26T07:16:17.585147+0000","last_became_peered":"2023-11-26T07:16:17.585147+0000","last_unstale":"2023-11-26T07:16:20.171672+0000","last_undegraded":"2023-11-26T07:16:20.171672+0000","last_fullsized":"2023-11-26T07:16:20.171672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:56:50.275379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606380+0000","last_change":"2023-11-26T07:16:18.600937+0000","last_active":"2023-11-26T07:16:19.606380+0000","last_peered":"2023-11-26T07:16:19.606380+0000","last_clean":"2023-11-26T07:16:19.606380+0000","last_became_active":"2023-11-26T07:16:18.600372+0000","last_became_peered":"2023-11-26T07:16:18.600372+0000","last_unstale":"2023-11-26T07:16:19.606380+0000","last_undegraded":"2023-11-26T07:16:19.606380+0000","last_fullsized":"2023-11-26T07:16:19.606380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:53.869386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171725+0000","last_change":"2023-11-26T07:16:17.578953+0000","last_active":"2023-11-26T07:16:20.171725+0000","last_peered":"2023-11-26T07:16:20.171725+0000","last_clean":"2023-11-26T07:16:20.171725+0000","last_became_active":"2023-11-26T07:16:17.578650+0000","last_became_peered":"2023-11-26T07:16:17.578650+0000","last_unstale":"2023-11-26T07:16:20.171725+0000","last_undegraded":"2023-11-26T07:16:20.171725+0000","last_fullsized":"2023-11-26T07:16:20.171725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:09:49.091099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606546+0000","last_change":"2023-11-26T07:16:18.597738+0000","last_active":"2023-11-26T07:16:19.606546+0000","last_peered":"2023-11-26T07:16:19.606546+0000","last_clean":"2023-11-26T07:16:19.606546+0000","last_became_active":"2023-11-26T07:16:18.597440+0000","last_became_peered":"2023-11-26T07:16:18.597440+0000","last_unstale":"2023-11-26T07:16:19.606546+0000","last_undegraded":"2023-11-26T07:16:19.606546+0000","last_fullsized":"2023-11-26T07:16:19.606546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:46.678079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171443+0000","last_change":"2023-11-26T07:16:17.593644+0000","last_active":"2023-11-26T07:16:20.171443+0000","last_peered":"2023-11-26T07:16:20.171443+0000","last_clean":"2023-11-26T07:16:20.171443+0000","last_became_active":"2023-11-26T07:16:17.593333+0000","last_became_peered":"2023-11-26T07:16:17.593333+0000","last_unstale":"2023-11-26T07:16:20.171443+0000","last_undegraded":"2023-11-26T07:16:20.171443+0000","last_fullsized":"2023-11-26T07:16:20.171443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:41.150316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171177+0000","last_change":"2023-11-26T07:16:18.599341+0000","last_active":"2023-11-26T07:16:20.171177+0000","last_peered":"2023-11-26T07:16:20.171177+0000","last_clean":"2023-11-26T07:16:20.171177+0000","last_became_active":"2023-11-26T07:16:18.599052+0000","last_became_peered":"2023-11-26T07:16:18.599052+0000","last_unstale":"2023-11-26T07:16:20.171177+0000","last_undegraded":"2023-11-26T07:16:20.171177+0000","last_fullsized":"2023-11-26T07:16:20.171177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:22:50.031595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171337+0000","last_change":"2023-11-26T07:16:17.585949+0000","last_active":"2023-11-26T07:16:20.171337+0000","last_peered":"2023-11-26T07:16:20.171337+0000","last_clean":"2023-11-26T07:16:20.171337+0000","last_became_active":"2023-11-26T07:16:17.585688+0000","last_became_peered":"2023-11-26T07:16:17.585688+0000","last_unstale":"2023-11-26T07:16:20.171337+0000","last_undegraded":"2023-11-26T07:16:20.171337+0000","last_fullsized":"2023-11-26T07:16:20.171337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:42:47.406959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.172089+0000","last_change":"2023-11-26T07:16:18.605464+0000","last_active":"2023-11-26T07:16:20.172089+0000","last_peered":"2023-11-26T07:16:20.172089+0000","last_clean":"2023-11-26T07:16:20.172089+0000","last_became_active":"2023-11-26T07:16:18.605180+0000","last_became_peered":"2023-11-26T07:16:18.605180+0000","last_unstale":"2023-11-26T07:16:20.172089+0000","last_undegraded":"2023-11-26T07:16:20.172089+0000","last_fullsized":"2023-11-26T07:16:20.172089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:59:20.892686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170939+0000","last_change":"2023-11-26T07:16:17.585612+0000","last_active":"2023-11-26T07:16:20.170939+0000","last_peered":"2023-11-26T07:16:20.170939+0000","last_clean":"2023-11-26T07:16:20.170939+0000","last_became_active":"2023-11-26T07:16:17.585332+0000","last_became_peered":"2023-11-26T07:16:17.585332+0000","last_unstale":"2023-11-26T07:16:20.170939+0000","last_undegraded":"2023-11-26T07:16:20.170939+0000","last_fullsized":"2023-11-26T07:16:20.170939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:09:53.739016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608938+0000","last_change":"2023-11-26T07:16:18.610149+0000","last_active":"2023-11-26T07:16:19.608938+0000","last_peered":"2023-11-26T07:16:19.608938+0000","last_clean":"2023-11-26T07:16:19.608938+0000","last_became_active":"2023-11-26T07:16:18.609849+0000","last_became_peered":"2023-11-26T07:16:18.609849+0000","last_unstale":"2023-11-26T07:16:19.608938+0000","last_undegraded":"2023-11-26T07:16:19.608938+0000","last_fullsized":"2023-11-26T07:16:19.608938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:07.718629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171008+0000","last_change":"2023-11-26T07:16:17.591559+0000","last_active":"2023-11-26T07:16:20.171008+0000","last_peered":"2023-11-26T07:16:20.171008+0000","last_clean":"2023-11-26T07:16:20.171008+0000","last_became_active":"2023-11-26T07:16:17.591285+0000","last_became_peered":"2023-11-26T07:16:17.591285+0000","last_unstale":"2023-11-26T07:16:20.171008+0000","last_undegraded":"2023-11-26T07:16:20.171008+0000","last_fullsized":"2023-11-26T07:16:20.171008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:56:50.275379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171534+0000","last_change":"2023-11-26T07:16:18.603137+0000","last_active":"2023-11-26T07:16:20.171534+0000","last_peered":"2023-11-26T07:16:20.171534+0000","last_clean":"2023-11-26T07:16:20.171534+0000","last_became_active":"2023-11-26T07:16:18.602867+0000","last_became_peered":"2023-11-26T07:16:18.602867+0000","last_unstale":"2023-11-26T07:16:20.171534+0000","last_undegraded":"2023-11-26T07:16:20.171534+0000","last_fullsized":"2023-11-26T07:16:20.171534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:20.475765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170948+0000","last_change":"2023-11-26T07:16:17.586972+0000","last_active":"2023-11-26T07:16:20.170948+0000","last_peered":"2023-11-26T07:16:20.170948+0000","last_clean":"2023-11-26T07:16:20.170948+0000","last_became_active":"2023-11-26T07:16:17.583592+0000","last_became_peered":"2023-11-26T07:16:17.583592+0000","last_unstale":"2023-11-26T07:16:20.170948+0000","last_undegraded":"2023-11-26T07:16:20.170948+0000","last_fullsized":"2023-11-26T07:16:20.170948+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:03:52.477073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607550+0000","last_change":"2023-11-26T07:16:18.607402+0000","last_active":"2023-11-26T07:16:19.607550+0000","last_peered":"2023-11-26T07:16:19.607550+0000","last_clean":"2023-11-26T07:16:19.607550+0000","last_became_active":"2023-11-26T07:16:18.607072+0000","last_became_peered":"2023-11-26T07:16:18.607072+0000","last_unstale":"2023-11-26T07:16:19.607550+0000","last_undegraded":"2023-11-26T07:16:19.607550+0000","last_fullsized":"2023-11-26T07:16:19.607550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:12:12.749341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606365+0000","last_change":"2023-11-26T07:16:17.589498+0000","last_active":"2023-11-26T07:16:19.606365+0000","last_peered":"2023-11-26T07:16:19.606365+0000","last_clean":"2023-11-26T07:16:19.606365+0000","last_became_active":"2023-11-26T07:16:17.588955+0000","last_became_peered":"2023-11-26T07:16:17.588955+0000","last_unstale":"2023-11-26T07:16:19.606365+0000","last_undegraded":"2023-11-26T07:16:19.606365+0000","last_fullsized":"2023-11-26T07:16:19.606365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:34.568596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170562+0000","last_change":"2023-11-26T07:16:18.607799+0000","last_active":"2023-11-26T07:16:20.170562+0000","last_peered":"2023-11-26T07:16:20.170562+0000","last_clean":"2023-11-26T07:16:20.170562+0000","last_became_active":"2023-11-26T07:16:18.606724+0000","last_became_peered":"2023-11-26T07:16:18.606724+0000","last_unstale":"2023-11-26T07:16:20.170562+0000","last_undegraded":"2023-11-26T07:16:20.170562+0000","last_fullsized":"2023-11-26T07:16:20.170562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:25:49.167252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170907+0000","last_change":"2023-11-26T07:16:17.590072+0000","last_active":"2023-11-26T07:16:20.170907+0000","last_peered":"2023-11-26T07:16:20.170907+0000","last_clean":"2023-11-26T07:16:20.170907+0000","last_became_active":"2023-11-26T07:16:17.589771+0000","last_became_peered":"2023-11-26T07:16:17.589771+0000","last_unstale":"2023-11-26T07:16:20.170907+0000","last_undegraded":"2023-11-26T07:16:20.170907+0000","last_fullsized":"2023-11-26T07:16:20.170907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:41.150316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170589+0000","last_change":"2023-11-26T07:16:18.601829+0000","last_active":"2023-11-26T07:16:20.170589+0000","last_peered":"2023-11-26T07:16:20.170589+0000","last_clean":"2023-11-26T07:16:20.170589+0000","last_became_active":"2023-11-26T07:16:18.601596+0000","last_became_peered":"2023-11-26T07:16:18.601596+0000","last_unstale":"2023-11-26T07:16:20.170589+0000","last_undegraded":"2023-11-26T07:16:20.170589+0000","last_fullsized":"2023-11-26T07:16:20.170589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:23.077029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.172356+0000","last_change":"2023-11-26T07:16:17.582108+0000","last_active":"2023-11-26T07:16:20.172356+0000","last_peered":"2023-11-26T07:16:20.172356+0000","last_clean":"2023-11-26T07:16:20.172356+0000","last_became_active":"2023-11-26T07:16:17.581840+0000","last_became_peered":"2023-11-26T07:16:17.581840+0000","last_unstale":"2023-11-26T07:16:20.172356+0000","last_undegraded":"2023-11-26T07:16:20.172356+0000","last_fullsized":"2023-11-26T07:16:20.172356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:44:49.695478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607565+0000","last_change":"2023-11-26T07:16:18.598723+0000","last_active":"2023-11-26T07:16:19.607565+0000","last_peered":"2023-11-26T07:16:19.607565+0000","last_clean":"2023-11-26T07:16:19.607565+0000","last_became_active":"2023-11-26T07:16:18.598441+0000","last_became_peered":"2023-11-26T07:16:18.598441+0000","last_unstale":"2023-11-26T07:16:19.607565+0000","last_undegraded":"2023-11-26T07:16:19.607565+0000","last_fullsized":"2023-11-26T07:16:19.607565+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:53.869386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171335+0000","last_change":"2023-11-26T07:16:17.593677+0000","last_active":"2023-11-26T07:16:20.171335+0000","last_peered":"2023-11-26T07:16:20.171335+0000","last_clean":"2023-11-26T07:16:20.171335+0000","last_became_active":"2023-11-26T07:16:17.593426+0000","last_became_peered":"2023-11-26T07:16:17.593426+0000","last_unstale":"2023-11-26T07:16:20.171335+0000","last_undegraded":"2023-11-26T07:16:20.171335+0000","last_fullsized":"2023-11-26T07:16:20.171335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:09:56.548526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171554+0000","last_change":"2023-11-26T07:16:18.605993+0000","last_active":"2023-11-26T07:16:20.171554+0000","last_peered":"2023-11-26T07:16:20.171554+0000","last_clean":"2023-11-26T07:16:20.171554+0000","last_became_active":"2023-11-26T07:16:18.605669+0000","last_became_peered":"2023-11-26T07:16:18.605669+0000","last_unstale":"2023-11-26T07:16:20.171554+0000","last_undegraded":"2023-11-26T07:16:20.171554+0000","last_fullsized":"2023-11-26T07:16:20.171554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:59:20.892686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171814+0000","last_change":"2023-11-26T07:16:17.584124+0000","last_active":"2023-11-26T07:16:20.171814+0000","last_peered":"2023-11-26T07:16:20.171814+0000","last_clean":"2023-11-26T07:16:20.171814+0000","last_became_active":"2023-11-26T07:16:17.583832+0000","last_became_peered":"2023-11-26T07:16:17.583832+0000","last_unstale":"2023-11-26T07:16:20.171814+0000","last_undegraded":"2023-11-26T07:16:20.171814+0000","last_fullsized":"2023-11-26T07:16:20.171814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:34.128300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608954+0000","last_change":"2023-11-26T07:16:18.613019+0000","last_active":"2023-11-26T07:16:19.608954+0000","last_peered":"2023-11-26T07:16:19.608954+0000","last_clean":"2023-11-26T07:16:19.608954+0000","last_became_active":"2023-11-26T07:16:18.612702+0000","last_became_peered":"2023-11-26T07:16:18.612702+0000","last_unstale":"2023-11-26T07:16:19.608954+0000","last_undegraded":"2023-11-26T07:16:19.608954+0000","last_fullsized":"2023-11-26T07:16:19.608954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:20.475765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171762+0000","last_change":"2023-11-26T07:16:17.590247+0000","last_active":"2023-11-26T07:16:20.171762+0000","last_peered":"2023-11-26T07:16:20.171762+0000","last_clean":"2023-11-26T07:16:20.171762+0000","last_became_active":"2023-11-26T07:16:17.589988+0000","last_became_peered":"2023-11-26T07:16:17.589988+0000","last_unstale":"2023-11-26T07:16:20.171762+0000","last_undegraded":"2023-11-26T07:16:20.171762+0000","last_fullsized":"2023-11-26T07:16:20.171762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:41:43.517205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171153+0000","last_change":"2023-11-26T07:16:18.610674+0000","last_active":"2023-11-26T07:16:20.171153+0000","last_peered":"2023-11-26T07:16:20.171153+0000","last_clean":"2023-11-26T07:16:20.171153+0000","last_became_active":"2023-11-26T07:16:18.610404+0000","last_became_peered":"2023-11-26T07:16:18.610404+0000","last_unstale":"2023-11-26T07:16:20.171153+0000","last_undegraded":"2023-11-26T07:16:20.171153+0000","last_fullsized":"2023-11-26T07:16:20.171153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:40:09.021844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606535+0000","last_change":"2023-11-26T07:16:17.585931+0000","last_active":"2023-11-26T07:16:19.606535+0000","last_peered":"2023-11-26T07:16:19.606535+0000","last_clean":"2023-11-26T07:16:19.606535+0000","last_became_active":"2023-11-26T07:16:17.585681+0000","last_became_peered":"2023-11-26T07:16:17.585681+0000","last_unstale":"2023-11-26T07:16:19.606535+0000","last_undegraded":"2023-11-26T07:16:19.606535+0000","last_fullsized":"2023-11-26T07:16:19.606535+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:49:22.020678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170989+0000","last_change":"2023-11-26T07:16:18.601095+0000","last_active":"2023-11-26T07:16:20.170989+0000","last_peered":"2023-11-26T07:16:20.170989+0000","last_clean":"2023-11-26T07:16:20.170989+0000","last_became_active":"2023-11-26T07:16:18.600757+0000","last_became_peered":"2023-11-26T07:16:18.600757+0000","last_unstale":"2023-11-26T07:16:20.170989+0000","last_undegraded":"2023-11-26T07:16:20.170989+0000","last_fullsized":"2023-11-26T07:16:20.170989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:07.718629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171326+0000","last_change":"2023-11-26T07:16:17.582741+0000","last_active":"2023-11-26T07:16:20.171326+0000","last_peered":"2023-11-26T07:16:20.171326+0000","last_clean":"2023-11-26T07:16:20.171326+0000","last_became_active":"2023-11-26T07:16:17.582472+0000","last_became_peered":"2023-11-26T07:16:17.582472+0000","last_unstale":"2023-11-26T07:16:20.171326+0000","last_undegraded":"2023-11-26T07:16:20.171326+0000","last_fullsized":"2023-11-26T07:16:20.171326+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:09:56.590410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171684+0000","last_change":"2023-11-26T07:16:18.603976+0000","last_active":"2023-11-26T07:16:20.171684+0000","last_peered":"2023-11-26T07:16:20.171684+0000","last_clean":"2023-11-26T07:16:20.171684+0000","last_became_active":"2023-11-26T07:16:18.603663+0000","last_became_peered":"2023-11-26T07:16:18.603663+0000","last_unstale":"2023-11-26T07:16:20.171684+0000","last_undegraded":"2023-11-26T07:16:20.171684+0000","last_fullsized":"2023-11-26T07:16:20.171684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:16.330339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171055+0000","last_change":"2023-11-26T07:16:17.588408+0000","last_active":"2023-11-26T07:16:20.171055+0000","last_peered":"2023-11-26T07:16:20.171055+0000","last_clean":"2023-11-26T07:16:20.171055+0000","last_became_active":"2023-11-26T07:16:17.588080+0000","last_became_peered":"2023-11-26T07:16:17.588080+0000","last_unstale":"2023-11-26T07:16:20.171055+0000","last_undegraded":"2023-11-26T07:16:20.171055+0000","last_fullsized":"2023-11-26T07:16:20.171055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:08:52.679738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170959+0000","last_change":"2023-11-26T07:16:18.598247+0000","last_active":"2023-11-26T07:16:20.170959+0000","last_peered":"2023-11-26T07:16:20.170959+0000","last_clean":"2023-11-26T07:16:20.170959+0000","last_became_active":"2023-11-26T07:16:18.597923+0000","last_became_peered":"2023-11-26T07:16:18.597923+0000","last_unstale":"2023-11-26T07:16:20.170959+0000","last_undegraded":"2023-11-26T07:16:20.170959+0000","last_fullsized":"2023-11-26T07:16:20.170959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:31:40.918037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.606244+0000","last_change":"2023-11-26T07:16:17.585908+0000","last_active":"2023-11-26T07:16:19.606244+0000","last_peered":"2023-11-26T07:16:19.606244+0000","last_clean":"2023-11-26T07:16:19.606244+0000","last_became_active":"2023-11-26T07:16:17.585624+0000","last_became_peered":"2023-11-26T07:16:17.585624+0000","last_unstale":"2023-11-26T07:16:19.606244+0000","last_undegraded":"2023-11-26T07:16:19.606244+0000","last_fullsized":"2023-11-26T07:16:19.606244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:40.325385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.609084+0000","last_change":"2023-11-26T07:16:18.609471+0000","last_active":"2023-11-26T07:16:19.609084+0000","last_peered":"2023-11-26T07:16:19.609084+0000","last_clean":"2023-11-26T07:16:19.609084+0000","last_became_active":"2023-11-26T07:16:18.609179+0000","last_became_peered":"2023-11-26T07:16:18.609179+0000","last_unstale":"2023-11-26T07:16:19.609084+0000","last_undegraded":"2023-11-26T07:16:19.609084+0000","last_fullsized":"2023-11-26T07:16:19.609084+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:28:14.679869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170843+0000","last_change":"2023-11-26T07:16:17.581389+0000","last_active":"2023-11-26T07:16:20.170843+0000","last_peered":"2023-11-26T07:16:20.170843+0000","last_clean":"2023-11-26T07:16:20.170843+0000","last_became_active":"2023-11-26T07:16:17.581119+0000","last_became_peered":"2023-11-26T07:16:17.581119+0000","last_unstale":"2023-11-26T07:16:20.170843+0000","last_undegraded":"2023-11-26T07:16:20.170843+0000","last_fullsized":"2023-11-26T07:16:20.170843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:22.835744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607981+0000","last_change":"2023-11-26T07:16:18.605266+0000","last_active":"2023-11-26T07:16:19.607981+0000","last_peered":"2023-11-26T07:16:19.607981+0000","last_clean":"2023-11-26T07:16:19.607981+0000","last_became_active":"2023-11-26T07:16:18.604983+0000","last_became_peered":"2023-11-26T07:16:18.604983+0000","last_unstale":"2023-11-26T07:16:19.607981+0000","last_undegraded":"2023-11-26T07:16:19.607981+0000","last_fullsized":"2023-11-26T07:16:19.607981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:52:47.905499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171901+0000","last_change":"2023-11-26T07:16:17.582872+0000","last_active":"2023-11-26T07:16:20.171901+0000","last_peered":"2023-11-26T07:16:20.171901+0000","last_clean":"2023-11-26T07:16:20.171901+0000","last_became_active":"2023-11-26T07:16:17.582623+0000","last_became_peered":"2023-11-26T07:16:17.582623+0000","last_unstale":"2023-11-26T07:16:20.171901+0000","last_undegraded":"2023-11-26T07:16:20.171901+0000","last_fullsized":"2023-11-26T07:16:20.171901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:26:20.166154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170587+0000","last_change":"2023-11-26T07:16:18.608196+0000","last_active":"2023-11-26T07:16:20.170587+0000","last_peered":"2023-11-26T07:16:20.170587+0000","last_clean":"2023-11-26T07:16:20.170587+0000","last_became_active":"2023-11-26T07:16:18.607929+0000","last_became_peered":"2023-11-26T07:16:18.607929+0000","last_unstale":"2023-11-26T07:16:20.170587+0000","last_undegraded":"2023-11-26T07:16:20.170587+0000","last_fullsized":"2023-11-26T07:16:20.170587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:20.475765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171030+0000","last_change":"2023-11-26T07:16:17.587193+0000","last_active":"2023-11-26T07:16:20.171030+0000","last_peered":"2023-11-26T07:16:20.171030+0000","last_clean":"2023-11-26T07:16:20.171030+0000","last_became_active":"2023-11-26T07:16:17.586894+0000","last_became_peered":"2023-11-26T07:16:17.586894+0000","last_unstale":"2023-11-26T07:16:20.171030+0000","last_undegraded":"2023-11-26T07:16:20.171030+0000","last_fullsized":"2023-11-26T07:16:20.171030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:44:06.246834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.609102+0000","last_change":"2023-11-26T07:16:18.610127+0000","last_active":"2023-11-26T07:16:19.609102+0000","last_peered":"2023-11-26T07:16:19.609102+0000","last_clean":"2023-11-26T07:16:19.609102+0000","last_became_active":"2023-11-26T07:16:18.609428+0000","last_became_peered":"2023-11-26T07:16:18.609428+0000","last_unstale":"2023-11-26T07:16:19.609102+0000","last_undegraded":"2023-11-26T07:16:19.609102+0000","last_fullsized":"2023-11-26T07:16:19.609102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:04:47.701141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171070+0000","last_change":"2023-11-26T07:16:17.588399+0000","last_active":"2023-11-26T07:16:20.171070+0000","last_peered":"2023-11-26T07:16:20.171070+0000","last_clean":"2023-11-26T07:16:20.171070+0000","last_became_active":"2023-11-26T07:16:17.588135+0000","last_became_peered":"2023-11-26T07:16:17.588135+0000","last_unstale":"2023-11-26T07:16:20.171070+0000","last_undegraded":"2023-11-26T07:16:20.171070+0000","last_fullsized":"2023-11-26T07:16:20.171070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:34.128300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170831+0000","last_change":"2023-11-26T07:16:18.597411+0000","last_active":"2023-11-26T07:16:20.170831+0000","last_peered":"2023-11-26T07:16:20.170831+0000","last_clean":"2023-11-26T07:16:20.170831+0000","last_became_active":"2023-11-26T07:16:18.596501+0000","last_became_peered":"2023-11-26T07:16:18.596501+0000","last_unstale":"2023-11-26T07:16:20.170831+0000","last_undegraded":"2023-11-26T07:16:20.170831+0000","last_fullsized":"2023-11-26T07:16:20.170831+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:41:44.521573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171580+0000","last_change":"2023-11-26T07:16:17.582720+0000","last_active":"2023-11-26T07:16:20.171580+0000","last_peered":"2023-11-26T07:16:20.171580+0000","last_clean":"2023-11-26T07:16:20.171580+0000","last_became_active":"2023-11-26T07:16:17.582268+0000","last_became_peered":"2023-11-26T07:16:17.582268+0000","last_unstale":"2023-11-26T07:16:20.171580+0000","last_undegraded":"2023-11-26T07:16:20.171580+0000","last_fullsized":"2023-11-26T07:16:20.171580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:09:56.548526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171974+0000","last_change":"2023-11-26T07:16:18.603811+0000","last_active":"2023-11-26T07:16:20.171974+0000","last_peered":"2023-11-26T07:16:20.171974+0000","last_clean":"2023-11-26T07:16:20.171974+0000","last_became_active":"2023-11-26T07:16:18.603540+0000","last_became_peered":"2023-11-26T07:16:18.603540+0000","last_unstale":"2023-11-26T07:16:20.171974+0000","last_undegraded":"2023-11-26T07:16:20.171974+0000","last_fullsized":"2023-11-26T07:16:20.171974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:28:14.679869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608429+0000","last_change":"2023-11-26T07:16:17.586817+0000","last_active":"2023-11-26T07:16:19.608429+0000","last_peered":"2023-11-26T07:16:19.608429+0000","last_clean":"2023-11-26T07:16:19.608429+0000","last_became_active":"2023-11-26T07:16:17.586505+0000","last_became_peered":"2023-11-26T07:16:17.586505+0000","last_unstale":"2023-11-26T07:16:19.608429+0000","last_undegraded":"2023-11-26T07:16:19.608429+0000","last_fullsized":"2023-11-26T07:16:19.608429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:09:56.548526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607332+0000","last_change":"2023-11-26T07:16:18.599629+0000","last_active":"2023-11-26T07:16:19.607332+0000","last_peered":"2023-11-26T07:16:19.607332+0000","last_clean":"2023-11-26T07:16:19.607332+0000","last_became_active":"2023-11-26T07:16:18.599367+0000","last_became_peered":"2023-11-26T07:16:18.599367+0000","last_unstale":"2023-11-26T07:16:19.607332+0000","last_undegraded":"2023-11-26T07:16:19.607332+0000","last_fullsized":"2023-11-26T07:16:19.607332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:47.337440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170516+0000","last_change":"2023-11-26T07:16:17.589253+0000","last_active":"2023-11-26T07:16:20.170516+0000","last_peered":"2023-11-26T07:16:20.170516+0000","last_clean":"2023-11-26T07:16:20.170516+0000","last_became_active":"2023-11-26T07:16:17.588770+0000","last_became_peered":"2023-11-26T07:16:17.588770+0000","last_unstale":"2023-11-26T07:16:20.170516+0000","last_undegraded":"2023-11-26T07:16:20.170516+0000","last_fullsized":"2023-11-26T07:16:20.170516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:50:26.219042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.170589+0000","last_change":"2023-11-26T07:16:18.605132+0000","last_active":"2023-11-26T07:16:20.170589+0000","last_peered":"2023-11-26T07:16:20.170589+0000","last_clean":"2023-11-26T07:16:20.170589+0000","last_became_active":"2023-11-26T07:16:18.604881+0000","last_became_peered":"2023-11-26T07:16:18.604881+0000","last_unstale":"2023-11-26T07:16:20.170589+0000","last_undegraded":"2023-11-26T07:16:20.170589+0000","last_fullsized":"2023-11-26T07:16:20.170589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:07.718629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.608310+0000","last_change":"2023-11-26T07:16:17.585783+0000","last_active":"2023-11-26T07:16:19.608310+0000","last_peered":"2023-11-26T07:16:19.608310+0000","last_clean":"2023-11-26T07:16:19.608310+0000","last_became_active":"2023-11-26T07:16:17.585518+0000","last_became_peered":"2023-11-26T07:16:17.585518+0000","last_unstale":"2023-11-26T07:16:19.608310+0000","last_undegraded":"2023-11-26T07:16:19.608310+0000","last_fullsized":"2023-11-26T07:16:19.608310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:44:49.695478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:20.171085+0000","last_change":"2023-11-26T07:16:18.606707+0000","last_active":"2023-11-26T07:16:20.171085+0000","last_peered":"2023-11-26T07:16:20.171085+0000","last_clean":"2023-11-26T07:16:20.171085+0000","last_became_active":"2023-11-26T07:16:18.606449+0000","last_became_peered":"2023-11-26T07:16:18.606449+0000","last_unstale":"2023-11-26T07:16:20.171085+0000","last_undegraded":"2023-11-26T07:16:20.171085+0000","last_fullsized":"2023-11-26T07:16:20.171085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:17.566986+0000","last_clean_scrub_stamp":"2023-11-26T07:16:17.566986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:46:23.472050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-26T07:16:19.607780+0000","last_change":"2023-11-26T07:16:17.581842+0000","last_active":"2023-11-26T07:16:19.607780+0000","last_peered":"2023-11-26T07:16:19.607780+0000","last_clean":"2023-11-26T07:16:19.607780+0000","last_became_active":"2023-11-26T07:16:17.581522+0000","last_became_peered":"2023-11-26T07:16:17.581522+0000","last_unstale":"2023-11-26T07:16:19.607780+0000","last_undegraded":"2023-11-26T07:16:19.607780+0000","last_fullsized":"2023-11-26T07:16:19.607780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:16:16.562618+0000","last_clean_scrub_stamp":"2023-11-26T07:16:16.562618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:04:46.696773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":268,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-26T07:16:18.588211+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:16:18.588211+0000","last_peered":"2023-11-26T07:16:18.588211+0000","last_clean":"2023-11-26T07:16:18.588211+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:16:18.588211+0000","last_undegraded":"2023-11-26T07:16:18.588211+0000","last_fullsized":"2023-11-26T07:16:18.588211+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28600,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343240,"statfs":{"total":96636764160,"available":96607477760,"internally_reserved":0,"allocated":1617920,"data_stored":1028668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.56200000000000006},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74299999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65600000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58699999999999997},"min":{"1min":0.40799999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88100000000000001,"5min":1.048,"15min":1.048},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.315,"15min":0.315},"max":{"1min":0.71599999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.55000000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82499999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.622},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79700000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.624}]},{"osd":3,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.46400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.876,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.51800000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.49199999999999999},"min":{"1min":0.34200000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.45500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.52800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":1.171,"15min":1.171},"last":0.504}]},{"osd":6,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.39500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.93899999999999995,"5min":1.361,"15min":1.361},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0409999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":438924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75900000000000001,"5min":1.159,"15min":1.159},"last":0.747},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55900000000000005,"15min":0.57699999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73699999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.67700000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80000000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56200000000000006,"15min":0.56299999999999994},"min":{"1min":0.46700000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71799999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80800000000000005,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72299999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56000000000000005,"15min":0.53700000000000003},"min":{"1min":0.433,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77600000000000002,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.70299999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.70999999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.5,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.747,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.33900000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71299999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50700000000000001},"min":{"1min":0.34399999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72099999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.625}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.439,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.747,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":438424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.33300000000000002},"min":{"1min":0.17899999999999999,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.20499999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.51000000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.33500000000000002},"min":{"1min":0.24199999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53300000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.58299999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.28799999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.27200000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.57099999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.38800000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83399999999999996,"5min":0.877,"15min":0.877},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86499999999999999,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55600000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79400000000000004,"5min":0.995,"15min":0.995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.62,"15min":0.629},"min":{"1min":0.45600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61699999999999999,"15min":0.62},"min":{"1min":0.42399999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83099999999999996,"5min":1.129,"15min":1.129},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.42599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83799999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.626},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91400000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.879,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28600,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343240,"statfs":{"total":96636764160,"available":96607477760,"internally_reserved":0,"allocated":1617920,"data_stored":1028632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26900000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.219},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.379,"15min":0.37},"min":{"1min":0.245,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.309}]},{"osd":2,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.29499999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.64300000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.249},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36299999999999999,"15min":0.34899999999999998},"min":{"1min":0.23200000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.34599999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40000000000000002,"15min":0.40300000000000002},"min":{"1min":0.29799999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.438,"15min":0.44},"min":{"1min":0.29799999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60699999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.33500000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77800000000000002,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.621,"15min":0.63500000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53600000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89000000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73699999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72999999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:15:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81899999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.65000000000000002,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.60799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":438812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.36099999999999999,"15min":0.36899999999999999},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.61199999999999999,"5min":0.627,"15min":0.627},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.376},"min":{"1min":0.26400000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.51100000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.35699999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.39000000000000001},"min":{"1min":0.26600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.53400000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.379,"15min":0.36799999999999999},"min":{"1min":0.28799999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38900000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"min":{"1min":0.28599999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.58799999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41799999999999998,"15min":0.42599999999999999},"min":{"1min":0.29599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68100000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.377}]},{"osd":4,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.67100000000000004},"min":{"1min":0.41499999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81299999999999994,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.39200000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55800000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.622,"15min":0.624},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90000000000000002,"5min":1.075,"15min":1.075},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61699999999999999,"15min":0.61799999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0309999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.66600000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.747},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.51300000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.004,"5min":1.03,"15min":1.03},"last":0.56799999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64700000000000002,"15min":0.65900000000000003},"min":{"1min":0.504,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94899999999999995,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.502,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96199999999999997,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.57799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":439842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35399999999999998,"15min":0.35999999999999999},"min":{"1min":0.23999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.629,"5min":1.284,"15min":1.284},"last":0.312},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.42799999999999999,"15min":0.47299999999999998},"min":{"1min":0.26800000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63600000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.26000000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.437,"15min":0.51500000000000001},"min":{"1min":0.25,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.58799999999999997,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.44700000000000001,"15min":0.49199999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.69399999999999995,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.35099999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46800000000000003,"15min":0.502},"min":{"1min":0.27900000000000003,"5min":0.247,"15min":0.247},"max":{"1min":0.61199999999999999,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41199999999999998,"15min":0.42699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.59799999999999998,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.37}]},{"osd":4,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.64600000000000002,"15min":0.67400000000000004},"min":{"1min":0.41499999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79000000000000004,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.63800000000000001,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81899999999999995,"5min":1.405,"15min":1.405},"last":0.45000000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.625,"15min":0.63700000000000001},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78100000000000003,"5min":1.377,"15min":1.377},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.71499999999999997,"15min":0.79500000000000004},"min":{"1min":0.42399999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89200000000000002,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.47099999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.67100000000000004},"min":{"1min":0.50600000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79800000000000004,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.71599999999999997,"15min":0.78900000000000003},"min":{"1min":0.434,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82599999999999996,"5min":4.883,"15min":4.883},"last":0.50700000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:15:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.80500000000000005},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.68899999999999995,"15min":0.73299999999999998},"min":{"1min":0.437,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.876,"5min":2.923,"15min":2.923},"last":0.46100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":438230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.60999999999999999},"min":{"1min":0.34300000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.79400000000000004,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68500000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.69499999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.375,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.65700000000000003},"min":{"1min":0.376,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91100000000000003,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.78600000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56000000000000005,"15min":0.53100000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.67300000000000004},"min":{"1min":0.438,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76000000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56200000000000006,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53600000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58299999999999996,"15min":0.60299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73899999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:15:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.58899999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.63600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.51000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":439864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53000000000000003,"15min":0.50800000000000001},"min":{"1min":0.29799999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.54300000000000004},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.76700000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.71199999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.65400000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.55400000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.749}]},{"osd":3,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.39300000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.627},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.751,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71999999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.35599999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.67300000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84699999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69099999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.78400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:16:23.093 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:16:23.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:16:23.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.255+0000 7ff16b81b700 1 Processor -- start 2023-11-26T07:16:23.257 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.255+0000 7ff16b81b700 1 -- start start 2023-11-26T07:16:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff16b81b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff16415e160 0x7ff1640b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff16b81b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff1640746a0 0x7ff16406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff16b81b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1640b3c60 con 0x7ff1640afc90 2023-11-26T07:16:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff16b81b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1640b4500 con 0x7ff1640746a0 2023-11-26T07:16:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff16b81b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1640b58c0 con 0x7ff16415e160 2023-11-26T07:16:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff169db8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff1640746a0 0x7ff16406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.259 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff169db8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff1640746a0 0x7ff16406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40646/0 (socket says 172.21.15.83:40646) 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff169db8700 1 -- 172.21.15.83:0/518115371 learned_addr learned my addr 172.21.15.83:0/518115371 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff1695b7700 1 --2- 172.21.15.83:0/518115371 >> v2:172.21.15.203:3300/0 conn(0x7ff16415e160 0x7ff1640b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff169db8700 1 -- 172.21.15.83:0/518115371 >> v2:172.21.15.203:3300/0 conn(0x7ff16415e160 msgr2=0x7ff1640b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff169db8700 1 --2- 172.21.15.83:0/518115371 >> v2:172.21.15.203:3300/0 conn(0x7ff16415e160 0x7ff1640b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff169db8700 1 -- 172.21.15.83:0/518115371 >> v1:172.21.15.203:6789/0 conn(0x7ff1640afc90 legacy=0x7ff164073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff169db8700 1 -- 172.21.15.83:0/518115371 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1640b0140 con 0x7ff1640746a0 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.256+0000 7ff1695b7700 1 --2- 172.21.15.83:0/518115371 >> v2:172.21.15.203:3300/0 conn(0x7ff16415e160 0x7ff1640b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff169db8700 1 --2- 172.21.15.83:0/518115371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1640746a0 0x7ff16406d930 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff16409efa0 tx=0x7ff15c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70018439274eaa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff15bfff700 1 -- 172.21.15.83:0/518115371 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00e070 con 0x7ff1640746a0 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff15bfff700 1 -- 172.21.15.83:0/518115371 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c003a20 con 0x7ff1640746a0 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff15bfff700 1 -- 172.21.15.83:0/518115371 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c003dd0 con 0x7ff1640746a0 2023-11-26T07:16:23.260 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff16b81b700 1 -- 172.21.15.83:0/518115371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1640746a0 msgr2=0x7ff16406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff16b81b700 1 --2- 172.21.15.83:0/518115371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1640746a0 0x7ff16406d930 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff16409efa0 tx=0x7ff15c00b330 comp rx=0 tx=0).stop 2023-11-26T07:16:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff16b81b700 1 -- 172.21.15.83:0/518115371 shutdown_connections 2023-11-26T07:16:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff16b81b700 1 --2- 172.21.15.83:0/518115371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1640746a0 0x7ff16406d930 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff16b81b700 1 --2- 172.21.15.83:0/518115371 >> v2:172.21.15.203:3300/0 conn(0x7ff16415e160 0x7ff1640b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff16b81b700 1 -- 172.21.15.83:0/518115371 >> 172.21.15.83:0/518115371 conn(0x7ff1640d0bf0 msgr2=0x7ff1640d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff16b81b700 1 -- 172.21.15.83:0/518115371 shutdown_connections 2023-11-26T07:16:23.261 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.257+0000 7ff16b81b700 1 -- 172.21.15.83:0/518115371 wait complete. 2023-11-26T07:16:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.259+0000 7ff16b81b700 1 Processor -- start 2023-11-26T07:16:23.263 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.259+0000 7ff16b81b700 1 -- start start 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.259+0000 7ff16b81b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1640746a0 0x7ff16415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.259+0000 7ff16b81b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1640afc90 0x7ff1641557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.259+0000 7ff16b81b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff16415e160 0x7ff1641f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.259+0000 7ff16b81b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff164155d00 con 0x7ff1640746a0 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.259+0000 7ff16b81b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff164155f00 con 0x7ff16415e160 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.259+0000 7ff16b81b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1641f46e0 con 0x7ff1640afc90 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.259+0000 7ff1695b7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1640746a0 0x7ff16415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.259+0000 7ff1695b7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1640746a0 0x7ff16415e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40648/0 (socket says 172.21.15.83:40648) 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.259+0000 7ff1695b7700 1 -- 172.21.15.83:0/2706215231 learned_addr learned my addr 172.21.15.83:0/2706215231 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff169db8700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff16415e160 0x7ff1641f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff168db6700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1640afc90 0x7ff1641557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff1695b7700 1 -- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1640afc90 msgr2=0x7ff1641557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff1695b7700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1640afc90 0x7ff1641557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff1695b7700 1 -- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff16415e160 msgr2=0x7ff1641f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff1695b7700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff16415e160 0x7ff1641f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff1695b7700 1 -- 172.21.15.83:0/2706215231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c009b30 con 0x7ff1640746a0 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff1695b7700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1640746a0 0x7ff16415e750 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff15400c7f0 tx=0x7ff15400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff159ffb700 1 -- 172.21.15.83:0/2706215231 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154011800 con 0x7ff1640746a0 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff159ffb700 1 -- 172.21.15.83:0/2706215231 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff154011960 con 0x7ff1640746a0 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff16b81b700 1 -- 172.21.15.83:0/2706215231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1641f49c0 con 0x7ff1640746a0 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff159ffb700 1 -- 172.21.15.83:0/2706215231 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154011d10 con 0x7ff1640746a0 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.260+0000 7ff16b81b700 1 -- 172.21.15.83:0/2706215231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1641561c0 con 0x7ff1640746a0 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.261+0000 7ff16b81b700 1 -- 172.21.15.83:0/2706215231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff164068fb0 con 0x7ff1640746a0 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.262+0000 7ff159ffb700 1 -- 172.21.15.83:0/2706215231 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7ff154018360 con 0x7ff1640746a0 2023-11-26T07:16:23.265 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.262+0000 7ff159ffb700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff14806b020 0x7ff14806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.262+0000 7ff159ffb700 1 -- 172.21.15.83:0/2706215231 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff154089810 con 0x7ff1640746a0 2023-11-26T07:16:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.262+0000 7ff168db6700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff14806b020 0x7ff14806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.263+0000 7ff168db6700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff14806b020 0x7ff14806d4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff164159430 tx=0x7ff160009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:23.268 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.266+0000 7ff159ffb700 1 -- 172.21.15.83:0/2706215231 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff1540546a0 con 0x7ff1640746a0 2023-11-26T07:16:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.539+0000 7ff16b81b700 1 -- 172.21.15.83:0/2706215231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff1641563e0 con 0x7ff1640746a0 2023-11-26T07:16:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.541+0000 7ff159ffb700 1 -- 172.21.15.83:0/2706215231 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff15401f0e0 con 0x7ff1640746a0 2023-11-26T07:16:23.542 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2023-11-26T07:16:23.542 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ== 2023-11-26T07:16:23.543 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2023-11-26T07:16:23.543 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2023-11-26T07:16:23.543 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:16:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 -- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff14806b020 msgr2=0x7ff14806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff14806b020 0x7ff14806d4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff164159430 tx=0x7ff160009040 comp rx=0 tx=0).stop 2023-11-26T07:16:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 -- 172.21.15.83:0/2706215231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1640746a0 msgr2=0x7ff16415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1640746a0 0x7ff16415e750 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff15400c7f0 tx=0x7ff15400cbb0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 -- 172.21.15.83:0/2706215231 shutdown_connections 2023-11-26T07:16:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff14806b020 0x7ff14806d4e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff16415e160 0x7ff1641f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1640746a0 0x7ff16415e750 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 --2- 172.21.15.83:0/2706215231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1640afc90 0x7ff1641557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 -- 172.21.15.83:0/2706215231 >> 172.21.15.83:0/2706215231 conn(0x7ff1640d0bf0 msgr2=0x7ff16415d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 -- 172.21.15.83:0/2706215231 shutdown_connections 2023-11-26T07:16:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.543+0000 7ff16b81b700 1 -- 172.21.15.83:0/2706215231 wait complete. 2023-11-26T07:16:23.546 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2023-11-26T07:16:23.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:23.567 INFO:tasks.ceph.mds.b.smithi203.stderr:2023-11-26T07:16:23.565+0000 7f2153425700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:16:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.676+0000 7fb64de50700 1 Processor -- start 2023-11-26T07:16:23.678 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.676+0000 7fb64de50700 1 -- start start 2023-11-26T07:16:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.677+0000 7fb64de50700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6480af4d0 0x7fb6480b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.677+0000 7fb64de50700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6480ae430 con 0x7fb64815e080 2023-11-26T07:16:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.677+0000 7fb64de50700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6480ae710 con 0x7fb64806c710 2023-11-26T07:16:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.677+0000 7fb64de50700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6480af0b0 con 0x7fb6480af4d0 2023-11-26T07:16:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.677+0000 7fb6477fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb64815e080 0x7fb64806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60696/0 (socket says 172.21.15.83:60696) 2023-11-26T07:16:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.677+0000 7fb6477fe700 1 -- 172.21.15.83:0/891713873 learned_addr learned my addr 172.21.15.83:0/891713873 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb647fff700 1 --2- 172.21.15.83:0/891713873 >> v2:172.21.15.203:3300/0 conn(0x7fb6480af4d0 0x7fb6480b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558009592 0 0) 0x7fb6480ae430 con 0x7fb64815e080 2023-11-26T07:16:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6340035b0 con 0x7fb64815e080 2023-11-26T07:16:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 677695961 0 0) 0x7fb6340035b0 con 0x7fb64815e080 2023-11-26T07:16:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 >> v2:172.21.15.203:3300/0 conn(0x7fb6480af4d0 msgr2=0x7fb6480b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 --2- 172.21.15.83:0/891713873 >> v2:172.21.15.203:3300/0 conn(0x7fb6480af4d0 0x7fb6480b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 >> v1:172.21.15.203:6789/0 conn(0x7fb64806c710 legacy=0x7fb64806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6480ae9f0 con 0x7fb64815e080 2023-11-26T07:16:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fb638001670 con 0x7fb64815e080 2023-11-26T07:16:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 >> v1:172.21.15.83:6789/0 conn(0x7fb64815e080 legacy=0x7fb64806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 --2- 172.21.15.83:0/891713873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb634003a00 0x7fb634003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 --2- 172.21.15.83:0/891713873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb634004850 0x7fb634004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 --2- 172.21.15.83:0/891713873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 0x7fb63400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480af0b0 con 0x7fb634005210 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63400c0b0 con 0x7fb634003a00 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63400c270 con 0x7fb634004850 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb6477fe700 1 --2- 172.21.15.83:0/891713873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb634003a00 0x7fb634003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb647fff700 1 --2- 172.21.15.83:0/891713873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 0x7fb63400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb63ffff700 1 --2- 172.21.15.83:0/891713873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb634004850 0x7fb634004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb647fff700 1 -- 172.21.15.83:0/891713873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb634004850 msgr2=0x7fb634004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb647fff700 1 --2- 172.21.15.83:0/891713873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb634004850 0x7fb634004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb647fff700 1 -- 172.21.15.83:0/891713873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb634003a00 msgr2=0x7fb634003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb647fff700 1 --2- 172.21.15.83:0/891713873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb634003a00 0x7fb634003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb647fff700 1 -- 172.21.15.83:0/891713873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6340035b0 con 0x7fb634005210 2023-11-26T07:16:23.683 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.678+0000 7fb647fff700 1 -- 172.21.15.83:0/891713873 reap_dead start 2023-11-26T07:16:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.679+0000 7fb647fff700 1 --2- 172.21.15.83:0/891713873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 0x7fb63400ba10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb640009fc0 tx=0x7fb634005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f58177d237d6b2d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.679+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb640003a20 con 0x7fb634005210 2023-11-26T07:16:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.679+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64000f040 con 0x7fb634005210 2023-11-26T07:16:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.679+0000 7fb646ffd700 1 -- 172.21.15.83:0/891713873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb640003de0 con 0x7fb634005210 2023-11-26T07:16:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.679+0000 7fb64de50700 1 -- 172.21.15.83:0/891713873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 msgr2=0x7fb63400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.679+0000 7fb64de50700 1 --2- 172.21.15.83:0/891713873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 0x7fb63400ba10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb640009fc0 tx=0x7fb634005880 comp rx=0 tx=0).stop 2023-11-26T07:16:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.679+0000 7fb64de50700 1 -- 172.21.15.83:0/891713873 shutdown_connections 2023-11-26T07:16:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.679+0000 7fb64de50700 1 --2- 172.21.15.83:0/891713873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 0x7fb63400ba10 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.679+0000 7fb64de50700 1 -- 172.21.15.83:0/891713873 >> 172.21.15.83:0/891713873 conn(0x7fb648071b90 msgr2=0x7fb648071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.679+0000 7fb64de50700 1 -- 172.21.15.83:0/891713873 shutdown_connections 2023-11-26T07:16:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.679+0000 7fb64de50700 1 -- 172.21.15.83:0/891713873 wait complete. 2023-11-26T07:16:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.681+0000 7fb64de50700 1 Processor -- start 2023-11-26T07:16:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.681+0000 7fb64de50700 1 -- start start 2023-11-26T07:16:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.681+0000 7fb64de50700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 0x7fb6481f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.681+0000 7fb64de50700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb64815e460 0x7fb64815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.681+0000 7fb64de50700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6481557b0 0x7fb648155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.681+0000 7fb64de50700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6481561a0 con 0x7fb634005210 2023-11-26T07:16:23.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.681+0000 7fb64de50700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb648153080 con 0x7fb6481557b0 2023-11-26T07:16:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.681+0000 7fb64de50700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64815f550 con 0x7fb64815e460 2023-11-26T07:16:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.681+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 0x7fb6481f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.681+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 0x7fb6481f2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40668/0 (socket says 172.21.15.83:40668) 2023-11-26T07:16:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.681+0000 7fb6477fe700 1 -- 172.21.15.83:0/3823195224 learned_addr learned my addr 172.21.15.83:0/3823195224 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb647fff700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6481557b0 0x7fb648155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb63ffff700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb64815e460 0x7fb64815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb6477fe700 1 -- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb64815e460 msgr2=0x7fb64815e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb6477fe700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb64815e460 0x7fb64815e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb6477fe700 1 -- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6481557b0 msgr2=0x7fb648155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb6477fe700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6481557b0 0x7fb648155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb6477fe700 1 -- 172.21.15.83:0/3823195224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64815ef90 con 0x7fb634005210 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb6477fe700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 0x7fb6481f2e10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb638007290 tx=0x7fb6380075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb644ff9700 1 -- 172.21.15.83:0/3823195224 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638011580 con 0x7fb634005210 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb644ff9700 1 -- 172.21.15.83:0/3823195224 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6380116e0 con 0x7fb634005210 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb64de50700 1 -- 172.21.15.83:0/3823195224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64815f270 con 0x7fb634005210 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb644ff9700 1 -- 172.21.15.83:0/3823195224 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638011a90 con 0x7fb634005210 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.682+0000 7fb64de50700 1 -- 172.21.15.83:0/3823195224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6480cda80 con 0x7fb634005210 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.683+0000 7fb64de50700 1 -- 172.21.15.83:0/3823195224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6480cdc40 con 0x7fb634005210 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.683+0000 7fb644ff9700 1 -- 172.21.15.83:0/3823195224 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fb638011bf0 con 0x7fb634005210 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.684+0000 7fb644ff9700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb62806aff0 0x7fb62806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.684+0000 7fb644ff9700 1 -- 172.21.15.83:0/3823195224 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6380896e0 con 0x7fb634005210 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.684+0000 7fb63ffff700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb62806aff0 0x7fb62806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:23.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.684+0000 7fb63ffff700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb62806aff0 0x7fb62806d4b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb64815a410 tx=0x7fb630009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:23.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.688+0000 7fb644ff9700 1 -- 172.21.15.83:0/3823195224 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb638054680 con 0x7fb634005210 2023-11-26T07:16:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.919+0000 7fb64de50700 1 -- 172.21.15.83:0/3823195224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6480753f0 con 0x7fb634005210 2023-11-26T07:16:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.920+0000 7fb644ff9700 1 -- 172.21.15.83:0/3823195224 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fb638057ca0 con 0x7fb634005210 2023-11-26T07:16:23.921 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:23.921 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1015690400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1015690400},{"type":"v1","addr":"172.21.15.203:6834","nonce":1015690400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2716428832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2716428832},{"type":"v1","addr":"172.21.15.83:6834","nonce":2716428832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/469139444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":469139444},{"type":"v1","addr":"172.21.15.83:6835","nonce":469139444}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:16:17.967531+0000","modified":"2023-11-26T07:16:21.013928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1901175330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1901175330},{"type":"v1","addr":"172.21.15.203:6835","nonce":1901175330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:16:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 -- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb62806aff0 msgr2=0x7fb62806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb62806aff0 0x7fb62806d4b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb64815a410 tx=0x7fb630009040 comp rx=0 tx=0).stop 2023-11-26T07:16:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 -- 172.21.15.83:0/3823195224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 msgr2=0x7fb6481f2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 0x7fb6481f2e10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb638007290 tx=0x7fb6380075a0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 -- 172.21.15.83:0/3823195224 shutdown_connections 2023-11-26T07:16:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb62806aff0 0x7fb62806d4b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6481557b0 0x7fb648155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb634005210 0x7fb6481f2e10 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 --2- 172.21.15.83:0/3823195224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb64815e460 0x7fb64815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 -- 172.21.15.83:0/3823195224 >> 172.21.15.83:0/3823195224 conn(0x7fb648071b90 msgr2=0x7fb6480b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 -- 172.21.15.83:0/3823195224 shutdown_connections 2023-11-26T07:16:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:23.922+0000 7fb64de50700 1 -- 172.21.15.83:0/3823195224 wait complete. 2023-11-26T07:16:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 55 2023-11-26T07:16:24.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:16:17.967531+0000', 'modified': '2023-11-26T07:16:21.013928+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7128}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7128': {'gid': 7128, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.203:6835/1901175330', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6833', 'nonce': 1901175330}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 1901175330}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:16:24.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:16:24.338 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-26T07:16:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.459+0000 7f78adaab700 1 Processor -- start 2023-11-26T07:16:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.460+0000 7f78adaab700 1 -- start start 2023-11-26T07:16:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.460+0000 7f78adaab700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78a80d3610 0x7f78a80b5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.460+0000 7f78adaab700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80b5e90 0x7f78a80b7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.460+0000 7f78adaab700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a80d2810 con 0x7f78a815e480 2023-11-26T07:16:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.460+0000 7f78adaab700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80b6300 con 0x7f78a80d3610 2023-11-26T07:16:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.460+0000 7f78adaab700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80d2a70 con 0x7f78a80b5e90 2023-11-26T07:16:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a6ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78a80d3610 0x7f78a80b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a7fff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80b5e90 0x7f78a80b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a7fff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80b5e90 0x7f78a80b7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:51138/0 (socket says 172.21.15.83:51138) 2023-11-26T07:16:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a6ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78a80d3610 0x7f78a80b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40130/0 (socket says 172.21.15.83:40130) 2023-11-26T07:16:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a7fff700 1 -- 172.21.15.83:0/2591904622 learned_addr learned my addr 172.21.15.83:0/2591904622 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a77fe700 1 --1- 172.21.15.83:0/2591904622 >> v1:172.21.15.203:6789/0 conn(0x7f78a815e480 0x7f78a80d2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:45334/0 (socket says 172.21.15.83:45334) 2023-11-26T07:16:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a6ffd700 1 -- 172.21.15.83:0/2591904622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80b5e90 msgr2=0x7f78a80b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/2591904622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80b5e90 0x7f78a80b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a6ffd700 1 -- 172.21.15.83:0/2591904622 >> v1:172.21.15.203:6789/0 conn(0x7f78a815e480 legacy=0x7f78a80d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a6ffd700 1 -- 172.21.15.83:0/2591904622 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a80d2d50 con 0x7f78a80d3610 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a7fff700 1 --2- 172.21.15.83:0/2591904622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80b5e90 0x7f78a80b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/2591904622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80d3610 0x7f78a80b5950 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f789c009840 tx=0x7f789c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17e924dc3741d76b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a67fc700 1 -- 172.21.15.83:0/2591904622 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c011830 con 0x7f78a80d3610 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a67fc700 1 -- 172.21.15.83:0/2591904622 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c011990 con 0x7f78a80d3610 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.461+0000 7f78a67fc700 1 -- 172.21.15.83:0/2591904622 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c011d20 con 0x7f78a80d3610 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.462+0000 7f78adaab700 1 -- 172.21.15.83:0/2591904622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80d3610 msgr2=0x7f78a80b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.462+0000 7f78adaab700 1 --2- 172.21.15.83:0/2591904622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80d3610 0x7f78a80b5950 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f789c009840 tx=0x7f789c009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.462+0000 7f78adaab700 1 -- 172.21.15.83:0/2591904622 shutdown_connections 2023-11-26T07:16:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.462+0000 7f78adaab700 1 --2- 172.21.15.83:0/2591904622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80b5e90 0x7f78a80b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.462+0000 7f78adaab700 1 --2- 172.21.15.83:0/2591904622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a80d3610 0x7f78a80b5950 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.462+0000 7f78adaab700 1 -- 172.21.15.83:0/2591904622 >> 172.21.15.83:0/2591904622 conn(0x7f78a80d0950 msgr2=0x7f78a80d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:24.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.462+0000 7f78adaab700 1 -- 172.21.15.83:0/2591904622 shutdown_connections 2023-11-26T07:16:24.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.462+0000 7f78adaab700 1 -- 172.21.15.83:0/2591904622 wait complete. 2023-11-26T07:16:24.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.463+0000 7f78adaab700 1 Processor -- start 2023-11-26T07:16:24.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.463+0000 7f78adaab700 1 -- start start 2023-11-26T07:16:24.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.464+0000 7f78adaab700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a80b5e90 0x7f78a815fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.464+0000 7f78adaab700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80d3610 0x7f78a80c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.464+0000 7f78adaab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a815e480 0x7f78a815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.464+0000 7f78adaab700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80c9ca0 con 0x7f78a80b5e90 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.464+0000 7f78adaab700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80c9f40 con 0x7f78a815e480 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.464+0000 7f78adaab700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80ca220 con 0x7f78a80d3610 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.464+0000 7f78a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a80b5e90 0x7f78a815fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.464+0000 7f78a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a80b5e90 0x7f78a815fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40682/0 (socket says 172.21.15.83:40682) 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.464+0000 7f78a77fe700 1 -- 172.21.15.83:0/1585465362 learned_addr learned my addr 172.21.15.83:0/1585465362 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.464+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80d3610 0x7f78a80c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.464+0000 7f78a7fff700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a815e480 0x7f78a815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.465+0000 7f78a77fe700 1 -- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80d3610 msgr2=0x7f78a80c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.465+0000 7f78a77fe700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80d3610 0x7f78a80c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.465+0000 7f78a77fe700 1 -- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a815e480 msgr2=0x7f78a815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.465+0000 7f78a77fe700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a815e480 0x7f78a815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.465+0000 7f78a77fe700 1 -- 172.21.15.83:0/1585465362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0009b30 con 0x7f78a80b5e90 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.465+0000 7f78a77fe700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a80b5e90 0x7f78a815fe80 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f789800c490 tx=0x7f789800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.465+0000 7f788ffff700 1 -- 172.21.15.83:0/1585465362 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800cea0 con 0x7f78a80b5e90 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.465+0000 7f788ffff700 1 -- 172.21.15.83:0/1585465362 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7898005cb0 con 0x7f78a80b5e90 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.465+0000 7f78adaab700 1 -- 172.21.15.83:0/1585465362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c0094a0 con 0x7f78a80b5e90 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.465+0000 7f788ffff700 1 -- 172.21.15.83:0/1585465362 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78980072d0 con 0x7f78a80b5e90 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.465+0000 7f78adaab700 1 -- 172.21.15.83:0/1585465362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a80ca7e0 con 0x7f78a80b5e90 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.466+0000 7f788ffff700 1 -- 172.21.15.83:0/1585465362 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f7898007430 con 0x7f78a80b5e90 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.467+0000 7f788ffff700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f789406aff0 0x7f789406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.467+0000 7f788ffff700 1 -- 172.21.15.83:0/1585465362 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7898088900 con 0x7f78a80b5e90 2023-11-26T07:16:24.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.467+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f789406aff0 0x7f789406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.467+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f789406aff0 0x7f789406d4b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f789c000c00 tx=0x7f789c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:24.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.469+0000 7f78b9ca4b80 1 Processor -- start 2023-11-26T07:16:24.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.469+0000 7f78b9ca4b80 1 -- start start 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047b9e080 0x55d047be4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047bb3ed0 0x55d047be4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9b490 0x55d047bc7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d047b11770 con 0x55d047bb3ed0 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d047af8750 con 0x55d047b9b490 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d047af6070 con 0x55d047b9e080 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047bb3ed0 0x55d047be4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047bb3ed0 0x55d047be4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40692/0 (socket says 172.21.15.83:40692) 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a6ffd700 1 -- 172.21.15.83:0/2846122063 learned_addr learned my addr 172.21.15.83:0/2846122063 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a77fe700 1 --2- 172.21.15.83:0/2846122063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047b9e080 0x55d047be4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a7fff700 1 --2- 172.21.15.83:0/2846122063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9b490 0x55d047bc7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a6ffd700 1 -- 172.21.15.83:0/2846122063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047b9e080 msgr2=0x55d047be4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/2846122063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047b9e080 0x55d047be4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a6ffd700 1 -- 172.21.15.83:0/2846122063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9b490 msgr2=0x55d047bc7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/2846122063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9b490 0x55d047bc7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a6ffd700 1 -- 172.21.15.83:0/2846122063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d04796e5e0 con 0x55d047bb3ed0 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a7fff700 1 --2- 172.21.15.83:0/2846122063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9b490 0x55d047bc7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/2846122063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047bb3ed0 0x55d047be4d80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f789c009b90 tx=0x7f789c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217a93e7671e9b69 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f788d7fa700 1 -- 172.21.15.83:0/2846122063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c02c070 con 0x55d047bb3ed0 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f788d7fa700 1 -- 172.21.15.83:0/2846122063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c0036a0 con 0x55d047bb3ed0 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.470+0000 7f788d7fa700 1 -- 172.21.15.83:0/2846122063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c020070 con 0x55d047bb3ed0 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.471+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/2846122063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047bb3ed0 msgr2=0x55d047be4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.471+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/2846122063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047bb3ed0 0x55d047be4d80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f789c009b90 tx=0x7f789c010000 comp rx=0 tx=0).stop 2023-11-26T07:16:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.471+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/2846122063 shutdown_connections 2023-11-26T07:16:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.471+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/2846122063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9b490 0x55d047bc7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.471+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/2846122063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047b9e080 0x55d047be4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.471+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/2846122063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047bb3ed0 0x55d047be4d80 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.471+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/2846122063 >> 172.21.15.83:0/2846122063 conn(0x55d047acc310 msgr2=0x55d047be2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.471+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/2846122063 shutdown_connections 2023-11-26T07:16:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.471+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/2846122063 wait complete. 2023-11-26T07:16:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.472+0000 7f78b9ca4b80 1 -- start start 2023-11-26T07:16:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.472+0000 7f78b9ca4b80 1 Processor -- start 2023-11-26T07:16:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bb3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9e080 0x55d047bcb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047bb3ed0 0x55d047bdf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d047af1960 con 0x55d047b9b490 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d047aab510 con 0x55d047b9e080 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d047aab1b0 con 0x55d047bb3ed0 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78b9ca4b80 10 client.0 _ll_register_callbacks cb 0x55d0479d6a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bb3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bb3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40696/0 (socket says 172.21.15.83:40696) 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78a77fe700 1 -- 172.21.15.83:0/242586587 learned_addr learned my addr 172.21.15.83:0/242586587 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9e080 0x55d047bcb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78a77fe700 1 -- 172.21.15.83:0/242586587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047bb3ed0 msgr2=0x55d047bdf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78a77fe700 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047bb3ed0 0x55d047bdf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78a77fe700 1 -- 172.21.15.83:0/242586587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9e080 msgr2=0x55d047bcb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78a77fe700 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9e080 0x55d047bcb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78a77fe700 1 -- 172.21.15.83:0/242586587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a80d2d50 con 0x55d047b9b490 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78a77fe700 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bb3360 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f78980070a0 tx=0x7f78980576a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84734c3ec95a58f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78867fc700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78867fc700 1 -- 172.21.15.83:0/242586587 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789805f8f0 con 0x55d047b9b490 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78867fc700 1 -- 172.21.15.83:0/242586587 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f789805f460 con 0x55d047b9b490 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/242586587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d047b90a40 con 0x55d047b9b490 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78867fc700 1 -- 172.21.15.83:0/242586587 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f789805f620 con 0x55d047b9b490 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78b9ca4b80 10 client.7245 fetch_fsmap learned FSMap version 55 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/242586587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55d04791ad90 con 0x55d047b9b490 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.473+0000 7f78867fc700 1 -- 172.21.15.83:0/242586587 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d047b90a40 con 0x55d047b9b490 2023-11-26T07:16:24.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.474+0000 7f78867fc700 1 -- 172.21.15.83:0/242586587 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7898053490 con 0x55d047b9b490 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.474+0000 7f78b9ca4b80 10 client.7245 fetch_fsmap finished waiting for FSMap version 55 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.474+0000 7f78b9ca4b80 10 client.7245 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.474+0000 7f78b9ca4b80 20 client.7245 populate_metadata read hostname 'smithi083' 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.474+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.203:6833/1901175330,v1:172.21.15.203:6835/1901175330] conn(0x55d047bdcda0 0x55d047bdd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.474+0000 7f78b9ca4b80 4 client.7245 mds_command: new command op to 7128 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.474+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/242586587 --> [v2:172.21.15.203:6833/1901175330,v1:172.21.15.203:6835/1901175330] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d047b6c840 con 0x55d047bdcda0 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.474+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.203:6833/1901175330,v1:172.21.15.203:6835/1901175330] conn(0x55d047bdcda0 0x55d047bdd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.474+0000 7f78867fc700 10 client.7245 ms_handle_connect on v2:172.21.15.203:6833/1901175330 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.474+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.203:6833/1901175330,v1:172.21.15.203:6835/1901175330] conn(0x55d047bdcda0 0x55d047bdd180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06b39049d2f2f13 server_cookie=656552052843f4b2 in_seq=0 out_seq=0 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78867fc700 1 -- 172.21.15.83:0/242586587 <== mds.0 v2:172.21.15.203:6833/1901175330 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d047b6c840 con 0x55d047bdcda0 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78867fc700 10 client.7245 handle_command_reply: tid=0 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 1 client.7245 shutdown 2023-11-26T07:16:24.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/242586587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 msgr2=0x55d047bb3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bb3360 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f78980070a0 tx=0x7f78980576a0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/242586587 shutdown_connections 2023-11-26T07:16:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047bb3ed0 0x55d047bdf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bb3360 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.203:6833/1901175330,v1:172.21.15.203:6835/1901175330] conn(0x55d047bdcda0 0x55d047bdd180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/242586587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9e080 0x55d047bcb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/242586587 >> 172.21.15.83:0/242586587 conn(0x55d047acc310 msgr2=0x55d047be6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78867fc700 0 client.7245 ms_handle_reset on v2:172.21.15.203:6833/1901175330 2023-11-26T07:16:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/242586587 shutdown_connections 2023-11-26T07:16:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/242586587 wait complete. 2023-11-26T07:16:24.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.475+0000 7f78b9ca4b80 20 client.7245 trim_cache size 0 max 16384 2023-11-26T07:16:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.493+0000 7f78b9ca4b80 1 Processor -- start 2023-11-26T07:16:24.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.493+0000 7f78b9ca4b80 1 -- start start 2023-11-26T07:16:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9b490 0x55d047c06240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9e080 0x55d047be2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047bb3ed0 0x55d047be0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0479708d0 con 0x55d047b9e080 2023-11-26T07:16:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d047b98d50 con 0x55d047b9b490 2023-11-26T07:16:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d047c06780 con 0x55d047bb3ed0 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78a6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9e080 0x55d047be2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78a6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9e080 0x55d047be2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40706/0 (socket says 172.21.15.83:40706) 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78a6ffd700 1 -- 172.21.15.83:0/3701622652 learned_addr learned my addr 172.21.15.83:0/3701622652 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78a6ffd700 1 -- 172.21.15.83:0/3701622652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047bb3ed0 msgr2=0x55d047be0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/3701622652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047bb3ed0 0x55d047be0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78a6ffd700 1 -- 172.21.15.83:0/3701622652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9b490 msgr2=0x55d047c06240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78a77fe700 1 --2- 172.21.15.83:0/3701622652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9b490 0x55d047c06240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/3701622652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9b490 0x55d047c06240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.494+0000 7f78a6ffd700 1 -- 172.21.15.83:0/3701622652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d047b9a330 con 0x55d047b9e080 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/3701622652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9e080 0x55d047be2ef0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f78a808c680 tx=0x7f789c009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03244242768e054 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f788dffb700 1 -- 172.21.15.83:0/3701622652 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c02c040 con 0x55d047b9e080 2023-11-26T07:16:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f788dffb700 1 -- 172.21.15.83:0/3701622652 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c003a90 con 0x55d047b9e080 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f788dffb700 1 -- 172.21.15.83:0/3701622652 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c003e40 con 0x55d047b9e080 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/3701622652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9e080 msgr2=0x55d047be2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/3701622652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9e080 0x55d047be2ef0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f78a808c680 tx=0x7f789c009210 comp rx=0 tx=0).stop 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/3701622652 shutdown_connections 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/3701622652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047bb3ed0 0x55d047be0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/3701622652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047b9b490 0x55d047c06240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/3701622652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9e080 0x55d047be2ef0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/3701622652 >> 172.21.15.83:0/3701622652 conn(0x55d047acc310 msgr2=0x55d047be56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/3701622652 shutdown_connections 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.495+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/3701622652 wait complete. 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.496+0000 7f78b9ca4b80 1 -- start start 2023-11-26T07:16:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.496+0000 7f78b9ca4b80 1 Processor -- start 2023-11-26T07:16:24.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bd8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047b9e080 0x55d047bc7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78b9ca4b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047bb3ed0 0x55d047bc8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d047c063f0 con 0x55d047b9b490 2023-11-26T07:16:24.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d047be1b80 con 0x55d047bb3ed0 2023-11-26T07:16:24.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78b9ca4b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d047bd9090 con 0x55d047b9e080 2023-11-26T07:16:24.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78b9ca4b80 10 client.0 _ll_register_callbacks cb 0x55d0479d6a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:16:24.503 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bd8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a77fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bd8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40710/0 (socket says 172.21.15.83:40710) 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a77fe700 1 -- 172.21.15.83:0/734605613 learned_addr learned my addr 172.21.15.83:0/734605613 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a7fff700 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047bb3ed0 0x55d047bc8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047b9e080 0x55d047bc7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a77fe700 1 -- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047b9e080 msgr2=0x55d047bc7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a77fe700 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047b9e080 0x55d047bc7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a77fe700 1 -- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047bb3ed0 msgr2=0x55d047bc8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a77fe700 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047bb3ed0 0x55d047bc8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a77fe700 1 -- 172.21.15.83:0/734605613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d04796e5e0 con 0x55d047b9b490 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a7fff700 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047bb3ed0 0x55d047bc8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78a77fe700 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bd8800 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f78980575e0 tx=0x7f7898000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47901806ebf3d707 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:24.504 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78867fc700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78867fc700 1 -- 172.21.15.83:0/734605613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789805f8f0 con 0x55d047b9b490 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78867fc700 1 -- 172.21.15.83:0/734605613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7898071030 con 0x55d047b9b490 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.497+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/734605613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f789c0036a0 con 0x55d047b9b490 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78867fc700 1 -- 172.21.15.83:0/734605613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f789805f460 con 0x55d047b9b490 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78b9ca4b80 10 client.7251 fetch_fsmap learned FSMap version 55 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/734605613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55d047be3c50 con 0x55d047b9b490 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78867fc700 1 -- 172.21.15.83:0/734605613 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c0036a0 con 0x55d047b9b490 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78867fc700 1 -- 172.21.15.83:0/734605613 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7898056600 con 0x55d047b9b490 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78b9ca4b80 10 client.7251 fetch_fsmap finished waiting for FSMap version 55 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78b9ca4b80 10 client.7251 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78b9ca4b80 20 client.7251 populate_metadata read hostname 'smithi083' 2023-11-26T07:16:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:6833/1901175330,v1:172.21.15.203:6835/1901175330] conn(0x55d047bc8040 0x55d047bc9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78b9ca4b80 4 client.7251 mds_command: new command op to 7128 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/734605613 --> [v2:172.21.15.203:6833/1901175330,v1:172.21.15.203:6835/1901175330] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d047ab7f80 con 0x55d047bc8040 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:6833/1901175330,v1:172.21.15.203:6835/1901175330] conn(0x55d047bc8040 0x55d047bc9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78867fc700 10 client.7251 ms_handle_connect on v2:172.21.15.203:6833/1901175330 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.498+0000 7f78a6ffd700 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:6833/1901175330,v1:172.21.15.203:6835/1901175330] conn(0x55d047bc8040 0x55d047bc9470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d3bda65f44706d server_cookie=5f98fa86ea707fd in_seq=0 out_seq=0 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78867fc700 1 -- 172.21.15.83:0/734605613 <== mds.0 v2:172.21.15.203:6833/1901175330 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d047ab7f80 con 0x55d047bc8040 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78867fc700 10 client.7251 handle_command_reply: tid=0 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 1 client.7251 shutdown 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/734605613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 msgr2=0x55d047bd8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bd8800 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f78980575e0 tx=0x7f7898000bc0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/734605613 shutdown_connections 2023-11-26T07:16:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:6833/1901175330,v1:172.21.15.203:6835/1901175330] conn(0x55d047bc8040 0x55d047bc9470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d047bb3ed0 0x55d047bc8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d047b9b490 0x55d047bd8800 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 1 --2- 172.21.15.83:0/734605613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d047b9e080 0x55d047bc7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78867fc700 0 client.7251 ms_handle_reset on v2:172.21.15.203:6833/1901175330 2023-11-26T07:16:24.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/734605613 >> 172.21.15.83:0/734605613 conn(0x55d047acc310 msgr2=0x55d047bdea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:24.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/734605613 shutdown_connections 2023-11-26T07:16:24.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 1 -- 172.21.15.83:0/734605613 wait complete. 2023-11-26T07:16:24.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78b9ca4b80 20 client.7251 trim_cache size 0 max 16384 2023-11-26T07:16:24.507 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:16:24.507 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "0aae1049-cc8c-4011-a01f-378673e57a96", 2023-11-26T07:16:24.508 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-26T07:16:24.508 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7128, 2023-11-26T07:16:24.508 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-26T07:16:24.508 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-26T07:16:24.508 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-26T07:16:24.508 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.9173977850000004, 2023-11-26T07:16:24.508 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 55, 2023-11-26T07:16:24.508 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 83, 2023-11-26T07:16:24.508 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 83, 2023-11-26T07:16:24.508 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 11.834637730000001 2023-11-26T07:16:24.508 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:16:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78adaab700 1 -- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f789406aff0 msgr2=0x7f789406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78adaab700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f789406aff0 0x7f789406d4b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f789c000c00 tx=0x7f789c008030 comp rx=0 tx=0).stop 2023-11-26T07:16:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78adaab700 1 -- 172.21.15.83:0/1585465362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a80b5e90 msgr2=0x7f78a815fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78adaab700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a80b5e90 0x7f78a815fe80 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f789800c490 tx=0x7f789800c7a0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78adaab700 1 -- 172.21.15.83:0/1585465362 shutdown_connections 2023-11-26T07:16:24.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78adaab700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f789406aff0 0x7f789406d4b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78adaab700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78a815e480 0x7f78a815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78adaab700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78a80b5e90 0x7f78a815fe80 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78adaab700 1 --2- 172.21.15.83:0/1585465362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78a80d3610 0x7f78a80c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.499+0000 7f78adaab700 1 -- 172.21.15.83:0/1585465362 >> 172.21.15.83:0/1585465362 conn(0x7f78a80d0950 msgr2=0x7f78a80d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:24.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.500+0000 7f78adaab700 1 -- 172.21.15.83:0/1585465362 shutdown_connections 2023-11-26T07:16:24.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.500+0000 7f78adaab700 1 -- 172.21.15.83:0/1585465362 wait complete. 2023-11-26T07:16:24.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:16:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.631+0000 7f89c7a77700 1 Processor -- start 2023-11-26T07:16:24.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.631+0000 7f89c7a77700 1 -- start start 2023-11-26T07:16:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c7a77700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89c0147780 0x7f89c015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c7a77700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f89c006dc20 0x7f89c0073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c7a77700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c015eb00 con 0x7f89c01473a0 2023-11-26T07:16:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c7a77700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c015ede0 con 0x7f89c006dc20 2023-11-26T07:16:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c7a77700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c015f0c0 con 0x7f89c0147780 2023-11-26T07:16:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c5813700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f89c01473a0 0x7f89c006c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60758/0 (socket says 172.21.15.83:60758) 2023-11-26T07:16:24.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c5813700 1 -- 172.21.15.83:0/3153209092 learned_addr learned my addr 172.21.15.83:0/3153209092 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c6014700 1 --2- 172.21.15.83:0/3153209092 >> v2:172.21.15.203:3300/0 conn(0x7f89c006dc20 0x7f89c0073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c5012700 1 --2- 172.21.15.83:0/3153209092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89c0147780 0x7f89c015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c5012700 1 -- 172.21.15.83:0/3153209092 >> v2:172.21.15.203:3300/0 conn(0x7f89c006dc20 msgr2=0x7f89c0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c5012700 1 --2- 172.21.15.83:0/3153209092 >> v2:172.21.15.203:3300/0 conn(0x7f89c006dc20 0x7f89c0073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c5012700 1 -- 172.21.15.83:0/3153209092 >> v1:172.21.15.83:6789/0 conn(0x7f89c01473a0 legacy=0x7f89c006c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c5012700 1 -- 172.21.15.83:0/3153209092 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c015f3a0 con 0x7f89c0147780 2023-11-26T07:16:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.632+0000 7f89c4811700 1 -- 172.21.15.83:0/3153209092 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474956756 0 0) 0x7f89c015eb00 con 0x7f89c01473a0 2023-11-26T07:16:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.633+0000 7f89c5012700 1 --2- 172.21.15.83:0/3153209092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89c0147780 0x7f89c015e4b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009840 tx=0x7f89b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e138836908152e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.633+0000 7f89c4811700 1 -- 172.21.15.83:0/3153209092 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0011830 con 0x7f89c0147780 2023-11-26T07:16:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.633+0000 7f89c4811700 1 -- 172.21.15.83:0/3153209092 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b0011990 con 0x7f89c0147780 2023-11-26T07:16:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.633+0000 7f89c4811700 1 -- 172.21.15.83:0/3153209092 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0011d20 con 0x7f89c0147780 2023-11-26T07:16:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.634+0000 7f89c7a77700 1 -- 172.21.15.83:0/3153209092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89c0147780 msgr2=0x7f89c015e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.634+0000 7f89c7a77700 1 --2- 172.21.15.83:0/3153209092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89c0147780 0x7f89c015e4b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009840 tx=0x7f89b0009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.634+0000 7f89c7a77700 1 -- 172.21.15.83:0/3153209092 shutdown_connections 2023-11-26T07:16:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.634+0000 7f89c7a77700 1 --2- 172.21.15.83:0/3153209092 >> v2:172.21.15.203:3300/0 conn(0x7f89c006dc20 0x7f89c0073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.634+0000 7f89c7a77700 1 --2- 172.21.15.83:0/3153209092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89c0147780 0x7f89c015e4b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.634+0000 7f89c7a77700 1 -- 172.21.15.83:0/3153209092 >> 172.21.15.83:0/3153209092 conn(0x7f89c00b3130 msgr2=0x7f89c00d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.634+0000 7f89c7a77700 1 -- 172.21.15.83:0/3153209092 shutdown_connections 2023-11-26T07:16:24.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.634+0000 7f89c7a77700 1 -- 172.21.15.83:0/3153209092 wait complete. 2023-11-26T07:16:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.635+0000 7f89c7a77700 1 Processor -- start 2023-11-26T07:16:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.635+0000 7f89c7a77700 1 -- start start 2023-11-26T07:16:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c7a77700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c006dc20 0x7f89c015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c7a77700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89c01473a0 0x7f89c013c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c7a77700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c0147780 0x7f89c00b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c7a77700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c013e950 con 0x7f89c006dc20 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c7a77700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00cd710 con 0x7f89c0147780 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c7a77700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00cd9b0 con 0x7f89c01473a0 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c5813700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c006dc20 0x7f89c015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c5813700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c006dc20 0x7f89c015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40724/0 (socket says 172.21.15.83:40724) 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c5813700 1 -- 172.21.15.83:0/4003179235 learned_addr learned my addr 172.21.15.83:0/4003179235 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c5012700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89c01473a0 0x7f89c013c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c6014700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c0147780 0x7f89c00b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c5813700 1 -- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89c01473a0 msgr2=0x7f89c013c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.636+0000 7f89c5813700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89c01473a0 0x7f89c013c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.637+0000 7f89c5813700 1 -- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c0147780 msgr2=0x7f89c00b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.637+0000 7f89c5813700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c0147780 0x7f89c00b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.637+0000 7f89c5813700 1 -- 172.21.15.83:0/4003179235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b00094a0 con 0x7f89c006dc20 2023-11-26T07:16:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.637+0000 7f89c6014700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c0147780 0x7f89c00b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.637+0000 7f89c5813700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c006dc20 0x7f89c015d120 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0011e0 tx=0x7f89bc00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.637+0000 7f89b67fc700 1 -- 172.21.15.83:0/4003179235 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc004c80 con 0x7f89c006dc20 2023-11-26T07:16:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.637+0000 7f89b67fc700 1 -- 172.21.15.83:0/4003179235 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc004de0 con 0x7f89c006dc20 2023-11-26T07:16:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.637+0000 7f89c7a77700 1 -- 172.21.15.83:0/4003179235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c00cdc90 con 0x7f89c006dc20 2023-11-26T07:16:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.637+0000 7f89b67fc700 1 -- 172.21.15.83:0/4003179235 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc007310 con 0x7f89c006dc20 2023-11-26T07:16:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.637+0000 7f89c7a77700 1 -- 172.21.15.83:0/4003179235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c00ce250 con 0x7f89c006dc20 2023-11-26T07:16:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.638+0000 7f89c7a77700 1 -- 172.21.15.83:0/4003179235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c0067f60 con 0x7f89c006dc20 2023-11-26T07:16:24.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.638+0000 7f89b67fc700 1 -- 172.21.15.83:0/4003179235 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f89bc007470 con 0x7f89c006dc20 2023-11-26T07:16:24.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.638+0000 7f89b67fc700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f89a406aff0 0x7f89a406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:24.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.639+0000 7f89b67fc700 1 -- 172.21.15.83:0/4003179235 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89bc0893a0 con 0x7f89c006dc20 2023-11-26T07:16:24.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.639+0000 7f89c5012700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f89a406aff0 0x7f89a406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:24.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.639+0000 7f89c5012700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f89a406aff0 0x7f89a406d4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009840 tx=0x7f89b001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:24.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.643+0000 7f89b67fc700 1 -- 172.21.15.83:0/4003179235 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89bc0543d0 con 0x7f89c006dc20 2023-11-26T07:16:24.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.878+0000 7f89c7a77700 1 -- 172.21.15.83:0/4003179235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f89c006c240 con 0x7f89c006dc20 2023-11-26T07:16:24.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.878+0000 7f89b67fc700 1 -- 172.21.15.83:0/4003179235 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7f89bc0579f0 con 0x7f89c006dc20 2023-11-26T07:16:24.880 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:16:24.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 -- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f89a406aff0 msgr2=0x7f89a406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f89a406aff0 0x7f89a406d4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009840 tx=0x7f89b001c040 comp rx=0 tx=0).stop 2023-11-26T07:16:24.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 -- 172.21.15.83:0/4003179235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c006dc20 msgr2=0x7f89c015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:24.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c006dc20 0x7f89c015d120 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0011e0 tx=0x7f89bc00ca30 comp rx=0 tx=0).stop 2023-11-26T07:16:24.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 -- 172.21.15.83:0/4003179235 shutdown_connections 2023-11-26T07:16:24.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f89a406aff0 0x7f89a406d4b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89c0147780 0x7f89c00b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c006dc20 0x7f89c015d120 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 --2- 172.21.15.83:0/4003179235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89c01473a0 0x7f89c013c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:24.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 -- 172.21.15.83:0/4003179235 >> 172.21.15.83:0/4003179235 conn(0x7f89c00b3130 msgr2=0x7f89c01466c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:24.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 -- 172.21.15.83:0/4003179235 shutdown_connections 2023-11-26T07:16:24.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:24.880+0000 7f89c7a77700 1 -- 172.21.15.83:0/4003179235 wait complete. 2023-11-26T07:16:24.893 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:16:24.894 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:16:24.894 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-26T07:16:24.894 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:16:24.894 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:16:24.894 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:16:24.894 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:16:24.894 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:16:24.894 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:16:24.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:24.895 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-26T07:16:24.911 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:16:24.911 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:16:24.912 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42101sec preferred_lft 42101sec 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::3a41:fab:adc5:5d6c/64 scope link noprefixroute 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::54b0:6eff:fe33:bb15/64 scope link 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether fe:46:b0:72:7b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:16:24.913 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::fc46:b0ff:fe72:7b3f/64 scope link 2023-11-26T07:16:24.914 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:24.914 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:16:24.914 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:16:24.914 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::7808:fdff:fe1b:770c/64 scope link 2023-11-26T07:16:24.914 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:24.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:24.914 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:16:24.977 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-26T07:16:24.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:24.979 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-26T07:16:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-26T07:16:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:16:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:16:25.037 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:16:25.038 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:16:25.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:25.038 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:16:25.038 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:16:25.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:16:25.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:16:25.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:16:25.038 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:16:25.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:25.141 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:16:25.157 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:16:25.157 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:16:25.157 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:16:25.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:25.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-26T07:16:25.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:16:25.223 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:16:25.280 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:16:25.280 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:16:25.280 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:16:25.336 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:16:25.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:25.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:16:25.421 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:16:25.421 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:16:25.448 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:16:25.448 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:16:25.448 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:16:25.448 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:16:25.448 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:16:25.448 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:16:25.449 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:16:25.449 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:16:25.518 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:16:25.518 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:16:25.518 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:16:25.519 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:16:25.519 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:16:25.519 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:16:25.519 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:16:25.519 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:16:25.519 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:16:25.519 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:16:25.519 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:16:25.520 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:16:25.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:25.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:16:25.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:25.547 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:16:25.635 INFO:teuthology.orchestra.run.smithi083.stdout:client7260 2023-11-26T07:16:25.636 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:16:25.636 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:25.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:16:25.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.812+0000 7fdc31343700 1 Processor -- start 2023-11-26T07:16:25.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.812+0000 7fdc31343700 1 -- start start 2023-11-26T07:16:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.813+0000 7fdc31343700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc2c15c160 0x7fdc2c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.813+0000 7fdc31343700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc2c0b2870 0x7fdc2c0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.813+0000 7fdc31343700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc2c0b02e0 0x7fdc2c0ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:25.816 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.813+0000 7fdc31343700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c0b2cf0 con 0x7fdc2c0b2870 2023-11-26T07:16:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.813+0000 7fdc31343700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c0b0770 con 0x7fdc2c0b02e0 2023-11-26T07:16:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.813+0000 7fdc31343700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c15c540 con 0x7fdc2c15c160 2023-11-26T07:16:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.813+0000 7fdc2a7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc2c0b2870 0x7fdc2c0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.813+0000 7fdc2a7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc2c0b2870 0x7fdc2c0afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40738/0 (socket says 172.21.15.83:40738) 2023-11-26T07:16:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.813+0000 7fdc2a7fc700 1 -- 172.21.15.83:0/3460226054 learned_addr learned my addr 172.21.15.83:0/3460226054 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc2b7fe700 1 --2- 172.21.15.83:0/3460226054 >> v2:172.21.15.203:3300/0 conn(0x7fdc2c0b02e0 0x7fdc2c0ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc2affd700 1 --2- 172.21.15.83:0/3460226054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc2c15c160 0x7fdc2c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc2a7fc700 1 -- 172.21.15.83:0/3460226054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc2c15c160 msgr2=0x7fdc2c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc2a7fc700 1 --2- 172.21.15.83:0/3460226054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc2c15c160 0x7fdc2c0622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc2a7fc700 1 -- 172.21.15.83:0/3460226054 >> v2:172.21.15.203:3300/0 conn(0x7fdc2c0b02e0 msgr2=0x7fdc2c0ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:25.817 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc2a7fc700 1 --2- 172.21.15.83:0/3460226054 >> v2:172.21.15.203:3300/0 conn(0x7fdc2c0b02e0 0x7fdc2c0ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc2a7fc700 1 -- 172.21.15.83:0/3460226054 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c0b2ee0 con 0x7fdc2c0b2870 2023-11-26T07:16:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc2a7fc700 1 --2- 172.21.15.83:0/3460226054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c0b2870 0x7fdc2c0afda0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0b56f0 tx=0x7fdc1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cfb00acd1f8ca3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc29ffb700 1 -- 172.21.15.83:0/3460226054 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00b860 con 0x7fdc2c0b2870 2023-11-26T07:16:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc29ffb700 1 -- 172.21.15.83:0/3460226054 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c00b9c0 con 0x7fdc2c0b2870 2023-11-26T07:16:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc29ffb700 1 -- 172.21.15.83:0/3460226054 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00bd50 con 0x7fdc2c0b2870 2023-11-26T07:16:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc31343700 1 -- 172.21.15.83:0/3460226054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c0b2870 msgr2=0x7fdc2c0afda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc31343700 1 --2- 172.21.15.83:0/3460226054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c0b2870 0x7fdc2c0afda0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0b56f0 tx=0x7fdc1c009c00 comp rx=0 tx=0).stop 2023-11-26T07:16:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc31343700 1 -- 172.21.15.83:0/3460226054 shutdown_connections 2023-11-26T07:16:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc31343700 1 --2- 172.21.15.83:0/3460226054 >> v2:172.21.15.203:3300/0 conn(0x7fdc2c0b02e0 0x7fdc2c0ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc31343700 1 --2- 172.21.15.83:0/3460226054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc2c15c160 0x7fdc2c0622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:25.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc31343700 1 --2- 172.21.15.83:0/3460226054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c0b2870 0x7fdc2c0afda0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.814+0000 7fdc31343700 1 -- 172.21.15.83:0/3460226054 >> 172.21.15.83:0/3460226054 conn(0x7fdc2c0b57b0 msgr2=0x7fdc2c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.815+0000 7fdc31343700 1 -- 172.21.15.83:0/3460226054 shutdown_connections 2023-11-26T07:16:25.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.815+0000 7fdc31343700 1 -- 172.21.15.83:0/3460226054 wait complete. 2023-11-26T07:16:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.816+0000 7fdc31343700 1 Processor -- start 2023-11-26T07:16:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.816+0000 7fdc31343700 1 -- start start 2023-11-26T07:16:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc31343700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc2c0b02e0 0x7fdc2c154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc31343700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c0b2870 0x7fdc2c15f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc31343700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c15c160 0x7fdc2c14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc31343700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c15f6a0 con 0x7fdc2c0b2870 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc31343700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c14d690 con 0x7fdc2c15c160 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc31343700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c14ccc0 con 0x7fdc2c0b02e0 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc2a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c0b2870 0x7fdc2c15f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc2a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c0b2870 0x7fdc2c15f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40744/0 (socket says 172.21.15.83:40744) 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc2a7fc700 1 -- 172.21.15.83:0/2289729991 learned_addr learned my addr 172.21.15.83:0/2289729991 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc2affd700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc2c0b02e0 0x7fdc2c154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc2b7fe700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c15c160 0x7fdc2c14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc2a7fc700 1 -- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc2c0b02e0 msgr2=0x7fdc2c154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc2a7fc700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc2c0b02e0 0x7fdc2c154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc2a7fc700 1 -- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c15c160 msgr2=0x7fdc2c14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc2a7fc700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c15c160 0x7fdc2c14d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc2a7fc700 1 -- 172.21.15.83:0/2289729991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c009550 con 0x7fdc2c0b2870 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc2a7fc700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c0b2870 0x7fdc2c15f160 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c06fd10 tx=0x7fdc1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc137fe700 1 -- 172.21.15.83:0/2289729991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c004240 con 0x7fdc2c0b2870 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc137fe700 1 -- 172.21.15.83:0/2289729991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c01f070 con 0x7fdc2c0b2870 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc137fe700 1 -- 172.21.15.83:0/2289729991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0045c0 con 0x7fdc2c0b2870 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc31343700 1 -- 172.21.15.83:0/2289729991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c153460 con 0x7fdc2c0b2870 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.817+0000 7fdc31343700 1 -- 172.21.15.83:0/2289729991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c14c670 con 0x7fdc2c0b2870 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.818+0000 7fdc31343700 1 -- 172.21.15.83:0/2289729991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c067f60 con 0x7fdc2c0b2870 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.819+0000 7fdc137fe700 1 -- 172.21.15.83:0/2289729991 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fdc1c021030 con 0x7fdc2c0b2870 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.819+0000 7fdc137fe700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1806aff0 0x7fdc1806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.819+0000 7fdc137fe700 1 -- 172.21.15.83:0/2289729991 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc1c08a8c0 con 0x7fdc2c0b2870 2023-11-26T07:16:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.819+0000 7fdc2affd700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1806aff0 0x7fdc1806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.820+0000 7fdc2affd700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1806aff0 0x7fdc1806d4b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdc20006b20 tx=0x7fdc20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:25.824+0000 7fdc137fe700 1 -- 172.21.15.83:0/2289729991 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc1c055850 con 0x7fdc2c0b2870 2023-11-26T07:16:26.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.058+0000 7fdc31343700 1 -- 172.21.15.83:0/2289729991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdc2c0b2ee0 con 0x7fdc2c0b2870 2023-11-26T07:16:26.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.059+0000 7fdc137fe700 1 -- 172.21.15.83:0/2289729991 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdc2c0b2ee0 con 0x7fdc2c0b2870 2023-11-26T07:16:26.060 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:16:26.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.060+0000 7fdc31343700 1 -- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1806aff0 msgr2=0x7fdc1806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:26.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.060+0000 7fdc31343700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1806aff0 0x7fdc1806d4b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdc20006b20 tx=0x7fdc20008040 comp rx=0 tx=0).stop 2023-11-26T07:16:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.060+0000 7fdc31343700 1 -- 172.21.15.83:0/2289729991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c0b2870 msgr2=0x7fdc2c15f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.060+0000 7fdc31343700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c0b2870 0x7fdc2c15f160 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c06fd10 tx=0x7fdc1c004750 comp rx=0 tx=0).stop 2023-11-26T07:16:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.061+0000 7fdc31343700 1 -- 172.21.15.83:0/2289729991 shutdown_connections 2023-11-26T07:16:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.061+0000 7fdc31343700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1806aff0 0x7fdc1806d4b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.061+0000 7fdc31343700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc2c15c160 0x7fdc2c14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.061+0000 7fdc31343700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc2c0b02e0 0x7fdc2c154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.061+0000 7fdc31343700 1 --2- 172.21.15.83:0/2289729991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c0b2870 0x7fdc2c15f160 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.061+0000 7fdc31343700 1 -- 172.21.15.83:0/2289729991 >> 172.21.15.83:0/2289729991 conn(0x7fdc2c0b57b0 msgr2=0x7fdc2c0afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.061+0000 7fdc31343700 1 -- 172.21.15.83:0/2289729991 shutdown_connections 2023-11-26T07:16:26.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.061+0000 7fdc31343700 1 -- 172.21.15.83:0/2289729991 wait complete. 2023-11-26T07:16:26.074 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:16:26.074 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:16:26.074 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2023-11-26T07:16:26.074 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:16:26.074 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:16:26.075 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:16:26.075 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:16:26.075 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:16:26.075 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:16:26.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:26.075 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2023-11-26T07:16:26.096 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:16:26.096 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:16:26.096 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:16:26.096 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:26.096 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-11-26T07:16:26.096 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:26.096 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42103sec preferred_lft 42103sec 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::64b0:e04:3aa8:a78c/64 scope link noprefixroute 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:16:26.097 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1092:a6ff:fe63:3da7/64 scope link 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::b88a:cff:fe60:2b6f/64 scope link 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether de:b3:89:ce:09:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::dcb3:89ff:fece:9cd/64 scope link 2023-11-26T07:16:26.098 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:16:26.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:26.099 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:16:26.166 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-11-26T07:16:26.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:26.168 DEBUG:teuthology.orchestra.run.smithi203:> route 2023-11-26T07:16:26.228 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2023-11-26T07:16:26.229 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:16:26.229 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:16:26.229 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:16:26.229 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:16:26.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:26.229 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:16:26.229 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:16:26.229 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:16:26.229 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:16:26.229 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:16:26.230 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:16:26.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:26.341 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:16:26.358 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:16:26.358 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:16:26.359 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:16:26.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:26.359 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2023-11-26T07:16:26.425 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:16:26.426 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:16:26.483 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:16:26.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:16:26.483 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:16:26.539 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:16:26.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:26.540 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:16:26.627 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:16:26.628 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:16:26.651 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:16:26.651 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:16:26.651 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:16:26.652 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:16:26.652 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:16:26.652 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:16:26.652 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:16:26.652 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:16:26.691 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:16:26.691 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:16:26.692 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:16:26.692 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:16:26.692 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:16:26.692 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:16:26.692 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:16:26.692 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:16:26.692 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:16:26.693 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:16:26.693 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:16:26.701 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:16:26.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:26.701 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:16:26.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:26.733 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:16:26.804 INFO:teuthology.orchestra.run.smithi203.stdout:client6502 2023-11-26T07:16:26.805 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:16:26.805 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:26.862 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:16:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.990+0000 7fd0b247c700 1 Processor -- start 2023-11-26T07:16:26.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.990+0000 7fd0b247c700 1 -- start start 2023-11-26T07:16:26.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.990+0000 7fd0b247c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd0ac0cf040 0x7fd0ac06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:26.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.990+0000 7fd0b247c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0d21c0 0x7fd0ac0af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:26.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0b247c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ac0d01c0 con 0x7fd0ac0702e0 2023-11-26T07:16:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0b247c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac070910 con 0x7fd0ac0cf040 2023-11-26T07:16:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0b247c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac070bf0 con 0x7fd0ac0d21c0 2023-11-26T07:16:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0abfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd0ac0cf040 0x7fd0ac06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0ab7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd0ac0702e0 0x7fd0ac0d1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60794/0 (socket says 172.21.15.83:60794) 2023-11-26T07:16:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0ab7fe700 1 -- 172.21.15.83:0/2678858677 learned_addr learned my addr 172.21.15.83:0/2678858677 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0abfff700 1 -- 172.21.15.83:0/2678858677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0d21c0 msgr2=0x7fd0ac0af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0b0a19700 1 --2- 172.21.15.83:0/2678858677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0d21c0 0x7fd0ac0af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0abfff700 1 --2- 172.21.15.83:0/2678858677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0d21c0 0x7fd0ac0af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0aaffd700 1 -- 172.21.15.83:0/2678858677 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683277799 0 0) 0x7fd0ac0d01c0 con 0x7fd0ac0702e0 2023-11-26T07:16:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0abfff700 1 -- 172.21.15.83:0/2678858677 >> v1:172.21.15.83:6789/0 conn(0x7fd0ac0702e0 legacy=0x7fd0ac0d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0abfff700 1 -- 172.21.15.83:0/2678858677 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac0aebb0 con 0x7fd0ac0cf040 2023-11-26T07:16:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0b0a19700 1 --2- 172.21.15.83:0/2678858677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0d21c0 0x7fd0ac0af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0abfff700 1 --2- 172.21.15.83:0/2678858677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0ac0cf040 0x7fd0ac06fda0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009840 tx=0x7fd09c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f805771bfd8b92a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0aaffd700 1 -- 172.21.15.83:0/2678858677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c011830 con 0x7fd0ac0cf040 2023-11-26T07:16:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0aaffd700 1 -- 172.21.15.83:0/2678858677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c011990 con 0x7fd0ac0cf040 2023-11-26T07:16:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.991+0000 7fd0aaffd700 1 -- 172.21.15.83:0/2678858677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c011d20 con 0x7fd0ac0cf040 2023-11-26T07:16:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.992+0000 7fd0b247c700 1 -- 172.21.15.83:0/2678858677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0ac0cf040 msgr2=0x7fd0ac06fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.992+0000 7fd0b247c700 1 --2- 172.21.15.83:0/2678858677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0ac0cf040 0x7fd0ac06fda0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009840 tx=0x7fd09c009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.992+0000 7fd0b247c700 1 -- 172.21.15.83:0/2678858677 shutdown_connections 2023-11-26T07:16:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.992+0000 7fd0b247c700 1 --2- 172.21.15.83:0/2678858677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0d21c0 0x7fd0ac0af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.992+0000 7fd0b247c700 1 --2- 172.21.15.83:0/2678858677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0ac0cf040 0x7fd0ac06fda0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.992+0000 7fd0b247c700 1 -- 172.21.15.83:0/2678858677 >> 172.21.15.83:0/2678858677 conn(0x7fd0ac09aa00 msgr2=0x7fd0ac09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.992+0000 7fd0b247c700 1 -- 172.21.15.83:0/2678858677 shutdown_connections 2023-11-26T07:16:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.992+0000 7fd0b247c700 1 -- 172.21.15.83:0/2678858677 wait complete. 2023-11-26T07:16:26.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.993+0000 7fd0b247c700 1 Processor -- start 2023-11-26T07:16:26.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.993+0000 7fd0b247c700 1 -- start start 2023-11-26T07:16:26.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0b247c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0702e0 0x7fd0ac1f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0b247c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0ac0cf040 0x7fd0ac14cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0b247c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d21c0 0x7fd0ac0b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0b247c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac0c8cd0 con 0x7fd0ac0cf040 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0b247c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac1f5b40 con 0x7fd0ac0d21c0 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0b247c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac0c8f70 con 0x7fd0ac0702e0 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0ab7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0ac0cf040 0x7fd0ac14cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0ab7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0ac0cf040 0x7fd0ac14cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40756/0 (socket says 172.21.15.83:40756) 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0ab7fe700 1 -- 172.21.15.83:0/3612672188 learned_addr learned my addr 172.21.15.83:0/3612672188 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0abfff700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0702e0 0x7fd0ac1f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0b0a19700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d21c0 0x7fd0ac0b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0ab7fe700 1 -- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0702e0 msgr2=0x7fd0ac1f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:26.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0ab7fe700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0702e0 0x7fd0ac1f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0ab7fe700 1 -- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d21c0 msgr2=0x7fd0ac0b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0ab7fe700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d21c0 0x7fd0ac0b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0ab7fe700 1 -- 172.21.15.83:0/3612672188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c0094a0 con 0x7fd0ac0cf040 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.994+0000 7fd0b0a19700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d21c0 0x7fd0ac0b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.995+0000 7fd0ab7fe700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0ac0cf040 0x7fd0ac14cc90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00011e0 tx=0x7fd0a000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.995+0000 7fd0a8ff9700 1 -- 172.21.15.83:0/3612672188 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0004c80 con 0x7fd0ac0cf040 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.995+0000 7fd0a8ff9700 1 -- 172.21.15.83:0/3612672188 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a0004e00 con 0x7fd0ac0cf040 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.995+0000 7fd0b247c700 1 -- 172.21.15.83:0/3612672188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ac0c9250 con 0x7fd0ac0cf040 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.995+0000 7fd0a8ff9700 1 -- 172.21.15.83:0/3612672188 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0007310 con 0x7fd0ac0cf040 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.995+0000 7fd0b247c700 1 -- 172.21.15.83:0/3612672188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ac0c9810 con 0x7fd0ac0cf040 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.996+0000 7fd0b247c700 1 -- 172.21.15.83:0/3612672188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ac067f60 con 0x7fd0ac0cf040 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.996+0000 7fd0a8ff9700 1 -- 172.21.15.83:0/3612672188 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fd0a0007470 con 0x7fd0ac0cf040 2023-11-26T07:16:27.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.996+0000 7fd0a8ff9700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd09806aff0 0x7fd09806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:27.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.997+0000 7fd0a8ff9700 1 -- 172.21.15.83:0/3612672188 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0a0089500 con 0x7fd0ac0cf040 2023-11-26T07:16:27.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.997+0000 7fd0abfff700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd09806aff0 0x7fd09806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:27.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:26.997+0000 7fd0abfff700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd09806aff0 0x7fd09806d4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0090b0 tx=0x7fd09c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:27.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.001+0000 7fd0a8ff9700 1 -- 172.21.15.83:0/3612672188 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0a00544a0 con 0x7fd0ac0cf040 2023-11-26T07:16:27.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.234+0000 7fd0b247c700 1 -- 172.21.15.83:0/3612672188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd0ac06eeb0 con 0x7fd0ac0cf040 2023-11-26T07:16:27.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:16:27.330 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:16:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.662+0000 7fd0a8ff9700 1 -- 172.21.15.83:0/3612672188 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd0a0057ac0 con 0x7fd0ac0cf040 2023-11-26T07:16:27.666 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 -- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd09806aff0 msgr2=0x7fd09806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:27.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd09806aff0 0x7fd09806d4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0090b0 tx=0x7fd09c008030 comp rx=0 tx=0).stop 2023-11-26T07:16:27.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 -- 172.21.15.83:0/3612672188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0ac0cf040 msgr2=0x7fd0ac14cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:27.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0ac0cf040 0x7fd0ac14cc90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00011e0 tx=0x7fd0a000cae0 comp rx=0 tx=0).stop 2023-11-26T07:16:27.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 -- 172.21.15.83:0/3612672188 shutdown_connections 2023-11-26T07:16:27.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd09806aff0 0x7fd09806d4b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:27.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d21c0 0x7fd0ac0b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:27.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0702e0 0x7fd0ac1f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:27.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 --2- 172.21.15.83:0/3612672188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0ac0cf040 0x7fd0ac14cc90 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:27.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 -- 172.21.15.83:0/3612672188 >> 172.21.15.83:0/3612672188 conn(0x7fd0ac09aa00 msgr2=0x7fd0ac0afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:27.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 -- 172.21.15.83:0/3612672188 shutdown_connections 2023-11-26T07:16:27.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.665+0000 7fd0b247c700 1 -- 172.21.15.83:0/3612672188 wait complete. 2023-11-26T07:16:27.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:16:27.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.799+0000 7feb3ee1e700 1 Processor -- start 2023-11-26T07:16:27.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.799+0000 7feb3ee1e700 1 -- start start 2023-11-26T07:16:27.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.800+0000 7feb3ee1e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb3806c650 0x7feb38165d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:27.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.800+0000 7feb3ee1e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3815f800 con 0x7feb380ce140 2023-11-26T07:16:27.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.800+0000 7feb3ee1e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb381414c0 con 0x7feb3815af10 2023-11-26T07:16:27.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.800+0000 7feb3ee1e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3815c300 con 0x7feb3806c650 2023-11-26T07:16:27.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.800+0000 7feb37fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7feb380ce140 0x7feb3815f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60806/0 (socket says 172.21.15.83:60806) 2023-11-26T07:16:27.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.800+0000 7feb3d3bb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feb3806c650 0x7feb38165d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:27.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.800+0000 7feb37fff700 1 -- 172.21.15.83:0/3118091866 learned_addr learned my addr 172.21.15.83:0/3118091866 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:27.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.801+0000 7feb3d3bb700 1 -- 172.21.15.83:0/3118091866 >> v1:172.21.15.203:6789/0 conn(0x7feb3815af10 legacy=0x7feb3815e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:27.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.801+0000 7feb3d3bb700 1 -- 172.21.15.83:0/3118091866 >> v1:172.21.15.83:6789/0 conn(0x7feb380ce140 legacy=0x7feb3815f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:27.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.801+0000 7feb3d3bb700 1 -- 172.21.15.83:0/3118091866 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380ce9f0 con 0x7feb3806c650 2023-11-26T07:16:27.803 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.801+0000 7feb377fe700 1 -- 172.21.15.83:0/3118091866 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395508318 0 0) 0x7feb3815f800 con 0x7feb380ce140 2023-11-26T07:16:27.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.801+0000 7feb3d3bb700 1 --2- 172.21.15.83:0/3118091866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb3806c650 0x7feb38165d70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7feb30009d90 tx=0x7feb3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f75415a3857ac96 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:27.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.801+0000 7feb377fe700 1 -- 172.21.15.83:0/3118091866 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3000e070 con 0x7feb3806c650 2023-11-26T07:16:27.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.801+0000 7feb377fe700 1 -- 172.21.15.83:0/3118091866 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3000bb90 con 0x7feb3806c650 2023-11-26T07:16:27.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.801+0000 7feb377fe700 1 -- 172.21.15.83:0/3118091866 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30003a20 con 0x7feb3806c650 2023-11-26T07:16:27.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.802+0000 7feb3ee1e700 1 -- 172.21.15.83:0/3118091866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb3806c650 msgr2=0x7feb38165d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:27.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.802+0000 7feb3ee1e700 1 --2- 172.21.15.83:0/3118091866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb3806c650 0x7feb38165d70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7feb30009d90 tx=0x7feb3000b330 comp rx=0 tx=0).stop 2023-11-26T07:16:27.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.802+0000 7feb3ee1e700 1 -- 172.21.15.83:0/3118091866 shutdown_connections 2023-11-26T07:16:27.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.802+0000 7feb3ee1e700 1 --2- 172.21.15.83:0/3118091866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb3806c650 0x7feb38165d70 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:27.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.802+0000 7feb3ee1e700 1 -- 172.21.15.83:0/3118091866 >> 172.21.15.83:0/3118091866 conn(0x7feb380cd570 msgr2=0x7feb380cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:27.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.802+0000 7feb3ee1e700 1 -- 172.21.15.83:0/3118091866 shutdown_connections 2023-11-26T07:16:27.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.802+0000 7feb3ee1e700 1 -- 172.21.15.83:0/3118091866 wait complete. 2023-11-26T07:16:27.807 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.803+0000 7feb3ee1e700 1 Processor -- start 2023-11-26T07:16:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.803+0000 7feb3ee1e700 1 -- start start 2023-11-26T07:16:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb3ee1e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb3806c650 0x7feb381342e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb3ee1e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb380ce140 0x7feb3813dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb3ee1e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb3815af10 0x7feb381337e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb3ee1e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb38154010 con 0x7feb380ce140 2023-11-26T07:16:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb3ee1e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb38135960 con 0x7feb3806c650 2023-11-26T07:16:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb3ee1e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb381348c0 con 0x7feb3815af10 2023-11-26T07:16:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb37fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb380ce140 0x7feb3813dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb37fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb380ce140 0x7feb3813dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40770/0 (socket says 172.21.15.83:40770) 2023-11-26T07:16:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb37fff700 1 -- 172.21.15.83:0/976390334 learned_addr learned my addr 172.21.15.83:0/976390334 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:27.808 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb37fff700 1 -- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb3815af10 msgr2=0x7feb381337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb37fff700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb3815af10 0x7feb381337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb3cbba700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb3806c650 0x7feb381342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb37fff700 1 -- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb3806c650 msgr2=0x7feb381342e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb37fff700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb3806c650 0x7feb381342e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb37fff700 1 -- 172.21.15.83:0/976390334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb30009a70 con 0x7feb380ce140 2023-11-26T07:16:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb37fff700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb380ce140 0x7feb3813dc10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c540 tx=0x7feb2400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb357fa700 1 -- 172.21.15.83:0/976390334 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24004c80 con 0x7feb380ce140 2023-11-26T07:16:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb357fa700 1 -- 172.21.15.83:0/976390334 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb24004de0 con 0x7feb380ce140 2023-11-26T07:16:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb3ee1e700 1 -- 172.21.15.83:0/976390334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb38134ba0 con 0x7feb380ce140 2023-11-26T07:16:27.809 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.804+0000 7feb357fa700 1 -- 172.21.15.83:0/976390334 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24007310 con 0x7feb380ce140 2023-11-26T07:16:27.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.805+0000 7feb3ee1e700 1 -- 172.21.15.83:0/976390334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb38135160 con 0x7feb380ce140 2023-11-26T07:16:27.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.806+0000 7feb3ee1e700 1 -- 172.21.15.83:0/976390334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb38067f60 con 0x7feb380ce140 2023-11-26T07:16:27.810 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.806+0000 7feb357fa700 1 -- 172.21.15.83:0/976390334 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7feb24007470 con 0x7feb380ce140 2023-11-26T07:16:27.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.806+0000 7feb357fa700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feb1c06aff0 0x7feb1c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:27.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.806+0000 7feb357fa700 1 -- 172.21.15.83:0/976390334 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb24089520 con 0x7feb380ce140 2023-11-26T07:16:27.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.807+0000 7feb3cbba700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feb1c06aff0 0x7feb1c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:27.811 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.807+0000 7feb3cbba700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feb1c06aff0 0x7feb1c06d4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb3806fc70 tx=0x7feb2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:27.812 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:27.811+0000 7feb357fa700 1 -- 172.21.15.83:0/976390334 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb240544c0 con 0x7feb380ce140 2023-11-26T07:16:28.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.029+0000 7feb3ee1e700 1 -- 172.21.15.83:0/976390334 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7feb380cefb0 con 0x7feb1c06aff0 2023-11-26T07:16:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.031+0000 7feb357fa700 1 -- 172.21.15.83:0/976390334 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7feb2c00a460 con 0x7feb1c06aff0 2023-11-26T07:16:28.033 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:16:28.033 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:28.033 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephfs" 2023-11-26T07:16:28.033 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:28.033 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:16:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 -- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feb1c06aff0 msgr2=0x7feb1c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feb1c06aff0 0x7feb1c06d4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb3806fc70 tx=0x7feb2c008040 comp rx=0 tx=0).stop 2023-11-26T07:16:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 -- 172.21.15.83:0/976390334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb380ce140 msgr2=0x7feb3813dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb380ce140 0x7feb3813dc10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c540 tx=0x7feb2400c900 comp rx=0 tx=0).stop 2023-11-26T07:16:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 -- 172.21.15.83:0/976390334 shutdown_connections 2023-11-26T07:16:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feb1c06aff0 0x7feb1c06d4b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb3815af10 0x7feb381337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb3806c650 0x7feb381342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 --2- 172.21.15.83:0/976390334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb380ce140 0x7feb3813dc10 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 -- 172.21.15.83:0/976390334 >> 172.21.15.83:0/976390334 conn(0x7feb380cd570 msgr2=0x7feb38071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 -- 172.21.15.83:0/976390334 shutdown_connections 2023-11-26T07:16:28.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.033+0000 7feb3ee1e700 1 -- 172.21.15.83:0/976390334 wait complete. 2023-11-26T07:16:28.047 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:16:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.169+0000 7f6a515e4700 1 Processor -- start 2023-11-26T07:16:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.169+0000 7f6a515e4700 1 -- start start 2023-11-26T07:16:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.169+0000 7f6a515e4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a4c15e140 0x7f6a4c0d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.169+0000 7f6a515e4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6a4c0d39d0 0x7f6a4c0d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.169+0000 7f6a515e4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a4c0b80e0 con 0x7f6a4c0b98c0 2023-11-26T07:16:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.169+0000 7f6a515e4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4c15e520 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.169+0000 7f6a515e4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c0d2930 con 0x7f6a4c15e140 2023-11-26T07:16:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a4b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6a4c0b98c0 0x7f6a4c0b59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60818/0 (socket says 172.21.15.83:60818) 2023-11-26T07:16:28.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a4a7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6a4c0d39d0 0x7f6a4c0d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a4b7fe700 1 -- 172.21.15.83:0/1654556780 learned_addr learned my addr 172.21.15.83:0/1654556780 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a49ffb700 1 -- 172.21.15.83:0/1654556780 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866929591 0 0) 0x7f6a44001490 con 0x7f6a4c0b98c0 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a49ffb700 1 -- 172.21.15.83:0/1654556780 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a340035b0 con 0x7f6a4c0b98c0 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a4affd700 1 --2- 172.21.15.83:0/1654556780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a4c15e140 0x7f6a4c0d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a4a7fc700 1 -- 172.21.15.83:0/1654556780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a4c15e140 msgr2=0x7f6a4c0d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a4a7fc700 1 --2- 172.21.15.83:0/1654556780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a4c15e140 0x7f6a4c0d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a4a7fc700 1 -- 172.21.15.83:0/1654556780 >> v1:172.21.15.83:6789/0 conn(0x7f6a4c0b98c0 legacy=0x7f6a4c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a4a7fc700 1 -- 172.21.15.83:0/1654556780 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c0d2c10 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a4affd700 1 --2- 172.21.15.83:0/1654556780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a4c15e140 0x7f6a4c0d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a4a7fc700 1 --2- 172.21.15.83:0/1654556780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a4c0d39d0 0x7f6a4c0d4400 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009840 tx=0x7f6a3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be5d753cda0a8f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a49ffb700 1 -- 172.21.15.83:0/1654556780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c011830 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a49ffb700 1 -- 172.21.15.83:0/1654556780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c011990 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.170+0000 7f6a49ffb700 1 -- 172.21.15.83:0/1654556780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c011d20 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.171+0000 7f6a515e4700 1 -- 172.21.15.83:0/1654556780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a4c0d39d0 msgr2=0x7f6a4c0d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.171+0000 7f6a515e4700 1 --2- 172.21.15.83:0/1654556780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a4c0d39d0 0x7f6a4c0d4400 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009840 tx=0x7f6a3c009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.171+0000 7f6a515e4700 1 -- 172.21.15.83:0/1654556780 shutdown_connections 2023-11-26T07:16:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.171+0000 7f6a515e4700 1 --2- 172.21.15.83:0/1654556780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a4c15e140 0x7f6a4c0d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.171+0000 7f6a515e4700 1 --2- 172.21.15.83:0/1654556780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a4c0d39d0 0x7f6a4c0d4400 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.171+0000 7f6a515e4700 1 -- 172.21.15.83:0/1654556780 >> 172.21.15.83:0/1654556780 conn(0x7f6a4c06b410 msgr2=0x7f6a4c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.171+0000 7f6a515e4700 1 -- 172.21.15.83:0/1654556780 shutdown_connections 2023-11-26T07:16:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.171+0000 7f6a515e4700 1 -- 172.21.15.83:0/1654556780 wait complete. 2023-11-26T07:16:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.172+0000 7f6a515e4700 1 Processor -- start 2023-11-26T07:16:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.172+0000 7f6a515e4700 1 -- start start 2023-11-26T07:16:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a515e4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a4c0b98c0 0x7f6a4c14ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a515e4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a4c0d39d0 0x7f6a4c0c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a515e4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a4c15e140 0x7f6a4c0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a515e4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c14f240 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a515e4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c0cb370 con 0x7f6a4c0b98c0 2023-11-26T07:16:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a515e4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c0b4380 con 0x7f6a4c15e140 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a4a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a4c0d39d0 0x7f6a4c0c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a4a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a4c0d39d0 0x7f6a4c0c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40784/0 (socket says 172.21.15.83:40784) 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a4a7fc700 1 -- 172.21.15.83:0/710099976 learned_addr learned my addr 172.21.15.83:0/710099976 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a4a7fc700 1 -- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a4c15e140 msgr2=0x7f6a4c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a4a7fc700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a4c15e140 0x7f6a4c0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a4a7fc700 1 -- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a4c0b98c0 msgr2=0x7f6a4c14ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a4a7fc700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a4c0b98c0 0x7f6a4c14ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a4a7fc700 1 -- 172.21.15.83:0/710099976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c0094a0 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.173+0000 7f6a4a7fc700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a4c0d39d0 0x7f6a4c0c78b0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0090b0 tx=0x7f6a3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.174+0000 7f6a337fe700 1 -- 172.21.15.83:0/710099976 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0036a0 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.174+0000 7f6a337fe700 1 -- 172.21.15.83:0/710099976 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c01f070 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.174+0000 7f6a515e4700 1 -- 172.21.15.83:0/710099976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4c0c8220 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.174+0000 7f6a337fe700 1 -- 172.21.15.83:0/710099976 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c003a20 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.174+0000 7f6a515e4700 1 -- 172.21.15.83:0/710099976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4c0c87e0 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.175+0000 7f6a515e4700 1 -- 172.21.15.83:0/710099976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4c068fb0 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.175+0000 7f6a337fe700 1 -- 172.21.15.83:0/710099976 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f6a3c003b80 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.175+0000 7f6a337fe700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6a3806aff0 0x7f6a3806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.175+0000 7f6a337fe700 1 -- 172.21.15.83:0/710099976 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a3c021070 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.176+0000 7f6a4affd700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6a3806aff0 0x7f6a3806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.176+0000 7f6a4affd700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6a3806aff0 0x7f6a3806d4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000a8b0 tx=0x7f6a40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:28.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.180+0000 7f6a337fe700 1 -- 172.21.15.83:0/710099976 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a3c056100 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.368+0000 7f6a515e4700 1 -- 172.21.15.83:0/710099976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6a4c15d940 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.368+0000 7f6a337fe700 1 -- 172.21.15.83:0/710099976 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6a3c059720 con 0x7f6a4c0d39d0 2023-11-26T07:16:28.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.370+0000 7f6a515e4700 1 -- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6a3806aff0 msgr2=0x7f6a3806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.370+0000 7f6a515e4700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6a3806aff0 0x7f6a3806d4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000a8b0 tx=0x7f6a40008040 comp rx=0 tx=0).stop 2023-11-26T07:16:28.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.371+0000 7f6a515e4700 1 -- 172.21.15.83:0/710099976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a4c0d39d0 msgr2=0x7f6a4c0c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.371+0000 7f6a515e4700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a4c0d39d0 0x7f6a4c0c78b0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0090b0 tx=0x7f6a3c004750 comp rx=0 tx=0).stop 2023-11-26T07:16:28.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.371+0000 7f6a515e4700 1 -- 172.21.15.83:0/710099976 shutdown_connections 2023-11-26T07:16:28.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.371+0000 7f6a515e4700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6a3806aff0 0x7f6a3806d4b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.371+0000 7f6a515e4700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a4c15e140 0x7f6a4c0cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.371+0000 7f6a515e4700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a4c0b98c0 0x7f6a4c14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.371+0000 7f6a515e4700 1 --2- 172.21.15.83:0/710099976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a4c0d39d0 0x7f6a4c0c78b0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.371+0000 7f6a515e4700 1 -- 172.21.15.83:0/710099976 >> 172.21.15.83:0/710099976 conn(0x7f6a4c06b410 msgr2=0x7f6a4c15f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:28.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.371+0000 7f6a515e4700 1 -- 172.21.15.83:0/710099976 shutdown_connections 2023-11-26T07:16:28.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.371+0000 7f6a515e4700 1 -- 172.21.15.83:0/710099976 wait complete. 2023-11-26T07:16:28.384 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2023-11-26T07:16:28.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.504+0000 7f7866cc4700 1 Processor -- start 2023-11-26T07:16:28.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.504+0000 7f7866cc4700 1 -- start start 2023-11-26T07:16:28.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.505+0000 7f7866cc4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f786006e370 0x7f786015b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.505+0000 7f7866cc4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f786006b0f0 0x7f786006b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.505+0000 7f7866cc4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786015e9f0 con 0x7f78600b49a0 2023-11-26T07:16:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.505+0000 7f7866cc4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786015ec90 con 0x7f786006e370 2023-11-26T07:16:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.505+0000 7f7866cc4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786015ef70 con 0x7f786006b0f0 2023-11-26T07:16:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.505+0000 7f785ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f786006e370 0x7f786015b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.505+0000 7f785ffff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f786006e370 0x7f786015b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40790/0 (socket says 172.21.15.83:40790) 2023-11-26T07:16:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.505+0000 7f785ffff700 1 -- 172.21.15.83:0/2323422691 learned_addr learned my addr 172.21.15.83:0/2323422691 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.505+0000 7f7865261700 1 --2- 172.21.15.83:0/2323422691 >> v2:172.21.15.203:3300/0 conn(0x7f786006b0f0 0x7f786006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f785ffff700 1 -- 172.21.15.83:0/2323422691 >> v2:172.21.15.203:3300/0 conn(0x7f786006b0f0 msgr2=0x7f786006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f785ffff700 1 --2- 172.21.15.83:0/2323422691 >> v2:172.21.15.203:3300/0 conn(0x7f786006b0f0 0x7f786006b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f785ffff700 1 -- 172.21.15.83:0/2323422691 >> v1:172.21.15.203:6789/0 conn(0x7f78600b49a0 legacy=0x7f78600b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f785ffff700 1 -- 172.21.15.83:0/2323422691 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786015f250 con 0x7f786006e370 2023-11-26T07:16:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f7865261700 1 --2- 172.21.15.83:0/2323422691 >> v2:172.21.15.203:3300/0 conn(0x7f786006b0f0 0x7f786006b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f785ffff700 1 --2- 172.21.15.83:0/2323422691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f786006e370 0x7f786015b210 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f786015c3b0 tx=0x7f784c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256afaf90f5aa0a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f785f7fe700 1 -- 172.21.15.83:0/2323422691 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c00dcf0 con 0x7f786006e370 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f785f7fe700 1 -- 172.21.15.83:0/2323422691 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784c00de50 con 0x7f786006e370 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f785f7fe700 1 -- 172.21.15.83:0/2323422691 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c004250 con 0x7f786006e370 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f7866cc4700 1 -- 172.21.15.83:0/2323422691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f786006e370 msgr2=0x7f786015b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f7866cc4700 1 --2- 172.21.15.83:0/2323422691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f786006e370 0x7f786015b210 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f786015c3b0 tx=0x7f784c00be70 comp rx=0 tx=0).stop 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f7866cc4700 1 -- 172.21.15.83:0/2323422691 shutdown_connections 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f7866cc4700 1 --2- 172.21.15.83:0/2323422691 >> v2:172.21.15.203:3300/0 conn(0x7f786006b0f0 0x7f786006b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f7866cc4700 1 --2- 172.21.15.83:0/2323422691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f786006e370 0x7f786015b210 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f7866cc4700 1 -- 172.21.15.83:0/2323422691 >> 172.21.15.83:0/2323422691 conn(0x7f78600b28f0 msgr2=0x7f78600b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f7866cc4700 1 -- 172.21.15.83:0/2323422691 shutdown_connections 2023-11-26T07:16:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.506+0000 7f7866cc4700 1 -- 172.21.15.83:0/2323422691 wait complete. 2023-11-26T07:16:28.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.508+0000 7f7866cc4700 1 Processor -- start 2023-11-26T07:16:28.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.508+0000 7f7866cc4700 1 -- start start 2023-11-26T07:16:28.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.508+0000 7f7866cc4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f786006b0f0 0x7f7860159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.508+0000 7f7866cc4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f786006e370 0x7f786015a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.508+0000 7f7866cc4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78600b49a0 0x7f7860156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.508+0000 7f7866cc4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78601f01f0 con 0x7f786006b0f0 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7866cc4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78601ef8c0 con 0x7f786006e370 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7866cc4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78601efba0 con 0x7f78600b49a0 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7864a60700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f786006b0f0 0x7f7860159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7864a60700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f786006b0f0 0x7f7860159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40796/0 (socket says 172.21.15.83:40796) 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7864a60700 1 -- 172.21.15.83:0/1538988968 learned_addr learned my addr 172.21.15.83:0/1538988968 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f785ffff700 1 --2- 172.21.15.83:0/1538988968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f786006e370 0x7f786015a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7865261700 1 --2- 172.21.15.83:0/1538988968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78600b49a0 0x7f7860156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7864a60700 1 -- 172.21.15.83:0/1538988968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78600b49a0 msgr2=0x7f7860156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7864a60700 1 --2- 172.21.15.83:0/1538988968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78600b49a0 0x7f7860156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7864a60700 1 -- 172.21.15.83:0/1538988968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f786006e370 msgr2=0x7f786015a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7864a60700 1 --2- 172.21.15.83:0/1538988968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f786006e370 0x7f786015a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7864a60700 1 -- 172.21.15.83:0/1538988968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c00b7f0 con 0x7f786006b0f0 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7864a60700 1 --2- 172.21.15.83:0/1538988968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f786006b0f0 0x7f7860159fc0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f785400c4d0 tx=0x7f785400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f785d7fa700 1 -- 172.21.15.83:0/1538988968 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854011790 con 0x7f786006b0f0 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f785d7fa700 1 -- 172.21.15.83:0/1538988968 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78540118f0 con 0x7f786006b0f0 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7866cc4700 1 -- 172.21.15.83:0/1538988968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78601efe80 con 0x7f786006b0f0 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f785d7fa700 1 -- 172.21.15.83:0/1538988968 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854011ca0 con 0x7f786006b0f0 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.509+0000 7f7866cc4700 1 -- 172.21.15.83:0/1538988968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7860157070 con 0x7f786006b0f0 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.510+0000 7f7866cc4700 1 -- 172.21.15.83:0/1538988968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7860068fb0 con 0x7f786006b0f0 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.511+0000 7f785d7fa700 1 -- 172.21.15.83:0/1538988968 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f785401c460 con 0x7f786006b0f0 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.511+0000 7f785d7fa700 1 --2- 172.21.15.83:0/1538988968 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f784406b020 0x7f784406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.511+0000 7f785d7fa700 1 -- 172.21.15.83:0/1538988968 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7854089530 con 0x7f786006b0f0 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.511+0000 7f785ffff700 1 --2- 172.21.15.83:0/1538988968 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f784406b020 0x7f784406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.512+0000 7f785ffff700 1 --2- 172.21.15.83:0/1538988968 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f784406b020 0x7f784406d4e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f786015c3b0 tx=0x7f784c012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.515+0000 7f785d7fa700 1 -- 172.21.15.83:0/1538988968 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7854054560 con 0x7f786006b0f0 2023-11-26T07:16:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:Invalid command: invalid chars #@ in abcd@# 2023-11-26T07:16:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:fs volume create [] : Create a CephFS volume 2023-11-26T07:16:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2023-11-26T07:16:28.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-26T07:16:28.738 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2023-11-26T07:16:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.858+0000 7f4ddfbcb700 1 Processor -- start 2023-11-26T07:16:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.858+0000 7f4ddfbcb700 1 -- start start 2023-11-26T07:16:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.859+0000 7f4ddfbcb700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4dd815bf50 0x7f4dd80ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.859+0000 7f4ddfbcb700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd80d0140 con 0x7f4dd80ce830 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.859+0000 7f4ddfbcb700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd80cf3c0 con 0x7f4dd80c6c30 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.859+0000 7f4ddfbcb700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd815c330 con 0x7f4dd815bf50 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.859+0000 7f4ddd166700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4dd80ce830 0x7f4dd80cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60844/0 (socket says 172.21.15.83:60844) 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.859+0000 7f4ddd166700 1 -- 172.21.15.83:0/670008216 learned_addr learned my addr 172.21.15.83:0/670008216 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.860+0000 7f4ddd967700 1 --2- 172.21.15.83:0/670008216 >> v2:172.21.15.203:3300/0 conn(0x7f4dd815bf50 0x7f4dd80ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.860+0000 7f4ddc965700 1 -- 172.21.15.83:0/670008216 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559515144 0 0) 0x7f4dd80d0140 con 0x7f4dd80ce830 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.860+0000 7f4ddc965700 1 -- 172.21.15.83:0/670008216 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dc40035b0 con 0x7f4dd80ce830 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.860+0000 7f4ddd967700 1 -- 172.21.15.83:0/670008216 >> v1:172.21.15.203:6789/0 conn(0x7f4dd80c6c30 legacy=0x7f4dd80cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.860+0000 7f4ddc965700 1 -- 172.21.15.83:0/670008216 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3386445419 0 0) 0x7f4dc40035b0 con 0x7f4dd80ce830 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.860+0000 7f4ddd967700 1 -- 172.21.15.83:0/670008216 >> v1:172.21.15.83:6789/0 conn(0x7f4dd80ce830 legacy=0x7f4dd80cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.860+0000 7f4ddd967700 1 -- 172.21.15.83:0/670008216 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd80d03e0 con 0x7f4dd815bf50 2023-11-26T07:16:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.860+0000 7f4ddd967700 1 --2- 172.21.15.83:0/670008216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd815bf50 0x7f4dd80ce2f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009840 tx=0x7f4dc8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c728c703efa3efef server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.860+0000 7f4ddc965700 1 -- 172.21.15.83:0/670008216 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4001670 con 0x7f4dd815bf50 2023-11-26T07:16:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.860+0000 7f4ddc965700 1 -- 172.21.15.83:0/670008216 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc8004030 con 0x7f4dd815bf50 2023-11-26T07:16:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.860+0000 7f4ddc965700 1 -- 172.21.15.83:0/670008216 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80043a0 con 0x7f4dd815bf50 2023-11-26T07:16:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.861+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/670008216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd815bf50 msgr2=0x7f4dd80ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.861+0000 7f4ddfbcb700 1 --2- 172.21.15.83:0/670008216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd815bf50 0x7f4dd80ce2f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009840 tx=0x7f4dc8009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.861+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/670008216 shutdown_connections 2023-11-26T07:16:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.861+0000 7f4ddfbcb700 1 --2- 172.21.15.83:0/670008216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd815bf50 0x7f4dd80ce2f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.861+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/670008216 >> 172.21.15.83:0/670008216 conn(0x7f4dd80b3780 msgr2=0x7f4dd80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.861+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/670008216 shutdown_connections 2023-11-26T07:16:28.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.861+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/670008216 wait complete. 2023-11-26T07:16:28.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.862+0000 7f4ddfbcb700 1 Processor -- start 2023-11-26T07:16:28.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.862+0000 7f4ddfbcb700 1 -- start start 2023-11-26T07:16:28.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4ddfbcb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd80c6c30 0x7f4dd81f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4ddfbcb700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dd80ce830 0x7f4dd8158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4ddfbcb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dd815bf50 0x7f4dd80cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4ddfbcb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd81f1c00 con 0x7f4dd815bf50 2023-11-26T07:16:28.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4ddfbcb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80b5a00 con 0x7f4dd80c6c30 2023-11-26T07:16:28.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4ddfbcb700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80b5ce0 con 0x7f4dd80ce830 2023-11-26T07:16:28.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4dde168700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dd815bf50 0x7f4dd80cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4dde168700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dd815bf50 0x7f4dd80cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40814/0 (socket says 172.21.15.83:40814) 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4dde168700 1 -- 172.21.15.83:0/2460957870 learned_addr learned my addr 172.21.15.83:0/2460957870 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4ddd967700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd80c6c30 0x7f4dd81f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4ddd166700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dd80ce830 0x7f4dd8158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4dde168700 1 -- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dd80ce830 msgr2=0x7f4dd8158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4dde168700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dd80ce830 0x7f4dd8158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4dde168700 1 -- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd80c6c30 msgr2=0x7f4dd81f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4dde168700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd80c6c30 0x7f4dd81f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4dde168700 1 -- 172.21.15.83:0/2460957870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc80094a0 con 0x7f4dd815bf50 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4ddd166700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dd80ce830 0x7f4dd8158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.863+0000 7f4dde168700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dd815bf50 0x7f4dd80cac90 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000cbc0 tx=0x7f4dd000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.864+0000 7f4dce7fc700 1 -- 172.21.15.83:0/2460957870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0003c00 con 0x7f4dd815bf50 2023-11-26T07:16:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.864+0000 7f4dce7fc700 1 -- 172.21.15.83:0/2460957870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd0003d60 con 0x7f4dd815bf50 2023-11-26T07:16:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.864+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/2460957870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd80b5fc0 con 0x7f4dd815bf50 2023-11-26T07:16:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.864+0000 7f4dce7fc700 1 -- 172.21.15.83:0/2460957870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000abf0 con 0x7f4dd815bf50 2023-11-26T07:16:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.864+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/2460957870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd80b6580 con 0x7f4dd815bf50 2023-11-26T07:16:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.865+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/2460957870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd80b6710 con 0x7f4dd815bf50 2023-11-26T07:16:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.865+0000 7f4dce7fc700 1 -- 172.21.15.83:0/2460957870 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f4dd000ad50 con 0x7f4dd815bf50 2023-11-26T07:16:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.865+0000 7f4dce7fc700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4dbc06aff0 0x7f4dbc06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.865+0000 7f4dce7fc700 1 -- 172.21.15.83:0/2460957870 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4dd0089900 con 0x7f4dd815bf50 2023-11-26T07:16:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.866+0000 7f4ddd967700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4dbc06aff0 0x7f4dbc06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.866+0000 7f4ddd967700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4dbc06aff0 0x7f4dbc06d4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80090b0 tx=0x7f4dc8013040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:28.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:28.870+0000 7f4dce7fc700 1 -- 172.21.15.83:0/2460957870 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4dd0054810 con 0x7f4dd815bf50 2023-11-26T07:16:29.086 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:29.085+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/2460957870 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dd80c5700 con 0x7f4dbc06aff0 2023-11-26T07:16:31.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.707+0000 7f4dce7fc700 1 -- 172.21.15.83:0/2460957870 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f4dc80036a0 con 0x7f4dbc06aff0 2023-11-26T07:16:31.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4dbc06aff0 msgr2=0x7f4dbc06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:31.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4dbc06aff0 0x7f4dbc06d4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80090b0 tx=0x7f4dc8013040 comp rx=0 tx=0).stop 2023-11-26T07:16:31.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/2460957870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dd815bf50 msgr2=0x7f4dd80cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:31.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dd815bf50 0x7f4dd80cac90 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000cbc0 tx=0x7f4dd000cf80 comp rx=0 tx=0).stop 2023-11-26T07:16:31.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/2460957870 shutdown_connections 2023-11-26T07:16:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4dbc06aff0 0x7f4dbc06d4b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dd815bf50 0x7f4dd80cac90 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4dd80c6c30 0x7f4dd81f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 --2- 172.21.15.83:0/2460957870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4dd80ce830 0x7f4dd8158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/2460957870 >> 172.21.15.83:0/2460957870 conn(0x7f4dd80b3780 msgr2=0x7f4dd80b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/2460957870 shutdown_connections 2023-11-26T07:16:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.710+0000 7f4ddfbcb700 1 -- 172.21.15.83:0/2460957870 wait complete. 2023-11-26T07:16:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:Volume created successfully (no MDS daemons created) 2023-11-26T07:16:31.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2023-11-26T07:16:31.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.853+0000 7f793028e700 1 Processor -- start 2023-11-26T07:16:31.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.854+0000 7f793028e700 1 -- start start 2023-11-26T07:16:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.854+0000 7f793028e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f79280cf610 0x7f79280add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.854+0000 7f793028e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792815f9e0 0x7f79280b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.854+0000 7f793028e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f792806b080 con 0x7f7928073f90 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.854+0000 7f793028e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792815c290 con 0x7f79280cf610 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.854+0000 7f793028e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79280cf9f0 con 0x7f792815f9e0 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792e82b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7928073f90 0x7f792806b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60876/0 (socket says 172.21.15.83:60876) 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792e82b700 1 -- 172.21.15.83:0/2391196339 learned_addr learned my addr 172.21.15.83:0/2391196339 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792e02a700 1 --2- 172.21.15.83:0/2391196339 >> v2:172.21.15.203:3300/0 conn(0x7f79280cf610 0x7f79280add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141404165 0 0) 0x7f792806b080 con 0x7f7928073f90 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79140035b0 con 0x7f7928073f90 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d829700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792815f9e0 0x7f79280b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3361853573 0 0) 0x7f79140035b0 con 0x7f7928073f90 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792815f9e0 msgr2=0x7f79280b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792815f9e0 0x7f79280b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 >> v2:172.21.15.203:3300/0 conn(0x7f79280cf610 msgr2=0x7f79280add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 --2- 172.21.15.83:0/2391196339 >> v2:172.21.15.203:3300/0 conn(0x7f79280cf610 0x7f79280add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792815ff80 con 0x7f7928073f90 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f79200019f0 con 0x7f7928073f90 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 >> v1:172.21.15.83:6789/0 conn(0x7f7928073f90 legacy=0x7f792806b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7914003a00 0x7f7914003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7914004850 0x7f7914004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792815f9e0 0x7f7914005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792815c290 con 0x7f7914004850 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79280cf9f0 con 0x7f7914003a00 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791400ba10 con 0x7f792815f9e0 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792d829700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7914004850 0x7f7914004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:31.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.855+0000 7f792e02a700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7914003a00 0x7f7914003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f792e82b700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792815f9e0 0x7f7914005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f792d829700 1 -- 172.21.15.83:0/2391196339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792815f9e0 msgr2=0x7f7914005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f792d829700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792815f9e0 0x7f7914005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f792d829700 1 -- 172.21.15.83:0/2391196339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7914003a00 msgr2=0x7f7914003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f792d829700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7914003a00 0x7f7914003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f792d829700 1 -- 172.21.15.83:0/2391196339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79140035b0 con 0x7f7914004850 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f792d829700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7914004850 0x7f7914004cd0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f7924009b60 tx=0x7f7914005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be60bbb68807b85f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792400e6c0 con 0x7f7914004850 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792400e820 con 0x7f7914004850 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f792d028700 1 -- 172.21.15.83:0/2391196339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792400eb90 con 0x7f7914004850 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f793028e700 1 -- 172.21.15.83:0/2391196339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7914004850 msgr2=0x7f7914004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f793028e700 1 --2- 172.21.15.83:0/2391196339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7914004850 0x7f7914004cd0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f7924009b60 tx=0x7f7914005880 comp rx=0 tx=0).stop 2023-11-26T07:16:31.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f792e82b700 1 -- 172.21.15.83:0/2391196339 reap_dead start 2023-11-26T07:16:31.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f793028e700 1 -- 172.21.15.83:0/2391196339 shutdown_connections 2023-11-26T07:16:31.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.856+0000 7f793028e700 1 -- 172.21.15.83:0/2391196339 >> 172.21.15.83:0/2391196339 conn(0x7f79280d0bf0 msgr2=0x7f79280d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:31.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.857+0000 7f793028e700 1 -- 172.21.15.83:0/2391196339 shutdown_connections 2023-11-26T07:16:31.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.857+0000 7f793028e700 1 -- 172.21.15.83:0/2391196339 wait complete. 2023-11-26T07:16:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.858+0000 7f793028e700 1 Processor -- start 2023-11-26T07:16:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.858+0000 7f793028e700 1 -- start start 2023-11-26T07:16:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.858+0000 7f793028e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7914004850 0x7f792806e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.858+0000 7f793028e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f792815c460 0x7f792815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.858+0000 7f793028e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792806f150 0x7f792814a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.858+0000 7f793028e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792815d630 con 0x7f7914004850 2023-11-26T07:16:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f793028e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792806ea60 con 0x7f792815c460 2023-11-26T07:16:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f793028e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792815ca20 con 0x7f792806f150 2023-11-26T07:16:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f792e02a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7914004850 0x7f792806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f792e02a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7914004850 0x7f792806e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40844/0 (socket says 172.21.15.83:40844) 2023-11-26T07:16:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f792e02a700 1 -- 172.21.15.83:0/3924739106 learned_addr learned my addr 172.21.15.83:0/3924739106 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f792d829700 1 --2- 172.21.15.83:0/3924739106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f792815c460 0x7f792815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f792e82b700 1 --2- 172.21.15.83:0/3924739106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792806f150 0x7f792814a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f792e02a700 1 -- 172.21.15.83:0/3924739106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792806f150 msgr2=0x7f792814a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f792e02a700 1 --2- 172.21.15.83:0/3924739106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f792806f150 0x7f792814a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f792e02a700 1 -- 172.21.15.83:0/3924739106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f792815c460 msgr2=0x7f792815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f792e02a700 1 --2- 172.21.15.83:0/3924739106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f792815c460 0x7f792815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f792e02a700 1 -- 172.21.15.83:0/3924739106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792815cd00 con 0x7f7914004850 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f792e02a700 1 --2- 172.21.15.83:0/3924739106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7914004850 0x7f792806e520 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7918011aa0 tx=0x7f7918011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f791effd700 1 -- 172.21.15.83:0/3924739106 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791800f940 con 0x7f7914004850 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f791effd700 1 -- 172.21.15.83:0/3924739106 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791800faa0 con 0x7f7914004850 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f791effd700 1 -- 172.21.15.83:0/3924739106 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791800fe50 con 0x7f7914004850 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f793028e700 1 -- 172.21.15.83:0/3924739106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792815cfe0 con 0x7f7914004850 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.859+0000 7f793028e700 1 -- 172.21.15.83:0/3924739106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792815e000 con 0x7f7914004850 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.860+0000 7f793028e700 1 -- 172.21.15.83:0/3924739106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792815d910 con 0x7f7914004850 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.861+0000 7f791effd700 1 -- 172.21.15.83:0/3924739106 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f7918014440 con 0x7f7914004850 2023-11-26T07:16:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.861+0000 7f791effd700 1 --2- 172.21.15.83:0/3924739106 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f790c06aff0 0x7f790c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.861+0000 7f791effd700 1 -- 172.21.15.83:0/3924739106 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f791808a220 con 0x7f7914004850 2023-11-26T07:16:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.861+0000 7f792d829700 1 --2- 172.21.15.83:0/3924739106 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f790c06aff0 0x7f790c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.862+0000 7f792d829700 1 --2- 172.21.15.83:0/3924739106 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f790c06aff0 0x7f790c06d4b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f79240090b0 tx=0x7f792401f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:31.866+0000 7f791effd700 1 -- 172.21.15.83:0/3924739106 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7918054f10 con 0x7f7914004850 2023-11-26T07:16:32.082 INFO:teuthology.orchestra.run.smithi083.stderr:Invalid command: invalid chars #@ in abcd@# 2023-11-26T07:16:32.082 INFO:teuthology.orchestra.run.smithi083.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2023-11-26T07:16:32.083 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2023-11-26T07:16:32.086 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-26T07:16:32.086 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2023-11-26T07:16:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.207+0000 7f082aef8700 1 Processor -- start 2023-11-26T07:16:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.208+0000 7f082aef8700 1 -- start start 2023-11-26T07:16:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.208+0000 7f082aef8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f082415e160 0x7f08240b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.208+0000 7f082aef8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240afc90 0x7f0824073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.208+0000 7f082aef8700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f08240744d0 0x7f082406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.208+0000 7f082aef8700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08240b4500 con 0x7f08240744d0 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.208+0000 7f082aef8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0824074960 con 0x7f082415e160 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.208+0000 7f082aef8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08240b3c60 con 0x7f08240afc90 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0829495700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f08240744d0 0x7f082406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0829495700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f08240744d0 0x7f082406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40856/0 (socket says 172.21.15.83:40856) 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0829495700 1 -- 172.21.15.83:0/3772765337 learned_addr learned my addr 172.21.15.83:0/3772765337 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0828c94700 1 --2- 172.21.15.83:0/3772765337 >> v2:172.21.15.203:3300/0 conn(0x7f082415e160 0x7f08240b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0823fff700 1 --2- 172.21.15.83:0/3772765337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240afc90 0x7f0824073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0829495700 1 -- 172.21.15.83:0/3772765337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240afc90 msgr2=0x7f0824073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0829495700 1 --2- 172.21.15.83:0/3772765337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240afc90 0x7f0824073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0829495700 1 -- 172.21.15.83:0/3772765337 >> v2:172.21.15.203:3300/0 conn(0x7f082415e160 msgr2=0x7f08240b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0829495700 1 --2- 172.21.15.83:0/3772765337 >> v2:172.21.15.203:3300/0 conn(0x7f082415e160 0x7f08240b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0829495700 1 -- 172.21.15.83:0/3772765337 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08240b58c0 con 0x7f08240744d0 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0828c94700 1 --2- 172.21.15.83:0/3772765337 >> v2:172.21.15.203:3300/0 conn(0x7f082415e160 0x7f08240b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f0829495700 1 --2- 172.21.15.83:0/3772765337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240744d0 0x7f082406d930 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f082409efa0 tx=0x7f081c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb615c857c7fdc7c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f08237fe700 1 -- 172.21.15.83:0/3772765337 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c00e070 con 0x7f08240744d0 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f08237fe700 1 -- 172.21.15.83:0/3772765337 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081c00b9c0 con 0x7f08240744d0 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.209+0000 7f08237fe700 1 -- 172.21.15.83:0/3772765337 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c00bd70 con 0x7f08240744d0 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.210+0000 7f082aef8700 1 -- 172.21.15.83:0/3772765337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240744d0 msgr2=0x7f082406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.210+0000 7f082aef8700 1 --2- 172.21.15.83:0/3772765337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240744d0 0x7f082406d930 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f082409efa0 tx=0x7f081c00b330 comp rx=0 tx=0).stop 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.210+0000 7f082aef8700 1 -- 172.21.15.83:0/3772765337 shutdown_connections 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.210+0000 7f082aef8700 1 --2- 172.21.15.83:0/3772765337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240744d0 0x7f082406d930 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.210+0000 7f082aef8700 1 --2- 172.21.15.83:0/3772765337 >> v2:172.21.15.203:3300/0 conn(0x7f082415e160 0x7f08240b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.210+0000 7f082aef8700 1 --2- 172.21.15.83:0/3772765337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240afc90 0x7f0824073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.210+0000 7f082aef8700 1 -- 172.21.15.83:0/3772765337 >> 172.21.15.83:0/3772765337 conn(0x7f08240d0bf0 msgr2=0x7f08240d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.210+0000 7f082aef8700 1 -- 172.21.15.83:0/3772765337 shutdown_connections 2023-11-26T07:16:32.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.210+0000 7f082aef8700 1 -- 172.21.15.83:0/3772765337 wait complete. 2023-11-26T07:16:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.211+0000 7f082aef8700 1 Processor -- start 2023-11-26T07:16:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.211+0000 7f082aef8700 1 -- start start 2023-11-26T07:16:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f082aef8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240744d0 0x7f08240b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f082aef8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240afc90 0x7f0824146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f082aef8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f082415e160 0x7f0824153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f082aef8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0824155de0 con 0x7f08240744d0 2023-11-26T07:16:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f082aef8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0824153560 con 0x7f082415e160 2023-11-26T07:16:32.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f082aef8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0824155ff0 con 0x7f08240afc90 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0828c94700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240744d0 0x7f08240b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0828c94700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240744d0 0x7f08240b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40858/0 (socket says 172.21.15.83:40858) 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0828c94700 1 -- 172.21.15.83:0/1376133670 learned_addr learned my addr 172.21.15.83:0/1376133670 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0829495700 1 --2- 172.21.15.83:0/1376133670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f082415e160 0x7f0824153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0823fff700 1 --2- 172.21.15.83:0/1376133670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240afc90 0x7f0824146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0828c94700 1 -- 172.21.15.83:0/1376133670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240afc90 msgr2=0x7f0824146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0828c94700 1 --2- 172.21.15.83:0/1376133670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08240afc90 0x7f0824146a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0828c94700 1 -- 172.21.15.83:0/1376133670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f082415e160 msgr2=0x7f0824153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0828c94700 1 --2- 172.21.15.83:0/1376133670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f082415e160 0x7f0824153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0828c94700 1 -- 172.21.15.83:0/1376133670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c009b30 con 0x7f08240744d0 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0829495700 1 --2- 172.21.15.83:0/1376133670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f082415e160 0x7f0824153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f0828c94700 1 --2- 172.21.15.83:0/1376133670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08240744d0 0x7f08240b23b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f081800c820 tx=0x7f081800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:32.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.212+0000 7f08217fa700 1 -- 172.21.15.83:0/1376133670 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818011800 con 0x7f08240744d0 2023-11-26T07:16:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.213+0000 7f08217fa700 1 -- 172.21.15.83:0/1376133670 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0818011960 con 0x7f08240744d0 2023-11-26T07:16:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.213+0000 7f082aef8700 1 -- 172.21.15.83:0/1376133670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08241562d0 con 0x7f08240744d0 2023-11-26T07:16:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.213+0000 7f08217fa700 1 -- 172.21.15.83:0/1376133670 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818011d10 con 0x7f08240744d0 2023-11-26T07:16:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.213+0000 7f082aef8700 1 -- 172.21.15.83:0/1376133670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082415ef70 con 0x7f08240744d0 2023-11-26T07:16:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.214+0000 7f082aef8700 1 -- 172.21.15.83:0/1376133670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0824068fb0 con 0x7f08240744d0 2023-11-26T07:16:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.214+0000 7f08217fa700 1 -- 172.21.15.83:0/1376133670 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f0818018360 con 0x7f08240744d0 2023-11-26T07:16:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.214+0000 7f08217fa700 1 --2- 172.21.15.83:0/1376133670 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f080806aff0 0x7f080806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.215+0000 7f08217fa700 1 -- 172.21.15.83:0/1376133670 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f0818089af0 con 0x7f08240744d0 2023-11-26T07:16:32.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.215+0000 7f0823fff700 1 --2- 172.21.15.83:0/1376133670 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f080806aff0 0x7f080806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.215+0000 7f0823fff700 1 --2- 172.21.15.83:0/1376133670 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f080806aff0 0x7f080806d4b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f08240d0280 tx=0x7f0810008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:32.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.219+0000 7f08217fa700 1 -- 172.21.15.83:0/1376133670 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0818054650 con 0x7f08240744d0 2023-11-26T07:16:32.443 INFO:teuthology.orchestra.run.smithi083.stderr:Invalid command: invalid chars @# in abcd@# 2023-11-26T07:16:32.443 INFO:teuthology.orchestra.run.smithi083.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2023-11-26T07:16:32.443 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: invalid command 2023-11-26T07:16:32.446 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-26T07:16:32.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2023-11-26T07:16:32.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.568+0000 7f771d3d7700 1 Processor -- start 2023-11-26T07:16:32.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.568+0000 7f771d3d7700 1 -- start start 2023-11-26T07:16:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.569+0000 7f771d3d7700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f771815bec0 0x7f771806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.569+0000 7f771d3d7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77180702e0 0x7f77180d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.569+0000 7f771d3d7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77180cc4a0 con 0x7f77180d2000 2023-11-26T07:16:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.569+0000 7f771d3d7700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7718070770 con 0x7f77180702e0 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.569+0000 7f771d3d7700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77180d03e0 con 0x7f771815bec0 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.569+0000 7f77167fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77180702e0 0x7f77180d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.569+0000 7f77167fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77180702e0 0x7f77180d1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40868/0 (socket says 172.21.15.83:40868) 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.569+0000 7f77167fc700 1 -- 172.21.15.83:0/3126462969 learned_addr learned my addr 172.21.15.83:0/3126462969 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.570+0000 7f7716ffd700 1 --2- 172.21.15.83:0/3126462969 >> v2:172.21.15.203:3300/0 conn(0x7f771815bec0 0x7f771806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.570+0000 7f77167fc700 1 -- 172.21.15.83:0/3126462969 >> v2:172.21.15.203:3300/0 conn(0x7f771815bec0 msgr2=0x7f771806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.570+0000 7f77167fc700 1 --2- 172.21.15.83:0/3126462969 >> v2:172.21.15.203:3300/0 conn(0x7f771815bec0 0x7f771806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.570+0000 7f77167fc700 1 -- 172.21.15.83:0/3126462969 >> v1:172.21.15.203:6789/0 conn(0x7f77180d2000 legacy=0x7f771815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.570+0000 7f77167fc700 1 -- 172.21.15.83:0/3126462969 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180d2430 con 0x7f77180702e0 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.570+0000 7f77167fc700 1 --2- 172.21.15.83:0/3126462969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77180702e0 0x7f77180d1ac0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f77180d2d00 tx=0x7f7700009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719d3a0ecc2a321f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.570+0000 7f7715ffb700 1 -- 172.21.15.83:0/3126462969 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770000b870 con 0x7f77180702e0 2023-11-26T07:16:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.570+0000 7f7715ffb700 1 -- 172.21.15.83:0/3126462969 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770000b9d0 con 0x7f77180702e0 2023-11-26T07:16:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.570+0000 7f7715ffb700 1 -- 172.21.15.83:0/3126462969 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770000bd60 con 0x7f77180702e0 2023-11-26T07:16:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.570+0000 7f771d3d7700 1 -- 172.21.15.83:0/3126462969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77180702e0 msgr2=0x7f77180d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.570+0000 7f771d3d7700 1 --2- 172.21.15.83:0/3126462969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77180702e0 0x7f77180d1ac0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f77180d2d00 tx=0x7f7700009bd0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.571+0000 7f771d3d7700 1 -- 172.21.15.83:0/3126462969 shutdown_connections 2023-11-26T07:16:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.571+0000 7f771d3d7700 1 --2- 172.21.15.83:0/3126462969 >> v2:172.21.15.203:3300/0 conn(0x7f771815bec0 0x7f771806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.571+0000 7f771d3d7700 1 --2- 172.21.15.83:0/3126462969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77180702e0 0x7f77180d1ac0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.571+0000 7f771d3d7700 1 -- 172.21.15.83:0/3126462969 >> 172.21.15.83:0/3126462969 conn(0x7f77180730a0 msgr2=0x7f771809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.571+0000 7f771d3d7700 1 -- 172.21.15.83:0/3126462969 shutdown_connections 2023-11-26T07:16:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.571+0000 7f771d3d7700 1 -- 172.21.15.83:0/3126462969 wait complete. 2023-11-26T07:16:32.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.572+0000 7f771d3d7700 1 Processor -- start 2023-11-26T07:16:32.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.572+0000 7f771d3d7700 1 -- start start 2023-11-26T07:16:32.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f771d3d7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77180702e0 0x7f771815c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f771d3d7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77180d2000 0x7f77180cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f771d3d7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771815bec0 0x7f771815ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f771d3d7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771815c9f0 con 0x7f77180d2000 2023-11-26T07:16:32.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f771d3d7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771815a980 con 0x7f77180702e0 2023-11-26T07:16:32.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f771d3d7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771815d7f0 con 0x7f771815bec0 2023-11-26T07:16:32.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f77167fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77180d2000 0x7f77180cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f77167fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77180d2000 0x7f77180cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40874/0 (socket says 172.21.15.83:40874) 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f77167fc700 1 -- 172.21.15.83:0/4294245522 learned_addr learned my addr 172.21.15.83:0/4294245522 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f7716ffd700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77180702e0 0x7f771815c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f77177fe700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771815bec0 0x7f771815ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f77167fc700 1 -- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771815bec0 msgr2=0x7f771815ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f77167fc700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771815bec0 0x7f771815ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f77167fc700 1 -- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77180702e0 msgr2=0x7f771815c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f77167fc700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77180702e0 0x7f771815c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f77167fc700 1 -- 172.21.15.83:0/4294245522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7700009550 con 0x7f77180d2000 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f7716ffd700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77180702e0 0x7f771815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f77167fc700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77180d2000 0x7f77180cb600 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7700000c00 tx=0x7f7700004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f770f7fe700 1 -- 172.21.15.83:0/4294245522 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700004520 con 0x7f77180d2000 2023-11-26T07:16:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f770f7fe700 1 -- 172.21.15.83:0/4294245522 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770001e070 con 0x7f77180d2000 2023-11-26T07:16:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f771d3d7700 1 -- 172.21.15.83:0/4294245522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771815cbe0 con 0x7f77180d2000 2023-11-26T07:16:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f770f7fe700 1 -- 172.21.15.83:0/4294245522 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77000038a0 con 0x7f77180d2000 2023-11-26T07:16:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.573+0000 7f771d3d7700 1 -- 172.21.15.83:0/4294245522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771815d1a0 con 0x7f77180d2000 2023-11-26T07:16:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.574+0000 7f771d3d7700 1 -- 172.21.15.83:0/4294245522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7718068fb0 con 0x7f77180d2000 2023-11-26T07:16:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.575+0000 7f770f7fe700 1 -- 172.21.15.83:0/4294245522 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f7700003a00 con 0x7f77180d2000 2023-11-26T07:16:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.575+0000 7f770f7fe700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f76f806aff0 0x7f76f806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.575+0000 7f770f7fe700 1 -- 172.21.15.83:0/4294245522 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f7700020070 con 0x7f77180d2000 2023-11-26T07:16:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.575+0000 7f7716ffd700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f76f806aff0 0x7f76f806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.576+0000 7f7716ffd700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f76f806aff0 0x7f76f806d4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7710009bc0 tx=0x7f7710003a60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.580+0000 7f770f7fe700 1 -- 172.21.15.83:0/4294245522 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7700055d60 con 0x7f77180d2000 2023-11-26T07:16:32.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:32.804+0000 7f771d3d7700 1 -- 172.21.15.83:0/4294245522 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f7718070980 con 0x7f76f806aff0 2023-11-26T07:16:32.810 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-26T07:16:32.809+0000 7fcd25823700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:16:32.836 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-26T07:16:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.824+0000 7f770f7fe700 1 -- 172.21.15.83:0/4294245522 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+78 (secure 0 0 0) 0x7f77100045c0 con 0x7f76f806aff0 2023-11-26T07:16:35.827 INFO:teuthology.orchestra.run.smithi083.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2023-11-26T07:16:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.827+0000 7f771d3d7700 1 -- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f76f806aff0 msgr2=0x7f76f806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.827+0000 7f771d3d7700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f76f806aff0 0x7f76f806d4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7710009bc0 tx=0x7f7710003a60 comp rx=0 tx=0).stop 2023-11-26T07:16:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.827+0000 7f771d3d7700 1 -- 172.21.15.83:0/4294245522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77180d2000 msgr2=0x7f77180cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.828+0000 7f771d3d7700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77180d2000 0x7f77180cb600 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7700000c00 tx=0x7f7700004750 comp rx=0 tx=0).stop 2023-11-26T07:16:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.828+0000 7f771d3d7700 1 -- 172.21.15.83:0/4294245522 shutdown_connections 2023-11-26T07:16:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.828+0000 7f771d3d7700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f76f806aff0 0x7f76f806d4b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.828+0000 7f771d3d7700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f771815bec0 0x7f771815ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.828+0000 7f771d3d7700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77180702e0 0x7f771815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.828+0000 7f771d3d7700 1 --2- 172.21.15.83:0/4294245522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77180d2000 0x7f77180cb600 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.828+0000 7f771d3d7700 1 -- 172.21.15.83:0/4294245522 >> 172.21.15.83:0/4294245522 conn(0x7f77180730a0 msgr2=0x7f77180b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.828+0000 7f771d3d7700 1 -- 172.21.15.83:0/4294245522 shutdown_connections 2023-11-26T07:16:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:35.828+0000 7f771d3d7700 1 -- 172.21.15.83:0/4294245522 wait complete. 2023-11-26T07:16:35.846 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:16:35.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:16:35.886 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:16:35.929 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:16:35.929 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:16:35.929 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:35.947 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:16:35.947 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:36.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.167+0000 7fc23b19f700 1 Processor -- start 2023-11-26T07:16:36.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.167+0000 7fc23b19f700 1 -- start start 2023-11-26T07:16:36.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc23b19f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc23406c710 0x7fc2340b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc23b19f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2340ae430 con 0x7fc2340af4d0 2023-11-26T07:16:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc23b19f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2340ae710 con 0x7fc23415e080 2023-11-26T07:16:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc23b19f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2340af0b0 con 0x7fc23406c710 2023-11-26T07:16:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc23973c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc2340af4d0 0x7fc23406e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60938/0 (socket says 172.21.15.83:60938) 2023-11-26T07:16:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc23973c700 1 -- 172.21.15.83:0/952593202 learned_addr learned my addr 172.21.15.83:0/952593202 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:36.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 872196232 0 0) 0x7fc2340ae430 con 0x7fc2340af4d0 2023-11-26T07:16:36.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2240035b0 con 0x7fc2340af4d0 2023-11-26T07:16:36.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc233fff700 1 --2- 172.21.15.83:0/952593202 >> v2:172.21.15.203:3300/0 conn(0x7fc23406c710 0x7fc2340b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3781489488 0 0) 0x7fc2240035b0 con 0x7fc2340af4d0 2023-11-26T07:16:36.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 >> v2:172.21.15.203:3300/0 conn(0x7fc23406c710 msgr2=0x7fc2340b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:36.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 --2- 172.21.15.83:0/952593202 >> v2:172.21.15.203:3300/0 conn(0x7fc23406c710 0x7fc2340b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 >> v1:172.21.15.203:6789/0 conn(0x7fc23415e080 legacy=0x7fc23406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2340ae9f0 con 0x7fc2340af4d0 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fc22c0019f0 con 0x7fc2340af4d0 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 >> v1:172.21.15.83:6789/0 conn(0x7fc2340af4d0 legacy=0x7fc23406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc224003a00 0x7fc224003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc224004850 0x7fc224004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc224005210 0x7fc22400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2340af0b0 con 0x7fc224004850 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22400c0b0 con 0x7fc224005210 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.168+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22400c270 con 0x7fc224003a00 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc233fff700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc224004850 0x7fc224004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc23973c700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc224005210 0x7fc22400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc238f3b700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc224003a00 0x7fc224003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc233fff700 1 -- 172.21.15.83:0/952593202 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc224003a00 msgr2=0x7fc224003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:36.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc233fff700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc224003a00 0x7fc224003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc233fff700 1 -- 172.21.15.83:0/952593202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc224005210 msgr2=0x7fc22400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:36.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc233fff700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc224005210 0x7fc22400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc233fff700 1 -- 172.21.15.83:0/952593202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2240035b0 con 0x7fc224004850 2023-11-26T07:16:36.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc23973c700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc224005210 0x7fc22400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:36.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc23973c700 1 -- 172.21.15.83:0/952593202 reap_dead start 2023-11-26T07:16:36.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc233fff700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc224004850 0x7fc224004cd0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fc220009b60 tx=0x7fc224005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100922e04f7d487f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:36.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22000e6c0 con 0x7fc224004850 2023-11-26T07:16:36.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22000e820 con 0x7fc224004850 2023-11-26T07:16:36.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.169+0000 7fc2337fe700 1 -- 172.21.15.83:0/952593202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22000eb90 con 0x7fc224004850 2023-11-26T07:16:36.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.170+0000 7fc23b19f700 1 -- 172.21.15.83:0/952593202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc224004850 msgr2=0x7fc224004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.170+0000 7fc23b19f700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc224004850 0x7fc224004cd0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fc220009b60 tx=0x7fc224005880 comp rx=0 tx=0).stop 2023-11-26T07:16:36.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.170+0000 7fc23b19f700 1 -- 172.21.15.83:0/952593202 shutdown_connections 2023-11-26T07:16:36.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.170+0000 7fc23b19f700 1 --2- 172.21.15.83:0/952593202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc224004850 0x7fc224004cd0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.170+0000 7fc23b19f700 1 -- 172.21.15.83:0/952593202 >> 172.21.15.83:0/952593202 conn(0x7fc234071b90 msgr2=0x7fc234071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:36.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.170+0000 7fc23b19f700 1 -- 172.21.15.83:0/952593202 shutdown_connections 2023-11-26T07:16:36.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.170+0000 7fc23b19f700 1 -- 172.21.15.83:0/952593202 wait complete. 2023-11-26T07:16:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.171+0000 7fc23b19f700 1 Processor -- start 2023-11-26T07:16:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.171+0000 7fc23b19f700 1 -- start start 2023-11-26T07:16:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc23b19f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc224004850 0x7fc2341536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc23b19f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc234155c20 0x7fc234156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc23b19f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2340b18c0 0x7fc2340b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc23b19f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2340b2290 con 0x7fc234155c20 2023-11-26T07:16:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc23b19f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc234153d60 con 0x7fc224004850 2023-11-26T07:16:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc23b19f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc234154040 con 0x7fc2340b18c0 2023-11-26T07:16:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc233fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc234155c20 0x7fc234156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc233fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc234155c20 0x7fc234156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40908/0 (socket says 172.21.15.83:40908) 2023-11-26T07:16:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc233fff700 1 -- 172.21.15.83:0/1207751355 learned_addr learned my addr 172.21.15.83:0/1207751355 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc233fff700 1 -- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2340b18c0 msgr2=0x7fc2340b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc23973c700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2340b18c0 0x7fc2340b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc233fff700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2340b18c0 0x7fc2340b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc238f3b700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc224004850 0x7fc2341536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc233fff700 1 -- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc224004850 msgr2=0x7fc2341536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc233fff700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc224004850 0x7fc2341536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc233fff700 1 -- 172.21.15.83:0/1207751355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc234154320 con 0x7fc234155c20 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc238f3b700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc224004850 0x7fc2341536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc233fff700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc234155c20 0x7fc234156450 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc22400c080 tx=0x7fc2200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.172+0000 7fc2317fa700 1 -- 172.21.15.83:0/1207751355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22000e6c0 con 0x7fc234155c20 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.173+0000 7fc2317fa700 1 -- 172.21.15.83:0/1207751355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc220003ad0 con 0x7fc234155c20 2023-11-26T07:16:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.173+0000 7fc23b19f700 1 -- 172.21.15.83:0/1207751355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc234154600 con 0x7fc234155c20 2023-11-26T07:16:36.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.173+0000 7fc2317fa700 1 -- 172.21.15.83:0/1207751355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2200045d0 con 0x7fc234155c20 2023-11-26T07:16:36.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.173+0000 7fc23b19f700 1 -- 172.21.15.83:0/1207751355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc234154bc0 con 0x7fc234155c20 2023-11-26T07:16:36.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.174+0000 7fc23b19f700 1 -- 172.21.15.83:0/1207751355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2340753f0 con 0x7fc234155c20 2023-11-26T07:16:36.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.174+0000 7fc2317fa700 1 -- 172.21.15.83:0/1207751355 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fc220013470 con 0x7fc234155c20 2023-11-26T07:16:36.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.174+0000 7fc2317fa700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc21806aff0 0x7fc21806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.174+0000 7fc2317fa700 1 -- 172.21.15.83:0/1207751355 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc22008afc0 con 0x7fc234155c20 2023-11-26T07:16:36.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.175+0000 7fc238f3b700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc21806aff0 0x7fc21806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.175+0000 7fc238f3b700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc21806aff0 0x7fc21806d4b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc228000c00 tx=0x7fc228009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:36.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.179+0000 7fc2317fa700 1 -- 172.21.15.83:0/1207751355 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc2200bde90 con 0x7fc234155c20 2023-11-26T07:16:36.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.378+0000 7fc23b19f700 1 -- 172.21.15.83:0/1207751355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc234067f60 con 0x7fc234155c20 2023-11-26T07:16:36.381 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.379+0000 7fc2317fa700 1 -- 172.21.15.83:0/1207751355 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16675 (secure 0 0 0) 0x7fc220017020 con 0x7fc234155c20 2023-11-26T07:16:36.382 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:36.382 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:35.818271+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:16:15.747889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:16:16.943249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6835/469139444":"2023-11-27T07:16:32.809518+0000","172.21.15.83:6833/469139444":"2023-11-27T07:16:32.809518+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 -- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc21806aff0 msgr2=0x7fc21806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc21806aff0 0x7fc21806d4b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc228000c00 tx=0x7fc228009040 comp rx=0 tx=0).stop 2023-11-26T07:16:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 -- 172.21.15.83:0/1207751355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc234155c20 msgr2=0x7fc234156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc234155c20 0x7fc234156450 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc22400c080 tx=0x7fc2200047c0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 -- 172.21.15.83:0/1207751355 shutdown_connections 2023-11-26T07:16:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc21806aff0 0x7fc21806d4b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2340b18c0 0x7fc2340b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc224004850 0x7fc2341536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 --2- 172.21.15.83:0/1207751355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc234155c20 0x7fc234156450 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 -- 172.21.15.83:0/1207751355 >> 172.21.15.83:0/1207751355 conn(0x7fc234071b90 msgr2=0x7fc2340b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 -- 172.21.15.83:0/1207751355 shutdown_connections 2023-11-26T07:16:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.381+0000 7fc23b19f700 1 -- 172.21.15.83:0/1207751355 wait complete. 2023-11-26T07:16:36.396 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:16:15.747889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:16:16.943249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:16:36.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:16:36.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.517+0000 7fb09dced700 1 Processor -- start 2023-11-26T07:16:36.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.517+0000 7fb09dced700 1 -- start start 2023-11-26T07:16:36.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb09dced700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb0980c6c30 0x7fb0980cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb09dced700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb0980cef40 0x7fb0980cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb09dced700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0980d0140 con 0x7fb09815bf50 2023-11-26T07:16:36.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb09dced700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb09815c330 con 0x7fb0980cef40 2023-11-26T07:16:36.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb09dced700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0980cf3d0 con 0x7fb0980c6c30 2023-11-26T07:16:36.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb097fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb0980cef40 0x7fb0980cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb097fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb0980cef40 0x7fb0980cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40918/0 (socket says 172.21.15.83:40918) 2023-11-26T07:16:36.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb097fff700 1 -- 172.21.15.83:0/520806477 learned_addr learned my addr 172.21.15.83:0/520806477 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:36.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb097fff700 1 -- 172.21.15.83:0/520806477 >> v2:172.21.15.203:3300/0 conn(0x7fb0980c6c30 msgr2=0x7fb0980cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb096ffd700 1 --2- 172.21.15.83:0/520806477 >> v2:172.21.15.203:3300/0 conn(0x7fb0980c6c30 0x7fb0980cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb097fff700 1 --2- 172.21.15.83:0/520806477 >> v2:172.21.15.203:3300/0 conn(0x7fb0980c6c30 0x7fb0980cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb097fff700 1 -- 172.21.15.83:0/520806477 >> v1:172.21.15.203:6789/0 conn(0x7fb09815bf50 legacy=0x7fb0980ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb097fff700 1 -- 172.21.15.83:0/520806477 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0980d03e0 con 0x7fb0980cef40 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb096ffd700 1 --2- 172.21.15.83:0/520806477 >> v2:172.21.15.203:3300/0 conn(0x7fb0980c6c30 0x7fb0980cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb097fff700 1 --2- 172.21.15.83:0/520806477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0980cef40 0x7fb0980cc6c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb098072420 tx=0x7fb09000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896e03a105aac80d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.518+0000 7fb0967fc700 1 -- 172.21.15.83:0/520806477 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09000e070 con 0x7fb0980cef40 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.519+0000 7fb0967fc700 1 -- 172.21.15.83:0/520806477 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb090003a20 con 0x7fb0980cef40 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.519+0000 7fb0967fc700 1 -- 172.21.15.83:0/520806477 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090003dd0 con 0x7fb0980cef40 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.519+0000 7fb09dced700 1 -- 172.21.15.83:0/520806477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0980cef40 msgr2=0x7fb0980cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.519+0000 7fb09dced700 1 --2- 172.21.15.83:0/520806477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0980cef40 0x7fb0980cc6c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb098072420 tx=0x7fb09000b330 comp rx=0 tx=0).stop 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.519+0000 7fb09dced700 1 -- 172.21.15.83:0/520806477 shutdown_connections 2023-11-26T07:16:36.523 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.519+0000 7fb09dced700 1 --2- 172.21.15.83:0/520806477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0980cef40 0x7fb0980cc6c0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.519+0000 7fb09dced700 1 --2- 172.21.15.83:0/520806477 >> v2:172.21.15.203:3300/0 conn(0x7fb0980c6c30 0x7fb0980cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.519+0000 7fb09dced700 1 -- 172.21.15.83:0/520806477 >> 172.21.15.83:0/520806477 conn(0x7fb0980b3780 msgr2=0x7fb0980b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:36.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.519+0000 7fb09dced700 1 -- 172.21.15.83:0/520806477 shutdown_connections 2023-11-26T07:16:36.524 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.519+0000 7fb09dced700 1 -- 172.21.15.83:0/520806477 wait complete. 2023-11-26T07:16:36.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.520+0000 7fb09dced700 1 Processor -- start 2023-11-26T07:16:36.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.521+0000 7fb09dced700 1 -- start start 2023-11-26T07:16:36.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.521+0000 7fb09dced700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0980c6c30 0x7fb0981f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.521+0000 7fb09dced700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0980cef40 0x7fb0981e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.521+0000 7fb09dced700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb09815bf50 0x7fb0980c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.521+0000 7fb09dced700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09806c170 con 0x7fb0980cef40 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.521+0000 7fb09dced700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09806c450 con 0x7fb0980c6c30 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.521+0000 7fb09dced700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09806c730 con 0x7fb09815bf50 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.521+0000 7fb096ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0980cef40 0x7fb0981e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.521+0000 7fb096ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0980cef40 0x7fb0981e5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40922/0 (socket says 172.21.15.83:40922) 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.521+0000 7fb096ffd700 1 -- 172.21.15.83:0/3824777953 learned_addr learned my addr 172.21.15.83:0/3824777953 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.522+0000 7fb096ffd700 1 -- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb09815bf50 msgr2=0x7fb0980c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.522+0000 7fb096ffd700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb09815bf50 0x7fb0980c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.522+0000 7fb096ffd700 1 -- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0980c6c30 msgr2=0x7fb0981f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.522+0000 7fb096ffd700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0980c6c30 0x7fb0981f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.522+0000 7fb096ffd700 1 -- 172.21.15.83:0/3824777953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090009b30 con 0x7fb0980cef40 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.522+0000 7fb096ffd700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0980cef40 0x7fb0981e5e10 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb08800c750 tx=0x7fb08800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:36.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.522+0000 7fb07ffff700 1 -- 172.21.15.83:0/3824777953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088011800 con 0x7fb0980cef40 2023-11-26T07:16:36.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.522+0000 7fb07ffff700 1 -- 172.21.15.83:0/3824777953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb088011960 con 0x7fb0980cef40 2023-11-26T07:16:36.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.522+0000 7fb09dced700 1 -- 172.21.15.83:0/3824777953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09806ca10 con 0x7fb0980cef40 2023-11-26T07:16:36.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.522+0000 7fb07ffff700 1 -- 172.21.15.83:0/3824777953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088011cd0 con 0x7fb0980cef40 2023-11-26T07:16:36.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.522+0000 7fb09dced700 1 -- 172.21.15.83:0/3824777953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0981f3840 con 0x7fb0980cef40 2023-11-26T07:16:36.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.523+0000 7fb09dced700 1 -- 172.21.15.83:0/3824777953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb098068fb0 con 0x7fb0980cef40 2023-11-26T07:16:36.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.523+0000 7fb07ffff700 1 -- 172.21.15.83:0/3824777953 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fb088021350 con 0x7fb0980cef40 2023-11-26T07:16:36.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.524+0000 7fb07ffff700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb08406aff0 0x7fb08406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.524+0000 7fb07ffff700 1 -- 172.21.15.83:0/3824777953 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb088089880 con 0x7fb0980cef40 2023-11-26T07:16:36.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.524+0000 7fb0977fe700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb08406aff0 0x7fb08406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.525+0000 7fb0977fe700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb08406aff0 0x7fb08406d4b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0096f0 tx=0x7fb08c006c50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:36.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.528+0000 7fb07ffff700 1 -- 172.21.15.83:0/3824777953 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb0880546b0 con 0x7fb0980cef40 2023-11-26T07:16:36.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.714+0000 7fb09dced700 1 -- 172.21.15.83:0/3824777953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0980b7ab0 con 0x7fb0980cef40 2023-11-26T07:16:36.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.714+0000 7fb07ffff700 1 -- 172.21.15.83:0/3824777953 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb08801e0a0 con 0x7fb0980cef40 2023-11-26T07:16:36.717 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:16:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 -- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb08406aff0 msgr2=0x7fb08406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb08406aff0 0x7fb08406d4b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0096f0 tx=0x7fb08c006c50 comp rx=0 tx=0).stop 2023-11-26T07:16:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 -- 172.21.15.83:0/3824777953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0980cef40 msgr2=0x7fb0981e5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0980cef40 0x7fb0981e5e10 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fb08800c750 tx=0x7fb08800cb10 comp rx=0 tx=0).stop 2023-11-26T07:16:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 -- 172.21.15.83:0/3824777953 shutdown_connections 2023-11-26T07:16:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb08406aff0 0x7fb08406d4b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb09815bf50 0x7fb0980c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.719 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0980c6c30 0x7fb0981f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 --2- 172.21.15.83:0/3824777953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0980cef40 0x7fb0981e5e10 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 -- 172.21.15.83:0/3824777953 >> 172.21.15.83:0/3824777953 conn(0x7fb0980b3780 msgr2=0x7fb0980ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 -- 172.21.15.83:0/3824777953 shutdown_connections 2023-11-26T07:16:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.716+0000 7fb09dced700 1 -- 172.21.15.83:0/3824777953 wait complete. 2023-11-26T07:16:36.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:16:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.851+0000 7f1cd5949700 1 Processor -- start 2023-11-26T07:16:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.852+0000 7f1cd5949700 1 -- start start 2023-11-26T07:16:36.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.852+0000 7f1cd5949700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cd015dfa0 0x7f1cd006ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.852+0000 7f1cd5949700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd006bf40 con 0x7f1cd00b6ec0 2023-11-26T07:16:36.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.852+0000 7f1cd5949700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd00b7a30 con 0x7f1cd00cff20 2023-11-26T07:16:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.852+0000 7f1cd5949700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd006c210 con 0x7f1cd015dfa0 2023-11-26T07:16:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.852+0000 7f1cce7fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1cd00b6ec0 0x7f1cd00b7320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:60970/0 (socket says 172.21.15.83:60970) 2023-11-26T07:16:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.852+0000 7f1cce7fc700 1 -- 172.21.15.83:0/1463890484 learned_addr learned my addr 172.21.15.83:0/1463890484 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1cceffd700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cd015dfa0 0x7f1cd006ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494224978 0 0) 0x7f1cd006bf40 con 0x7f1cd00b6ec0 2023-11-26T07:16:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb80035b0 con 0x7f1cd00b6ec0 2023-11-26T07:16:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305489714 0 0) 0x7f1cd00b7a30 con 0x7f1cd00cff20 2023-11-26T07:16:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cd006bf40 con 0x7f1cd00cff20 2023-11-26T07:16:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3070413263 0 0) 0x7f1cb80035b0 con 0x7f1cd00b6ec0 2023-11-26T07:16:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cd015dfa0 msgr2=0x7f1cd006ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cd015dfa0 0x7f1cd006ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 >> v1:172.21.15.203:6789/0 conn(0x7f1cd00cff20 legacy=0x7f1cd00d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd00b4b30 con 0x7f1cd00b6ec0 2023-11-26T07:16:36.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f1cc0001670 con 0x7f1cd00b6ec0 2023-11-26T07:16:36.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 >> v1:172.21.15.83:6789/0 conn(0x7f1cd00b6ec0 legacy=0x7f1cd00b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cd015dfa0 0x7f1cb8003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb8004170 0x7f1cb8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cb8004d90 0x7f1cb8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd006c210 con 0x7f1cb8004170 2023-11-26T07:16:36.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb800bb50 con 0x7f1cb8004d90 2023-11-26T07:16:36.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb800bd10 con 0x7f1cd015dfa0 2023-11-26T07:16:36.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1cce7fc700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb8004170 0x7f1cb8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1cceffd700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cd015dfa0 0x7f1cb8003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1ccf7fe700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cb8004d90 0x7f1cb8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1cce7fc700 1 -- 172.21.15.83:0/1463890484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cd015dfa0 msgr2=0x7f1cb8003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1cce7fc700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cd015dfa0 0x7f1cb8003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1cce7fc700 1 -- 172.21.15.83:0/1463890484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cb8004d90 msgr2=0x7f1cb8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1cce7fc700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cb8004d90 0x7f1cb8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1cce7fc700 1 -- 172.21.15.83:0/1463890484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb80035b0 con 0x7f1cb8004170 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.853+0000 7f1cce7fc700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb8004170 0x7f1cb8004850 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c160 tx=0x7f1cb8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530f7c7aef2e485a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.854+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000cbe0 con 0x7f1cb8004170 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.854+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc000cd60 con 0x7f1cb8004170 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.854+0000 7f1ccdffb700 1 -- 172.21.15.83:0/1463890484 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0005ba0 con 0x7f1cb8004170 2023-11-26T07:16:36.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.854+0000 7f1cd5949700 1 -- 172.21.15.83:0/1463890484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb8004170 msgr2=0x7f1cb8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.854+0000 7f1cd5949700 1 --2- 172.21.15.83:0/1463890484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb8004170 0x7f1cb8004850 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c160 tx=0x7f1cb8005880 comp rx=0 tx=0).stop 2023-11-26T07:16:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.854+0000 7f1ccf7fe700 1 -- 172.21.15.83:0/1463890484 reap_dead start 2023-11-26T07:16:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.854+0000 7f1cd5949700 1 -- 172.21.15.83:0/1463890484 shutdown_connections 2023-11-26T07:16:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.854+0000 7f1cd5949700 1 -- 172.21.15.83:0/1463890484 >> 172.21.15.83:0/1463890484 conn(0x7f1cd00b3810 msgr2=0x7f1cd00b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.854+0000 7f1cd5949700 1 -- 172.21.15.83:0/1463890484 shutdown_connections 2023-11-26T07:16:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.854+0000 7f1cd5949700 1 -- 172.21.15.83:0/1463890484 wait complete. 2023-11-26T07:16:36.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.856+0000 7f1cd5949700 1 Processor -- start 2023-11-26T07:16:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.856+0000 7f1cd5949700 1 -- start start 2023-11-26T07:16:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1cd5949700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cb8004170 0x7f1cd0160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1cd5949700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd009bc00 0x7f1cd0151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1cd5949700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd009c080 0x7f1cd0151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1cd5949700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0151aa0 con 0x7f1cd009c080 2023-11-26T07:16:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1cd5949700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd009c650 con 0x7f1cd009bc00 2023-11-26T07:16:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1cd5949700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd009c930 con 0x7f1cb8004170 2023-11-26T07:16:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd009c080 0x7f1cd0151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd009c080 0x7f1cd0151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40944/0 (socket says 172.21.15.83:40944) 2023-11-26T07:16:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1ccf7fe700 1 -- 172.21.15.83:0/3289571034 learned_addr learned my addr 172.21.15.83:0/3289571034 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1cce7fc700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd009bc00 0x7f1cd0151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1cceffd700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cb8004170 0x7f1cd0160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1ccf7fe700 1 -- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cb8004170 msgr2=0x7f1cd0160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1ccf7fe700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cb8004170 0x7f1cd0160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1ccf7fe700 1 -- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd009bc00 msgr2=0x7f1cd0151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1ccf7fe700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd009bc00 0x7f1cd0151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.857+0000 7f1ccf7fe700 1 -- 172.21.15.83:0/3289571034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd009cc10 con 0x7f1cd009c080 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.858+0000 7f1ccf7fe700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd009c080 0x7f1cd0151560 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0072c60 tx=0x7f1cc800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.858+0000 7f1cb7fff700 1 -- 172.21.15.83:0/3289571034 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8003c10 con 0x7f1cd009c080 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.858+0000 7f1cb7fff700 1 -- 172.21.15.83:0/3289571034 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc8003d70 con 0x7f1cd009c080 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.858+0000 7f1cd5949700 1 -- 172.21.15.83:0/3289571034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd009cef0 con 0x7f1cd009c080 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.858+0000 7f1cb7fff700 1 -- 172.21.15.83:0/3289571034 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8005310 con 0x7f1cd009c080 2023-11-26T07:16:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.858+0000 7f1cd5949700 1 -- 172.21.15.83:0/3289571034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd00c82a0 con 0x7f1cd009c080 2023-11-26T07:16:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.859+0000 7f1cd5949700 1 -- 172.21.15.83:0/3289571034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd006f480 con 0x7f1cd009c080 2023-11-26T07:16:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.859+0000 7f1cb7fff700 1 -- 172.21.15.83:0/3289571034 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f1cc8005470 con 0x7f1cd009c080 2023-11-26T07:16:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.860+0000 7f1cb7fff700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1cbc06aff0 0x7f1cbc06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.860+0000 7f1cb7fff700 1 -- 172.21.15.83:0/3289571034 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1cc8089950 con 0x7f1cd009c080 2023-11-26T07:16:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.860+0000 7f1cceffd700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1cbc06aff0 0x7f1cbc06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.861+0000 7f1cceffd700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1cbc06aff0 0x7f1cbc06d4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009a80 tx=0x7f1cc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:36.864+0000 7f1cb7fff700 1 -- 172.21.15.83:0/3289571034 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1cc8054760 con 0x7f1cd009c080 2023-11-26T07:16:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.051+0000 7f1cd5949700 1 -- 172.21.15.83:0/3289571034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1cd00c8430 con 0x7f1cd009c080 2023-11-26T07:16:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.051+0000 7f1cb7fff700 1 -- 172.21.15.83:0/3289571034 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1cc801a310 con 0x7f1cd009c080 2023-11-26T07:16:37.054 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:16:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.053+0000 7f1cd5949700 1 -- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1cbc06aff0 msgr2=0x7f1cbc06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.053+0000 7f1cd5949700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1cbc06aff0 0x7f1cbc06d4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009a80 tx=0x7f1cc4008040 comp rx=0 tx=0).stop 2023-11-26T07:16:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.053+0000 7f1cd5949700 1 -- 172.21.15.83:0/3289571034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd009c080 msgr2=0x7f1cd0151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.053+0000 7f1cd5949700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd009c080 0x7f1cd0151560 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0072c60 tx=0x7f1cc800ef00 comp rx=0 tx=0).stop 2023-11-26T07:16:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.053+0000 7f1cd5949700 1 -- 172.21.15.83:0/3289571034 shutdown_connections 2023-11-26T07:16:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.053+0000 7f1cd5949700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1cbc06aff0 0x7f1cbc06d4b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.053+0000 7f1cd5949700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd009c080 0x7f1cd0151560 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.053+0000 7f1cd5949700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1cb8004170 0x7f1cd0160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.053+0000 7f1cd5949700 1 --2- 172.21.15.83:0/3289571034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cd009bc00 0x7f1cd0151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.053+0000 7f1cd5949700 1 -- 172.21.15.83:0/3289571034 >> 172.21.15.83:0/3289571034 conn(0x7f1cd00b3810 msgr2=0x7f1cd006ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.054+0000 7f1cd5949700 1 -- 172.21.15.83:0/3289571034 shutdown_connections 2023-11-26T07:16:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.054+0000 7f1cd5949700 1 -- 172.21.15.83:0/3289571034 wait complete. 2023-11-26T07:16:37.068 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:16:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.189+0000 7fdd860da700 1 Processor -- start 2023-11-26T07:16:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.189+0000 7fdd860da700 1 -- start start 2023-11-26T07:16:37.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.189+0000 7fdd860da700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdd8015e160 0x7fdd800b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.189+0000 7fdd860da700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdd800746a0 0x7fdd8006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.189+0000 7fdd860da700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd800b3c60 con 0x7fdd800afc90 2023-11-26T07:16:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.189+0000 7fdd860da700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd800b4500 con 0x7fdd8015e160 2023-11-26T07:16:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.189+0000 7fdd860da700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd800b58c0 con 0x7fdd800746a0 2023-11-26T07:16:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdd8015e160 0x7fdd800b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdd8015e160 0x7fdd800b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40948/0 (socket says 172.21.15.83:40948) 2023-11-26T07:16:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7f7fe700 1 -- 172.21.15.83:0/1459175298 learned_addr learned my addr 172.21.15.83:0/1459175298 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7ffff700 1 --2- 172.21.15.83:0/1459175298 >> v2:172.21.15.203:3300/0 conn(0x7fdd800746a0 0x7fdd8006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7f7fe700 1 -- 172.21.15.83:0/1459175298 >> v2:172.21.15.203:3300/0 conn(0x7fdd800746a0 msgr2=0x7fdd8006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7f7fe700 1 --2- 172.21.15.83:0/1459175298 >> v2:172.21.15.203:3300/0 conn(0x7fdd800746a0 0x7fdd8006d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7f7fe700 1 -- 172.21.15.83:0/1459175298 >> v1:172.21.15.203:6789/0 conn(0x7fdd800afc90 legacy=0x7fdd80073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7f7fe700 1 -- 172.21.15.83:0/1459175298 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd800b0140 con 0x7fdd8015e160 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7f7fe700 1 --2- 172.21.15.83:0/1459175298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd8015e160 0x7fdd800b5da0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fdd800d16e0 tx=0x7fdd74009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649d8896b2702e74 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7e7fc700 1 -- 172.21.15.83:0/1459175298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd74004230 con 0x7fdd8015e160 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7e7fc700 1 -- 172.21.15.83:0/1459175298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd74004390 con 0x7fdd8015e160 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.190+0000 7fdd7e7fc700 1 -- 172.21.15.83:0/1459175298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd740036a0 con 0x7fdd8015e160 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.191+0000 7fdd860da700 1 -- 172.21.15.83:0/1459175298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd8015e160 msgr2=0x7fdd800b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.191+0000 7fdd860da700 1 --2- 172.21.15.83:0/1459175298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd8015e160 0x7fdd800b5da0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fdd800d16e0 tx=0x7fdd74009bd0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.191+0000 7fdd860da700 1 -- 172.21.15.83:0/1459175298 shutdown_connections 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.191+0000 7fdd860da700 1 --2- 172.21.15.83:0/1459175298 >> v2:172.21.15.203:3300/0 conn(0x7fdd800746a0 0x7fdd8006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.191+0000 7fdd860da700 1 --2- 172.21.15.83:0/1459175298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd8015e160 0x7fdd800b5da0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.191+0000 7fdd860da700 1 -- 172.21.15.83:0/1459175298 >> 172.21.15.83:0/1459175298 conn(0x7fdd800d0bf0 msgr2=0x7fdd800d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:37.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.191+0000 7fdd860da700 1 -- 172.21.15.83:0/1459175298 shutdown_connections 2023-11-26T07:16:37.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.191+0000 7fdd860da700 1 -- 172.21.15.83:0/1459175298 wait complete. 2023-11-26T07:16:37.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.192+0000 7fdd860da700 1 Processor -- start 2023-11-26T07:16:37.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.192+0000 7fdd860da700 1 -- start start 2023-11-26T07:16:37.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd860da700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800746a0 0x7fdd80150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd860da700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd800afc90 0x7fdd80160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd860da700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd8015e160 0x7fdd800c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd860da700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80150ad0 con 0x7fdd800746a0 2023-11-26T07:16:37.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd860da700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80150580 con 0x7fdd800afc90 2023-11-26T07:16:37.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd860da700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd801f4530 con 0x7fdd8015e160 2023-11-26T07:16:37.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd7f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800746a0 0x7fdd80150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd7f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800746a0 0x7fdd80150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40954/0 (socket says 172.21.15.83:40954) 2023-11-26T07:16:37.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd7f7fe700 1 -- 172.21.15.83:0/3973368291 learned_addr learned my addr 172.21.15.83:0/3973368291 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:37.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd7effd700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd800afc90 0x7fdd80160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd7ffff700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd8015e160 0x7fdd800c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd7f7fe700 1 -- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd8015e160 msgr2=0x7fdd800c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd7f7fe700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd8015e160 0x7fdd800c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd7f7fe700 1 -- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd800afc90 msgr2=0x7fdd80160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd7f7fe700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd800afc90 0x7fdd80160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd7f7fe700 1 -- 172.21.15.83:0/3973368291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd74009550 con 0x7fdd800746a0 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.193+0000 7fdd7effd700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd800afc90 0x7fdd80160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.194+0000 7fdd7f7fe700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800746a0 0x7fdd80150040 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdd80072ed0 tx=0x7fdd74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.194+0000 7fdd67fff700 1 -- 172.21.15.83:0/3973368291 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd74011630 con 0x7fdd800746a0 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.194+0000 7fdd67fff700 1 -- 172.21.15.83:0/3973368291 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd74011790 con 0x7fdd800746a0 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.194+0000 7fdd860da700 1 -- 172.21.15.83:0/3973368291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd801f4810 con 0x7fdd800746a0 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.194+0000 7fdd67fff700 1 -- 172.21.15.83:0/3973368291 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd74011b40 con 0x7fdd800746a0 2023-11-26T07:16:37.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.194+0000 7fdd860da700 1 -- 172.21.15.83:0/3973368291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd801f4dd0 con 0x7fdd800746a0 2023-11-26T07:16:37.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.195+0000 7fdd860da700 1 -- 172.21.15.83:0/3973368291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd80068fb0 con 0x7fdd800746a0 2023-11-26T07:16:37.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.195+0000 7fdd67fff700 1 -- 172.21.15.83:0/3973368291 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fdd74011ca0 con 0x7fdd800746a0 2023-11-26T07:16:37.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.195+0000 7fdd67fff700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd6c06aff0 0x7fdd6c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.195+0000 7fdd67fff700 1 -- 172.21.15.83:0/3973368291 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdd7408afb0 con 0x7fdd800746a0 2023-11-26T07:16:37.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.196+0000 7fdd7effd700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd6c06aff0 0x7fdd6c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.196+0000 7fdd7effd700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd6c06aff0 0x7fdd6c06d4b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd80150c80 tx=0x7fdd70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.200+0000 7fdd67fff700 1 -- 172.21.15.83:0/3973368291 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd740bde90 con 0x7fdd800746a0 2023-11-26T07:16:37.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.387+0000 7fdd860da700 1 -- 172.21.15.83:0/3973368291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd8006c140 con 0x7fdd800746a0 2023-11-26T07:16:37.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.388+0000 7fdd67fff700 1 -- 172.21.15.83:0/3973368291 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd7401f020 con 0x7fdd800746a0 2023-11-26T07:16:37.390 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:16:37.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 -- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd6c06aff0 msgr2=0x7fdd6c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd6c06aff0 0x7fdd6c06d4b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd80150c80 tx=0x7fdd70009040 comp rx=0 tx=0).stop 2023-11-26T07:16:37.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 -- 172.21.15.83:0/3973368291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800746a0 msgr2=0x7fdd80150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800746a0 0x7fdd80150040 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdd80072ed0 tx=0x7fdd74004750 comp rx=0 tx=0).stop 2023-11-26T07:16:37.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 -- 172.21.15.83:0/3973368291 shutdown_connections 2023-11-26T07:16:37.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdd6c06aff0 0x7fdd6c06d4b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd8015e160 0x7fdd800c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800746a0 0x7fdd80150040 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.393 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 --2- 172.21.15.83:0/3973368291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd800afc90 0x7fdd80160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 -- 172.21.15.83:0/3973368291 >> 172.21.15.83:0/3973368291 conn(0x7fdd800d0bf0 msgr2=0x7fdd8006e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 -- 172.21.15.83:0/3973368291 shutdown_connections 2023-11-26T07:16:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.390+0000 7fdd860da700 1 -- 172.21.15.83:0/3973368291 wait complete. 2023-11-26T07:16:37.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:16:37.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.524+0000 7faec028b700 1 Processor -- start 2023-11-26T07:16:37.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.524+0000 7faec028b700 1 -- start start 2023-11-26T07:16:37.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faec028b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faeb80d2380 0x7faeb81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faec028b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb80b22d0 con 0x7faeb80b3370 2023-11-26T07:16:37.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faec028b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb80b25b0 con 0x7faeb815bdd0 2023-11-26T07:16:37.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faec028b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb80b2f50 con 0x7faeb80d2380 2023-11-26T07:16:37.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd826700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faeb80b3370 0x7faeb80d1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:32772/0 (socket says 172.21.15.83:32772) 2023-11-26T07:16:37.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd826700 1 -- 172.21.15.83:0/3184275802 learned_addr learned my addr 172.21.15.83:0/3184275802 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:37.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebe828700 1 --2- 172.21.15.83:0/3184275802 >> v2:172.21.15.203:3300/0 conn(0x7faeb80d2380 0x7faeb81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277909860 0 0) 0x7faeb80b22d0 con 0x7faeb80b3370 2023-11-26T07:16:37.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faea40035b0 con 0x7faeb80b3370 2023-11-26T07:16:37.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1856888429 0 0) 0x7faea40035b0 con 0x7faeb80b3370 2023-11-26T07:16:37.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 >> v2:172.21.15.203:3300/0 conn(0x7faeb80d2380 msgr2=0x7faeb81604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:37.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd025700 1 --2- 172.21.15.83:0/3184275802 >> v2:172.21.15.203:3300/0 conn(0x7faeb80d2380 0x7faeb81604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 >> v1:172.21.15.203:6789/0 conn(0x7faeb815bdd0 legacy=0x7faeb80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb80b2890 con 0x7faeb80b3370 2023-11-26T07:16:37.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7faea8001670 con 0x7faeb80b3370 2023-11-26T07:16:37.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 >> v1:172.21.15.83:6789/0 conn(0x7faeb80b3370 legacy=0x7faeb80d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd025700 1 --2- 172.21.15.83:0/3184275802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea4003a00 0x7faea4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.525+0000 7faebd025700 1 --2- 172.21.15.83:0/3184275802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faea4004400 0x7faea40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebd025700 1 --2- 172.21.15.83:0/3184275802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faea40050d0 0x7faea400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb80b2f50 con 0x7faea4003a00 2023-11-26T07:16:37.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea400c030 con 0x7faea4004400 2023-11-26T07:16:37.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea400c1f0 con 0x7faea40050d0 2023-11-26T07:16:37.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebe027700 1 --2- 172.21.15.83:0/3184275802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea4003a00 0x7faea4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebd826700 1 --2- 172.21.15.83:0/3184275802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faea4004400 0x7faea40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebe828700 1 --2- 172.21.15.83:0/3184275802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faea40050d0 0x7faea400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebe027700 1 -- 172.21.15.83:0/3184275802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faea40050d0 msgr2=0x7faea400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:37.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebe027700 1 --2- 172.21.15.83:0/3184275802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faea40050d0 0x7faea400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebe027700 1 -- 172.21.15.83:0/3184275802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faea4004400 msgr2=0x7faea40048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:37.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebe027700 1 --2- 172.21.15.83:0/3184275802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faea4004400 0x7faea40048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebe027700 1 -- 172.21.15.83:0/3184275802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea40035b0 con 0x7faea4003a00 2023-11-26T07:16:37.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebe828700 1 -- 172.21.15.83:0/3184275802 reap_dead start 2023-11-26T07:16:37.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebe027700 1 --2- 172.21.15.83:0/3184275802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea4003a00 0x7faea4003de0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7faeb40018d0 tx=0x7faea4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39177aae84917fcd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:37.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb400c690 con 0x7faea4003a00 2023-11-26T07:16:37.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb400c810 con 0x7faea4003a00 2023-11-26T07:16:37.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.526+0000 7faebd025700 1 -- 172.21.15.83:0/3184275802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb400cbc0 con 0x7faea4003a00 2023-11-26T07:16:37.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.527+0000 7faec028b700 1 -- 172.21.15.83:0/3184275802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea4003a00 msgr2=0x7faea4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.527+0000 7faec028b700 1 --2- 172.21.15.83:0/3184275802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea4003a00 0x7faea4003de0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7faeb40018d0 tx=0x7faea4005880 comp rx=0 tx=0).stop 2023-11-26T07:16:37.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.527+0000 7faec028b700 1 -- 172.21.15.83:0/3184275802 shutdown_connections 2023-11-26T07:16:37.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.527+0000 7faec028b700 1 --2- 172.21.15.83:0/3184275802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea4003a00 0x7faea4003de0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.527+0000 7faec028b700 1 -- 172.21.15.83:0/3184275802 >> 172.21.15.83:0/3184275802 conn(0x7faeb80731a0 msgr2=0x7faeb809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:37.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.527+0000 7faec028b700 1 -- 172.21.15.83:0/3184275802 shutdown_connections 2023-11-26T07:16:37.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.527+0000 7faec028b700 1 -- 172.21.15.83:0/3184275802 wait complete. 2023-11-26T07:16:37.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.528+0000 7faec028b700 1 Processor -- start 2023-11-26T07:16:37.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.528+0000 7faec028b700 1 -- start start 2023-11-26T07:16:37.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faec028b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faea4003a00 0x7faeb814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faec028b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb80cdc20 0x7faeb80ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.537 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faec028b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faeb80c6530 0x7faeb80c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faec028b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb80ce5e0 con 0x7faeb80c6530 2023-11-26T07:16:37.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faec028b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb80b76d0 con 0x7faeb80cdc20 2023-11-26T07:16:37.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faec028b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb80b6910 con 0x7faea4003a00 2023-11-26T07:16:37.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faebe828700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faeb80c6530 0x7faeb80c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faebe828700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faeb80c6530 0x7faeb80c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40978/0 (socket says 172.21.15.83:40978) 2023-11-26T07:16:37.538 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faebe828700 1 -- 172.21.15.83:0/3901649636 learned_addr learned my addr 172.21.15.83:0/3901649636 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faebe027700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faea4003a00 0x7faeb814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faebe828700 1 -- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faea4003a00 msgr2=0x7faeb814cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faebd826700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb80cdc20 0x7faeb80ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faebe828700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faea4003a00 0x7faeb814cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faebe828700 1 -- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb80cdc20 msgr2=0x7faeb80ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faebe828700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb80cdc20 0x7faeb80ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faebe828700 1 -- 172.21.15.83:0/3901649636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb80b6bf0 con 0x7faeb80c6530 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faebe828700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faeb80c6530 0x7faeb80c69b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7faeb8071560 tx=0x7faeb00114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faeaeffd700 1 -- 172.21.15.83:0/3901649636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb000e070 con 0x7faeb80c6530 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faeaeffd700 1 -- 172.21.15.83:0/3901649636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb0003c30 con 0x7faeb80c6530 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faec028b700 1 -- 172.21.15.83:0/3901649636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb80b6ed0 con 0x7faeb80c6530 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faeaeffd700 1 -- 172.21.15.83:0/3901649636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb0003e20 con 0x7faeb80c6530 2023-11-26T07:16:37.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.529+0000 7faec028b700 1 -- 172.21.15.83:0/3901649636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb80c6ef0 con 0x7faeb80c6530 2023-11-26T07:16:37.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.530+0000 7faec028b700 1 -- 172.21.15.83:0/3901649636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb8067f60 con 0x7faeb80c6530 2023-11-26T07:16:37.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.531+0000 7faeaeffd700 1 -- 172.21.15.83:0/3901649636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7faeb0011a10 con 0x7faeb80c6530 2023-11-26T07:16:37.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.531+0000 7faeaeffd700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae9c06aff0 0x7fae9c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:37.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.531+0000 7faeaeffd700 1 -- 172.21.15.83:0/3901649636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faeb0089cb0 con 0x7faeb80c6530 2023-11-26T07:16:37.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.532+0000 7faebe027700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae9c06aff0 0x7fae9c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:37.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.532+0000 7faebe027700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae9c06aff0 0x7fae9c06d4b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faeb4014040 tx=0x7faeb4005040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:37.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.536+0000 7faeaeffd700 1 -- 172.21.15.83:0/3901649636 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faeb0054ae0 con 0x7faeb80c6530 2023-11-26T07:16:37.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.742+0000 7faec028b700 1 -- 172.21.15.83:0/3901649636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faeb80b7c60 con 0x7faeb80c6530 2023-11-26T07:16:37.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.742+0000 7faeaeffd700 1 -- 172.21.15.83:0/3901649636 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v90) v1 ==== 80+0+120 (secure 0 0 0) 0x7faeb0058100 con 0x7faeb80c6530 2023-11-26T07:16:37.744 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6835/469139444 2023-11-27T07:16:32.809518+0000 2023-11-26T07:16:37.745 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6833/469139444 2023-11-27T07:16:32.809518+0000 2023-11-26T07:16:37.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 -- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae9c06aff0 msgr2=0x7fae9c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae9c06aff0 0x7fae9c06d4b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faeb4014040 tx=0x7faeb4005040 comp rx=0 tx=0).stop 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 -- 172.21.15.83:0/3901649636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faeb80c6530 msgr2=0x7faeb80c69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faeb80c6530 0x7faeb80c69b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7faeb8071560 tx=0x7faeb00114b0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 -- 172.21.15.83:0/3901649636 shutdown_connections 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fae9c06aff0 0x7fae9c06d4b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faeb80c6530 0x7faeb80c69b0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faea4003a00 0x7faeb814cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 --2- 172.21.15.83:0/3901649636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb80cdc20 0x7faeb80ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 -- 172.21.15.83:0/3901649636 >> 172.21.15.83:0/3901649636 conn(0x7faeb80731a0 msgr2=0x7faeb806eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 -- 172.21.15.83:0/3901649636 shutdown_connections 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:37.744+0000 7faec028b700 1 -- 172.21.15.83:0/3901649636 wait complete. 2023-11-26T07:16:37.748 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2023-11-26T07:16:37.759 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:16:37.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:37.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:16:37.801 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:16:37.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:37.801 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:16:37.858 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:16:37.858 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:37.879 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:16:37.879 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:37.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.057+0000 7fdc35631700 1 Processor -- start 2023-11-26T07:16:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.057+0000 7fdc35631700 1 -- start start 2023-11-26T07:16:38.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.057+0000 7fdc35631700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdc3015bd30 0x7fdc300cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.057+0000 7fdc35631700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc30162920 0x7fdc300cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.057+0000 7fdc35631700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc3015e6e0 0x7fdc3015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.057+0000 7fdc35631700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc30161bf0 con 0x7fdc3015e6e0 2023-11-26T07:16:38.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.057+0000 7fdc35631700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc300cff40 con 0x7fdc3015bd30 2023-11-26T07:16:38.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.057+0000 7fdc35631700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3015f450 con 0x7fdc30162920 2023-11-26T07:16:38.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.057+0000 7fdc2f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc3015e6e0 0x7fdc3015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.058+0000 7fdc2f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdc3015e6e0 0x7fdc3015eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40982/0 (socket says 172.21.15.83:40982) 2023-11-26T07:16:38.061 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.058+0000 7fdc2f7fe700 1 -- 172.21.15.83:0/1532024646 learned_addr learned my addr 172.21.15.83:0/1532024646 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.058+0000 7fdc2f7fe700 1 -- 172.21.15.83:0/1532024646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc30162920 msgr2=0x7fdc300cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.058+0000 7fdc2f7fe700 1 --2- 172.21.15.83:0/1532024646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc30162920 0x7fdc300cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.058+0000 7fdc2f7fe700 1 -- 172.21.15.83:0/1532024646 >> v2:172.21.15.203:3300/0 conn(0x7fdc3015bd30 msgr2=0x7fdc300cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.058+0000 7fdc2f7fe700 1 --2- 172.21.15.83:0/1532024646 >> v2:172.21.15.203:3300/0 conn(0x7fdc3015bd30 0x7fdc300cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.058+0000 7fdc2f7fe700 1 -- 172.21.15.83:0/1532024646 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc30161e40 con 0x7fdc3015e6e0 2023-11-26T07:16:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.058+0000 7fdc2f7fe700 1 --2- 172.21.15.83:0/1532024646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc3015e6e0 0x7fdc3015eb70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdc300723e0 tx=0x7fdc2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b8d1ce7ccbb33c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.058+0000 7fdc2dffb700 1 -- 172.21.15.83:0/1532024646 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2800e070 con 0x7fdc3015e6e0 2023-11-26T07:16:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.058+0000 7fdc2dffb700 1 -- 172.21.15.83:0/1532024646 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2800b9c0 con 0x7fdc3015e6e0 2023-11-26T07:16:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.058+0000 7fdc2dffb700 1 -- 172.21.15.83:0/1532024646 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2800bd70 con 0x7fdc3015e6e0 2023-11-26T07:16:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.059+0000 7fdc35631700 1 -- 172.21.15.83:0/1532024646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc3015e6e0 msgr2=0x7fdc3015eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.059+0000 7fdc35631700 1 --2- 172.21.15.83:0/1532024646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc3015e6e0 0x7fdc3015eb70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdc300723e0 tx=0x7fdc2800b330 comp rx=0 tx=0).stop 2023-11-26T07:16:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.059+0000 7fdc35631700 1 -- 172.21.15.83:0/1532024646 shutdown_connections 2023-11-26T07:16:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.059+0000 7fdc35631700 1 --2- 172.21.15.83:0/1532024646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc3015e6e0 0x7fdc3015eb70 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.059+0000 7fdc35631700 1 --2- 172.21.15.83:0/1532024646 >> v2:172.21.15.203:3300/0 conn(0x7fdc3015bd30 0x7fdc300cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.059+0000 7fdc35631700 1 --2- 172.21.15.83:0/1532024646 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc30162920 0x7fdc300cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.059+0000 7fdc35631700 1 -- 172.21.15.83:0/1532024646 >> 172.21.15.83:0/1532024646 conn(0x7fdc30072710 msgr2=0x7fdc30072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.059+0000 7fdc35631700 1 -- 172.21.15.83:0/1532024646 shutdown_connections 2023-11-26T07:16:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.059+0000 7fdc35631700 1 -- 172.21.15.83:0/1532024646 wait complete. 2023-11-26T07:16:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.060+0000 7fdc35631700 1 Processor -- start 2023-11-26T07:16:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.060+0000 7fdc35631700 1 -- start start 2023-11-26T07:16:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc35631700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc3015bd30 0x7fdc3006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc35631700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc3015e6e0 0x7fdc30156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc35631700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc30162920 0x7fdc300c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc35631700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc30157bf0 con 0x7fdc30162920 2023-11-26T07:16:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc35631700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc301614c0 con 0x7fdc3015bd30 2023-11-26T07:16:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc35631700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc30156db0 con 0x7fdc3015e6e0 2023-11-26T07:16:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc2f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc30162920 0x7fdc300c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc2f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc30162920 0x7fdc300c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40992/0 (socket says 172.21.15.83:40992) 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc2f7fe700 1 -- 172.21.15.83:0/4214059720 learned_addr learned my addr 172.21.15.83:0/4214059720 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc2f7fe700 1 -- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc3015e6e0 msgr2=0x7fdc30156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc2effd700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc3015bd30 0x7fdc3006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc2f7fe700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc3015e6e0 0x7fdc30156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc2f7fe700 1 -- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc3015bd30 msgr2=0x7fdc3006e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc2f7fe700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc3015bd30 0x7fdc3006e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc2f7fe700 1 -- 172.21.15.83:0/4214059720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc28009b30 con 0x7fdc30162920 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc2f7fe700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc30162920 0x7fdc300c9c90 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800b370 tx=0x7fdc28004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.061+0000 7fdc177fe700 1 -- 172.21.15.83:0/4214059720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28004310 con 0x7fdc30162920 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.062+0000 7fdc177fe700 1 -- 172.21.15.83:0/4214059720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc28004490 con 0x7fdc30162920 2023-11-26T07:16:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.062+0000 7fdc177fe700 1 -- 172.21.15.83:0/4214059720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28004840 con 0x7fdc30162920 2023-11-26T07:16:38.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.062+0000 7fdc35631700 1 -- 172.21.15.83:0/4214059720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc300ca1d0 con 0x7fdc30162920 2023-11-26T07:16:38.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.062+0000 7fdc35631700 1 -- 172.21.15.83:0/4214059720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc30157350 con 0x7fdc30162920 2023-11-26T07:16:38.068 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.063+0000 7fdc35631700 1 -- 172.21.15.83:0/4214059720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc300ccb20 con 0x7fdc30162920 2023-11-26T07:16:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.063+0000 7fdc177fe700 1 -- 172.21.15.83:0/4214059720 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fdc2801c440 con 0x7fdc30162920 2023-11-26T07:16:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.064+0000 7fdc177fe700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1c06b020 0x7fdc1c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.064+0000 7fdc177fe700 1 -- 172.21.15.83:0/4214059720 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdc2808b440 con 0x7fdc30162920 2023-11-26T07:16:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.064+0000 7fdc2effd700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1c06b020 0x7fdc1c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.065+0000 7fdc2effd700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1c06b020 0x7fdc1c06d4e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc24006fd0 tx=0x7fdc24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.068+0000 7fdc177fe700 1 -- 172.21.15.83:0/4214059720 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc28056260 con 0x7fdc30162920 2023-11-26T07:16:38.262 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.260+0000 7fdc35631700 1 -- 172.21.15.83:0/4214059720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc300751e0 con 0x7fdc30162920 2023-11-26T07:16:38.264 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.261+0000 7fdc177fe700 1 -- 172.21.15.83:0/4214059720 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16675 (secure 0 0 0) 0x7fdc28059880 con 0x7fdc30162920 2023-11-26T07:16:38.264 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:38.264 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:35.818271+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:16:15.747889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:16:16.943249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6835/469139444":"2023-11-27T07:16:32.809518+0000","172.21.15.83:6833/469139444":"2023-11-27T07:16:32.809518+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:38.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 -- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1c06b020 msgr2=0x7fdc1c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1c06b020 0x7fdc1c06d4e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc24006fd0 tx=0x7fdc24008040 comp rx=0 tx=0).stop 2023-11-26T07:16:38.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 -- 172.21.15.83:0/4214059720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc30162920 msgr2=0x7fdc300c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc30162920 0x7fdc300c9c90 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800b370 tx=0x7fdc28004ad0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 -- 172.21.15.83:0/4214059720 shutdown_connections 2023-11-26T07:16:38.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fdc1c06b020 0x7fdc1c06d4e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc30162920 0x7fdc300c9c90 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.266 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdc3015bd30 0x7fdc3006e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 --2- 172.21.15.83:0/4214059720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdc3015e6e0 0x7fdc30156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 -- 172.21.15.83:0/4214059720 >> 172.21.15.83:0/4214059720 conn(0x7fdc30072710 msgr2=0x7fdc30150ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:38.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 -- 172.21.15.83:0/4214059720 shutdown_connections 2023-11-26T07:16:38.267 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.263+0000 7fdc35631700 1 -- 172.21.15.83:0/4214059720 wait complete. 2023-11-26T07:16:38.278 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:16:15.747889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:16:16.943249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:16:38.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:16:38.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.400+0000 7f6317aa2700 1 Processor -- start 2023-11-26T07:16:38.402 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.400+0000 7f6317aa2700 1 -- start start 2023-11-26T07:16:38.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f6317aa2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f631015e2d0 0x7f631006ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f6317aa2700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f63100ceb70 0x7f63100cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f6317aa2700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63100cf550 con 0x7f63100b4230 2023-11-26T07:16:38.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f6317aa2700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631015e6b0 con 0x7f63100ceb70 2023-11-26T07:16:38.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f6317aa2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63100cc9b0 con 0x7f631015e2d0 2023-11-26T07:16:38.405 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f631603f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f63100ceb70 0x7f63100cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f631583e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f631015e2d0 0x7f631006ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f631503d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f63100b4230 0x7f63100ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:45654/0 (socket says 172.21.15.83:45654) 2023-11-26T07:16:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f631503d700 1 -- 172.21.15.83:0/887692599 learned_addr learned my addr 172.21.15.83:0/887692599 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f631583e700 1 -- 172.21.15.83:0/887692599 >> v2:172.21.15.203:3300/0 conn(0x7f63100ceb70 msgr2=0x7f63100cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f631583e700 1 --2- 172.21.15.83:0/887692599 >> v2:172.21.15.203:3300/0 conn(0x7f63100ceb70 0x7f63100cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f631583e700 1 -- 172.21.15.83:0/887692599 >> v1:172.21.15.203:6789/0 conn(0x7f63100b4230 legacy=0x7f63100ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f631483c700 1 -- 172.21.15.83:0/887692599 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889506699 0 0) 0x7f63100cf550 con 0x7f63100b4230 2023-11-26T07:16:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f631583e700 1 -- 172.21.15.83:0/887692599 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63100cf000 con 0x7f631015e2d0 2023-11-26T07:16:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.401+0000 7f631603f700 1 --2- 172.21.15.83:0/887692599 >> v2:172.21.15.203:3300/0 conn(0x7f63100ceb70 0x7f63100cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f631583e700 1 --2- 172.21.15.83:0/887692599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f631015e2d0 0x7f631006ef80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f630c009840 tx=0x7f630c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b82765e164aabd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f631483c700 1 -- 172.21.15.83:0/887692599 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c011830 con 0x7f631015e2d0 2023-11-26T07:16:38.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f631483c700 1 -- 172.21.15.83:0/887692599 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c011990 con 0x7f631015e2d0 2023-11-26T07:16:38.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f631483c700 1 -- 172.21.15.83:0/887692599 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c011d20 con 0x7f631015e2d0 2023-11-26T07:16:38.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f6317aa2700 1 -- 172.21.15.83:0/887692599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f631015e2d0 msgr2=0x7f631006ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f6317aa2700 1 --2- 172.21.15.83:0/887692599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f631015e2d0 0x7f631006ef80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f630c009840 tx=0x7f630c009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:38.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f6317aa2700 1 -- 172.21.15.83:0/887692599 shutdown_connections 2023-11-26T07:16:38.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f6317aa2700 1 --2- 172.21.15.83:0/887692599 >> v2:172.21.15.203:3300/0 conn(0x7f63100ceb70 0x7f63100cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f6317aa2700 1 --2- 172.21.15.83:0/887692599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f631015e2d0 0x7f631006ef80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f6317aa2700 1 -- 172.21.15.83:0/887692599 >> 172.21.15.83:0/887692599 conn(0x7f63100b3710 msgr2=0x7f63100b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:38.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f6317aa2700 1 -- 172.21.15.83:0/887692599 shutdown_connections 2023-11-26T07:16:38.407 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.402+0000 7f6317aa2700 1 -- 172.21.15.83:0/887692599 wait complete. 2023-11-26T07:16:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.404+0000 7f6317aa2700 1 Processor -- start 2023-11-26T07:16:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.404+0000 7f6317aa2700 1 -- start start 2023-11-26T07:16:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.404+0000 7f6317aa2700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63100b4230 0x7f631015b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.404+0000 7f6317aa2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63100ceb70 0x7f63100cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.404+0000 7f6317aa2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f631015e2d0 0x7f63100b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.404+0000 7f6317aa2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100cac00 con 0x7f631015e2d0 2023-11-26T07:16:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.404+0000 7f6317aa2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631015b870 con 0x7f63100ceb70 2023-11-26T07:16:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.404+0000 7f6317aa2700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63100c9ff0 con 0x7f63100b4230 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f631603f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f631015e2d0 0x7f63100b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f631603f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f631015e2d0 0x7f63100b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41002/0 (socket says 172.21.15.83:41002) 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f631603f700 1 -- 172.21.15.83:0/953171581 learned_addr learned my addr 172.21.15.83:0/953171581 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f631583e700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63100b4230 0x7f631015b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f631503d700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63100ceb70 0x7f63100cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f631603f700 1 -- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63100b4230 msgr2=0x7f631015b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f631603f700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63100b4230 0x7f631015b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f631603f700 1 -- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63100ceb70 msgr2=0x7f63100cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f631603f700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63100ceb70 0x7f63100cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f631603f700 1 -- 172.21.15.83:0/953171581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6308009a70 con 0x7f631015e2d0 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f631603f700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f631015e2d0 0x7f63100b6ef0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f630800efc0 tx=0x7f630800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f63067fc700 1 -- 172.21.15.83:0/953171581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630800ca90 con 0x7f631015e2d0 2023-11-26T07:16:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f63067fc700 1 -- 172.21.15.83:0/953171581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630800f040 con 0x7f631015e2d0 2023-11-26T07:16:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f63067fc700 1 -- 172.21.15.83:0/953171581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630800ce10 con 0x7f631015e2d0 2023-11-26T07:16:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f6317aa2700 1 -- 172.21.15.83:0/953171581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c0094a0 con 0x7f631015e2d0 2023-11-26T07:16:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.405+0000 7f6317aa2700 1 -- 172.21.15.83:0/953171581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63100ca5b0 con 0x7f631015e2d0 2023-11-26T07:16:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.406+0000 7f6317aa2700 1 -- 172.21.15.83:0/953171581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6310072f00 con 0x7f631015e2d0 2023-11-26T07:16:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.407+0000 7f63067fc700 1 -- 172.21.15.83:0/953171581 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f6308014480 con 0x7f631015e2d0 2023-11-26T07:16:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.407+0000 7f63067fc700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62f406aff0 0x7f62f406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.407+0000 7f63067fc700 1 -- 172.21.15.83:0/953171581 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6308089fb0 con 0x7f631015e2d0 2023-11-26T07:16:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.407+0000 7f631583e700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62f406aff0 0x7f62f406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.408+0000 7f631583e700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62f406aff0 0x7f62f406d4b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f630c000c00 tx=0x7f630c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:38.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.411+0000 7f63067fc700 1 -- 172.21.15.83:0/953171581 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63080bce90 con 0x7f631015e2d0 2023-11-26T07:16:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.600+0000 7f6317aa2700 1 -- 172.21.15.83:0/953171581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6310068fb0 con 0x7f631015e2d0 2023-11-26T07:16:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.600+0000 7f63067fc700 1 -- 172.21.15.83:0/953171581 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6308058870 con 0x7f631015e2d0 2023-11-26T07:16:38.602 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:16:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 -- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62f406aff0 msgr2=0x7f62f406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62f406aff0 0x7f62f406d4b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f630c000c00 tx=0x7f630c008030 comp rx=0 tx=0).stop 2023-11-26T07:16:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 -- 172.21.15.83:0/953171581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f631015e2d0 msgr2=0x7f63100b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f631015e2d0 0x7f63100b6ef0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f630800efc0 tx=0x7f630800c3a0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 -- 172.21.15.83:0/953171581 shutdown_connections 2023-11-26T07:16:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62f406aff0 0x7f62f406d4b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f631015e2d0 0x7f63100b6ef0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63100b4230 0x7f631015b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 --2- 172.21.15.83:0/953171581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63100ceb70 0x7f63100cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 -- 172.21.15.83:0/953171581 >> 172.21.15.83:0/953171581 conn(0x7f63100b3710 msgr2=0x7f63100cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 -- 172.21.15.83:0/953171581 shutdown_connections 2023-11-26T07:16:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.602+0000 7f6317aa2700 1 -- 172.21.15.83:0/953171581 wait complete. 2023-11-26T07:16:38.617 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:16:38.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.738+0000 7f57273c0700 1 Processor -- start 2023-11-26T07:16:38.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.738+0000 7f57273c0700 1 -- start start 2023-11-26T07:16:38.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.739+0000 7f57273c0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f572015e160 0x7f57200b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.739+0000 7f57273c0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f57200afc90 0x7f5720073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.739+0000 7f57273c0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57200744d0 0x7f572006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.739+0000 7f57273c0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57200b4500 con 0x7f57200744d0 2023-11-26T07:16:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.739+0000 7f57273c0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5720074960 con 0x7f57200afc90 2023-11-26T07:16:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.739+0000 7f57273c0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57200b3c60 con 0x7f572015e160 2023-11-26T07:16:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.739+0000 7f572595d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57200744d0 0x7f572006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.739+0000 7f572595d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57200744d0 0x7f572006d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41006/0 (socket says 172.21.15.83:41006) 2023-11-26T07:16:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.739+0000 7f572595d700 1 -- 172.21.15.83:0/1235214961 learned_addr learned my addr 172.21.15.83:0/1235214961 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.740+0000 7f572495b700 1 --2- 172.21.15.83:0/1235214961 >> v2:172.21.15.203:3300/0 conn(0x7f57200afc90 0x7f5720073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.740+0000 7f572515c700 1 --2- 172.21.15.83:0/1235214961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f572015e160 0x7f57200b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.740+0000 7f572595d700 1 -- 172.21.15.83:0/1235214961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f572015e160 msgr2=0x7f57200b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.740+0000 7f572595d700 1 --2- 172.21.15.83:0/1235214961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f572015e160 0x7f57200b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.740+0000 7f572595d700 1 -- 172.21.15.83:0/1235214961 >> v2:172.21.15.203:3300/0 conn(0x7f57200afc90 msgr2=0x7f5720073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.740+0000 7f572595d700 1 --2- 172.21.15.83:0/1235214961 >> v2:172.21.15.203:3300/0 conn(0x7f57200afc90 0x7f5720073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.740+0000 7f572595d700 1 -- 172.21.15.83:0/1235214961 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57200b58c0 con 0x7f57200744d0 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.740+0000 7f572595d700 1 --2- 172.21.15.83:0/1235214961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57200744d0 0x7f572006d930 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f572009f020 tx=0x7f571800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa929bb0741025ae server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.740+0000 7f5717fff700 1 -- 172.21.15.83:0/1235214961 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571800e070 con 0x7f57200744d0 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.740+0000 7f5717fff700 1 -- 172.21.15.83:0/1235214961 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571800b9c0 con 0x7f57200744d0 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.740+0000 7f5717fff700 1 -- 172.21.15.83:0/1235214961 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571800bd70 con 0x7f57200744d0 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.741+0000 7f57273c0700 1 -- 172.21.15.83:0/1235214961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57200744d0 msgr2=0x7f572006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.741+0000 7f57273c0700 1 --2- 172.21.15.83:0/1235214961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57200744d0 0x7f572006d930 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f572009f020 tx=0x7f571800b330 comp rx=0 tx=0).stop 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.741+0000 7f57273c0700 1 -- 172.21.15.83:0/1235214961 shutdown_connections 2023-11-26T07:16:38.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.741+0000 7f57273c0700 1 --2- 172.21.15.83:0/1235214961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57200744d0 0x7f572006d930 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.741+0000 7f57273c0700 1 --2- 172.21.15.83:0/1235214961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f572015e160 0x7f57200b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.741+0000 7f57273c0700 1 --2- 172.21.15.83:0/1235214961 >> v2:172.21.15.203:3300/0 conn(0x7f57200afc90 0x7f5720073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.741+0000 7f57273c0700 1 -- 172.21.15.83:0/1235214961 >> 172.21.15.83:0/1235214961 conn(0x7f57200d0bf0 msgr2=0x7f57200d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.741+0000 7f57273c0700 1 -- 172.21.15.83:0/1235214961 shutdown_connections 2023-11-26T07:16:38.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.741+0000 7f57273c0700 1 -- 172.21.15.83:0/1235214961 wait complete. 2023-11-26T07:16:38.748 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.742+0000 7f57273c0700 1 Processor -- start 2023-11-26T07:16:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.742+0000 7f57273c0700 1 -- start start 2023-11-26T07:16:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f57273c0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57200744d0 0x7f572015e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f57273c0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57200afc90 0x7f57201f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f57273c0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f572015e160 0x7f5720147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f57273c0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572015ead0 con 0x7f57200744d0 2023-11-26T07:16:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f57273c0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57201f4900 con 0x7f57200afc90 2023-11-26T07:16:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f57273c0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572015ecc0 con 0x7f572015e160 2023-11-26T07:16:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f572515c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57200744d0 0x7f572015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f572515c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57200744d0 0x7f572015e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41010/0 (socket says 172.21.15.83:41010) 2023-11-26T07:16:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f572515c700 1 -- 172.21.15.83:0/2785915825 learned_addr learned my addr 172.21.15.83:0/2785915825 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f572515c700 1 -- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f572015e160 msgr2=0x7f5720147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f572515c700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f572015e160 0x7f5720147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f572515c700 1 -- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57200afc90 msgr2=0x7f57201f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f572515c700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57200afc90 0x7f57201f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f572515c700 1 -- 172.21.15.83:0/2785915825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5718009b30 con 0x7f57200744d0 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f572515c700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57200744d0 0x7f572015e590 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f571000c7f0 tx=0x7f571000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f5715ffb700 1 -- 172.21.15.83:0/2785915825 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710011800 con 0x7f57200744d0 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f5715ffb700 1 -- 172.21.15.83:0/2785915825 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5710011960 con 0x7f57200744d0 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f57273c0700 1 -- 172.21.15.83:0/2785915825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572015efa0 con 0x7f57200744d0 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.743+0000 7f5715ffb700 1 -- 172.21.15.83:0/2785915825 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710011cf0 con 0x7f57200744d0 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.744+0000 7f57273c0700 1 -- 172.21.15.83:0/2785915825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572015f8c0 con 0x7f57200744d0 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.744+0000 7f57273c0700 1 -- 172.21.15.83:0/2785915825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572015f190 con 0x7f57200744d0 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.745+0000 7f5715ffb700 1 -- 172.21.15.83:0/2785915825 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f5710018360 con 0x7f57200744d0 2023-11-26T07:16:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.745+0000 7f5715ffb700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f570406aff0 0x7f570406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:38.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.745+0000 7f5715ffb700 1 -- 172.21.15.83:0/2785915825 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5710089750 con 0x7f57200744d0 2023-11-26T07:16:38.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.745+0000 7f572495b700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f570406aff0 0x7f570406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:38.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.746+0000 7f572495b700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f570406aff0 0x7f570406d4b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f572015d880 tx=0x7f571c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.749+0000 7f5715ffb700 1 -- 172.21.15.83:0/2785915825 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5710054580 con 0x7f57200744d0 2023-11-26T07:16:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.941+0000 7f57273c0700 1 -- 172.21.15.83:0/2785915825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f572015f390 con 0x7f57200744d0 2023-11-26T07:16:38.943 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.941+0000 7f5715ffb700 1 -- 172.21.15.83:0/2785915825 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5710057ba0 con 0x7f57200744d0 2023-11-26T07:16:38.944 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:16:38.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 -- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f570406aff0 msgr2=0x7f570406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f570406aff0 0x7f570406d4b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f572015d880 tx=0x7f571c008040 comp rx=0 tx=0).stop 2023-11-26T07:16:38.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 -- 172.21.15.83:0/2785915825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57200744d0 msgr2=0x7f572015e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57200744d0 0x7f572015e590 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f571000c7f0 tx=0x7f571000cbb0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 -- 172.21.15.83:0/2785915825 shutdown_connections 2023-11-26T07:16:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f570406aff0 0x7f570406d4b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f572015e160 0x7f5720147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57200744d0 0x7f572015e590 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 --2- 172.21.15.83:0/2785915825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57200afc90 0x7f57201f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 -- 172.21.15.83:0/2785915825 >> 172.21.15.83:0/2785915825 conn(0x7f57200d0bf0 msgr2=0x7f5720071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 -- 172.21.15.83:0/2785915825 shutdown_connections 2023-11-26T07:16:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:38.943+0000 7f57273c0700 1 -- 172.21.15.83:0/2785915825 wait complete. 2023-11-26T07:16:38.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:16:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.078+0000 7f54d4344700 1 Processor -- start 2023-11-26T07:16:39.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.078+0000 7f54d4344700 1 -- start start 2023-11-26T07:16:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.079+0000 7f54d4344700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f54cc15ada0 0x7f54cc06f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.079+0000 7f54d4344700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f54cc06d3c0 0x7f54cc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.079+0000 7f54d4344700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54cc0cccd0 con 0x7f54cc0afb40 2023-11-26T07:16:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.079+0000 7f54d4344700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc0cd2e0 con 0x7f54cc15ada0 2023-11-26T07:16:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.079+0000 7f54d4344700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc0cd5c0 con 0x7f54cc06d3c0 2023-11-26T07:16:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d20e0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f54cc15ada0 0x7f54cc06f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d20e0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f54cc15ada0 0x7f54cc06f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41018/0 (socket says 172.21.15.83:41018) 2023-11-26T07:16:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d20e0700 1 -- 172.21.15.83:0/3598816105 learned_addr learned my addr 172.21.15.83:0/3598816105 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d28e1700 1 --2- 172.21.15.83:0/3598816105 >> v2:172.21.15.203:3300/0 conn(0x7f54cc06d3c0 0x7f54cc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d20e0700 1 -- 172.21.15.83:0/3598816105 >> v2:172.21.15.203:3300/0 conn(0x7f54cc06d3c0 msgr2=0x7f54cc06d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d20e0700 1 --2- 172.21.15.83:0/3598816105 >> v2:172.21.15.203:3300/0 conn(0x7f54cc06d3c0 0x7f54cc06d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d20e0700 1 -- 172.21.15.83:0/3598816105 >> v1:172.21.15.203:6789/0 conn(0x7f54cc0afb40 legacy=0x7f54cc0cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d20e0700 1 -- 172.21.15.83:0/3598816105 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc06fd50 con 0x7f54cc15ada0 2023-11-26T07:16:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d20e0700 1 --2- 172.21.15.83:0/3598816105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54cc15ada0 0x7f54cc06f810 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0d0210 tx=0x7f54bc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7be8d999f8d4489 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d10de700 1 -- 172.21.15.83:0/3598816105 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc00dcf0 con 0x7f54cc15ada0 2023-11-26T07:16:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d10de700 1 -- 172.21.15.83:0/3598816105 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc00de50 con 0x7f54cc15ada0 2023-11-26T07:16:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.080+0000 7f54d10de700 1 -- 172.21.15.83:0/3598816105 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc004250 con 0x7f54cc15ada0 2023-11-26T07:16:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.081+0000 7f54d4344700 1 -- 172.21.15.83:0/3598816105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54cc15ada0 msgr2=0x7f54cc06f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.081+0000 7f54d4344700 1 --2- 172.21.15.83:0/3598816105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54cc15ada0 0x7f54cc06f810 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0d0210 tx=0x7f54bc00be70 comp rx=0 tx=0).stop 2023-11-26T07:16:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.081+0000 7f54d4344700 1 -- 172.21.15.83:0/3598816105 shutdown_connections 2023-11-26T07:16:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.081+0000 7f54d4344700 1 --2- 172.21.15.83:0/3598816105 >> v2:172.21.15.203:3300/0 conn(0x7f54cc06d3c0 0x7f54cc06d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.081+0000 7f54d4344700 1 --2- 172.21.15.83:0/3598816105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54cc15ada0 0x7f54cc06f810 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.081+0000 7f54d4344700 1 -- 172.21.15.83:0/3598816105 >> 172.21.15.83:0/3598816105 conn(0x7f54cc0b2050 msgr2=0x7f54cc0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.081+0000 7f54d4344700 1 -- 172.21.15.83:0/3598816105 shutdown_connections 2023-11-26T07:16:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.081+0000 7f54d4344700 1 -- 172.21.15.83:0/3598816105 wait complete. 2023-11-26T07:16:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.082+0000 7f54d4344700 1 Processor -- start 2023-11-26T07:16:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.082+0000 7f54d4344700 1 -- start start 2023-11-26T07:16:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d4344700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54cc06d3c0 0x7f54cc15c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d4344700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54cc0afb40 0x7f54cc15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d4344700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54cc15ada0 0x7f54cc1ef070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d4344700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc157df0 con 0x7f54cc15ada0 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d4344700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc157170 con 0x7f54cc06d3c0 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d4344700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc157450 con 0x7f54cc0afb40 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d28e1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54cc15ada0 0x7f54cc1ef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d28e1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54cc15ada0 0x7f54cc1ef070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41028/0 (socket says 172.21.15.83:41028) 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d28e1700 1 -- 172.21.15.83:0/1061623471 learned_addr learned my addr 172.21.15.83:0/1061623471 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d18df700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54cc0afb40 0x7f54cc15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d20e0700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54cc06d3c0 0x7f54cc15c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d28e1700 1 -- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54cc0afb40 msgr2=0x7f54cc15c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d28e1700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54cc0afb40 0x7f54cc15c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d28e1700 1 -- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54cc06d3c0 msgr2=0x7f54cc15c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d28e1700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54cc06d3c0 0x7f54cc15c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d28e1700 1 -- 172.21.15.83:0/1061623471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc00b7f0 con 0x7f54cc15ada0 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d28e1700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54cc15ada0 0x7f54cc1ef070 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f54c400cd70 tx=0x7f54c400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54c2ffd700 1 -- 172.21.15.83:0/1061623471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4003a20 con 0x7f54cc15ada0 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54c2ffd700 1 -- 172.21.15.83:0/1061623471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c4003b80 con 0x7f54cc15ada0 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d4344700 1 -- 172.21.15.83:0/1061623471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54cc157730 con 0x7f54cc15ada0 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54c2ffd700 1 -- 172.21.15.83:0/1061623471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4004020 con 0x7f54cc15ada0 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.083+0000 7f54d4344700 1 -- 172.21.15.83:0/1061623471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc15bca0 con 0x7f54cc15ada0 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.084+0000 7f54d4344700 1 -- 172.21.15.83:0/1061623471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54cc068d90 con 0x7f54cc15ada0 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.085+0000 7f54c2ffd700 1 -- 172.21.15.83:0/1061623471 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f54c4004180 con 0x7f54cc15ada0 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.085+0000 7f54c2ffd700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f54b006aff0 0x7f54b006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.085+0000 7f54c2ffd700 1 -- 172.21.15.83:0/1061623471 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f54c4012070 con 0x7f54cc15ada0 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.085+0000 7f54d20e0700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f54b006aff0 0x7f54b006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.090 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.086+0000 7f54d20e0700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f54b006aff0 0x7f54b006d4b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54bc000c00 tx=0x7f54bc012000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:39.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.090+0000 7f54c2ffd700 1 -- 172.21.15.83:0/1061623471 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f54c4054f40 con 0x7f54cc15ada0 2023-11-26T07:16:39.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.279+0000 7f54d4344700 1 -- 172.21.15.83:0/1061623471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54cc157f30 con 0x7f54cc15ada0 2023-11-26T07:16:39.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.279+0000 7f54c2ffd700 1 -- 172.21.15.83:0/1061623471 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f54cc157f30 con 0x7f54cc15ada0 2023-11-26T07:16:39.282 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:16:39.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.281+0000 7f54d4344700 1 -- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f54b006aff0 msgr2=0x7f54b006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.281+0000 7f54d4344700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f54b006aff0 0x7f54b006d4b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54bc000c00 tx=0x7f54bc012000 comp rx=0 tx=0).stop 2023-11-26T07:16:39.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.281+0000 7f54d4344700 1 -- 172.21.15.83:0/1061623471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54cc15ada0 msgr2=0x7f54cc1ef070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.281+0000 7f54d4344700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54cc15ada0 0x7f54cc1ef070 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f54c400cd70 tx=0x7f54c400a570 comp rx=0 tx=0).stop 2023-11-26T07:16:39.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.282+0000 7f54d4344700 1 -- 172.21.15.83:0/1061623471 shutdown_connections 2023-11-26T07:16:39.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.282+0000 7f54d4344700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f54b006aff0 0x7f54b006d4b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.282+0000 7f54d4344700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54cc15ada0 0x7f54cc1ef070 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.282+0000 7f54d4344700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54cc06d3c0 0x7f54cc15c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.282+0000 7f54d4344700 1 --2- 172.21.15.83:0/1061623471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54cc0afb40 0x7f54cc15c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.282+0000 7f54d4344700 1 -- 172.21.15.83:0/1061623471 >> 172.21.15.83:0/1061623471 conn(0x7f54cc0b2050 msgr2=0x7f54cc0ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:39.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.282+0000 7f54d4344700 1 -- 172.21.15.83:0/1061623471 shutdown_connections 2023-11-26T07:16:39.285 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.282+0000 7f54d4344700 1 -- 172.21.15.83:0/1061623471 wait complete. 2023-11-26T07:16:39.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:16:39.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.415+0000 7ffb46812700 1 Processor -- start 2023-11-26T07:16:39.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.415+0000 7ffb46812700 1 -- start start 2023-11-26T07:16:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb46812700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ffb400b3370 0x7ffb401604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb46812700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb400d1c70 0x7ffb400d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb46812700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb400b22d0 con 0x7ffb4015bdd0 2023-11-26T07:16:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb46812700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb400b2f50 con 0x7ffb400d1c70 2023-11-26T07:16:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb46812700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb400d2640 con 0x7ffb400b3370 2023-11-26T07:16:39.420 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb44daf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb400d1c70 0x7ffb400d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb44daf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ffb400d1c70 0x7ffb400d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41036/0 (socket says 172.21.15.83:41036) 2023-11-26T07:16:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb44daf700 1 -- 172.21.15.83:0/594187597 learned_addr learned my addr 172.21.15.83:0/594187597 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb3f7fe700 1 --2- 172.21.15.83:0/594187597 >> v2:172.21.15.203:3300/0 conn(0x7ffb400b3370 0x7ffb401604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb44daf700 1 -- 172.21.15.83:0/594187597 >> v2:172.21.15.203:3300/0 conn(0x7ffb400b3370 msgr2=0x7ffb401604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb44daf700 1 --2- 172.21.15.83:0/594187597 >> v2:172.21.15.203:3300/0 conn(0x7ffb400b3370 0x7ffb401604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb44daf700 1 -- 172.21.15.83:0/594187597 >> v1:172.21.15.203:6789/0 conn(0x7ffb4015bdd0 legacy=0x7ffb400b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.421 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb44daf700 1 -- 172.21.15.83:0/594187597 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400b25b0 con 0x7ffb400d1c70 2023-11-26T07:16:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb3f7fe700 1 --2- 172.21.15.83:0/594187597 >> v2:172.21.15.203:3300/0 conn(0x7ffb400b3370 0x7ffb401604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb44daf700 1 --2- 172.21.15.83:0/594187597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb400d1c70 0x7ffb400d2100 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ffb4015f910 tx=0x7ffb3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b96f6b75a09db9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb3effd700 1 -- 172.21.15.83:0/594187597 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3800e070 con 0x7ffb400d1c70 2023-11-26T07:16:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb3effd700 1 -- 172.21.15.83:0/594187597 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3800b9c0 con 0x7ffb400d1c70 2023-11-26T07:16:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.416+0000 7ffb3effd700 1 -- 172.21.15.83:0/594187597 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3800bd70 con 0x7ffb400d1c70 2023-11-26T07:16:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.417+0000 7ffb46812700 1 -- 172.21.15.83:0/594187597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb400d1c70 msgr2=0x7ffb400d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.422 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.417+0000 7ffb46812700 1 --2- 172.21.15.83:0/594187597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb400d1c70 0x7ffb400d2100 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7ffb4015f910 tx=0x7ffb3800b330 comp rx=0 tx=0).stop 2023-11-26T07:16:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.417+0000 7ffb46812700 1 -- 172.21.15.83:0/594187597 shutdown_connections 2023-11-26T07:16:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.417+0000 7ffb46812700 1 --2- 172.21.15.83:0/594187597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb400d1c70 0x7ffb400d2100 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.417+0000 7ffb46812700 1 --2- 172.21.15.83:0/594187597 >> v2:172.21.15.203:3300/0 conn(0x7ffb400b3370 0x7ffb401604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.417+0000 7ffb46812700 1 -- 172.21.15.83:0/594187597 >> 172.21.15.83:0/594187597 conn(0x7ffb400731a0 msgr2=0x7ffb4009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.417+0000 7ffb46812700 1 -- 172.21.15.83:0/594187597 shutdown_connections 2023-11-26T07:16:39.423 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.417+0000 7ffb46812700 1 -- 172.21.15.83:0/594187597 wait complete. 2023-11-26T07:16:39.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.418+0000 7ffb46812700 1 Processor -- start 2023-11-26T07:16:39.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.418+0000 7ffb46812700 1 -- start start 2023-11-26T07:16:39.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb46812700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb400b3370 0x7ffb401572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb46812700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb400d1c70 0x7ffb400c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.426 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb46812700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb4015bdd0 0x7ffb4006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb46812700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400c9580 con 0x7ffb400d1c70 2023-11-26T07:16:39.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb46812700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4006ce20 con 0x7ffb4015bdd0 2023-11-26T07:16:39.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb46812700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400c8f30 con 0x7ffb400b3370 2023-11-26T07:16:39.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb400d1c70 0x7ffb400c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb3f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb400d1c70 0x7ffb400c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41040/0 (socket says 172.21.15.83:41040) 2023-11-26T07:16:39.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb3f7fe700 1 -- 172.21.15.83:0/256461816 learned_addr learned my addr 172.21.15.83:0/256461816 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:39.427 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb3ffff700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb400b3370 0x7ffb401572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb44daf700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb4015bdd0 0x7ffb4006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb3f7fe700 1 -- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb400b3370 msgr2=0x7ffb401572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb3f7fe700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb400b3370 0x7ffb401572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb3f7fe700 1 -- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb4015bdd0 msgr2=0x7ffb4006c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb3f7fe700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb4015bdd0 0x7ffb4006c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.419+0000 7ffb3f7fe700 1 -- 172.21.15.83:0/256461816 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb38009b30 con 0x7ffb400d1c70 2023-11-26T07:16:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.420+0000 7ffb3f7fe700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb400d1c70 0x7ffb400c9a10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ffb3400e810 tx=0x7ffb3400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:39.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.420+0000 7ffb3cff9700 1 -- 172.21.15.83:0/256461816 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3400c070 con 0x7ffb400d1c70 2023-11-26T07:16:39.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.420+0000 7ffb3cff9700 1 -- 172.21.15.83:0/256461816 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb34004230 con 0x7ffb400d1c70 2023-11-26T07:16:39.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.420+0000 7ffb46812700 1 -- 172.21.15.83:0/256461816 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb400c9210 con 0x7ffb400d1c70 2023-11-26T07:16:39.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.420+0000 7ffb3cff9700 1 -- 172.21.15.83:0/256461816 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb340036a0 con 0x7ffb400d1c70 2023-11-26T07:16:39.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.420+0000 7ffb46812700 1 -- 172.21.15.83:0/256461816 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb400c9f50 con 0x7ffb400d1c70 2023-11-26T07:16:39.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.421+0000 7ffb46812700 1 -- 172.21.15.83:0/256461816 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb400c96f0 con 0x7ffb400d1c70 2023-11-26T07:16:39.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.421+0000 7ffb3cff9700 1 -- 172.21.15.83:0/256461816 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7ffb34003800 con 0x7ffb400d1c70 2023-11-26T07:16:39.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.422+0000 7ffb3cff9700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ffb2c06aff0 0x7ffb2c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.422+0000 7ffb3cff9700 1 -- 172.21.15.83:0/256461816 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffb34089540 con 0x7ffb400d1c70 2023-11-26T07:16:39.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.422+0000 7ffb3ffff700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ffb2c06aff0 0x7ffb2c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.422+0000 7ffb3ffff700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ffb2c06aff0 0x7ffb2c06d4b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffb30007fd0 tx=0x7ffb30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:39.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.426+0000 7ffb3cff9700 1 -- 172.21.15.83:0/256461816 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb34054370 con 0x7ffb400d1c70 2023-11-26T07:16:39.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.631+0000 7ffb46812700 1 -- 172.21.15.83:0/256461816 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffb400c8d90 con 0x7ffb400d1c70 2023-11-26T07:16:39.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.631+0000 7ffb3cff9700 1 -- 172.21.15.83:0/256461816 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v90) v1 ==== 80+0+120 (secure 0 0 0) 0x7ffb400c8d90 con 0x7ffb400d1c70 2023-11-26T07:16:39.634 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6835/469139444 2023-11-27T07:16:32.809518+0000 2023-11-26T07:16:39.634 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6833/469139444 2023-11-27T07:16:32.809518+0000 2023-11-26T07:16:39.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 -- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ffb2c06aff0 msgr2=0x7ffb2c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ffb2c06aff0 0x7ffb2c06d4b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffb30007fd0 tx=0x7ffb30009040 comp rx=0 tx=0).stop 2023-11-26T07:16:39.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 -- 172.21.15.83:0/256461816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb400d1c70 msgr2=0x7ffb400c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb400d1c70 0x7ffb400c9a10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ffb3400e810 tx=0x7ffb3400eb20 comp rx=0 tx=0).stop 2023-11-26T07:16:39.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 -- 172.21.15.83:0/256461816 shutdown_connections 2023-11-26T07:16:39.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ffb2c06aff0 0x7ffb2c06d4b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb4015bdd0 0x7ffb4006c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb400b3370 0x7ffb401572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 --2- 172.21.15.83:0/256461816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb400d1c70 0x7ffb400c9a10 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 -- 172.21.15.83:0/256461816 >> 172.21.15.83:0/256461816 conn(0x7ffb400731a0 msgr2=0x7ffb400afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:39.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 -- 172.21.15.83:0/256461816 shutdown_connections 2023-11-26T07:16:39.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.633+0000 7ffb46812700 1 -- 172.21.15.83:0/256461816 wait complete. 2023-11-26T07:16:39.638 INFO:teuthology.orchestra.run.smithi083.stderr:listed 2 entries 2023-11-26T07:16:39.648 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:16:39.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:39.648 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:16:39.693 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:16:39.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:16:39.693 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:16:39.712 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:16:39.712 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:39.729 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:16:39.729 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:16:39.730 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:39.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:16:39.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:39.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.890+0000 7f54b4830700 1 Processor -- start 2023-11-26T07:16:39.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.890+0000 7f54b4830700 1 -- start start 2023-11-26T07:16:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.891+0000 7f54b4830700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b015c140 0x7f54b00cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.891+0000 7f54b4830700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b00cc780 con 0x7f54b015bd60 2023-11-26T07:16:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.891+0000 7f54b4830700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b00cc9b0 con 0x7f54b00cf2b0 2023-11-26T07:16:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.891+0000 7f54b4830700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00ccc90 con 0x7f54b015c140 2023-11-26T07:16:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ae59c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f54b015bd60 0x7f54b00d1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:32854/0 (socket says 172.21.15.83:32854) 2023-11-26T07:16:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ae59c700 1 -- 172.21.15.83:0/2380188103 learned_addr learned my addr 172.21.15.83:0/2380188103 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657021178 0 0) 0x7f54b00cc780 con 0x7f54b015bd60 2023-11-26T07:16:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54add9b700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b015c140 0x7f54b00cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f549c0035b0 con 0x7f54b015bd60 2023-11-26T07:16:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 106424110 0 0) 0x7f549c0035b0 con 0x7f54b015bd60 2023-11-26T07:16:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b015c140 msgr2=0x7f54b00cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b015c140 0x7f54b00cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 >> v1:172.21.15.203:6789/0 conn(0x7f54b00cf2b0 legacy=0x7f54b00c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b00ccf70 con 0x7f54b015bd60 2023-11-26T07:16:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f5498001670 con 0x7f54b015bd60 2023-11-26T07:16:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 >> v1:172.21.15.83:6789/0 conn(0x7f54b015bd60 legacy=0x7f54b00d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b015c140 0x7f549c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f549c003fb0 0x7f549c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 0x7f549c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00ccc90 con 0x7f549c004d90 2023-11-26T07:16:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c00baf0 con 0x7f549c003fb0 2023-11-26T07:16:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c00bcb0 con 0x7f54b015c140 2023-11-26T07:16:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54ae59c700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b015c140 0x7f549c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54aed9d700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 0x7f549c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.892+0000 7f54add9b700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f549c003fb0 0x7f549c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54aed9d700 1 -- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b015c140 msgr2=0x7f549c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54aed9d700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b015c140 0x7f549c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54aed9d700 1 -- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f549c003fb0 msgr2=0x7f549c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54aed9d700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f549c003fb0 0x7f549c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54aed9d700 1 -- 172.21.15.83:0/2380188103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c0035b0 con 0x7f549c004d90 2023-11-26T07:16:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54ae59c700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b015c140 0x7f549c003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54aed9d700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 0x7f549c005210 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f54a800c160 tx=0x7f549c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c985eed749f26e59 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8001db0 con 0x7f549c004d90 2023-11-26T07:16:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a800cca0 con 0x7f549c004d90 2023-11-26T07:16:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54ad59a700 1 -- 172.21.15.83:0/2380188103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8003c40 con 0x7f549c004d90 2023-11-26T07:16:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54b4830700 1 -- 172.21.15.83:0/2380188103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 msgr2=0x7f549c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54b4830700 1 --2- 172.21.15.83:0/2380188103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 0x7f549c005210 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f54a800c160 tx=0x7f549c005880 comp rx=0 tx=0).stop 2023-11-26T07:16:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54aed9d700 1 -- 172.21.15.83:0/2380188103 reap_dead start 2023-11-26T07:16:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54b4830700 1 -- 172.21.15.83:0/2380188103 shutdown_connections 2023-11-26T07:16:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.893+0000 7f54b4830700 1 -- 172.21.15.83:0/2380188103 >> 172.21.15.83:0/2380188103 conn(0x7f54b0073130 msgr2=0x7f54b009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.894+0000 7f54b4830700 1 -- 172.21.15.83:0/2380188103 shutdown_connections 2023-11-26T07:16:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.894+0000 7f54b4830700 1 -- 172.21.15.83:0/2380188103 wait complete. 2023-11-26T07:16:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.895+0000 7f54b4830700 1 Processor -- start 2023-11-26T07:16:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.895+0000 7f54b4830700 1 -- start start 2023-11-26T07:16:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54b4830700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 0x7f54b0159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54b4830700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54b014f300 0x7f54b015aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54b4830700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b00b4300 0x7f54b00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54b4830700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00b3ac0 con 0x7f549c004d90 2023-11-26T07:16:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54b4830700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00cbbe0 con 0x7f54b014f300 2023-11-26T07:16:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54b4830700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b00cbec0 con 0x7f54b00b4300 2023-11-26T07:16:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54ae59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 0x7f54b0159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54ae59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 0x7f54b0159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41058/0 (socket says 172.21.15.83:41058) 2023-11-26T07:16:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54ae59c700 1 -- 172.21.15.83:0/4023579970 learned_addr learned my addr 172.21.15.83:0/4023579970 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54add9b700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54b014f300 0x7f54b015aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54aed9d700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b00b4300 0x7f54b00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54ae59c700 1 -- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b00b4300 msgr2=0x7f54b00caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54ae59c700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b00b4300 0x7f54b00caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54ae59c700 1 -- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54b014f300 msgr2=0x7f54b015aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54ae59c700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54b014f300 0x7f54b015aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54ae59c700 1 -- 172.21.15.83:0/4023579970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b00b7210 con 0x7f549c004d90 2023-11-26T07:16:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54add9b700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54b014f300 0x7f54b015aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54ae59c700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 0x7f54b0159420 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f54980073f0 tx=0x7f54980077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54a6ffd700 1 -- 172.21.15.83:0/4023579970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549800f990 con 0x7f549c004d90 2023-11-26T07:16:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54a6ffd700 1 -- 172.21.15.83:0/4023579970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549800faf0 con 0x7f549c004d90 2023-11-26T07:16:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54b4830700 1 -- 172.21.15.83:0/4023579970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b00b74f0 con 0x7f549c004d90 2023-11-26T07:16:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54a6ffd700 1 -- 172.21.15.83:0/4023579970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549800fea0 con 0x7f549c004d90 2023-11-26T07:16:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.896+0000 7f54b4830700 1 -- 172.21.15.83:0/4023579970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b00b4960 con 0x7f549c004d90 2023-11-26T07:16:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.897+0000 7f54b4830700 1 -- 172.21.15.83:0/4023579970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b0067f60 con 0x7f549c004d90 2023-11-26T07:16:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.898+0000 7f54a6ffd700 1 -- 172.21.15.83:0/4023579970 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f549801d460 con 0x7f549c004d90 2023-11-26T07:16:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.898+0000 7f54a6ffd700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f549006aff0 0x7f549006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.898+0000 7f54a6ffd700 1 -- 172.21.15.83:0/4023579970 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5498089a70 con 0x7f549c004d90 2023-11-26T07:16:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.898+0000 7f54add9b700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f549006aff0 0x7f549006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.899+0000 7f54add9b700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f549006aff0 0x7f549006d4b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f54b00cec10 tx=0x7f54a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:39.902+0000 7f54a6ffd700 1 -- 172.21.15.83:0/4023579970 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f54980549b0 con 0x7f549c004d90 2023-11-26T07:16:40.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.136+0000 7f54b4830700 1 -- 172.21.15.83:0/4023579970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54b00b4100 con 0x7f549c004d90 2023-11-26T07:16:40.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.136+0000 7f54a6ffd700 1 -- 172.21.15.83:0/4023579970 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f5498057fd0 con 0x7f549c004d90 2023-11-26T07:16:40.139 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:40.139 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1015690400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1015690400},{"type":"v1","addr":"172.21.15.203:6834","nonce":1015690400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2716428832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2716428832},{"type":"v1","addr":"172.21.15.83:6834","nonce":2716428832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/558945070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":558945070},{"type":"v1","addr":"172.21.15.83:6835","nonce":558945070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:16:17.967531+0000","modified":"2023-11-26T07:16:21.013928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1901175330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1901175330},{"type":"v1","addr":"172.21.15.203:6835","nonce":1901175330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:16:40.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 -- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f549006aff0 msgr2=0x7f549006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f549006aff0 0x7f549006d4b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f54b00cec10 tx=0x7f54a0008040 comp rx=0 tx=0).stop 2023-11-26T07:16:40.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 -- 172.21.15.83:0/4023579970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 msgr2=0x7f54b0159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 0x7f54b0159420 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f54980073f0 tx=0x7f54980077b0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 -- 172.21.15.83:0/4023579970 shutdown_connections 2023-11-26T07:16:40.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f549006aff0 0x7f549006d4b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f54b00b4300 0x7f54b00caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f549c004d90 0x7f54b0159420 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 --2- 172.21.15.83:0/4023579970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54b014f300 0x7f54b015aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 -- 172.21.15.83:0/4023579970 >> 172.21.15.83:0/4023579970 conn(0x7f54b0073130 msgr2=0x7f54b00d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 -- 172.21.15.83:0/4023579970 shutdown_connections 2023-11-26T07:16:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.138+0000 7f54b4830700 1 -- 172.21.15.83:0/4023579970 wait complete. 2023-11-26T07:16:40.142 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 60 2023-11-26T07:16:40.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.271+0000 7f2f29ac6700 1 Processor -- start 2023-11-26T07:16:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.271+0000 7f2f29ac6700 1 -- start start 2023-11-26T07:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.272+0000 7f2f29ac6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f2413e000 0x7f2f2415ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.272+0000 7f2f29ac6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2f2415efb0 0x7f2f240cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.272+0000 7f2f29ac6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f240b4290 con 0x7f2f240cfd40 2023-11-26T07:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.272+0000 7f2f29ac6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2415f430 con 0x7f2f2415efb0 2023-11-26T07:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.272+0000 7f2f29ac6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2413e570 con 0x7f2f2413e000 2023-11-26T07:16:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.272+0000 7f2f22ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2f2415efb0 0x7f2f240cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.272+0000 7f2f22ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2f2415efb0 0x7f2f240cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41066/0 (socket says 172.21.15.83:41066) 2023-11-26T07:16:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.272+0000 7f2f22ffd700 1 -- 172.21.15.83:0/4238706870 learned_addr learned my addr 172.21.15.83:0/4238706870 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f237fe700 1 --2- 172.21.15.83:0/4238706870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f2413e000 0x7f2f2415ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f22ffd700 1 -- 172.21.15.83:0/4238706870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f2413e000 msgr2=0x7f2f2415ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f22ffd700 1 --2- 172.21.15.83:0/4238706870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f2413e000 0x7f2f2415ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f22ffd700 1 -- 172.21.15.83:0/4238706870 >> v1:172.21.15.203:6789/0 conn(0x7f2f240cfd40 legacy=0x7f2f2415b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f22ffd700 1 -- 172.21.15.83:0/4238706870 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f240d05d0 con 0x7f2f2415efb0 2023-11-26T07:16:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f22ffd700 1 --2- 172.21.15.83:0/4238706870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f2415efb0 0x7f2f240cf800 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00bb90 tx=0x7f2f0c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b864b324d4d0265 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f227fc700 1 -- 172.21.15.83:0/4238706870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c00dd50 con 0x7f2f2415efb0 2023-11-26T07:16:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f227fc700 1 -- 172.21.15.83:0/4238706870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c010470 con 0x7f2f2415efb0 2023-11-26T07:16:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f227fc700 1 -- 172.21.15.83:0/4238706870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c010800 con 0x7f2f2415efb0 2023-11-26T07:16:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f29ac6700 1 -- 172.21.15.83:0/4238706870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f2415efb0 msgr2=0x7f2f240cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f29ac6700 1 --2- 172.21.15.83:0/4238706870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f2415efb0 0x7f2f240cf800 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00bb90 tx=0x7f2f0c00bea0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.273+0000 7f2f29ac6700 1 -- 172.21.15.83:0/4238706870 shutdown_connections 2023-11-26T07:16:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.274+0000 7f2f29ac6700 1 --2- 172.21.15.83:0/4238706870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f2413e000 0x7f2f2415ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.274+0000 7f2f29ac6700 1 --2- 172.21.15.83:0/4238706870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f2415efb0 0x7f2f240cf800 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.274+0000 7f2f29ac6700 1 -- 172.21.15.83:0/4238706870 >> 172.21.15.83:0/4238706870 conn(0x7f2f24072990 msgr2=0x7f2f2406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.274+0000 7f2f29ac6700 1 -- 172.21.15.83:0/4238706870 shutdown_connections 2023-11-26T07:16:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.274+0000 7f2f29ac6700 1 -- 172.21.15.83:0/4238706870 wait complete. 2023-11-26T07:16:40.280 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.275+0000 7f2f29ac6700 1 Processor -- start 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.275+0000 7f2f29ac6700 1 -- start start 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.276+0000 7f2f29ac6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240cfd40 0x7f2f2406e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.276+0000 7f2f29ac6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f2413e000 0x7f2f2413fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.276+0000 7f2f29ac6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f2415efb0 0x7f2f2413cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.276+0000 7f2f29ac6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24152610 con 0x7f2f240cfd40 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.276+0000 7f2f29ac6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2406eac0 con 0x7f2f2415efb0 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.276+0000 7f2f29ac6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f241f3060 con 0x7f2f2413e000 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.276+0000 7f2f237fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240cfd40 0x7f2f2406e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.276+0000 7f2f237fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240cfd40 0x7f2f2406e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41070/0 (socket says 172.21.15.83:41070) 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.276+0000 7f2f237fe700 1 -- 172.21.15.83:0/1900506446 learned_addr learned my addr 172.21.15.83:0/1900506446 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.276+0000 7f2f22ffd700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f2413e000 0x7f2f2413fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f237fe700 1 -- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f2413e000 msgr2=0x7f2f2413fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.276+0000 7f2f23fff700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f2415efb0 0x7f2f2413cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f237fe700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f2413e000 0x7f2f2413fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f237fe700 1 -- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f2415efb0 msgr2=0x7f2f2413cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f237fe700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f2415efb0 0x7f2f2413cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f237fe700 1 -- 172.21.15.83:0/1900506446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c00b7f0 con 0x7f2f240cfd40 2023-11-26T07:16:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f23fff700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f2415efb0 0x7f2f2413cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f22ffd700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f2413e000 0x7f2f2413fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f237fe700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240cfd40 0x7f2f2406e580 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400e810 tx=0x7f2f1400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f1bfff700 1 -- 172.21.15.83:0/1900506446 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14013610 con 0x7f2f240cfd40 2023-11-26T07:16:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f1bfff700 1 -- 172.21.15.83:0/1900506446 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f14013770 con 0x7f2f240cfd40 2023-11-26T07:16:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f29ac6700 1 -- 172.21.15.83:0/1900506446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f241f3340 con 0x7f2f240cfd40 2023-11-26T07:16:40.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f1bfff700 1 -- 172.21.15.83:0/1900506446 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14013b20 con 0x7f2f240cfd40 2023-11-26T07:16:40.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.277+0000 7f2f29ac6700 1 -- 172.21.15.83:0/1900506446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f241f3900 con 0x7f2f240cfd40 2023-11-26T07:16:40.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.278+0000 7f2f29ac6700 1 -- 172.21.15.83:0/1900506446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f240b3f00 con 0x7f2f240cfd40 2023-11-26T07:16:40.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.278+0000 7f2f1bfff700 1 -- 172.21.15.83:0/1900506446 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f2f14013c80 con 0x7f2f240cfd40 2023-11-26T07:16:40.283 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.279+0000 7f2f1bfff700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2f0406aff0 0x7f2f0406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.279+0000 7f2f1bfff700 1 -- 172.21.15.83:0/1900506446 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2f14089610 con 0x7f2f240cfd40 2023-11-26T07:16:40.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.279+0000 7f2f22ffd700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2f0406aff0 0x7f2f0406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.284 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.279+0000 7f2f22ffd700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2f0406aff0 0x7f2f0406d4b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f24140c60 tx=0x7f2f0c013000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:40.286 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.283+0000 7f2f1bfff700 1 -- 172.21.15.83:0/1900506446 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f240b3f00 con 0x7f2f240cfd40 2023-11-26T07:16:40.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.475+0000 7f2f29ac6700 1 -- 172.21.15.83:0/1900506446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f240b3f00 con 0x7f2f240cfd40 2023-11-26T07:16:40.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.476+0000 7f2f1bfff700 1 -- 172.21.15.83:0/1900506446 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16675 (secure 0 0 0) 0x7f2f240b3f00 con 0x7f2f240cfd40 2023-11-26T07:16:40.479 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:40.479 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:35.818271+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:16:15.747889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:16:16.943249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6835/469139444":"2023-11-27T07:16:32.809518+0000","172.21.15.83:6833/469139444":"2023-11-27T07:16:32.809518+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.478+0000 7f2f29ac6700 1 -- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2f0406aff0 msgr2=0x7f2f0406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.478+0000 7f2f29ac6700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2f0406aff0 0x7f2f0406d4b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f24140c60 tx=0x7f2f0c013000 comp rx=0 tx=0).stop 2023-11-26T07:16:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.478+0000 7f2f29ac6700 1 -- 172.21.15.83:0/1900506446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240cfd40 msgr2=0x7f2f2406e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.478+0000 7f2f29ac6700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240cfd40 0x7f2f2406e580 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400e810 tx=0x7f2f1400ebd0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.478+0000 7f2f29ac6700 1 -- 172.21.15.83:0/1900506446 shutdown_connections 2023-11-26T07:16:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.478+0000 7f2f29ac6700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2f0406aff0 0x7f2f0406d4b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.478+0000 7f2f29ac6700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f2415efb0 0x7f2f2413cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.478+0000 7f2f29ac6700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f240cfd40 0x7f2f2406e580 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.478+0000 7f2f29ac6700 1 --2- 172.21.15.83:0/1900506446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f2413e000 0x7f2f2413fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.478+0000 7f2f29ac6700 1 -- 172.21.15.83:0/1900506446 >> 172.21.15.83:0/1900506446 conn(0x7f2f24072990 msgr2=0x7f2f24073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:40.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.479+0000 7f2f29ac6700 1 -- 172.21.15.83:0/1900506446 shutdown_connections 2023-11-26T07:16:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.479+0000 7f2f29ac6700 1 -- 172.21.15.83:0/1900506446 wait complete. 2023-11-26T07:16:40.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:16:15.747889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:16:16.943249+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:16:40.494 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:16:40.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.615+0000 7f764549e700 1 Processor -- start 2023-11-26T07:16:40.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.615+0000 7f764549e700 1 -- start start 2023-11-26T07:16:40.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.615+0000 7f764549e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f764015deb0 0x7f76400af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.619 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.615+0000 7f764549e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76400afb50 0x7f76400b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.615+0000 7f764549e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76400713b0 0x7f7640071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f764549e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76400af100 con 0x7f76400713b0 2023-11-26T07:16:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f764549e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7640071da0 con 0x7f764015deb0 2023-11-26T07:16:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f764549e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76400ae250 con 0x7f76400afb50 2023-11-26T07:16:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76400713b0 0x7f7640071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f76400713b0 0x7f7640071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41082/0 (socket says 172.21.15.83:41082) 2023-11-26T07:16:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763f7fe700 1 -- 172.21.15.83:0/954655007 learned_addr learned my addr 172.21.15.83:0/954655007 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763e7fc700 1 --2- 172.21.15.83:0/954655007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76400afb50 0x7f76400b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763effd700 1 --2- 172.21.15.83:0/954655007 >> v2:172.21.15.203:3300/0 conn(0x7f764015deb0 0x7f76400af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763f7fe700 1 -- 172.21.15.83:0/954655007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76400afb50 msgr2=0x7f76400b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:40.620 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763f7fe700 1 --2- 172.21.15.83:0/954655007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76400afb50 0x7f76400b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763f7fe700 1 -- 172.21.15.83:0/954655007 >> v2:172.21.15.203:3300/0 conn(0x7f764015deb0 msgr2=0x7f76400af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763f7fe700 1 --2- 172.21.15.83:0/954655007 >> v2:172.21.15.203:3300/0 conn(0x7f764015deb0 0x7f76400af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763f7fe700 1 -- 172.21.15.83:0/954655007 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400ae480 con 0x7f76400713b0 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763f7fe700 1 --2- 172.21.15.83:0/954655007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76400713b0 0x7f7640071860 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f76400b5ab0 tx=0x7f763800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62d2301a1f89a94 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763dffb700 1 -- 172.21.15.83:0/954655007 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763800e070 con 0x7f76400713b0 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763dffb700 1 -- 172.21.15.83:0/954655007 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763800b980 con 0x7f76400713b0 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.616+0000 7f763dffb700 1 -- 172.21.15.83:0/954655007 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763800bd30 con 0x7f76400713b0 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.617+0000 7f764549e700 1 -- 172.21.15.83:0/954655007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76400713b0 msgr2=0x7f7640071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.617+0000 7f764549e700 1 --2- 172.21.15.83:0/954655007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76400713b0 0x7f7640071860 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f76400b5ab0 tx=0x7f763800b330 comp rx=0 tx=0).stop 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.617+0000 7f764549e700 1 -- 172.21.15.83:0/954655007 shutdown_connections 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.617+0000 7f764549e700 1 --2- 172.21.15.83:0/954655007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76400713b0 0x7f7640071860 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.621 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.617+0000 7f764549e700 1 --2- 172.21.15.83:0/954655007 >> v2:172.21.15.203:3300/0 conn(0x7f764015deb0 0x7f76400af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.617+0000 7f764549e700 1 --2- 172.21.15.83:0/954655007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76400afb50 0x7f76400b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.617+0000 7f764549e700 1 -- 172.21.15.83:0/954655007 >> 172.21.15.83:0/954655007 conn(0x7f7640074170 msgr2=0x7f764006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:40.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.617+0000 7f764549e700 1 -- 172.21.15.83:0/954655007 shutdown_connections 2023-11-26T07:16:40.622 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.617+0000 7f764549e700 1 -- 172.21.15.83:0/954655007 wait complete. 2023-11-26T07:16:40.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.618+0000 7f764549e700 1 Processor -- start 2023-11-26T07:16:40.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.618+0000 7f764549e700 1 -- start start 2023-11-26T07:16:40.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f764549e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76400713b0 0x7f764014fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f764549e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76400afb50 0x7f7640155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.624 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f764549e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764015deb0 0x7f7640155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f764549e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76401f4ee0 con 0x7f76400afb50 2023-11-26T07:16:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f764549e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7640150000 con 0x7f76400713b0 2023-11-26T07:16:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f764549e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76401f4010 con 0x7f764015deb0 2023-11-26T07:16:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f763e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76400afb50 0x7f7640155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f763e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76400afb50 0x7f7640155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41086/0 (socket says 172.21.15.83:41086) 2023-11-26T07:16:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f763e7fc700 1 -- 172.21.15.83:0/2796761904 learned_addr learned my addr 172.21.15.83:0/2796761904 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f763effd700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76400713b0 0x7f764014fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f763f7fe700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764015deb0 0x7f7640155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f763e7fc700 1 -- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764015deb0 msgr2=0x7f7640155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f763e7fc700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764015deb0 0x7f7640155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f763e7fc700 1 -- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76400713b0 msgr2=0x7f764014fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f763e7fc700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76400713b0 0x7f764014fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f763e7fc700 1 -- 172.21.15.83:0/2796761904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7638009b30 con 0x7f76400afb50 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.619+0000 7f763f7fe700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764015deb0 0x7f7640155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.620+0000 7f763e7fc700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76400afb50 0x7f7640155ab0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f763400e7e0 tx=0x7f763400eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.620+0000 7f76277fe700 1 -- 172.21.15.83:0/2796761904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763400c070 con 0x7f76400afb50 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.620+0000 7f76277fe700 1 -- 172.21.15.83:0/2796761904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7634013610 con 0x7f76400afb50 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.620+0000 7f764549e700 1 -- 172.21.15.83:0/2796761904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76401f42d0 con 0x7f76400afb50 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.620+0000 7f76277fe700 1 -- 172.21.15.83:0/2796761904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76340139a0 con 0x7f76400afb50 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.620+0000 7f764549e700 1 -- 172.21.15.83:0/2796761904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76401f4890 con 0x7f76400afb50 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.621+0000 7f764549e700 1 -- 172.21.15.83:0/2796761904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76401f5090 con 0x7f76400afb50 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.621+0000 7f76277fe700 1 -- 172.21.15.83:0/2796761904 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f7634013b00 con 0x7f76400afb50 2023-11-26T07:16:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.622+0000 7f76277fe700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f762c06aff0 0x7f762c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.622+0000 7f76277fe700 1 -- 172.21.15.83:0/2796761904 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7634089570 con 0x7f76400afb50 2023-11-26T07:16:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.622+0000 7f763effd700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f762c06aff0 0x7f762c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.622+0000 7f763effd700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f762c06aff0 0x7f762c06d4b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76401557e0 tx=0x7f7630008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:40.628 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.626+0000 7f76277fe700 1 -- 172.21.15.83:0/2796761904 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76401f5090 con 0x7f76400afb50 2023-11-26T07:16:40.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.815+0000 7f764549e700 1 -- 172.21.15.83:0/2796761904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76401f5090 con 0x7f76400afb50 2023-11-26T07:16:40.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.816+0000 7f76277fe700 1 -- 172.21.15.83:0/2796761904 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76401f5090 con 0x7f76400afb50 2023-11-26T07:16:40.819 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:16:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 -- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f762c06aff0 msgr2=0x7f762c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f762c06aff0 0x7f762c06d4b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76401557e0 tx=0x7f7630008040 comp rx=0 tx=0).stop 2023-11-26T07:16:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 -- 172.21.15.83:0/2796761904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76400afb50 msgr2=0x7f7640155ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.821 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76400afb50 0x7f7640155ab0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f763400e7e0 tx=0x7f763400eaf0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 -- 172.21.15.83:0/2796761904 shutdown_connections 2023-11-26T07:16:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f762c06aff0 0x7f762c06d4b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f764015deb0 0x7f7640155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76400713b0 0x7f764014fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 --2- 172.21.15.83:0/2796761904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76400afb50 0x7f7640155ab0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 -- 172.21.15.83:0/2796761904 >> 172.21.15.83:0/2796761904 conn(0x7f7640074170 msgr2=0x7f764015b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:40.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 -- 172.21.15.83:0/2796761904 shutdown_connections 2023-11-26T07:16:40.823 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.818+0000 7f764549e700 1 -- 172.21.15.83:0/2796761904 wait complete. 2023-11-26T07:16:40.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:16:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.952+0000 7f1878f03700 1 Processor -- start 2023-11-26T07:16:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.952+0000 7f1878f03700 1 -- start start 2023-11-26T07:16:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.953+0000 7f1878f03700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f187415e250 0x7f1874073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.957 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.953+0000 7f1878f03700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18740afea0 0x7f18740744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.953+0000 7f1878f03700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18740b48f0 con 0x7f18740b0560 2023-11-26T07:16:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.953+0000 7f1878f03700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187415e630 con 0x7f18740afea0 2023-11-26T07:16:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.953+0000 7f1878f03700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18740b4bd0 con 0x7f187415e250 2023-11-26T07:16:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f1872d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18740afea0 0x7f18740744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f1872d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18740afea0 0x7f18740744d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41096/0 (socket says 172.21.15.83:41096) 2023-11-26T07:16:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f1872d9d700 1 -- 172.21.15.83:0/3559516546 learned_addr learned my addr 172.21.15.83:0/3559516546 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f187259c700 1 --2- 172.21.15.83:0/3559516546 >> v2:172.21.15.203:3300/0 conn(0x7f187415e250 0x7f1874073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f1872d9d700 1 -- 172.21.15.83:0/3559516546 >> v2:172.21.15.203:3300/0 conn(0x7f187415e250 msgr2=0x7f1874073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f1872d9d700 1 --2- 172.21.15.83:0/3559516546 >> v2:172.21.15.203:3300/0 conn(0x7f187415e250 0x7f1874073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f1872d9d700 1 -- 172.21.15.83:0/3559516546 >> v1:172.21.15.203:6789/0 conn(0x7f18740b0560 legacy=0x7f1874161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:40.958 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f1872d9d700 1 -- 172.21.15.83:0/3559516546 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18740b50d0 con 0x7f18740afea0 2023-11-26T07:16:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f1872d9d700 1 --2- 172.21.15.83:0/3559516546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18740afea0 0x7f18740744d0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f1874073420 tx=0x7f186c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845a3ae36e3981eb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f187159a700 1 -- 172.21.15.83:0/3559516546 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c00e070 con 0x7f18740afea0 2023-11-26T07:16:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f187159a700 1 -- 172.21.15.83:0/3559516546 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186c003a20 con 0x7f18740afea0 2023-11-26T07:16:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.954+0000 7f187159a700 1 -- 172.21.15.83:0/3559516546 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c003dd0 con 0x7f18740afea0 2023-11-26T07:16:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.955+0000 7f1878f03700 1 -- 172.21.15.83:0/3559516546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18740afea0 msgr2=0x7f18740744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.955+0000 7f1878f03700 1 --2- 172.21.15.83:0/3559516546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18740afea0 0x7f18740744d0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f1874073420 tx=0x7f186c00b330 comp rx=0 tx=0).stop 2023-11-26T07:16:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.955+0000 7f1878f03700 1 -- 172.21.15.83:0/3559516546 shutdown_connections 2023-11-26T07:16:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.955+0000 7f1878f03700 1 --2- 172.21.15.83:0/3559516546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18740afea0 0x7f18740744d0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.955+0000 7f1878f03700 1 --2- 172.21.15.83:0/3559516546 >> v2:172.21.15.203:3300/0 conn(0x7f187415e250 0x7f1874073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.955+0000 7f1878f03700 1 -- 172.21.15.83:0/3559516546 >> 172.21.15.83:0/3559516546 conn(0x7f1874072960 msgr2=0x7f187406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:40.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.955+0000 7f1878f03700 1 -- 172.21.15.83:0/3559516546 shutdown_connections 2023-11-26T07:16:40.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.955+0000 7f1878f03700 1 -- 172.21.15.83:0/3559516546 wait complete. 2023-11-26T07:16:40.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.956+0000 7f1878f03700 1 Processor -- start 2023-11-26T07:16:40.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.956+0000 7f1878f03700 1 -- start start 2023-11-26T07:16:40.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1878f03700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18740afea0 0x7f18741523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1878f03700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18740b0560 0x7f18741f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1878f03700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187415e250 0x7f18741480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1878f03700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18741f8580 con 0x7f187415e250 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1878f03700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1874158460 con 0x7f18740b0560 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1878f03700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1874158740 con 0x7f18740afea0 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1872d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187415e250 0x7f18741480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1872d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187415e250 0x7f18741480e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41102/0 (socket says 172.21.15.83:41102) 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1872d9d700 1 -- 172.21.15.83:0/1141640849 learned_addr learned my addr 172.21.15.83:0/1141640849 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1871d9b700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18740b0560 0x7f18741f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f187259c700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18740afea0 0x7f18741523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1872d9d700 1 -- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18740afea0 msgr2=0x7f18741523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1872d9d700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18740afea0 0x7f18741523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1872d9d700 1 -- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18740b0560 msgr2=0x7f18741f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1872d9d700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18740b0560 0x7f18741f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1872d9d700 1 -- 172.21.15.83:0/1141640849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c009b30 con 0x7f187415e250 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1871d9b700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18740b0560 0x7f18741f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.957+0000 7f1872d9d700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187415e250 0x7f18741480e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f186c015040 tx=0x7f186c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.958+0000 7f1862ffd700 1 -- 172.21.15.83:0/1141640849 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c004440 con 0x7f187415e250 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.958+0000 7f1862ffd700 1 -- 172.21.15.83:0/1141640849 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186c0045c0 con 0x7f187415e250 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.958+0000 7f1878f03700 1 -- 172.21.15.83:0/1141640849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1874158a20 con 0x7f187415e250 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.958+0000 7f1862ffd700 1 -- 172.21.15.83:0/1141640849 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c004970 con 0x7f187415e250 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.958+0000 7f1878f03700 1 -- 172.21.15.83:0/1141640849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1874148c20 con 0x7f187415e250 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.959+0000 7f1878f03700 1 -- 172.21.15.83:0/1141640849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1874068fb0 con 0x7f187415e250 2023-11-26T07:16:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.959+0000 7f1862ffd700 1 -- 172.21.15.83:0/1141640849 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f186c012430 con 0x7f187415e250 2023-11-26T07:16:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.959+0000 7f1862ffd700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185406aff0 0x7f185406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.959+0000 7f1862ffd700 1 -- 172.21.15.83:0/1141640849 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f186c08ad40 con 0x7f187415e250 2023-11-26T07:16:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.960+0000 7f187259c700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185406aff0 0x7f185406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.960+0000 7f187259c700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185406aff0 0x7f185406d4b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f186800b9b0 tx=0x7f1868008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:40.964+0000 7f1862ffd700 1 -- 172.21.15.83:0/1141640849 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f186c055c00 con 0x7f187415e250 2023-11-26T07:16:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.152+0000 7f1878f03700 1 -- 172.21.15.83:0/1141640849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1874158ee0 con 0x7f187415e250 2023-11-26T07:16:41.155 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.153+0000 7f1862ffd700 1 -- 172.21.15.83:0/1141640849 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f186c017020 con 0x7f187415e250 2023-11-26T07:16:41.155 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:16:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 -- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185406aff0 msgr2=0x7f185406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185406aff0 0x7f185406d4b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f186800b9b0 tx=0x7f1868008040 comp rx=0 tx=0).stop 2023-11-26T07:16:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 -- 172.21.15.83:0/1141640849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187415e250 msgr2=0x7f18741480e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187415e250 0x7f18741480e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f186c015040 tx=0x7f186c004ad0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.158 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 -- 172.21.15.83:0/1141640849 shutdown_connections 2023-11-26T07:16:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185406aff0 0x7f185406d4b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f187415e250 0x7f18741480e0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18740afea0 0x7f18741523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 --2- 172.21.15.83:0/1141640849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18740b0560 0x7f18741f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 -- 172.21.15.83:0/1141640849 >> 172.21.15.83:0/1141640849 conn(0x7f1874072960 msgr2=0x7f187415a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 -- 172.21.15.83:0/1141640849 shutdown_connections 2023-11-26T07:16:41.159 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.155+0000 7f1878f03700 1 -- 172.21.15.83:0/1141640849 wait complete. 2023-11-26T07:16:41.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:16:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.290+0000 7f7f11a41700 1 Processor -- start 2023-11-26T07:16:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.291+0000 7f7f11a41700 1 -- start start 2023-11-26T07:16:41.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.291+0000 7f7f11a41700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f0c0b4310 0x7f7f0c06bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.291+0000 7f7f11a41700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0c06ba50 con 0x7f7f0c0ce430 2023-11-26T07:16:41.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.291+0000 7f7f11a41700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0c06b120 con 0x7f7f0c15e080 2023-11-26T07:16:41.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.291+0000 7f7f11a41700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0d1200 con 0x7f7f0c0b4310 2023-11-26T07:16:41.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.291+0000 7f7f0b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7f0c0ce430 0x7f7f0c0cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:32918/0 (socket says 172.21.15.83:32918) 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.291+0000 7f7f0b7fe700 1 -- 172.21.15.83:0/539163751 learned_addr learned my addr 172.21.15.83:0/539163751 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f0a7fc700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f0c0b4310 0x7f7f0c06bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761496948 0 0) 0x7f7f0c06ba50 con 0x7f7f0c0ce430 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ef40035b0 con 0x7f7f0c0ce430 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871405894 0 0) 0x7f7f0c06b120 con 0x7f7f0c15e080 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f0c06ba50 con 0x7f7f0c15e080 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3978727389 0 0) 0x7f7ef40035b0 con 0x7f7f0c0ce430 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f0c0b4310 msgr2=0x7f7f0c06bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f0c0b4310 0x7f7f0c06bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 >> v1:172.21.15.203:6789/0 conn(0x7f7f0c15e080 legacy=0x7f7f0c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c06b400 con 0x7f7f0c0ce430 2023-11-26T07:16:41.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f7f040019f0 con 0x7f7f0c0ce430 2023-11-26T07:16:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 >> v1:172.21.15.83:6789/0 conn(0x7f7f0c0ce430 legacy=0x7f7f0c0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ef4003bc0 0x7f7ef4003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ef4004850 0x7f7ef4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef4005210 0x7f7ef4014220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0d1200 con 0x7f7ef4005210 2023-11-26T07:16:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4014890 con 0x7f7ef4003bc0 2023-11-26T07:16:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef4014a50 con 0x7f7ef4004850 2023-11-26T07:16:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f0a7fc700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f0c0b4310 0x7f7f0c06bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f0b7fe700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef4005210 0x7f7ef4014220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f0affd700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ef4003bc0 0x7f7ef4003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.292+0000 7f7f0a7fc700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ef4004850 0x7f7ef4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f0b7fe700 1 -- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ef4004850 msgr2=0x7f7ef4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f0b7fe700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ef4004850 0x7f7ef4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f0b7fe700 1 -- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ef4003bc0 msgr2=0x7f7ef4003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f0b7fe700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ef4003bc0 0x7f7ef4003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f0b7fe700 1 -- 172.21.15.83:0/539163751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef40035b0 con 0x7f7ef4005210 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f0a7fc700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ef4004850 0x7f7ef4004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f0b7fe700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef4005210 0x7f7ef4014220 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400a7a0 tx=0x7f7ef4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53fbee5031f244d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04012600 con 0x7f7ef4005210 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f04012780 con 0x7f7ef4005210 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f09ffb700 1 -- 172.21.15.83:0/539163751 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04012b30 con 0x7f7ef4005210 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f11a41700 1 -- 172.21.15.83:0/539163751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef4005210 msgr2=0x7f7ef4014220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f11a41700 1 --2- 172.21.15.83:0/539163751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef4005210 0x7f7ef4014220 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400a7a0 tx=0x7f7ef4005880 comp rx=0 tx=0).stop 2023-11-26T07:16:41.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f0b7fe700 1 -- 172.21.15.83:0/539163751 reap_dead start 2023-11-26T07:16:41.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f11a41700 1 -- 172.21.15.83:0/539163751 shutdown_connections 2023-11-26T07:16:41.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f11a41700 1 -- 172.21.15.83:0/539163751 >> 172.21.15.83:0/539163751 conn(0x7f7f0c0b3780 msgr2=0x7f7f0c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:41.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f11a41700 1 -- 172.21.15.83:0/539163751 shutdown_connections 2023-11-26T07:16:41.300 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.293+0000 7f7f11a41700 1 -- 172.21.15.83:0/539163751 wait complete. 2023-11-26T07:16:41.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.295+0000 7f7f11a41700 1 Processor -- start 2023-11-26T07:16:41.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.295+0000 7f7f11a41700 1 -- start start 2023-11-26T07:16:41.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f11a41700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c0ccd80 0x7f7f0c1583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f11a41700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f0c0cd160 0x7f7f0c0b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f11a41700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f0c0b4870 0x7f7f0c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f11a41700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0b78f0 con 0x7f7f0c0cd160 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f11a41700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0aed00 con 0x7f7f0c0ccd80 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f11a41700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c0b7050 con 0x7f7f0c0b4870 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f0a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f0c0cd160 0x7f7f0c0b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f0a7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f0c0cd160 0x7f7f0c0b67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41120/0 (socket says 172.21.15.83:41120) 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f0a7fc700 1 -- 172.21.15.83:0/2011009495 learned_addr learned my addr 172.21.15.83:0/2011009495 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f0a7fc700 1 -- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f0c0b4870 msgr2=0x7f7f0c0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f0b7fe700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f0c0b4870 0x7f7f0c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f0affd700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c0ccd80 0x7f7f0c1583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f0a7fc700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f0c0b4870 0x7f7f0c0b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f0a7fc700 1 -- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c0ccd80 msgr2=0x7f7f0c1583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f0a7fc700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c0ccd80 0x7f7f0c1583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f0a7fc700 1 -- 172.21.15.83:0/2011009495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c0b7330 con 0x7f7f0c0cd160 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.296+0000 7f7f0affd700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c0ccd80 0x7f7f0c1583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.297+0000 7f7f0a7fc700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f0c0cd160 0x7f7f0c0b67b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0118b0 tx=0x7f7efc011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.297+0000 7f7ef3fff700 1 -- 172.21.15.83:0/2011009495 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00f870 con 0x7f7f0c0cd160 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.297+0000 7f7ef3fff700 1 -- 172.21.15.83:0/2011009495 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc00f9d0 con 0x7f7f0c0cd160 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.297+0000 7f7f11a41700 1 -- 172.21.15.83:0/2011009495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0c0b7610 con 0x7f7f0c0cd160 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.297+0000 7f7ef3fff700 1 -- 172.21.15.83:0/2011009495 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00fd60 con 0x7f7f0c0cd160 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.297+0000 7f7f11a41700 1 -- 172.21.15.83:0/2011009495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0c0b5260 con 0x7f7f0c0cd160 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.298+0000 7f7f11a41700 1 -- 172.21.15.83:0/2011009495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0c068fb0 con 0x7f7f0c0cd160 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.298+0000 7f7ef3fff700 1 -- 172.21.15.83:0/2011009495 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f7efc015460 con 0x7f7f0c0cd160 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.299+0000 7f7ef3fff700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7ef806aff0 0x7f7ef806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.299+0000 7f7ef3fff700 1 -- 172.21.15.83:0/2011009495 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7efc08a120 con 0x7f7f0c0cd160 2023-11-26T07:16:41.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.299+0000 7f7f0affd700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7ef806aff0 0x7f7ef806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.299+0000 7f7f0affd700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7ef806aff0 0x7f7ef806d4b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f00007fd0 tx=0x7f7f00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:41.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.303+0000 7f7ef3fff700 1 -- 172.21.15.83:0/2011009495 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7efc054f30 con 0x7f7f0c0cd160 2023-11-26T07:16:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.490+0000 7f7f11a41700 1 -- 172.21.15.83:0/2011009495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f0c0aee40 con 0x7f7f0c0cd160 2023-11-26T07:16:41.493 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.490+0000 7f7ef3fff700 1 -- 172.21.15.83:0/2011009495 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f0c0aee40 con 0x7f7f0c0cd160 2023-11-26T07:16:41.493 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:16:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.492+0000 7f7f11a41700 1 -- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7ef806aff0 msgr2=0x7f7ef806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.492+0000 7f7f11a41700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7ef806aff0 0x7f7ef806d4b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f00007fd0 tx=0x7f7f00009040 comp rx=0 tx=0).stop 2023-11-26T07:16:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.492+0000 7f7f11a41700 1 -- 172.21.15.83:0/2011009495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f0c0cd160 msgr2=0x7f7f0c0b67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.492+0000 7f7f11a41700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f0c0cd160 0x7f7f0c0b67b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0118b0 tx=0x7f7efc011c70 comp rx=0 tx=0).stop 2023-11-26T07:16:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.492+0000 7f7f11a41700 1 -- 172.21.15.83:0/2011009495 shutdown_connections 2023-11-26T07:16:41.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.492+0000 7f7f11a41700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7ef806aff0 0x7f7ef806d4b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.492+0000 7f7f11a41700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f0c0b4870 0x7f7f0c0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.492+0000 7f7f11a41700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f0c0ccd80 0x7f7f0c1583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.492+0000 7f7f11a41700 1 --2- 172.21.15.83:0/2011009495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f0c0cd160 0x7f7f0c0b67b0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.493+0000 7f7f11a41700 1 -- 172.21.15.83:0/2011009495 >> 172.21.15.83:0/2011009495 conn(0x7f7f0c0b3780 msgr2=0x7f7f0c159510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:41.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.493+0000 7f7f11a41700 1 -- 172.21.15.83:0/2011009495 shutdown_connections 2023-11-26T07:16:41.496 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.493+0000 7f7f11a41700 1 -- 172.21.15.83:0/2011009495 wait complete. 2023-11-26T07:16:41.507 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.627+0000 7ff7c1644700 1 Processor -- start 2023-11-26T07:16:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.627+0000 7ff7c1644700 1 -- start start 2023-11-26T07:16:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.628+0000 7ff7c1644700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc15e1e0 0x7ff7bc0d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.628+0000 7ff7c1644700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7bc0d3a00 0x7ff7bc0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.628+0000 7ff7c1644700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7bc0d20b0 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.628+0000 7ff7c1644700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7bc0d3150 con 0x7ff7bc0d3a00 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.628+0000 7ff7c1644700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc0d1240 con 0x7ff7bc15e1e0 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.628+0000 7ff7ba7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7bc0d3a00 0x7ff7bc0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.628+0000 7ff7ba7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff7bc0d3a00 0x7ff7bc0d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41126/0 (socket says 172.21.15.83:41126) 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.628+0000 7ff7ba7fc700 1 -- 172.21.15.83:0/2532842281 learned_addr learned my addr 172.21.15.83:0/2532842281 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.628+0000 7ff7baffd700 1 --2- 172.21.15.83:0/2532842281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc15e1e0 0x7ff7bc0d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7ba7fc700 1 -- 172.21.15.83:0/2532842281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc15e1e0 msgr2=0x7ff7bc0d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7ba7fc700 1 --2- 172.21.15.83:0/2532842281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc15e1e0 0x7ff7bc0d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7ba7fc700 1 -- 172.21.15.83:0/2532842281 >> v1:172.21.15.203:6789/0 conn(0x7ff7bc0d0db0 legacy=0x7ff7bc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7b9ffb700 1 -- 172.21.15.83:0/2532842281 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181162667 0 0) 0x7ff7bc0d20b0 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7ba7fc700 1 -- 172.21.15.83:0/2532842281 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc0d2390 con 0x7ff7bc0d3a00 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7ba7fc700 1 --2- 172.21.15.83:0/2532842281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc0d3a00 0x7ff7bc0d0870 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00098f0 tx=0x7ff7b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3412d68ab24bc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7b9ffb700 1 -- 172.21.15.83:0/2532842281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0011830 con 0x7ff7bc0d3a00 2023-11-26T07:16:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7b9ffb700 1 -- 172.21.15.83:0/2532842281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b0011990 con 0x7ff7bc0d3a00 2023-11-26T07:16:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7b9ffb700 1 -- 172.21.15.83:0/2532842281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0011d20 con 0x7ff7bc0d3a00 2023-11-26T07:16:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7c1644700 1 -- 172.21.15.83:0/2532842281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc0d3a00 msgr2=0x7ff7bc0d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7c1644700 1 --2- 172.21.15.83:0/2532842281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc0d3a00 0x7ff7bc0d0870 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00098f0 tx=0x7ff7b0009c00 comp rx=0 tx=0).stop 2023-11-26T07:16:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7c1644700 1 -- 172.21.15.83:0/2532842281 shutdown_connections 2023-11-26T07:16:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7c1644700 1 --2- 172.21.15.83:0/2532842281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc15e1e0 0x7ff7bc0d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7c1644700 1 --2- 172.21.15.83:0/2532842281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc0d3a00 0x7ff7bc0d0870 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.629+0000 7ff7c1644700 1 -- 172.21.15.83:0/2532842281 >> 172.21.15.83:0/2532842281 conn(0x7ff7bc0b5b50 msgr2=0x7ff7bc0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.630+0000 7ff7c1644700 1 -- 172.21.15.83:0/2532842281 shutdown_connections 2023-11-26T07:16:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.630+0000 7ff7c1644700 1 -- 172.21.15.83:0/2532842281 wait complete. 2023-11-26T07:16:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.631+0000 7ff7c1644700 1 Processor -- start 2023-11-26T07:16:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.631+0000 7ff7c1644700 1 -- start start 2023-11-26T07:16:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7c1644700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc0d0db0 0x7ff7bc1f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7c1644700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0d3a00 0x7ff7bc1f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7c1644700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7bc15e1e0 0x7ff7bc0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7c1644700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc15f830 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7c1644700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc1f45e0 con 0x7ff7bc15e1e0 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7c1644700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc1f48c0 con 0x7ff7bc0d3a00 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7baffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc0d0db0 0x7ff7bc1f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7baffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc0d0db0 0x7ff7bc1f3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41130/0 (socket says 172.21.15.83:41130) 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7baffd700 1 -- 172.21.15.83:0/3989162771 learned_addr learned my addr 172.21.15.83:0/3989162771 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7baffd700 1 -- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0d3a00 msgr2=0x7ff7bc1f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7baffd700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0d3a00 0x7ff7bc1f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7bb7fe700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7bc15e1e0 0x7ff7bc0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7baffd700 1 -- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7bc15e1e0 msgr2=0x7ff7bc0b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.632+0000 7ff7baffd700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7bc15e1e0 0x7ff7bc0b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.633+0000 7ff7baffd700 1 -- 172.21.15.83:0/3989162771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b0009550 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.633+0000 7ff7bb7fe700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7bc15e1e0 0x7ff7bc0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.633+0000 7ff7baffd700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc0d0db0 0x7ff7bc1f3a90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00e870 tx=0x7ff7ac00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.633+0000 7ff7a37fe700 1 -- 172.21.15.83:0/3989162771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac009590 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.633+0000 7ff7a37fe700 1 -- 172.21.15.83:0/3989162771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac00ee70 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.633+0000 7ff7c1644700 1 -- 172.21.15.83:0/3989162771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc15fa80 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.633+0000 7ff7a37fe700 1 -- 172.21.15.83:0/3989162771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00ca50 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.633+0000 7ff7c1644700 1 -- 172.21.15.83:0/3989162771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc0cdcc0 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.634+0000 7ff7c1644700 1 -- 172.21.15.83:0/3989162771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7bc1600d0 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.634+0000 7ff7a37fe700 1 -- 172.21.15.83:0/3989162771 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7ff7ac00cbb0 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.635+0000 7ff7a37fe700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff7a806aff0 0x7ff7a806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.635+0000 7ff7a37fe700 1 -- 172.21.15.83:0/3989162771 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff7ac0896a0 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.635+0000 7ff7ba7fc700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff7a806aff0 0x7ff7a806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:41.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.635+0000 7ff7ba7fc700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff7a806aff0 0x7ff7a806d4b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc069200 tx=0x7ff7b0010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:41.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.639+0000 7ff7a37fe700 1 -- 172.21.15.83:0/3989162771 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff7ac054670 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.874+0000 7ff7c1644700 1 -- 172.21.15.83:0/3989162771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7bc15e5c0 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.875+0000 7ff7a37fe700 1 -- 172.21.15.83:0/3989162771 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4918 (secure 0 0 0) 0x7ff7ac00ce60 con 0x7ff7bc0d0db0 2023-11-26T07:16:41.877 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:41.878 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1015690400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1015690400},{"type":"v1","addr":"172.21.15.203:6834","nonce":1015690400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2716428832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2716428832},{"type":"v1","addr":"172.21.15.83:6834","nonce":2716428832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/558945070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":558945070},{"type":"v1","addr":"172.21.15.83:6835","nonce":558945070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:16:17.967531+0000","modified":"2023-11-26T07:16:21.013928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1901175330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1901175330},{"type":"v1","addr":"172.21.15.203:6835","nonce":1901175330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:16:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 -- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff7a806aff0 msgr2=0x7ff7a806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff7a806aff0 0x7ff7a806d4b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc069200 tx=0x7ff7b0010000 comp rx=0 tx=0).stop 2023-11-26T07:16:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 -- 172.21.15.83:0/3989162771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc0d0db0 msgr2=0x7ff7bc1f3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc0d0db0 0x7ff7bc1f3a90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00e870 tx=0x7ff7ac00ec30 comp rx=0 tx=0).stop 2023-11-26T07:16:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 -- 172.21.15.83:0/3989162771 shutdown_connections 2023-11-26T07:16:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff7a806aff0 0x7ff7a806d4b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7bc15e1e0 0x7ff7bc0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc0d0db0 0x7ff7bc1f3a90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 --2- 172.21.15.83:0/3989162771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0d3a00 0x7ff7bc1f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 -- 172.21.15.83:0/3989162771 >> 172.21.15.83:0/3989162771 conn(0x7ff7bc0b5b50 msgr2=0x7ff7bc0b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 -- 172.21.15.83:0/3989162771 shutdown_connections 2023-11-26T07:16:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:41.877+0000 7ff7c1644700 1 -- 172.21.15.83:0/3989162771 wait complete. 2023-11-26T07:16:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 60 2023-11-26T07:16:41.891 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.009+0000 7fe8f143c700 1 Processor -- start 2023-11-26T07:16:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.009+0000 7fe8f143c700 1 -- start start 2023-11-26T07:16:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.010+0000 7fe8f143c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8ec15e1e0 0x7fe8ec0d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.010+0000 7fe8f143c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0d3a00 0x7fe8ec0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8f143c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec0d20b0 con 0x7fe8ec0d0db0 2023-11-26T07:16:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8f143c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ec0d3150 con 0x7fe8ec15e1e0 2023-11-26T07:16:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8f143c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0d1240 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8eaffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8ec15e1e0 0x7fe8ec0d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8eb7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe8ec0d0db0 0x7fe8ec06aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:32950/0 (socket says 172.21.15.83:32950) 2023-11-26T07:16:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8eaffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe8ec15e1e0 0x7fe8ec0d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41138/0 (socket says 172.21.15.83:41138) 2023-11-26T07:16:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8eb7fe700 1 -- 172.21.15.83:0/4290038056 learned_addr learned my addr 172.21.15.83:0/4290038056 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8ea7fc700 1 --2- 172.21.15.83:0/4290038056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0d3a00 0x7fe8ec0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8eaffd700 1 -- 172.21.15.83:0/4290038056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0d3a00 msgr2=0x7fe8ec0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8eaffd700 1 --2- 172.21.15.83:0/4290038056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0d3a00 0x7fe8ec0d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8eaffd700 1 -- 172.21.15.83:0/4290038056 >> v1:172.21.15.83:6789/0 conn(0x7fe8ec0d0db0 legacy=0x7fe8ec06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8eaffd700 1 -- 172.21.15.83:0/4290038056 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec0d2390 con 0x7fe8ec15e1e0 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8e9ffb700 1 -- 172.21.15.83:0/4290038056 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862803688 0 0) 0x7fe8ec0d20b0 con 0x7fe8ec0d0db0 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8eaffd700 1 --2- 172.21.15.83:0/4290038056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec15e1e0 0x7fe8ec0d34c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009840 tx=0x7fe8dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a76919b72e0f00 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8e9ffb700 1 -- 172.21.15.83:0/4290038056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc011830 con 0x7fe8ec15e1e0 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8e9ffb700 1 -- 172.21.15.83:0/4290038056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc011990 con 0x7fe8ec15e1e0 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.011+0000 7fe8e9ffb700 1 -- 172.21.15.83:0/4290038056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc011d20 con 0x7fe8ec15e1e0 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.012+0000 7fe8f143c700 1 -- 172.21.15.83:0/4290038056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec15e1e0 msgr2=0x7fe8ec0d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.012+0000 7fe8f143c700 1 --2- 172.21.15.83:0/4290038056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec15e1e0 0x7fe8ec0d34c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009840 tx=0x7fe8dc009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.012+0000 7fe8f143c700 1 -- 172.21.15.83:0/4290038056 shutdown_connections 2023-11-26T07:16:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.012+0000 7fe8f143c700 1 --2- 172.21.15.83:0/4290038056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec15e1e0 0x7fe8ec0d34c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.012+0000 7fe8f143c700 1 --2- 172.21.15.83:0/4290038056 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec0d3a00 0x7fe8ec0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.012+0000 7fe8f143c700 1 -- 172.21.15.83:0/4290038056 >> 172.21.15.83:0/4290038056 conn(0x7fe8ec0b5b50 msgr2=0x7fe8ec0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:42.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.012+0000 7fe8f143c700 1 -- 172.21.15.83:0/4290038056 shutdown_connections 2023-11-26T07:16:42.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.012+0000 7fe8f143c700 1 -- 172.21.15.83:0/4290038056 wait complete. 2023-11-26T07:16:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.013+0000 7fe8f143c700 1 Processor -- start 2023-11-26T07:16:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.013+0000 7fe8f143c700 1 -- start start 2023-11-26T07:16:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8f143c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ec0d0db0 0x7fe8ec0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8f143c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d3a00 0x7fe8ec0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8f143c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec15e1e0 0x7fe8ec1544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8f143c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec1582d0 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8f143c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0cc2e0 con 0x7fe8ec0d0db0 2023-11-26T07:16:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8f143c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0cb760 con 0x7fe8ec15e1e0 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8ea7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d3a00 0x7fe8ec0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8ea7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d3a00 0x7fe8ec0b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41146/0 (socket says 172.21.15.83:41146) 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8ea7fc700 1 -- 172.21.15.83:0/523120034 learned_addr learned my addr 172.21.15.83:0/523120034 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8eaffd700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ec0d0db0 0x7fe8ec0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8ea7fc700 1 -- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec15e1e0 msgr2=0x7fe8ec1544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8ea7fc700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec15e1e0 0x7fe8ec1544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8ea7fc700 1 -- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ec0d0db0 msgr2=0x7fe8ec0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8ea7fc700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ec0d0db0 0x7fe8ec0caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.014+0000 7fe8ea7fc700 1 -- 172.21.15.83:0/523120034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0094a0 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.015+0000 7fe8ea7fc700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d3a00 0x7fe8ec0b3f10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000e810 tx=0x7fe8e000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.015+0000 7fe8d37fe700 1 -- 172.21.15.83:0/523120034 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0013610 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.015+0000 7fe8d37fe700 1 -- 172.21.15.83:0/523120034 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e0013770 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.015+0000 7fe8f143c700 1 -- 172.21.15.83:0/523120034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec0cba40 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.015+0000 7fe8d37fe700 1 -- 172.21.15.83:0/523120034 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0013b20 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.015+0000 7fe8f143c700 1 -- 172.21.15.83:0/523120034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec0cc000 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.016+0000 7fe8f143c700 1 -- 172.21.15.83:0/523120034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec068670 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.016+0000 7fe8d37fe700 1 -- 172.21.15.83:0/523120034 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fe8e0013c80 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.016+0000 7fe8d37fe700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8d806aff0 0x7fe8d806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.017+0000 7fe8d37fe700 1 -- 172.21.15.83:0/523120034 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe8e00895e0 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.017+0000 7fe8eaffd700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8d806aff0 0x7fe8d806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.017+0000 7fe8eaffd700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8d806aff0 0x7fe8d806d4b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009b90 tx=0x7fe8dc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:42.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.021+0000 7fe8d37fe700 1 -- 172.21.15.83:0/523120034 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8e0054380 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.209+0000 7fe8f143c700 1 -- 172.21.15.83:0/523120034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8ec0751e0 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.211+0000 7fe8d37fe700 1 -- 172.21.15.83:0/523120034 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16675 (secure 0 0 0) 0x7fe8e00579a0 con 0x7fe8ec0d3a00 2023-11-26T07:16:42.213 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:42.213 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:35.818271+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:16:15.747889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:16:16.943249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6835/469139444":"2023-11-27T07:16:32.809518+0000","172.21.15.83:6833/469139444":"2023-11-27T07:16:32.809518+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:42.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 -- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8d806aff0 msgr2=0x7fe8d806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8d806aff0 0x7fe8d806d4b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009b90 tx=0x7fe8dc008030 comp rx=0 tx=0).stop 2023-11-26T07:16:42.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 -- 172.21.15.83:0/523120034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d3a00 msgr2=0x7fe8ec0b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d3a00 0x7fe8ec0b3f10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000e810 tx=0x7fe8e000ebd0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 -- 172.21.15.83:0/523120034 shutdown_connections 2023-11-26T07:16:42.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8d806aff0 0x7fe8d806d4b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe8ec15e1e0 0x7fe8ec1544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8ec0d0db0 0x7fe8ec0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 --2- 172.21.15.83:0/523120034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8ec0d3a00 0x7fe8ec0b3f10 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 -- 172.21.15.83:0/523120034 >> 172.21.15.83:0/523120034 conn(0x7fe8ec0b5b50 msgr2=0x7fe8ec0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:42.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 -- 172.21.15.83:0/523120034 shutdown_connections 2023-11-26T07:16:42.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.213+0000 7fe8f143c700 1 -- 172.21.15.83:0/523120034 wait complete. 2023-11-26T07:16:42.229 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:16:42.230 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:42.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.348+0000 7f21be851700 1 Processor -- start 2023-11-26T07:16:42.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.348+0000 7f21be851700 1 -- start start 2023-11-26T07:16:42.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.348+0000 7f21be851700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f21b8073f90 0x7f21b80ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.348+0000 7f21be851700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b815c140 0x7f21b80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.348+0000 7f21be851700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b8074e90 con 0x7f21b806ee50 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.348+0000 7f21be851700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b80ce5d0 con 0x7f21b8073f90 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.348+0000 7f21be851700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b815c700 con 0x7f21b815c140 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b7fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f21b8073f90 0x7f21b80ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b77fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f21b806ee50 0x7f21b8074370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:32962/0 (socket says 172.21.15.83:32962) 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b7fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f21b8073f90 0x7f21b80ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41152/0 (socket says 172.21.15.83:41152) 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b77fe700 1 -- 172.21.15.83:0/1147625890 learned_addr learned my addr 172.21.15.83:0/1147625890 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b7fff700 1 -- 172.21.15.83:0/1147625890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b815c140 msgr2=0x7f21b80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21bcdee700 1 --2- 172.21.15.83:0/1147625890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b815c140 0x7f21b80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b6ffd700 1 -- 172.21.15.83:0/1147625890 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975099668 0 0) 0x7f21b8074e90 con 0x7f21b806ee50 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b7fff700 1 --2- 172.21.15.83:0/1147625890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b815c140 0x7f21b80cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b7fff700 1 -- 172.21.15.83:0/1147625890 >> v1:172.21.15.83:6789/0 conn(0x7f21b806ee50 legacy=0x7f21b8074370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b7fff700 1 -- 172.21.15.83:0/1147625890 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b815c9e0 con 0x7f21b8073f90 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b7fff700 1 --2- 172.21.15.83:0/1147625890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b8073f90 0x7f21b80ce090 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009840 tx=0x7f21a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7d101e2768ae54 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b6ffd700 1 -- 172.21.15.83:0/1147625890 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8011830 con 0x7f21b8073f90 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b6ffd700 1 -- 172.21.15.83:0/1147625890 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a8011990 con 0x7f21b8073f90 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.349+0000 7f21b6ffd700 1 -- 172.21.15.83:0/1147625890 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8011d20 con 0x7f21b8073f90 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.350+0000 7f21be851700 1 -- 172.21.15.83:0/1147625890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b8073f90 msgr2=0x7f21b80ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.350+0000 7f21be851700 1 --2- 172.21.15.83:0/1147625890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b8073f90 0x7f21b80ce090 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009840 tx=0x7f21a8009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.350+0000 7f21be851700 1 -- 172.21.15.83:0/1147625890 shutdown_connections 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.350+0000 7f21be851700 1 --2- 172.21.15.83:0/1147625890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b815c140 0x7f21b80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.350+0000 7f21be851700 1 --2- 172.21.15.83:0/1147625890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b8073f90 0x7f21b80ce090 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.350+0000 7f21be851700 1 -- 172.21.15.83:0/1147625890 >> 172.21.15.83:0/1147625890 conn(0x7f21b80cd570 msgr2=0x7f21b80cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.350+0000 7f21be851700 1 -- 172.21.15.83:0/1147625890 shutdown_connections 2023-11-26T07:16:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.350+0000 7f21be851700 1 -- 172.21.15.83:0/1147625890 wait complete. 2023-11-26T07:16:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.351+0000 7f21be851700 1 Processor -- start 2023-11-26T07:16:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.352+0000 7f21be851700 1 -- start start 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.352+0000 7f21be851700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b806ee50 0x7f21b8146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.352+0000 7f21be851700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b8073f90 0x7f21b81e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.352+0000 7f21be851700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b815c140 0x7f21b81e7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.352+0000 7f21be851700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b814c460 con 0x7f21b815c140 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.352+0000 7f21be851700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b815b910 con 0x7f21b8073f90 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.352+0000 7f21be851700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b814bac0 con 0x7f21b806ee50 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21bcdee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b815c140 0x7f21b81e7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21bcdee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b815c140 0x7f21b81e7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41162/0 (socket says 172.21.15.83:41162) 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21bcdee700 1 -- 172.21.15.83:0/3328476336 learned_addr learned my addr 172.21.15.83:0/3328476336 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21b7fff700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b806ee50 0x7f21b8146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21b77fe700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b8073f90 0x7f21b81e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21bcdee700 1 -- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b806ee50 msgr2=0x7f21b8146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21bcdee700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b806ee50 0x7f21b8146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21bcdee700 1 -- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b8073f90 msgr2=0x7f21b81e79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21bcdee700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b8073f90 0x7f21b81e79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21bcdee700 1 -- 172.21.15.83:0/3328476336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a80094a0 con 0x7f21b815c140 2023-11-26T07:16:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21bcdee700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b815c140 0x7f21b81e7f00 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f21b000cc30 tx=0x7f21b000a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21b4ff9700 1 -- 172.21.15.83:0/3328476336 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b000ab70 con 0x7f21b815c140 2023-11-26T07:16:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21b4ff9700 1 -- 172.21.15.83:0/3328476336 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b000acd0 con 0x7f21b815c140 2023-11-26T07:16:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21be851700 1 -- 172.21.15.83:0/3328476336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b814bda0 con 0x7f21b815c140 2023-11-26T07:16:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21b4ff9700 1 -- 172.21.15.83:0/3328476336 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b0003c40 con 0x7f21b815c140 2023-11-26T07:16:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.353+0000 7f21be851700 1 -- 172.21.15.83:0/3328476336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b8146cb0 con 0x7f21b815c140 2023-11-26T07:16:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.354+0000 7f21be851700 1 -- 172.21.15.83:0/3328476336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b814c5a0 con 0x7f21b815c140 2023-11-26T07:16:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.355+0000 7f21b4ff9700 1 -- 172.21.15.83:0/3328476336 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f21b000f3f0 con 0x7f21b815c140 2023-11-26T07:16:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.355+0000 7f21b4ff9700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f219806aff0 0x7f219806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.355+0000 7f21b4ff9700 1 -- 172.21.15.83:0/3328476336 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f21b0089b10 con 0x7f21b815c140 2023-11-26T07:16:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.355+0000 7f21b7fff700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f219806aff0 0x7f219806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.356+0000 7f21b7fff700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f219806aff0 0x7f219806d4b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21a80090b0 tx=0x7f21a8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.360+0000 7f21b4ff9700 1 -- 172.21.15.83:0/3328476336 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21b814c5a0 con 0x7f21b815c140 2023-11-26T07:16:42.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.592+0000 7f21be851700 1 -- 172.21.15.83:0/3328476336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21b814c5a0 con 0x7f21b815c140 2023-11-26T07:16:42.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.593+0000 7f21b4ff9700 1 -- 172.21.15.83:0/3328476336 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f21b814c5a0 con 0x7f21b815c140 2023-11-26T07:16:42.595 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:42.595 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1015690400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1015690400},{"type":"v1","addr":"172.21.15.203:6834","nonce":1015690400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2716428832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2716428832},{"type":"v1","addr":"172.21.15.83:6834","nonce":2716428832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/558945070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":558945070},{"type":"v1","addr":"172.21.15.83:6835","nonce":558945070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:16:17.967531+0000","modified":"2023-11-26T07:16:21.013928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1901175330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1901175330},{"type":"v1","addr":"172.21.15.203:6835","nonce":1901175330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:16:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 -- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f219806aff0 msgr2=0x7f219806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f219806aff0 0x7f219806d4b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21a80090b0 tx=0x7f21a8008030 comp rx=0 tx=0).stop 2023-11-26T07:16:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 -- 172.21.15.83:0/3328476336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b815c140 msgr2=0x7f21b81e7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b815c140 0x7f21b81e7f00 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f21b000cc30 tx=0x7f21b000a1f0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 -- 172.21.15.83:0/3328476336 shutdown_connections 2023-11-26T07:16:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f219806aff0 0x7f219806d4b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b815c140 0x7f21b81e7f00 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21b806ee50 0x7f21b8146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 --2- 172.21.15.83:0/3328476336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21b8073f90 0x7f21b81e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 -- 172.21.15.83:0/3328476336 >> 172.21.15.83:0/3328476336 conn(0x7f21b80cd570 msgr2=0x7f21b80cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:42.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 -- 172.21.15.83:0/3328476336 shutdown_connections 2023-11-26T07:16:42.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.595+0000 7f21be851700 1 -- 172.21.15.83:0/3328476336 wait complete. 2023-11-26T07:16:42.599 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 60 2023-11-26T07:16:42.611 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:42.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.729+0000 7f12e9a9e700 1 Processor -- start 2023-11-26T07:16:42.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.729+0000 7f12e9a9e700 1 -- start start 2023-11-26T07:16:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e9a9e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f12e415e080 0x7f12e40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e9a9e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f12e406c710 0x7f12e40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e9a9e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e40ae430 con 0x7f12e40af4d0 2023-11-26T07:16:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e9a9e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e40af0b0 con 0x7f12e406c710 2023-11-26T07:16:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e9a9e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e40b05a0 con 0x7f12e415e080 2023-11-26T07:16:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e3fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f12e406c710 0x7f12e40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e3fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f12e406c710 0x7f12e40afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41170/0 (socket says 172.21.15.83:41170) 2023-11-26T07:16:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e3fff700 1 -- 172.21.15.83:0/1456414062 learned_addr learned my addr 172.21.15.83:0/1456414062 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e37fe700 1 --2- 172.21.15.83:0/1456414062 >> v2:172.21.15.203:3300/0 conn(0x7f12e415e080 0x7f12e40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e3fff700 1 -- 172.21.15.83:0/1456414062 >> v2:172.21.15.203:3300/0 conn(0x7f12e415e080 msgr2=0x7f12e40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e3fff700 1 --2- 172.21.15.83:0/1456414062 >> v2:172.21.15.203:3300/0 conn(0x7f12e415e080 0x7f12e40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e3fff700 1 -- 172.21.15.83:0/1456414062 >> v1:172.21.15.203:6789/0 conn(0x7f12e40af4d0 legacy=0x7f12e406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e3fff700 1 -- 172.21.15.83:0/1456414062 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e40ae710 con 0x7f12e406c710 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.730+0000 7f12e3fff700 1 --2- 172.21.15.83:0/1456414062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12e406c710 0x7f12e40afb90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f12e4072a90 tx=0x7f12dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1f97c41fcd35eb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.731+0000 7f12e27fc700 1 -- 172.21.15.83:0/1456414062 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc00e070 con 0x7f12e406c710 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.731+0000 7f12e27fc700 1 -- 172.21.15.83:0/1456414062 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc003a20 con 0x7f12e406c710 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.731+0000 7f12e27fc700 1 -- 172.21.15.83:0/1456414062 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc003dd0 con 0x7f12e406c710 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.731+0000 7f12e9a9e700 1 -- 172.21.15.83:0/1456414062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12e406c710 msgr2=0x7f12e40afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.731+0000 7f12e9a9e700 1 --2- 172.21.15.83:0/1456414062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12e406c710 0x7f12e40afb90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f12e4072a90 tx=0x7f12dc00b330 comp rx=0 tx=0).stop 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.731+0000 7f12e9a9e700 1 -- 172.21.15.83:0/1456414062 shutdown_connections 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.731+0000 7f12e9a9e700 1 --2- 172.21.15.83:0/1456414062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12e406c710 0x7f12e40afb90 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.731+0000 7f12e9a9e700 1 --2- 172.21.15.83:0/1456414062 >> v2:172.21.15.203:3300/0 conn(0x7f12e415e080 0x7f12e40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.731+0000 7f12e9a9e700 1 -- 172.21.15.83:0/1456414062 >> 172.21.15.83:0/1456414062 conn(0x7f12e4071b90 msgr2=0x7f12e4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.731+0000 7f12e9a9e700 1 -- 172.21.15.83:0/1456414062 shutdown_connections 2023-11-26T07:16:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.731+0000 7f12e9a9e700 1 -- 172.21.15.83:0/1456414062 wait complete. 2023-11-26T07:16:42.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.732+0000 7f12e9a9e700 1 Processor -- start 2023-11-26T07:16:42.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.733+0000 7f12e9a9e700 1 -- start start 2023-11-26T07:16:42.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.733+0000 7f12e9a9e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12e406c710 0x7f12e414c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.733+0000 7f12e9a9e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12e40af4d0 0x7f12e41f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.733+0000 7f12e9a9e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12e415e080 0x7f12e40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.733+0000 7f12e9a9e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e40ce6d0 con 0x7f12e40af4d0 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.733+0000 7f12e9a9e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e40ce9b0 con 0x7f12e406c710 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.733+0000 7f12e9a9e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e40cec90 con 0x7f12e415e080 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.733+0000 7f12e2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12e40af4d0 0x7f12e41f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.733+0000 7f12e2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12e40af4d0 0x7f12e41f41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41174/0 (socket says 172.21.15.83:41174) 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.733+0000 7f12e2ffd700 1 -- 172.21.15.83:0/26421297 learned_addr learned my addr 172.21.15.83:0/26421297 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12e37fe700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12e406c710 0x7f12e414c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12e3fff700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12e415e080 0x7f12e40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12e2ffd700 1 -- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12e415e080 msgr2=0x7f12e40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12e2ffd700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12e415e080 0x7f12e40cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12e2ffd700 1 -- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12e406c710 msgr2=0x7f12e414c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12e2ffd700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12e406c710 0x7f12e414c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12e2ffd700 1 -- 172.21.15.83:0/26421297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc009b30 con 0x7f12e40af4d0 2023-11-26T07:16:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12e2ffd700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12e40af4d0 0x7f12e41f41a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f12d400c4a0 tx=0x7f12d400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12dbfff700 1 -- 172.21.15.83:0/26421297 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4011790 con 0x7f12e40af4d0 2023-11-26T07:16:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12dbfff700 1 -- 172.21.15.83:0/26421297 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d40118f0 con 0x7f12e40af4d0 2023-11-26T07:16:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12e9a9e700 1 -- 172.21.15.83:0/26421297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e40cef70 con 0x7f12e40af4d0 2023-11-26T07:16:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12dbfff700 1 -- 172.21.15.83:0/26421297 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4011ca0 con 0x7f12e40af4d0 2023-11-26T07:16:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.734+0000 7f12e9a9e700 1 -- 172.21.15.83:0/26421297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e41f3b30 con 0x7f12e40af4d0 2023-11-26T07:16:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.735+0000 7f12e9a9e700 1 -- 172.21.15.83:0/26421297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e40cf5c0 con 0x7f12e40af4d0 2023-11-26T07:16:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.736+0000 7f12dbfff700 1 -- 172.21.15.83:0/26421297 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f12d401c460 con 0x7f12e40af4d0 2023-11-26T07:16:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.736+0000 7f12dbfff700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f12c406aff0 0x7f12c406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.736+0000 7f12dbfff700 1 -- 172.21.15.83:0/26421297 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f12d40894a0 con 0x7f12e40af4d0 2023-11-26T07:16:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.737+0000 7f12e37fe700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f12c406aff0 0x7f12c406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.737+0000 7f12e37fe700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f12c406aff0 0x7f12c406d4b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12cc006b20 tx=0x7f12cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.741+0000 7f12dbfff700 1 -- 172.21.15.83:0/26421297 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12d40543e0 con 0x7f12e40af4d0 2023-11-26T07:16:42.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.974+0000 7f12e9a9e700 1 -- 172.21.15.83:0/26421297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12e40753f0 con 0x7f12e40af4d0 2023-11-26T07:16:42.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.975+0000 7f12dbfff700 1 -- 172.21.15.83:0/26421297 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f12d4057a00 con 0x7f12e40af4d0 2023-11-26T07:16:42.977 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:42.977 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1015690400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1015690400},{"type":"v1","addr":"172.21.15.203:6834","nonce":1015690400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2716428832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2716428832},{"type":"v1","addr":"172.21.15.83:6834","nonce":2716428832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/558945070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":558945070},{"type":"v1","addr":"172.21.15.83:6835","nonce":558945070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:16:17.967531+0000","modified":"2023-11-26T07:16:21.013928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.203:6835/1901175330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":1901175330},{"type":"v1","addr":"172.21.15.203:6835","nonce":1901175330}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:16:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 -- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f12c406aff0 msgr2=0x7f12c406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f12c406aff0 0x7f12c406d4b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12cc006b20 tx=0x7f12cc008040 comp rx=0 tx=0).stop 2023-11-26T07:16:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 -- 172.21.15.83:0/26421297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12e40af4d0 msgr2=0x7f12e41f41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12e40af4d0 0x7f12e41f41a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f12d400c4a0 tx=0x7f12d400c7b0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 -- 172.21.15.83:0/26421297 shutdown_connections 2023-11-26T07:16:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f12c406aff0 0x7f12c406d4b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f12e415e080 0x7f12e40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12e406c710 0x7f12e414c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 --2- 172.21.15.83:0/26421297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12e40af4d0 0x7f12e41f41a0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 -- 172.21.15.83:0/26421297 >> 172.21.15.83:0/26421297 conn(0x7f12e4071b90 msgr2=0x7f12e40b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 -- 172.21.15.83:0/26421297 shutdown_connections 2023-11-26T07:16:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:42.977+0000 7f12e9a9e700 1 -- 172.21.15.83:0/26421297 wait complete. 2023-11-26T07:16:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 60 2023-11-26T07:16:42.991 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.109+0000 7fbb2ed2b700 1 Processor -- start 2023-11-26T07:16:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.109+0000 7fbb2ed2b700 1 -- start start 2023-11-26T07:16:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.110+0000 7fbb2ed2b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbb2815e080 0x7fbb280b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.110+0000 7fbb2ed2b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280af4d0 0x7fbb280afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.110+0000 7fbb2ed2b700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb280ae430 con 0x7fbb2806c000 2023-11-26T07:16:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.110+0000 7fbb2ed2b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb280af0b0 con 0x7fbb2815e080 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.110+0000 7fbb2ed2b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280b05a0 con 0x7fbb280af4d0 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.110+0000 7fbb2d2c8700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbb2806c000 0x7fbb28073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:32992/0 (socket says 172.21.15.83:32992) 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.110+0000 7fbb2d2c8700 1 -- 172.21.15.83:0/1274259799 learned_addr learned my addr 172.21.15.83:0/1274259799 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb27fff700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280af4d0 0x7fbb280afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb2cac7700 1 --2- 172.21.15.83:0/1274259799 >> v2:172.21.15.203:3300/0 conn(0x7fbb2815e080 0x7fbb280b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635809208 0 0) 0x7fbb280ae430 con 0x7fbb2806c000 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb180035b0 con 0x7fbb2806c000 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 470764554 0 0) 0x7fbb180035b0 con 0x7fbb2806c000 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280af4d0 msgr2=0x7fbb280afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280af4d0 0x7fbb280afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 >> v2:172.21.15.203:3300/0 conn(0x7fbb2815e080 msgr2=0x7fbb280b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 --2- 172.21.15.83:0/1274259799 >> v2:172.21.15.203:3300/0 conn(0x7fbb2815e080 0x7fbb280b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb280ae710 con 0x7fbb2806c000 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fbb200019f0 con 0x7fbb2806c000 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 >> v1:172.21.15.83:6789/0 conn(0x7fbb2806c000 legacy=0x7fbb28073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb18003a30 0x7fbb18003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb18004850 0x7fbb18004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280af4d0 0x7fbb18005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280af0b0 con 0x7fbb18004850 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280b05a0 con 0x7fbb18003a30 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1800ba10 con 0x7fbb280af4d0 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb27fff700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb18004850 0x7fbb18004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb2cac7700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb18003a30 0x7fbb18003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb2d2c8700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280af4d0 0x7fbb18005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb27fff700 1 -- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280af4d0 msgr2=0x7fbb18005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb27fff700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280af4d0 0x7fbb18005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb27fff700 1 -- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb18003a30 msgr2=0x7fbb18003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb27fff700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb18003a30 0x7fbb18003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb27fff700 1 -- 172.21.15.83:0/1274259799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180035b0 con 0x7fbb18004850 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.111+0000 7fbb2d2c8700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280af4d0 0x7fbb18005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.112+0000 7fbb27fff700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb18004850 0x7fbb18004cd0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009b30 tx=0x7fbb18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c7cce929a0342f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.112+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400e6c0 con 0x7fbb18004850 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.112+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1400e820 con 0x7fbb18004850 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.112+0000 7fbb277fe700 1 -- 172.21.15.83:0/1274259799 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400eb90 con 0x7fbb18004850 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.112+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/1274259799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb18004850 msgr2=0x7fbb18004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.112+0000 7fbb2ed2b700 1 --2- 172.21.15.83:0/1274259799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb18004850 0x7fbb18004cd0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009b30 tx=0x7fbb18005880 comp rx=0 tx=0).stop 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.112+0000 7fbb2d2c8700 1 -- 172.21.15.83:0/1274259799 reap_dead start 2023-11-26T07:16:43.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.112+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/1274259799 shutdown_connections 2023-11-26T07:16:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.112+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/1274259799 >> 172.21.15.83:0/1274259799 conn(0x7fbb28071b90 msgr2=0x7fbb28071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.112+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/1274259799 shutdown_connections 2023-11-26T07:16:43.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.112+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/1274259799 wait complete. 2023-11-26T07:16:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.114+0000 7fbb2ed2b700 1 Processor -- start 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.114+0000 7fbb2ed2b700 1 -- start start 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2ed2b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb18004850 0x7fbb28148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2ed2b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280b4ff0 0x7fbb280b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2ed2b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb2815fc40 0x7fbb281600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2ed2b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28157380 con 0x7fbb2815fc40 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2ed2b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280b59b0 con 0x7fbb18004850 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2ed2b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28160a50 con 0x7fbb280b4ff0 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2d2c8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb2815fc40 0x7fbb281600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2d2c8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb2815fc40 0x7fbb281600a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41196/0 (socket says 172.21.15.83:41196) 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2d2c8700 1 -- 172.21.15.83:0/869942982 learned_addr learned my addr 172.21.15.83:0/869942982 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb27fff700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280b4ff0 0x7fbb280b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2cac7700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb18004850 0x7fbb28148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2d2c8700 1 -- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280b4ff0 msgr2=0x7fbb280b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2d2c8700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280b4ff0 0x7fbb280b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2d2c8700 1 -- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb18004850 msgr2=0x7fbb28148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2d2c8700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb18004850 0x7fbb28148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb2d2c8700 1 -- 172.21.15.83:0/869942982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb28160d30 con 0x7fbb2815fc40 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.115+0000 7fbb27fff700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280b4ff0 0x7fbb280b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.116+0000 7fbb2cac7700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb18004850 0x7fbb28148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.116+0000 7fbb2d2c8700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb2815fc40 0x7fbb281600a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fbb20000f80 tx=0x7fbb20001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.116+0000 7fbb257fa700 1 -- 172.21.15.83:0/869942982 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000ed10 con 0x7fbb2815fc40 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.116+0000 7fbb257fa700 1 -- 172.21.15.83:0/869942982 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2000ee90 con 0x7fbb2815fc40 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.116+0000 7fbb257fa700 1 -- 172.21.15.83:0/869942982 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20013660 con 0x7fbb2815fc40 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.116+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/869942982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb28161010 con 0x7fbb2815fc40 2023-11-26T07:16:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.116+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/869942982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb281612f0 con 0x7fbb2815fc40 2023-11-26T07:16:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.117+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/869942982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb280753f0 con 0x7fbb2815fc40 2023-11-26T07:16:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.117+0000 7fbb257fa700 1 -- 172.21.15.83:0/869942982 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fbb200137c0 con 0x7fbb2815fc40 2023-11-26T07:16:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.118+0000 7fbb257fa700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbb0c06aff0 0x7fbb0c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.118+0000 7fbb257fa700 1 -- 172.21.15.83:0/869942982 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbb20089db0 con 0x7fbb2815fc40 2023-11-26T07:16:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.118+0000 7fbb2cac7700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbb0c06aff0 0x7fbb0c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.118+0000 7fbb2cac7700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbb0c06aff0 0x7fbb0c06d4b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009bd0 tx=0x7fbb1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.122+0000 7fbb257fa700 1 -- 172.21.15.83:0/869942982 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb20054d80 con 0x7fbb2815fc40 2023-11-26T07:16:43.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.314+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/869942982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb28067f60 con 0x7fbb2815fc40 2023-11-26T07:16:43.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.315+0000 7fbb257fa700 1 -- 172.21.15.83:0/869942982 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16675 (secure 0 0 0) 0x7fbb200583a0 con 0x7fbb2815fc40 2023-11-26T07:16:43.318 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:43.318 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":90,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:35.818271+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:16:15.747889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:16:16.943249+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6835/469139444":"2023-11-27T07:16:32.809518+0000","172.21.15.83:6833/469139444":"2023-11-27T07:16:32.809518+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:43.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.317+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbb0c06aff0 msgr2=0x7fbb0c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.317+0000 7fbb2ed2b700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbb0c06aff0 0x7fbb0c06d4b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009bd0 tx=0x7fbb1c008040 comp rx=0 tx=0).stop 2023-11-26T07:16:43.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.317+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/869942982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb2815fc40 msgr2=0x7fbb281600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.317+0000 7fbb2ed2b700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb2815fc40 0x7fbb281600a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fbb20000f80 tx=0x7fbb20001ae0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.317+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/869942982 shutdown_connections 2023-11-26T07:16:43.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.318+0000 7fbb2ed2b700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbb0c06aff0 0x7fbb0c06d4b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.318+0000 7fbb2ed2b700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb2815fc40 0x7fbb281600a0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.318+0000 7fbb2ed2b700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb18004850 0x7fbb28148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.318+0000 7fbb2ed2b700 1 --2- 172.21.15.83:0/869942982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb280b4ff0 0x7fbb280b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.318+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/869942982 >> 172.21.15.83:0/869942982 conn(0x7fbb28071b90 msgr2=0x7fbb280b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:43.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.318+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/869942982 shutdown_connections 2023-11-26T07:16:43.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.318+0000 7fbb2ed2b700 1 -- 172.21.15.83:0/869942982 wait complete. 2023-11-26T07:16:43.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:16:43.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.450+0000 7f0f48a30700 1 Processor -- start 2023-11-26T07:16:43.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.450+0000 7f0f48a30700 1 -- start start 2023-11-26T07:16:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.451+0000 7f0f48a30700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0f440ce090 0x7f0f4406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.451+0000 7f0f48a30700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f44073f90 0x7f0f44074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.451+0000 7f0f48a30700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4406e7f0 con 0x7f0f4415e2a0 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.451+0000 7f0f48a30700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f44074960 con 0x7f0f44073f90 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.451+0000 7f0f48a30700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4406dec0 con 0x7f0f440ce090 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.451+0000 7f0f42d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f44073f90 0x7f0f44074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.451+0000 7f0f42d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f44073f90 0x7f0f44074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41202/0 (socket says 172.21.15.83:41202) 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.451+0000 7f0f4259c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f0f4415e2a0 0x7f0f4406ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33008/0 (socket says 172.21.15.83:33008) 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.451+0000 7f0f42d9d700 1 -- 172.21.15.83:0/3447101392 learned_addr learned my addr 172.21.15.83:0/3447101392 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.452+0000 7f0f42d9d700 1 -- 172.21.15.83:0/3447101392 >> v2:172.21.15.203:3300/0 conn(0x7f0f440ce090 msgr2=0x7f0f4406ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.452+0000 7f0f41d9b700 1 --2- 172.21.15.83:0/3447101392 >> v2:172.21.15.203:3300/0 conn(0x7f0f440ce090 0x7f0f4406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.452+0000 7f0f42d9d700 1 --2- 172.21.15.83:0/3447101392 >> v2:172.21.15.203:3300/0 conn(0x7f0f440ce090 0x7f0f4406ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.452+0000 7f0f42d9d700 1 -- 172.21.15.83:0/3447101392 >> v1:172.21.15.83:6789/0 conn(0x7f0f4415e2a0 legacy=0x7f0f4406ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.452+0000 7f0f4159a700 1 -- 172.21.15.83:0/3447101392 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705599146 0 0) 0x7f0f4406e7f0 con 0x7f0f4415e2a0 2023-11-26T07:16:43.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.452+0000 7f0f42d9d700 1 -- 172.21.15.83:0/3447101392 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4406e1a0 con 0x7f0f44073f90 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.452+0000 7f0f41d9b700 1 --2- 172.21.15.83:0/3447101392 >> v2:172.21.15.203:3300/0 conn(0x7f0f440ce090 0x7f0f4406ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.452+0000 7f0f42d9d700 1 --2- 172.21.15.83:0/3447101392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f44073f90 0x7f0f44074420 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0f44071840 tx=0x7f0f3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1abb9dd5bc32807 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.452+0000 7f0f4159a700 1 -- 172.21.15.83:0/3447101392 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00e070 con 0x7f0f44073f90 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.452+0000 7f0f4159a700 1 -- 172.21.15.83:0/3447101392 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c00b9c0 con 0x7f0f44073f90 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.452+0000 7f0f4159a700 1 -- 172.21.15.83:0/3447101392 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00bd70 con 0x7f0f44073f90 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.453+0000 7f0f48a30700 1 -- 172.21.15.83:0/3447101392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f44073f90 msgr2=0x7f0f44074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.453+0000 7f0f48a30700 1 --2- 172.21.15.83:0/3447101392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f44073f90 0x7f0f44074420 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0f44071840 tx=0x7f0f3c00b330 comp rx=0 tx=0).stop 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.453+0000 7f0f48a30700 1 -- 172.21.15.83:0/3447101392 shutdown_connections 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.453+0000 7f0f48a30700 1 --2- 172.21.15.83:0/3447101392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f44073f90 0x7f0f44074420 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.453+0000 7f0f48a30700 1 --2- 172.21.15.83:0/3447101392 >> v2:172.21.15.203:3300/0 conn(0x7f0f440ce090 0x7f0f4406ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.453+0000 7f0f48a30700 1 -- 172.21.15.83:0/3447101392 >> 172.21.15.83:0/3447101392 conn(0x7f0f440cd570 msgr2=0x7f0f440cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:43.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.453+0000 7f0f48a30700 1 -- 172.21.15.83:0/3447101392 shutdown_connections 2023-11-26T07:16:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.453+0000 7f0f48a30700 1 -- 172.21.15.83:0/3447101392 wait complete. 2023-11-26T07:16:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.454+0000 7f0f48a30700 1 Processor -- start 2023-11-26T07:16:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.454+0000 7f0f48a30700 1 -- start start 2023-11-26T07:16:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f48a30700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f44073f90 0x7f0f44146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f48a30700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f440ce090 0x7f0f44158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f48a30700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4415e2a0 0x7f0f441484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f48a30700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44158610 con 0x7f0f4415e2a0 2023-11-26T07:16:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f48a30700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44146d70 con 0x7f0f44073f90 2023-11-26T07:16:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f48a30700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4415f1a0 con 0x7f0f440ce090 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f42d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4415e2a0 0x7f0f441484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f42d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4415e2a0 0x7f0f441484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41210/0 (socket says 172.21.15.83:41210) 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f42d9d700 1 -- 172.21.15.83:0/3595513769 learned_addr learned my addr 172.21.15.83:0/3595513769 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f42d9d700 1 -- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f440ce090 msgr2=0x7f0f44158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f42d9d700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f440ce090 0x7f0f44158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f4259c700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f44073f90 0x7f0f44146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f42d9d700 1 -- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f44073f90 msgr2=0x7f0f44146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f42d9d700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f44073f90 0x7f0f44146490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f42d9d700 1 -- 172.21.15.83:0/3595513769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c009b30 con 0x7f0f4415e2a0 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f42d9d700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4415e2a0 0x7f0f441484f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0f44073510 tx=0x7f0f3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f32ffd700 1 -- 172.21.15.83:0/3595513769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c0099b0 con 0x7f0f4415e2a0 2023-11-26T07:16:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f32ffd700 1 -- 172.21.15.83:0/3595513769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c0092e0 con 0x7f0f4415e2a0 2023-11-26T07:16:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f32ffd700 1 -- 172.21.15.83:0/3595513769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c01c640 con 0x7f0f4415e2a0 2023-11-26T07:16:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.455+0000 7f0f48a30700 1 -- 172.21.15.83:0/3595513769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f44149090 con 0x7f0f4415e2a0 2023-11-26T07:16:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.456+0000 7f0f48a30700 1 -- 172.21.15.83:0/3595513769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f44158230 con 0x7f0f4415e2a0 2023-11-26T07:16:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.456+0000 7f0f48a30700 1 -- 172.21.15.83:0/3595513769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f440cf050 con 0x7f0f4415e2a0 2023-11-26T07:16:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.457+0000 7f0f32ffd700 1 -- 172.21.15.83:0/3595513769 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f0f3c01c7a0 con 0x7f0f4415e2a0 2023-11-26T07:16:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.457+0000 7f0f32ffd700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f2406aff0 0x7f0f2406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.457+0000 7f0f32ffd700 1 -- 172.21.15.83:0/3595513769 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0f3c08b5d0 con 0x7f0f4415e2a0 2023-11-26T07:16:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.458+0000 7f0f4259c700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f2406aff0 0x7f0f2406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.458+0000 7f0f4259c700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f2406aff0 0x7f0f2406d4b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f38000f60 tx=0x7f0f38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.462+0000 7f0f32ffd700 1 -- 172.21.15.83:0/3595513769 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f3c056490 con 0x7f0f4415e2a0 2023-11-26T07:16:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.696+0000 7f0f48a30700 1 -- 172.21.15.83:0/3595513769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0f44158750 con 0x7f0f4415e2a0 2023-11-26T07:16:43.699 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-26T07:16:43.697+0000 7fcd25823700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:16:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.698+0000 7f0f32ffd700 1 -- 172.21.15.83:0/3595513769 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0f44158750 con 0x7f0f4415e2a0 2023-11-26T07:16:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 -- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f2406aff0 msgr2=0x7f0f2406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f2406aff0 0x7f0f2406d4b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f38000f60 tx=0x7f0f38009040 comp rx=0 tx=0).stop 2023-11-26T07:16:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 -- 172.21.15.83:0/3595513769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4415e2a0 msgr2=0x7f0f441484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4415e2a0 0x7f0f441484f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0f44073510 tx=0x7f0f3c004ad0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 -- 172.21.15.83:0/3595513769 shutdown_connections 2023-11-26T07:16:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f2406aff0 0x7f0f2406d4b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f4415e2a0 0x7f0f441484f0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f44073f90 0x7f0f44146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 --2- 172.21.15.83:0/3595513769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f440ce090 0x7f0f44158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 -- 172.21.15.83:0/3595513769 >> 172.21.15.83:0/3595513769 conn(0x7f0f440cd570 msgr2=0x7f0f440cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 -- 172.21.15.83:0/3595513769 shutdown_connections 2023-11-26T07:16:43.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.701+0000 7f0f48a30700 1 -- 172.21.15.83:0/3595513769 wait complete. 2023-11-26T07:16:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:16:43.705 INFO:tasks.ceph.mds.b.smithi203.stderr: -3588> 2023-11-26T07:16:23.565+0000 7f2153425700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:16:43.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:16:43.737 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-11-26T07:16:43.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.835+0000 7f9d6758f700 1 Processor -- start 2023-11-26T07:16:43.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.836+0000 7f9d6758f700 1 -- start start 2023-11-26T07:16:43.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.836+0000 7f9d6758f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d6015b370 0x7f9d6015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.836+0000 7f9d6758f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d600b3c60 0x7f9d600b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.836+0000 7f9d6758f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d600ae5e0 con 0x7f9d601587f0 2023-11-26T07:16:43.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.836+0000 7f9d6758f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d600b4610 con 0x7f9d600b3c60 2023-11-26T07:16:43.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.836+0000 7f9d6758f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6015be30 con 0x7f9d6015b370 2023-11-26T07:16:43.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d65b2c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d600b3c60 0x7f9d600b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d65b2c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9d600b3c60 0x7f9d600b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41218/0 (socket says 172.21.15.83:41218) 2023-11-26T07:16:43.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d65b2c700 1 -- 172.21.15.83:0/2472994108 learned_addr learned my addr 172.21.15.83:0/2472994108 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:43.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d64b2a700 1 --2- 172.21.15.83:0/2472994108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d6015b370 0x7f9d6015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d65b2c700 1 -- 172.21.15.83:0/2472994108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d6015b370 msgr2=0x7f9d6015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:43.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d65b2c700 1 --2- 172.21.15.83:0/2472994108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d6015b370 0x7f9d6015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d65b2c700 1 -- 172.21.15.83:0/2472994108 >> v1:172.21.15.203:6789/0 conn(0x7f9d601587f0 legacy=0x7f9d6006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d65b2c700 1 -- 172.21.15.83:0/2472994108 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6015c110 con 0x7f9d600b3c60 2023-11-26T07:16:43.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d65b2c700 1 --2- 172.21.15.83:0/2472994108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d600b3c60 0x7f9d600b40d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800d150 tx=0x7f9d5800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9bbd97c30cb49b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:43.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d57fff700 1 -- 172.21.15.83:0/2472994108 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58010070 con 0x7f9d600b3c60 2023-11-26T07:16:43.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d57fff700 1 -- 172.21.15.83:0/2472994108 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5800dd10 con 0x7f9d600b3c60 2023-11-26T07:16:43.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.837+0000 7f9d57fff700 1 -- 172.21.15.83:0/2472994108 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800b310 con 0x7f9d600b3c60 2023-11-26T07:16:43.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.838+0000 7f9d6758f700 1 -- 172.21.15.83:0/2472994108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d600b3c60 msgr2=0x7f9d600b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.838+0000 7f9d6758f700 1 --2- 172.21.15.83:0/2472994108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d600b3c60 0x7f9d600b40d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800d150 tx=0x7f9d5800d460 comp rx=0 tx=0).stop 2023-11-26T07:16:43.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.838+0000 7f9d6758f700 1 -- 172.21.15.83:0/2472994108 shutdown_connections 2023-11-26T07:16:43.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.838+0000 7f9d6758f700 1 --2- 172.21.15.83:0/2472994108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d600b3c60 0x7f9d600b40d0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.838+0000 7f9d6758f700 1 --2- 172.21.15.83:0/2472994108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d6015b370 0x7f9d6015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.838+0000 7f9d6758f700 1 -- 172.21.15.83:0/2472994108 >> 172.21.15.83:0/2472994108 conn(0x7f9d600d0bf0 msgr2=0x7f9d600d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:43.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.838+0000 7f9d6758f700 1 -- 172.21.15.83:0/2472994108 shutdown_connections 2023-11-26T07:16:43.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.838+0000 7f9d6758f700 1 -- 172.21.15.83:0/2472994108 wait complete. 2023-11-26T07:16:43.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.839+0000 7f9d6758f700 1 Processor -- start 2023-11-26T07:16:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.839+0000 7f9d6758f700 1 -- start start 2023-11-26T07:16:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d6758f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d600b3c60 0x7f9d601f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d6758f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d601587f0 0x7f9d60154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d6758f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d6015b370 0x7f9d601f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d6758f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d601f47c0 con 0x7f9d601587f0 2023-11-26T07:16:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d6758f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6009b740 con 0x7f9d6015b370 2023-11-26T07:16:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d6758f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6009d1b0 con 0x7f9d600b3c60 2023-11-26T07:16:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d64b2a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d601587f0 0x7f9d60154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d64b2a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d601587f0 0x7f9d60154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41222/0 (socket says 172.21.15.83:41222) 2023-11-26T07:16:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d64b2a700 1 -- 172.21.15.83:0/3710979039 learned_addr learned my addr 172.21.15.83:0/3710979039 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d6532b700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d600b3c60 0x7f9d601f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d65b2c700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d6015b370 0x7f9d601f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d64b2a700 1 -- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d600b3c60 msgr2=0x7f9d601f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d64b2a700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d600b3c60 0x7f9d601f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d64b2a700 1 -- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d6015b370 msgr2=0x7f9d601f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d64b2a700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d6015b370 0x7f9d601f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d64b2a700 1 -- 172.21.15.83:0/3710979039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5800bd40 con 0x7f9d601587f0 2023-11-26T07:16:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d64b2a700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d601587f0 0x7f9d60154380 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00e870 tx=0x7f9d5c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d55ffb700 1 -- 172.21.15.83:0/3710979039 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c009590 con 0x7f9d601587f0 2023-11-26T07:16:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d55ffb700 1 -- 172.21.15.83:0/3710979039 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c013680 con 0x7f9d601587f0 2023-11-26T07:16:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.840+0000 7f9d55ffb700 1 -- 172.21.15.83:0/3710979039 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0139f0 con 0x7f9d601587f0 2023-11-26T07:16:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.841+0000 7f9d6758f700 1 -- 172.21.15.83:0/3710979039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6009d490 con 0x7f9d601587f0 2023-11-26T07:16:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.841+0000 7f9d6758f700 1 -- 172.21.15.83:0/3710979039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6009b410 con 0x7f9d601587f0 2023-11-26T07:16:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.842+0000 7f9d6758f700 1 -- 172.21.15.83:0/3710979039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6009d010 con 0x7f9d601587f0 2023-11-26T07:16:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.842+0000 7f9d55ffb700 1 -- 172.21.15.83:0/3710979039 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f9d5c013b50 con 0x7f9d601587f0 2023-11-26T07:16:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.842+0000 7f9d55ffb700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9d4406aff0 0x7f9d4406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.843+0000 7f9d55ffb700 1 -- 172.21.15.83:0/3710979039 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f9d5c0895d0 con 0x7f9d601587f0 2023-11-26T07:16:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.843+0000 7f9d6532b700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9d4406aff0 0x7f9d4406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.843+0000 7f9d6532b700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9d4406aff0 0x7f9d4406d4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9d6009b190 tx=0x7f9d50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:43.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:43.847+0000 7f9d55ffb700 1 -- 172.21.15.83:0/3710979039 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d6009d010 con 0x7f9d601587f0 2023-11-26T07:16:44.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.086+0000 7f9d6758f700 1 -- 172.21.15.83:0/3710979039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9d6009d010 con 0x7f9d601587f0 2023-11-26T07:16:44.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.700+0000 7f9d55ffb700 1 -- 172.21.15.83:0/3710979039 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9d6009d010 con 0x7f9d601587f0 2023-11-26T07:16:44.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 -- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9d4406aff0 msgr2=0x7f9d4406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:44.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9d4406aff0 0x7f9d4406d4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9d6009b190 tx=0x7f9d50009040 comp rx=0 tx=0).stop 2023-11-26T07:16:44.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 -- 172.21.15.83:0/3710979039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d601587f0 msgr2=0x7f9d60154380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:44.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d601587f0 0x7f9d60154380 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00e870 tx=0x7f9d5c00eb80 comp rx=0 tx=0).stop 2023-11-26T07:16:44.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 -- 172.21.15.83:0/3710979039 shutdown_connections 2023-11-26T07:16:44.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9d4406aff0 0x7f9d4406d4b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:44.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d6015b370 0x7f9d601f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:44.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9d600b3c60 0x7f9d601f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:44.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 --2- 172.21.15.83:0/3710979039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d601587f0 0x7f9d60154380 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:44.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 -- 172.21.15.83:0/3710979039 >> 172.21.15.83:0/3710979039 conn(0x7f9d600d0bf0 msgr2=0x7f9d6006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:44.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 -- 172.21.15.83:0/3710979039 shutdown_connections 2023-11-26T07:16:44.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.702+0000 7f9d6758f700 1 -- 172.21.15.83:0/3710979039 wait complete. 2023-11-26T07:16:44.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:16:44.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.844+0000 7fa4a076e700 1 Processor -- start 2023-11-26T07:16:44.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.844+0000 7fa4a076e700 1 -- start start 2023-11-26T07:16:44.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.845+0000 7fa4a076e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa4980cf040 0x7fa49806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:44.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.845+0000 7fa4a076e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa4980702e0 0x7fa4980d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.845+0000 7fa4a076e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4980d01c0 con 0x7fa4980d1ff0 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.845+0000 7fa4a076e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa498070910 con 0x7fa4980702e0 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.845+0000 7fa4a076e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa498070bf0 con 0x7fa4980cf040 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.845+0000 7fa49dd09700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa4980702e0 0x7fa4980d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49dd09700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa4980702e0 0x7fa4980d1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41230/0 (socket says 172.21.15.83:41230) 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49dd09700 1 -- 172.21.15.83:0/4034657853 learned_addr learned my addr 172.21.15.83:0/4034657853 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49e50a700 1 --2- 172.21.15.83:0/4034657853 >> v2:172.21.15.203:3300/0 conn(0x7fa4980cf040 0x7fa49806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49dd09700 1 -- 172.21.15.83:0/4034657853 >> v2:172.21.15.203:3300/0 conn(0x7fa4980cf040 msgr2=0x7fa49806fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49dd09700 1 --2- 172.21.15.83:0/4034657853 >> v2:172.21.15.203:3300/0 conn(0x7fa4980cf040 0x7fa49806fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49dd09700 1 -- 172.21.15.83:0/4034657853 >> v1:172.21.15.203:6789/0 conn(0x7fa4980d1ff0 legacy=0x7fa49815f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49dd09700 1 -- 172.21.15.83:0/4034657853 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980aebb0 con 0x7fa4980702e0 2023-11-26T07:16:44.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49e50a700 1 --2- 172.21.15.83:0/4034657853 >> v2:172.21.15.203:3300/0 conn(0x7fa4980cf040 0x7fa49806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49dd09700 1 --2- 172.21.15.83:0/4034657853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4980702e0 0x7fa4980d1ab0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa4980cf690 tx=0x7fa494009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5a65fa7e547fec server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49d508700 1 -- 172.21.15.83:0/4034657853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494004230 con 0x7fa4980702e0 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49d508700 1 -- 172.21.15.83:0/4034657853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa494004390 con 0x7fa4980702e0 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.846+0000 7fa49d508700 1 -- 172.21.15.83:0/4034657853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4940036a0 con 0x7fa4980702e0 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.847+0000 7fa4a076e700 1 -- 172.21.15.83:0/4034657853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4980702e0 msgr2=0x7fa4980d1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.847+0000 7fa4a076e700 1 --2- 172.21.15.83:0/4034657853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4980702e0 0x7fa4980d1ab0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa4980cf690 tx=0x7fa494009bd0 comp rx=0 tx=0).stop 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.847+0000 7fa4a076e700 1 -- 172.21.15.83:0/4034657853 shutdown_connections 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.847+0000 7fa4a076e700 1 --2- 172.21.15.83:0/4034657853 >> v2:172.21.15.203:3300/0 conn(0x7fa4980cf040 0x7fa49806fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.847+0000 7fa4a076e700 1 --2- 172.21.15.83:0/4034657853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4980702e0 0x7fa4980d1ab0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.847+0000 7fa4a076e700 1 -- 172.21.15.83:0/4034657853 >> 172.21.15.83:0/4034657853 conn(0x7fa49809aa00 msgr2=0x7fa49809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.847+0000 7fa4a076e700 1 -- 172.21.15.83:0/4034657853 shutdown_connections 2023-11-26T07:16:44.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.847+0000 7fa4a076e700 1 -- 172.21.15.83:0/4034657853 wait complete. 2023-11-26T07:16:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.848+0000 7fa4a076e700 1 Processor -- start 2023-11-26T07:16:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.848+0000 7fa4a076e700 1 -- start start 2023-11-26T07:16:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa4a076e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4980702e0 0x7fa4980b6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa4a076e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4980cf040 0x7fa4980b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa4a076e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4980d1ff0 0x7fa4980c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa4a076e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49815d6c0 con 0x7fa4980cf040 2023-11-26T07:16:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa4a076e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980ca8b0 con 0x7fa4980d1ff0 2023-11-26T07:16:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa4a076e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980cab50 con 0x7fa4980702e0 2023-11-26T07:16:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa49dd09700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4980cf040 0x7fa4980b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:44.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa49dd09700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4980cf040 0x7fa4980b6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41236/0 (socket says 172.21.15.83:41236) 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa49dd09700 1 -- 172.21.15.83:0/2019417871 learned_addr learned my addr 172.21.15.83:0/2019417871 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa49dd09700 1 -- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4980702e0 msgr2=0x7fa4980b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa49dd09700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4980702e0 0x7fa4980b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa49dd09700 1 -- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4980d1ff0 msgr2=0x7fa4980c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa49dd09700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4980d1ff0 0x7fa4980c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa49dd09700 1 -- 172.21.15.83:0/2019417871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa494009550 con 0x7fa4980cf040 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa49dd09700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4980cf040 0x7fa4980b6d40 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa49806e5d0 tx=0x7fa494004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa48effd700 1 -- 172.21.15.83:0/2019417871 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494003d80 con 0x7fa4980cf040 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa48effd700 1 -- 172.21.15.83:0/2019417871 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49401f070 con 0x7fa4980cf040 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa4a076e700 1 -- 172.21.15.83:0/2019417871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4980cae30 con 0x7fa4980cf040 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa48effd700 1 -- 172.21.15.83:0/2019417871 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494011870 con 0x7fa4980cf040 2023-11-26T07:16:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.849+0000 7fa4a076e700 1 -- 172.21.15.83:0/2019417871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4980cb3f0 con 0x7fa4980cf040 2023-11-26T07:16:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.850+0000 7fa4a076e700 1 -- 172.21.15.83:0/2019417871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa498067f60 con 0x7fa4980cf040 2023-11-26T07:16:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.851+0000 7fa48effd700 1 -- 172.21.15.83:0/2019417871 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fa494021030 con 0x7fa4980cf040 2023-11-26T07:16:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.851+0000 7fa48effd700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa47c06aff0 0x7fa47c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.851+0000 7fa48effd700 1 -- 172.21.15.83:0/2019417871 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fa49408ae10 con 0x7fa4980cf040 2023-11-26T07:16:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.851+0000 7fa49e50a700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa47c06aff0 0x7fa47c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.852+0000 7fa49e50a700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa47c06aff0 0x7fa47c06d4b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa488007920 tx=0x7fa488008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:44.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:44.856+0000 7fa48effd700 1 -- 172.21.15.83:0/2019417871 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa494055d90 con 0x7fa4980cf040 2023-11-26T07:16:45.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.048+0000 7fa4a076e700 1 -- 172.21.15.83:0/2019417871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa4980c8cb0 con 0x7fa4980cf040 2023-11-26T07:16:45.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.868+0000 7fa48effd700 1 -- 172.21.15.83:0/2019417871 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa4940593b0 con 0x7fa4980cf040 2023-11-26T07:16:45.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 -- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa47c06aff0 msgr2=0x7fa47c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:45.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa47c06aff0 0x7fa47c06d4b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa488007920 tx=0x7fa488008040 comp rx=0 tx=0).stop 2023-11-26T07:16:45.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 -- 172.21.15.83:0/2019417871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4980cf040 msgr2=0x7fa4980b6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:45.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4980cf040 0x7fa4980b6d40 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa49806e5d0 tx=0x7fa494004750 comp rx=0 tx=0).stop 2023-11-26T07:16:45.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 -- 172.21.15.83:0/2019417871 shutdown_connections 2023-11-26T07:16:45.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa47c06aff0 0x7fa47c06d4b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:45.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4980d1ff0 0x7fa4980c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:45.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4980702e0 0x7fa4980b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:45.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 --2- 172.21.15.83:0/2019417871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4980cf040 0x7fa4980b6d40 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:45.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 -- 172.21.15.83:0/2019417871 >> 172.21.15.83:0/2019417871 conn(0x7fa49809aa00 msgr2=0x7fa498061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:45.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 -- 172.21.15.83:0/2019417871 shutdown_connections 2023-11-26T07:16:45.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:45.871+0000 7fa4a076e700 1 -- 172.21.15.83:0/2019417871 wait complete. 2023-11-26T07:16:45.875 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:16:45.890 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:16:46.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.017+0000 7efc0c006700 1 Processor -- start 2023-11-26T07:16:46.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.017+0000 7efc0c006700 1 -- start start 2023-11-26T07:16:46.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.018+0000 7efc0c006700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efc0406b200 0x7efc0415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:46.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.018+0000 7efc0c006700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc040cfe00 con 0x7efc040d1890 2023-11-26T07:16:46.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.018+0000 7efc0c006700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc0406b700 con 0x7efc0415df10 2023-11-26T07:16:46.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.018+0000 7efc0c006700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc0406c380 con 0x7efc0406b200 2023-11-26T07:16:46.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.018+0000 7efc0a5a3700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efc040d1890 0x7efc040d0270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33054/0 (socket says 172.21.15.83:33054) 2023-11-26T07:16:46.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.018+0000 7efc0a5a3700 1 -- 172.21.15.83:0/969510189 learned_addr learned my addr 172.21.15.83:0/969510189 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:46.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.018+0000 7efc095a1700 1 --2- 172.21.15.83:0/969510189 >> v2:172.21.15.203:3300/0 conn(0x7efc0406b200 0x7efc0415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:46.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.018+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837197995 0 0) 0x7efc040cfe00 con 0x7efc040d1890 2023-11-26T07:16:46.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.018+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbf00035b0 con 0x7efc040d1890 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572738701 0 0) 0x7efbf00035b0 con 0x7efc040d1890 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 >> v2:172.21.15.203:3300/0 conn(0x7efc0406b200 msgr2=0x7efc0415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 --2- 172.21.15.83:0/969510189 >> v2:172.21.15.203:3300/0 conn(0x7efc0406b200 0x7efc0415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 >> v1:172.21.15.203:6789/0 conn(0x7efc0415df10 legacy=0x7efc040cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0406b9e0 con 0x7efc040d1890 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7efbfc0019f0 con 0x7efc040d1890 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 >> v1:172.21.15.83:6789/0 conn(0x7efc040d1890 legacy=0x7efc040d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf0003a00 0x7efbf0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efbf0004850 0x7efbf0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efbf0005210 0x7efbf000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf000c180 con 0x7efbf0003a00 2023-11-26T07:16:46.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf000c2f0 con 0x7efbf0005210 2023-11-26T07:16:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf000c4b0 con 0x7efbf0004850 2023-11-26T07:16:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc09da2700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf0003a00 0x7efbf0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc0a5a3700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efbf0005210 0x7efbf000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc095a1700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efbf0004850 0x7efbf0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc09da2700 1 -- 172.21.15.83:0/969510189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efbf0004850 msgr2=0x7efbf0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc09da2700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efbf0004850 0x7efbf0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc09da2700 1 -- 172.21.15.83:0/969510189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efbf0005210 msgr2=0x7efbf000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc09da2700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efbf0005210 0x7efbf000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc09da2700 1 -- 172.21.15.83:0/969510189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf00035b0 con 0x7efbf0003a00 2023-11-26T07:16:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc0a5a3700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efbf0005210 0x7efbf000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:46.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc0a5a3700 1 -- 172.21.15.83:0/969510189 reap_dead start 2023-11-26T07:16:46.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc09da2700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf0003a00 0x7efbf0003e20 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7efbf40017c0 tx=0x7efbf0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99663211a369e56 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:46.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40052d0 con 0x7efbf0003a00 2023-11-26T07:16:46.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf4005430 con 0x7efbf0003a00 2023-11-26T07:16:46.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.019+0000 7efc08da0700 1 -- 172.21.15.83:0/969510189 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf40038c0 con 0x7efbf0003a00 2023-11-26T07:16:46.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.020+0000 7efc0c006700 1 -- 172.21.15.83:0/969510189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf0003a00 msgr2=0x7efbf0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:46.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.020+0000 7efc0c006700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf0003a00 0x7efbf0003e20 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7efbf40017c0 tx=0x7efbf0005880 comp rx=0 tx=0).stop 2023-11-26T07:16:46.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.020+0000 7efc0c006700 1 -- 172.21.15.83:0/969510189 shutdown_connections 2023-11-26T07:16:46.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.020+0000 7efc0c006700 1 --2- 172.21.15.83:0/969510189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efbf0003a00 0x7efbf0003e20 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:46.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.020+0000 7efc0c006700 1 -- 172.21.15.83:0/969510189 >> 172.21.15.83:0/969510189 conn(0x7efc0409aa00 msgr2=0x7efc0409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:46.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.020+0000 7efc0c006700 1 -- 172.21.15.83:0/969510189 shutdown_connections 2023-11-26T07:16:46.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.020+0000 7efc0c006700 1 -- 172.21.15.83:0/969510189 wait complete. 2023-11-26T07:16:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.021+0000 7efc0c006700 1 Processor -- start 2023-11-26T07:16:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.022+0000 7efc0c006700 1 -- start start 2023-11-26T07:16:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.022+0000 7efc0c006700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efbf0003a00 0x7efc041f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.022+0000 7efc0c006700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04149d60 0x7efc0414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.022+0000 7efc0c006700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0414c310 0x7efc041553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:46.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.022+0000 7efc0c006700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc04154f10 con 0x7efc0414c310 2023-11-26T07:16:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.022+0000 7efc0c006700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0414c7c0 con 0x7efc04149d60 2023-11-26T07:16:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.022+0000 7efc0c006700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0415c680 con 0x7efbf0003a00 2023-11-26T07:16:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc0a5a3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0414c310 0x7efc041553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc0a5a3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0414c310 0x7efc041553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41258/0 (socket says 172.21.15.83:41258) 2023-11-26T07:16:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc0a5a3700 1 -- 172.21.15.83:0/4097062450 learned_addr learned my addr 172.21.15.83:0/4097062450 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc09da2700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efbf0003a00 0x7efc041f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc095a1700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04149d60 0x7efc0414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc0a5a3700 1 -- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efbf0003a00 msgr2=0x7efc041f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc0a5a3700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efbf0003a00 0x7efc041f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc0a5a3700 1 -- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04149d60 msgr2=0x7efc0414bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc0a5a3700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04149d60 0x7efc0414bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc0a5a3700 1 -- 172.21.15.83:0/4097062450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc041545e0 con 0x7efc0414c310 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc09da2700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efbf0003a00 0x7efc041f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efc0a5a3700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0414c310 0x7efc041553a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7efbfc002010 tx=0x7efbfc007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.023+0000 7efbfa7fc700 1 -- 172.21.15.83:0/4097062450 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc00ec60 con 0x7efc0414c310 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.024+0000 7efbfa7fc700 1 -- 172.21.15.83:0/4097062450 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc00edc0 con 0x7efc0414c310 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.024+0000 7efc0c006700 1 -- 172.21.15.83:0/4097062450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc041548c0 con 0x7efc0414c310 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.024+0000 7efbfa7fc700 1 -- 172.21.15.83:0/4097062450 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc013660 con 0x7efc0414c310 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.024+0000 7efc0c006700 1 -- 172.21.15.83:0/4097062450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc040b4ce0 con 0x7efc0414c310 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.025+0000 7efc0c006700 1 -- 172.21.15.83:0/4097062450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc040751e0 con 0x7efc0414c310 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.026+0000 7efbfa7fc700 1 -- 172.21.15.83:0/4097062450 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7efbfc0137c0 con 0x7efc0414c310 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.026+0000 7efbfa7fc700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efbe806b0c0 0x7efbe806d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.026+0000 7efbfa7fc700 1 -- 172.21.15.83:0/4097062450 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5443+0+0 (secure 0 0 0) 0x7efbfc089ca0 con 0x7efc0414c310 2023-11-26T07:16:46.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.027+0000 7efc09da2700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efbe806b0c0 0x7efbe806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.027+0000 7efc09da2700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efbe806b0c0 0x7efbe806d580 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005040 tx=0x7efbf4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:46.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.031+0000 7efbfa7fc700 1 -- 172.21.15.83:0/4097062450 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efbfc054db0 con 0x7efc0414c310 2023-11-26T07:16:46.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.220+0000 7efc0c006700 1 -- 172.21.15.83:0/4097062450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efc04155050 con 0x7efc0414c310 2023-11-26T07:16:46.877 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.874+0000 7efbfa7fc700 1 -- 172.21.15.83:0/4097062450 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) v1 ==== 169+0+0 (secure 0 0 0) 0x7efc04155050 con 0x7efc0414c310 2023-11-26T07:16:46.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 -- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efbe806b0c0 msgr2=0x7efbe806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:46.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efbe806b0c0 0x7efbe806d580 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005040 tx=0x7efbf4009030 comp rx=0 tx=0).stop 2023-11-26T07:16:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 -- 172.21.15.83:0/4097062450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0414c310 msgr2=0x7efc041553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0414c310 0x7efc041553a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7efbfc002010 tx=0x7efbfc007c90 comp rx=0 tx=0).stop 2023-11-26T07:16:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 -- 172.21.15.83:0/4097062450 shutdown_connections 2023-11-26T07:16:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efbe806b0c0 0x7efbe806d580 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0414c310 0x7efc041553a0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efbf0003a00 0x7efc041f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 --2- 172.21.15.83:0/4097062450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04149d60 0x7efc0414bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:46.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 -- 172.21.15.83:0/4097062450 >> 172.21.15.83:0/4097062450 conn(0x7efc0409aa00 msgr2=0x7efc040d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:46.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 -- 172.21.15.83:0/4097062450 shutdown_connections 2023-11-26T07:16:46.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:46.879+0000 7efc0c006700 1 -- 172.21.15.83:0/4097062450 wait complete. 2023-11-26T07:16:46.883 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2023-11-26T07:16:46.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:16:47.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.024+0000 7f4ef4b9c700 1 Processor -- start 2023-11-26T07:16:47.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.024+0000 7f4ef4b9c700 1 -- start start 2023-11-26T07:16:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4ef4b9c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef015e390 0x7f4ef00b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4ef4b9c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4ef006f480 0x7f4ef006e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4ef4b9c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef00b5ea0 con 0x7f4ef0073f90 2023-11-26T07:16:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4ef4b9c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef00b4530 con 0x7f4ef006f480 2023-11-26T07:16:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4ef4b9c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00b3d00 con 0x7f4ef015e390 2023-11-26T07:16:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eeed9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4ef0073f90 0x7f4ef006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33074/0 (socket says 172.21.15.83:33074) 2023-11-26T07:16:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eeed9d700 1 -- 172.21.15.83:0/3972361043 learned_addr learned my addr 172.21.15.83:0/3972361043 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eee59c700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef015e390 0x7f4ef00b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eedd9b700 1 --2- 172.21.15.83:0/3972361043 >> v2:172.21.15.203:3300/0 conn(0x7f4ef006f480 0x7f4ef006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090850738 0 0) 0x7f4ef00b5ea0 con 0x7f4ef0073f90 2023-11-26T07:16:47.031 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed80035b0 con 0x7f4ef0073f90 2023-11-26T07:16:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 383863584 0 0) 0x7f4ed80035b0 con 0x7f4ef0073f90 2023-11-26T07:16:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef015e390 msgr2=0x7f4ef00b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eed59a700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef015e390 0x7f4ef00b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 >> v2:172.21.15.203:3300/0 conn(0x7f4ef006f480 msgr2=0x7f4ef006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eed59a700 1 --2- 172.21.15.83:0/3972361043 >> v2:172.21.15.203:3300/0 conn(0x7f4ef006f480 0x7f4ef006e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef00b3f70 con 0x7f4ef0073f90 2023-11-26T07:16:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.025+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f4ee80019f0 con 0x7f4ef0073f90 2023-11-26T07:16:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 >> v1:172.21.15.83:6789/0 conn(0x7f4ef0073f90 legacy=0x7f4ef006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eed59a700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8003a00 0x7f4ed8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eed59a700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed8004850 0x7f4ed8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eed59a700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed8005210 0x7f4ed800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00b4530 con 0x7f4ed8003a00 2023-11-26T07:16:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00b3d00 con 0x7f4ed8005210 2023-11-26T07:16:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed800bfa0 con 0x7f4ed8004850 2023-11-26T07:16:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eee59c700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef015e390 0x7f4ef00b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eedd9b700 1 --2- 172.21.15.83:0/3972361043 >> v2:172.21.15.203:3300/0 conn(0x7f4ef006f480 0x7f4ef006e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eee59c700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8003a00 0x7f4ed8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eeed9d700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed8005210 0x7f4ed800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eedd9b700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed8004850 0x7f4ed8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eee59c700 1 -- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed8004850 msgr2=0x7f4ed8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eee59c700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed8004850 0x7f4ed8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eee59c700 1 -- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed8005210 msgr2=0x7f4ed800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eee59c700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed8005210 0x7f4ed800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eee59c700 1 -- 172.21.15.83:0/3972361043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed80035b0 con 0x7f4ed8003a00 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eee59c700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8003a00 0x7f4ed8003e20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009ba0 tx=0x7f4ed8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fc8fc2a293d804 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0004030 con 0x7f4ed8003a00 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee0004190 con 0x7f4ed8003a00 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.026+0000 7f4eed59a700 1 -- 172.21.15.83:0/3972361043 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0004540 con 0x7f4ed8003a00 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.027+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/3972361043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8003a00 msgr2=0x7f4ed8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.027+0000 7f4ef4b9c700 1 --2- 172.21.15.83:0/3972361043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8003a00 0x7f4ed8003e20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009ba0 tx=0x7f4ed8005880 comp rx=0 tx=0).stop 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.027+0000 7f4eeed9d700 1 -- 172.21.15.83:0/3972361043 reap_dead start 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.027+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/3972361043 shutdown_connections 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.027+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/3972361043 >> 172.21.15.83:0/3972361043 conn(0x7f4ef00d0bf0 msgr2=0x7f4ef00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:47.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.027+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/3972361043 shutdown_connections 2023-11-26T07:16:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.027+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/3972361043 wait complete. 2023-11-26T07:16:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.028+0000 7f4ef4b9c700 1 Processor -- start 2023-11-26T07:16:47.035 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.028+0000 7f4ef4b9c700 1 -- start start 2023-11-26T07:16:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4ef4b9c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed8003a00 0x7f4ef0161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4ef4b9c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef01f4d20 0x7f4ef006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4ef4b9c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ef006de70 0x7f4ef0154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4ef4b9c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef006e320 con 0x7f4ef006de70 2023-11-26T07:16:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4ef4b9c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef01616c0 con 0x7f4ed8003a00 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4ef4b9c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef015f870 con 0x7f4ef01f4d20 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4eeed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ef006de70 0x7f4ef0154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4eeed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ef006de70 0x7f4ef0154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41278/0 (socket says 172.21.15.83:41278) 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4eeed9d700 1 -- 172.21.15.83:0/1511290282 learned_addr learned my addr 172.21.15.83:0/1511290282 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4eee59c700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed8003a00 0x7f4ef0161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4eedd9b700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef01f4d20 0x7f4ef006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4eeed9d700 1 -- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef01f4d20 msgr2=0x7f4ef006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4eeed9d700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef01f4d20 0x7f4ef006d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4eeed9d700 1 -- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed8003a00 msgr2=0x7f4ef0161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4eeed9d700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed8003a00 0x7f4ef0161180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.029+0000 7f4eeed9d700 1 -- 172.21.15.83:0/1511290282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef015fae0 con 0x7f4ef006de70 2023-11-26T07:16:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.030+0000 7f4eeed9d700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ef006de70 0x7f4ef0154a40 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0075050 tx=0x7f4ee800a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.030+0000 7f4edeffd700 1 -- 172.21.15.83:0/1511290282 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee800ad00 con 0x7f4ef006de70 2023-11-26T07:16:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.030+0000 7f4edeffd700 1 -- 172.21.15.83:0/1511290282 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee80133f0 con 0x7f4ef006de70 2023-11-26T07:16:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.030+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/1511290282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef015fdc0 con 0x7f4ef006de70 2023-11-26T07:16:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.030+0000 7f4edeffd700 1 -- 172.21.15.83:0/1511290282 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee800aea0 con 0x7f4ef006de70 2023-11-26T07:16:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.030+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/1511290282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef0160380 con 0x7f4ef006de70 2023-11-26T07:16:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.031+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/1511290282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef0160510 con 0x7f4ef006de70 2023-11-26T07:16:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.031+0000 7f4edeffd700 1 -- 172.21.15.83:0/1511290282 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f4ee80137a0 con 0x7f4ef006de70 2023-11-26T07:16:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.032+0000 7f4edeffd700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4ed006aff0 0x7f4ed006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.032+0000 7f4edeffd700 1 -- 172.21.15.83:0/1511290282 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f4ee8089ca0 con 0x7f4ef006de70 2023-11-26T07:16:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.032+0000 7f4eee59c700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4ed006aff0 0x7f4ed006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.032+0000 7f4eee59c700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4ed006aff0 0x7f4ed006d4b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00aeb20 tx=0x7f4ee0020040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.036+0000 7f4edeffd700 1 -- 172.21.15.83:0/1511290282 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ee8054ee0 con 0x7f4ef006de70 2023-11-26T07:16:47.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.225+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/1511290282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4ef0068fb0 con 0x7f4ef006de70 2023-11-26T07:16:47.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.228+0000 7f4edeffd700 1 -- 172.21.15.83:0/1511290282 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4ee8058500 con 0x7f4ef006de70 2023-11-26T07:16:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4ed006aff0 msgr2=0x7f4ed006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4ed006aff0 0x7f4ed006d4b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00aeb20 tx=0x7f4ee0020040 comp rx=0 tx=0).stop 2023-11-26T07:16:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/1511290282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ef006de70 msgr2=0x7f4ef0154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ef006de70 0x7f4ef0154a40 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0075050 tx=0x7f4ee800a640 comp rx=0 tx=0).stop 2023-11-26T07:16:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/1511290282 shutdown_connections 2023-11-26T07:16:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4ed006aff0 0x7f4ed006d4b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ef006de70 0x7f4ef0154a40 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed8003a00 0x7f4ef0161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 --2- 172.21.15.83:0/1511290282 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ef01f4d20 0x7f4ef006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/1511290282 >> 172.21.15.83:0/1511290282 conn(0x7f4ef00d0bf0 msgr2=0x7f4ef00af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:47.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/1511290282 shutdown_connections 2023-11-26T07:16:47.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.230+0000 7f4ef4b9c700 1 -- 172.21.15.83:0/1511290282 wait complete. 2023-11-26T07:16:47.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2023-11-26T07:16:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.362+0000 7f0d32c73700 1 Processor -- start 2023-11-26T07:16:47.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.362+0000 7f0d32c73700 1 -- start start 2023-11-26T07:16:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.363+0000 7f0d32c73700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 0x7f0d2c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.363+0000 7f0d32c73700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0d2c0d1c70 0x7f0d2c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.363+0000 7f0d32c73700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2c0b22d0 con 0x7f0d2c15bdd0 2023-11-26T07:16:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.363+0000 7f0d32c73700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b2f50 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.363+0000 7f0d32c73700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0d2640 con 0x7f0d2c0b3370 2023-11-26T07:16:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.363+0000 7f0d2bfff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 0x7f0d2c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.363+0000 7f0d31210700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0d2c0d1c70 0x7f0d2c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.363+0000 7f0d30a0f700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0d2c15bdd0 0x7f0d2c0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:45940/0 (socket says 172.21.15.83:45940) 2023-11-26T07:16:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.363+0000 7f0d30a0f700 1 -- 172.21.15.83:0/3832035324 learned_addr learned my addr 172.21.15.83:0/3832035324 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.363+0000 7f0d31210700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0d2c0d1c70 0x7f0d2c0d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40736/0 (socket says 172.21.15.83:40736) 2023-11-26T07:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.363+0000 7f0d2bfff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 0x7f0d2c1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:51742/0 (socket says 172.21.15.83:51742) 2023-11-26T07:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d2bfff700 1 -- 172.21.15.83:0/3832035324 >> v2:172.21.15.203:3300/0 conn(0x7f0d2c0d1c70 msgr2=0x7f0d2c0d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d2bfff700 1 --2- 172.21.15.83:0/3832035324 >> v2:172.21.15.203:3300/0 conn(0x7f0d2c0d1c70 0x7f0d2c0d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d2bfff700 1 -- 172.21.15.83:0/3832035324 >> v1:172.21.15.203:6789/0 conn(0x7f0d2c15bdd0 legacy=0x7f0d2c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d2bfff700 1 -- 172.21.15.83:0/3832035324 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c0b25b0 con 0x7f0d2c0b3370 2023-11-26T07:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d31210700 1 --2- 172.21.15.83:0/3832035324 >> v2:172.21.15.203:3300/0 conn(0x7f0d2c0d1c70 0x7f0d2c0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d2bfff700 1 --2- 172.21.15.83:0/3832035324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 0x7f0d2c1604e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009840 tx=0x7f0d18009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=282a86dedf4f8450 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d2b7fe700 1 -- 172.21.15.83:0/3832035324 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18011830 con 0x7f0d2c0b3370 2023-11-26T07:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d2b7fe700 1 -- 172.21.15.83:0/3832035324 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d18011990 con 0x7f0d2c0b3370 2023-11-26T07:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d2b7fe700 1 -- 172.21.15.83:0/3832035324 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18011d20 con 0x7f0d2c0b3370 2023-11-26T07:16:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d32c73700 1 -- 172.21.15.83:0/3832035324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 msgr2=0x7f0d2c1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d32c73700 1 --2- 172.21.15.83:0/3832035324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 0x7f0d2c1604e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009840 tx=0x7f0d18009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d32c73700 1 -- 172.21.15.83:0/3832035324 shutdown_connections 2023-11-26T07:16:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d32c73700 1 --2- 172.21.15.83:0/3832035324 >> v2:172.21.15.203:3300/0 conn(0x7f0d2c0d1c70 0x7f0d2c0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d32c73700 1 --2- 172.21.15.83:0/3832035324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 0x7f0d2c1604e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.364+0000 7f0d32c73700 1 -- 172.21.15.83:0/3832035324 >> 172.21.15.83:0/3832035324 conn(0x7f0d2c0731a0 msgr2=0x7f0d2c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.365+0000 7f0d32c73700 1 -- 172.21.15.83:0/3832035324 shutdown_connections 2023-11-26T07:16:47.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.365+0000 7f0d32c73700 1 -- 172.21.15.83:0/3832035324 wait complete. 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.366+0000 7f0d32c73700 1 Processor -- start 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.366+0000 7f0d32c73700 1 -- start start 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d32c73700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 0x7f0d2c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d32c73700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d2c0d1c70 0x7f0d2c15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d32c73700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d2c15bdd0 0x7f0d2c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d32c73700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c06cf60 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d32c73700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0cf370 con 0x7f0d2c15bdd0 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d32c73700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c1f83f0 con 0x7f0d2c0b3370 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d2bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d2c0d1c70 0x7f0d2c15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d2bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d2c0d1c70 0x7f0d2c15b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41290/0 (socket says 172.21.15.83:41290) 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d2bfff700 1 -- 172.21.15.83:0/2978457989 learned_addr learned my addr 172.21.15.83:0/2978457989 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d30a0f700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 0x7f0d2c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d31210700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d2c15bdd0 0x7f0d2c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d2bfff700 1 -- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 msgr2=0x7f0d2c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d2bfff700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 0x7f0d2c06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d2bfff700 1 -- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d2c15bdd0 msgr2=0x7f0d2c0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d2bfff700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d2c15bdd0 0x7f0d2c0cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d2bfff700 1 -- 172.21.15.83:0/2978457989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d24009a70 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d2bfff700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d2c0d1c70 0x7f0d2c15b180 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009fd0 tx=0x7f0d18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d297fa700 1 -- 172.21.15.83:0/2978457989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d180045d0 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d297fa700 1 -- 172.21.15.83:0/2978457989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1801f070 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d32c73700 1 -- 172.21.15.83:0/2978457989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d180094a0 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.367+0000 7f0d297fa700 1 -- 172.21.15.83:0/2978457989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d180038e0 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.368+0000 7f0d32c73700 1 -- 172.21.15.83:0/2978457989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2c1f7910 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.368+0000 7f0d32c73700 1 -- 172.21.15.83:0/2978457989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c0cd800 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.369+0000 7f0d297fa700 1 -- 172.21.15.83:0/2978457989 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f0d18003a40 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.369+0000 7f0d297fa700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0d1006aff0 0x7f0d1006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.369+0000 7f0d297fa700 1 -- 172.21.15.83:0/2978457989 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0d18021070 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.369+0000 7f0d30a0f700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0d1006aff0 0x7f0d1006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:47.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.370+0000 7f0d30a0f700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0d1006aff0 0x7f0d1006d4b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d20005910 tx=0x7f0d20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:47.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.374+0000 7f0d297fa700 1 -- 172.21.15.83:0/2978457989 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d18055ec0 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.645+0000 7f0d32c73700 1 -- 172.21.15.83:0/2978457989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7f0d2c067f60 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.876+0000 7f0d297fa700 1 -- 172.21.15.83:0/2978457989 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v294) v1 ==== 146+0+0 (secure 0 0 0) 0x7f0d1802b0d0 con 0x7f0d2c0d1c70 2023-11-26T07:16:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 -- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0d1006aff0 msgr2=0x7f0d1006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0d1006aff0 0x7f0d1006d4b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d20005910 tx=0x7f0d20009040 comp rx=0 tx=0).stop 2023-11-26T07:16:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 -- 172.21.15.83:0/2978457989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d2c0d1c70 msgr2=0x7f0d2c15b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d2c0d1c70 0x7f0d2c15b180 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009fd0 tx=0x7f0d18004750 comp rx=0 tx=0).stop 2023-11-26T07:16:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 -- 172.21.15.83:0/2978457989 shutdown_connections 2023-11-26T07:16:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0d1006aff0 0x7f0d1006d4b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d2c15bdd0 0x7f0d2c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d2c0b3370 0x7f0d2c06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 --2- 172.21.15.83:0/2978457989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d2c0d1c70 0x7f0d2c15b180 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:47.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 -- 172.21.15.83:0/2978457989 >> 172.21.15.83:0/2978457989 conn(0x7f0d2c0731a0 msgr2=0x7f0d2c0aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:47.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 -- 172.21.15.83:0/2978457989 shutdown_connections 2023-11-26T07:16:47.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:47.878+0000 7f0d32c73700 1 -- 172.21.15.83:0/2978457989 wait complete. 2023-11-26T07:16:47.893 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2023-11-26T07:16:47.893 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2023-11-26T07:16:47.894 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:16:47.894 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:48.018 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.016+0000 7f20395e6700 1 Processor -- start 2023-11-26T07:16:48.019 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.016+0000 7f20395e6700 1 -- start start 2023-11-26T07:16:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20395e6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f203415e1e0 0x7f20340d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20395e6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20340d3a00 0x7f20340d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20395e6700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20340d20b0 con 0x7f20340d0db0 2023-11-26T07:16:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20395e6700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20340d3150 con 0x7f203415e1e0 2023-11-26T07:16:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20395e6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20340d1240 con 0x7f20340d3a00 2023-11-26T07:16:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f2032ffd700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f203415e1e0 0x7f20340d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20327fc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20340d3a00 0x7f20340d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20337fe700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f20340d0db0 0x7f203406aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:45958/0 (socket says 172.21.15.83:45958) 2023-11-26T07:16:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20337fe700 1 -- 172.21.15.83:0/3164674060 learned_addr learned my addr 172.21.15.83:0/3164674060 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20327fc700 1 -- 172.21.15.83:0/3164674060 >> v2:172.21.15.203:3300/0 conn(0x7f203415e1e0 msgr2=0x7f20340d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20327fc700 1 --2- 172.21.15.83:0/3164674060 >> v2:172.21.15.203:3300/0 conn(0x7f203415e1e0 0x7f20340d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20327fc700 1 -- 172.21.15.83:0/3164674060 >> v1:172.21.15.203:6789/0 conn(0x7f20340d0db0 legacy=0x7f203406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f20327fc700 1 -- 172.21.15.83:0/3164674060 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20340d2390 con 0x7f20340d3a00 2023-11-26T07:16:48.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.017+0000 7f2032ffd700 1 --2- 172.21.15.83:0/3164674060 >> v2:172.21.15.203:3300/0 conn(0x7f203415e1e0 0x7f20340d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:16:48.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f20327fc700 1 --2- 172.21.15.83:0/3164674060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20340d3a00 0x7f20340d0870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2024009840 tx=0x7f2024009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945ac312d4118036 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:48.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f2031ffb700 1 -- 172.21.15.83:0/3164674060 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024011830 con 0x7f20340d3a00 2023-11-26T07:16:48.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f2031ffb700 1 -- 172.21.15.83:0/3164674060 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2024011990 con 0x7f20340d3a00 2023-11-26T07:16:48.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f2031ffb700 1 -- 172.21.15.83:0/3164674060 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2024011d20 con 0x7f20340d3a00 2023-11-26T07:16:48.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f20395e6700 1 -- 172.21.15.83:0/3164674060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20340d3a00 msgr2=0x7f20340d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f20395e6700 1 --2- 172.21.15.83:0/3164674060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20340d3a00 0x7f20340d0870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2024009840 tx=0x7f2024009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f20395e6700 1 -- 172.21.15.83:0/3164674060 shutdown_connections 2023-11-26T07:16:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f20395e6700 1 --2- 172.21.15.83:0/3164674060 >> v2:172.21.15.203:3300/0 conn(0x7f203415e1e0 0x7f20340d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f20395e6700 1 --2- 172.21.15.83:0/3164674060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20340d3a00 0x7f20340d0870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f20395e6700 1 -- 172.21.15.83:0/3164674060 >> 172.21.15.83:0/3164674060 conn(0x7f20340b5b50 msgr2=0x7f20340b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f20395e6700 1 -- 172.21.15.83:0/3164674060 shutdown_connections 2023-11-26T07:16:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.018+0000 7f20395e6700 1 -- 172.21.15.83:0/3164674060 wait complete. 2023-11-26T07:16:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.020+0000 7f20395e6700 1 Processor -- start 2023-11-26T07:16:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.020+0000 7f20395e6700 1 -- start start 2023-11-26T07:16:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20395e6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20340d0db0 0x7f20341f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20395e6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20340d3a00 0x7f20340caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.025 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20395e6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203415e1e0 0x7f203415c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20395e6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20340cb470 con 0x7f203415e1e0 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20395e6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20340cb710 con 0x7f20340d0db0 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20395e6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20340cb9f0 con 0x7f20340d3a00 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20337fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203415e1e0 0x7f203415c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20337fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203415e1e0 0x7f203415c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41306/0 (socket says 172.21.15.83:41306) 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20337fe700 1 -- 172.21.15.83:0/3085730802 learned_addr learned my addr 172.21.15.83:0/3085730802 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f2032ffd700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20340d0db0 0x7f20341f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20327fc700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20340d3a00 0x7f20340caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20337fe700 1 -- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20340d3a00 msgr2=0x7f20340caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20337fe700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20340d3a00 0x7f20340caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20337fe700 1 -- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20340d0db0 msgr2=0x7f20341f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20337fe700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20340d0db0 0x7f20341f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20337fe700 1 -- 172.21.15.83:0/3085730802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280094a0 con 0x7f203415e1e0 2023-11-26T07:16:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20337fe700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203415e1e0 0x7f203415c380 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f202c00cbc0 tx=0x7f202c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f201b7fe700 1 -- 172.21.15.83:0/3085730802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00a9e0 con 0x7f203415e1e0 2023-11-26T07:16:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f201b7fe700 1 -- 172.21.15.83:0/3085730802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c00ab40 con 0x7f203415e1e0 2023-11-26T07:16:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f20395e6700 1 -- 172.21.15.83:0/3085730802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20240094a0 con 0x7f203415e1e0 2023-11-26T07:16:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.021+0000 7f201b7fe700 1 -- 172.21.15.83:0/3085730802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c012400 con 0x7f203415e1e0 2023-11-26T07:16:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.022+0000 7f20395e6700 1 -- 172.21.15.83:0/3085730802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20340cbfb0 con 0x7f203415e1e0 2023-11-26T07:16:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.023+0000 7f20395e6700 1 -- 172.21.15.83:0/3085730802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20340cc600 con 0x7f203415e1e0 2023-11-26T07:16:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.023+0000 7f201b7fe700 1 -- 172.21.15.83:0/3085730802 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f202c01b3b0 con 0x7f203415e1e0 2023-11-26T07:16:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.023+0000 7f201b7fe700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f202006b020 0x7f202006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.023+0000 7f201b7fe700 1 -- 172.21.15.83:0/3085730802 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f202c088f70 con 0x7f203415e1e0 2023-11-26T07:16:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.023+0000 7f2032ffd700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f202006b020 0x7f202006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.024+0000 7f2032ffd700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f202006b020 0x7f202006d4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20340691b0 tx=0x7f2028009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:48.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.028+0000 7f201b7fe700 1 -- 172.21.15.83:0/3085730802 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f202c0bbe90 con 0x7f203415e1e0 2023-11-26T07:16:48.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.217+0000 7f20395e6700 1 -- 172.21.15.83:0/3085730802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20340751e0 con 0x7f203415e1e0 2023-11-26T07:16:48.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.218+0000 7f201b7fe700 1 -- 172.21.15.83:0/3085730802 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+13105 (secure 0 0 0) 0x7f202c054260 con 0x7f203415e1e0 2023-11-26T07:16:48.220 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:48.220 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":93,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:46.870249+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:6835/1901175330":"2023-11-27T07:16:43.698215+0000","172.21.15.203:6833/1901175330":"2023-11-27T07:16:43.698215+0000","172.21.15.83:6835/469139444":"2023-11-27T07:16:32.809518+0000","172.21.15.83:6833/469139444":"2023-11-27T07:16:32.809518+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:48.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 -- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f202006b020 msgr2=0x7f202006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f202006b020 0x7f202006d4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20340691b0 tx=0x7f2028009210 comp rx=0 tx=0).stop 2023-11-26T07:16:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 -- 172.21.15.83:0/3085730802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203415e1e0 msgr2=0x7f203415c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203415e1e0 0x7f203415c380 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f202c00cbc0 tx=0x7f202c00ced0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 -- 172.21.15.83:0/3085730802 shutdown_connections 2023-11-26T07:16:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f202006b020 0x7f202006d4e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f203415e1e0 0x7f203415c380 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20340d0db0 0x7f20341f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 --2- 172.21.15.83:0/3085730802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20340d3a00 0x7f20340caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 -- 172.21.15.83:0/3085730802 >> 172.21.15.83:0/3085730802 conn(0x7f20340b5b50 msgr2=0x7f20340b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 -- 172.21.15.83:0/3085730802 shutdown_connections 2023-11-26T07:16:48.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.220+0000 7f20395e6700 1 -- 172.21.15.83:0/3085730802 wait complete. 2023-11-26T07:16:48.235 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:16:48.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:16:48.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.355+0000 7f1e81947700 1 Processor -- start 2023-11-26T07:16:48.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.355+0000 7f1e81947700 1 -- start start 2023-11-26T07:16:48.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.356+0000 7f1e81947700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1e7c0afb40 0x7f1e7c0cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.356+0000 7f1e81947700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e7c0cfa80 0x7f1e7c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.356+0000 7f1e81947700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7c0cccd0 con 0x7f1e7c15ada0 2023-11-26T07:16:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.356+0000 7f1e81947700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7c0cd2e0 con 0x7f1e7c0cfa80 2023-11-26T07:16:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.356+0000 7f1e81947700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7c0cd5c0 con 0x7f1e7c0afb40 2023-11-26T07:16:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.356+0000 7f1e7b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e7c0cfa80 0x7f1e7c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.356+0000 7f1e7b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1e7c0cfa80 0x7f1e7c06d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41314/0 (socket says 172.21.15.83:41314) 2023-11-26T07:16:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.356+0000 7f1e7b7fe700 1 -- 172.21.15.83:0/2996070687 learned_addr learned my addr 172.21.15.83:0/2996070687 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.356+0000 7f1e7a7fc700 1 --2- 172.21.15.83:0/2996070687 >> v2:172.21.15.203:3300/0 conn(0x7f1e7c0afb40 0x7f1e7c0cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e7b7fe700 1 -- 172.21.15.83:0/2996070687 >> v2:172.21.15.203:3300/0 conn(0x7f1e7c0afb40 msgr2=0x7f1e7c0cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e7b7fe700 1 --2- 172.21.15.83:0/2996070687 >> v2:172.21.15.203:3300/0 conn(0x7f1e7c0afb40 0x7f1e7c0cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e7b7fe700 1 -- 172.21.15.83:0/2996070687 >> v1:172.21.15.203:6789/0 conn(0x7f1e7c15ada0 legacy=0x7f1e7c06f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e7b7fe700 1 -- 172.21.15.83:0/2996070687 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c062440 con 0x7f1e7c0cfa80 2023-11-26T07:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e7b7fe700 1 --2- 172.21.15.83:0/2996070687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7c0cfa80 0x7f1e7c06d3c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c06c010 tx=0x7f1e7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4332f2ec82058 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e79ffb700 1 -- 172.21.15.83:0/2996070687 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400e070 con 0x7f1e7c0cfa80 2023-11-26T07:16:48.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e79ffb700 1 -- 172.21.15.83:0/2996070687 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7400b910 con 0x7f1e7c0cfa80 2023-11-26T07:16:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e79ffb700 1 -- 172.21.15.83:0/2996070687 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400bcc0 con 0x7f1e7c0cfa80 2023-11-26T07:16:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e81947700 1 -- 172.21.15.83:0/2996070687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7c0cfa80 msgr2=0x7f1e7c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e81947700 1 --2- 172.21.15.83:0/2996070687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7c0cfa80 0x7f1e7c06d3c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c06c010 tx=0x7f1e7400b330 comp rx=0 tx=0).stop 2023-11-26T07:16:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e81947700 1 -- 172.21.15.83:0/2996070687 shutdown_connections 2023-11-26T07:16:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e81947700 1 --2- 172.21.15.83:0/2996070687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7c0cfa80 0x7f1e7c06d3c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e81947700 1 --2- 172.21.15.83:0/2996070687 >> v2:172.21.15.203:3300/0 conn(0x7f1e7c0afb40 0x7f1e7c0cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e81947700 1 -- 172.21.15.83:0/2996070687 >> 172.21.15.83:0/2996070687 conn(0x7f1e7c0b2050 msgr2=0x7f1e7c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:48.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.357+0000 7f1e81947700 1 -- 172.21.15.83:0/2996070687 shutdown_connections 2023-11-26T07:16:48.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.358+0000 7f1e81947700 1 -- 172.21.15.83:0/2996070687 wait complete. 2023-11-26T07:16:48.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.359+0000 7f1e81947700 1 Processor -- start 2023-11-26T07:16:48.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.359+0000 7f1e81947700 1 -- start start 2023-11-26T07:16:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e81947700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e7c0afb40 0x7f1e7c156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e81947700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e7c0cfa80 0x7f1e7c14e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e81947700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7c15ada0 0x7f1e7c14f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e81947700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c156fc0 con 0x7f1e7c15ada0 2023-11-26T07:16:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e81947700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c1559e0 con 0x7f1e7c0afb40 2023-11-26T07:16:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e81947700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c155cc0 con 0x7f1e7c0cfa80 2023-11-26T07:16:48.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e7b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7c15ada0 0x7f1e7c14f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e7b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7c15ada0 0x7f1e7c14f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41320/0 (socket says 172.21.15.83:41320) 2023-11-26T07:16:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e7b7fe700 1 -- 172.21.15.83:0/1556682924 learned_addr learned my addr 172.21.15.83:0/1556682924 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e7affd700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e7c0afb40 0x7f1e7c156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e7a7fc700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e7c0cfa80 0x7f1e7c14e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e7b7fe700 1 -- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e7c0cfa80 msgr2=0x7f1e7c14e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e7b7fe700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e7c0cfa80 0x7f1e7c14e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e7b7fe700 1 -- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e7c0afb40 msgr2=0x7f1e7c156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e7b7fe700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e7c0afb40 0x7f1e7c156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e7b7fe700 1 -- 172.21.15.83:0/1556682924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74009b30 con 0x7f1e7c15ada0 2023-11-26T07:16:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e7b7fe700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7c15ada0 0x7f1e7c14f780 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c070e40 tx=0x7f1e74004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e63fff700 1 -- 172.21.15.83:0/1556682924 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74004880 con 0x7f1e7c15ada0 2023-11-26T07:16:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e63fff700 1 -- 172.21.15.83:0/1556682924 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e740099b0 con 0x7f1e7c15ada0 2023-11-26T07:16:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e81947700 1 -- 172.21.15.83:0/1556682924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7c155fa0 con 0x7f1e7c15ada0 2023-11-26T07:16:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e63fff700 1 -- 172.21.15.83:0/1556682924 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7401c680 con 0x7f1e7c15ada0 2023-11-26T07:16:48.369 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.360+0000 7f1e81947700 1 -- 172.21.15.83:0/1556682924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7c156560 con 0x7f1e7c15ada0 2023-11-26T07:16:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.361+0000 7f1e81947700 1 -- 172.21.15.83:0/1556682924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7c068d90 con 0x7f1e7c15ada0 2023-11-26T07:16:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.362+0000 7f1e63fff700 1 -- 172.21.15.83:0/1556682924 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f1e7401c7e0 con 0x7f1e7c15ada0 2023-11-26T07:16:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.362+0000 7f1e63fff700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1e6806aff0 0x7f1e6806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.362+0000 7f1e63fff700 1 -- 172.21.15.83:0/1556682924 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f1e7408b130 con 0x7f1e7c15ada0 2023-11-26T07:16:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.362+0000 7f1e7affd700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1e6806aff0 0x7f1e6806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.363+0000 7f1e7affd700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1e6806aff0 0x7f1e6806d4b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e70007fd0 tx=0x7f1e70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:48.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.367+0000 7f1e63fff700 1 -- 172.21.15.83:0/1556682924 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1e740562f0 con 0x7f1e7c15ada0 2023-11-26T07:16:48.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.555+0000 7f1e81947700 1 -- 172.21.15.83:0/1556682924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e7c1566f0 con 0x7f1e7c15ada0 2023-11-26T07:16:48.557 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.555+0000 7f1e63fff700 1 -- 172.21.15.83:0/1556682924 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e74059910 con 0x7f1e7c15ada0 2023-11-26T07:16:48.558 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:16:48.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 -- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1e6806aff0 msgr2=0x7f1e6806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1e6806aff0 0x7f1e6806d4b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e70007fd0 tx=0x7f1e70009040 comp rx=0 tx=0).stop 2023-11-26T07:16:48.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 -- 172.21.15.83:0/1556682924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7c15ada0 msgr2=0x7f1e7c14f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7c15ada0 0x7f1e7c14f780 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c070e40 tx=0x7f1e74004ad0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 -- 172.21.15.83:0/1556682924 shutdown_connections 2023-11-26T07:16:48.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1e6806aff0 0x7f1e6806d4b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e7c15ada0 0x7f1e7c14f780 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e7c0afb40 0x7f1e7c156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 --2- 172.21.15.83:0/1556682924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e7c0cfa80 0x7f1e7c14e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 -- 172.21.15.83:0/1556682924 >> 172.21.15.83:0/1556682924 conn(0x7f1e7c0b2050 msgr2=0x7f1e7c0b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:48.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 -- 172.21.15.83:0/1556682924 shutdown_connections 2023-11-26T07:16:48.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.557+0000 7f1e81947700 1 -- 172.21.15.83:0/1556682924 wait complete. 2023-11-26T07:16:48.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2023-11-26T07:16:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.689+0000 7f3de7bf0700 1 Processor -- start 2023-11-26T07:16:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.689+0000 7f3de7bf0700 1 -- start start 2023-11-26T07:16:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de7bf0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3de01446a0 0x7f3de00b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de7bf0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3de00adf00 0x7f3de006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de7bf0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de00af950 con 0x7f3de006c540 2023-11-26T07:16:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de7bf0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de0144a80 con 0x7f3de00adf00 2023-11-26T07:16:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de7bf0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de006c9b0 con 0x7f3de01446a0 2023-11-26T07:16:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de518b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3de00adf00 0x7f3de006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de518b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3de00adf00 0x7f3de006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41326/0 (socket says 172.21.15.83:41326) 2023-11-26T07:16:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de518b700 1 -- 172.21.15.83:0/2961217485 learned_addr learned my addr 172.21.15.83:0/2961217485 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de598c700 1 --2- 172.21.15.83:0/2961217485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3de01446a0 0x7f3de00b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de518b700 1 -- 172.21.15.83:0/2961217485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3de01446a0 msgr2=0x7f3de00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de518b700 1 --2- 172.21.15.83:0/2961217485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3de01446a0 0x7f3de00b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.690+0000 7f3de518b700 1 -- 172.21.15.83:0/2961217485 >> v1:172.21.15.203:6789/0 conn(0x7f3de006c540 legacy=0x7f3de006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de518b700 1 -- 172.21.15.83:0/2961217485 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de00afc30 con 0x7f3de00adf00 2023-11-26T07:16:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de598c700 1 --2- 172.21.15.83:0/2961217485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3de01446a0 0x7f3de00b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de518b700 1 --2- 172.21.15.83:0/2961217485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00adf00 0x7f3de006c000 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009920 tx=0x7f3ddc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3a5f9b65cfaaea server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de498a700 1 -- 172.21.15.83:0/2961217485 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004230 con 0x7f3de00adf00 2023-11-26T07:16:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de498a700 1 -- 172.21.15.83:0/2961217485 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc004390 con 0x7f3de00adf00 2023-11-26T07:16:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de498a700 1 -- 172.21.15.83:0/2961217485 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc009e70 con 0x7f3de00adf00 2023-11-26T07:16:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de7bf0700 1 -- 172.21.15.83:0/2961217485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00adf00 msgr2=0x7f3de006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de7bf0700 1 --2- 172.21.15.83:0/2961217485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00adf00 0x7f3de006c000 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009920 tx=0x7f3ddc009c30 comp rx=0 tx=0).stop 2023-11-26T07:16:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de7bf0700 1 -- 172.21.15.83:0/2961217485 shutdown_connections 2023-11-26T07:16:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de7bf0700 1 --2- 172.21.15.83:0/2961217485 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3de01446a0 0x7f3de00b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de7bf0700 1 --2- 172.21.15.83:0/2961217485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00adf00 0x7f3de006c000 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de7bf0700 1 -- 172.21.15.83:0/2961217485 >> 172.21.15.83:0/2961217485 conn(0x7f3de0071b90 msgr2=0x7f3de0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.691+0000 7f3de7bf0700 1 -- 172.21.15.83:0/2961217485 shutdown_connections 2023-11-26T07:16:48.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.692+0000 7f3de7bf0700 1 -- 172.21.15.83:0/2961217485 wait complete. 2023-11-26T07:16:48.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.693+0000 7f3de7bf0700 1 Processor -- start 2023-11-26T07:16:48.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.693+0000 7f3de7bf0700 1 -- start start 2023-11-26T07:16:48.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.693+0000 7f3de7bf0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de006c540 0x7f3de015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.693+0000 7f3de7bf0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3de00adf00 0x7f3de015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.693+0000 7f3de7bf0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3de01446a0 0x7f3de015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.693+0000 7f3de7bf0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de015e320 con 0x7f3de006c540 2023-11-26T07:16:48.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.693+0000 7f3de7bf0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de015c5f0 con 0x7f3de01446a0 2023-11-26T07:16:48.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.693+0000 7f3de7bf0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de015b9e0 con 0x7f3de00adf00 2023-11-26T07:16:48.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de598c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de006c540 0x7f3de015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de598c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de006c540 0x7f3de015ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41332/0 (socket says 172.21.15.83:41332) 2023-11-26T07:16:48.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de598c700 1 -- 172.21.15.83:0/1143297265 learned_addr learned my addr 172.21.15.83:0/1143297265 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:48.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de518b700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3de00adf00 0x7f3de015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de618d700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3de01446a0 0x7f3de015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de598c700 1 -- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3de00adf00 msgr2=0x7f3de015b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de598c700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3de00adf00 0x7f3de015b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de598c700 1 -- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3de01446a0 msgr2=0x7f3de015dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:48.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de598c700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3de01446a0 0x7f3de015dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:48.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de598c700 1 -- 172.21.15.83:0/1143297265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc009550 con 0x7f3de006c540 2023-11-26T07:16:48.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de598c700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de006c540 0x7f3de015ca80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000c7f0 tx=0x7f3dd000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:48.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3dd67fc700 1 -- 172.21.15.83:0/1143297265 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0011800 con 0x7f3de006c540 2023-11-26T07:16:48.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3dd67fc700 1 -- 172.21.15.83:0/1143297265 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0011960 con 0x7f3de006c540 2023-11-26T07:16:48.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de7bf0700 1 -- 172.21.15.83:0/1143297265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de015bcc0 con 0x7f3de006c540 2023-11-26T07:16:48.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3dd67fc700 1 -- 172.21.15.83:0/1143297265 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0011cf0 con 0x7f3de006c540 2023-11-26T07:16:48.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.694+0000 7f3de7bf0700 1 -- 172.21.15.83:0/1143297265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de015c280 con 0x7f3de006c540 2023-11-26T07:16:48.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.695+0000 7f3de7bf0700 1 -- 172.21.15.83:0/1143297265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de00753f0 con 0x7f3de006c540 2023-11-26T07:16:48.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.696+0000 7f3dd67fc700 1 -- 172.21.15.83:0/1143297265 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f3dd0018360 con 0x7f3de006c540 2023-11-26T07:16:48.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.696+0000 7f3dd67fc700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3dc406aff0 0x7f3dc406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:48.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.696+0000 7f3dd67fc700 1 -- 172.21.15.83:0/1143297265 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3dd00893e0 con 0x7f3de006c540 2023-11-26T07:16:48.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.696+0000 7f3de518b700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3dc406aff0 0x7f3dc406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:48.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.697+0000 7f3de518b700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3dc406aff0 0x7f3dc406d4b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3de00ae2e0 tx=0x7f3ddc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:48.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.700+0000 7f3dd67fc700 1 -- 172.21.15.83:0/1143297265 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3dd0054510 con 0x7f3de006c540 2023-11-26T07:16:48.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:48.970+0000 7f3de7bf0700 1 -- 172.21.15.83:0/1143297265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7f3de015c8d0 con 0x7f3de006c540 2023-11-26T07:16:49.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.881+0000 7f3dd67fc700 1 -- 172.21.15.83:0/1143297265 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v296) v1 ==== 152+0+0 (secure 0 0 0) 0x7f3dd0057b30 con 0x7f3de006c540 2023-11-26T07:16:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 -- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3dc406aff0 msgr2=0x7f3dc406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3dc406aff0 0x7f3dc406d4b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3de00ae2e0 tx=0x7f3ddc008030 comp rx=0 tx=0).stop 2023-11-26T07:16:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 -- 172.21.15.83:0/1143297265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de006c540 msgr2=0x7f3de015ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de006c540 0x7f3de015ca80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000c7f0 tx=0x7f3dd000cb00 comp rx=0 tx=0).stop 2023-11-26T07:16:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 -- 172.21.15.83:0/1143297265 shutdown_connections 2023-11-26T07:16:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3dc406aff0 0x7f3dc406d4b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3de01446a0 0x7f3de015dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:49.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de006c540 0x7f3de015ca80 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 --2- 172.21.15.83:0/1143297265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3de00adf00 0x7f3de015b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 -- 172.21.15.83:0/1143297265 >> 172.21.15.83:0/1143297265 conn(0x7f3de0071b90 msgr2=0x7f3de00d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 -- 172.21.15.83:0/1143297265 shutdown_connections 2023-11-26T07:16:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:49.884+0000 7f3de7bf0700 1 -- 172.21.15.83:0/1143297265 wait complete. 2023-11-26T07:16:49.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:16:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.028+0000 7f28e0e74700 1 Processor -- start 2023-11-26T07:16:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.028+0000 7f28e0e74700 1 -- start start 2023-11-26T07:16:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28e0e74700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f28dc15e160 0x7f28dc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28e0e74700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f28dc0746a0 0x7f28dc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28e0e74700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28dc0b3c60 con 0x7f28dc0afc90 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28e0e74700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28dc0b4500 con 0x7f28dc15e160 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28e0e74700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28dc0b58c0 con 0x7f28dc0746a0 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28da59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f28dc15e160 0x7f28dc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28da59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f28dc15e160 0x7f28dc0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41338/0 (socket says 172.21.15.83:41338) 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28da59c700 1 -- 172.21.15.83:0/3120797380 learned_addr learned my addr 172.21.15.83:0/3120797380 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28da59c700 1 -- 172.21.15.83:0/3120797380 >> v2:172.21.15.203:3300/0 conn(0x7f28dc0746a0 msgr2=0x7f28dc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28da59c700 1 --2- 172.21.15.83:0/3120797380 >> v2:172.21.15.203:3300/0 conn(0x7f28dc0746a0 0x7f28dc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28da59c700 1 -- 172.21.15.83:0/3120797380 >> v1:172.21.15.203:6789/0 conn(0x7f28dc0afc90 legacy=0x7f28dc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28da59c700 1 -- 172.21.15.83:0/3120797380 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc0b0140 con 0x7f28dc15e160 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28da59c700 1 --2- 172.21.15.83:0/3120797380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28dc15e160 0x7f28dc0b5da0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0d16e0 tx=0x7f28cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7909408c38b65283 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28d959a700 1 -- 172.21.15.83:0/3120797380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc004230 con 0x7f28dc15e160 2023-11-26T07:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28d959a700 1 -- 172.21.15.83:0/3120797380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc004390 con 0x7f28dc15e160 2023-11-26T07:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.029+0000 7f28d959a700 1 -- 172.21.15.83:0/3120797380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc0036a0 con 0x7f28dc15e160 2023-11-26T07:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.030+0000 7f28e0e74700 1 -- 172.21.15.83:0/3120797380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28dc15e160 msgr2=0x7f28dc0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.030+0000 7f28e0e74700 1 --2- 172.21.15.83:0/3120797380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28dc15e160 0x7f28dc0b5da0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0d16e0 tx=0x7f28cc009bd0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.030+0000 7f28e0e74700 1 -- 172.21.15.83:0/3120797380 shutdown_connections 2023-11-26T07:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.030+0000 7f28e0e74700 1 --2- 172.21.15.83:0/3120797380 >> v2:172.21.15.203:3300/0 conn(0x7f28dc0746a0 0x7f28dc06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.030+0000 7f28e0e74700 1 --2- 172.21.15.83:0/3120797380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28dc15e160 0x7f28dc0b5da0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.030+0000 7f28e0e74700 1 -- 172.21.15.83:0/3120797380 >> 172.21.15.83:0/3120797380 conn(0x7f28dc0d0bf0 msgr2=0x7f28dc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.030+0000 7f28e0e74700 1 -- 172.21.15.83:0/3120797380 shutdown_connections 2023-11-26T07:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.030+0000 7f28e0e74700 1 -- 172.21.15.83:0/3120797380 wait complete. 2023-11-26T07:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.031+0000 7f28e0e74700 1 Processor -- start 2023-11-26T07:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.031+0000 7f28e0e74700 1 -- start start 2023-11-26T07:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28e0e74700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28dc0746a0 0x7f28dc154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28e0e74700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28dc0afc90 0x7f28dc1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28e0e74700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28dc15e160 0x7f28dc1f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28e0e74700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc15e880 con 0x7f28dc0746a0 2023-11-26T07:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28e0e74700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc15eb60 con 0x7f28dc15e160 2023-11-26T07:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28e0e74700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc15ee40 con 0x7f28dc0afc90 2023-11-26T07:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28da59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28dc0746a0 0x7f28dc154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28da59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28dc0746a0 0x7f28dc154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41344/0 (socket says 172.21.15.83:41344) 2023-11-26T07:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28da59c700 1 -- 172.21.15.83:0/2982575397 learned_addr learned my addr 172.21.15.83:0/2982575397 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28dad9d700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28dc15e160 0x7f28dc1f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28d9d9b700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28dc0afc90 0x7f28dc1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28da59c700 1 -- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28dc0afc90 msgr2=0x7f28dc1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28da59c700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28dc0afc90 0x7f28dc1550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28da59c700 1 -- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28dc15e160 msgr2=0x7f28dc1f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28da59c700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28dc15e160 0x7f28dc1f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28da59c700 1 -- 172.21.15.83:0/2982575397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc009550 con 0x7f28dc0746a0 2023-11-26T07:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.032+0000 7f28d9d9b700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28dc0afc90 0x7f28dc1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.033+0000 7f28da59c700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28dc0746a0 0x7f28dc154bb0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000c00 tx=0x7f28cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.033+0000 7f28caffd700 1 -- 172.21.15.83:0/2982575397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc011830 con 0x7f28dc0746a0 2023-11-26T07:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.033+0000 7f28caffd700 1 -- 172.21.15.83:0/2982575397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc0119b0 con 0x7f28dc0746a0 2023-11-26T07:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.033+0000 7f28e0e74700 1 -- 172.21.15.83:0/2982575397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc15f120 con 0x7f28dc0746a0 2023-11-26T07:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.033+0000 7f28caffd700 1 -- 172.21.15.83:0/2982575397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc011d60 con 0x7f28dc0746a0 2023-11-26T07:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.033+0000 7f28e0e74700 1 -- 172.21.15.83:0/2982575397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc0c7880 con 0x7f28dc0746a0 2023-11-26T07:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.034+0000 7f28e0e74700 1 -- 172.21.15.83:0/2982575397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28dc068fb0 con 0x7f28dc0746a0 2023-11-26T07:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.034+0000 7f28caffd700 1 -- 172.21.15.83:0/2982575397 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f28cc01b440 con 0x7f28dc0746a0 2023-11-26T07:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.034+0000 7f28caffd700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f28bc06aff0 0x7f28bc06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.035+0000 7f28caffd700 1 -- 172.21.15.83:0/2982575397 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f28cc08a9d0 con 0x7f28dc0746a0 2023-11-26T07:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.035+0000 7f28d9d9b700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f28bc06aff0 0x7f28bc06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.035+0000 7f28d9d9b700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f28bc06aff0 0x7f28bc06d4b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f28dc1f8030 tx=0x7f28d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.039+0000 7f28caffd700 1 -- 172.21.15.83:0/2982575397 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28cc055c00 con 0x7f28dc0746a0 2023-11-26T07:16:50.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.228+0000 7f28e0e74700 1 -- 172.21.15.83:0/2982575397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f28dc0b46b0 con 0x7f28dc0746a0 2023-11-26T07:16:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.231+0000 7f28caffd700 1 -- 172.21.15.83:0/2982575397 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f28dc0b46b0 con 0x7f28dc0746a0 2023-11-26T07:16:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 -- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f28bc06aff0 msgr2=0x7f28bc06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f28bc06aff0 0x7f28bc06d4b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f28dc1f8030 tx=0x7f28d0009040 comp rx=0 tx=0).stop 2023-11-26T07:16:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 -- 172.21.15.83:0/2982575397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28dc0746a0 msgr2=0x7f28dc154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28dc0746a0 0x7f28dc154bb0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000c00 tx=0x7f28cc004750 comp rx=0 tx=0).stop 2023-11-26T07:16:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 -- 172.21.15.83:0/2982575397 shutdown_connections 2023-11-26T07:16:50.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f28bc06aff0 0x7f28bc06d4b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28dc15e160 0x7f28dc1f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28dc0746a0 0x7f28dc154bb0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 --2- 172.21.15.83:0/2982575397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28dc0afc90 0x7f28dc1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 -- 172.21.15.83:0/2982575397 >> 172.21.15.83:0/2982575397 conn(0x7f28dc0d0bf0 msgr2=0x7f28dc15d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:50.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 -- 172.21.15.83:0/2982575397 shutdown_connections 2023-11-26T07:16:50.237 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.233+0000 7f28e0e74700 1 -- 172.21.15.83:0/2982575397 wait complete. 2023-11-26T07:16:50.248 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:16:50.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:16:50.285 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:16:50.334 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:16:50.334 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:16:50.334 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:50.351 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:16:50.352 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:16:50.352 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:50.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:16:50.371 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:16:50.371 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:50.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:16:50.409 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:16:50.409 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:16:50.430 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:16:50.430 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:16:50.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:16:50.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.566+0000 7fea3206f700 1 Processor -- start 2023-11-26T07:16:50.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.566+0000 7fea3206f700 1 -- start start 2023-11-26T07:16:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea3206f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea2c15bdd0 0x7fea2c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea3206f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fea2c0d1c70 0x7fea2c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea3206f700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2c0b22d0 con 0x7fea2c0b3370 2023-11-26T07:16:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea3206f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2c0b2f50 con 0x7fea2c0d1c70 2023-11-26T07:16:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea3206f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0d2640 con 0x7fea2c15bdd0 2023-11-26T07:16:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea2bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fea2c0d1c70 0x7fea2c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea2bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fea2c0d1c70 0x7fea2c0d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41356/0 (socket says 172.21.15.83:41356) 2023-11-26T07:16:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea2bfff700 1 -- 172.21.15.83:0/1948668236 learned_addr learned my addr 172.21.15.83:0/1948668236 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea2b7fe700 1 --2- 172.21.15.83:0/1948668236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea2c15bdd0 0x7fea2c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea2bfff700 1 -- 172.21.15.83:0/1948668236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea2c15bdd0 msgr2=0x7fea2c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea2bfff700 1 --2- 172.21.15.83:0/1948668236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea2c15bdd0 0x7fea2c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea2bfff700 1 -- 172.21.15.83:0/1948668236 >> v1:172.21.15.203:6789/0 conn(0x7fea2c0b3370 legacy=0x7fea2c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.567+0000 7fea2bfff700 1 -- 172.21.15.83:0/1948668236 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0b25b0 con 0x7fea2c0d1c70 2023-11-26T07:16:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea2bfff700 1 --2- 172.21.15.83:0/1948668236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea2c0d1c70 0x7fea2c0d2100 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fea24009e50 tx=0x7fea2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bedae67f32f5e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea2a7fc700 1 -- 172.21.15.83:0/1948668236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2400e070 con 0x7fea2c0d1c70 2023-11-26T07:16:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea2a7fc700 1 -- 172.21.15.83:0/1948668236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2400b9c0 con 0x7fea2c0d1c70 2023-11-26T07:16:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea2a7fc700 1 -- 172.21.15.83:0/1948668236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2400bd70 con 0x7fea2c0d1c70 2023-11-26T07:16:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea3206f700 1 -- 172.21.15.83:0/1948668236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea2c0d1c70 msgr2=0x7fea2c0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea3206f700 1 --2- 172.21.15.83:0/1948668236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea2c0d1c70 0x7fea2c0d2100 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fea24009e50 tx=0x7fea2400b330 comp rx=0 tx=0).stop 2023-11-26T07:16:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea3206f700 1 -- 172.21.15.83:0/1948668236 shutdown_connections 2023-11-26T07:16:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea3206f700 1 --2- 172.21.15.83:0/1948668236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea2c0d1c70 0x7fea2c0d2100 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea3206f700 1 --2- 172.21.15.83:0/1948668236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea2c15bdd0 0x7fea2c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea3206f700 1 -- 172.21.15.83:0/1948668236 >> 172.21.15.83:0/1948668236 conn(0x7fea2c0731a0 msgr2=0x7fea2c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea3206f700 1 -- 172.21.15.83:0/1948668236 shutdown_connections 2023-11-26T07:16:50.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.568+0000 7fea3206f700 1 -- 172.21.15.83:0/1948668236 wait complete. 2023-11-26T07:16:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.569+0000 7fea3206f700 1 Processor -- start 2023-11-26T07:16:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.570+0000 7fea3206f700 1 -- start start 2023-11-26T07:16:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.570+0000 7fea3206f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea2c0b3370 0x7fea2c0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.570+0000 7fea3206f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea2c0d1c70 0x7fea2c157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.570+0000 7fea3206f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea2c15bdd0 0x7fea2c158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.576 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.570+0000 7fea3206f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c158580 con 0x7fea2c15bdd0 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.570+0000 7fea3206f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c144180 con 0x7fea2c0b3370 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.570+0000 7fea3206f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0b6910 con 0x7fea2c0d1c70 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea2bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea2c15bdd0 0x7fea2c158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea2bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea2c15bdd0 0x7fea2c158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41362/0 (socket says 172.21.15.83:41362) 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea2bfff700 1 -- 172.21.15.83:0/4251861321 learned_addr learned my addr 172.21.15.83:0/4251861321 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea2affd700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea2c0d1c70 0x7fea2c157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea2b7fe700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea2c0b3370 0x7fea2c0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea2bfff700 1 -- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea2c0d1c70 msgr2=0x7fea2c157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea2bfff700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea2c0d1c70 0x7fea2c157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea2bfff700 1 -- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea2c0b3370 msgr2=0x7fea2c0c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea2bfff700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea2c0b3370 0x7fea2c0c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.577 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea2bfff700 1 -- 172.21.15.83:0/4251861321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea24009b30 con 0x7fea2c15bdd0 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea2bfff700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea2c15bdd0 0x7fea2c158040 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0716d0 tx=0x7fea24004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea13fff700 1 -- 172.21.15.83:0/4251861321 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24004800 con 0x7fea2c15bdd0 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea13fff700 1 -- 172.21.15.83:0/4251861321 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea24004960 con 0x7fea2c15bdd0 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea3206f700 1 -- 172.21.15.83:0/4251861321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2c0b6bf0 con 0x7fea2c15bdd0 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea13fff700 1 -- 172.21.15.83:0/4251861321 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2401c680 con 0x7fea2c15bdd0 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.571+0000 7fea3206f700 1 -- 172.21.15.83:0/4251861321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2c0b71b0 con 0x7fea2c15bdd0 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.572+0000 7fea3206f700 1 -- 172.21.15.83:0/4251861321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2c067f60 con 0x7fea2c15bdd0 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.572+0000 7fea13fff700 1 -- 172.21.15.83:0/4251861321 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fea2401c7e0 con 0x7fea2c15bdd0 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.573+0000 7fea13fff700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fea1806aff0 0x7fea1806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.573+0000 7fea13fff700 1 -- 172.21.15.83:0/4251861321 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fea2408b170 con 0x7fea2c15bdd0 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.573+0000 7fea2b7fe700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fea1806aff0 0x7fea1806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:50.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.574+0000 7fea2b7fe700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fea1806aff0 0x7fea1806d4b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0b77f0 tx=0x7fea20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:50.579 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.577+0000 7fea13fff700 1 -- 172.21.15.83:0/4251861321 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea24056330 con 0x7fea2c15bdd0 2023-11-26T07:16:50.814 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.812+0000 7fea3206f700 1 -- 172.21.15.83:0/4251861321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea2c0d21e0 con 0x7fea2c15bdd0 2023-11-26T07:16:50.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.812+0000 7fea13fff700 1 -- 172.21.15.83:0/4251861321 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fea24059950 con 0x7fea2c15bdd0 2023-11-26T07:16:50.815 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:50.815 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":62,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6364,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/1015690400","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1015690400},{"type":"v1","addr":"172.21.15.203:6834","nonce":1015690400}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/4133563498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":4133563498},{"type":"v1","addr":"172.21.15.203:6835","nonce":4133563498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":7110,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2716428832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2716428832},{"type":"v1","addr":"172.21.15.83:6834","nonce":2716428832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7326,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/558945070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":558945070},{"type":"v1","addr":"172.21.15.83:6835","nonce":558945070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[]} 2023-11-26T07:16:50.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 -- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fea1806aff0 msgr2=0x7fea1806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:50.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fea1806aff0 0x7fea1806d4b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0b77f0 tx=0x7fea20008040 comp rx=0 tx=0).stop 2023-11-26T07:16:50.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 -- 172.21.15.83:0/4251861321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea2c15bdd0 msgr2=0x7fea2c158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:50.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea2c15bdd0 0x7fea2c158040 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0716d0 tx=0x7fea24004ad0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 -- 172.21.15.83:0/4251861321 shutdown_connections 2023-11-26T07:16:50.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fea1806aff0 0x7fea1806d4b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea2c15bdd0 0x7fea2c158040 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.818 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea2c0b3370 0x7fea2c0c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 --2- 172.21.15.83:0/4251861321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea2c0d1c70 0x7fea2c157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:50.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 -- 172.21.15.83:0/4251861321 >> 172.21.15.83:0/4251861321 conn(0x7fea2c0731a0 msgr2=0x7fea2c0b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:50.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 -- 172.21.15.83:0/4251861321 shutdown_connections 2023-11-26T07:16:50.819 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:50.815+0000 7fea3206f700 1 -- 172.21.15.83:0/4251861321 wait complete. 2023-11-26T07:16:50.819 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 62 2023-11-26T07:16:50.830 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:16:50.830 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:16:50.830 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:16:50.830 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:16:50.830 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:16:50.830 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:16:50.831 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:16:50.831 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:16:50.831 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:16:50.831 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:16:50.831 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:16:50.831 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:16:50.832 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:16:50.832 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:16:50.832 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:16:50.832 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:16:50.839 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:16:50.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:16:50.843 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:16:50.843 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:16:50.844 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:16:50.844 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:16:50.881 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:16:50.881 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:50.885 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:16:50.886 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:50.886 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:16:50.886 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:50.957 INFO:tasks.ceph.mds.c.smithi083.stdout:starting mds.c at 2023-11-26T07:16:50.976 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2023-11-26T07:16:50.976 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2023-11-26T07:16:56.839 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:16:56.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:16:56.842 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:16:56.842 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:56.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:16:56.931 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-26T07:16:56.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.977+0000 7f15fb02e700 1 Processor -- start 2023-11-26T07:16:56.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.977+0000 7f15fb02e700 1 -- start start 2023-11-26T07:16:56.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15fb02e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15f40ce4d0 0x7f15f40d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:56.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15fb02e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f15f40ce910 0x7f15f40d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:56.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15fb02e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f40d0150 con 0x7f15f415e080 2023-11-26T07:16:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15fb02e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f415e460 con 0x7f15f40ce910 2023-11-26T07:16:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15fb02e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40cc9b0 con 0x7f15f40ce4d0 2023-11-26T07:16:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15f95cb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f15f40ce910 0x7f15f40d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15f8dca700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f15f415e080 0x7f15f40cc2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33198/0 (socket says 172.21.15.83:33198) 2023-11-26T07:16:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15f8dca700 1 -- 172.21.15.83:0/3625954649 learned_addr learned my addr 172.21.15.83:0/3625954649 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15f3fff700 1 --2- 172.21.15.83:0/3625954649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15f40ce4d0 0x7f15f40d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15f95cb700 1 -- 172.21.15.83:0/3625954649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15f40ce4d0 msgr2=0x7f15f40d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15f95cb700 1 --2- 172.21.15.83:0/3625954649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15f40ce4d0 0x7f15f40d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15f95cb700 1 -- 172.21.15.83:0/3625954649 >> v1:172.21.15.83:6789/0 conn(0x7f15f415e080 legacy=0x7f15f40cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15f95cb700 1 -- 172.21.15.83:0/3625954649 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f40cee50 con 0x7f15f40ce910 2023-11-26T07:16:56.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15f37fe700 1 -- 172.21.15.83:0/3625954649 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982254471 0 0) 0x7f15f40d0150 con 0x7f15f415e080 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.978+0000 7f15f95cb700 1 --2- 172.21.15.83:0/3625954649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f40ce910 0x7f15f40d2120 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009dc0 tx=0x7f15ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4e7bc90c0855e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.979+0000 7f15f37fe700 1 -- 172.21.15.83:0/3625954649 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00e070 con 0x7f15f40ce910 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.979+0000 7f15f37fe700 1 -- 172.21.15.83:0/3625954649 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec00bb90 con 0x7f15f40ce910 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.979+0000 7f15f37fe700 1 -- 172.21.15.83:0/3625954649 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec012430 con 0x7f15f40ce910 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.979+0000 7f15fb02e700 1 -- 172.21.15.83:0/3625954649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f40ce910 msgr2=0x7f15f40d2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.979+0000 7f15fb02e700 1 --2- 172.21.15.83:0/3625954649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f40ce910 0x7f15f40d2120 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009dc0 tx=0x7f15ec00b330 comp rx=0 tx=0).stop 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.979+0000 7f15fb02e700 1 -- 172.21.15.83:0/3625954649 shutdown_connections 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.979+0000 7f15fb02e700 1 --2- 172.21.15.83:0/3625954649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f40ce910 0x7f15f40d2120 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.979+0000 7f15fb02e700 1 --2- 172.21.15.83:0/3625954649 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15f40ce4d0 0x7f15f40d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.979+0000 7f15fb02e700 1 -- 172.21.15.83:0/3625954649 >> 172.21.15.83:0/3625954649 conn(0x7f15f4073130 msgr2=0x7f15f409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.979+0000 7f15fb02e700 1 -- 172.21.15.83:0/3625954649 shutdown_connections 2023-11-26T07:16:56.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.979+0000 7f15fb02e700 1 -- 172.21.15.83:0/3625954649 wait complete. 2023-11-26T07:16:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.980+0000 7f15fb02e700 1 Processor -- start 2023-11-26T07:16:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.980+0000 7f15fb02e700 1 -- start start 2023-11-26T07:16:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.981+0000 7f15fb02e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f40ce4d0 0x7f15f4159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.981+0000 7f15fb02e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15f40ce910 0x7f15f414b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:56.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.981+0000 7f15fb02e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f415e080 0x7f15f40ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.981+0000 7f15fb02e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f415b2e0 con 0x7f15f40ce4d0 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.981+0000 7f15fb02e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4159850 con 0x7f15f415e080 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.981+0000 7f15fb02e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f415a660 con 0x7f15f40ce910 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.981+0000 7f15f8dca700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f40ce4d0 0x7f15f4159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.981+0000 7f15f8dca700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f40ce4d0 0x7f15f4159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41396/0 (socket says 172.21.15.83:41396) 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.981+0000 7f15f8dca700 1 -- 172.21.15.83:0/1621263716 learned_addr learned my addr 172.21.15.83:0/1621263716 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f3fff700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15f40ce910 0x7f15f414b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f95cb700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f415e080 0x7f15f40ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f8dca700 1 -- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15f40ce910 msgr2=0x7f15f414b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f8dca700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15f40ce910 0x7f15f414b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f8dca700 1 -- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f415e080 msgr2=0x7f15f40ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f8dca700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f415e080 0x7f15f40ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:56.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f8dca700 1 -- 172.21.15.83:0/1621263716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec009a70 con 0x7f15f40ce4d0 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f95cb700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f415e080 0x7f15f40ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f8dca700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f40ce4d0 0x7f15f4159310 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f15e800c550 tx=0x7f15e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f17fa700 1 -- 172.21.15.83:0/1621263716 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8004c80 con 0x7f15f40ce4d0 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f17fa700 1 -- 172.21.15.83:0/1621263716 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e8004e00 con 0x7f15f40ce4d0 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15fb02e700 1 -- 172.21.15.83:0/1621263716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f415a940 con 0x7f15f40ce4d0 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15f17fa700 1 -- 172.21.15.83:0/1621263716 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8007310 con 0x7f15f40ce4d0 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.982+0000 7f15fb02e700 1 -- 172.21.15.83:0/1621263716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f415af00 con 0x7f15f40ce4d0 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.983+0000 7f15fb02e700 1 -- 172.21.15.83:0/1621263716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f4070b00 con 0x7f15f40ce4d0 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.983+0000 7f15f17fa700 1 -- 172.21.15.83:0/1621263716 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f15e8007470 con 0x7f15f40ce4d0 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.984+0000 7f15f17fa700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f15d806aff0 0x7f15d806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.984+0000 7f15f17fa700 1 -- 172.21.15.83:0/1621263716 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f15e80891b0 con 0x7f15f40ce4d0 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.984+0000 7f15f3fff700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f15d806aff0 0x7f15d806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:56.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.985+0000 7f15f3fff700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f15d806aff0 0x7f15d806d4b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f15f41f7320 tx=0x7f15e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:56.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:56.989+0000 7f15f17fa700 1 -- 172.21.15.83:0/1621263716 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15e80543f0 con 0x7f15f40ce4d0 2023-11-26T07:16:57.153 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-26T07:16:57.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:57.194+0000 7f15fb02e700 1 -- 172.21.15.83:0/1621263716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f15f4067f60 con 0x7f15f40ce4d0 2023-11-26T07:16:57.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:16:57.389 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:16:58.378 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.376+0000 7f15f17fa700 1 -- 172.21.15.83:0/1621263716 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v98) v1 ==== 97+0+0 (secure 0 0 0) 0x7f15e8057a10 con 0x7f15f40ce4d0 2023-11-26T07:16:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 -- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f15d806aff0 msgr2=0x7f15d806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.382 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f15d806aff0 0x7f15d806d4b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f15f41f7320 tx=0x7f15e0008040 comp rx=0 tx=0).stop 2023-11-26T07:16:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 -- 172.21.15.83:0/1621263716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f40ce4d0 msgr2=0x7f15f4159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f40ce4d0 0x7f15f4159310 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f15e800c550 tx=0x7f15e800c910 comp rx=0 tx=0).stop 2023-11-26T07:16:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 -- 172.21.15.83:0/1621263716 shutdown_connections 2023-11-26T07:16:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f15d806aff0 0x7f15d806d4b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15f415e080 0x7f15f40ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f40ce4d0 0x7f15f4159310 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 --2- 172.21.15.83:0/1621263716 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15f40ce910 0x7f15f414b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 -- 172.21.15.83:0/1621263716 >> 172.21.15.83:0/1621263716 conn(0x7f15f4073130 msgr2=0x7f15f406ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 -- 172.21.15.83:0/1621263716 shutdown_connections 2023-11-26T07:16:58.384 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.380+0000 7f15fb02e700 1 -- 172.21.15.83:0/1621263716 wait complete. 2023-11-26T07:16:58.384 INFO:teuthology.orchestra.run.smithi083.stderr: removed all blocklist entries 2023-11-26T07:16:58.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:16:58.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.520+0000 7fa547cbf700 1 Processor -- start 2023-11-26T07:16:58.522 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.520+0000 7fa547cbf700 1 -- start start 2023-11-26T07:16:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa547cbf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5401446a0 0x7fa5400b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa547cbf700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5400af950 con 0x7fa5400adf00 2023-11-26T07:16:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa547cbf700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5400afc30 con 0x7fa54006c710 2023-11-26T07:16:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa547cbf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa540144a80 con 0x7fa5401446a0 2023-11-26T07:16:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa54525a700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa5400adf00 0x7fa54006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33218/0 (socket says 172.21.15.83:33218) 2023-11-26T07:16:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa54525a700 1 -- 172.21.15.83:0/1320163440 learned_addr learned my addr 172.21.15.83:0/1320163440 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa545a5b700 1 --2- 172.21.15.83:0/1320163440 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5401446a0 0x7fa5400b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa544a59700 1 -- 172.21.15.83:0/1320163440 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755819360 0 0) 0x7fa5400af950 con 0x7fa5400adf00 2023-11-26T07:16:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa544a59700 1 -- 172.21.15.83:0/1320163440 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa52c0035b0 con 0x7fa5400adf00 2023-11-26T07:16:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa544a59700 1 -- 172.21.15.83:0/1320163440 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631189863 0 0) 0x7fa5400afc30 con 0x7fa54006c710 2023-11-26T07:16:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa544a59700 1 -- 172.21.15.83:0/1320163440 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5400af950 con 0x7fa54006c710 2023-11-26T07:16:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.521+0000 7fa544a59700 1 -- 172.21.15.83:0/1320163440 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1533923679 0 0) 0x7fa52c0035b0 con 0x7fa5400adf00 2023-11-26T07:16:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.522+0000 7fa545a5b700 1 -- 172.21.15.83:0/1320163440 >> v1:172.21.15.203:6789/0 conn(0x7fa54006c710 legacy=0x7fa5400b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.522+0000 7fa545a5b700 1 -- 172.21.15.83:0/1320163440 >> v1:172.21.15.83:6789/0 conn(0x7fa5400adf00 legacy=0x7fa54006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.522+0000 7fa545a5b700 1 -- 172.21.15.83:0/1320163440 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400aff10 con 0x7fa5401446a0 2023-11-26T07:16:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.522+0000 7fa545a5b700 1 --2- 172.21.15.83:0/1320163440 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5401446a0 0x7fa5400b5ae0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009870 tx=0x7fa53c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0c79e41dfda593 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.522+0000 7fa544a59700 1 -- 172.21.15.83:0/1320163440 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530001670 con 0x7fa5401446a0 2023-11-26T07:16:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.522+0000 7fa544a59700 1 -- 172.21.15.83:0/1320163440 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53c004030 con 0x7fa5401446a0 2023-11-26T07:16:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.522+0000 7fa544a59700 1 -- 172.21.15.83:0/1320163440 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0043c0 con 0x7fa5401446a0 2023-11-26T07:16:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.522+0000 7fa547cbf700 1 -- 172.21.15.83:0/1320163440 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5401446a0 msgr2=0x7fa5400b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.522+0000 7fa547cbf700 1 --2- 172.21.15.83:0/1320163440 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5401446a0 0x7fa5400b5ae0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009870 tx=0x7fa53c009b80 comp rx=0 tx=0).stop 2023-11-26T07:16:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.523+0000 7fa547cbf700 1 -- 172.21.15.83:0/1320163440 shutdown_connections 2023-11-26T07:16:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.523+0000 7fa547cbf700 1 --2- 172.21.15.83:0/1320163440 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5401446a0 0x7fa5400b5ae0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.523+0000 7fa547cbf700 1 -- 172.21.15.83:0/1320163440 >> 172.21.15.83:0/1320163440 conn(0x7fa540071b90 msgr2=0x7fa540071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.523+0000 7fa547cbf700 1 -- 172.21.15.83:0/1320163440 shutdown_connections 2023-11-26T07:16:58.529 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.523+0000 7fa547cbf700 1 -- 172.21.15.83:0/1320163440 wait complete. 2023-11-26T07:16:58.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.524+0000 7fa547cbf700 1 Processor -- start 2023-11-26T07:16:58.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.524+0000 7fa547cbf700 1 -- start start 2023-11-26T07:16:58.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa547cbf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa54006c710 0x7fa5401eee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa547cbf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5400adf00 0x7fa5401f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa547cbf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401446a0 0x7fa540073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa547cbf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400c6330 con 0x7fa5401446a0 2023-11-26T07:16:58.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa547cbf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5401ef3c0 con 0x7fa5400adf00 2023-11-26T07:16:58.532 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa547cbf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54009dff0 con 0x7fa54006c710 2023-11-26T07:16:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa54625c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401446a0 0x7fa540073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa54625c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401446a0 0x7fa540073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41414/0 (socket says 172.21.15.83:41414) 2023-11-26T07:16:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa54625c700 1 -- 172.21.15.83:0/1831436139 learned_addr learned my addr 172.21.15.83:0/1831436139 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa545a5b700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa54006c710 0x7fa5401eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa54525a700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5400adf00 0x7fa5401f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa54625c700 1 -- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa54006c710 msgr2=0x7fa5401eee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.525+0000 7fa54625c700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa54006c710 0x7fa5401eee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.526+0000 7fa54625c700 1 -- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5400adf00 msgr2=0x7fa5401f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.526+0000 7fa54625c700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5400adf00 0x7fa5401f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.526+0000 7fa54625c700 1 -- 172.21.15.83:0/1831436139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c0094a0 con 0x7fa5401446a0 2023-11-26T07:16:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.526+0000 7fa54525a700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5400adf00 0x7fa5401f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.526+0000 7fa54625c700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401446a0 0x7fa540073f90 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fa53800cbc0 tx=0x7fa53800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.526+0000 7fa5367fc700 1 -- 172.21.15.83:0/1831436139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538003c00 con 0x7fa5401446a0 2023-11-26T07:16:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.526+0000 7fa5367fc700 1 -- 172.21.15.83:0/1831436139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa538003d60 con 0x7fa5401446a0 2023-11-26T07:16:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.526+0000 7fa547cbf700 1 -- 172.21.15.83:0/1831436139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54009e2d0 con 0x7fa5401446a0 2023-11-26T07:16:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.526+0000 7fa5367fc700 1 -- 172.21.15.83:0/1831436139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53800ab00 con 0x7fa5401446a0 2023-11-26T07:16:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.526+0000 7fa547cbf700 1 -- 172.21.15.83:0/1831436139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5400c5f50 con 0x7fa5401446a0 2023-11-26T07:16:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.527+0000 7fa547cbf700 1 -- 172.21.15.83:0/1831436139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5400753f0 con 0x7fa5401446a0 2023-11-26T07:16:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.527+0000 7fa5367fc700 1 -- 172.21.15.83:0/1831436139 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fa53800ac60 con 0x7fa5401446a0 2023-11-26T07:16:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.527+0000 7fa5367fc700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa52406aff0 0x7fa52406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.528+0000 7fa5367fc700 1 -- 172.21.15.83:0/1831436139 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa5380893e0 con 0x7fa5401446a0 2023-11-26T07:16:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.528+0000 7fa545a5b700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa52406aff0 0x7fa52406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.528+0000 7fa545a5b700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa52406aff0 0x7fa52406d4b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0097c0 tx=0x7fa53c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.532+0000 7fa5367fc700 1 -- 172.21.15.83:0/1831436139 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa538054760 con 0x7fa5401446a0 2023-11-26T07:16:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.802+0000 7fa547cbf700 1 -- 172.21.15.83:0/1831436139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa5400c6470 con 0x7fa5401446a0 2023-11-26T07:16:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.803+0000 7fa5367fc700 1 -- 172.21.15.83:0/1831436139 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa5400c6470 con 0x7fa5401446a0 2023-11-26T07:16:58.805 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:58.806 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:16:58.806 INFO:teuthology.orchestra.run.smithi083.stdout: "auth_dump": [ 2023-11-26T07:16:58.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.806 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.a", 2023-11-26T07:16:58.806 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlA8pxNxAAt5NuewAjm7yPThCguJ6sEA==", 2023-11-26T07:16:58.806 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.807 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:58.807 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:16:58.807 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:16:58.807 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.807 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.807 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.808 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.b", 2023-11-26T07:16:58.808 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlHBgDCBAAk6egig5krY7kiMiHkMKrWg==", 2023-11-26T07:16:58.808 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.808 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:58.808 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:16:58.808 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:16:58.808 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.809 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.809 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.809 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.c", 2023-11-26T07:16:58.809 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlbKEQAxAAM2tBBQwM+tt2d8JU37YURg==", 2023-11-26T07:16:58.809 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.809 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:58.810 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:16:58.810 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:16:58.810 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.810 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.810 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.810 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mds.d", 2023-11-26T07:16:58.811 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlI3ZHDxAArtLmBVGr/vAB/YGwEFfhGw==", 2023-11-26T07:16:58.811 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.811 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:58.811 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:16:58.811 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:16:58.811 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.811 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.812 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.812 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.0", 2023-11-26T07:16:58.812 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBo72JlhuSTKRAA6LRDOHBQVl5TL5tIHzyfzA==", 2023-11-26T07:16:58.812 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.812 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:58.812 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:58.813 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.813 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.813 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.1", 2023-11-26T07:16:58.813 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBr72JlteU6KRAAuuHTYosYpl+KptywVmaeHA==", 2023-11-26T07:16:58.813 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.814 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:58.814 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:58.814 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.814 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.815 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.2", 2023-11-26T07:16:58.815 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBu72JlEb6dKBAAebp4pnu9XZsSRJpojTZcvQ==", 2023-11-26T07:16:58.815 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.815 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:58.815 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:58.815 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.815 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.816 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.816 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.3", 2023-11-26T07:16:58.816 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBx72JlbTJ3KBAA1JUxxAoyCzTtfgubUxSJEA==", 2023-11-26T07:16:58.816 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.816 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:58.816 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:58.817 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.817 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.817 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.817 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.4", 2023-11-26T07:16:58.817 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB372JlufqqLhAAm6ZUt9juWj5yAIDlnCXqWg==", 2023-11-26T07:16:58.817 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.818 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:58.818 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:58.818 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.818 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.818 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.5", 2023-11-26T07:16:58.819 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB672Jl1L3vLRAAX06GcsntcmB+7L1CsfnJzw==", 2023-11-26T07:16:58.819 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.819 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:58.819 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:58.819 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.820 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.820 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.6", 2023-11-26T07:16:58.820 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQB972Jlv41HLRAALtuyuREvGmTuDZt/RT0UcA==", 2023-11-26T07:16:58.820 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.820 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:58.820 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:58.820 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.821 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.821 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.821 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "osd.7", 2023-11-26T07:16:58.821 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQCA72JlKCHsLBAAAXDzA7AN36wcLjhK97nlkg==", 2023-11-26T07:16:58.821 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.821 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow profile osd", 2023-11-26T07:16:58.822 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile osd", 2023-11-26T07:16:58.822 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.822 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.822 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.822 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.0", 2023-11-26T07:16:58.823 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ==", 2023-11-26T07:16:58.823 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.823 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:58.823 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow r", 2023-11-26T07:16:58.823 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:16:58.823 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.824 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.824 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.1", 2023-11-26T07:16:58.824 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JleNpIGhAAKwoQj4zB9YOxpFMD28juCA==", 2023-11-26T07:16:58.824 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.824 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:58.824 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:16:58.824 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:16:58.825 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:16:58.825 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.825 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.825 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.2", 2023-11-26T07:16:58.825 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JldAHlFhAAb3u9KljU9vssDl54zgno2Q==", 2023-11-26T07:16:58.825 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.826 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:58.826 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:16:58.826 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:16:58.826 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:16:58.826 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.826 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.827 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.3", 2023-11-26T07:16:58.827 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBl72JluiQUIBAA06mqpeCzgpbSLo2/fulfmw==", 2023-11-26T07:16:58.827 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.827 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow", 2023-11-26T07:16:58.827 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow r", 2023-11-26T07:16:58.827 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow rw", 2023-11-26T07:16:58.827 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow rwx" 2023-11-26T07:16:58.828 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.828 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "client.admin", 2023-11-26T07:16:58.828 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlD4jbGhAA05xtSS5EzQTuhBjVqS2JpA==", 2023-11-26T07:16:58.828 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.828 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:16:58.829 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr": "allow *", 2023-11-26T07:16:58.829 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow *", 2023-11-26T07:16:58.829 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.829 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.829 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.x", 2023-11-26T07:16:58.830 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72JlsA0VNBAAfSlnVquhI+vpLFwmgLGGVA==", 2023-11-26T07:16:58.830 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.830 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:16:58.830 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:16:58.830 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.830 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:16:58.831 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:58.831 INFO:teuthology.orchestra.run.smithi083.stdout: "entity": "mgr.y", 2023-11-26T07:16:58.831 INFO:teuthology.orchestra.run.smithi083.stdout: "key": "AQBk72Jl3qkjMBAA+jcxYuDG/UHhB7W7HDwmaw==", 2023-11-26T07:16:58.831 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": { 2023-11-26T07:16:58.831 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": "allow *", 2023-11-26T07:16:58.831 INFO:teuthology.orchestra.run.smithi083.stdout: "mon": "allow profile mgr", 2023-11-26T07:16:58.831 INFO:teuthology.orchestra.run.smithi083.stdout: "osd": "allow *" 2023-11-26T07:16:58.832 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.832 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:58.832 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:16:58.832 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:16:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 -- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa52406aff0 msgr2=0x7fa52406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa52406aff0 0x7fa52406d4b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0097c0 tx=0x7fa53c008030 comp rx=0 tx=0).stop 2023-11-26T07:16:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 -- 172.21.15.83:0/1831436139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401446a0 msgr2=0x7fa540073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401446a0 0x7fa540073f90 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fa53800cbc0 tx=0x7fa53800ced0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 -- 172.21.15.83:0/1831436139 shutdown_connections 2023-11-26T07:16:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa52406aff0 0x7fa52406d4b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5401446a0 0x7fa540073f90 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa54006c710 0x7fa5401eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 --2- 172.21.15.83:0/1831436139 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5400adf00 0x7fa5401f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 -- 172.21.15.83:0/1831436139 >> 172.21.15.83:0/1831436139 conn(0x7fa540071b90 msgr2=0x7fa54015d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 -- 172.21.15.83:0/1831436139 shutdown_connections 2023-11-26T07:16:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.805+0000 7fa547cbf700 1 -- 172.21.15.83:0/1831436139 wait complete. 2023-11-26T07:16:58.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:16:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.953+0000 7ff74c56a700 1 Processor -- start 2023-11-26T07:16:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.953+0000 7ff74c56a700 1 -- start start 2023-11-26T07:16:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.954+0000 7ff74c56a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff7440cf2b0 0x7ff7440c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.954+0000 7ff74c56a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7440cc780 con 0x7ff74415c140 2023-11-26T07:16:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.954+0000 7ff74c56a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7440cc9b0 con 0x7ff74415bd60 2023-11-26T07:16:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.954+0000 7ff74c56a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7440ccc90 con 0x7ff7440cf2b0 2023-11-26T07:16:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.954+0000 7ff749b05700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff74415c140 0x7ff7440cda50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33228/0 (socket says 172.21.15.83:33228) 2023-11-26T07:16:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.954+0000 7ff749b05700 1 -- 172.21.15.83:0/4050632812 learned_addr learned my addr 172.21.15.83:0/4050632812 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff74ab07700 1 --2- 172.21.15.83:0/4050632812 >> v2:172.21.15.203:3300/0 conn(0x7ff7440cf2b0 0x7ff7440c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319177693 0 0) 0x7ff7440cc780 con 0x7ff74415c140 2023-11-26T07:16:58.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7300035b0 con 0x7ff74415c140 2023-11-26T07:16:58.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 187158437 0 0) 0x7ff7300035b0 con 0x7ff74415c140 2023-11-26T07:16:58.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 >> v2:172.21.15.203:3300/0 conn(0x7ff7440cf2b0 msgr2=0x7ff7440c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:58.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 --2- 172.21.15.83:0/4050632812 >> v2:172.21.15.203:3300/0 conn(0x7ff7440cf2b0 0x7ff7440c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 >> v1:172.21.15.203:6789/0 conn(0x7ff74415bd60 legacy=0x7ff7440d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.961 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440ccf70 con 0x7ff74415c140 2023-11-26T07:16:58.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7ff740001670 con 0x7ff74415c140 2023-11-26T07:16:58.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 >> v1:172.21.15.83:6789/0 conn(0x7ff74415c140 legacy=0x7ff7440cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff730003a00 0x7ff730003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff730004850 0x7ff730004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff730005210 0x7ff73000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440ccc90 con 0x7ff730003a00 2023-11-26T07:16:58.962 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73000c0b0 con 0x7ff730004850 2023-11-26T07:16:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73000c270 con 0x7ff730005210 2023-11-26T07:16:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff74a306700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff730003a00 0x7ff730003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749b05700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff730004850 0x7ff730004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff74ab07700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff730005210 0x7ff73000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff74a306700 1 -- 172.21.15.83:0/4050632812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff730005210 msgr2=0x7ff73000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff74a306700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff730005210 0x7ff73000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff74a306700 1 -- 172.21.15.83:0/4050632812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff730004850 msgr2=0x7ff730004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff74a306700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff730004850 0x7ff730004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff74a306700 1 -- 172.21.15.83:0/4050632812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300035b0 con 0x7ff730003a00 2023-11-26T07:16:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff749b05700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff730004850 0x7ff730004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.955+0000 7ff74ab07700 1 -- 172.21.15.83:0/4050632812 reap_dead start 2023-11-26T07:16:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.956+0000 7ff74a306700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff730003a00 0x7ff730003e20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff7340017c0 tx=0x7ff730005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4536f18b20cd31 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.956+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7340052d0 con 0x7ff730003a00 2023-11-26T07:16:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.956+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff734005430 con 0x7ff730003a00 2023-11-26T07:16:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.956+0000 7ff749304700 1 -- 172.21.15.83:0/4050632812 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7340038c0 con 0x7ff730003a00 2023-11-26T07:16:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.956+0000 7ff74c56a700 1 -- 172.21.15.83:0/4050632812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff730003a00 msgr2=0x7ff730003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.956+0000 7ff74c56a700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff730003a00 0x7ff730003e20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff7340017c0 tx=0x7ff730005880 comp rx=0 tx=0).stop 2023-11-26T07:16:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.956+0000 7ff74c56a700 1 -- 172.21.15.83:0/4050632812 shutdown_connections 2023-11-26T07:16:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.956+0000 7ff74c56a700 1 --2- 172.21.15.83:0/4050632812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff730003a00 0x7ff730003e20 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.956+0000 7ff74c56a700 1 -- 172.21.15.83:0/4050632812 >> 172.21.15.83:0/4050632812 conn(0x7ff744073130 msgr2=0x7ff74409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:58.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.956+0000 7ff74c56a700 1 -- 172.21.15.83:0/4050632812 shutdown_connections 2023-11-26T07:16:58.966 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.956+0000 7ff74c56a700 1 -- 172.21.15.83:0/4050632812 wait complete. 2023-11-26T07:16:58.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.958+0000 7ff74c56a700 1 Processor -- start 2023-11-26T07:16:58.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.958+0000 7ff74c56a700 1 -- start start 2023-11-26T07:16:58.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.958+0000 7ff74c56a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff730003a00 0x7ff74406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.958+0000 7ff74c56a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff744159fb0 0x7ff74406c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.958+0000 7ff74c56a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7440c6a20 0x7ff744152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.958+0000 7ff74c56a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74415a430 con 0x7ff7440c6a20 2023-11-26T07:16:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.958+0000 7ff74c56a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74406b5f0 con 0x7ff730003a00 2023-11-26T07:16:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.958+0000 7ff74c56a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74406b8d0 con 0x7ff744159fb0 2023-11-26T07:16:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.958+0000 7ff74ab07700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7440c6a20 0x7ff744152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.958+0000 7ff74ab07700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7440c6a20 0x7ff744152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41436/0 (socket says 172.21.15.83:41436) 2023-11-26T07:16:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.958+0000 7ff74ab07700 1 -- 172.21.15.83:0/3588345587 learned_addr learned my addr 172.21.15.83:0/3588345587 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff74a306700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff730003a00 0x7ff74406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff749b05700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff744159fb0 0x7ff74406c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff74ab07700 1 -- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff744159fb0 msgr2=0x7ff74406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff74ab07700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff744159fb0 0x7ff74406c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.970 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff74ab07700 1 -- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff730003a00 msgr2=0x7ff74406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff74ab07700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff730003a00 0x7ff74406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff74ab07700 1 -- 172.21.15.83:0/3588345587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74406bbb0 con 0x7ff7440c6a20 2023-11-26T07:16:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff749b05700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff744159fb0 0x7ff74406c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff74ab07700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7440c6a20 0x7ff744152e50 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff7440cf720 tx=0x7ff73c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff73affd700 1 -- 172.21.15.83:0/3588345587 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00e070 con 0x7ff7440c6a20 2023-11-26T07:16:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff73affd700 1 -- 172.21.15.83:0/3588345587 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c003cf0 con 0x7ff7440c6a20 2023-11-26T07:16:58.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff73affd700 1 -- 172.21.15.83:0/3588345587 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0117e0 con 0x7ff7440c6a20 2023-11-26T07:16:58.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff74c56a700 1 -- 172.21.15.83:0/3588345587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74406be90 con 0x7ff7440c6a20 2023-11-26T07:16:58.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.959+0000 7ff74c56a700 1 -- 172.21.15.83:0/3588345587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74406cbd0 con 0x7ff7440c6a20 2023-11-26T07:16:58.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.960+0000 7ff74c56a700 1 -- 172.21.15.83:0/3588345587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74406c4e0 con 0x7ff7440c6a20 2023-11-26T07:16:58.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.960+0000 7ff73affd700 1 -- 172.21.15.83:0/3588345587 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7ff73c011940 con 0x7ff7440c6a20 2023-11-26T07:16:58.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.961+0000 7ff73affd700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff72806aff0 0x7ff72806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:58.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.961+0000 7ff73affd700 1 -- 172.21.15.83:0/3588345587 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff73c089830 con 0x7ff7440c6a20 2023-11-26T07:16:58.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.961+0000 7ff74a306700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff72806aff0 0x7ff72806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:58.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.962+0000 7ff74a306700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff72806aff0 0x7ff72806d4b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff734000c00 tx=0x7ff734009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:58.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:58.965+0000 7ff73affd700 1 -- 172.21.15.83:0/3588345587 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff73c054970 con 0x7ff7440c6a20 2023-11-26T07:16:59.157 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.155+0000 7ff74c56a700 1 -- 172.21.15.83:0/3588345587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff74406cd60 con 0x7ff7440c6a20 2023-11-26T07:16:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.156+0000 7ff73affd700 1 -- 172.21.15.83:0/3588345587 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+12846 (secure 0 0 0) 0x7ff73c057f90 con 0x7ff7440c6a20 2023-11-26T07:16:59.163 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:59.163 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":98,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:16:58.375941+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:16:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 -- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff72806aff0 msgr2=0x7ff72806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff72806aff0 0x7ff72806d4b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff734000c00 tx=0x7ff734009030 comp rx=0 tx=0).stop 2023-11-26T07:16:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 -- 172.21.15.83:0/3588345587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7440c6a20 msgr2=0x7ff744152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7440c6a20 0x7ff744152e50 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff7440cf720 tx=0x7ff73c009f70 comp rx=0 tx=0).stop 2023-11-26T07:16:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 -- 172.21.15.83:0/3588345587 shutdown_connections 2023-11-26T07:16:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff72806aff0 0x7ff72806d4b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7440c6a20 0x7ff744152e50 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff730003a00 0x7ff74406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 --2- 172.21.15.83:0/3588345587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff744159fb0 0x7ff74406c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 -- 172.21.15.83:0/3588345587 >> 172.21.15.83:0/3588345587 conn(0x7ff744073130 msgr2=0x7ff7440c75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 -- 172.21.15.83:0/3588345587 shutdown_connections 2023-11-26T07:16:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.158+0000 7ff74c56a700 1 -- 172.21.15.83:0/3588345587 wait complete. 2023-11-26T07:16:59.172 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:16:59.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:16:59.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.293+0000 7fbc450b4700 1 Processor -- start 2023-11-26T07:16:59.295 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.293+0000 7fbc450b4700 1 -- start start 2023-11-26T07:16:59.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc450b4700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc401587f0 0x7fbc4006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc450b4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbc4015b370 0x7fbc4015b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc450b4700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4006b530 con 0x7fbc400b3c60 2023-11-26T07:16:59.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc450b4700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc400ae5e0 con 0x7fbc401587f0 2023-11-26T07:16:59.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc450b4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4015be30 con 0x7fbc4015b370 2023-11-26T07:16:59.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3ed9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc401587f0 0x7fbc4006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3f59e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fbc400b3c60 0x7fbc400af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33252/0 (socket says 172.21.15.83:33252) 2023-11-26T07:16:59.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3f59e700 1 -- 172.21.15.83:0/2145389858 learned_addr learned my addr 172.21.15.83:0/2145389858 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:59.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3e59c700 1 --2- 172.21.15.83:0/2145389858 >> v2:172.21.15.203:3300/0 conn(0x7fbc4015b370 0x7fbc4015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3ed9d700 1 -- 172.21.15.83:0/2145389858 >> v2:172.21.15.203:3300/0 conn(0x7fbc4015b370 msgr2=0x7fbc4015b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:59.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3ed9d700 1 --2- 172.21.15.83:0/2145389858 >> v2:172.21.15.203:3300/0 conn(0x7fbc4015b370 0x7fbc4015b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3ed9d700 1 -- 172.21.15.83:0/2145389858 >> v1:172.21.15.83:6789/0 conn(0x7fbc400b3c60 legacy=0x7fbc400af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.298 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3dd9b700 1 -- 172.21.15.83:0/2145389858 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 514028293 0 0) 0x7fbc4006b530 con 0x7fbc400b3c60 2023-11-26T07:16:59.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3ed9d700 1 -- 172.21.15.83:0/2145389858 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4015c110 con 0x7fbc401587f0 2023-11-26T07:16:59.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3ed9d700 1 --2- 172.21.15.83:0/2145389858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc401587f0 0x7fbc4006aff0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009840 tx=0x7fbc34009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5501babe7510d45c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:59.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3dd9b700 1 -- 172.21.15.83:0/2145389858 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34011830 con 0x7fbc401587f0 2023-11-26T07:16:59.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3dd9b700 1 -- 172.21.15.83:0/2145389858 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc34011990 con 0x7fbc401587f0 2023-11-26T07:16:59.299 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.294+0000 7fbc3dd9b700 1 -- 172.21.15.83:0/2145389858 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34011d20 con 0x7fbc401587f0 2023-11-26T07:16:59.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.295+0000 7fbc450b4700 1 -- 172.21.15.83:0/2145389858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc401587f0 msgr2=0x7fbc4006aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.295+0000 7fbc450b4700 1 --2- 172.21.15.83:0/2145389858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc401587f0 0x7fbc4006aff0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009840 tx=0x7fbc34009b50 comp rx=0 tx=0).stop 2023-11-26T07:16:59.302 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.295+0000 7fbc450b4700 1 -- 172.21.15.83:0/2145389858 shutdown_connections 2023-11-26T07:16:59.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.295+0000 7fbc450b4700 1 --2- 172.21.15.83:0/2145389858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc401587f0 0x7fbc4006aff0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.295+0000 7fbc450b4700 1 --2- 172.21.15.83:0/2145389858 >> v2:172.21.15.203:3300/0 conn(0x7fbc4015b370 0x7fbc4015b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.295+0000 7fbc450b4700 1 -- 172.21.15.83:0/2145389858 >> 172.21.15.83:0/2145389858 conn(0x7fbc400d0bf0 msgr2=0x7fbc400d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:59.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.295+0000 7fbc450b4700 1 -- 172.21.15.83:0/2145389858 shutdown_connections 2023-11-26T07:16:59.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.295+0000 7fbc450b4700 1 -- 172.21.15.83:0/2145389858 wait complete. 2023-11-26T07:16:59.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.296+0000 7fbc450b4700 1 Processor -- start 2023-11-26T07:16:59.303 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.296+0000 7fbc450b4700 1 -- start start 2023-11-26T07:16:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.297+0000 7fbc450b4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc400b3c60 0x7fbc4015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.297+0000 7fbc450b4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc401587f0 0x7fbc4006ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.297+0000 7fbc450b4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4015b370 0x7fbc4006f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.297+0000 7fbc450b4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc401f4990 con 0x7fbc4015b370 2023-11-26T07:16:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.297+0000 7fbc450b4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc401f0410 con 0x7fbc400b3c60 2023-11-26T07:16:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.297+0000 7fbc450b4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc401f3bd0 con 0x7fbc401587f0 2023-11-26T07:16:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.297+0000 7fbc3f59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4015b370 0x7fbc4006f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.297+0000 7fbc3f59e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4015b370 0x7fbc4006f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41450/0 (socket says 172.21.15.83:41450) 2023-11-26T07:16:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.297+0000 7fbc3f59e700 1 -- 172.21.15.83:0/191932115 learned_addr learned my addr 172.21.15.83:0/191932115 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.297+0000 7fbc3ed9d700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc400b3c60 0x7fbc4015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc3e59c700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc401587f0 0x7fbc4006ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc3f59e700 1 -- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc401587f0 msgr2=0x7fbc4006ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc3f59e700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc401587f0 0x7fbc4006ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc3f59e700 1 -- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc400b3c60 msgr2=0x7fbc4015ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc3f59e700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc400b3c60 0x7fbc4015ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc3f59e700 1 -- 172.21.15.83:0/191932115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc340094a0 con 0x7fbc4015b370 2023-11-26T07:16:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc3e59c700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc401587f0 0x7fbc4006ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:16:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc3f59e700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4015b370 0x7fbc4006f460 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800cc70 tx=0x7fbc3800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc277fe700 1 -- 172.21.15.83:0/191932115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3800ce70 con 0x7fbc4015b370 2023-11-26T07:16:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc277fe700 1 -- 172.21.15.83:0/191932115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3800aba0 con 0x7fbc4015b370 2023-11-26T07:16:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc450b4700 1 -- 172.21.15.83:0/191932115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc401f3eb0 con 0x7fbc4015b370 2023-11-26T07:16:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc277fe700 1 -- 172.21.15.83:0/191932115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc380070d0 con 0x7fbc4015b370 2023-11-26T07:16:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.298+0000 7fbc450b4700 1 -- 172.21.15.83:0/191932115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc401f4470 con 0x7fbc4015b370 2023-11-26T07:16:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.299+0000 7fbc450b4700 1 -- 172.21.15.83:0/191932115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc40068fb0 con 0x7fbc4015b370 2023-11-26T07:16:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.299+0000 7fbc277fe700 1 -- 172.21.15.83:0/191932115 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fbc38007230 con 0x7fbc4015b370 2023-11-26T07:16:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.300+0000 7fbc277fe700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbc2c06aff0 0x7fbc2c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.300+0000 7fbc277fe700 1 -- 172.21.15.83:0/191932115 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbc3808ba20 con 0x7fbc4015b370 2023-11-26T07:16:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.300+0000 7fbc3ed9d700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbc2c06aff0 0x7fbc2c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.301+0000 7fbc3ed9d700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbc2c06aff0 0x7fbc2c06d4b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc340090b0 tx=0x7fbc34008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:59.308 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.304+0000 7fbc277fe700 1 -- 172.21.15.83:0/191932115 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc38056d10 con 0x7fbc4015b370 2023-11-26T07:16:59.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.492+0000 7fbc450b4700 1 -- 172.21.15.83:0/191932115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc4006be20 con 0x7fbc4015b370 2023-11-26T07:16:59.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.493+0000 7fbc277fe700 1 -- 172.21.15.83:0/191932115 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc38015070 con 0x7fbc4015b370 2023-11-26T07:16:59.495 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:16:59.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 -- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbc2c06aff0 msgr2=0x7fbc2c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbc2c06aff0 0x7fbc2c06d4b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc340090b0 tx=0x7fbc34008030 comp rx=0 tx=0).stop 2023-11-26T07:16:59.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 -- 172.21.15.83:0/191932115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4015b370 msgr2=0x7fbc4006f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4015b370 0x7fbc4006f460 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800cc70 tx=0x7fbc3800a340 comp rx=0 tx=0).stop 2023-11-26T07:16:59.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 -- 172.21.15.83:0/191932115 shutdown_connections 2023-11-26T07:16:59.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fbc2c06aff0 0x7fbc2c06d4b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4015b370 0x7fbc4006f460 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc400b3c60 0x7fbc4015ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 --2- 172.21.15.83:0/191932115 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc401587f0 0x7fbc4006ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 -- 172.21.15.83:0/191932115 >> 172.21.15.83:0/191932115 conn(0x7fbc400d0bf0 msgr2=0x7fbc400b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:59.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 -- 172.21.15.83:0/191932115 shutdown_connections 2023-11-26T07:16:59.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.495+0000 7fbc450b4700 1 -- 172.21.15.83:0/191932115 wait complete. 2023-11-26T07:16:59.509 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:16:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.628+0000 7f62f3309700 1 Processor -- start 2023-11-26T07:16:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.628+0000 7f62f3309700 1 -- start start 2023-11-26T07:16:59.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.629+0000 7f62f3309700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62ec15ddd0 0x7f62ec0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.629+0000 7f62f3309700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec0b1d40 con 0x7f62ec15e1b0 2023-11-26T07:16:59.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.629+0000 7f62f3309700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec0b28c0 con 0x7f62ec0d2240 2023-11-26T07:16:59.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.629+0000 7f62f3309700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec0b1f90 con 0x7f62ec15ddd0 2023-11-26T07:16:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.630+0000 7f62f10a5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62ec15ddd0 0x7f62ec0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.630+0000 7f62f10a5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62ec15ddd0 0x7f62ec0b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41454/0 (socket says 172.21.15.83:41454) 2023-11-26T07:16:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.630+0000 7f62f08a4700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f62ec15e1b0 0x7f62ec0d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33262/0 (socket says 172.21.15.83:33262) 2023-11-26T07:16:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.630+0000 7f62f10a5700 1 -- 172.21.15.83:0/1838193238 learned_addr learned my addr 172.21.15.83:0/1838193238 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.630+0000 7f62f10a5700 1 -- 172.21.15.83:0/1838193238 >> v1:172.21.15.203:6789/0 conn(0x7f62ec0d2240 legacy=0x7f62ec06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.630+0000 7f62f10a5700 1 -- 172.21.15.83:0/1838193238 >> v1:172.21.15.83:6789/0 conn(0x7f62ec15e1b0 legacy=0x7f62ec0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.630+0000 7f62f10a5700 1 -- 172.21.15.83:0/1838193238 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec0b2270 con 0x7f62ec15ddd0 2023-11-26T07:16:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.630+0000 7f62f10a5700 1 --2- 172.21.15.83:0/1838193238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec15ddd0 0x7f62ec0b1680 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009920 tx=0x7f62dc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f97b6d9a46fce8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.630+0000 7f62e3fff700 1 -- 172.21.15.83:0/1838193238 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc004230 con 0x7f62ec15ddd0 2023-11-26T07:16:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.630+0000 7f62e3fff700 1 -- 172.21.15.83:0/1838193238 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62dc004390 con 0x7f62ec15ddd0 2023-11-26T07:16:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.630+0000 7f62e3fff700 1 -- 172.21.15.83:0/1838193238 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc009e70 con 0x7f62ec15ddd0 2023-11-26T07:16:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.631+0000 7f62f3309700 1 -- 172.21.15.83:0/1838193238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec15ddd0 msgr2=0x7f62ec0b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.631+0000 7f62f3309700 1 --2- 172.21.15.83:0/1838193238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec15ddd0 0x7f62ec0b1680 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009920 tx=0x7f62dc009c30 comp rx=0 tx=0).stop 2023-11-26T07:16:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.631+0000 7f62f3309700 1 -- 172.21.15.83:0/1838193238 shutdown_connections 2023-11-26T07:16:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.631+0000 7f62f3309700 1 --2- 172.21.15.83:0/1838193238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec15ddd0 0x7f62ec0b1680 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.631+0000 7f62f3309700 1 -- 172.21.15.83:0/1838193238 >> 172.21.15.83:0/1838193238 conn(0x7f62ec0731a0 msgr2=0x7f62ec09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.631+0000 7f62f3309700 1 -- 172.21.15.83:0/1838193238 shutdown_connections 2023-11-26T07:16:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.631+0000 7f62f3309700 1 -- 172.21.15.83:0/1838193238 wait complete. 2023-11-26T07:16:59.638 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.632+0000 7f62f3309700 1 Processor -- start 2023-11-26T07:16:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.632+0000 7f62f3309700 1 -- start start 2023-11-26T07:16:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f3309700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62ec0d2240 0x7f62ec0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f3309700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec15ddd0 0x7f62ec09bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f3309700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec15e1b0 0x7f62ec14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f3309700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec09c140 con 0x7f62ec15ddd0 2023-11-26T07:16:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f3309700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec09cf40 con 0x7f62ec15e1b0 2023-11-26T07:16:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f3309700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec09c330 con 0x7f62ec0d2240 2023-11-26T07:16:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f08a4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec15ddd0 0x7f62ec09bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f08a4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec15ddd0 0x7f62ec09bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41462/0 (socket says 172.21.15.83:41462) 2023-11-26T07:16:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f08a4700 1 -- 172.21.15.83:0/3523351456 learned_addr learned my addr 172.21.15.83:0/3523351456 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f08a4700 1 -- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62ec0d2240 msgr2=0x7f62ec0b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f08a4700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62ec0d2240 0x7f62ec0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f08a4700 1 -- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec15e1b0 msgr2=0x7f62ec14d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f08a4700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec15e1b0 0x7f62ec14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f08a4700 1 -- 172.21.15.83:0/3523351456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc009550 con 0x7f62ec15ddd0 2023-11-26T07:16:59.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f08a4700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec15ddd0 0x7f62ec09bc00 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f62e800c430 tx=0x7f62e800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:59.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62e1ffb700 1 -- 172.21.15.83:0/3523351456 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e8005cb0 con 0x7f62ec15ddd0 2023-11-26T07:16:59.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62e1ffb700 1 -- 172.21.15.83:0/3523351456 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e8005e10 con 0x7f62ec15ddd0 2023-11-26T07:16:59.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62f3309700 1 -- 172.21.15.83:0/3523351456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec0b2ed0 con 0x7f62ec15ddd0 2023-11-26T07:16:59.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.633+0000 7f62e1ffb700 1 -- 172.21.15.83:0/3523351456 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e80072d0 con 0x7f62ec15ddd0 2023-11-26T07:16:59.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.634+0000 7f62f3309700 1 -- 172.21.15.83:0/3523351456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec09c8f0 con 0x7f62ec15ddd0 2023-11-26T07:16:59.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.635+0000 7f62f3309700 1 -- 172.21.15.83:0/3523351456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec09dba0 con 0x7f62ec15ddd0 2023-11-26T07:16:59.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.635+0000 7f62e1ffb700 1 -- 172.21.15.83:0/3523351456 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f62e8007430 con 0x7f62ec15ddd0 2023-11-26T07:16:59.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.635+0000 7f62e1ffb700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62d006aff0 0x7f62d006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.635+0000 7f62e1ffb700 1 -- 172.21.15.83:0/3523351456 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f62e8088620 con 0x7f62ec15ddd0 2023-11-26T07:16:59.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.636+0000 7f62f10a5700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62d006aff0 0x7f62d006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.636+0000 7f62f10a5700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62d006aff0 0x7f62d006d4b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f62dc000c00 tx=0x7f62dc010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:59.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.640+0000 7f62e1ffb700 1 -- 172.21.15.83:0/3523351456 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62e8053a50 con 0x7f62ec15ddd0 2023-11-26T07:16:59.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.834+0000 7f62f3309700 1 -- 172.21.15.83:0/3523351456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f62ec0b2d30 con 0x7f62ec15ddd0 2023-11-26T07:16:59.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.834+0000 7f62e1ffb700 1 -- 172.21.15.83:0/3523351456 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) v1 ==== 84+0+65 (secure 0 0 0) 0x7f62ec0b2d30 con 0x7f62ec15ddd0 2023-11-26T07:16:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:16:59.837 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:16:59.837 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:16:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2023-11-26T07:16:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2023-11-26T07:16:59.838 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:16:59.838 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:16:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.836+0000 7f62f3309700 1 -- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62d006aff0 msgr2=0x7f62d006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.836+0000 7f62f3309700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62d006aff0 0x7f62d006d4b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f62dc000c00 tx=0x7f62dc010000 comp rx=0 tx=0).stop 2023-11-26T07:16:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.836+0000 7f62f3309700 1 -- 172.21.15.83:0/3523351456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec15ddd0 msgr2=0x7f62ec09bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.836+0000 7f62f3309700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec15ddd0 0x7f62ec09bc00 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f62e800c430 tx=0x7f62e800c7f0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.836+0000 7f62f3309700 1 -- 172.21.15.83:0/3523351456 shutdown_connections 2023-11-26T07:16:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.836+0000 7f62f3309700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f62d006aff0 0x7f62d006d4b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.836+0000 7f62f3309700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec15e1b0 0x7f62ec14d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.836+0000 7f62f3309700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62ec0d2240 0x7f62ec0b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.836+0000 7f62f3309700 1 --2- 172.21.15.83:0/3523351456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec15ddd0 0x7f62ec09bc00 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.836+0000 7f62f3309700 1 -- 172.21.15.83:0/3523351456 >> 172.21.15.83:0/3523351456 conn(0x7f62ec0731a0 msgr2=0x7f62ec070370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.837+0000 7f62f3309700 1 -- 172.21.15.83:0/3523351456 shutdown_connections 2023-11-26T07:16:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.837+0000 7f62f3309700 1 -- 172.21.15.83:0/3523351456 wait complete. 2023-11-26T07:16:59.851 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:16:59.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:16:59.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.973+0000 7fc6d35a4700 1 Processor -- start 2023-11-26T07:16:59.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.974+0000 7fc6d35a4700 1 -- start start 2023-11-26T07:16:59.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.974+0000 7fc6d35a4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6cc0d1ab0 0x7fc6cc0d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.974+0000 7fc6d35a4700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc6cc0af3b0 0x7fc6cc0af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.974+0000 7fc6d35a4700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6cc0d01c0 con 0x7fc6cc0cf040 2023-11-26T07:16:59.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.974+0000 7fc6d35a4700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc0d2470 con 0x7fc6cc0af3b0 2023-11-26T07:16:59.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.974+0000 7fc6d35a4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc070910 con 0x7fc6cc0d1ab0 2023-11-26T07:16:59.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6d1b41700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc6cc0af3b0 0x7fc6cc0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6d1340700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fc6cc0cf040 0x7fc6cc06fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:46126/0 (socket says 172.21.15.83:46126) 2023-11-26T07:16:59.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6d0b3f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6cc0d1ab0 0x7fc6cc0d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6d1340700 1 -- 172.21.15.83:0/3202207150 learned_addr learned my addr 172.21.15.83:0/3202207150 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:16:59.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6d1b41700 1 -- 172.21.15.83:0/3202207150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6cc0d1ab0 msgr2=0x7fc6cc0d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:16:59.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6d1b41700 1 --2- 172.21.15.83:0/3202207150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6cc0d1ab0 0x7fc6cc0d1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6d1b41700 1 -- 172.21.15.83:0/3202207150 >> v1:172.21.15.203:6789/0 conn(0x7fc6cc0cf040 legacy=0x7fc6cc06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6c3fff700 1 -- 172.21.15.83:0/3202207150 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658935715 0 0) 0x7fc6cc0d01c0 con 0x7fc6cc0cf040 2023-11-26T07:16:59.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6d1b41700 1 -- 172.21.15.83:0/3202207150 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc070bf0 con 0x7fc6cc0af3b0 2023-11-26T07:16:59.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6d1b41700 1 --2- 172.21.15.83:0/3202207150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6cc0af3b0 0x7fc6cc0af800 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009d90 tx=0x7fc6c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1531754e6f536b95 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:59.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6c3fff700 1 -- 172.21.15.83:0/3202207150 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c400e070 con 0x7fc6cc0af3b0 2023-11-26T07:16:59.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6c3fff700 1 -- 172.21.15.83:0/3202207150 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c400bb20 con 0x7fc6cc0af3b0 2023-11-26T07:16:59.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.975+0000 7fc6c3fff700 1 -- 172.21.15.83:0/3202207150 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4012430 con 0x7fc6cc0af3b0 2023-11-26T07:16:59.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.976+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3202207150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6cc0af3b0 msgr2=0x7fc6cc0af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.976+0000 7fc6d35a4700 1 --2- 172.21.15.83:0/3202207150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6cc0af3b0 0x7fc6cc0af800 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009d90 tx=0x7fc6c400b330 comp rx=0 tx=0).stop 2023-11-26T07:16:59.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.976+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3202207150 shutdown_connections 2023-11-26T07:16:59.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.976+0000 7fc6d35a4700 1 --2- 172.21.15.83:0/3202207150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6cc0af3b0 0x7fc6cc0af800 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.976+0000 7fc6d35a4700 1 --2- 172.21.15.83:0/3202207150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6cc0d1ab0 0x7fc6cc0d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.976+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3202207150 >> 172.21.15.83:0/3202207150 conn(0x7fc6cc09aa00 msgr2=0x7fc6cc09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:16:59.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.976+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3202207150 shutdown_connections 2023-11-26T07:16:59.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.976+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3202207150 wait complete. 2023-11-26T07:16:59.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.977+0000 7fc6d35a4700 1 Processor -- start 2023-11-26T07:16:59.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.977+0000 7fc6d35a4700 1 -- start start 2023-11-26T07:16:59.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d35a4700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6cc0af3b0 0x7fc6cc0c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d35a4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6cc0cf040 0x7fc6cc15c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d35a4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6cc0d1ab0 0x7fc6cc1f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d35a4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0c8ae0 con 0x7fc6cc0cf040 2023-11-26T07:16:59.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d35a4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc15c8f0 con 0x7fc6cc0d1ab0 2023-11-26T07:16:59.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d35a4700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0c8520 con 0x7fc6cc0af3b0 2023-11-26T07:16:59.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d0b3f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6cc0cf040 0x7fc6cc15c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d0b3f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6cc0cf040 0x7fc6cc15c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41476/0 (socket says 172.21.15.83:41476) 2023-11-26T07:16:59.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d0b3f700 1 -- 172.21.15.83:0/3720546752 learned_addr learned my addr 172.21.15.83:0/3720546752 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:16:59.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d1340700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6cc0af3b0 0x7fc6cc0c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d1b41700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6cc0d1ab0 0x7fc6cc1f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d0b3f700 1 -- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6cc0af3b0 msgr2=0x7fc6cc0c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d0b3f700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6cc0af3b0 0x7fc6cc0c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d0b3f700 1 -- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6cc0d1ab0 msgr2=0x7fc6cc1f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:16:59.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d0b3f700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6cc0d1ab0 0x7fc6cc1f1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:16:59.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d0b3f700 1 -- 172.21.15.83:0/3720546752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4009a70 con 0x7fc6cc0cf040 2023-11-26T07:16:59.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.978+0000 7fc6d1b41700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6cc0d1ab0 0x7fc6cc1f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:16:59.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.979+0000 7fc6d0b3f700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6cc0cf040 0x7fc6cc15c3b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800e9b0 tx=0x7fc6c800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:59.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.979+0000 7fc6c1ffb700 1 -- 172.21.15.83:0/3720546752 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8009710 con 0x7fc6cc0cf040 2023-11-26T07:16:59.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.979+0000 7fc6c1ffb700 1 -- 172.21.15.83:0/3720546752 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c800c850 con 0x7fc6cc0cf040 2023-11-26T07:16:59.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.979+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3720546752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc0c8800 con 0x7fc6cc0cf040 2023-11-26T07:16:59.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.979+0000 7fc6c1ffb700 1 -- 172.21.15.83:0/3720546752 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c800cbc0 con 0x7fc6cc0cf040 2023-11-26T07:16:59.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.979+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3720546752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc1f19e0 con 0x7fc6cc0cf040 2023-11-26T07:16:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.980+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3720546752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc067f60 con 0x7fc6cc0cf040 2023-11-26T07:16:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.980+0000 7fc6c1ffb700 1 -- 172.21.15.83:0/3720546752 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fc6c800cd20 con 0x7fc6cc0cf040 2023-11-26T07:16:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.981+0000 7fc6c1ffb700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc6b006aff0 0x7fc6b006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:16:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.981+0000 7fc6c1ffb700 1 -- 172.21.15.83:0/3720546752 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc6c8089550 con 0x7fc6cc0cf040 2023-11-26T07:16:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.981+0000 7fc6d1340700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc6b006aff0 0x7fc6b006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:16:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.981+0000 7fc6d1340700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc6b006aff0 0x7fc6b006d4b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000f60 tx=0x7fc6bc007be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:16:59.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:16:59.985+0000 7fc6c1ffb700 1 -- 172.21.15.83:0/3720546752 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc6c8054840 con 0x7fc6cc0cf040 2023-11-26T07:17:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.174+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3720546752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc6cc0b0620 con 0x7fc6cc0cf040 2023-11-26T07:17:00.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.559+0000 7fc6c1ffb700 1 -- 172.21.15.83:0/3720546752 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v99) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc6c8057e60 con 0x7fc6cc0cf040 2023-11-26T07:17:00.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc6b006aff0 msgr2=0x7fc6b006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:00.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc6b006aff0 0x7fc6b006d4b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000f60 tx=0x7fc6bc007be0 comp rx=0 tx=0).stop 2023-11-26T07:17:00.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3720546752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6cc0cf040 msgr2=0x7fc6cc15c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:00.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6cc0cf040 0x7fc6cc15c3b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800e9b0 tx=0x7fc6c800ed70 comp rx=0 tx=0).stop 2023-11-26T07:17:00.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3720546752 shutdown_connections 2023-11-26T07:17:00.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc6b006aff0 0x7fc6b006d4b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:00.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6cc0d1ab0 0x7fc6cc1f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:00.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6cc0af3b0 0x7fc6cc0c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:00.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 --2- 172.21.15.83:0/3720546752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6cc0cf040 0x7fc6cc15c3b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:00.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3720546752 >> 172.21.15.83:0/3720546752 conn(0x7fc6cc09aa00 msgr2=0x7fc6cc15f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:00.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3720546752 shutdown_connections 2023-11-26T07:17:00.567 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.563+0000 7fc6d35a4700 1 -- 172.21.15.83:0/3720546752 wait complete. 2023-11-26T07:17:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2023-11-26T07:17:00.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:17:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.707+0000 7f087d39d700 1 Processor -- start 2023-11-26T07:17:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.707+0000 7f087d39d700 1 -- start start 2023-11-26T07:17:00.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.707+0000 7f087d39d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f08780cf240 0x7f08780d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.707+0000 7f087d39d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08780d3970 0x7f08780b5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.707+0000 7f087d39d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08780d1aa0 con 0x7f087815e1d0 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.707+0000 7f087d39d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08780d1cf0 con 0x7f08780cf240 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.707+0000 7f087d39d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08780d1fd0 con 0x7f08780d3970 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f08767fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f08780cf240 0x7f08780d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f0876ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f087815e1d0 0x7f08780d2ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33290/0 (socket says 172.21.15.83:33290) 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f08767fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f08780cf240 0x7f08780d3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41484/0 (socket says 172.21.15.83:41484) 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f0876ffd700 1 -- 172.21.15.83:0/2912244331 learned_addr learned my addr 172.21.15.83:0/2912244331 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f08777fe700 1 --2- 172.21.15.83:0/2912244331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08780d3970 0x7f08780b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f08767fc700 1 -- 172.21.15.83:0/2912244331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08780d3970 msgr2=0x7f08780b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f08767fc700 1 --2- 172.21.15.83:0/2912244331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08780d3970 0x7f08780b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f08767fc700 1 -- 172.21.15.83:0/2912244331 >> v1:172.21.15.83:6789/0 conn(0x7f087815e1d0 legacy=0x7f08780d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f08767fc700 1 -- 172.21.15.83:0/2912244331 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780d22b0 con 0x7f08780cf240 2023-11-26T07:17:00.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f0875ffb700 1 -- 172.21.15.83:0/2912244331 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237981349 0 0) 0x7f08780d1aa0 con 0x7f087815e1d0 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f08777fe700 1 --2- 172.21.15.83:0/2912244331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08780d3970 0x7f08780b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f08767fc700 1 --2- 172.21.15.83:0/2912244331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08780cf240 0x7f08780d3430 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f086c009840 tx=0x7f086c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e23139929405d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f0875ffb700 1 -- 172.21.15.83:0/2912244331 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c011830 con 0x7f08780cf240 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f0875ffb700 1 -- 172.21.15.83:0/2912244331 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086c011990 con 0x7f08780cf240 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.708+0000 7f0875ffb700 1 -- 172.21.15.83:0/2912244331 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c011d20 con 0x7f08780cf240 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.709+0000 7f087d39d700 1 -- 172.21.15.83:0/2912244331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08780cf240 msgr2=0x7f08780d3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.709+0000 7f087d39d700 1 --2- 172.21.15.83:0/2912244331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08780cf240 0x7f08780d3430 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f086c009840 tx=0x7f086c009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.709+0000 7f087d39d700 1 -- 172.21.15.83:0/2912244331 shutdown_connections 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.709+0000 7f087d39d700 1 --2- 172.21.15.83:0/2912244331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08780d3970 0x7f08780b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.709+0000 7f087d39d700 1 --2- 172.21.15.83:0/2912244331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08780cf240 0x7f08780d3430 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.709+0000 7f087d39d700 1 -- 172.21.15.83:0/2912244331 >> 172.21.15.83:0/2912244331 conn(0x7f08780d0950 msgr2=0x7f08780d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.709+0000 7f087d39d700 1 -- 172.21.15.83:0/2912244331 shutdown_connections 2023-11-26T07:17:00.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.709+0000 7f087d39d700 1 -- 172.21.15.83:0/2912244331 wait complete. 2023-11-26T07:17:00.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.710+0000 7f087d39d700 1 Processor -- start 2023-11-26T07:17:00.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.710+0000 7f087d39d700 1 -- start start 2023-11-26T07:17:00.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f087d39d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08780cf240 0x7f08780c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f087d39d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08780d3970 0x7f087815f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f087d39d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087815e1d0 0x7f087815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f087d39d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08780c8200 con 0x7f087815e1d0 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f087d39d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08780ab0a0 con 0x7f08780cf240 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f087d39d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087815cbf0 con 0x7f08780d3970 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f08777fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087815e1d0 0x7f087815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f08777fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087815e1d0 0x7f087815fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41492/0 (socket says 172.21.15.83:41492) 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f08777fe700 1 -- 172.21.15.83:0/639561899 learned_addr learned my addr 172.21.15.83:0/639561899 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f08767fc700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08780d3970 0x7f087815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f0876ffd700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08780cf240 0x7f08780c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f08777fe700 1 -- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08780d3970 msgr2=0x7f087815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f08777fe700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08780d3970 0x7f087815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f08777fe700 1 -- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08780cf240 msgr2=0x7f08780c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f08777fe700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08780cf240 0x7f08780c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f08777fe700 1 -- 172.21.15.83:0/639561899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c0094a0 con 0x7f087815e1d0 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.711+0000 7f0876ffd700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08780cf240 0x7f08780c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.712+0000 7f08777fe700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087815e1d0 0x7f087815fcc0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f087000cef0 tx=0x7f087000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.712+0000 7f085f7fe700 1 -- 172.21.15.83:0/639561899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000adf0 con 0x7f087815e1d0 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.712+0000 7f085f7fe700 1 -- 172.21.15.83:0/639561899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0870015440 con 0x7f087815e1d0 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.712+0000 7f087d39d700 1 -- 172.21.15.83:0/639561899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08780aadc0 con 0x7f087815e1d0 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.712+0000 7f085f7fe700 1 -- 172.21.15.83:0/639561899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700157f0 con 0x7f087815e1d0 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.712+0000 7f087d39d700 1 -- 172.21.15.83:0/639561899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087814d780 con 0x7f087815e1d0 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.713+0000 7f087d39d700 1 -- 172.21.15.83:0/639561899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087814d300 con 0x7f087815e1d0 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.713+0000 7f085f7fe700 1 -- 172.21.15.83:0/639561899 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f087000d430 con 0x7f087815e1d0 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.713+0000 7f085f7fe700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f086406aff0 0x7f086406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.714+0000 7f085f7fe700 1 -- 172.21.15.83:0/639561899 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f0870089ae0 con 0x7f087815e1d0 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.714+0000 7f0876ffd700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f086406aff0 0x7f086406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:00.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.714+0000 7f0876ffd700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f086406aff0 0x7f086406d4b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0868007fd0 tx=0x7f0868009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:00.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.718+0000 7f085f7fe700 1 -- 172.21.15.83:0/639561899 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f087814d300 con 0x7f087815e1d0 2023-11-26T07:17:00.909 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:00.906+0000 7f087d39d700 1 -- 172.21.15.83:0/639561899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f087814d300 con 0x7f087815e1d0 2023-11-26T07:17:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.565+0000 7f085f7fe700 1 -- 172.21.15.83:0/639561899 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v100) v1 ==== 170+0+0 (secure 0 0 0) 0x7f087814d300 con 0x7f087815e1d0 2023-11-26T07:17:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.569+0000 7f087d39d700 1 -- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f086406aff0 msgr2=0x7f086406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.569+0000 7f087d39d700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f086406aff0 0x7f086406d4b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0868007fd0 tx=0x7f0868009040 comp rx=0 tx=0).stop 2023-11-26T07:17:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.569+0000 7f087d39d700 1 -- 172.21.15.83:0/639561899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087815e1d0 msgr2=0x7f087815fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.569+0000 7f087d39d700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087815e1d0 0x7f087815fcc0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f087000cef0 tx=0x7f087000a340 comp rx=0 tx=0).stop 2023-11-26T07:17:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.569+0000 7f087d39d700 1 -- 172.21.15.83:0/639561899 shutdown_connections 2023-11-26T07:17:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.569+0000 7f087d39d700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f086406aff0 0x7f086406d4b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.569+0000 7f087d39d700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087815e1d0 0x7f087815fcc0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.569+0000 7f087d39d700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08780cf240 0x7f08780c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.569+0000 7f087d39d700 1 --2- 172.21.15.83:0/639561899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f08780d3970 0x7f087815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.569+0000 7f087d39d700 1 -- 172.21.15.83:0/639561899 >> 172.21.15.83:0/639561899 conn(0x7f08780d0950 msgr2=0x7f08780b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:01.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.569+0000 7f087d39d700 1 -- 172.21.15.83:0/639561899 shutdown_connections 2023-11-26T07:17:01.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.570+0000 7f087d39d700 1 -- 172.21.15.83:0/639561899 wait complete. 2023-11-26T07:17:01.585 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2023-11-26T07:17:01.601 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:17:01.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.724+0000 7fd808c98700 1 Processor -- start 2023-11-26T07:17:01.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.724+0000 7fd808c98700 1 -- start start 2023-11-26T07:17:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.725+0000 7fd808c98700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd80415e160 0x7fd8040b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.725+0000 7fd808c98700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8040746a0 0x7fd80406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.725+0000 7fd808c98700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8040b3c60 con 0x7fd8040afc90 2023-11-26T07:17:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.725+0000 7fd808c98700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8040b4500 con 0x7fd80415e160 2023-11-26T07:17:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.725+0000 7fd808c98700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040b58c0 con 0x7fd8040746a0 2023-11-26T07:17:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.725+0000 7fd80259c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd80415e160 0x7fd8040b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.725+0000 7fd802d9d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8040746a0 0x7fd80406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.725+0000 7fd801d9b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd8040afc90 0x7fd804073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:46156/0 (socket says 172.21.15.83:46156) 2023-11-26T07:17:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.725+0000 7fd801d9b700 1 -- 172.21.15.83:0/3129359936 learned_addr learned my addr 172.21.15.83:0/3129359936 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.726+0000 7fd80159a700 1 -- 172.21.15.83:0/3129359936 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998388956 0 0) 0x7fd8040b3c60 con 0x7fd8040afc90 2023-11-26T07:17:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.726+0000 7fd80159a700 1 -- 172.21.15.83:0/3129359936 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7ec0035b0 con 0x7fd8040afc90 2023-11-26T07:17:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.726+0000 7fd802d9d700 1 -- 172.21.15.83:0/3129359936 >> v2:172.21.15.203:3300/0 conn(0x7fd80415e160 msgr2=0x7fd8040b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.726+0000 7fd802d9d700 1 --2- 172.21.15.83:0/3129359936 >> v2:172.21.15.203:3300/0 conn(0x7fd80415e160 0x7fd8040b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.726+0000 7fd802d9d700 1 -- 172.21.15.83:0/3129359936 >> v1:172.21.15.203:6789/0 conn(0x7fd8040afc90 legacy=0x7fd804073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.726+0000 7fd802d9d700 1 -- 172.21.15.83:0/3129359936 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8040b0140 con 0x7fd8040746a0 2023-11-26T07:17:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.726+0000 7fd80259c700 1 --2- 172.21.15.83:0/3129359936 >> v2:172.21.15.203:3300/0 conn(0x7fd80415e160 0x7fd8040b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.726+0000 7fd802d9d700 1 --2- 172.21.15.83:0/3129359936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8040746a0 0x7fd80406d930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009d90 tx=0x7fd7fc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d084750e10de0b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.726+0000 7fd80159a700 1 -- 172.21.15.83:0/3129359936 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc00e070 con 0x7fd8040746a0 2023-11-26T07:17:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.726+0000 7fd80159a700 1 -- 172.21.15.83:0/3129359936 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc00bb20 con 0x7fd8040746a0 2023-11-26T07:17:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.726+0000 7fd80159a700 1 -- 172.21.15.83:0/3129359936 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc00be90 con 0x7fd8040746a0 2023-11-26T07:17:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.727+0000 7fd808c98700 1 -- 172.21.15.83:0/3129359936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8040746a0 msgr2=0x7fd80406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.727+0000 7fd808c98700 1 --2- 172.21.15.83:0/3129359936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8040746a0 0x7fd80406d930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009d90 tx=0x7fd7fc00b330 comp rx=0 tx=0).stop 2023-11-26T07:17:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.727+0000 7fd808c98700 1 -- 172.21.15.83:0/3129359936 shutdown_connections 2023-11-26T07:17:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.727+0000 7fd808c98700 1 --2- 172.21.15.83:0/3129359936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8040746a0 0x7fd80406d930 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.727+0000 7fd808c98700 1 --2- 172.21.15.83:0/3129359936 >> v2:172.21.15.203:3300/0 conn(0x7fd80415e160 0x7fd8040b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.727+0000 7fd808c98700 1 -- 172.21.15.83:0/3129359936 >> 172.21.15.83:0/3129359936 conn(0x7fd8040d0bf0 msgr2=0x7fd8040d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.727+0000 7fd808c98700 1 -- 172.21.15.83:0/3129359936 shutdown_connections 2023-11-26T07:17:01.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.727+0000 7fd808c98700 1 -- 172.21.15.83:0/3129359936 wait complete. 2023-11-26T07:17:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.728+0000 7fd808c98700 1 Processor -- start 2023-11-26T07:17:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.728+0000 7fd808c98700 1 -- start start 2023-11-26T07:17:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd808c98700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8040746a0 0x7fd80415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd808c98700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8040afc90 0x7fd8041601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd808c98700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd80415e160 0x7fd8041f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd808c98700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8041606f0 con 0x7fd80415e160 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd808c98700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8041608e0 con 0x7fd8040afc90 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd808c98700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd804160bc0 con 0x7fd8040746a0 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd802d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd80415e160 0x7fd8041f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd802d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd80415e160 0x7fd8041f30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41506/0 (socket says 172.21.15.83:41506) 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd802d9d700 1 -- 172.21.15.83:0/369102258 learned_addr learned my addr 172.21.15.83:0/369102258 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd80259c700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8040746a0 0x7fd80415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd801d9b700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8040afc90 0x7fd8041601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd802d9d700 1 -- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8040746a0 msgr2=0x7fd80415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd802d9d700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8040746a0 0x7fd80415d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd802d9d700 1 -- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8040afc90 msgr2=0x7fd8041601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd802d9d700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8040afc90 0x7fd8041601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd802d9d700 1 -- 172.21.15.83:0/369102258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f40094a0 con 0x7fd80415e160 2023-11-26T07:17:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd801d9b700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8040afc90 0x7fd8041601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.729+0000 7fd802d9d700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd80415e160 0x7fd8041f30a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc014040 tx=0x7fd7fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.730+0000 7fd7f2ffd700 1 -- 172.21.15.83:0/369102258 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0048b0 con 0x7fd80415e160 2023-11-26T07:17:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.730+0000 7fd808c98700 1 -- 172.21.15.83:0/369102258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7fc009a70 con 0x7fd80415e160 2023-11-26T07:17:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.730+0000 7fd7f2ffd700 1 -- 172.21.15.83:0/369102258 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc020070 con 0x7fd80415e160 2023-11-26T07:17:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.730+0000 7fd7f2ffd700 1 -- 172.21.15.83:0/369102258 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0276c0 con 0x7fd80415e160 2023-11-26T07:17:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.730+0000 7fd808c98700 1 -- 172.21.15.83:0/369102258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd804161180 con 0x7fd80415e160 2023-11-26T07:17:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.731+0000 7fd808c98700 1 -- 172.21.15.83:0/369102258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd804161370 con 0x7fd80415e160 2023-11-26T07:17:01.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.731+0000 7fd7f2ffd700 1 -- 172.21.15.83:0/369102258 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fd7fc031430 con 0x7fd80415e160 2023-11-26T07:17:01.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.732+0000 7fd7f2ffd700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd7e406aff0 0x7fd7e406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:01.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.732+0000 7fd7f2ffd700 1 -- 172.21.15.83:0/369102258 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd7fc022070 con 0x7fd80415e160 2023-11-26T07:17:01.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.732+0000 7fd80259c700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd7e406aff0 0x7fd7e406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:01.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.732+0000 7fd80259c700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd7e406aff0 0x7fd7e406d4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009cd0 tx=0x7fd7f4009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.736+0000 7fd7f2ffd700 1 -- 172.21.15.83:0/369102258 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd7fc056a60 con 0x7fd80415e160 2023-11-26T07:17:01.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:01.972+0000 7fd808c98700 1 -- 172.21.15.83:0/369102258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd804068fb0 con 0x7fd80415e160 2023-11-26T07:17:02.582 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.580+0000 7fd7f2ffd700 1 -- 172.21.15.83:0/369102258 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v65) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd7fc02c0c0 con 0x7fd80415e160 2023-11-26T07:17:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.583+0000 7fd808c98700 1 -- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd7e406aff0 msgr2=0x7fd7e406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.583+0000 7fd808c98700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd7e406aff0 0x7fd7e406d4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009cd0 tx=0x7fd7f4009210 comp rx=0 tx=0).stop 2023-11-26T07:17:02.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.583+0000 7fd808c98700 1 -- 172.21.15.83:0/369102258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd80415e160 msgr2=0x7fd8041f30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.583+0000 7fd808c98700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd80415e160 0x7fd8041f30a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc014040 tx=0x7fd7fc004ad0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.584+0000 7fd808c98700 1 -- 172.21.15.83:0/369102258 shutdown_connections 2023-11-26T07:17:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.584+0000 7fd808c98700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd7e406aff0 0x7fd7e406d4b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.584+0000 7fd808c98700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd80415e160 0x7fd8041f30a0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.584+0000 7fd808c98700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd8040746a0 0x7fd80415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.584+0000 7fd808c98700 1 --2- 172.21.15.83:0/369102258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8040afc90 0x7fd8041601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.584+0000 7fd808c98700 1 -- 172.21.15.83:0/369102258 >> 172.21.15.83:0/369102258 conn(0x7fd8040d0bf0 msgr2=0x7fd8040b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.584+0000 7fd808c98700 1 -- 172.21.15.83:0/369102258 shutdown_connections 2023-11-26T07:17:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.584+0000 7fd808c98700 1 -- 172.21.15.83:0/369102258 wait complete. 2023-11-26T07:17:02.594 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 16 and data pool 17 2023-11-26T07:17:02.606 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:02.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.724+0000 7f260928a700 1 Processor -- start 2023-11-26T07:17:02.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.725+0000 7f260928a700 1 -- start start 2023-11-26T07:17:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.725+0000 7f260928a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f260415a3b0 0x7f26040b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:02.729 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.725+0000 7f260928a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f260415deb0 0x7f260415e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.725+0000 7f260928a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260415a790 con 0x7f26040cf360 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.725+0000 7f260928a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040cf8b0 con 0x7f260415deb0 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.725+0000 7f260928a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040b4c40 con 0x7f260415a3b0 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.725+0000 7f26037fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f260415deb0 0x7f260415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.725+0000 7f26027fc700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f26040cf360 0x7f26040b3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33334/0 (socket says 172.21.15.83:33334) 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.725+0000 7f26037fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f260415deb0 0x7f260415e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41528/0 (socket says 172.21.15.83:41528) 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.725+0000 7f26027fc700 1 -- 172.21.15.83:0/2451021264 learned_addr learned my addr 172.21.15.83:0/2451021264 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.726+0000 7f2602ffd700 1 --2- 172.21.15.83:0/2451021264 >> v2:172.21.15.203:3300/0 conn(0x7f260415a3b0 0x7f26040b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.726+0000 7f26037fe700 1 -- 172.21.15.83:0/2451021264 >> v2:172.21.15.203:3300/0 conn(0x7f260415a3b0 msgr2=0x7f26040b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.726+0000 7f26037fe700 1 --2- 172.21.15.83:0/2451021264 >> v2:172.21.15.203:3300/0 conn(0x7f260415a3b0 0x7f26040b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.726+0000 7f26037fe700 1 -- 172.21.15.83:0/2451021264 >> v1:172.21.15.83:6789/0 conn(0x7f26040cf360 legacy=0x7f26040b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:02.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.726+0000 7f26037fe700 1 -- 172.21.15.83:0/2451021264 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2604070c60 con 0x7f260415deb0 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.726+0000 7f2601ffb700 1 -- 172.21.15.83:0/2451021264 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113906583 0 0) 0x7f260415a790 con 0x7f26040cf360 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.726+0000 7f26037fe700 1 --2- 172.21.15.83:0/2451021264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f260415deb0 0x7f260415e340 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f260404fca0 tx=0x7f25fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7580905ac280b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.726+0000 7f2601ffb700 1 -- 172.21.15.83:0/2451021264 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc00e070 con 0x7f260415deb0 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.726+0000 7f2601ffb700 1 -- 172.21.15.83:0/2451021264 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc00bbd0 con 0x7f260415deb0 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.726+0000 7f2601ffb700 1 -- 172.21.15.83:0/2451021264 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc012430 con 0x7f260415deb0 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.727+0000 7f260928a700 1 -- 172.21.15.83:0/2451021264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f260415deb0 msgr2=0x7f260415e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.727+0000 7f260928a700 1 --2- 172.21.15.83:0/2451021264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f260415deb0 0x7f260415e340 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f260404fca0 tx=0x7f25fc00b330 comp rx=0 tx=0).stop 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.727+0000 7f260928a700 1 -- 172.21.15.83:0/2451021264 shutdown_connections 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.727+0000 7f260928a700 1 --2- 172.21.15.83:0/2451021264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f260415deb0 0x7f260415e340 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.727+0000 7f260928a700 1 --2- 172.21.15.83:0/2451021264 >> v2:172.21.15.203:3300/0 conn(0x7f260415a3b0 0x7f26040b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.727+0000 7f260928a700 1 -- 172.21.15.83:0/2451021264 >> 172.21.15.83:0/2451021264 conn(0x7f26040b1740 msgr2=0x7f26040b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:02.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.727+0000 7f260928a700 1 -- 172.21.15.83:0/2451021264 shutdown_connections 2023-11-26T07:17:02.732 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.727+0000 7f260928a700 1 -- 172.21.15.83:0/2451021264 wait complete. 2023-11-26T07:17:02.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.728+0000 7f260928a700 1 Processor -- start 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.728+0000 7f260928a700 1 -- start start 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f260928a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040cf360 0x7f2604166d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f260928a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f260415a3b0 0x7f26041472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f260928a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415deb0 0x7f2604147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f260928a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2604152250 con 0x7f260415a3b0 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f260928a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2604151c90 con 0x7f260415deb0 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f260928a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2604151f70 con 0x7f26040cf360 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f26027fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f260415a3b0 0x7f26041472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f26027fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f260415a3b0 0x7f26041472a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41532/0 (socket says 172.21.15.83:41532) 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f26027fc700 1 -- 172.21.15.83:0/1998002543 learned_addr learned my addr 172.21.15.83:0/1998002543 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f2602ffd700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040cf360 0x7f2604166d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:02.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f26037fe700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415deb0 0x7f2604147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f26027fc700 1 -- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040cf360 msgr2=0x7f2604166d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f26027fc700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040cf360 0x7f2604166d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f26027fc700 1 -- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415deb0 msgr2=0x7f2604147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f26027fc700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415deb0 0x7f2604147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f26027fc700 1 -- 172.21.15.83:0/1998002543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc009a70 con 0x7f260415a3b0 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f26037fe700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415deb0 0x7f2604147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f26027fc700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f260415a3b0 0x7f26041472a0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f25f800c520 tx=0x7f25f800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f25eb7fe700 1 -- 172.21.15.83:0/1998002543 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8004c80 con 0x7f260415a3b0 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.729+0000 7f25eb7fe700 1 -- 172.21.15.83:0/1998002543 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f8004de0 con 0x7f260415a3b0 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.730+0000 7f260928a700 1 -- 172.21.15.83:0/1998002543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2604152b50 con 0x7f260415a3b0 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.730+0000 7f25eb7fe700 1 -- 172.21.15.83:0/1998002543 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f80072d0 con 0x7f260415a3b0 2023-11-26T07:17:02.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.730+0000 7f260928a700 1 -- 172.21.15.83:0/1998002543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26041477e0 con 0x7f260415a3b0 2023-11-26T07:17:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.730+0000 7f260928a700 1 -- 172.21.15.83:0/1998002543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26041529b0 con 0x7f260415a3b0 2023-11-26T07:17:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.731+0000 7f25eb7fe700 1 -- 172.21.15.83:0/1998002543 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f25f8007430 con 0x7f260415a3b0 2023-11-26T07:17:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.731+0000 7f25eb7fe700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f25f006aff0 0x7f25f006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.731+0000 7f25eb7fe700 1 -- 172.21.15.83:0/1998002543 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25f80892e0 con 0x7f260415a3b0 2023-11-26T07:17:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.732+0000 7f2602ffd700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f25f006aff0 0x7f25f006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:02.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.732+0000 7f2602ffd700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f25f006aff0 0x7f25f006d4b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25f4006b20 tx=0x7f25f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:02.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.736+0000 7f25eb7fe700 1 -- 172.21.15.83:0/1998002543 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26041529b0 con 0x7f260415a3b0 2023-11-26T07:17:02.930 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.928+0000 7f260928a700 1 -- 172.21.15.83:0/1998002543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26041529b0 con 0x7f260415a3b0 2023-11-26T07:17:02.931 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.929+0000 7f25eb7fe700 1 -- 172.21.15.83:0/1998002543 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16528 (secure 0 0 0) 0x7f26041529b0 con 0x7f260415a3b0 2023-11-26T07:17:02.932 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:02.932 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":101,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:17:01.974384+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:00.194045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:00.926172+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:02.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 -- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f25f006aff0 msgr2=0x7f25f006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:02.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f25f006aff0 0x7f25f006d4b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25f4006b20 tx=0x7f25f4008040 comp rx=0 tx=0).stop 2023-11-26T07:17:02.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 -- 172.21.15.83:0/1998002543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f260415a3b0 msgr2=0x7f26041472a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:02.934 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f260415a3b0 0x7f26041472a0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f25f800c520 tx=0x7f25f800c830 comp rx=0 tx=0).stop 2023-11-26T07:17:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 -- 172.21.15.83:0/1998002543 shutdown_connections 2023-11-26T07:17:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f25f006aff0 0x7f25f006d4b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f260415deb0 0x7f2604147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26040cf360 0x7f2604166d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 --2- 172.21.15.83:0/1998002543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f260415a3b0 0x7f26041472a0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 -- 172.21.15.83:0/1998002543 >> 172.21.15.83:0/1998002543 conn(0x7f26040b1740 msgr2=0x7f260414b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 -- 172.21.15.83:0/1998002543 shutdown_connections 2023-11-26T07:17:02.935 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:02.932+0000 7f260928a700 1 -- 172.21.15.83:0/1998002543 wait complete. 2023-11-26T07:17:02.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb820e26700 1 Processor -- start 2023-11-26T07:17:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb820e26700 1 -- start start 2023-11-26T07:17:03.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb820e26700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c06c710 0x7fb81c165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb820e26700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c0af950 con 0x7fb81c1446a0 2023-11-26T07:17:03.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb820e26700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c0afc30 con 0x7fb81c0adf00 2023-11-26T07:17:03.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb820e26700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c144a80 con 0x7fb81c06c710 2023-11-26T07:17:03.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81a59c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb81c1446a0 0x7fb81c0b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33346/0 (socket says 172.21.15.83:33346) 2023-11-26T07:17:03.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81a59c700 1 -- 172.21.15.83:0/2024196989 learned_addr learned my addr 172.21.15.83:0/2024196989 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:03.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198558661 0 0) 0x7fb81c0af950 con 0x7fb81c1446a0 2023-11-26T07:17:03.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8040035b0 con 0x7fb81c1446a0 2023-11-26T07:17:03.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81ad9d700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c06c710 0x7fb81c165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2427457675 0 0) 0x7fb8040035b0 con 0x7fb81c1446a0 2023-11-26T07:17:03.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c06c710 msgr2=0x7fb81c165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c06c710 0x7fb81c165b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 >> v1:172.21.15.203:6789/0 conn(0x7fb81c0adf00 legacy=0x7fb81c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c0aff10 con 0x7fb81c1446a0 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fb80c001670 con 0x7fb81c1446a0 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 >> v1:172.21.15.83:6789/0 conn(0x7fb81c1446a0 legacy=0x7fb81c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb804003a30 0x7fb804003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c06c710 0x7fb804004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb804004d90 0x7fb804005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c144a80 con 0x7fb804004d90 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80400bac0 con 0x7fb804003a30 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.065+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80400de90 con 0x7fb81c06c710 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb81ad9d700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb804004d90 0x7fb804005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.071 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb81a59c700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb804003a30 0x7fb804003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb819d9b700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c06c710 0x7fb804004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb81ad9d700 1 -- 172.21.15.83:0/2024196989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c06c710 msgr2=0x7fb804004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb81ad9d700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c06c710 0x7fb804004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb81ad9d700 1 -- 172.21.15.83:0/2024196989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb804003a30 msgr2=0x7fb804003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb81ad9d700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb804003a30 0x7fb804003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb81ad9d700 1 -- 172.21.15.83:0/2024196989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040035b0 con 0x7fb804004d90 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb81ad9d700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb804004d90 0x7fb804005210 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb814009f90 tx=0x7fb804005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2eae101490c482f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81400ec30 con 0x7fb804004d90 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81400f040 con 0x7fb804004d90 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.066+0000 7fb81959a700 1 -- 172.21.15.83:0/2024196989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814014480 con 0x7fb804004d90 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.067+0000 7fb820e26700 1 -- 172.21.15.83:0/2024196989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb804004d90 msgr2=0x7fb804005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:03.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.067+0000 7fb820e26700 1 --2- 172.21.15.83:0/2024196989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb804004d90 0x7fb804005210 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb814009f90 tx=0x7fb804005880 comp rx=0 tx=0).stop 2023-11-26T07:17:03.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.067+0000 7fb81ad9d700 1 -- 172.21.15.83:0/2024196989 reap_dead start 2023-11-26T07:17:03.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.067+0000 7fb820e26700 1 -- 172.21.15.83:0/2024196989 shutdown_connections 2023-11-26T07:17:03.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.067+0000 7fb820e26700 1 -- 172.21.15.83:0/2024196989 >> 172.21.15.83:0/2024196989 conn(0x7fb81c071b90 msgr2=0x7fb81c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:03.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.067+0000 7fb820e26700 1 -- 172.21.15.83:0/2024196989 shutdown_connections 2023-11-26T07:17:03.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.067+0000 7fb820e26700 1 -- 172.21.15.83:0/2024196989 wait complete. 2023-11-26T07:17:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.068+0000 7fb820e26700 1 Processor -- start 2023-11-26T07:17:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.068+0000 7fb820e26700 1 -- start start 2023-11-26T07:17:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb820e26700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb804004d90 0x7fb81c1494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb820e26700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c14ac30 0x7fb81c14c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb820e26700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c14b0b0 0x7fb81c14ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb820e26700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c14d410 con 0x7fb81c14b0b0 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb820e26700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c149b90 con 0x7fb804004d90 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb820e26700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c149e70 con 0x7fb81c14ac30 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb81ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c14b0b0 0x7fb81c14ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb81ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c14b0b0 0x7fb81c14ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41554/0 (socket says 172.21.15.83:41554) 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb81ad9d700 1 -- 172.21.15.83:0/3849890931 learned_addr learned my addr 172.21.15.83:0/3849890931 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb819d9b700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c14ac30 0x7fb81c14c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb81a59c700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb804004d90 0x7fb81c1494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb81ad9d700 1 -- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c14ac30 msgr2=0x7fb81c14c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb81ad9d700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c14ac30 0x7fb81c14c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb81ad9d700 1 -- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb804004d90 msgr2=0x7fb81c1494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb81ad9d700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb804004d90 0x7fb81c1494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb81ad9d700 1 -- 172.21.15.83:0/3849890931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c14a150 con 0x7fb81c14b0b0 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb819d9b700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c14ac30 0x7fb81c14c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.069+0000 7fb81ad9d700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c14b0b0 0x7fb81c14ced0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb8040047b0 tx=0x7fb814009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.070+0000 7fb80affd700 1 -- 172.21.15.83:0/3849890931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814014c70 con 0x7fb81c14b0b0 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.070+0000 7fb80affd700 1 -- 172.21.15.83:0/3849890931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb814014df0 con 0x7fb81c14b0b0 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.070+0000 7fb820e26700 1 -- 172.21.15.83:0/3849890931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c14a430 con 0x7fb81c14b0b0 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.070+0000 7fb80affd700 1 -- 172.21.15.83:0/3849890931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8140266b0 con 0x7fb81c14b0b0 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.070+0000 7fb820e26700 1 -- 172.21.15.83:0/3849890931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c14b7a0 con 0x7fb81c14b0b0 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.071+0000 7fb820e26700 1 -- 172.21.15.83:0/3849890931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c0753f0 con 0x7fb81c14b0b0 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.071+0000 7fb80affd700 1 -- 172.21.15.83:0/3849890931 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fb814023070 con 0x7fb81c14b0b0 2023-11-26T07:17:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.072+0000 7fb80affd700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb7fc06aff0 0x7fb7fc06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.072+0000 7fb80affd700 1 -- 172.21.15.83:0/3849890931 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb81408ae20 con 0x7fb81c14b0b0 2023-11-26T07:17:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.072+0000 7fb81a59c700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb7fc06aff0 0x7fb7fc06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.072+0000 7fb81a59c700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb7fc06aff0 0x7fb7fc06d4b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb80c001050 tx=0x7fb80c000f80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:03.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.076+0000 7fb80affd700 1 -- 172.21.15.83:0/3849890931 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb814055e50 con 0x7fb81c14b0b0 2023-11-26T07:17:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.267+0000 7fb820e26700 1 -- 172.21.15.83:0/3849890931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb81c067f60 con 0x7fb81c14b0b0 2023-11-26T07:17:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.268+0000 7fb80affd700 1 -- 172.21.15.83:0/3849890931 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16528 (secure 0 0 0) 0x7fb814059470 con 0x7fb81c14b0b0 2023-11-26T07:17:03.271 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:03.271 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":101,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:17:01.974384+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:00.194045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:00.926172+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.270+0000 7fb820e26700 1 -- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb7fc06aff0 msgr2=0x7fb7fc06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.270+0000 7fb820e26700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb7fc06aff0 0x7fb7fc06d4b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb80c001050 tx=0x7fb80c000f80 comp rx=0 tx=0).stop 2023-11-26T07:17:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.270+0000 7fb820e26700 1 -- 172.21.15.83:0/3849890931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c14b0b0 msgr2=0x7fb81c14ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.270+0000 7fb820e26700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c14b0b0 0x7fb81c14ced0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fb8040047b0 tx=0x7fb814009750 comp rx=0 tx=0).stop 2023-11-26T07:17:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.271+0000 7fb820e26700 1 -- 172.21.15.83:0/3849890931 shutdown_connections 2023-11-26T07:17:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.271+0000 7fb820e26700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb7fc06aff0 0x7fb7fc06d4b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.271+0000 7fb820e26700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb81c14b0b0 0x7fb81c14ced0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.271+0000 7fb820e26700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb804004d90 0x7fb81c1494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.271+0000 7fb820e26700 1 --2- 172.21.15.83:0/3849890931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb81c14ac30 0x7fb81c14c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.271+0000 7fb820e26700 1 -- 172.21.15.83:0/3849890931 >> 172.21.15.83:0/3849890931 conn(0x7fb81c071b90 msgr2=0x7fb81c1403e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.271+0000 7fb820e26700 1 -- 172.21.15.83:0/3849890931 shutdown_connections 2023-11-26T07:17:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.271+0000 7fb820e26700 1 -- 172.21.15.83:0/3849890931 wait complete. 2023-11-26T07:17:03.286 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:17:03.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f031e189700 1 Processor -- start 2023-11-26T07:17:03.408 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f031e189700 1 -- start start 2023-11-26T07:17:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f031e189700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f03180af520 0x7f03180afb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f031e189700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f03180746a0 0x7f031806d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f031e189700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031815bf70 con 0x7f03180b0070 2023-11-26T07:17:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f031e189700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031815ee00 con 0x7f03180af520 2023-11-26T07:17:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f031e189700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031815e4f0 con 0x7f03180746a0 2023-11-26T07:17:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f03177fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f03180af520 0x7f03180afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f03177fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f03180af520 0x7f03180afb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41558/0 (socket says 172.21.15.83:41558) 2023-11-26T07:17:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f0316ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f03180b0070 0x7f0318073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33366/0 (socket says 172.21.15.83:33366) 2023-11-26T07:17:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f03177fe700 1 -- 172.21.15.83:0/2348464251 learned_addr learned my addr 172.21.15.83:0/2348464251 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f0317fff700 1 --2- 172.21.15.83:0/2348464251 >> v2:172.21.15.203:3300/0 conn(0x7f03180746a0 0x7f031806d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f03177fe700 1 -- 172.21.15.83:0/2348464251 >> v2:172.21.15.203:3300/0 conn(0x7f03180746a0 msgr2=0x7f031806d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f03177fe700 1 --2- 172.21.15.83:0/2348464251 >> v2:172.21.15.203:3300/0 conn(0x7f03180746a0 0x7f031806d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f03177fe700 1 -- 172.21.15.83:0/2348464251 >> v1:172.21.15.83:6789/0 conn(0x7f03180b0070 legacy=0x7f0318073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f03177fe700 1 -- 172.21.15.83:0/2348464251 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180b04f0 con 0x7f03180af520 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f03167fc700 1 -- 172.21.15.83:0/2348464251 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166846612 0 0) 0x7f0308001110 con 0x7f03180b0070 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f0317fff700 1 --2- 172.21.15.83:0/2348464251 >> v2:172.21.15.203:3300/0 conn(0x7f03180746a0 0x7f031806d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.407+0000 7f03177fe700 1 --2- 172.21.15.83:0/2348464251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03180af520 0x7f03180afb30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0318161a10 tx=0x7f0300009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2014885dc7ef0a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.408+0000 7f03167fc700 1 -- 172.21.15.83:0/2348464251 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030000b870 con 0x7f03180af520 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.408+0000 7f03167fc700 1 -- 172.21.15.83:0/2348464251 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030000b9d0 con 0x7f03180af520 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.408+0000 7f03167fc700 1 -- 172.21.15.83:0/2348464251 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030000bd60 con 0x7f03180af520 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.408+0000 7f031e189700 1 -- 172.21.15.83:0/2348464251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03180af520 msgr2=0x7f03180afb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.408+0000 7f031e189700 1 --2- 172.21.15.83:0/2348464251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03180af520 0x7f03180afb30 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0318161a10 tx=0x7f0300009bd0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.408+0000 7f031e189700 1 -- 172.21.15.83:0/2348464251 shutdown_connections 2023-11-26T07:17:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.408+0000 7f031e189700 1 --2- 172.21.15.83:0/2348464251 >> v2:172.21.15.203:3300/0 conn(0x7f03180746a0 0x7f031806d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.408+0000 7f031e189700 1 --2- 172.21.15.83:0/2348464251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03180af520 0x7f03180afb30 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.408+0000 7f031e189700 1 -- 172.21.15.83:0/2348464251 >> 172.21.15.83:0/2348464251 conn(0x7f03180b28f0 msgr2=0x7f03180b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.408+0000 7f031e189700 1 -- 172.21.15.83:0/2348464251 shutdown_connections 2023-11-26T07:17:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.408+0000 7f031e189700 1 -- 172.21.15.83:0/2348464251 wait complete. 2023-11-26T07:17:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.409+0000 7f031e189700 1 Processor -- start 2023-11-26T07:17:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.410+0000 7f031e189700 1 -- start start 2023-11-26T07:17:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.410+0000 7f031e189700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03180746a0 0x7f0318161110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.410+0000 7f031e189700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180af520 0x7f031813e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.410+0000 7f031e189700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b0070 0x7f031815f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.410+0000 7f031e189700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318161650 con 0x7f03180746a0 2023-11-26T07:17:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.410+0000 7f031e189700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318160070 con 0x7f03180b0070 2023-11-26T07:17:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.410+0000 7f031e189700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0318160350 con 0x7f03180af520 2023-11-26T07:17:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.410+0000 7f03177fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03180746a0 0x7f0318161110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.410+0000 7f03177fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03180746a0 0x7f0318161110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41564/0 (socket says 172.21.15.83:41564) 2023-11-26T07:17:03.415 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.410+0000 7f03177fe700 1 -- 172.21.15.83:0/1661199771 learned_addr learned my addr 172.21.15.83:0/1661199771 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f0317fff700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b0070 0x7f031815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f0316ffd700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180af520 0x7f031813e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f03177fe700 1 -- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180af520 msgr2=0x7f031813e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f03177fe700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180af520 0x7f031813e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f03177fe700 1 -- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b0070 msgr2=0x7f031815f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f03177fe700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b0070 0x7f031815f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f03177fe700 1 -- 172.21.15.83:0/1661199771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0300009550 con 0x7f03180746a0 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f03177fe700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03180746a0 0x7f0318161110 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0300009c10 tx=0x7f0300004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f030ffff700 1 -- 172.21.15.83:0/1661199771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300004310 con 0x7f03180746a0 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f030ffff700 1 -- 172.21.15.83:0/1661199771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030001f070 con 0x7f03180746a0 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f031e189700 1 -- 172.21.15.83:0/1661199771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0318160630 con 0x7f03180746a0 2023-11-26T07:17:03.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f030ffff700 1 -- 172.21.15.83:0/1661199771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03000036a0 con 0x7f03180746a0 2023-11-26T07:17:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.411+0000 7f031e189700 1 -- 172.21.15.83:0/1661199771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0318160bf0 con 0x7f03180746a0 2023-11-26T07:17:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.412+0000 7f031e189700 1 -- 172.21.15.83:0/1661199771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0318160d80 con 0x7f03180746a0 2023-11-26T07:17:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.413+0000 7f030ffff700 1 -- 172.21.15.83:0/1661199771 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f0300021030 con 0x7f03180746a0 2023-11-26T07:17:03.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.413+0000 7f030ffff700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f02f806aff0 0x7f02f806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:03.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.413+0000 7f030ffff700 1 -- 172.21.15.83:0/1661199771 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f030008af50 con 0x7f03180746a0 2023-11-26T07:17:03.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.413+0000 7f0316ffd700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f02f806aff0 0x7f02f806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:03.418 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.414+0000 7f0316ffd700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f02f806aff0 0x7f02f806d4b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0318153f30 tx=0x7f0308007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:03.419 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.418+0000 7f030ffff700 1 -- 172.21.15.83:0/1661199771 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0300056180 con 0x7f03180746a0 2023-11-26T07:17:03.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:03.663+0000 7f031e189700 1 -- 172.21.15.83:0/1661199771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f03180b5030 con 0x7f03180746a0 2023-11-26T07:17:03.864 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-26T07:17:04.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.593+0000 7f030ffff700 1 -- 172.21.15.83:0/1661199771 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7f03000597a0 con 0x7f03180746a0 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 -- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f02f806aff0 msgr2=0x7f02f806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f02f806aff0 0x7f02f806d4b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0318153f30 tx=0x7f0308007c90 comp rx=0 tx=0).stop 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 -- 172.21.15.83:0/1661199771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03180746a0 msgr2=0x7f0318161110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03180746a0 0x7f0318161110 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0300009c10 tx=0x7f0300004750 comp rx=0 tx=0).stop 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 -- 172.21.15.83:0/1661199771 shutdown_connections 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f02f806aff0 0x7f02f806d4b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03180b0070 0x7f031815f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03180746a0 0x7f0318161110 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 --2- 172.21.15.83:0/1661199771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03180af520 0x7f031813e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 -- 172.21.15.83:0/1661199771 >> 172.21.15.83:0/1661199771 conn(0x7f03180b28f0 msgr2=0x7f0318159e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 -- 172.21.15.83:0/1661199771 shutdown_connections 2023-11-26T07:17:04.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.597+0000 7f031e189700 1 -- 172.21.15.83:0/1661199771 wait complete. 2023-11-26T07:17:04.613 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:17:04.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:17:04.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.740+0000 7f92754ef700 1 Processor -- start 2023-11-26T07:17:04.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.740+0000 7f92754ef700 1 -- start start 2023-11-26T07:17:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.741+0000 7f92754ef700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92700aedc0 0x7f92700ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.741+0000 7f92754ef700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927015e290 0x7f92700af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.741+0000 7f92754ef700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927006e400 con 0x7f92700ad870 2023-11-26T07:17:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.741+0000 7f92754ef700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927006fde0 con 0x7f92700aedc0 2023-11-26T07:17:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.741+0000 7f92754ef700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92700af6e0 con 0x7f927015e290 2023-11-26T07:17:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.741+0000 7f926effd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92700aedc0 0x7f92700ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926effd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f92700aedc0 0x7f92700ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41572/0 (socket says 172.21.15.83:41572) 2023-11-26T07:17:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926effd700 1 -- 172.21.15.83:0/1035126181 learned_addr learned my addr 172.21.15.83:0/1035126181 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926e7fc700 1 --2- 172.21.15.83:0/1035126181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927015e290 0x7f92700af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:04.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926effd700 1 -- 172.21.15.83:0/1035126181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927015e290 msgr2=0x7f92700af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926effd700 1 --2- 172.21.15.83:0/1035126181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927015e290 0x7f92700af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926effd700 1 -- 172.21.15.83:0/1035126181 >> v1:172.21.15.203:6789/0 conn(0x7f92700ad870 legacy=0x7f927006f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926effd700 1 -- 172.21.15.83:0/1035126181 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927006e630 con 0x7f92700aedc0 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926e7fc700 1 --2- 172.21.15.83:0/1035126181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927015e290 0x7f92700af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926effd700 1 --2- 172.21.15.83:0/1035126181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92700aedc0 0x7f92700ae790 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f92640098f0 tx=0x7f9264009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e69fda6278c6de server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926dffb700 1 -- 172.21.15.83:0/1035126181 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264004230 con 0x7f92700aedc0 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926dffb700 1 -- 172.21.15.83:0/1035126181 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264004390 con 0x7f92700aedc0 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.742+0000 7f926dffb700 1 -- 172.21.15.83:0/1035126181 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640036a0 con 0x7f92700aedc0 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.743+0000 7f92754ef700 1 -- 172.21.15.83:0/1035126181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92700aedc0 msgr2=0x7f92700ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.743+0000 7f92754ef700 1 --2- 172.21.15.83:0/1035126181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92700aedc0 0x7f92700ae790 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f92640098f0 tx=0x7f9264009c00 comp rx=0 tx=0).stop 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.743+0000 7f92754ef700 1 -- 172.21.15.83:0/1035126181 shutdown_connections 2023-11-26T07:17:04.746 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.743+0000 7f92754ef700 1 --2- 172.21.15.83:0/1035126181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92700aedc0 0x7f92700ae790 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:04.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.743+0000 7f92754ef700 1 --2- 172.21.15.83:0/1035126181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927015e290 0x7f92700af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:04.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.743+0000 7f92754ef700 1 -- 172.21.15.83:0/1035126181 >> 172.21.15.83:0/1035126181 conn(0x7f92700b2050 msgr2=0x7f92700b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:04.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.743+0000 7f92754ef700 1 -- 172.21.15.83:0/1035126181 shutdown_connections 2023-11-26T07:17:04.747 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.743+0000 7f92754ef700 1 -- 172.21.15.83:0/1035126181 wait complete. 2023-11-26T07:17:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.744+0000 7f92754ef700 1 Processor -- start 2023-11-26T07:17:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.744+0000 7f92754ef700 1 -- start start 2023-11-26T07:17:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f92754ef700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92700ad870 0x7f9270153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f92754ef700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92700aedc0 0x7f92700c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f92754ef700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927015e290 0x7f92700c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f92754ef700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92700c85c0 con 0x7f92700aedc0 2023-11-26T07:17:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f92754ef700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92700c88a0 con 0x7f92700ad870 2023-11-26T07:17:04.749 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f92754ef700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92700c8b80 con 0x7f927015e290 2023-11-26T07:17:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f926e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92700aedc0 0x7f92700c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f926e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92700aedc0 0x7f92700c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41580/0 (socket says 172.21.15.83:41580) 2023-11-26T07:17:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f926e7fc700 1 -- 172.21.15.83:0/467683354 learned_addr learned my addr 172.21.15.83:0/467683354 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f926e7fc700 1 -- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927015e290 msgr2=0x7f92700c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f926effd700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92700ad870 0x7f9270153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f926e7fc700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927015e290 0x7f92700c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f926e7fc700 1 -- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92700ad870 msgr2=0x7f9270153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f926e7fc700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92700ad870 0x7f9270153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f926e7fc700 1 -- 172.21.15.83:0/467683354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9264009550 con 0x7f92700aedc0 2023-11-26T07:17:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f926e7fc700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92700aedc0 0x7f92700c7c70 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f926000c7f0 tx=0x7f926000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:04.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f92577fe700 1 -- 172.21.15.83:0/467683354 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260011800 con 0x7f92700aedc0 2023-11-26T07:17:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f92577fe700 1 -- 172.21.15.83:0/467683354 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9260011960 con 0x7f92700aedc0 2023-11-26T07:17:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f92754ef700 1 -- 172.21.15.83:0/467683354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92700c5460 con 0x7f92700aedc0 2023-11-26T07:17:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f92577fe700 1 -- 172.21.15.83:0/467683354 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260011d10 con 0x7f92700aedc0 2023-11-26T07:17:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.745+0000 7f92754ef700 1 -- 172.21.15.83:0/467683354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92700c8390 con 0x7f92700aedc0 2023-11-26T07:17:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.746+0000 7f92754ef700 1 -- 172.21.15.83:0/467683354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9270068d90 con 0x7f92700aedc0 2023-11-26T07:17:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.747+0000 7f92577fe700 1 -- 172.21.15.83:0/467683354 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f9260018360 con 0x7f92700aedc0 2023-11-26T07:17:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.747+0000 7f92577fe700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f925c06aff0 0x7f925c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.747+0000 7f92577fe700 1 -- 172.21.15.83:0/467683354 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92600897b0 con 0x7f92700aedc0 2023-11-26T07:17:04.751 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.748+0000 7f926effd700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f925c06aff0 0x7f925c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:04.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.748+0000 7f926effd700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f925c06aff0 0x7f925c06d4b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f92701604a0 tx=0x7f9264008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:04.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.752+0000 7f92577fe700 1 -- 172.21.15.83:0/467683354 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9260054640 con 0x7f92700aedc0 2023-11-26T07:17:04.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:04.984+0000 7f92754ef700 1 -- 172.21.15.83:0/467683354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f92700c52c0 con 0x7f92700aedc0 2023-11-26T07:17:05.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.594+0000 7f92577fe700 1 -- 172.21.15.83:0/467683354 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v69) v1 ==== 122+0+0 (secure 0 0 0) 0x7f92700c52c0 con 0x7f92700aedc0 2023-11-26T07:17:05.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 -- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f925c06aff0 msgr2=0x7f925c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:05.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f925c06aff0 0x7f925c06d4b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f92701604a0 tx=0x7f9264008030 comp rx=0 tx=0).stop 2023-11-26T07:17:05.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 -- 172.21.15.83:0/467683354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92700aedc0 msgr2=0x7f92700c7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:05.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92700aedc0 0x7f92700c7c70 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f926000c7f0 tx=0x7f926000cbb0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 -- 172.21.15.83:0/467683354 shutdown_connections 2023-11-26T07:17:05.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f925c06aff0 0x7f925c06d4b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f927015e290 0x7f92700c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92700ad870 0x7f9270153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 --2- 172.21.15.83:0/467683354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92700aedc0 0x7f92700c7c70 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 -- 172.21.15.83:0/467683354 >> 172.21.15.83:0/467683354 conn(0x7f92700b2050 msgr2=0x7f92700b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:05.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 -- 172.21.15.83:0/467683354 shutdown_connections 2023-11-26T07:17:05.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.597+0000 7f92754ef700 1 -- 172.21.15.83:0/467683354 wait complete. 2023-11-26T07:17:05.612 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:17:05.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.733+0000 7fd6412b9700 1 Processor -- start 2023-11-26T07:17:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.733+0000 7fd6412b9700 1 -- start start 2023-11-26T07:17:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd6412b9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd63c06ed40 0x7fd63c0afa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd6412b9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd63c073f90 0x7fd63c0743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd6412b9700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0748e0 con 0x7fd63c15deb0 2023-11-26T07:17:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd6412b9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c0affa0 con 0x7fd63c06ed40 2023-11-26T07:17:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd6412b9700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c06dca0 con 0x7fd63c073f90 2023-11-26T07:17:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd63a7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd63c06ed40 0x7fd63c0afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd63affd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd63c15deb0 0x7fd63c0b5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33394/0 (socket says 172.21.15.83:33394) 2023-11-26T07:17:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd63affd700 1 -- 172.21.15.83:0/2991197744 learned_addr learned my addr 172.21.15.83:0/2991197744 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd63a7fc700 1 -- 172.21.15.83:0/2991197744 >> v2:172.21.15.203:3300/0 conn(0x7fd63c073f90 msgr2=0x7fd63c0743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd63a7fc700 1 --2- 172.21.15.83:0/2991197744 >> v2:172.21.15.203:3300/0 conn(0x7fd63c073f90 0x7fd63c0743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd63a7fc700 1 -- 172.21.15.83:0/2991197744 >> v1:172.21.15.83:6789/0 conn(0x7fd63c15deb0 legacy=0x7fd63c0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd63a7fc700 1 -- 172.21.15.83:0/2991197744 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c06df80 con 0x7fd63c06ed40 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd639ffb700 1 -- 172.21.15.83:0/2991197744 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297423657 0 0) 0x7fd63c0748e0 con 0x7fd63c15deb0 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd63a7fc700 1 --2- 172.21.15.83:0/2991197744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd63c06ed40 0x7fd63c0afa60 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0b5190 tx=0x7fd624009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f3974c4cd69720 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd639ffb700 1 -- 172.21.15.83:0/2991197744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624011830 con 0x7fd63c06ed40 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd639ffb700 1 -- 172.21.15.83:0/2991197744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd624011990 con 0x7fd63c06ed40 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.734+0000 7fd639ffb700 1 -- 172.21.15.83:0/2991197744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624011d20 con 0x7fd63c06ed40 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.735+0000 7fd6412b9700 1 -- 172.21.15.83:0/2991197744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd63c06ed40 msgr2=0x7fd63c0afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.735+0000 7fd6412b9700 1 --2- 172.21.15.83:0/2991197744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd63c06ed40 0x7fd63c0afa60 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0b5190 tx=0x7fd624009b20 comp rx=0 tx=0).stop 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.735+0000 7fd6412b9700 1 -- 172.21.15.83:0/2991197744 shutdown_connections 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.735+0000 7fd6412b9700 1 --2- 172.21.15.83:0/2991197744 >> v2:172.21.15.203:3300/0 conn(0x7fd63c073f90 0x7fd63c0743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.735+0000 7fd6412b9700 1 --2- 172.21.15.83:0/2991197744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd63c06ed40 0x7fd63c0afa60 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.735+0000 7fd6412b9700 1 -- 172.21.15.83:0/2991197744 >> 172.21.15.83:0/2991197744 conn(0x7fd63c0d0ba0 msgr2=0x7fd63c0d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:05.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.735+0000 7fd6412b9700 1 -- 172.21.15.83:0/2991197744 shutdown_connections 2023-11-26T07:17:05.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.735+0000 7fd6412b9700 1 -- 172.21.15.83:0/2991197744 wait complete. 2023-11-26T07:17:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.736+0000 7fd6412b9700 1 Processor -- start 2023-11-26T07:17:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.736+0000 7fd6412b9700 1 -- start start 2023-11-26T07:17:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd6412b9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd63c06ed40 0x7fd63c14e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd6412b9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd63c073f90 0x7fd63c0ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:05.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd6412b9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd63c15deb0 0x7fd63c0cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:05.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd6412b9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1f3f60 con 0x7fd63c15deb0 2023-11-26T07:17:05.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd6412b9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1f3bc0 con 0x7fd63c06ed40 2023-11-26T07:17:05.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd6412b9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c151e70 con 0x7fd63c073f90 2023-11-26T07:17:05.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd63c15deb0 0x7fd63c0cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:05.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd63c15deb0 0x7fd63c0cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41596/0 (socket says 172.21.15.83:41596) 2023-11-26T07:17:05.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd63b7fe700 1 -- 172.21.15.83:0/3897552388 learned_addr learned my addr 172.21.15.83:0/3897552388 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:05.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd63a7fc700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd63c073f90 0x7fd63c0ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:05.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd63affd700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd63c06ed40 0x7fd63c14e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:05.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd63b7fe700 1 -- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd63c073f90 msgr2=0x7fd63c0ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:05.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd63b7fe700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd63c073f90 0x7fd63c0ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.742 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd63b7fe700 1 -- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd63c06ed40 msgr2=0x7fd63c14e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd63b7fe700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd63c06ed40 0x7fd63c14e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd63b7fe700 1 -- 172.21.15.83:0/3897552388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6240094a0 con 0x7fd63c15deb0 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.737+0000 7fd63a7fc700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd63c073f90 0x7fd63c0ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.738+0000 7fd63b7fe700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd63c15deb0 0x7fd63c0cacd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00b550 tx=0x7fd62c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.738+0000 7fd6337fe700 1 -- 172.21.15.83:0/3897552388 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c011830 con 0x7fd63c15deb0 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.738+0000 7fd6337fe700 1 -- 172.21.15.83:0/3897552388 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c011990 con 0x7fd63c15deb0 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.738+0000 7fd6412b9700 1 -- 172.21.15.83:0/3897552388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c152150 con 0x7fd63c15deb0 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.738+0000 7fd6337fe700 1 -- 172.21.15.83:0/3897552388 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c011d40 con 0x7fd63c15deb0 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.738+0000 7fd6412b9700 1 -- 172.21.15.83:0/3897552388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c14ee00 con 0x7fd63c15deb0 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.739+0000 7fd6412b9700 1 -- 172.21.15.83:0/3897552388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c068480 con 0x7fd63c15deb0 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.739+0000 7fd6337fe700 1 -- 172.21.15.83:0/3897552388 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fd62c01a400 con 0x7fd63c15deb0 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.740+0000 7fd6337fe700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd61c06aff0 0x7fd61c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:05.743 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.740+0000 7fd6337fe700 1 -- 172.21.15.83:0/3897552388 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd62c089790 con 0x7fd63c15deb0 2023-11-26T07:17:05.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.740+0000 7fd63affd700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd61c06aff0 0x7fd61c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:05.744 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.741+0000 7fd63affd700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd61c06aff0 0x7fd61c06d4b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd6340012e0 tx=0x7fd634001970 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:05.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.744+0000 7fd6337fe700 1 -- 172.21.15.83:0/3897552388 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd62c0547c0 con 0x7fd63c15deb0 2023-11-26T07:17:05.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.978+0000 7fd6412b9700 1 -- 172.21.15.83:0/3897552388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd63c1f40a0 con 0x7fd63c15deb0 2023-11-26T07:17:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.978+0000 7fd6337fe700 1 -- 172.21.15.83:0/3897552388 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fd63c1f40a0 con 0x7fd63c15deb0 2023-11-26T07:17:05.980 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:05.980 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/136760159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":136760159},{"type":"v1","addr":"172.21.15.203:6834","nonce":136760159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3723980912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":3723980912},{"type":"v1","addr":"172.21.15.203:6835","nonce":3723980912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3900154869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3900154869},{"type":"v1","addr":"172.21.15.83:6833","nonce":3900154869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:01.974687+0000","modified":"2023-11-26T07:17:04.985222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7551":{"gid":7551,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3327659486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3327659486},{"type":"v1","addr":"172.21.15.83:6835","nonce":3327659486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:17:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.980+0000 7fd6412b9700 1 -- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd61c06aff0 msgr2=0x7fd61c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.980+0000 7fd6412b9700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd61c06aff0 0x7fd61c06d4b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd6340012e0 tx=0x7fd634001970 comp rx=0 tx=0).stop 2023-11-26T07:17:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.980+0000 7fd6412b9700 1 -- 172.21.15.83:0/3897552388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd63c15deb0 msgr2=0x7fd63c0cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.980+0000 7fd6412b9700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd63c15deb0 0x7fd63c0cacd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00b550 tx=0x7fd62c00b910 comp rx=0 tx=0).stop 2023-11-26T07:17:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.981+0000 7fd6412b9700 1 -- 172.21.15.83:0/3897552388 shutdown_connections 2023-11-26T07:17:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.981+0000 7fd6412b9700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fd61c06aff0 0x7fd61c06d4b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.981+0000 7fd6412b9700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd63c15deb0 0x7fd63c0cacd0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.981+0000 7fd6412b9700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd63c06ed40 0x7fd63c14e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.981+0000 7fd6412b9700 1 --2- 172.21.15.83:0/3897552388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd63c073f90 0x7fd63c0ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.981+0000 7fd6412b9700 1 -- 172.21.15.83:0/3897552388 >> 172.21.15.83:0/3897552388 conn(0x7fd63c0d0ba0 msgr2=0x7fd63c074430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.981+0000 7fd6412b9700 1 -- 172.21.15.83:0/3897552388 shutdown_connections 2023-11-26T07:17:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:05.981+0000 7fd6412b9700 1 -- 172.21.15.83:0/3897552388 wait complete. 2023-11-26T07:17:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2023-11-26T07:17:05.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.113+0000 7fb7aa3f3700 1 Processor -- start 2023-11-26T07:17:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.113+0000 7fb7aa3f3700 1 -- start start 2023-11-26T07:17:06.116 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7aa3f3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7a415bdd0 0x7fb7a41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7aa3f3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7a40b3370 0x7fb7a40b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7aa3f3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a40b2030 con 0x7fb7a40d1c70 2023-11-26T07:17:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7aa3f3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a40b2f50 con 0x7fb7a415bdd0 2023-11-26T07:17:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7aa3f3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40b22d0 con 0x7fb7a40b3370 2023-11-26T07:17:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7a3fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb7a415bdd0 0x7fb7a41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7a8990700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb7a40d1c70 0x7fb7a40ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33412/0 (socket says 172.21.15.83:33412) 2023-11-26T07:17:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7a8990700 1 -- 172.21.15.83:0/496752891 learned_addr learned my addr 172.21.15.83:0/496752891 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7a37fe700 1 --2- 172.21.15.83:0/496752891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7a40b3370 0x7fb7a40b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7a3fff700 1 -- 172.21.15.83:0/496752891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7a40b3370 msgr2=0x7fb7a40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7a2ffd700 1 -- 172.21.15.83:0/496752891 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468934820 0 0) 0x7fb7a40b2030 con 0x7fb7a40d1c70 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7a3fff700 1 --2- 172.21.15.83:0/496752891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7a40b3370 0x7fb7a40b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7a3fff700 1 -- 172.21.15.83:0/496752891 >> v1:172.21.15.83:6789/0 conn(0x7fb7a40d1c70 legacy=0x7fb7a40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7a3fff700 1 -- 172.21.15.83:0/496752891 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a40b25b0 con 0x7fb7a415bdd0 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7a3fff700 1 --2- 172.21.15.83:0/496752891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a415bdd0 0x7fb7a41604e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb798009840 tx=0x7fb798009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20d2cfa35d5b217 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.114+0000 7fb7a2ffd700 1 -- 172.21.15.83:0/496752891 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798011830 con 0x7fb7a415bdd0 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.115+0000 7fb7a2ffd700 1 -- 172.21.15.83:0/496752891 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb798011990 con 0x7fb7a415bdd0 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.115+0000 7fb7a2ffd700 1 -- 172.21.15.83:0/496752891 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798011d20 con 0x7fb7a415bdd0 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.115+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/496752891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a415bdd0 msgr2=0x7fb7a41604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.115+0000 7fb7aa3f3700 1 --2- 172.21.15.83:0/496752891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a415bdd0 0x7fb7a41604e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb798009840 tx=0x7fb798009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.115+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/496752891 shutdown_connections 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.115+0000 7fb7aa3f3700 1 --2- 172.21.15.83:0/496752891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a415bdd0 0x7fb7a41604e0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.115+0000 7fb7aa3f3700 1 --2- 172.21.15.83:0/496752891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7a40b3370 0x7fb7a40b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.115+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/496752891 >> 172.21.15.83:0/496752891 conn(0x7fb7a40731a0 msgr2=0x7fb7a409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:06.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.115+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/496752891 shutdown_connections 2023-11-26T07:17:06.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.115+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/496752891 wait complete. 2023-11-26T07:17:06.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.116+0000 7fb7aa3f3700 1 Processor -- start 2023-11-26T07:17:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.116+0000 7fb7aa3f3700 1 -- start start 2023-11-26T07:17:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.117+0000 7fb7aa3f3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a40b3370 0x7fb7a4154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.117+0000 7fb7aa3f3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7a40d1c70 0x7fb7a41e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.117+0000 7fb7aa3f3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7a415bdd0 0x7fb7a40c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.117+0000 7fb7aa3f3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a41e77b0 con 0x7fb7a40b3370 2023-11-26T07:17:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.117+0000 7fb7aa3f3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a41e5760 con 0x7fb7a40d1c70 2023-11-26T07:17:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.117+0000 7fb7aa3f3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a41e74d0 con 0x7fb7a415bdd0 2023-11-26T07:17:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.117+0000 7fb7a3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a40b3370 0x7fb7a4154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.117+0000 7fb7a3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a40b3370 0x7fb7a4154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41606/0 (socket says 172.21.15.83:41606) 2023-11-26T07:17:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.117+0000 7fb7a3fff700 1 -- 172.21.15.83:0/1501769682 learned_addr learned my addr 172.21.15.83:0/1501769682 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.117+0000 7fb7a3fff700 1 -- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7a415bdd0 msgr2=0x7fb7a40c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.117+0000 7fb7a3fff700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7a415bdd0 0x7fb7a40c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.118+0000 7fb7a3fff700 1 -- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7a40d1c70 msgr2=0x7fb7a41e6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.118+0000 7fb7a3fff700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7a40d1c70 0x7fb7a41e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.118+0000 7fb7a3fff700 1 -- 172.21.15.83:0/1501769682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7980094a0 con 0x7fb7a40b3370 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.118+0000 7fb7a3fff700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a40b3370 0x7fb7a4154b30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb798014040 tx=0x7fb798004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.118+0000 7fb7a0ff9700 1 -- 172.21.15.83:0/1501769682 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7980045f0 con 0x7fb7a40b3370 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.118+0000 7fb7a0ff9700 1 -- 172.21.15.83:0/1501769682 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79801e070 con 0x7fb7a40b3370 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.118+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/1501769682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a41e59b0 con 0x7fb7a40b3370 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.118+0000 7fb7a0ff9700 1 -- 172.21.15.83:0/1501769682 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798003ae0 con 0x7fb7a40b3370 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.118+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/1501769682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a41e5f70 con 0x7fb7a40b3370 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.119+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/1501769682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a40d0520 con 0x7fb7a40b3370 2023-11-26T07:17:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.119+0000 7fb7a0ff9700 1 -- 172.21.15.83:0/1501769682 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fb798003c40 con 0x7fb7a40b3370 2023-11-26T07:17:06.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.120+0000 7fb7a0ff9700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb79006aff0 0x7fb79006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.120+0000 7fb7a0ff9700 1 -- 172.21.15.83:0/1501769682 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb79808a820 con 0x7fb7a40b3370 2023-11-26T07:17:06.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.120+0000 7fb7a37fe700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb79006aff0 0x7fb79006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.120+0000 7fb7a37fe700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb79006aff0 0x7fb79006d4b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40c87f0 tx=0x7fb794008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:06.125 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.124+0000 7fb7a0ff9700 1 -- 172.21.15.83:0/1501769682 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb798055850 con 0x7fb7a40b3370 2023-11-26T07:17:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.315+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/1501769682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7a41e6190 con 0x7fb7a40b3370 2023-11-26T07:17:06.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.317+0000 7fb7a0ff9700 1 -- 172.21.15.83:0/1501769682 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16515 (secure 0 0 0) 0x7fb798015020 con 0x7fb7a40b3370 2023-11-26T07:17:06.318 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:06.319 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":102,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:17:03.554036+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:00.194045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:00.926172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb79006aff0 msgr2=0x7fb79006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb79006aff0 0x7fb79006d4b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40c87f0 tx=0x7fb794008040 comp rx=0 tx=0).stop 2023-11-26T07:17:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/1501769682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a40b3370 msgr2=0x7fb7a4154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a40b3370 0x7fb7a4154b30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb798014040 tx=0x7fb798004750 comp rx=0 tx=0).stop 2023-11-26T07:17:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/1501769682 shutdown_connections 2023-11-26T07:17:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb79006aff0 0x7fb79006d4b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7a415bdd0 0x7fb7a40c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7a40b3370 0x7fb7a4154b30 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 --2- 172.21.15.83:0/1501769682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7a40d1c70 0x7fb7a41e6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/1501769682 >> 172.21.15.83:0/1501769682 conn(0x7fb7a40731a0 msgr2=0x7fb7a40ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/1501769682 shutdown_connections 2023-11-26T07:17:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.319+0000 7fb7aa3f3700 1 -- 172.21.15.83:0/1501769682 wait complete. 2023-11-26T07:17:06.333 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:17:06.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:06.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.451+0000 7f447e172700 1 Processor -- start 2023-11-26T07:17:06.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.451+0000 7f447e172700 1 -- start start 2023-11-26T07:17:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f447e172700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f447806c710 0x7f44780b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f447e172700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44780ae430 con 0x7f44780af4d0 2023-11-26T07:17:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f447e172700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44780ae710 con 0x7f447815e080 2023-11-26T07:17:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f447e172700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44780af0b0 con 0x7f447806c710 2023-11-26T07:17:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f4477fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f44780af4d0 0x7f447806e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33426/0 (socket says 172.21.15.83:33426) 2023-11-26T07:17:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f4477fff700 1 -- 172.21.15.83:0/2549303404 learned_addr learned my addr 172.21.15.83:0/2549303404 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f4476ffd700 1 --2- 172.21.15.83:0/2549303404 >> v2:172.21.15.203:3300/0 conn(0x7f447806c710 0x7f44780b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259774928 0 0) 0x7f44780ae430 con 0x7f44780af4d0 2023-11-26T07:17:06.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44600035b0 con 0x7f44780af4d0 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305924423 0 0) 0x7f44780ae710 con 0x7f447815e080 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44780ae430 con 0x7f447815e080 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2197104476 0 0) 0x7f44600035b0 con 0x7f44780af4d0 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 >> v2:172.21.15.203:3300/0 conn(0x7f447806c710 msgr2=0x7f44780b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 --2- 172.21.15.83:0/2549303404 >> v2:172.21.15.203:3300/0 conn(0x7f447806c710 0x7f44780b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 >> v1:172.21.15.203:6789/0 conn(0x7f447815e080 legacy=0x7f447806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780ae9f0 con 0x7f44780af4d0 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f44700019f0 con 0x7f44780af4d0 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 >> v1:172.21.15.83:6789/0 conn(0x7f44780af4d0 legacy=0x7f447806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4460003b90 0x7f4460003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 0x7f4460004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4460005210 0x7f446000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44780af0b0 con 0x7f4460004850 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446000c090 con 0x7f4460003b90 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446000c250 con 0x7f4460005210 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.452+0000 7f4476ffd700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 0x7f4460004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f44777fe700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4460003b90 0x7f4460003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f4477fff700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4460005210 0x7f446000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f4476ffd700 1 -- 172.21.15.83:0/2549303404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4460005210 msgr2=0x7f446000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f4476ffd700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4460005210 0x7f446000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f4476ffd700 1 -- 172.21.15.83:0/2549303404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4460003b90 msgr2=0x7f4460003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f4476ffd700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4460003b90 0x7f4460003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f4476ffd700 1 -- 172.21.15.83:0/2549303404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44600035b0 con 0x7f4460004850 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f44777fe700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4460003b90 0x7f4460003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:06.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f4477fff700 1 -- 172.21.15.83:0/2549303404 reap_dead start 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f4476ffd700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 0x7f4460004cd0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f446c009b60 tx=0x7f4460005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c037724aef867eb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00e6c0 con 0x7f4460004850 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c00e820 con 0x7f4460004850 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f44767fc700 1 -- 172.21.15.83:0/2549303404 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c00eb90 con 0x7f4460004850 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f447e172700 1 -- 172.21.15.83:0/2549303404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 msgr2=0x7f4460004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f447e172700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 0x7f4460004cd0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f446c009b60 tx=0x7f4460005880 comp rx=0 tx=0).stop 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.453+0000 7f447e172700 1 -- 172.21.15.83:0/2549303404 shutdown_connections 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.454+0000 7f447e172700 1 --2- 172.21.15.83:0/2549303404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 0x7f4460004cd0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.454+0000 7f447e172700 1 -- 172.21.15.83:0/2549303404 >> 172.21.15.83:0/2549303404 conn(0x7f4478071b90 msgr2=0x7f4478071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.454+0000 7f447e172700 1 -- 172.21.15.83:0/2549303404 shutdown_connections 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.454+0000 7f447e172700 1 -- 172.21.15.83:0/2549303404 wait complete. 2023-11-26T07:17:06.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.455+0000 7f447e172700 1 Processor -- start 2023-11-26T07:17:06.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.455+0000 7f447e172700 1 -- start start 2023-11-26T07:17:06.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f447e172700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 0x7f44780cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f447e172700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4478156dc0 0x7f4478160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f447e172700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44780cda40 0x7f44780cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f447e172700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4478141f70 con 0x7f4460004850 2023-11-26T07:17:06.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f447e172700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4478161480 con 0x7f44780cda40 2023-11-26T07:17:06.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f447e172700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4478156930 con 0x7f4478156dc0 2023-11-26T07:17:06.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f44777fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 0x7f44780cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f44777fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 0x7f44780cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41626/0 (socket says 172.21.15.83:41626) 2023-11-26T07:17:06.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f44777fe700 1 -- 172.21.15.83:0/3571432177 learned_addr learned my addr 172.21.15.83:0/3571432177 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f4476ffd700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4478156dc0 0x7f4478160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f4477fff700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44780cda40 0x7f44780cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f44777fe700 1 -- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4478156dc0 msgr2=0x7f4478160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f44777fe700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4478156dc0 0x7f4478160f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f44777fe700 1 -- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44780cda40 msgr2=0x7f44780cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f44777fe700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44780cda40 0x7f44780cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f44777fe700 1 -- 172.21.15.83:0/3571432177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478153f00 con 0x7f4460004850 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f44777fe700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 0x7f44780cf020 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f446800e7b0 tx=0x7f446800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f445ffff700 1 -- 172.21.15.83:0/3571432177 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44680052e0 con 0x7f4460004850 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f445ffff700 1 -- 172.21.15.83:0/3571432177 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468005440 con 0x7f4460004850 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f447e172700 1 -- 172.21.15.83:0/3571432177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44781562e0 con 0x7f4460004850 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f445ffff700 1 -- 172.21.15.83:0/3571432177 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44680038a0 con 0x7f4460004850 2023-11-26T07:17:06.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.456+0000 7f447e172700 1 -- 172.21.15.83:0/3571432177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44781541e0 con 0x7f4460004850 2023-11-26T07:17:06.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.457+0000 7f447e172700 1 -- 172.21.15.83:0/3571432177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44780753f0 con 0x7f4460004850 2023-11-26T07:17:06.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.458+0000 7f445ffff700 1 -- 172.21.15.83:0/3571432177 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f4468003a00 con 0x7f4460004850 2023-11-26T07:17:06.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.458+0000 7f445ffff700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f446406aff0 0x7f446406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.458+0000 7f445ffff700 1 -- 172.21.15.83:0/3571432177 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4468088d90 con 0x7f4460004850 2023-11-26T07:17:06.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.458+0000 7f4476ffd700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f446406aff0 0x7f446406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.459+0000 7f4476ffd700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f446406aff0 0x7f446406d4b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f446c0090b0 tx=0x7f446c01e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:06.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.463+0000 7f445ffff700 1 -- 172.21.15.83:0/3571432177 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4468053de0 con 0x7f4460004850 2023-11-26T07:17:06.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.647+0000 7f447e172700 1 -- 172.21.15.83:0/3571432177 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44780cf560 con 0x7f446406aff0 2023-11-26T07:17:06.657 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.656+0000 7f445ffff700 1 -- 172.21.15.83:0/3571432177 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289766 (secure 0 0 0) 0x7f44780cf560 con 0x7f446406aff0 2023-11-26T07:17:06.661 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:06.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.659+0000 7f447e172700 1 -- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f446406aff0 msgr2=0x7f446406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.659+0000 7f447e172700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f446406aff0 0x7f446406d4b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f446c0090b0 tx=0x7f446c01e040 comp rx=0 tx=0).stop 2023-11-26T07:17:06.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.659+0000 7f447e172700 1 -- 172.21.15.83:0/3571432177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 msgr2=0x7f44780cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.659+0000 7f447e172700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 0x7f44780cf020 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f446800e7b0 tx=0x7f446800eac0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.659+0000 7f447e172700 1 -- 172.21.15.83:0/3571432177 shutdown_connections 2023-11-26T07:17:06.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.659+0000 7f447e172700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f446406aff0 0x7f446406d4b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.659+0000 7f447e172700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44780cda40 0x7f44780cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.659+0000 7f447e172700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4460004850 0x7f44780cf020 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.659+0000 7f447e172700 1 --2- 172.21.15.83:0/3571432177 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4478156dc0 0x7f4478160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.659+0000 7f447e172700 1 -- 172.21.15.83:0/3571432177 >> 172.21.15.83:0/3571432177 conn(0x7f4478071b90 msgr2=0x7f447806bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:06.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.660+0000 7f447e172700 1 -- 172.21.15.83:0/3571432177 shutdown_connections 2023-11-26T07:17:06.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.660+0000 7f447e172700 1 -- 172.21.15.83:0/3571432177 wait complete. 2023-11-26T07:17:06.664 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:17:06.673 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2023-11-26T07:17:06.320904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248168,"kb_used_data":11144,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754726552,"statfs":{"total":773094113280,"available":772839989248,"internally_reserved":0,"allocated":11411456,"data_stored":5636508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.757198"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.560331+0000","last_change":"2023-11-26T07:17:02.575475+0000","last_active":"2023-11-26T07:17:03.560331+0000","last_peered":"2023-11-26T07:17:03.560331+0000","last_clean":"2023-11-26T07:17:03.560331+0000","last_became_active":"2023-11-26T07:17:02.575169+0000","last_became_peered":"2023-11-26T07:17:02.575169+0000","last_unstale":"2023-11-26T07:17:03.560331+0000","last_undegraded":"2023-11-26T07:17:03.560331+0000","last_fullsized":"2023-11-26T07:17:03.560331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:25:55.927869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625946+0000","last_change":"2023-11-26T07:17:02.568368+0000","last_active":"2023-11-26T07:17:03.625946+0000","last_peered":"2023-11-26T07:17:03.625946+0000","last_clean":"2023-11-26T07:17:03.625946+0000","last_became_active":"2023-11-26T07:17:02.568078+0000","last_became_peered":"2023-11-26T07:17:02.568078+0000","last_unstale":"2023-11-26T07:17:03.625946+0000","last_undegraded":"2023-11-26T07:17:03.625946+0000","last_fullsized":"2023-11-26T07:17:03.625946+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:11:27.335118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563685+0000","last_change":"2023-11-26T07:17:02.568861+0000","last_active":"2023-11-26T07:17:03.563685+0000","last_peered":"2023-11-26T07:17:03.563685+0000","last_clean":"2023-11-26T07:17:03.563685+0000","last_became_active":"2023-11-26T07:17:02.568578+0000","last_became_peered":"2023-11-26T07:17:02.568578+0000","last_unstale":"2023-11-26T07:17:03.563685+0000","last_undegraded":"2023-11-26T07:17:03.563685+0000","last_fullsized":"2023-11-26T07:17:03.563685+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:05.947486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559766+0000","last_change":"2023-11-26T07:17:02.569388+0000","last_active":"2023-11-26T07:17:03.559766+0000","last_peered":"2023-11-26T07:17:03.559766+0000","last_clean":"2023-11-26T07:17:03.559766+0000","last_became_active":"2023-11-26T07:17:02.569107+0000","last_became_peered":"2023-11-26T07:17:02.569107+0000","last_unstale":"2023-11-26T07:17:03.559766+0000","last_undegraded":"2023-11-26T07:17:03.559766+0000","last_fullsized":"2023-11-26T07:17:03.559766+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:05:12.369764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625454+0000","last_change":"2023-11-26T07:17:02.563096+0000","last_active":"2023-11-26T07:17:03.625454+0000","last_peered":"2023-11-26T07:17:03.625454+0000","last_clean":"2023-11-26T07:17:03.625454+0000","last_became_active":"2023-11-26T07:17:02.562658+0000","last_became_peered":"2023-11-26T07:17:02.562658+0000","last_unstale":"2023-11-26T07:17:03.625454+0000","last_undegraded":"2023-11-26T07:17:03.625454+0000","last_fullsized":"2023-11-26T07:17:03.625454+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:06:49.465873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626085+0000","last_change":"2023-11-26T07:17:02.566712+0000","last_active":"2023-11-26T07:17:03.626085+0000","last_peered":"2023-11-26T07:17:03.626085+0000","last_clean":"2023-11-26T07:17:03.626085+0000","last_became_active":"2023-11-26T07:17:02.566441+0000","last_became_peered":"2023-11-26T07:17:02.566441+0000","last_unstale":"2023-11-26T07:17:03.626085+0000","last_undegraded":"2023-11-26T07:17:03.626085+0000","last_fullsized":"2023-11-26T07:17:03.626085+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:56:03.418914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.562720+0000","last_change":"2023-11-26T07:17:02.576715+0000","last_active":"2023-11-26T07:17:03.562720+0000","last_peered":"2023-11-26T07:17:03.562720+0000","last_clean":"2023-11-26T07:17:03.562720+0000","last_became_active":"2023-11-26T07:17:02.576439+0000","last_became_peered":"2023-11-26T07:17:02.576439+0000","last_unstale":"2023-11-26T07:17:03.562720+0000","last_undegraded":"2023-11-26T07:17:03.562720+0000","last_fullsized":"2023-11-26T07:17:03.562720+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:57:37.656434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563387+0000","last_change":"2023-11-26T07:17:02.569790+0000","last_active":"2023-11-26T07:17:03.563387+0000","last_peered":"2023-11-26T07:17:03.563387+0000","last_clean":"2023-11-26T07:17:03.563387+0000","last_became_active":"2023-11-26T07:17:02.569501+0000","last_became_peered":"2023-11-26T07:17:02.569501+0000","last_unstale":"2023-11-26T07:17:03.563387+0000","last_undegraded":"2023-11-26T07:17:03.563387+0000","last_fullsized":"2023-11-26T07:17:03.563387+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:16.936993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.560200+0000","last_change":"2023-11-26T07:17:02.573486+0000","last_active":"2023-11-26T07:17:03.560200+0000","last_peered":"2023-11-26T07:17:03.560200+0000","last_clean":"2023-11-26T07:17:03.560200+0000","last_became_active":"2023-11-26T07:17:02.573157+0000","last_became_peered":"2023-11-26T07:17:02.573157+0000","last_unstale":"2023-11-26T07:17:03.560200+0000","last_undegraded":"2023-11-26T07:17:03.560200+0000","last_fullsized":"2023-11-26T07:17:03.560200+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:33:46.779337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559777+0000","last_change":"2023-11-26T07:17:02.572458+0000","last_active":"2023-11-26T07:17:03.559777+0000","last_peered":"2023-11-26T07:17:03.559777+0000","last_clean":"2023-11-26T07:17:03.559777+0000","last_became_active":"2023-11-26T07:17:02.572180+0000","last_became_peered":"2023-11-26T07:17:02.572180+0000","last_unstale":"2023-11-26T07:17:03.559777+0000","last_undegraded":"2023-11-26T07:17:03.559777+0000","last_fullsized":"2023-11-26T07:17:03.559777+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:33:31.205055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625966+0000","last_change":"2023-11-26T07:17:02.576567+0000","last_active":"2023-11-26T07:17:03.625966+0000","last_peered":"2023-11-26T07:17:03.625966+0000","last_clean":"2023-11-26T07:17:03.625966+0000","last_became_active":"2023-11-26T07:17:02.576308+0000","last_became_peered":"2023-11-26T07:17:02.576308+0000","last_unstale":"2023-11-26T07:17:03.625966+0000","last_undegraded":"2023-11-26T07:17:03.625966+0000","last_fullsized":"2023-11-26T07:17:03.625966+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:25:26.027867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559906+0000","last_change":"2023-11-26T07:17:02.571015+0000","last_active":"2023-11-26T07:17:03.559906+0000","last_peered":"2023-11-26T07:17:03.559906+0000","last_clean":"2023-11-26T07:17:03.559906+0000","last_became_active":"2023-11-26T07:17:02.570649+0000","last_became_peered":"2023-11-26T07:17:02.570649+0000","last_unstale":"2023-11-26T07:17:03.559906+0000","last_undegraded":"2023-11-26T07:17:03.559906+0000","last_fullsized":"2023-11-26T07:17:03.559906+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:39:21.005917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563656+0000","last_change":"2023-11-26T07:17:02.572715+0000","last_active":"2023-11-26T07:17:03.563656+0000","last_peered":"2023-11-26T07:17:03.563656+0000","last_clean":"2023-11-26T07:17:03.563656+0000","last_became_active":"2023-11-26T07:17:02.572035+0000","last_became_peered":"2023-11-26T07:17:02.572035+0000","last_unstale":"2023-11-26T07:17:03.563656+0000","last_undegraded":"2023-11-26T07:17:03.563656+0000","last_fullsized":"2023-11-26T07:17:03.563656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:28:54.491028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563646+0000","last_change":"2023-11-26T07:17:02.576459+0000","last_active":"2023-11-26T07:17:03.563646+0000","last_peered":"2023-11-26T07:17:03.563646+0000","last_clean":"2023-11-26T07:17:03.563646+0000","last_became_active":"2023-11-26T07:17:02.576185+0000","last_became_peered":"2023-11-26T07:17:02.576185+0000","last_unstale":"2023-11-26T07:17:03.563646+0000","last_undegraded":"2023-11-26T07:17:03.563646+0000","last_fullsized":"2023-11-26T07:17:03.563646+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:53:36.673023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625842+0000","last_change":"2023-11-26T07:17:02.573948+0000","last_active":"2023-11-26T07:17:03.625842+0000","last_peered":"2023-11-26T07:17:03.625842+0000","last_clean":"2023-11-26T07:17:03.625842+0000","last_became_active":"2023-11-26T07:17:02.572690+0000","last_became_peered":"2023-11-26T07:17:02.572690+0000","last_unstale":"2023-11-26T07:17:03.625842+0000","last_undegraded":"2023-11-26T07:17:03.625842+0000","last_fullsized":"2023-11-26T07:17:03.625842+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:16:18.265358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559655+0000","last_change":"2023-11-26T07:17:02.572635+0000","last_active":"2023-11-26T07:17:03.559655+0000","last_peered":"2023-11-26T07:17:03.559655+0000","last_clean":"2023-11-26T07:17:03.559655+0000","last_became_active":"2023-11-26T07:17:02.572325+0000","last_became_peered":"2023-11-26T07:17:02.572325+0000","last_unstale":"2023-11-26T07:17:03.559655+0000","last_undegraded":"2023-11-26T07:17:03.559655+0000","last_fullsized":"2023-11-26T07:17:03.559655+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:53:20.301655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626377+0000","last_change":"2023-11-26T07:17:02.575984+0000","last_active":"2023-11-26T07:17:03.626377+0000","last_peered":"2023-11-26T07:17:03.626377+0000","last_clean":"2023-11-26T07:17:03.626377+0000","last_became_active":"2023-11-26T07:17:02.575687+0000","last_became_peered":"2023-11-26T07:17:02.575687+0000","last_unstale":"2023-11-26T07:17:03.626377+0000","last_undegraded":"2023-11-26T07:17:03.626377+0000","last_fullsized":"2023-11-26T07:17:03.626377+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:53:36.673023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626125+0000","last_change":"2023-11-26T07:17:02.574826+0000","last_active":"2023-11-26T07:17:03.626125+0000","last_peered":"2023-11-26T07:17:03.626125+0000","last_clean":"2023-11-26T07:17:03.626125+0000","last_became_active":"2023-11-26T07:17:02.574534+0000","last_became_peered":"2023-11-26T07:17:02.574534+0000","last_unstale":"2023-11-26T07:17:03.626125+0000","last_undegraded":"2023-11-26T07:17:03.626125+0000","last_fullsized":"2023-11-26T07:17:03.626125+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:10.197851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563768+0000","last_change":"2023-11-26T07:17:02.568118+0000","last_active":"2023-11-26T07:17:03.563768+0000","last_peered":"2023-11-26T07:17:03.563768+0000","last_clean":"2023-11-26T07:17:03.563768+0000","last_became_active":"2023-11-26T07:17:02.567453+0000","last_became_peered":"2023-11-26T07:17:02.567453+0000","last_unstale":"2023-11-26T07:17:03.563768+0000","last_undegraded":"2023-11-26T07:17:03.563768+0000","last_fullsized":"2023-11-26T07:17:03.563768+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:15:11.809794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625888+0000","last_change":"2023-11-26T07:17:02.565875+0000","last_active":"2023-11-26T07:17:03.625888+0000","last_peered":"2023-11-26T07:17:03.625888+0000","last_clean":"2023-11-26T07:17:03.625888+0000","last_became_active":"2023-11-26T07:17:02.565604+0000","last_became_peered":"2023-11-26T07:17:02.565604+0000","last_unstale":"2023-11-26T07:17:03.625888+0000","last_undegraded":"2023-11-26T07:17:03.625888+0000","last_fullsized":"2023-11-26T07:17:03.625888+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:24.170651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625816+0000","last_change":"2023-11-26T07:17:02.565776+0000","last_active":"2023-11-26T07:17:03.625816+0000","last_peered":"2023-11-26T07:17:03.625816+0000","last_clean":"2023-11-26T07:17:03.625816+0000","last_became_active":"2023-11-26T07:17:02.565485+0000","last_became_peered":"2023-11-26T07:17:02.565485+0000","last_unstale":"2023-11-26T07:17:03.625816+0000","last_undegraded":"2023-11-26T07:17:03.625816+0000","last_fullsized":"2023-11-26T07:17:03.625816+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:10.197851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625794+0000","last_change":"2023-11-26T07:17:02.575638+0000","last_active":"2023-11-26T07:17:03.625794+0000","last_peered":"2023-11-26T07:17:03.625794+0000","last_clean":"2023-11-26T07:17:03.625794+0000","last_became_active":"2023-11-26T07:17:02.575164+0000","last_became_peered":"2023-11-26T07:17:02.575164+0000","last_unstale":"2023-11-26T07:17:03.625794+0000","last_undegraded":"2023-11-26T07:17:03.625794+0000","last_fullsized":"2023-11-26T07:17:03.625794+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:15:11.809794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625397+0000","last_change":"2023-11-26T07:17:02.565508+0000","last_active":"2023-11-26T07:17:03.625397+0000","last_peered":"2023-11-26T07:17:03.625397+0000","last_clean":"2023-11-26T07:17:03.625397+0000","last_became_active":"2023-11-26T07:17:02.565049+0000","last_became_peered":"2023-11-26T07:17:02.565049+0000","last_unstale":"2023-11-26T07:17:03.625397+0000","last_undegraded":"2023-11-26T07:17:03.625397+0000","last_fullsized":"2023-11-26T07:17:03.625397+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:04:23.067790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625968+0000","last_change":"2023-11-26T07:17:02.566025+0000","last_active":"2023-11-26T07:17:03.625968+0000","last_peered":"2023-11-26T07:17:03.625968+0000","last_clean":"2023-11-26T07:17:03.625968+0000","last_became_active":"2023-11-26T07:17:02.565634+0000","last_became_peered":"2023-11-26T07:17:02.565634+0000","last_unstale":"2023-11-26T07:17:03.625968+0000","last_undegraded":"2023-11-26T07:17:03.625968+0000","last_fullsized":"2023-11-26T07:17:03.625968+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:16.936993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626366+0000","last_change":"2023-11-26T07:17:02.575799+0000","last_active":"2023-11-26T07:17:03.626366+0000","last_peered":"2023-11-26T07:17:03.626366+0000","last_clean":"2023-11-26T07:17:03.626366+0000","last_became_active":"2023-11-26T07:17:02.575542+0000","last_became_peered":"2023-11-26T07:17:02.575542+0000","last_unstale":"2023-11-26T07:17:03.626366+0000","last_undegraded":"2023-11-26T07:17:03.626366+0000","last_fullsized":"2023-11-26T07:17:03.626366+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:17:22.322087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559910+0000","last_change":"2023-11-26T07:17:02.571878+0000","last_active":"2023-11-26T07:17:03.559910+0000","last_peered":"2023-11-26T07:17:03.559910+0000","last_clean":"2023-11-26T07:17:03.559910+0000","last_became_active":"2023-11-26T07:17:02.571586+0000","last_became_peered":"2023-11-26T07:17:02.571586+0000","last_unstale":"2023-11-26T07:17:03.559910+0000","last_undegraded":"2023-11-26T07:17:03.559910+0000","last_fullsized":"2023-11-26T07:17:03.559910+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:53.814028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626568+0000","last_change":"2023-11-26T07:17:02.572736+0000","last_active":"2023-11-26T07:17:03.626568+0000","last_peered":"2023-11-26T07:17:03.626568+0000","last_clean":"2023-11-26T07:17:03.626568+0000","last_became_active":"2023-11-26T07:17:02.572462+0000","last_became_peered":"2023-11-26T07:17:02.572462+0000","last_unstale":"2023-11-26T07:17:03.626568+0000","last_undegraded":"2023-11-26T07:17:03.626568+0000","last_fullsized":"2023-11-26T07:17:03.626568+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:43:51.163573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625997+0000","last_change":"2023-11-26T07:17:02.567421+0000","last_active":"2023-11-26T07:17:03.625997+0000","last_peered":"2023-11-26T07:17:03.625997+0000","last_clean":"2023-11-26T07:17:03.625997+0000","last_became_active":"2023-11-26T07:17:02.567137+0000","last_became_peered":"2023-11-26T07:17:02.567137+0000","last_unstale":"2023-11-26T07:17:03.625997+0000","last_undegraded":"2023-11-26T07:17:03.625997+0000","last_fullsized":"2023-11-26T07:17:03.625997+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:28:00.740325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625934+0000","last_change":"2023-11-26T07:17:02.572812+0000","last_active":"2023-11-26T07:17:03.625934+0000","last_peered":"2023-11-26T07:17:03.625934+0000","last_clean":"2023-11-26T07:17:03.625934+0000","last_became_active":"2023-11-26T07:17:02.572515+0000","last_became_peered":"2023-11-26T07:17:02.572515+0000","last_unstale":"2023-11-26T07:17:03.625934+0000","last_undegraded":"2023-11-26T07:17:03.625934+0000","last_fullsized":"2023-11-26T07:17:03.625934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:39.771470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.562647+0000","last_change":"2023-11-26T07:17:02.573494+0000","last_active":"2023-11-26T07:17:03.562647+0000","last_peered":"2023-11-26T07:17:03.562647+0000","last_clean":"2023-11-26T07:17:03.562647+0000","last_became_active":"2023-11-26T07:17:02.573221+0000","last_became_peered":"2023-11-26T07:17:02.573221+0000","last_unstale":"2023-11-26T07:17:03.562647+0000","last_undegraded":"2023-11-26T07:17:03.562647+0000","last_fullsized":"2023-11-26T07:17:03.562647+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:41.020210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559713+0000","last_change":"2023-11-26T07:17:02.572009+0000","last_active":"2023-11-26T07:17:03.559713+0000","last_peered":"2023-11-26T07:17:03.559713+0000","last_clean":"2023-11-26T07:17:03.559713+0000","last_became_active":"2023-11-26T07:17:02.571558+0000","last_became_peered":"2023-11-26T07:17:02.571558+0000","last_unstale":"2023-11-26T07:17:03.559713+0000","last_undegraded":"2023-11-26T07:17:03.559713+0000","last_fullsized":"2023-11-26T07:17:03.559713+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:35:52.938929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626053+0000","last_change":"2023-11-26T07:17:02.568891+0000","last_active":"2023-11-26T07:17:03.626053+0000","last_peered":"2023-11-26T07:17:03.626053+0000","last_clean":"2023-11-26T07:17:03.626053+0000","last_became_active":"2023-11-26T07:17:02.568619+0000","last_became_peered":"2023-11-26T07:17:02.568619+0000","last_unstale":"2023-11-26T07:17:03.626053+0000","last_undegraded":"2023-11-26T07:17:03.626053+0000","last_fullsized":"2023-11-26T07:17:03.626053+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:37:18.698945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626400+0000","last_change":"2023-11-26T07:17:02.567327+0000","last_active":"2023-11-26T07:17:03.626400+0000","last_peered":"2023-11-26T07:17:03.626400+0000","last_clean":"2023-11-26T07:17:03.626400+0000","last_became_active":"2023-11-26T07:17:02.567039+0000","last_became_peered":"2023-11-26T07:17:02.567039+0000","last_unstale":"2023-11-26T07:17:03.626400+0000","last_undegraded":"2023-11-26T07:17:03.626400+0000","last_fullsized":"2023-11-26T07:17:03.626400+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:21.174137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"101'4","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.594520+0000","last_change":"2023-11-26T07:17:01.584404+0000","last_active":"2023-11-26T07:17:02.594520+0000","last_peered":"2023-11-26T07:17:02.594520+0000","last_clean":"2023-11-26T07:17:02.594520+0000","last_became_active":"2023-11-26T07:17:01.584088+0000","last_became_peered":"2023-11-26T07:17:01.584088+0000","last_unstale":"2023-11-26T07:17:02.594520+0000","last_undegraded":"2023-11-26T07:17:02.594520+0000","last_fullsized":"2023-11-26T07:17:02.594520+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:04:11.625643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626425+0000","last_change":"2023-11-26T07:17:02.582406+0000","last_active":"2023-11-26T07:17:03.626425+0000","last_peered":"2023-11-26T07:17:03.626425+0000","last_clean":"2023-11-26T07:17:03.626425+0000","last_became_active":"2023-11-26T07:17:02.582095+0000","last_became_peered":"2023-11-26T07:17:02.582095+0000","last_unstale":"2023-11-26T07:17:03.626425+0000","last_undegraded":"2023-11-26T07:17:03.626425+0000","last_fullsized":"2023-11-26T07:17:03.626425+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:11:27.335118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.591688+0000","last_change":"2023-11-26T07:17:01.583711+0000","last_active":"2023-11-26T07:17:02.591688+0000","last_peered":"2023-11-26T07:17:02.591688+0000","last_clean":"2023-11-26T07:17:02.591688+0000","last_became_active":"2023-11-26T07:17:01.583168+0000","last_became_peered":"2023-11-26T07:17:01.583168+0000","last_unstale":"2023-11-26T07:17:02.591688+0000","last_undegraded":"2023-11-26T07:17:02.591688+0000","last_fullsized":"2023-11-26T07:17:02.591688+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:27.918900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625469+0000","last_change":"2023-11-26T07:17:02.569200+0000","last_active":"2023-11-26T07:17:03.625469+0000","last_peered":"2023-11-26T07:17:03.625469+0000","last_clean":"2023-11-26T07:17:03.625469+0000","last_became_active":"2023-11-26T07:17:02.568908+0000","last_became_peered":"2023-11-26T07:17:02.568908+0000","last_unstale":"2023-11-26T07:17:03.625469+0000","last_undegraded":"2023-11-26T07:17:03.625469+0000","last_fullsized":"2023-11-26T07:17:03.625469+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:06:13.267251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.592893+0000","last_change":"2023-11-26T07:17:01.588258+0000","last_active":"2023-11-26T07:17:02.592893+0000","last_peered":"2023-11-26T07:17:02.592893+0000","last_clean":"2023-11-26T07:17:02.592893+0000","last_became_active":"2023-11-26T07:17:01.587985+0000","last_became_peered":"2023-11-26T07:17:01.587985+0000","last_unstale":"2023-11-26T07:17:02.592893+0000","last_undegraded":"2023-11-26T07:17:02.592893+0000","last_fullsized":"2023-11-26T07:17:02.592893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:53:30.894666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563844+0000","last_change":"2023-11-26T07:17:02.570781+0000","last_active":"2023-11-26T07:17:03.563844+0000","last_peered":"2023-11-26T07:17:03.563844+0000","last_clean":"2023-11-26T07:17:03.563844+0000","last_became_active":"2023-11-26T07:17:02.570531+0000","last_became_peered":"2023-11-26T07:17:02.570531+0000","last_unstale":"2023-11-26T07:17:03.563844+0000","last_undegraded":"2023-11-26T07:17:03.563844+0000","last_fullsized":"2023-11-26T07:17:03.563844+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:50.422694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.593190+0000","last_change":"2023-11-26T07:17:01.590141+0000","last_active":"2023-11-26T07:17:02.593190+0000","last_peered":"2023-11-26T07:17:02.593190+0000","last_clean":"2023-11-26T07:17:02.593190+0000","last_became_active":"2023-11-26T07:17:01.589899+0000","last_became_peered":"2023-11-26T07:17:01.589899+0000","last_unstale":"2023-11-26T07:17:02.593190+0000","last_undegraded":"2023-11-26T07:17:02.593190+0000","last_fullsized":"2023-11-26T07:17:02.593190+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:04:02.800106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559803+0000","last_change":"2023-11-26T07:17:02.573870+0000","last_active":"2023-11-26T07:17:03.559803+0000","last_peered":"2023-11-26T07:17:03.559803+0000","last_clean":"2023-11-26T07:17:03.559803+0000","last_became_active":"2023-11-26T07:17:02.573620+0000","last_became_peered":"2023-11-26T07:17:02.573620+0000","last_unstale":"2023-11-26T07:17:03.559803+0000","last_undegraded":"2023-11-26T07:17:03.559803+0000","last_fullsized":"2023-11-26T07:17:03.559803+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:37:19.895847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.592173+0000","last_change":"2023-11-26T07:17:01.588225+0000","last_active":"2023-11-26T07:17:02.592173+0000","last_peered":"2023-11-26T07:17:02.592173+0000","last_clean":"2023-11-26T07:17:02.592173+0000","last_became_active":"2023-11-26T07:17:01.587905+0000","last_became_peered":"2023-11-26T07:17:01.587905+0000","last_unstale":"2023-11-26T07:17:02.592173+0000","last_undegraded":"2023-11-26T07:17:02.592173+0000","last_fullsized":"2023-11-26T07:17:02.592173+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:28:36.298533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625982+0000","last_change":"2023-11-26T07:17:02.567623+0000","last_active":"2023-11-26T07:17:03.625982+0000","last_peered":"2023-11-26T07:17:03.625982+0000","last_clean":"2023-11-26T07:17:03.625982+0000","last_became_active":"2023-11-26T07:17:02.567227+0000","last_became_peered":"2023-11-26T07:17:02.567227+0000","last_unstale":"2023-11-26T07:17:03.625982+0000","last_undegraded":"2023-11-26T07:17:03.625982+0000","last_fullsized":"2023-11-26T07:17:03.625982+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:59.383908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.590568+0000","last_change":"2023-11-26T07:17:01.585551+0000","last_active":"2023-11-26T07:17:02.590568+0000","last_peered":"2023-11-26T07:17:02.590568+0000","last_clean":"2023-11-26T07:17:02.590568+0000","last_became_active":"2023-11-26T07:17:01.585270+0000","last_became_peered":"2023-11-26T07:17:01.585270+0000","last_unstale":"2023-11-26T07:17:02.590568+0000","last_undegraded":"2023-11-26T07:17:02.590568+0000","last_fullsized":"2023-11-26T07:17:02.590568+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:09.192327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559742+0000","last_change":"2023-11-26T07:17:02.571080+0000","last_active":"2023-11-26T07:17:03.559742+0000","last_peered":"2023-11-26T07:17:03.559742+0000","last_clean":"2023-11-26T07:17:03.559742+0000","last_became_active":"2023-11-26T07:17:02.570791+0000","last_became_peered":"2023-11-26T07:17:02.570791+0000","last_unstale":"2023-11-26T07:17:03.559742+0000","last_undegraded":"2023-11-26T07:17:03.559742+0000","last_fullsized":"2023-11-26T07:17:03.559742+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:58.389755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.558565+0000","last_change":"2023-11-26T07:17:01.579199+0000","last_active":"2023-11-26T07:17:02.558565+0000","last_peered":"2023-11-26T07:17:02.558565+0000","last_clean":"2023-11-26T07:17:02.558565+0000","last_became_active":"2023-11-26T07:17:01.578831+0000","last_became_peered":"2023-11-26T07:17:01.578831+0000","last_unstale":"2023-11-26T07:17:02.558565+0000","last_undegraded":"2023-11-26T07:17:02.558565+0000","last_fullsized":"2023-11-26T07:17:02.558565+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:27.918900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.562470+0000","last_change":"2023-11-26T07:17:02.576186+0000","last_active":"2023-11-26T07:17:03.562470+0000","last_peered":"2023-11-26T07:17:03.562470+0000","last_clean":"2023-11-26T07:17:03.562470+0000","last_became_active":"2023-11-26T07:17:02.575861+0000","last_became_peered":"2023-11-26T07:17:02.575861+0000","last_unstale":"2023-11-26T07:17:03.562470+0000","last_undegraded":"2023-11-26T07:17:03.562470+0000","last_fullsized":"2023-11-26T07:17:03.562470+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:08:10.681717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.561540+0000","last_change":"2023-11-26T07:17:01.580094+0000","last_active":"2023-11-26T07:17:02.561540+0000","last_peered":"2023-11-26T07:17:02.561540+0000","last_clean":"2023-11-26T07:17:02.561540+0000","last_became_active":"2023-11-26T07:17:01.579827+0000","last_became_peered":"2023-11-26T07:17:01.579827+0000","last_unstale":"2023-11-26T07:17:02.561540+0000","last_undegraded":"2023-11-26T07:17:02.561540+0000","last_fullsized":"2023-11-26T07:17:02.561540+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:49.417170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.562830+0000","last_change":"2023-11-26T07:17:02.569496+0000","last_active":"2023-11-26T07:17:03.562830+0000","last_peered":"2023-11-26T07:17:03.562830+0000","last_clean":"2023-11-26T07:17:03.562830+0000","last_became_active":"2023-11-26T07:17:02.569221+0000","last_became_peered":"2023-11-26T07:17:02.569221+0000","last_unstale":"2023-11-26T07:17:03.562830+0000","last_undegraded":"2023-11-26T07:17:03.562830+0000","last_fullsized":"2023-11-26T07:17:03.562830+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:47.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.559321+0000","last_change":"2023-11-26T07:17:01.588205+0000","last_active":"2023-11-26T07:17:02.559321+0000","last_peered":"2023-11-26T07:17:02.559321+0000","last_clean":"2023-11-26T07:17:02.559321+0000","last_became_active":"2023-11-26T07:17:01.587870+0000","last_became_peered":"2023-11-26T07:17:01.587870+0000","last_unstale":"2023-11-26T07:17:02.559321+0000","last_undegraded":"2023-11-26T07:17:02.559321+0000","last_fullsized":"2023-11-26T07:17:02.559321+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:50:06.014213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626417+0000","last_change":"2023-11-26T07:17:02.574834+0000","last_active":"2023-11-26T07:17:03.626417+0000","last_peered":"2023-11-26T07:17:03.626417+0000","last_clean":"2023-11-26T07:17:03.626417+0000","last_became_active":"2023-11-26T07:17:02.574559+0000","last_became_peered":"2023-11-26T07:17:02.574559+0000","last_unstale":"2023-11-26T07:17:03.626417+0000","last_undegraded":"2023-11-26T07:17:03.626417+0000","last_fullsized":"2023-11-26T07:17:03.626417+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:39.771470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.556759+0000","last_change":"2023-11-26T07:17:01.584174+0000","last_active":"2023-11-26T07:17:02.556759+0000","last_peered":"2023-11-26T07:17:02.556759+0000","last_clean":"2023-11-26T07:17:02.556759+0000","last_became_active":"2023-11-26T07:17:01.583882+0000","last_became_peered":"2023-11-26T07:17:01.583882+0000","last_unstale":"2023-11-26T07:17:02.556759+0000","last_undegraded":"2023-11-26T07:17:02.556759+0000","last_fullsized":"2023-11-26T07:17:02.556759+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:56:19.009627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625207+0000","last_change":"2023-11-26T07:17:02.568393+0000","last_active":"2023-11-26T07:17:03.625207+0000","last_peered":"2023-11-26T07:17:03.625207+0000","last_clean":"2023-11-26T07:17:03.625207+0000","last_became_active":"2023-11-26T07:17:02.568127+0000","last_became_peered":"2023-11-26T07:17:02.568127+0000","last_unstale":"2023-11-26T07:17:03.625207+0000","last_undegraded":"2023-11-26T07:17:03.625207+0000","last_fullsized":"2023-11-26T07:17:03.625207+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:24.997966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"101'3","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.595024+0000","last_change":"2023-11-26T07:17:01.587817+0000","last_active":"2023-11-26T07:17:02.595024+0000","last_peered":"2023-11-26T07:17:02.595024+0000","last_clean":"2023-11-26T07:17:02.595024+0000","last_became_active":"2023-11-26T07:17:01.587506+0000","last_became_peered":"2023-11-26T07:17:01.587506+0000","last_unstale":"2023-11-26T07:17:02.595024+0000","last_undegraded":"2023-11-26T07:17:02.595024+0000","last_fullsized":"2023-11-26T07:17:02.595024+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:17:03.354854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.562571+0000","last_change":"2023-11-26T07:17:02.569186+0000","last_active":"2023-11-26T07:17:03.562571+0000","last_peered":"2023-11-26T07:17:03.562571+0000","last_clean":"2023-11-26T07:17:03.562571+0000","last_became_active":"2023-11-26T07:17:02.568872+0000","last_became_peered":"2023-11-26T07:17:02.568872+0000","last_unstale":"2023-11-26T07:17:03.562571+0000","last_undegraded":"2023-11-26T07:17:03.562571+0000","last_fullsized":"2023-11-26T07:17:03.562571+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:50.939570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"101'6","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.597881+0000","last_change":"2023-11-26T07:17:01.583421+0000","last_active":"2023-11-26T07:17:02.597881+0000","last_peered":"2023-11-26T07:17:02.597881+0000","last_clean":"2023-11-26T07:17:02.597881+0000","last_became_active":"2023-11-26T07:17:01.582917+0000","last_became_peered":"2023-11-26T07:17:01.582917+0000","last_unstale":"2023-11-26T07:17:02.597881+0000","last_undegraded":"2023-11-26T07:17:02.597881+0000","last_fullsized":"2023-11-26T07:17:02.597881+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:11.911032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625919+0000","last_change":"2023-11-26T07:17:02.567391+0000","last_active":"2023-11-26T07:17:03.625919+0000","last_peered":"2023-11-26T07:17:03.625919+0000","last_clean":"2023-11-26T07:17:03.625919+0000","last_became_active":"2023-11-26T07:17:02.567100+0000","last_became_peered":"2023-11-26T07:17:02.567100+0000","last_unstale":"2023-11-26T07:17:03.625919+0000","last_undegraded":"2023-11-26T07:17:03.625919+0000","last_fullsized":"2023-11-26T07:17:03.625919+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:50.422694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.594586+0000","last_change":"2023-11-26T07:17:01.584515+0000","last_active":"2023-11-26T07:17:02.594586+0000","last_peered":"2023-11-26T07:17:02.594586+0000","last_clean":"2023-11-26T07:17:02.594586+0000","last_became_active":"2023-11-26T07:17:01.584221+0000","last_became_peered":"2023-11-26T07:17:01.584221+0000","last_unstale":"2023-11-26T07:17:02.594586+0000","last_undegraded":"2023-11-26T07:17:02.594586+0000","last_fullsized":"2023-11-26T07:17:02.594586+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:35:29.667246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625574+0000","last_change":"2023-11-26T07:17:02.566731+0000","last_active":"2023-11-26T07:17:03.625574+0000","last_peered":"2023-11-26T07:17:03.625574+0000","last_clean":"2023-11-26T07:17:03.625574+0000","last_became_active":"2023-11-26T07:17:02.566432+0000","last_became_peered":"2023-11-26T07:17:02.566432+0000","last_unstale":"2023-11-26T07:17:03.625574+0000","last_undegraded":"2023-11-26T07:17:03.625574+0000","last_fullsized":"2023-11-26T07:17:03.625574+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:05.947486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.571342+0000","last_change":"2023-11-26T07:17:01.586046+0000","last_active":"2023-11-26T07:17:02.571342+0000","last_peered":"2023-11-26T07:17:02.571342+0000","last_clean":"2023-11-26T07:17:02.571342+0000","last_became_active":"2023-11-26T07:17:01.585739+0000","last_became_peered":"2023-11-26T07:17:01.585739+0000","last_unstale":"2023-11-26T07:17:02.571342+0000","last_undegraded":"2023-11-26T07:17:02.571342+0000","last_fullsized":"2023-11-26T07:17:02.571342+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:36.667119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625216+0000","last_change":"2023-11-26T07:17:02.568173+0000","last_active":"2023-11-26T07:17:03.625216+0000","last_peered":"2023-11-26T07:17:03.625216+0000","last_clean":"2023-11-26T07:17:03.625216+0000","last_became_active":"2023-11-26T07:17:02.567887+0000","last_became_peered":"2023-11-26T07:17:02.567887+0000","last_unstale":"2023-11-26T07:17:03.625216+0000","last_undegraded":"2023-11-26T07:17:03.625216+0000","last_fullsized":"2023-11-26T07:17:03.625216+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:17:22.322087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"101'8","reported_seq":20,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.595220+0000","last_change":"2023-11-26T07:17:01.578474+0000","last_active":"2023-11-26T07:17:02.595220+0000","last_peered":"2023-11-26T07:17:02.595220+0000","last_clean":"2023-11-26T07:17:02.595220+0000","last_became_active":"2023-11-26T07:17:01.578190+0000","last_became_peered":"2023-11-26T07:17:01.578190+0000","last_unstale":"2023-11-26T07:17:02.595220+0000","last_undegraded":"2023-11-26T07:17:02.595220+0000","last_fullsized":"2023-11-26T07:17:02.595220+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:12:55.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625951+0000","last_change":"2023-11-26T07:17:02.573796+0000","last_active":"2023-11-26T07:17:03.625951+0000","last_peered":"2023-11-26T07:17:03.625951+0000","last_clean":"2023-11-26T07:17:03.625951+0000","last_became_active":"2023-11-26T07:17:02.573389+0000","last_became_peered":"2023-11-26T07:17:02.573389+0000","last_unstale":"2023-11-26T07:17:03.625951+0000","last_undegraded":"2023-11-26T07:17:03.625951+0000","last_fullsized":"2023-11-26T07:17:03.625951+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:50:51.562727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.594818+0000","last_change":"2023-11-26T07:17:01.582348+0000","last_active":"2023-11-26T07:17:02.594818+0000","last_peered":"2023-11-26T07:17:02.594818+0000","last_clean":"2023-11-26T07:17:02.594818+0000","last_became_active":"2023-11-26T07:17:01.582055+0000","last_became_peered":"2023-11-26T07:17:01.582055+0000","last_unstale":"2023-11-26T07:17:02.594818+0000","last_undegraded":"2023-11-26T07:17:02.594818+0000","last_fullsized":"2023-11-26T07:17:02.594818+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:26:01.288923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626940+0000","last_change":"2023-11-26T07:17:02.571365+0000","last_active":"2023-11-26T07:17:03.626940+0000","last_peered":"2023-11-26T07:17:03.626940+0000","last_clean":"2023-11-26T07:17:03.626940+0000","last_became_active":"2023-11-26T07:17:02.571091+0000","last_became_peered":"2023-11-26T07:17:02.571091+0000","last_unstale":"2023-11-26T07:17:03.626940+0000","last_undegraded":"2023-11-26T07:17:03.626940+0000","last_fullsized":"2023-11-26T07:17:03.626940+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:08:00.531777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.592985+0000","last_change":"2023-11-26T07:17:01.580502+0000","last_active":"2023-11-26T07:17:02.592985+0000","last_peered":"2023-11-26T07:17:02.592985+0000","last_clean":"2023-11-26T07:17:02.592985+0000","last_became_active":"2023-11-26T07:17:01.580205+0000","last_became_peered":"2023-11-26T07:17:01.580205+0000","last_unstale":"2023-11-26T07:17:02.592985+0000","last_undegraded":"2023-11-26T07:17:02.592985+0000","last_fullsized":"2023-11-26T07:17:02.592985+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:57:10.571315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563545+0000","last_change":"2023-11-26T07:17:02.574330+0000","last_active":"2023-11-26T07:17:03.563545+0000","last_peered":"2023-11-26T07:17:03.563545+0000","last_clean":"2023-11-26T07:17:03.563545+0000","last_became_active":"2023-11-26T07:17:02.574046+0000","last_became_peered":"2023-11-26T07:17:02.574046+0000","last_unstale":"2023-11-26T07:17:03.563545+0000","last_undegraded":"2023-11-26T07:17:03.563545+0000","last_fullsized":"2023-11-26T07:17:03.563545+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:17:22.322087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.593922+0000","last_change":"2023-11-26T07:17:01.585237+0000","last_active":"2023-11-26T07:17:02.593922+0000","last_peered":"2023-11-26T07:17:02.593922+0000","last_clean":"2023-11-26T07:17:02.593922+0000","last_became_active":"2023-11-26T07:17:01.584906+0000","last_became_peered":"2023-11-26T07:17:01.584906+0000","last_unstale":"2023-11-26T07:17:02.593922+0000","last_undegraded":"2023-11-26T07:17:02.593922+0000","last_fullsized":"2023-11-26T07:17:02.593922+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:57:11.459533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563902+0000","last_change":"2023-11-26T07:17:02.568317+0000","last_active":"2023-11-26T07:17:03.563902+0000","last_peered":"2023-11-26T07:17:03.563902+0000","last_clean":"2023-11-26T07:17:03.563902+0000","last_became_active":"2023-11-26T07:17:02.567770+0000","last_became_peered":"2023-11-26T07:17:02.567770+0000","last_unstale":"2023-11-26T07:17:03.563902+0000","last_undegraded":"2023-11-26T07:17:03.563902+0000","last_fullsized":"2023-11-26T07:17:03.563902+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:00.281782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.562531+0000","last_change":"2023-11-26T07:17:01.586521+0000","last_active":"2023-11-26T07:17:02.562531+0000","last_peered":"2023-11-26T07:17:02.562531+0000","last_clean":"2023-11-26T07:17:02.562531+0000","last_became_active":"2023-11-26T07:17:01.586279+0000","last_became_peered":"2023-11-26T07:17:01.586279+0000","last_unstale":"2023-11-26T07:17:02.562531+0000","last_undegraded":"2023-11-26T07:17:02.562531+0000","last_fullsized":"2023-11-26T07:17:02.562531+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:32.725095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563532+0000","last_change":"2023-11-26T07:17:02.569825+0000","last_active":"2023-11-26T07:17:03.563532+0000","last_peered":"2023-11-26T07:17:03.563532+0000","last_clean":"2023-11-26T07:17:03.563532+0000","last_became_active":"2023-11-26T07:17:02.569514+0000","last_became_peered":"2023-11-26T07:17:02.569514+0000","last_unstale":"2023-11-26T07:17:03.563532+0000","last_undegraded":"2023-11-26T07:17:03.563532+0000","last_fullsized":"2023-11-26T07:17:03.563532+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:24.997966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.592762+0000","last_change":"2023-11-26T07:17:01.583363+0000","last_active":"2023-11-26T07:17:02.592762+0000","last_peered":"2023-11-26T07:17:02.592762+0000","last_clean":"2023-11-26T07:17:02.592762+0000","last_became_active":"2023-11-26T07:17:01.583097+0000","last_became_peered":"2023-11-26T07:17:01.583097+0000","last_unstale":"2023-11-26T07:17:02.592762+0000","last_undegraded":"2023-11-26T07:17:02.592762+0000","last_fullsized":"2023-11-26T07:17:02.592762+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:53:30.894666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559588+0000","last_change":"2023-11-26T07:17:02.573267+0000","last_active":"2023-11-26T07:17:03.559588+0000","last_peered":"2023-11-26T07:17:03.559588+0000","last_clean":"2023-11-26T07:17:03.559588+0000","last_became_active":"2023-11-26T07:17:02.572996+0000","last_became_peered":"2023-11-26T07:17:02.572996+0000","last_unstale":"2023-11-26T07:17:03.559588+0000","last_undegraded":"2023-11-26T07:17:03.559588+0000","last_fullsized":"2023-11-26T07:17:03.559588+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:25:53.916291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.559713+0000","last_change":"2023-11-26T07:17:01.588512+0000","last_active":"2023-11-26T07:17:02.559713+0000","last_peered":"2023-11-26T07:17:02.559713+0000","last_clean":"2023-11-26T07:17:02.559713+0000","last_became_active":"2023-11-26T07:17:01.588116+0000","last_became_peered":"2023-11-26T07:17:01.588116+0000","last_unstale":"2023-11-26T07:17:02.559713+0000","last_undegraded":"2023-11-26T07:17:02.559713+0000","last_fullsized":"2023-11-26T07:17:02.559713+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:16:36.858554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563397+0000","last_change":"2023-11-26T07:17:02.567641+0000","last_active":"2023-11-26T07:17:03.563397+0000","last_peered":"2023-11-26T07:17:03.563397+0000","last_clean":"2023-11-26T07:17:03.563397+0000","last_became_active":"2023-11-26T07:17:02.567326+0000","last_became_peered":"2023-11-26T07:17:02.567326+0000","last_unstale":"2023-11-26T07:17:03.563397+0000","last_undegraded":"2023-11-26T07:17:03.563397+0000","last_fullsized":"2023-11-26T07:17:03.563397+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:59.383908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.563397+0000","last_change":"2023-11-26T07:17:01.588360+0000","last_active":"2023-11-26T07:17:02.563397+0000","last_peered":"2023-11-26T07:17:02.563397+0000","last_clean":"2023-11-26T07:17:02.563397+0000","last_became_active":"2023-11-26T07:17:01.587441+0000","last_became_peered":"2023-11-26T07:17:01.587441+0000","last_unstale":"2023-11-26T07:17:02.563397+0000","last_undegraded":"2023-11-26T07:17:02.563397+0000","last_fullsized":"2023-11-26T07:17:02.563397+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:45.951301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625268+0000","last_change":"2023-11-26T07:17:02.568858+0000","last_active":"2023-11-26T07:17:03.625268+0000","last_peered":"2023-11-26T07:17:03.625268+0000","last_clean":"2023-11-26T07:17:03.625268+0000","last_became_active":"2023-11-26T07:17:02.568340+0000","last_became_peered":"2023-11-26T07:17:02.568340+0000","last_unstale":"2023-11-26T07:17:03.625268+0000","last_undegraded":"2023-11-26T07:17:03.625268+0000","last_fullsized":"2023-11-26T07:17:03.625268+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:53:36.673023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.561685+0000","last_change":"2023-11-26T07:17:01.584757+0000","last_active":"2023-11-26T07:17:02.561685+0000","last_peered":"2023-11-26T07:17:02.561685+0000","last_clean":"2023-11-26T07:17:02.561685+0000","last_became_active":"2023-11-26T07:17:01.582979+0000","last_became_peered":"2023-11-26T07:17:01.582979+0000","last_unstale":"2023-11-26T07:17:02.561685+0000","last_undegraded":"2023-11-26T07:17:02.561685+0000","last_fullsized":"2023-11-26T07:17:02.561685+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:51.644212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626263+0000","last_change":"2023-11-26T07:17:02.573945+0000","last_active":"2023-11-26T07:17:03.626263+0000","last_peered":"2023-11-26T07:17:03.626263+0000","last_clean":"2023-11-26T07:17:03.626263+0000","last_became_active":"2023-11-26T07:17:02.573683+0000","last_became_peered":"2023-11-26T07:17:02.573683+0000","last_unstale":"2023-11-26T07:17:03.626263+0000","last_undegraded":"2023-11-26T07:17:03.626263+0000","last_fullsized":"2023-11-26T07:17:03.626263+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:24.170651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.594573+0000","last_change":"2023-11-26T07:17:01.584426+0000","last_active":"2023-11-26T07:17:02.594573+0000","last_peered":"2023-11-26T07:17:02.594573+0000","last_clean":"2023-11-26T07:17:02.594573+0000","last_became_active":"2023-11-26T07:17:01.584060+0000","last_became_peered":"2023-11-26T07:17:01.584060+0000","last_unstale":"2023-11-26T07:17:02.594573+0000","last_undegraded":"2023-11-26T07:17:02.594573+0000","last_fullsized":"2023-11-26T07:17:02.594573+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:27.918900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.560316+0000","last_change":"2023-11-26T07:17:02.570101+0000","last_active":"2023-11-26T07:17:03.560316+0000","last_peered":"2023-11-26T07:17:03.560316+0000","last_clean":"2023-11-26T07:17:03.560316+0000","last_became_active":"2023-11-26T07:17:02.569816+0000","last_became_peered":"2023-11-26T07:17:02.569816+0000","last_unstale":"2023-11-26T07:17:03.560316+0000","last_undegraded":"2023-11-26T07:17:03.560316+0000","last_fullsized":"2023-11-26T07:17:03.560316+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:11:18.280217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.571636+0000","last_change":"2023-11-26T07:17:01.582420+0000","last_active":"2023-11-26T07:17:02.571636+0000","last_peered":"2023-11-26T07:17:02.571636+0000","last_clean":"2023-11-26T07:17:02.571636+0000","last_became_active":"2023-11-26T07:17:01.582134+0000","last_became_peered":"2023-11-26T07:17:01.582134+0000","last_unstale":"2023-11-26T07:17:02.571636+0000","last_undegraded":"2023-11-26T07:17:02.571636+0000","last_fullsized":"2023-11-26T07:17:02.571636+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:56:19.009627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559983+0000","last_change":"2023-11-26T07:17:02.570866+0000","last_active":"2023-11-26T07:17:03.559983+0000","last_peered":"2023-11-26T07:17:03.559983+0000","last_clean":"2023-11-26T07:17:03.559983+0000","last_became_active":"2023-11-26T07:17:02.570277+0000","last_became_peered":"2023-11-26T07:17:02.570277+0000","last_unstale":"2023-11-26T07:17:03.559983+0000","last_undegraded":"2023-11-26T07:17:03.559983+0000","last_fullsized":"2023-11-26T07:17:03.559983+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:23:58.105639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.562069+0000","last_change":"2023-11-26T07:17:01.583471+0000","last_active":"2023-11-26T07:17:02.562069+0000","last_peered":"2023-11-26T07:17:02.562069+0000","last_clean":"2023-11-26T07:17:02.562069+0000","last_became_active":"2023-11-26T07:17:01.583032+0000","last_became_peered":"2023-11-26T07:17:01.583032+0000","last_unstale":"2023-11-26T07:17:02.562069+0000","last_undegraded":"2023-11-26T07:17:02.562069+0000","last_fullsized":"2023-11-26T07:17:02.562069+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:53:59.068613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625703+0000","last_change":"2023-11-26T07:17:02.574936+0000","last_active":"2023-11-26T07:17:03.625703+0000","last_peered":"2023-11-26T07:17:03.625703+0000","last_clean":"2023-11-26T07:17:03.625703+0000","last_became_active":"2023-11-26T07:17:02.574568+0000","last_became_peered":"2023-11-26T07:17:02.574568+0000","last_unstale":"2023-11-26T07:17:03.625703+0000","last_undegraded":"2023-11-26T07:17:03.625703+0000","last_fullsized":"2023-11-26T07:17:03.625703+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:04:23.067790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.557196+0000","last_change":"2023-11-26T07:17:01.590145+0000","last_active":"2023-11-26T07:17:02.557196+0000","last_peered":"2023-11-26T07:17:02.557196+0000","last_clean":"2023-11-26T07:17:02.557196+0000","last_became_active":"2023-11-26T07:17:01.589827+0000","last_became_peered":"2023-11-26T07:17:01.589827+0000","last_unstale":"2023-11-26T07:17:02.557196+0000","last_undegraded":"2023-11-26T07:17:02.557196+0000","last_fullsized":"2023-11-26T07:17:02.557196+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:38.765945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625984+0000","last_change":"2023-11-26T07:17:02.566604+0000","last_active":"2023-11-26T07:17:03.625984+0000","last_peered":"2023-11-26T07:17:03.625984+0000","last_clean":"2023-11-26T07:17:03.625984+0000","last_became_active":"2023-11-26T07:17:02.566197+0000","last_became_peered":"2023-11-26T07:17:02.566197+0000","last_unstale":"2023-11-26T07:17:03.625984+0000","last_undegraded":"2023-11-26T07:17:03.625984+0000","last_fullsized":"2023-11-26T07:17:03.625984+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:16.936993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.562707+0000","last_change":"2023-11-26T07:17:01.584165+0000","last_active":"2023-11-26T07:17:02.562707+0000","last_peered":"2023-11-26T07:17:02.562707+0000","last_clean":"2023-11-26T07:17:02.562707+0000","last_became_active":"2023-11-26T07:17:01.583874+0000","last_became_peered":"2023-11-26T07:17:01.583874+0000","last_unstale":"2023-11-26T07:17:02.562707+0000","last_undegraded":"2023-11-26T07:17:02.562707+0000","last_fullsized":"2023-11-26T07:17:02.562707+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:29.830617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626111+0000","last_change":"2023-11-26T07:17:02.566616+0000","last_active":"2023-11-26T07:17:03.626111+0000","last_peered":"2023-11-26T07:17:03.626111+0000","last_clean":"2023-11-26T07:17:03.626111+0000","last_became_active":"2023-11-26T07:17:02.565131+0000","last_became_peered":"2023-11-26T07:17:02.565131+0000","last_unstale":"2023-11-26T07:17:03.626111+0000","last_undegraded":"2023-11-26T07:17:03.626111+0000","last_fullsized":"2023-11-26T07:17:03.626111+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:24.997966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.558113+0000","last_change":"2023-11-26T07:17:01.588026+0000","last_active":"2023-11-26T07:17:02.558113+0000","last_peered":"2023-11-26T07:17:02.558113+0000","last_clean":"2023-11-26T07:17:02.558113+0000","last_became_active":"2023-11-26T07:17:01.587682+0000","last_became_peered":"2023-11-26T07:17:01.587682+0000","last_unstale":"2023-11-26T07:17:02.558113+0000","last_undegraded":"2023-11-26T07:17:02.558113+0000","last_fullsized":"2023-11-26T07:17:02.558113+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:46.666070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563604+0000","last_change":"2023-11-26T07:17:02.576408+0000","last_active":"2023-11-26T07:17:03.563604+0000","last_peered":"2023-11-26T07:17:03.563604+0000","last_clean":"2023-11-26T07:17:03.563604+0000","last_became_active":"2023-11-26T07:17:02.576118+0000","last_became_peered":"2023-11-26T07:17:02.576118+0000","last_unstale":"2023-11-26T07:17:03.563604+0000","last_undegraded":"2023-11-26T07:17:03.563604+0000","last_fullsized":"2023-11-26T07:17:03.563604+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:28:00.740325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.559746+0000","last_change":"2023-11-26T07:17:01.588719+0000","last_active":"2023-11-26T07:17:02.559746+0000","last_peered":"2023-11-26T07:17:02.559746+0000","last_clean":"2023-11-26T07:17:02.559746+0000","last_became_active":"2023-11-26T07:17:01.587960+0000","last_became_peered":"2023-11-26T07:17:01.587960+0000","last_unstale":"2023-11-26T07:17:02.559746+0000","last_undegraded":"2023-11-26T07:17:02.559746+0000","last_fullsized":"2023-11-26T07:17:02.559746+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:35:29.667246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559518+0000","last_change":"2023-11-26T07:17:02.570394+0000","last_active":"2023-11-26T07:17:03.559518+0000","last_peered":"2023-11-26T07:17:03.559518+0000","last_clean":"2023-11-26T07:17:03.559518+0000","last_became_active":"2023-11-26T07:17:02.570105+0000","last_became_peered":"2023-11-26T07:17:02.570105+0000","last_unstale":"2023-11-26T07:17:03.559518+0000","last_undegraded":"2023-11-26T07:17:03.559518+0000","last_fullsized":"2023-11-26T07:17:03.559518+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:22.278810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.562198+0000","last_change":"2023-11-26T07:17:01.585608+0000","last_active":"2023-11-26T07:17:02.562198+0000","last_peered":"2023-11-26T07:17:02.562198+0000","last_clean":"2023-11-26T07:17:02.562198+0000","last_became_active":"2023-11-26T07:17:01.583084+0000","last_became_peered":"2023-11-26T07:17:01.583084+0000","last_unstale":"2023-11-26T07:17:02.562198+0000","last_undegraded":"2023-11-26T07:17:02.562198+0000","last_fullsized":"2023-11-26T07:17:02.562198+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:08:12.474873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626046+0000","last_change":"2023-11-26T07:17:02.570586+0000","last_active":"2023-11-26T07:17:03.626046+0000","last_peered":"2023-11-26T07:17:03.626046+0000","last_clean":"2023-11-26T07:17:03.626046+0000","last_became_active":"2023-11-26T07:17:02.570242+0000","last_became_peered":"2023-11-26T07:17:02.570242+0000","last_unstale":"2023-11-26T07:17:03.626046+0000","last_undegraded":"2023-11-26T07:17:03.626046+0000","last_fullsized":"2023-11-26T07:17:03.626046+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:28:54.491028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.564818+0000","last_change":"2023-11-26T07:17:01.585782+0000","last_active":"2023-11-26T07:17:02.564818+0000","last_peered":"2023-11-26T07:17:02.564818+0000","last_clean":"2023-11-26T07:17:02.564818+0000","last_became_active":"2023-11-26T07:17:01.585381+0000","last_became_peered":"2023-11-26T07:17:01.585381+0000","last_unstale":"2023-11-26T07:17:02.564818+0000","last_undegraded":"2023-11-26T07:17:02.564818+0000","last_fullsized":"2023-11-26T07:17:02.564818+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:58:00.348529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":287,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.560818+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:17:02.560818+0000","last_peered":"2023-11-26T07:17:02.560818+0000","last_clean":"2023-11-26T07:17:02.560818+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:17:02.560818+0000","last_undegraded":"2023-11-26T07:17:02.560818+0000","last_fullsized":"2023-11-26T07:17:02.560818+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1871872,"data_stored":1147353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.56200000000000006},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74299999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58699999999999997},"min":{"1min":0.40799999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88100000000000001,"5min":1.048,"15min":1.048},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.315,"15min":0.315},"max":{"1min":0.71599999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.436}]},{"osd":2,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82499999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.622},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79700000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.40699999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.46400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.876,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.45900000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.49199999999999999},"min":{"1min":0.34200000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.45500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.64100000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":1.171,"15min":1.171},"last":0.59399999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.39500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.93899999999999995,"5min":1.361,"15min":1.361},"last":0.629},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0409999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.68300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28272,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343568,"statfs":{"total":96636764160,"available":96607813632,"internally_reserved":0,"allocated":1273856,"data_stored":556439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.747,"5min":1.159,"15min":1.159},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.61899999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.44},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.40600000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72299999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.65300000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.60499999999999998,"15min":0.55200000000000005},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.8689999999999998,"5min":4.8689999999999998,"15min":4.8689999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70999999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.38800000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.501,"15min":0.51300000000000001},"min":{"1min":0.29399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72599999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73299999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56799999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.74099999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.41699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.57299999999999995},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76400000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.501}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":556493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.34899999999999998,"15min":0.33500000000000002},"min":{"1min":0.221,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.311},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35099999999999998,"15min":0.34699999999999998},"min":{"1min":0.23400000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.41599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34799999999999998,"15min":0.33600000000000002},"min":{"1min":0.254,"5min":0.214,"15min":0.214},"max":{"1min":0.48099999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.38800000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.39600000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40699999999999997,"15min":0.39100000000000001},"min":{"1min":0.30399999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.59099999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.441},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.32900000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.34300000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60999999999999999,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.69399999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.879},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61699999999999999,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40600000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91700000000000004,"5min":1.129,"15min":1.129},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86199999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63700000000000001,"15min":0.628},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.435,"15min":0.435},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":1148461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.34899999999999998,"15min":0.35599999999999998},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.53200000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.247},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.372,"15min":0.36799999999999999},"min":{"1min":0.24299999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.44900000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.29599999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.249,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48999999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36799999999999999,"15min":0.35099999999999998},"min":{"1min":0.26000000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.315}]},{"osd":3,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.39100000000000001,"15min":0.40000000000000002},"min":{"1min":0.28000000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60899999999999999,"5min":0.75,"15min":0.75},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.42599999999999999,"15min":0.436},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.45900000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.377}]},{"osd":4,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.626,"15min":0.60999999999999999},"min":{"1min":0.436,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.64900000000000002,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.84299999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.63},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.752,"5min":1.046,"15min":1.046},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61699999999999999,"15min":0.623},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.44500000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.45200000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.628},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63600000000000001,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77200000000000002,"5min":1.087,"15min":1.087},"last":0.48399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":556369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.35099999999999998,"15min":0.36599999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47699999999999998,"5min":0.627,"15min":0.627},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.36299999999999999,"15min":0.374},"min":{"1min":0.24099999999999999,"5min":0.186,"15min":0.186},"max":{"1min":0.55000000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.504}]},{"osd":1,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51200000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.29899999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.51000000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.47499999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36799999999999999,"15min":0.371},"min":{"1min":0.24399999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.51300000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.44},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41099999999999998,"15min":0.42299999999999999},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54600000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48899999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60999999999999999,"15min":0.66300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.69999999999999996,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.44500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84399999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.67800000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60899999999999999,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73599999999999999,"5min":1.075,"15min":1.075},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76500000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.70099999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.751,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.628},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.44600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.76900000000000002,"5min":1.03,"15min":1.03},"last":0.66800000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.65400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.627,"15min":0.623},"min":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82399999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.71299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":558051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.34499999999999997,"15min":0.35699999999999998},"min":{"1min":0.20699999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.44700000000000001,"5min":1.284,"15min":1.284},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.39900000000000002,"15min":0.46300000000000002},"min":{"1min":0.23999999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59899999999999998,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.26600000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.39100000000000001,"15min":0.499},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.47299999999999998,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.222},{"interface":"front","average":{"1min":0.38,"5min":0.41999999999999998,"15min":0.48299999999999998},"min":{"1min":0.26600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.55800000000000005,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.443,"15min":0.49399999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.52800000000000002,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.42199999999999999},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51900000000000002,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.28699999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.622,"15min":0.66600000000000004},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81299999999999994,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.747},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.624,"15min":0.64600000000000002},"min":{"1min":0.45000000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85999999999999999,"5min":1.405,"15min":1.405},"last":0.57799999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61399999999999999,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":1.377,"15min":1.377},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66800000000000004,"15min":0.77900000000000003},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97599999999999998,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.68700000000000006}]},{"osd":6,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63,"15min":0.66400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93500000000000005,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.67100000000000004,"15min":0.77400000000000002},"min":{"1min":0.47399999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84999999999999998,"5min":4.883,"15min":4.883},"last":0.53900000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65900000000000003,"15min":0.78700000000000003},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89700000000000002,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.66200000000000003,"15min":0.72399999999999998},"min":{"1min":0.443,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.873,"5min":2.923,"15min":2.923},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28268,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343572,"statfs":{"total":96636764160,"available":96607817728,"internally_reserved":0,"allocated":1269760,"data_stored":556369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57299999999999995,"15min":0.60599999999999998},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.78300000000000003,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54900000000000004,"15min":0.53200000000000003},"min":{"1min":0.39400000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75700000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.50800000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.52000000000000002},"min":{"1min":0.33500000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.74299999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.66600000000000004},"min":{"1min":0.40500000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.753,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.72599999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57299999999999995,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73499999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.622}]},{"osd":5,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55500000000000005,"15min":0.56200000000000006},"min":{"1min":0.39200000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65000000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.42599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73699999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.60399999999999998},"min":{"1min":0.42399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79800000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.79600000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.81000000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.625,"15min":0.63600000000000001},"min":{"1min":0.51000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.78800000000000003,"5min":1.01,"15min":1.01},"last":0.74299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28276,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343564,"statfs":{"total":96636764160,"available":96607809536,"internally_reserved":0,"allocated":1277952,"data_stored":556973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53000000000000003,"15min":0.50800000000000001},"min":{"1min":0.29799999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.54300000000000004},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.46000000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.71199999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.44800000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.55400000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.39300000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.751,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.58999999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.35599999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.60699999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84699999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.54800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:06.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.832+0000 7faf63bec700 1 Processor -- start 2023-11-26T07:17:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.832+0000 7faf63bec700 1 -- start start 2023-11-26T07:17:06.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf63bec700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faf5c15bdd0 0x7faf5c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf63bec700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf5c0b3370 0x7faf5c0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf63bec700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5c0b2030 con 0x7faf5c0d1c70 2023-11-26T07:17:06.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf63bec700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c0b2f50 con 0x7faf5c0b3370 2023-11-26T07:17:06.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf63bec700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c0b22d0 con 0x7faf5c15bdd0 2023-11-26T07:17:06.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf61187700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf5c0b3370 0x7faf5c0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf61187700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faf5c0b3370 0x7faf5c0b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41636/0 (socket says 172.21.15.83:41636) 2023-11-26T07:17:06.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf61187700 1 -- 172.21.15.83:0/3210292035 learned_addr learned my addr 172.21.15.83:0/3210292035 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf61988700 1 --2- 172.21.15.83:0/3210292035 >> v2:172.21.15.203:3300/0 conn(0x7faf5c15bdd0 0x7faf5c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf61187700 1 -- 172.21.15.83:0/3210292035 >> v2:172.21.15.203:3300/0 conn(0x7faf5c15bdd0 msgr2=0x7faf5c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf61187700 1 --2- 172.21.15.83:0/3210292035 >> v2:172.21.15.203:3300/0 conn(0x7faf5c15bdd0 0x7faf5c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf61187700 1 -- 172.21.15.83:0/3210292035 >> v1:172.21.15.203:6789/0 conn(0x7faf5c0d1c70 legacy=0x7faf5c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf61187700 1 -- 172.21.15.83:0/3210292035 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c0b25b0 con 0x7faf5c0b3370 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf61988700 1 --2- 172.21.15.83:0/3210292035 >> v2:172.21.15.203:3300/0 conn(0x7faf5c15bdd0 0x7faf5c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.833+0000 7faf61187700 1 --2- 172.21.15.83:0/3210292035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf5c0b3370 0x7faf5c0b1af0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7faf5c06ec10 tx=0x7faf58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c3373379afb593 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.834+0000 7faf60986700 1 -- 172.21.15.83:0/3210292035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5800b870 con 0x7faf5c0b3370 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.834+0000 7faf60986700 1 -- 172.21.15.83:0/3210292035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5800b9d0 con 0x7faf5c0b3370 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.834+0000 7faf60986700 1 -- 172.21.15.83:0/3210292035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5800bd60 con 0x7faf5c0b3370 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.834+0000 7faf63bec700 1 -- 172.21.15.83:0/3210292035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf5c0b3370 msgr2=0x7faf5c0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.834+0000 7faf63bec700 1 --2- 172.21.15.83:0/3210292035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf5c0b3370 0x7faf5c0b1af0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7faf5c06ec10 tx=0x7faf58009bd0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.834+0000 7faf63bec700 1 -- 172.21.15.83:0/3210292035 shutdown_connections 2023-11-26T07:17:06.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.834+0000 7faf63bec700 1 --2- 172.21.15.83:0/3210292035 >> v2:172.21.15.203:3300/0 conn(0x7faf5c15bdd0 0x7faf5c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.834+0000 7faf63bec700 1 --2- 172.21.15.83:0/3210292035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf5c0b3370 0x7faf5c0b1af0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.834+0000 7faf63bec700 1 -- 172.21.15.83:0/3210292035 >> 172.21.15.83:0/3210292035 conn(0x7faf5c0731a0 msgr2=0x7faf5c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:06.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.834+0000 7faf63bec700 1 -- 172.21.15.83:0/3210292035 shutdown_connections 2023-11-26T07:17:06.838 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.834+0000 7faf63bec700 1 -- 172.21.15.83:0/3210292035 wait complete. 2023-11-26T07:17:06.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.835+0000 7faf63bec700 1 Processor -- start 2023-11-26T07:17:06.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.835+0000 7faf63bec700 1 -- start start 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.836+0000 7faf63bec700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5c0b3370 0x7faf5c0b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.836+0000 7faf63bec700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf5c0d1c70 0x7faf5c0b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.836+0000 7faf63bec700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf5c15bdd0 0x7faf5c156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.836+0000 7faf63bec700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0b7570 con 0x7faf5c15bdd0 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.836+0000 7faf63bec700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0b77a0 con 0x7faf5c0d1c70 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.836+0000 7faf63bec700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c1572c0 con 0x7faf5c0b3370 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.836+0000 7faf62189700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf5c15bdd0 0x7faf5c156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.836+0000 7faf62189700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf5c15bdd0 0x7faf5c156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41644/0 (socket says 172.21.15.83:41644) 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.836+0000 7faf62189700 1 -- 172.21.15.83:0/385490328 learned_addr learned my addr 172.21.15.83:0/385490328 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.836+0000 7faf61988700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5c0b3370 0x7faf5c0b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.836+0000 7faf61187700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf5c0d1c70 0x7faf5c0b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.837+0000 7faf62189700 1 -- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5c0b3370 msgr2=0x7faf5c0b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.837+0000 7faf62189700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5c0b3370 0x7faf5c0b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.837+0000 7faf62189700 1 -- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf5c0d1c70 msgr2=0x7faf5c0b7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.837+0000 7faf62189700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf5c0d1c70 0x7faf5c0b7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.837+0000 7faf62189700 1 -- 172.21.15.83:0/385490328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf58009550 con 0x7faf5c15bdd0 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.837+0000 7faf62189700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf5c15bdd0 0x7faf5c156d80 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7faf5400cc50 tx=0x7faf5400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.837+0000 7faf527fc700 1 -- 172.21.15.83:0/385490328 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54003c00 con 0x7faf5c15bdd0 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.837+0000 7faf527fc700 1 -- 172.21.15.83:0/385490328 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf54003d60 con 0x7faf5c15bdd0 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.837+0000 7faf63bec700 1 -- 172.21.15.83:0/385490328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c15c880 con 0x7faf5c15bdd0 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.837+0000 7faf527fc700 1 -- 172.21.15.83:0/385490328 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5400aab0 con 0x7faf5c15bdd0 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.837+0000 7faf63bec700 1 -- 172.21.15.83:0/385490328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c15ce40 con 0x7faf5c15bdd0 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.838+0000 7faf63bec700 1 -- 172.21.15.83:0/385490328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c067f60 con 0x7faf5c15bdd0 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.838+0000 7faf527fc700 1 -- 172.21.15.83:0/385490328 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7faf5400ac10 con 0x7faf5c15bdd0 2023-11-26T07:17:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.839+0000 7faf527fc700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faf4006aff0 0x7faf4006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:06.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.839+0000 7faf527fc700 1 -- 172.21.15.83:0/385490328 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf54089740 con 0x7faf5c15bdd0 2023-11-26T07:17:06.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.839+0000 7faf61988700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faf4006aff0 0x7faf4006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:06.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.839+0000 7faf61988700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faf4006aff0 0x7faf4006d4b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c940 tx=0x7faf4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:06.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:06.843+0000 7faf527fc700 1 -- 172.21.15.83:0/385490328 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf540546d0 con 0x7faf5c15bdd0 2023-11-26T07:17:07.034 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.033+0000 7faf63bec700 1 -- 172.21.15.83:0/385490328 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf5c0b2b70 con 0x7faf4006aff0 2023-11-26T07:17:07.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.041+0000 7faf527fc700 1 -- 172.21.15.83:0/385490328 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289766 (secure 0 0 0) 0x7faf4c00ce60 con 0x7faf4006aff0 2023-11-26T07:17:07.046 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:07.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 -- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faf4006aff0 msgr2=0x7faf4006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faf4006aff0 0x7faf4006d4b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c940 tx=0x7faf4c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 -- 172.21.15.83:0/385490328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf5c15bdd0 msgr2=0x7faf5c156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf5c15bdd0 0x7faf5c156d80 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7faf5400cc50 tx=0x7faf5400cf60 comp rx=0 tx=0).stop 2023-11-26T07:17:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 -- 172.21.15.83:0/385490328 shutdown_connections 2023-11-26T07:17:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faf4006aff0 0x7faf4006d4b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf5c15bdd0 0x7faf5c156d80 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5c0b3370 0x7faf5c0b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 --2- 172.21.15.83:0/385490328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf5c0d1c70 0x7faf5c0b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 -- 172.21.15.83:0/385490328 >> 172.21.15.83:0/385490328 conn(0x7faf5c0731a0 msgr2=0x7faf5c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 -- 172.21.15.83:0/385490328 shutdown_connections 2023-11-26T07:17:07.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.045+0000 7faf63bec700 1 -- 172.21.15.83:0/385490328 wait complete. 2023-11-26T07:17:07.050 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:17:07.058 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2023-11-26T07:17:06.320904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248168,"kb_used_data":11144,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754726552,"statfs":{"total":773094113280,"available":772839989248,"internally_reserved":0,"allocated":11411456,"data_stored":5636508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.757198"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.560331+0000","last_change":"2023-11-26T07:17:02.575475+0000","last_active":"2023-11-26T07:17:03.560331+0000","last_peered":"2023-11-26T07:17:03.560331+0000","last_clean":"2023-11-26T07:17:03.560331+0000","last_became_active":"2023-11-26T07:17:02.575169+0000","last_became_peered":"2023-11-26T07:17:02.575169+0000","last_unstale":"2023-11-26T07:17:03.560331+0000","last_undegraded":"2023-11-26T07:17:03.560331+0000","last_fullsized":"2023-11-26T07:17:03.560331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:25:55.927869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625946+0000","last_change":"2023-11-26T07:17:02.568368+0000","last_active":"2023-11-26T07:17:03.625946+0000","last_peered":"2023-11-26T07:17:03.625946+0000","last_clean":"2023-11-26T07:17:03.625946+0000","last_became_active":"2023-11-26T07:17:02.568078+0000","last_became_peered":"2023-11-26T07:17:02.568078+0000","last_unstale":"2023-11-26T07:17:03.625946+0000","last_undegraded":"2023-11-26T07:17:03.625946+0000","last_fullsized":"2023-11-26T07:17:03.625946+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:11:27.335118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563685+0000","last_change":"2023-11-26T07:17:02.568861+0000","last_active":"2023-11-26T07:17:03.563685+0000","last_peered":"2023-11-26T07:17:03.563685+0000","last_clean":"2023-11-26T07:17:03.563685+0000","last_became_active":"2023-11-26T07:17:02.568578+0000","last_became_peered":"2023-11-26T07:17:02.568578+0000","last_unstale":"2023-11-26T07:17:03.563685+0000","last_undegraded":"2023-11-26T07:17:03.563685+0000","last_fullsized":"2023-11-26T07:17:03.563685+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:05.947486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559766+0000","last_change":"2023-11-26T07:17:02.569388+0000","last_active":"2023-11-26T07:17:03.559766+0000","last_peered":"2023-11-26T07:17:03.559766+0000","last_clean":"2023-11-26T07:17:03.559766+0000","last_became_active":"2023-11-26T07:17:02.569107+0000","last_became_peered":"2023-11-26T07:17:02.569107+0000","last_unstale":"2023-11-26T07:17:03.559766+0000","last_undegraded":"2023-11-26T07:17:03.559766+0000","last_fullsized":"2023-11-26T07:17:03.559766+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:05:12.369764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625454+0000","last_change":"2023-11-26T07:17:02.563096+0000","last_active":"2023-11-26T07:17:03.625454+0000","last_peered":"2023-11-26T07:17:03.625454+0000","last_clean":"2023-11-26T07:17:03.625454+0000","last_became_active":"2023-11-26T07:17:02.562658+0000","last_became_peered":"2023-11-26T07:17:02.562658+0000","last_unstale":"2023-11-26T07:17:03.625454+0000","last_undegraded":"2023-11-26T07:17:03.625454+0000","last_fullsized":"2023-11-26T07:17:03.625454+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:06:49.465873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626085+0000","last_change":"2023-11-26T07:17:02.566712+0000","last_active":"2023-11-26T07:17:03.626085+0000","last_peered":"2023-11-26T07:17:03.626085+0000","last_clean":"2023-11-26T07:17:03.626085+0000","last_became_active":"2023-11-26T07:17:02.566441+0000","last_became_peered":"2023-11-26T07:17:02.566441+0000","last_unstale":"2023-11-26T07:17:03.626085+0000","last_undegraded":"2023-11-26T07:17:03.626085+0000","last_fullsized":"2023-11-26T07:17:03.626085+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:56:03.418914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.562720+0000","last_change":"2023-11-26T07:17:02.576715+0000","last_active":"2023-11-26T07:17:03.562720+0000","last_peered":"2023-11-26T07:17:03.562720+0000","last_clean":"2023-11-26T07:17:03.562720+0000","last_became_active":"2023-11-26T07:17:02.576439+0000","last_became_peered":"2023-11-26T07:17:02.576439+0000","last_unstale":"2023-11-26T07:17:03.562720+0000","last_undegraded":"2023-11-26T07:17:03.562720+0000","last_fullsized":"2023-11-26T07:17:03.562720+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:57:37.656434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563387+0000","last_change":"2023-11-26T07:17:02.569790+0000","last_active":"2023-11-26T07:17:03.563387+0000","last_peered":"2023-11-26T07:17:03.563387+0000","last_clean":"2023-11-26T07:17:03.563387+0000","last_became_active":"2023-11-26T07:17:02.569501+0000","last_became_peered":"2023-11-26T07:17:02.569501+0000","last_unstale":"2023-11-26T07:17:03.563387+0000","last_undegraded":"2023-11-26T07:17:03.563387+0000","last_fullsized":"2023-11-26T07:17:03.563387+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:16.936993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.560200+0000","last_change":"2023-11-26T07:17:02.573486+0000","last_active":"2023-11-26T07:17:03.560200+0000","last_peered":"2023-11-26T07:17:03.560200+0000","last_clean":"2023-11-26T07:17:03.560200+0000","last_became_active":"2023-11-26T07:17:02.573157+0000","last_became_peered":"2023-11-26T07:17:02.573157+0000","last_unstale":"2023-11-26T07:17:03.560200+0000","last_undegraded":"2023-11-26T07:17:03.560200+0000","last_fullsized":"2023-11-26T07:17:03.560200+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:33:46.779337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559777+0000","last_change":"2023-11-26T07:17:02.572458+0000","last_active":"2023-11-26T07:17:03.559777+0000","last_peered":"2023-11-26T07:17:03.559777+0000","last_clean":"2023-11-26T07:17:03.559777+0000","last_became_active":"2023-11-26T07:17:02.572180+0000","last_became_peered":"2023-11-26T07:17:02.572180+0000","last_unstale":"2023-11-26T07:17:03.559777+0000","last_undegraded":"2023-11-26T07:17:03.559777+0000","last_fullsized":"2023-11-26T07:17:03.559777+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:33:31.205055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625966+0000","last_change":"2023-11-26T07:17:02.576567+0000","last_active":"2023-11-26T07:17:03.625966+0000","last_peered":"2023-11-26T07:17:03.625966+0000","last_clean":"2023-11-26T07:17:03.625966+0000","last_became_active":"2023-11-26T07:17:02.576308+0000","last_became_peered":"2023-11-26T07:17:02.576308+0000","last_unstale":"2023-11-26T07:17:03.625966+0000","last_undegraded":"2023-11-26T07:17:03.625966+0000","last_fullsized":"2023-11-26T07:17:03.625966+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:25:26.027867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559906+0000","last_change":"2023-11-26T07:17:02.571015+0000","last_active":"2023-11-26T07:17:03.559906+0000","last_peered":"2023-11-26T07:17:03.559906+0000","last_clean":"2023-11-26T07:17:03.559906+0000","last_became_active":"2023-11-26T07:17:02.570649+0000","last_became_peered":"2023-11-26T07:17:02.570649+0000","last_unstale":"2023-11-26T07:17:03.559906+0000","last_undegraded":"2023-11-26T07:17:03.559906+0000","last_fullsized":"2023-11-26T07:17:03.559906+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:39:21.005917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563656+0000","last_change":"2023-11-26T07:17:02.572715+0000","last_active":"2023-11-26T07:17:03.563656+0000","last_peered":"2023-11-26T07:17:03.563656+0000","last_clean":"2023-11-26T07:17:03.563656+0000","last_became_active":"2023-11-26T07:17:02.572035+0000","last_became_peered":"2023-11-26T07:17:02.572035+0000","last_unstale":"2023-11-26T07:17:03.563656+0000","last_undegraded":"2023-11-26T07:17:03.563656+0000","last_fullsized":"2023-11-26T07:17:03.563656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:28:54.491028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563646+0000","last_change":"2023-11-26T07:17:02.576459+0000","last_active":"2023-11-26T07:17:03.563646+0000","last_peered":"2023-11-26T07:17:03.563646+0000","last_clean":"2023-11-26T07:17:03.563646+0000","last_became_active":"2023-11-26T07:17:02.576185+0000","last_became_peered":"2023-11-26T07:17:02.576185+0000","last_unstale":"2023-11-26T07:17:03.563646+0000","last_undegraded":"2023-11-26T07:17:03.563646+0000","last_fullsized":"2023-11-26T07:17:03.563646+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:53:36.673023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625842+0000","last_change":"2023-11-26T07:17:02.573948+0000","last_active":"2023-11-26T07:17:03.625842+0000","last_peered":"2023-11-26T07:17:03.625842+0000","last_clean":"2023-11-26T07:17:03.625842+0000","last_became_active":"2023-11-26T07:17:02.572690+0000","last_became_peered":"2023-11-26T07:17:02.572690+0000","last_unstale":"2023-11-26T07:17:03.625842+0000","last_undegraded":"2023-11-26T07:17:03.625842+0000","last_fullsized":"2023-11-26T07:17:03.625842+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:16:18.265358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559655+0000","last_change":"2023-11-26T07:17:02.572635+0000","last_active":"2023-11-26T07:17:03.559655+0000","last_peered":"2023-11-26T07:17:03.559655+0000","last_clean":"2023-11-26T07:17:03.559655+0000","last_became_active":"2023-11-26T07:17:02.572325+0000","last_became_peered":"2023-11-26T07:17:02.572325+0000","last_unstale":"2023-11-26T07:17:03.559655+0000","last_undegraded":"2023-11-26T07:17:03.559655+0000","last_fullsized":"2023-11-26T07:17:03.559655+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:53:20.301655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626377+0000","last_change":"2023-11-26T07:17:02.575984+0000","last_active":"2023-11-26T07:17:03.626377+0000","last_peered":"2023-11-26T07:17:03.626377+0000","last_clean":"2023-11-26T07:17:03.626377+0000","last_became_active":"2023-11-26T07:17:02.575687+0000","last_became_peered":"2023-11-26T07:17:02.575687+0000","last_unstale":"2023-11-26T07:17:03.626377+0000","last_undegraded":"2023-11-26T07:17:03.626377+0000","last_fullsized":"2023-11-26T07:17:03.626377+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:53:36.673023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626125+0000","last_change":"2023-11-26T07:17:02.574826+0000","last_active":"2023-11-26T07:17:03.626125+0000","last_peered":"2023-11-26T07:17:03.626125+0000","last_clean":"2023-11-26T07:17:03.626125+0000","last_became_active":"2023-11-26T07:17:02.574534+0000","last_became_peered":"2023-11-26T07:17:02.574534+0000","last_unstale":"2023-11-26T07:17:03.626125+0000","last_undegraded":"2023-11-26T07:17:03.626125+0000","last_fullsized":"2023-11-26T07:17:03.626125+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:10.197851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563768+0000","last_change":"2023-11-26T07:17:02.568118+0000","last_active":"2023-11-26T07:17:03.563768+0000","last_peered":"2023-11-26T07:17:03.563768+0000","last_clean":"2023-11-26T07:17:03.563768+0000","last_became_active":"2023-11-26T07:17:02.567453+0000","last_became_peered":"2023-11-26T07:17:02.567453+0000","last_unstale":"2023-11-26T07:17:03.563768+0000","last_undegraded":"2023-11-26T07:17:03.563768+0000","last_fullsized":"2023-11-26T07:17:03.563768+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:15:11.809794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625888+0000","last_change":"2023-11-26T07:17:02.565875+0000","last_active":"2023-11-26T07:17:03.625888+0000","last_peered":"2023-11-26T07:17:03.625888+0000","last_clean":"2023-11-26T07:17:03.625888+0000","last_became_active":"2023-11-26T07:17:02.565604+0000","last_became_peered":"2023-11-26T07:17:02.565604+0000","last_unstale":"2023-11-26T07:17:03.625888+0000","last_undegraded":"2023-11-26T07:17:03.625888+0000","last_fullsized":"2023-11-26T07:17:03.625888+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:24.170651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625816+0000","last_change":"2023-11-26T07:17:02.565776+0000","last_active":"2023-11-26T07:17:03.625816+0000","last_peered":"2023-11-26T07:17:03.625816+0000","last_clean":"2023-11-26T07:17:03.625816+0000","last_became_active":"2023-11-26T07:17:02.565485+0000","last_became_peered":"2023-11-26T07:17:02.565485+0000","last_unstale":"2023-11-26T07:17:03.625816+0000","last_undegraded":"2023-11-26T07:17:03.625816+0000","last_fullsized":"2023-11-26T07:17:03.625816+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:10.197851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625794+0000","last_change":"2023-11-26T07:17:02.575638+0000","last_active":"2023-11-26T07:17:03.625794+0000","last_peered":"2023-11-26T07:17:03.625794+0000","last_clean":"2023-11-26T07:17:03.625794+0000","last_became_active":"2023-11-26T07:17:02.575164+0000","last_became_peered":"2023-11-26T07:17:02.575164+0000","last_unstale":"2023-11-26T07:17:03.625794+0000","last_undegraded":"2023-11-26T07:17:03.625794+0000","last_fullsized":"2023-11-26T07:17:03.625794+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:15:11.809794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625397+0000","last_change":"2023-11-26T07:17:02.565508+0000","last_active":"2023-11-26T07:17:03.625397+0000","last_peered":"2023-11-26T07:17:03.625397+0000","last_clean":"2023-11-26T07:17:03.625397+0000","last_became_active":"2023-11-26T07:17:02.565049+0000","last_became_peered":"2023-11-26T07:17:02.565049+0000","last_unstale":"2023-11-26T07:17:03.625397+0000","last_undegraded":"2023-11-26T07:17:03.625397+0000","last_fullsized":"2023-11-26T07:17:03.625397+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:04:23.067790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625968+0000","last_change":"2023-11-26T07:17:02.566025+0000","last_active":"2023-11-26T07:17:03.625968+0000","last_peered":"2023-11-26T07:17:03.625968+0000","last_clean":"2023-11-26T07:17:03.625968+0000","last_became_active":"2023-11-26T07:17:02.565634+0000","last_became_peered":"2023-11-26T07:17:02.565634+0000","last_unstale":"2023-11-26T07:17:03.625968+0000","last_undegraded":"2023-11-26T07:17:03.625968+0000","last_fullsized":"2023-11-26T07:17:03.625968+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:16.936993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626366+0000","last_change":"2023-11-26T07:17:02.575799+0000","last_active":"2023-11-26T07:17:03.626366+0000","last_peered":"2023-11-26T07:17:03.626366+0000","last_clean":"2023-11-26T07:17:03.626366+0000","last_became_active":"2023-11-26T07:17:02.575542+0000","last_became_peered":"2023-11-26T07:17:02.575542+0000","last_unstale":"2023-11-26T07:17:03.626366+0000","last_undegraded":"2023-11-26T07:17:03.626366+0000","last_fullsized":"2023-11-26T07:17:03.626366+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:17:22.322087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559910+0000","last_change":"2023-11-26T07:17:02.571878+0000","last_active":"2023-11-26T07:17:03.559910+0000","last_peered":"2023-11-26T07:17:03.559910+0000","last_clean":"2023-11-26T07:17:03.559910+0000","last_became_active":"2023-11-26T07:17:02.571586+0000","last_became_peered":"2023-11-26T07:17:02.571586+0000","last_unstale":"2023-11-26T07:17:03.559910+0000","last_undegraded":"2023-11-26T07:17:03.559910+0000","last_fullsized":"2023-11-26T07:17:03.559910+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:53.814028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626568+0000","last_change":"2023-11-26T07:17:02.572736+0000","last_active":"2023-11-26T07:17:03.626568+0000","last_peered":"2023-11-26T07:17:03.626568+0000","last_clean":"2023-11-26T07:17:03.626568+0000","last_became_active":"2023-11-26T07:17:02.572462+0000","last_became_peered":"2023-11-26T07:17:02.572462+0000","last_unstale":"2023-11-26T07:17:03.626568+0000","last_undegraded":"2023-11-26T07:17:03.626568+0000","last_fullsized":"2023-11-26T07:17:03.626568+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:43:51.163573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625997+0000","last_change":"2023-11-26T07:17:02.567421+0000","last_active":"2023-11-26T07:17:03.625997+0000","last_peered":"2023-11-26T07:17:03.625997+0000","last_clean":"2023-11-26T07:17:03.625997+0000","last_became_active":"2023-11-26T07:17:02.567137+0000","last_became_peered":"2023-11-26T07:17:02.567137+0000","last_unstale":"2023-11-26T07:17:03.625997+0000","last_undegraded":"2023-11-26T07:17:03.625997+0000","last_fullsized":"2023-11-26T07:17:03.625997+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:28:00.740325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625934+0000","last_change":"2023-11-26T07:17:02.572812+0000","last_active":"2023-11-26T07:17:03.625934+0000","last_peered":"2023-11-26T07:17:03.625934+0000","last_clean":"2023-11-26T07:17:03.625934+0000","last_became_active":"2023-11-26T07:17:02.572515+0000","last_became_peered":"2023-11-26T07:17:02.572515+0000","last_unstale":"2023-11-26T07:17:03.625934+0000","last_undegraded":"2023-11-26T07:17:03.625934+0000","last_fullsized":"2023-11-26T07:17:03.625934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:39.771470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.562647+0000","last_change":"2023-11-26T07:17:02.573494+0000","last_active":"2023-11-26T07:17:03.562647+0000","last_peered":"2023-11-26T07:17:03.562647+0000","last_clean":"2023-11-26T07:17:03.562647+0000","last_became_active":"2023-11-26T07:17:02.573221+0000","last_became_peered":"2023-11-26T07:17:02.573221+0000","last_unstale":"2023-11-26T07:17:03.562647+0000","last_undegraded":"2023-11-26T07:17:03.562647+0000","last_fullsized":"2023-11-26T07:17:03.562647+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:41.020210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559713+0000","last_change":"2023-11-26T07:17:02.572009+0000","last_active":"2023-11-26T07:17:03.559713+0000","last_peered":"2023-11-26T07:17:03.559713+0000","last_clean":"2023-11-26T07:17:03.559713+0000","last_became_active":"2023-11-26T07:17:02.571558+0000","last_became_peered":"2023-11-26T07:17:02.571558+0000","last_unstale":"2023-11-26T07:17:03.559713+0000","last_undegraded":"2023-11-26T07:17:03.559713+0000","last_fullsized":"2023-11-26T07:17:03.559713+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:35:52.938929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626053+0000","last_change":"2023-11-26T07:17:02.568891+0000","last_active":"2023-11-26T07:17:03.626053+0000","last_peered":"2023-11-26T07:17:03.626053+0000","last_clean":"2023-11-26T07:17:03.626053+0000","last_became_active":"2023-11-26T07:17:02.568619+0000","last_became_peered":"2023-11-26T07:17:02.568619+0000","last_unstale":"2023-11-26T07:17:03.626053+0000","last_undegraded":"2023-11-26T07:17:03.626053+0000","last_fullsized":"2023-11-26T07:17:03.626053+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:37:18.698945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626400+0000","last_change":"2023-11-26T07:17:02.567327+0000","last_active":"2023-11-26T07:17:03.626400+0000","last_peered":"2023-11-26T07:17:03.626400+0000","last_clean":"2023-11-26T07:17:03.626400+0000","last_became_active":"2023-11-26T07:17:02.567039+0000","last_became_peered":"2023-11-26T07:17:02.567039+0000","last_unstale":"2023-11-26T07:17:03.626400+0000","last_undegraded":"2023-11-26T07:17:03.626400+0000","last_fullsized":"2023-11-26T07:17:03.626400+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:32:21.174137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"101'4","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.594520+0000","last_change":"2023-11-26T07:17:01.584404+0000","last_active":"2023-11-26T07:17:02.594520+0000","last_peered":"2023-11-26T07:17:02.594520+0000","last_clean":"2023-11-26T07:17:02.594520+0000","last_became_active":"2023-11-26T07:17:01.584088+0000","last_became_peered":"2023-11-26T07:17:01.584088+0000","last_unstale":"2023-11-26T07:17:02.594520+0000","last_undegraded":"2023-11-26T07:17:02.594520+0000","last_fullsized":"2023-11-26T07:17:02.594520+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:04:11.625643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626425+0000","last_change":"2023-11-26T07:17:02.582406+0000","last_active":"2023-11-26T07:17:03.626425+0000","last_peered":"2023-11-26T07:17:03.626425+0000","last_clean":"2023-11-26T07:17:03.626425+0000","last_became_active":"2023-11-26T07:17:02.582095+0000","last_became_peered":"2023-11-26T07:17:02.582095+0000","last_unstale":"2023-11-26T07:17:03.626425+0000","last_undegraded":"2023-11-26T07:17:03.626425+0000","last_fullsized":"2023-11-26T07:17:03.626425+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:11:27.335118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.591688+0000","last_change":"2023-11-26T07:17:01.583711+0000","last_active":"2023-11-26T07:17:02.591688+0000","last_peered":"2023-11-26T07:17:02.591688+0000","last_clean":"2023-11-26T07:17:02.591688+0000","last_became_active":"2023-11-26T07:17:01.583168+0000","last_became_peered":"2023-11-26T07:17:01.583168+0000","last_unstale":"2023-11-26T07:17:02.591688+0000","last_undegraded":"2023-11-26T07:17:02.591688+0000","last_fullsized":"2023-11-26T07:17:02.591688+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:27.918900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625469+0000","last_change":"2023-11-26T07:17:02.569200+0000","last_active":"2023-11-26T07:17:03.625469+0000","last_peered":"2023-11-26T07:17:03.625469+0000","last_clean":"2023-11-26T07:17:03.625469+0000","last_became_active":"2023-11-26T07:17:02.568908+0000","last_became_peered":"2023-11-26T07:17:02.568908+0000","last_unstale":"2023-11-26T07:17:03.625469+0000","last_undegraded":"2023-11-26T07:17:03.625469+0000","last_fullsized":"2023-11-26T07:17:03.625469+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:06:13.267251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.592893+0000","last_change":"2023-11-26T07:17:01.588258+0000","last_active":"2023-11-26T07:17:02.592893+0000","last_peered":"2023-11-26T07:17:02.592893+0000","last_clean":"2023-11-26T07:17:02.592893+0000","last_became_active":"2023-11-26T07:17:01.587985+0000","last_became_peered":"2023-11-26T07:17:01.587985+0000","last_unstale":"2023-11-26T07:17:02.592893+0000","last_undegraded":"2023-11-26T07:17:02.592893+0000","last_fullsized":"2023-11-26T07:17:02.592893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:53:30.894666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563844+0000","last_change":"2023-11-26T07:17:02.570781+0000","last_active":"2023-11-26T07:17:03.563844+0000","last_peered":"2023-11-26T07:17:03.563844+0000","last_clean":"2023-11-26T07:17:03.563844+0000","last_became_active":"2023-11-26T07:17:02.570531+0000","last_became_peered":"2023-11-26T07:17:02.570531+0000","last_unstale":"2023-11-26T07:17:03.563844+0000","last_undegraded":"2023-11-26T07:17:03.563844+0000","last_fullsized":"2023-11-26T07:17:03.563844+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:50.422694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.593190+0000","last_change":"2023-11-26T07:17:01.590141+0000","last_active":"2023-11-26T07:17:02.593190+0000","last_peered":"2023-11-26T07:17:02.593190+0000","last_clean":"2023-11-26T07:17:02.593190+0000","last_became_active":"2023-11-26T07:17:01.589899+0000","last_became_peered":"2023-11-26T07:17:01.589899+0000","last_unstale":"2023-11-26T07:17:02.593190+0000","last_undegraded":"2023-11-26T07:17:02.593190+0000","last_fullsized":"2023-11-26T07:17:02.593190+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:04:02.800106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559803+0000","last_change":"2023-11-26T07:17:02.573870+0000","last_active":"2023-11-26T07:17:03.559803+0000","last_peered":"2023-11-26T07:17:03.559803+0000","last_clean":"2023-11-26T07:17:03.559803+0000","last_became_active":"2023-11-26T07:17:02.573620+0000","last_became_peered":"2023-11-26T07:17:02.573620+0000","last_unstale":"2023-11-26T07:17:03.559803+0000","last_undegraded":"2023-11-26T07:17:03.559803+0000","last_fullsized":"2023-11-26T07:17:03.559803+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:37:19.895847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.592173+0000","last_change":"2023-11-26T07:17:01.588225+0000","last_active":"2023-11-26T07:17:02.592173+0000","last_peered":"2023-11-26T07:17:02.592173+0000","last_clean":"2023-11-26T07:17:02.592173+0000","last_became_active":"2023-11-26T07:17:01.587905+0000","last_became_peered":"2023-11-26T07:17:01.587905+0000","last_unstale":"2023-11-26T07:17:02.592173+0000","last_undegraded":"2023-11-26T07:17:02.592173+0000","last_fullsized":"2023-11-26T07:17:02.592173+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:28:36.298533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625982+0000","last_change":"2023-11-26T07:17:02.567623+0000","last_active":"2023-11-26T07:17:03.625982+0000","last_peered":"2023-11-26T07:17:03.625982+0000","last_clean":"2023-11-26T07:17:03.625982+0000","last_became_active":"2023-11-26T07:17:02.567227+0000","last_became_peered":"2023-11-26T07:17:02.567227+0000","last_unstale":"2023-11-26T07:17:03.625982+0000","last_undegraded":"2023-11-26T07:17:03.625982+0000","last_fullsized":"2023-11-26T07:17:03.625982+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:59.383908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.590568+0000","last_change":"2023-11-26T07:17:01.585551+0000","last_active":"2023-11-26T07:17:02.590568+0000","last_peered":"2023-11-26T07:17:02.590568+0000","last_clean":"2023-11-26T07:17:02.590568+0000","last_became_active":"2023-11-26T07:17:01.585270+0000","last_became_peered":"2023-11-26T07:17:01.585270+0000","last_unstale":"2023-11-26T07:17:02.590568+0000","last_undegraded":"2023-11-26T07:17:02.590568+0000","last_fullsized":"2023-11-26T07:17:02.590568+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:51:09.192327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559742+0000","last_change":"2023-11-26T07:17:02.571080+0000","last_active":"2023-11-26T07:17:03.559742+0000","last_peered":"2023-11-26T07:17:03.559742+0000","last_clean":"2023-11-26T07:17:03.559742+0000","last_became_active":"2023-11-26T07:17:02.570791+0000","last_became_peered":"2023-11-26T07:17:02.570791+0000","last_unstale":"2023-11-26T07:17:03.559742+0000","last_undegraded":"2023-11-26T07:17:03.559742+0000","last_fullsized":"2023-11-26T07:17:03.559742+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:58.389755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.558565+0000","last_change":"2023-11-26T07:17:01.579199+0000","last_active":"2023-11-26T07:17:02.558565+0000","last_peered":"2023-11-26T07:17:02.558565+0000","last_clean":"2023-11-26T07:17:02.558565+0000","last_became_active":"2023-11-26T07:17:01.578831+0000","last_became_peered":"2023-11-26T07:17:01.578831+0000","last_unstale":"2023-11-26T07:17:02.558565+0000","last_undegraded":"2023-11-26T07:17:02.558565+0000","last_fullsized":"2023-11-26T07:17:02.558565+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:27.918900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.562470+0000","last_change":"2023-11-26T07:17:02.576186+0000","last_active":"2023-11-26T07:17:03.562470+0000","last_peered":"2023-11-26T07:17:03.562470+0000","last_clean":"2023-11-26T07:17:03.562470+0000","last_became_active":"2023-11-26T07:17:02.575861+0000","last_became_peered":"2023-11-26T07:17:02.575861+0000","last_unstale":"2023-11-26T07:17:03.562470+0000","last_undegraded":"2023-11-26T07:17:03.562470+0000","last_fullsized":"2023-11-26T07:17:03.562470+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:08:10.681717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.561540+0000","last_change":"2023-11-26T07:17:01.580094+0000","last_active":"2023-11-26T07:17:02.561540+0000","last_peered":"2023-11-26T07:17:02.561540+0000","last_clean":"2023-11-26T07:17:02.561540+0000","last_became_active":"2023-11-26T07:17:01.579827+0000","last_became_peered":"2023-11-26T07:17:01.579827+0000","last_unstale":"2023-11-26T07:17:02.561540+0000","last_undegraded":"2023-11-26T07:17:02.561540+0000","last_fullsized":"2023-11-26T07:17:02.561540+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:49.417170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.562830+0000","last_change":"2023-11-26T07:17:02.569496+0000","last_active":"2023-11-26T07:17:03.562830+0000","last_peered":"2023-11-26T07:17:03.562830+0000","last_clean":"2023-11-26T07:17:03.562830+0000","last_became_active":"2023-11-26T07:17:02.569221+0000","last_became_peered":"2023-11-26T07:17:02.569221+0000","last_unstale":"2023-11-26T07:17:03.562830+0000","last_undegraded":"2023-11-26T07:17:03.562830+0000","last_fullsized":"2023-11-26T07:17:03.562830+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:47.671595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.559321+0000","last_change":"2023-11-26T07:17:01.588205+0000","last_active":"2023-11-26T07:17:02.559321+0000","last_peered":"2023-11-26T07:17:02.559321+0000","last_clean":"2023-11-26T07:17:02.559321+0000","last_became_active":"2023-11-26T07:17:01.587870+0000","last_became_peered":"2023-11-26T07:17:01.587870+0000","last_unstale":"2023-11-26T07:17:02.559321+0000","last_undegraded":"2023-11-26T07:17:02.559321+0000","last_fullsized":"2023-11-26T07:17:02.559321+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:50:06.014213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626417+0000","last_change":"2023-11-26T07:17:02.574834+0000","last_active":"2023-11-26T07:17:03.626417+0000","last_peered":"2023-11-26T07:17:03.626417+0000","last_clean":"2023-11-26T07:17:03.626417+0000","last_became_active":"2023-11-26T07:17:02.574559+0000","last_became_peered":"2023-11-26T07:17:02.574559+0000","last_unstale":"2023-11-26T07:17:03.626417+0000","last_undegraded":"2023-11-26T07:17:03.626417+0000","last_fullsized":"2023-11-26T07:17:03.626417+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:39.771470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.556759+0000","last_change":"2023-11-26T07:17:01.584174+0000","last_active":"2023-11-26T07:17:02.556759+0000","last_peered":"2023-11-26T07:17:02.556759+0000","last_clean":"2023-11-26T07:17:02.556759+0000","last_became_active":"2023-11-26T07:17:01.583882+0000","last_became_peered":"2023-11-26T07:17:01.583882+0000","last_unstale":"2023-11-26T07:17:02.556759+0000","last_undegraded":"2023-11-26T07:17:02.556759+0000","last_fullsized":"2023-11-26T07:17:02.556759+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:56:19.009627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625207+0000","last_change":"2023-11-26T07:17:02.568393+0000","last_active":"2023-11-26T07:17:03.625207+0000","last_peered":"2023-11-26T07:17:03.625207+0000","last_clean":"2023-11-26T07:17:03.625207+0000","last_became_active":"2023-11-26T07:17:02.568127+0000","last_became_peered":"2023-11-26T07:17:02.568127+0000","last_unstale":"2023-11-26T07:17:03.625207+0000","last_undegraded":"2023-11-26T07:17:03.625207+0000","last_fullsized":"2023-11-26T07:17:03.625207+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:24.997966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"101'3","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.595024+0000","last_change":"2023-11-26T07:17:01.587817+0000","last_active":"2023-11-26T07:17:02.595024+0000","last_peered":"2023-11-26T07:17:02.595024+0000","last_clean":"2023-11-26T07:17:02.595024+0000","last_became_active":"2023-11-26T07:17:01.587506+0000","last_became_peered":"2023-11-26T07:17:01.587506+0000","last_unstale":"2023-11-26T07:17:02.595024+0000","last_undegraded":"2023-11-26T07:17:02.595024+0000","last_fullsized":"2023-11-26T07:17:02.595024+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:17:03.354854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.562571+0000","last_change":"2023-11-26T07:17:02.569186+0000","last_active":"2023-11-26T07:17:03.562571+0000","last_peered":"2023-11-26T07:17:03.562571+0000","last_clean":"2023-11-26T07:17:03.562571+0000","last_became_active":"2023-11-26T07:17:02.568872+0000","last_became_peered":"2023-11-26T07:17:02.568872+0000","last_unstale":"2023-11-26T07:17:03.562571+0000","last_undegraded":"2023-11-26T07:17:03.562571+0000","last_fullsized":"2023-11-26T07:17:03.562571+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:50.939570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"101'6","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.597881+0000","last_change":"2023-11-26T07:17:01.583421+0000","last_active":"2023-11-26T07:17:02.597881+0000","last_peered":"2023-11-26T07:17:02.597881+0000","last_clean":"2023-11-26T07:17:02.597881+0000","last_became_active":"2023-11-26T07:17:01.582917+0000","last_became_peered":"2023-11-26T07:17:01.582917+0000","last_unstale":"2023-11-26T07:17:02.597881+0000","last_undegraded":"2023-11-26T07:17:02.597881+0000","last_fullsized":"2023-11-26T07:17:02.597881+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:11.911032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625919+0000","last_change":"2023-11-26T07:17:02.567391+0000","last_active":"2023-11-26T07:17:03.625919+0000","last_peered":"2023-11-26T07:17:03.625919+0000","last_clean":"2023-11-26T07:17:03.625919+0000","last_became_active":"2023-11-26T07:17:02.567100+0000","last_became_peered":"2023-11-26T07:17:02.567100+0000","last_unstale":"2023-11-26T07:17:03.625919+0000","last_undegraded":"2023-11-26T07:17:03.625919+0000","last_fullsized":"2023-11-26T07:17:03.625919+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:50.422694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.594586+0000","last_change":"2023-11-26T07:17:01.584515+0000","last_active":"2023-11-26T07:17:02.594586+0000","last_peered":"2023-11-26T07:17:02.594586+0000","last_clean":"2023-11-26T07:17:02.594586+0000","last_became_active":"2023-11-26T07:17:01.584221+0000","last_became_peered":"2023-11-26T07:17:01.584221+0000","last_unstale":"2023-11-26T07:17:02.594586+0000","last_undegraded":"2023-11-26T07:17:02.594586+0000","last_fullsized":"2023-11-26T07:17:02.594586+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:35:29.667246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625574+0000","last_change":"2023-11-26T07:17:02.566731+0000","last_active":"2023-11-26T07:17:03.625574+0000","last_peered":"2023-11-26T07:17:03.625574+0000","last_clean":"2023-11-26T07:17:03.625574+0000","last_became_active":"2023-11-26T07:17:02.566432+0000","last_became_peered":"2023-11-26T07:17:02.566432+0000","last_unstale":"2023-11-26T07:17:03.625574+0000","last_undegraded":"2023-11-26T07:17:03.625574+0000","last_fullsized":"2023-11-26T07:17:03.625574+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:05:05.947486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.571342+0000","last_change":"2023-11-26T07:17:01.586046+0000","last_active":"2023-11-26T07:17:02.571342+0000","last_peered":"2023-11-26T07:17:02.571342+0000","last_clean":"2023-11-26T07:17:02.571342+0000","last_became_active":"2023-11-26T07:17:01.585739+0000","last_became_peered":"2023-11-26T07:17:01.585739+0000","last_unstale":"2023-11-26T07:17:02.571342+0000","last_undegraded":"2023-11-26T07:17:02.571342+0000","last_fullsized":"2023-11-26T07:17:02.571342+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:36.667119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625216+0000","last_change":"2023-11-26T07:17:02.568173+0000","last_active":"2023-11-26T07:17:03.625216+0000","last_peered":"2023-11-26T07:17:03.625216+0000","last_clean":"2023-11-26T07:17:03.625216+0000","last_became_active":"2023-11-26T07:17:02.567887+0000","last_became_peered":"2023-11-26T07:17:02.567887+0000","last_unstale":"2023-11-26T07:17:03.625216+0000","last_undegraded":"2023-11-26T07:17:03.625216+0000","last_fullsized":"2023-11-26T07:17:03.625216+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:17:22.322087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"101'8","reported_seq":20,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.595220+0000","last_change":"2023-11-26T07:17:01.578474+0000","last_active":"2023-11-26T07:17:02.595220+0000","last_peered":"2023-11-26T07:17:02.595220+0000","last_clean":"2023-11-26T07:17:02.595220+0000","last_became_active":"2023-11-26T07:17:01.578190+0000","last_became_peered":"2023-11-26T07:17:01.578190+0000","last_unstale":"2023-11-26T07:17:02.595220+0000","last_undegraded":"2023-11-26T07:17:02.595220+0000","last_fullsized":"2023-11-26T07:17:02.595220+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:12:55.738508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625951+0000","last_change":"2023-11-26T07:17:02.573796+0000","last_active":"2023-11-26T07:17:03.625951+0000","last_peered":"2023-11-26T07:17:03.625951+0000","last_clean":"2023-11-26T07:17:03.625951+0000","last_became_active":"2023-11-26T07:17:02.573389+0000","last_became_peered":"2023-11-26T07:17:02.573389+0000","last_unstale":"2023-11-26T07:17:03.625951+0000","last_undegraded":"2023-11-26T07:17:03.625951+0000","last_fullsized":"2023-11-26T07:17:03.625951+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:50:51.562727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.594818+0000","last_change":"2023-11-26T07:17:01.582348+0000","last_active":"2023-11-26T07:17:02.594818+0000","last_peered":"2023-11-26T07:17:02.594818+0000","last_clean":"2023-11-26T07:17:02.594818+0000","last_became_active":"2023-11-26T07:17:01.582055+0000","last_became_peered":"2023-11-26T07:17:01.582055+0000","last_unstale":"2023-11-26T07:17:02.594818+0000","last_undegraded":"2023-11-26T07:17:02.594818+0000","last_fullsized":"2023-11-26T07:17:02.594818+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:26:01.288923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626940+0000","last_change":"2023-11-26T07:17:02.571365+0000","last_active":"2023-11-26T07:17:03.626940+0000","last_peered":"2023-11-26T07:17:03.626940+0000","last_clean":"2023-11-26T07:17:03.626940+0000","last_became_active":"2023-11-26T07:17:02.571091+0000","last_became_peered":"2023-11-26T07:17:02.571091+0000","last_unstale":"2023-11-26T07:17:03.626940+0000","last_undegraded":"2023-11-26T07:17:03.626940+0000","last_fullsized":"2023-11-26T07:17:03.626940+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:08:00.531777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.592985+0000","last_change":"2023-11-26T07:17:01.580502+0000","last_active":"2023-11-26T07:17:02.592985+0000","last_peered":"2023-11-26T07:17:02.592985+0000","last_clean":"2023-11-26T07:17:02.592985+0000","last_became_active":"2023-11-26T07:17:01.580205+0000","last_became_peered":"2023-11-26T07:17:01.580205+0000","last_unstale":"2023-11-26T07:17:02.592985+0000","last_undegraded":"2023-11-26T07:17:02.592985+0000","last_fullsized":"2023-11-26T07:17:02.592985+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:57:10.571315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563545+0000","last_change":"2023-11-26T07:17:02.574330+0000","last_active":"2023-11-26T07:17:03.563545+0000","last_peered":"2023-11-26T07:17:03.563545+0000","last_clean":"2023-11-26T07:17:03.563545+0000","last_became_active":"2023-11-26T07:17:02.574046+0000","last_became_peered":"2023-11-26T07:17:02.574046+0000","last_unstale":"2023-11-26T07:17:03.563545+0000","last_undegraded":"2023-11-26T07:17:03.563545+0000","last_fullsized":"2023-11-26T07:17:03.563545+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:17:22.322087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.593922+0000","last_change":"2023-11-26T07:17:01.585237+0000","last_active":"2023-11-26T07:17:02.593922+0000","last_peered":"2023-11-26T07:17:02.593922+0000","last_clean":"2023-11-26T07:17:02.593922+0000","last_became_active":"2023-11-26T07:17:01.584906+0000","last_became_peered":"2023-11-26T07:17:01.584906+0000","last_unstale":"2023-11-26T07:17:02.593922+0000","last_undegraded":"2023-11-26T07:17:02.593922+0000","last_fullsized":"2023-11-26T07:17:02.593922+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:57:11.459533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563902+0000","last_change":"2023-11-26T07:17:02.568317+0000","last_active":"2023-11-26T07:17:03.563902+0000","last_peered":"2023-11-26T07:17:03.563902+0000","last_clean":"2023-11-26T07:17:03.563902+0000","last_became_active":"2023-11-26T07:17:02.567770+0000","last_became_peered":"2023-11-26T07:17:02.567770+0000","last_unstale":"2023-11-26T07:17:03.563902+0000","last_undegraded":"2023-11-26T07:17:03.563902+0000","last_fullsized":"2023-11-26T07:17:03.563902+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:13:00.281782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.562531+0000","last_change":"2023-11-26T07:17:01.586521+0000","last_active":"2023-11-26T07:17:02.562531+0000","last_peered":"2023-11-26T07:17:02.562531+0000","last_clean":"2023-11-26T07:17:02.562531+0000","last_became_active":"2023-11-26T07:17:01.586279+0000","last_became_peered":"2023-11-26T07:17:01.586279+0000","last_unstale":"2023-11-26T07:17:02.562531+0000","last_undegraded":"2023-11-26T07:17:02.562531+0000","last_fullsized":"2023-11-26T07:17:02.562531+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:28:32.725095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563532+0000","last_change":"2023-11-26T07:17:02.569825+0000","last_active":"2023-11-26T07:17:03.563532+0000","last_peered":"2023-11-26T07:17:03.563532+0000","last_clean":"2023-11-26T07:17:03.563532+0000","last_became_active":"2023-11-26T07:17:02.569514+0000","last_became_peered":"2023-11-26T07:17:02.569514+0000","last_unstale":"2023-11-26T07:17:03.563532+0000","last_undegraded":"2023-11-26T07:17:03.563532+0000","last_fullsized":"2023-11-26T07:17:03.563532+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:24.997966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.592762+0000","last_change":"2023-11-26T07:17:01.583363+0000","last_active":"2023-11-26T07:17:02.592762+0000","last_peered":"2023-11-26T07:17:02.592762+0000","last_clean":"2023-11-26T07:17:02.592762+0000","last_became_active":"2023-11-26T07:17:01.583097+0000","last_became_peered":"2023-11-26T07:17:01.583097+0000","last_unstale":"2023-11-26T07:17:02.592762+0000","last_undegraded":"2023-11-26T07:17:02.592762+0000","last_fullsized":"2023-11-26T07:17:02.592762+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:53:30.894666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559588+0000","last_change":"2023-11-26T07:17:02.573267+0000","last_active":"2023-11-26T07:17:03.559588+0000","last_peered":"2023-11-26T07:17:03.559588+0000","last_clean":"2023-11-26T07:17:03.559588+0000","last_became_active":"2023-11-26T07:17:02.572996+0000","last_became_peered":"2023-11-26T07:17:02.572996+0000","last_unstale":"2023-11-26T07:17:03.559588+0000","last_undegraded":"2023-11-26T07:17:03.559588+0000","last_fullsized":"2023-11-26T07:17:03.559588+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:25:53.916291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.559713+0000","last_change":"2023-11-26T07:17:01.588512+0000","last_active":"2023-11-26T07:17:02.559713+0000","last_peered":"2023-11-26T07:17:02.559713+0000","last_clean":"2023-11-26T07:17:02.559713+0000","last_became_active":"2023-11-26T07:17:01.588116+0000","last_became_peered":"2023-11-26T07:17:01.588116+0000","last_unstale":"2023-11-26T07:17:02.559713+0000","last_undegraded":"2023-11-26T07:17:02.559713+0000","last_fullsized":"2023-11-26T07:17:02.559713+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:16:36.858554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563397+0000","last_change":"2023-11-26T07:17:02.567641+0000","last_active":"2023-11-26T07:17:03.563397+0000","last_peered":"2023-11-26T07:17:03.563397+0000","last_clean":"2023-11-26T07:17:03.563397+0000","last_became_active":"2023-11-26T07:17:02.567326+0000","last_became_peered":"2023-11-26T07:17:02.567326+0000","last_unstale":"2023-11-26T07:17:03.563397+0000","last_undegraded":"2023-11-26T07:17:03.563397+0000","last_fullsized":"2023-11-26T07:17:03.563397+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:59.383908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.563397+0000","last_change":"2023-11-26T07:17:01.588360+0000","last_active":"2023-11-26T07:17:02.563397+0000","last_peered":"2023-11-26T07:17:02.563397+0000","last_clean":"2023-11-26T07:17:02.563397+0000","last_became_active":"2023-11-26T07:17:01.587441+0000","last_became_peered":"2023-11-26T07:17:01.587441+0000","last_unstale":"2023-11-26T07:17:02.563397+0000","last_undegraded":"2023-11-26T07:17:02.563397+0000","last_fullsized":"2023-11-26T07:17:02.563397+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:09:45.951301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625268+0000","last_change":"2023-11-26T07:17:02.568858+0000","last_active":"2023-11-26T07:17:03.625268+0000","last_peered":"2023-11-26T07:17:03.625268+0000","last_clean":"2023-11-26T07:17:03.625268+0000","last_became_active":"2023-11-26T07:17:02.568340+0000","last_became_peered":"2023-11-26T07:17:02.568340+0000","last_unstale":"2023-11-26T07:17:03.625268+0000","last_undegraded":"2023-11-26T07:17:03.625268+0000","last_fullsized":"2023-11-26T07:17:03.625268+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:53:36.673023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.561685+0000","last_change":"2023-11-26T07:17:01.584757+0000","last_active":"2023-11-26T07:17:02.561685+0000","last_peered":"2023-11-26T07:17:02.561685+0000","last_clean":"2023-11-26T07:17:02.561685+0000","last_became_active":"2023-11-26T07:17:01.582979+0000","last_became_peered":"2023-11-26T07:17:01.582979+0000","last_unstale":"2023-11-26T07:17:02.561685+0000","last_undegraded":"2023-11-26T07:17:02.561685+0000","last_fullsized":"2023-11-26T07:17:02.561685+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:18:51.644212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626263+0000","last_change":"2023-11-26T07:17:02.573945+0000","last_active":"2023-11-26T07:17:03.626263+0000","last_peered":"2023-11-26T07:17:03.626263+0000","last_clean":"2023-11-26T07:17:03.626263+0000","last_became_active":"2023-11-26T07:17:02.573683+0000","last_became_peered":"2023-11-26T07:17:02.573683+0000","last_unstale":"2023-11-26T07:17:03.626263+0000","last_undegraded":"2023-11-26T07:17:03.626263+0000","last_fullsized":"2023-11-26T07:17:03.626263+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:24.170651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.594573+0000","last_change":"2023-11-26T07:17:01.584426+0000","last_active":"2023-11-26T07:17:02.594573+0000","last_peered":"2023-11-26T07:17:02.594573+0000","last_clean":"2023-11-26T07:17:02.594573+0000","last_became_active":"2023-11-26T07:17:01.584060+0000","last_became_peered":"2023-11-26T07:17:01.584060+0000","last_unstale":"2023-11-26T07:17:02.594573+0000","last_undegraded":"2023-11-26T07:17:02.594573+0000","last_fullsized":"2023-11-26T07:17:02.594573+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:34:27.918900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.560316+0000","last_change":"2023-11-26T07:17:02.570101+0000","last_active":"2023-11-26T07:17:03.560316+0000","last_peered":"2023-11-26T07:17:03.560316+0000","last_clean":"2023-11-26T07:17:03.560316+0000","last_became_active":"2023-11-26T07:17:02.569816+0000","last_became_peered":"2023-11-26T07:17:02.569816+0000","last_unstale":"2023-11-26T07:17:03.560316+0000","last_undegraded":"2023-11-26T07:17:03.560316+0000","last_fullsized":"2023-11-26T07:17:03.560316+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:11:18.280217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.571636+0000","last_change":"2023-11-26T07:17:01.582420+0000","last_active":"2023-11-26T07:17:02.571636+0000","last_peered":"2023-11-26T07:17:02.571636+0000","last_clean":"2023-11-26T07:17:02.571636+0000","last_became_active":"2023-11-26T07:17:01.582134+0000","last_became_peered":"2023-11-26T07:17:01.582134+0000","last_unstale":"2023-11-26T07:17:02.571636+0000","last_undegraded":"2023-11-26T07:17:02.571636+0000","last_fullsized":"2023-11-26T07:17:02.571636+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:56:19.009627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559983+0000","last_change":"2023-11-26T07:17:02.570866+0000","last_active":"2023-11-26T07:17:03.559983+0000","last_peered":"2023-11-26T07:17:03.559983+0000","last_clean":"2023-11-26T07:17:03.559983+0000","last_became_active":"2023-11-26T07:17:02.570277+0000","last_became_peered":"2023-11-26T07:17:02.570277+0000","last_unstale":"2023-11-26T07:17:03.559983+0000","last_undegraded":"2023-11-26T07:17:03.559983+0000","last_fullsized":"2023-11-26T07:17:03.559983+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:23:58.105639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.562069+0000","last_change":"2023-11-26T07:17:01.583471+0000","last_active":"2023-11-26T07:17:02.562069+0000","last_peered":"2023-11-26T07:17:02.562069+0000","last_clean":"2023-11-26T07:17:02.562069+0000","last_became_active":"2023-11-26T07:17:01.583032+0000","last_became_peered":"2023-11-26T07:17:01.583032+0000","last_unstale":"2023-11-26T07:17:02.562069+0000","last_undegraded":"2023-11-26T07:17:02.562069+0000","last_fullsized":"2023-11-26T07:17:02.562069+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:53:59.068613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625703+0000","last_change":"2023-11-26T07:17:02.574936+0000","last_active":"2023-11-26T07:17:03.625703+0000","last_peered":"2023-11-26T07:17:03.625703+0000","last_clean":"2023-11-26T07:17:03.625703+0000","last_became_active":"2023-11-26T07:17:02.574568+0000","last_became_peered":"2023-11-26T07:17:02.574568+0000","last_unstale":"2023-11-26T07:17:03.625703+0000","last_undegraded":"2023-11-26T07:17:03.625703+0000","last_fullsized":"2023-11-26T07:17:03.625703+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:04:23.067790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.557196+0000","last_change":"2023-11-26T07:17:01.590145+0000","last_active":"2023-11-26T07:17:02.557196+0000","last_peered":"2023-11-26T07:17:02.557196+0000","last_clean":"2023-11-26T07:17:02.557196+0000","last_became_active":"2023-11-26T07:17:01.589827+0000","last_became_peered":"2023-11-26T07:17:01.589827+0000","last_unstale":"2023-11-26T07:17:02.557196+0000","last_undegraded":"2023-11-26T07:17:02.557196+0000","last_fullsized":"2023-11-26T07:17:02.557196+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:38.765945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.625984+0000","last_change":"2023-11-26T07:17:02.566604+0000","last_active":"2023-11-26T07:17:03.625984+0000","last_peered":"2023-11-26T07:17:03.625984+0000","last_clean":"2023-11-26T07:17:03.625984+0000","last_became_active":"2023-11-26T07:17:02.566197+0000","last_became_peered":"2023-11-26T07:17:02.566197+0000","last_unstale":"2023-11-26T07:17:03.625984+0000","last_undegraded":"2023-11-26T07:17:03.625984+0000","last_fullsized":"2023-11-26T07:17:03.625984+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:16.936993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.562707+0000","last_change":"2023-11-26T07:17:01.584165+0000","last_active":"2023-11-26T07:17:02.562707+0000","last_peered":"2023-11-26T07:17:02.562707+0000","last_clean":"2023-11-26T07:17:02.562707+0000","last_became_active":"2023-11-26T07:17:01.583874+0000","last_became_peered":"2023-11-26T07:17:01.583874+0000","last_unstale":"2023-11-26T07:17:02.562707+0000","last_undegraded":"2023-11-26T07:17:02.562707+0000","last_fullsized":"2023-11-26T07:17:02.562707+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:29.830617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626111+0000","last_change":"2023-11-26T07:17:02.566616+0000","last_active":"2023-11-26T07:17:03.626111+0000","last_peered":"2023-11-26T07:17:03.626111+0000","last_clean":"2023-11-26T07:17:03.626111+0000","last_became_active":"2023-11-26T07:17:02.565131+0000","last_became_peered":"2023-11-26T07:17:02.565131+0000","last_unstale":"2023-11-26T07:17:03.626111+0000","last_undegraded":"2023-11-26T07:17:03.626111+0000","last_fullsized":"2023-11-26T07:17:03.626111+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:24.997966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.558113+0000","last_change":"2023-11-26T07:17:01.588026+0000","last_active":"2023-11-26T07:17:02.558113+0000","last_peered":"2023-11-26T07:17:02.558113+0000","last_clean":"2023-11-26T07:17:02.558113+0000","last_became_active":"2023-11-26T07:17:01.587682+0000","last_became_peered":"2023-11-26T07:17:01.587682+0000","last_unstale":"2023-11-26T07:17:02.558113+0000","last_undegraded":"2023-11-26T07:17:02.558113+0000","last_fullsized":"2023-11-26T07:17:02.558113+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:37:46.666070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.563604+0000","last_change":"2023-11-26T07:17:02.576408+0000","last_active":"2023-11-26T07:17:03.563604+0000","last_peered":"2023-11-26T07:17:03.563604+0000","last_clean":"2023-11-26T07:17:03.563604+0000","last_became_active":"2023-11-26T07:17:02.576118+0000","last_became_peered":"2023-11-26T07:17:02.576118+0000","last_unstale":"2023-11-26T07:17:03.563604+0000","last_undegraded":"2023-11-26T07:17:03.563604+0000","last_fullsized":"2023-11-26T07:17:03.563604+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:28:00.740325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.559746+0000","last_change":"2023-11-26T07:17:01.588719+0000","last_active":"2023-11-26T07:17:02.559746+0000","last_peered":"2023-11-26T07:17:02.559746+0000","last_clean":"2023-11-26T07:17:02.559746+0000","last_became_active":"2023-11-26T07:17:01.587960+0000","last_became_peered":"2023-11-26T07:17:01.587960+0000","last_unstale":"2023-11-26T07:17:02.559746+0000","last_undegraded":"2023-11-26T07:17:02.559746+0000","last_fullsized":"2023-11-26T07:17:02.559746+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:35:29.667246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.559518+0000","last_change":"2023-11-26T07:17:02.570394+0000","last_active":"2023-11-26T07:17:03.559518+0000","last_peered":"2023-11-26T07:17:03.559518+0000","last_clean":"2023-11-26T07:17:03.559518+0000","last_became_active":"2023-11-26T07:17:02.570105+0000","last_became_peered":"2023-11-26T07:17:02.570105+0000","last_unstale":"2023-11-26T07:17:03.559518+0000","last_undegraded":"2023-11-26T07:17:03.559518+0000","last_fullsized":"2023-11-26T07:17:03.559518+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:22.278810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.562198+0000","last_change":"2023-11-26T07:17:01.585608+0000","last_active":"2023-11-26T07:17:02.562198+0000","last_peered":"2023-11-26T07:17:02.562198+0000","last_clean":"2023-11-26T07:17:02.562198+0000","last_became_active":"2023-11-26T07:17:01.583084+0000","last_became_peered":"2023-11-26T07:17:01.583084+0000","last_unstale":"2023-11-26T07:17:02.562198+0000","last_undegraded":"2023-11-26T07:17:02.562198+0000","last_fullsized":"2023-11-26T07:17:02.562198+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:08:12.474873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:03.626046+0000","last_change":"2023-11-26T07:17:02.570586+0000","last_active":"2023-11-26T07:17:03.626046+0000","last_peered":"2023-11-26T07:17:03.626046+0000","last_clean":"2023-11-26T07:17:03.626046+0000","last_became_active":"2023-11-26T07:17:02.570242+0000","last_became_peered":"2023-11-26T07:17:02.570242+0000","last_unstale":"2023-11-26T07:17:03.626046+0000","last_undegraded":"2023-11-26T07:17:03.626046+0000","last_fullsized":"2023-11-26T07:17:03.626046+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:01.561678+0000","last_clean_scrub_stamp":"2023-11-26T07:17:01.561678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:28:54.491028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.564818+0000","last_change":"2023-11-26T07:17:01.585782+0000","last_active":"2023-11-26T07:17:02.564818+0000","last_peered":"2023-11-26T07:17:02.564818+0000","last_clean":"2023-11-26T07:17:02.564818+0000","last_became_active":"2023-11-26T07:17:01.585381+0000","last_became_peered":"2023-11-26T07:17:01.585381+0000","last_unstale":"2023-11-26T07:17:02.564818+0000","last_undegraded":"2023-11-26T07:17:02.564818+0000","last_fullsized":"2023-11-26T07:17:02.564818+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:00.556154+0000","last_clean_scrub_stamp":"2023-11-26T07:17:00.556154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:58:00.348529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":287,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:17:02.560818+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:17:02.560818+0000","last_peered":"2023-11-26T07:17:02.560818+0000","last_clean":"2023-11-26T07:17:02.560818+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:17:02.560818+0000","last_undegraded":"2023-11-26T07:17:02.560818+0000","last_fullsized":"2023-11-26T07:17:02.560818+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1871872,"data_stored":1147353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.56200000000000006},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.36899999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.74299999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58699999999999997},"min":{"1min":0.40799999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.88100000000000001,"5min":1.048,"15min":1.048},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.433,"5min":0.315,"15min":0.315},"max":{"1min":0.71599999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.436}]},{"osd":2,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82499999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.622,"15min":0.622},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.79700000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.40699999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.46400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.876,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.45900000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.49199999999999999},"min":{"1min":0.34200000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.45500000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.64100000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":1.171,"15min":1.171},"last":0.59399999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.63800000000000001},"min":{"1min":0.39500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.93899999999999995,"5min":1.361,"15min":1.361},"last":0.629},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0409999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.68300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28272,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343568,"statfs":{"total":96636764160,"available":96607813632,"internally_reserved":0,"allocated":1273856,"data_stored":556439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.747,"5min":1.159,"15min":1.159},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75700000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.61899999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78100000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.44},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.40600000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72299999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.65300000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.60499999999999998,"15min":0.55200000000000005},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.8689999999999998,"5min":4.8689999999999998,"15min":4.8689999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70999999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.38800000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.501,"15min":0.51300000000000001},"min":{"1min":0.29399999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72599999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73299999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56799999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.74099999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.41699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.57299999999999995},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76400000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.501}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":556493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.34899999999999998,"15min":0.33500000000000002},"min":{"1min":0.221,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.311},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35099999999999998,"15min":0.34699999999999998},"min":{"1min":0.23400000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"last":0.41599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34799999999999998,"15min":0.33600000000000002},"min":{"1min":0.254,"5min":0.214,"15min":0.214},"max":{"1min":0.48099999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.38800000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.39600000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40699999999999997,"15min":0.39100000000000001},"min":{"1min":0.30399999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.59099999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.441},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.32900000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.34300000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60999999999999999,"15min":0.60299999999999998},"min":{"1min":0.436,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.69399999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.879},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61699999999999999,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40600000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91700000000000004,"5min":1.129,"15min":1.129},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.62,"15min":0.622},"min":{"1min":0.39100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86199999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63700000000000001,"15min":0.628},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.435,"15min":0.435},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1884160,"data_stored":1148461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.34899999999999998,"15min":0.35599999999999998},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.53200000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.247},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.372,"15min":0.36799999999999999},"min":{"1min":0.24299999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.44900000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.29599999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.249,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48999999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36799999999999999,"15min":0.35099999999999998},"min":{"1min":0.26000000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.315}]},{"osd":3,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.39100000000000001,"15min":0.40000000000000002},"min":{"1min":0.28000000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.60899999999999999,"5min":0.75,"15min":0.75},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.42599999999999999,"15min":0.436},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.45900000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.377}]},{"osd":4,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.626,"15min":0.60999999999999999},"min":{"1min":0.436,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.64900000000000002,"15min":0.66600000000000004},"min":{"1min":0.45500000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.84299999999999997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.63},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.752,"5min":1.046,"15min":1.046},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61299999999999999,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61699999999999999,"15min":0.623},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60399999999999998,"15min":0.61599999999999999},"min":{"1min":0.44500000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.45200000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.628},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63600000000000001,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77200000000000002,"5min":1.087,"15min":1.087},"last":0.48399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":556369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.35099999999999998,"15min":0.36599999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47699999999999998,"5min":0.627,"15min":0.627},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.36299999999999999,"15min":0.374},"min":{"1min":0.24099999999999999,"5min":0.186,"15min":0.186},"max":{"1min":0.55000000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.504}]},{"osd":1,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.51200000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.29899999999999999,"5min":0.245,"15min":0.245},"max":{"1min":0.51000000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.47499999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.36799999999999999,"15min":0.371},"min":{"1min":0.24399999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.51300000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.44},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41099999999999998,"15min":0.42299999999999999},"min":{"1min":0.315,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54600000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48899999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60999999999999999,"15min":0.66300000000000003},"min":{"1min":0.39900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.69999999999999996,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.44500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84399999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.67800000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60899999999999999,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73599999999999999,"5min":1.075,"15min":1.075},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76500000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.70099999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.751,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.628},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.44600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.76900000000000002,"5min":1.03,"15min":1.03},"last":0.66800000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.65400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75600000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.627,"15min":0.623},"min":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82399999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.71299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":558051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.34499999999999997,"15min":0.35699999999999998},"min":{"1min":0.20699999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.44700000000000001,"5min":1.284,"15min":1.284},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.39900000000000002,"15min":0.46300000000000002},"min":{"1min":0.23999999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59899999999999998,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.26600000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.39100000000000001,"15min":0.499},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.47299999999999998,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.222},{"interface":"front","average":{"1min":0.38,"5min":0.41999999999999998,"15min":0.48299999999999998},"min":{"1min":0.26600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.55800000000000005,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.443,"15min":0.49399999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.52800000000000002,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.42199999999999999},"min":{"1min":0.24099999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51900000000000002,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.28699999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.622,"15min":0.66600000000000004},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81299999999999994,"5min":2.9500000000000002,"15min":2.9500000000000002},"last":0.747},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.624,"15min":0.64600000000000002},"min":{"1min":0.45000000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85999999999999999,"5min":1.405,"15min":1.405},"last":0.57799999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61399999999999999,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":1.377,"15min":1.377},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.66800000000000004,"15min":0.77900000000000003},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97599999999999998,"5min":4.8129999999999997,"15min":4.8129999999999997},"last":0.68700000000000006}]},{"osd":6,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63,"15min":0.66400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93500000000000005,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.67100000000000004,"15min":0.77400000000000002},"min":{"1min":0.47399999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84999999999999998,"5min":4.883,"15min":4.883},"last":0.53900000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.65900000000000003,"15min":0.78700000000000003},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89700000000000002,"5min":4.8499999999999996,"15min":4.8499999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.66200000000000003,"15min":0.72399999999999998},"min":{"1min":0.443,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.873,"5min":2.923,"15min":2.923},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28268,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343572,"statfs":{"total":96636764160,"available":96607817728,"internally_reserved":0,"allocated":1269760,"data_stored":556369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57299999999999995,"15min":0.60599999999999998},"min":{"1min":0.36099999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.78300000000000003,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54900000000000004,"15min":0.53200000000000003},"min":{"1min":0.39400000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75700000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.50800000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.52000000000000002},"min":{"1min":0.33500000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.74299999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":2.9820000000000002,"15min":2.9820000000000002},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.53700000000000003},"min":{"1min":0.377,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.66600000000000004},"min":{"1min":0.40500000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.753,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.72599999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57299999999999995,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73499999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.622}]},{"osd":5,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55500000000000005,"15min":0.56200000000000006},"min":{"1min":0.39200000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77400000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65000000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.42599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73699999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.60399999999999998},"min":{"1min":0.42399999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79800000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.79600000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.41299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.81000000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.625,"15min":0.63600000000000001},"min":{"1min":0.51000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.78800000000000003,"5min":1.01,"15min":1.01},"last":0.74299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28276,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343564,"statfs":{"total":96636764160,"available":96607809536,"internally_reserved":0,"allocated":1277952,"data_stored":556973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53000000000000003,"15min":0.50800000000000001},"min":{"1min":0.29799999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.748,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.54300000000000004},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.46000000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.71199999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.44800000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.55400000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.39000000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47799999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.39300000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59999999999999998,"15min":0.57499999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.751,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.58999999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.35599999999999998,"5min":0.311,"15min":0.311},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.60699999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84699999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Nov 26 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.54800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:07.063 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:17:07.063 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:17:07.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.224+0000 7efd11932700 1 Processor -- start 2023-11-26T07:17:07.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.224+0000 7efd11932700 1 -- start start 2023-11-26T07:17:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.225+0000 7efd11932700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efd0c0d39d0 0x7efd0c0b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.225+0000 7efd11932700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c0d2930 con 0x7efd0c15e140 2023-11-26T07:17:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.225+0000 7efd11932700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c0d2c10 con 0x7efd0c0d45d0 2023-11-26T07:17:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.225+0000 7efd11932700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0c15e520 con 0x7efd0c0d39d0 2023-11-26T07:17:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.225+0000 7efd0b7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efd0c0d39d0 0x7efd0c0b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.225+0000 7efd0affd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efd0c15e140 0x7efd0c0d3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33458/0 (socket says 172.21.15.83:33458) 2023-11-26T07:17:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.225+0000 7efd0affd700 1 -- 172.21.15.83:0/817662008 learned_addr learned my addr 172.21.15.83:0/817662008 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.225+0000 7efd0b7fe700 1 -- 172.21.15.83:0/817662008 >> v1:172.21.15.203:6789/0 conn(0x7efd0c0d45d0 legacy=0x7efd0c0b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.225+0000 7efd0b7fe700 1 -- 172.21.15.83:0/817662008 >> v1:172.21.15.83:6789/0 conn(0x7efd0c15e140 legacy=0x7efd0c0d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.225+0000 7efd09ffb700 1 -- 172.21.15.83:0/817662008 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894647226 0 0) 0x7efd0c0d2930 con 0x7efd0c15e140 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.225+0000 7efd0b7fe700 1 -- 172.21.15.83:0/817662008 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c0d2ef0 con 0x7efd0c0d39d0 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.226+0000 7efd0b7fe700 1 --2- 172.21.15.83:0/817662008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c0d39d0 0x7efd0c0b77e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7efd04009d90 tx=0x7efd0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f70ceead0bdca0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.226+0000 7efd09ffb700 1 -- 172.21.15.83:0/817662008 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0400e070 con 0x7efd0c0d39d0 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.226+0000 7efd09ffb700 1 -- 172.21.15.83:0/817662008 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0400bbd0 con 0x7efd0c0d39d0 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.226+0000 7efd09ffb700 1 -- 172.21.15.83:0/817662008 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04012430 con 0x7efd0c0d39d0 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.226+0000 7efd11932700 1 -- 172.21.15.83:0/817662008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c0d39d0 msgr2=0x7efd0c0b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.226+0000 7efd11932700 1 --2- 172.21.15.83:0/817662008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c0d39d0 0x7efd0c0b77e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7efd04009d90 tx=0x7efd0400b330 comp rx=0 tx=0).stop 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.226+0000 7efd11932700 1 -- 172.21.15.83:0/817662008 shutdown_connections 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.226+0000 7efd11932700 1 --2- 172.21.15.83:0/817662008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c0d39d0 0x7efd0c0b77e0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.226+0000 7efd11932700 1 -- 172.21.15.83:0/817662008 >> 172.21.15.83:0/817662008 conn(0x7efd0c06b410 msgr2=0x7efd0c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:07.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.226+0000 7efd11932700 1 -- 172.21.15.83:0/817662008 shutdown_connections 2023-11-26T07:17:07.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.226+0000 7efd11932700 1 -- 172.21.15.83:0/817662008 wait complete. 2023-11-26T07:17:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.228+0000 7efd11932700 1 Processor -- start 2023-11-26T07:17:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.228+0000 7efd11932700 1 -- start start 2023-11-26T07:17:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd11932700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd0c0d39d0 0x7efd0c15c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd11932700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd0c0d45d0 0x7efd0c0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd11932700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15e140 0x7efd0c0cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd11932700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c15c540 con 0x7efd0c15e140 2023-11-26T07:17:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd11932700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0c8fe0 con 0x7efd0c0d45d0 2023-11-26T07:17:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd11932700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0c86b0 con 0x7efd0c0d39d0 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd0b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15e140 0x7efd0c0cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd0b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15e140 0x7efd0c0cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41658/0 (socket says 172.21.15.83:41658) 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd0b7fe700 1 -- 172.21.15.83:0/1052840783 learned_addr learned my addr 172.21.15.83:0/1052840783 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd0affd700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd0c0d39d0 0x7efd0c15c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd0a7fc700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd0c0d45d0 0x7efd0c0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd0b7fe700 1 -- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd0c0d39d0 msgr2=0x7efd0c15c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd0b7fe700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd0c0d39d0 0x7efd0c15c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd0b7fe700 1 -- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd0c0d45d0 msgr2=0x7efd0c0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd0b7fe700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd0c0d45d0 0x7efd0c0ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd0b7fe700 1 -- 172.21.15.83:0/1052840783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd04009a70 con 0x7efd0c15e140 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.229+0000 7efd0a7fc700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd0c0d45d0 0x7efd0c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.230+0000 7efd0b7fe700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15e140 0x7efd0c0cd150 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7efd04014040 tx=0x7efd04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.230+0000 7efcf3fff700 1 -- 172.21.15.83:0/1052840783 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04012430 con 0x7efd0c15e140 2023-11-26T07:17:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.230+0000 7efcf3fff700 1 -- 172.21.15.83:0/1052840783 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd04020070 con 0x7efd0c15e140 2023-11-26T07:17:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.230+0000 7efd11932700 1 -- 172.21.15.83:0/1052840783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0c0c8990 con 0x7efd0c15e140 2023-11-26T07:17:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.230+0000 7efcf3fff700 1 -- 172.21.15.83:0/1052840783 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04004280 con 0x7efd0c15e140 2023-11-26T07:17:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.230+0000 7efd11932700 1 -- 172.21.15.83:0/1052840783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0c0cbfd0 con 0x7efd0c15e140 2023-11-26T07:17:07.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.231+0000 7efd11932700 1 -- 172.21.15.83:0/1052840783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0c068fb0 con 0x7efd0c15e140 2023-11-26T07:17:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.231+0000 7efcf3fff700 1 -- 172.21.15.83:0/1052840783 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7efd040043e0 con 0x7efd0c15e140 2023-11-26T07:17:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.232+0000 7efcf3fff700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efcf806aff0 0x7efcf806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.232+0000 7efcf3fff700 1 -- 172.21.15.83:0/1052840783 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd04022070 con 0x7efd0c15e140 2023-11-26T07:17:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.232+0000 7efd0affd700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efcf806aff0 0x7efcf806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.232+0000 7efd0affd700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efcf806aff0 0x7efcf806d4b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efd00007fd0 tx=0x7efd00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.236+0000 7efcf3fff700 1 -- 172.21.15.83:0/1052840783 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd04056770 con 0x7efd0c15e140 2023-11-26T07:17:07.382 INFO:tasks.ceph.mds.a.smithi083.stderr:2023-11-26T07:17:07.381+0000 7f647223c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:17:07.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.504+0000 7efd11932700 1 -- 172.21.15.83:0/1052840783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7efd0c0d1bb0 con 0x7efd0c15e140 2023-11-26T07:17:07.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.506+0000 7efcf3fff700 1 -- 172.21.15.83:0/1052840783 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7efd04059d90 con 0x7efd0c15e140 2023-11-26T07:17:07.508 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2023-11-26T07:17:07.508 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBl72JlRnGZExAAEIZNDknqljAIKEfTz94fCQ== 2023-11-26T07:17:07.508 INFO:teuthology.orchestra.run.smithi083.stdout: caps mds = "allow" 2023-11-26T07:17:07.508 INFO:teuthology.orchestra.run.smithi083.stdout: caps mon = "allow r" 2023-11-26T07:17:07.509 INFO:teuthology.orchestra.run.smithi083.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:17:07.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.509+0000 7efd11932700 1 -- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efcf806aff0 msgr2=0x7efcf806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.509+0000 7efd11932700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efcf806aff0 0x7efcf806d4b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efd00007fd0 tx=0x7efd00009040 comp rx=0 tx=0).stop 2023-11-26T07:17:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.509+0000 7efd11932700 1 -- 172.21.15.83:0/1052840783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15e140 msgr2=0x7efd0c0cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.509+0000 7efd11932700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15e140 0x7efd0c0cd150 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7efd04014040 tx=0x7efd04004ad0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.510+0000 7efd11932700 1 -- 172.21.15.83:0/1052840783 shutdown_connections 2023-11-26T07:17:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.510+0000 7efd11932700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efcf806aff0 0x7efcf806d4b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.510+0000 7efd11932700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0c15e140 0x7efd0c0cd150 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.510+0000 7efd11932700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd0c0d39d0 0x7efd0c15c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.510+0000 7efd11932700 1 --2- 172.21.15.83:0/1052840783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd0c0d45d0 0x7efd0c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.510+0000 7efd11932700 1 -- 172.21.15.83:0/1052840783 >> 172.21.15.83:0/1052840783 conn(0x7efd0c06b410 msgr2=0x7efd0c0b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.510+0000 7efd11932700 1 -- 172.21.15.83:0/1052840783 shutdown_connections 2023-11-26T07:17:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.510+0000 7efd11932700 1 -- 172.21.15.83:0/1052840783 wait complete. 2023-11-26T07:17:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:updated caps for client.0 2023-11-26T07:17:07.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:07.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.642+0000 7fe918384700 1 Processor -- start 2023-11-26T07:17:07.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.642+0000 7fe918384700 1 -- start start 2023-11-26T07:17:07.646 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.643+0000 7fe918384700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe91015e080 0x7fe9100b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.643+0000 7fe918384700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe9100af4d0 0x7fe9100afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.643+0000 7fe918384700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe91006c000 0x7fe91006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.643+0000 7fe918384700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9100af0b0 con 0x7fe91006c000 2023-11-26T07:17:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.643+0000 7fe918384700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91006c9f0 con 0x7fe9100af4d0 2023-11-26T07:17:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.643+0000 7fe918384700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9100b05a0 con 0x7fe91015e080 2023-11-26T07:17:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe916921700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe91006c000 0x7fe91006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe916921700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe91006c000 0x7fe91006c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41666/0 (socket says 172.21.15.83:41666) 2023-11-26T07:17:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe916921700 1 -- 172.21.15.83:0/3774512408 learned_addr learned my addr 172.21.15.83:0/3774512408 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe91591f700 1 --2- 172.21.15.83:0/3774512408 >> v2:172.21.15.203:3300/0 conn(0x7fe9100af4d0 0x7fe9100afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe916120700 1 --2- 172.21.15.83:0/3774512408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe91015e080 0x7fe9100b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe916921700 1 -- 172.21.15.83:0/3774512408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe91015e080 msgr2=0x7fe9100b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe916921700 1 --2- 172.21.15.83:0/3774512408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe91015e080 0x7fe9100b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe916921700 1 -- 172.21.15.83:0/3774512408 >> v2:172.21.15.203:3300/0 conn(0x7fe9100af4d0 msgr2=0x7fe9100afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe916921700 1 --2- 172.21.15.83:0/3774512408 >> v2:172.21.15.203:3300/0 conn(0x7fe9100af4d0 0x7fe9100afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe916921700 1 -- 172.21.15.83:0/3774512408 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9100ae430 con 0x7fe91006c000 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe916921700 1 --2- 172.21.15.83:0/3774512408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe91006c000 0x7fe91006c4b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe91006d380 tx=0x7fe90800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c396e5cac13ece6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe91511e700 1 -- 172.21.15.83:0/3774512408 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800e070 con 0x7fe91006c000 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe91511e700 1 -- 172.21.15.83:0/3774512408 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90800b8d0 con 0x7fe91006c000 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.644+0000 7fe91511e700 1 -- 172.21.15.83:0/3774512408 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90800bc80 con 0x7fe91006c000 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.645+0000 7fe918384700 1 -- 172.21.15.83:0/3774512408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe91006c000 msgr2=0x7fe91006c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.645+0000 7fe918384700 1 --2- 172.21.15.83:0/3774512408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe91006c000 0x7fe91006c4b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe91006d380 tx=0x7fe90800b330 comp rx=0 tx=0).stop 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.645+0000 7fe918384700 1 -- 172.21.15.83:0/3774512408 shutdown_connections 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.645+0000 7fe918384700 1 --2- 172.21.15.83:0/3774512408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe91006c000 0x7fe91006c4b0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.645+0000 7fe918384700 1 --2- 172.21.15.83:0/3774512408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe91015e080 0x7fe9100b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.645+0000 7fe918384700 1 --2- 172.21.15.83:0/3774512408 >> v2:172.21.15.203:3300/0 conn(0x7fe9100af4d0 0x7fe9100afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.645+0000 7fe918384700 1 -- 172.21.15.83:0/3774512408 >> 172.21.15.83:0/3774512408 conn(0x7fe910071b90 msgr2=0x7fe910071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.645+0000 7fe918384700 1 -- 172.21.15.83:0/3774512408 shutdown_connections 2023-11-26T07:17:07.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.645+0000 7fe918384700 1 -- 172.21.15.83:0/3774512408 wait complete. 2023-11-26T07:17:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.646+0000 7fe918384700 1 Processor -- start 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.646+0000 7fe918384700 1 -- start start 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe918384700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe91006c000 0x7fe910157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe918384700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9100af4d0 0x7fe91015ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe918384700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe91015e080 0x7fe9101604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe918384700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe910158950 con 0x7fe91006c000 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe918384700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe910157cd0 con 0x7fe9100af4d0 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe918384700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe910157fb0 con 0x7fe91015e080 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe916120700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe91006c000 0x7fe910157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe916120700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe91006c000 0x7fe910157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41668/0 (socket says 172.21.15.83:41668) 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe916120700 1 -- 172.21.15.83:0/3955473499 learned_addr learned my addr 172.21.15.83:0/3955473499 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe916120700 1 -- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe91015e080 msgr2=0x7fe9101604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe916921700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe91015e080 0x7fe9101604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe916120700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe91015e080 0x7fe9101604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe91591f700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9100af4d0 0x7fe91015ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe916120700 1 -- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9100af4d0 msgr2=0x7fe91015ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe916120700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9100af4d0 0x7fe91015ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe916120700 1 -- 172.21.15.83:0/3955473499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe908009b30 con 0x7fe91006c000 2023-11-26T07:17:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe916120700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe91006c000 0x7fe910157250 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe90000e870 tx=0x7fe90000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe906ffd700 1 -- 172.21.15.83:0/3955473499 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900009590 con 0x7fe91006c000 2023-11-26T07:17:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe906ffd700 1 -- 172.21.15.83:0/3955473499 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90000ee70 con 0x7fe91006c000 2023-11-26T07:17:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe918384700 1 -- 172.21.15.83:0/3955473499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe910158290 con 0x7fe91006c000 2023-11-26T07:17:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.647+0000 7fe906ffd700 1 -- 172.21.15.83:0/3955473499 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90000ca50 con 0x7fe91006c000 2023-11-26T07:17:07.653 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.648+0000 7fe918384700 1 -- 172.21.15.83:0/3955473499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe910156a00 con 0x7fe91006c000 2023-11-26T07:17:07.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.648+0000 7fe918384700 1 -- 172.21.15.83:0/3955473499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9100753f0 con 0x7fe91006c000 2023-11-26T07:17:07.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.649+0000 7fe906ffd700 1 -- 172.21.15.83:0/3955473499 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fe90000cbb0 con 0x7fe91006c000 2023-11-26T07:17:07.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.649+0000 7fe906ffd700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8f406aff0 0x7fe8f406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.649+0000 7fe906ffd700 1 -- 172.21.15.83:0/3955473499 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9000896f0 con 0x7fe91006c000 2023-11-26T07:17:07.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.649+0000 7fe91591f700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8f406aff0 0x7fe8f406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.654 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.650+0000 7fe91591f700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8f406aff0 0x7fe8f406d4b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe91015a250 tx=0x7fe90c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.654+0000 7fe906ffd700 1 -- 172.21.15.83:0/3955473499 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe900054680 con 0x7fe91006c000 2023-11-26T07:17:07.890 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.889+0000 7fe918384700 1 -- 172.21.15.83:0/3955473499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe910067f60 con 0x7fe91006c000 2023-11-26T07:17:07.891 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.890+0000 7fe906ffd700 1 -- 172.21.15.83:0/3955473499 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fe90000ce60 con 0x7fe91006c000 2023-11-26T07:17:07.891 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:07.891 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/136760159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":136760159},{"type":"v1","addr":"172.21.15.203:6834","nonce":136760159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3723980912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":3723980912},{"type":"v1","addr":"172.21.15.203:6835","nonce":3723980912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3900154869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3900154869},{"type":"v1","addr":"172.21.15.83:6833","nonce":3900154869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:01.974687+0000","modified":"2023-11-26T07:17:04.985222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7551":{"gid":7551,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3327659486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3327659486},{"type":"v1","addr":"172.21.15.83:6835","nonce":3327659486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:17:07.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 -- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8f406aff0 msgr2=0x7fe8f406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8f406aff0 0x7fe8f406d4b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe91015a250 tx=0x7fe90c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 -- 172.21.15.83:0/3955473499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe91006c000 msgr2=0x7fe910157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe91006c000 0x7fe910157250 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe90000e870 tx=0x7fe90000ec30 comp rx=0 tx=0).stop 2023-11-26T07:17:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 -- 172.21.15.83:0/3955473499 shutdown_connections 2023-11-26T07:17:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe8f406aff0 0x7fe8f406d4b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe91015e080 0x7fe9101604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe91006c000 0x7fe910157250 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 --2- 172.21.15.83:0/3955473499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9100af4d0 0x7fe91015ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 -- 172.21.15.83:0/3955473499 >> 172.21.15.83:0/3955473499 conn(0x7fe910071b90 msgr2=0x7fe9100b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 -- 172.21.15.83:0/3955473499 shutdown_connections 2023-11-26T07:17:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:07.892+0000 7fe918384700 1 -- 172.21.15.83:0/3955473499 wait complete. 2023-11-26T07:17:07.895 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2023-11-26T07:17:08.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:17:01.974687+0000', 'modified': '2023-11-26T07:17:04.985222+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7551}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7551': {'gid': 7551, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6835/3327659486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 3327659486}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 3327659486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:17:08.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:17:08.308 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-26T07:17:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.429+0000 7f5ca66a7700 1 Processor -- start 2023-11-26T07:17:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.429+0000 7f5ca66a7700 1 -- start start 2023-11-26T07:17:08.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca66a7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ca006c710 0x7f5ca00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca66a7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ca00af4d0 0x7f5ca00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca66a7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca00ae430 con 0x7f5ca015e080 2023-11-26T07:17:08.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca66a7700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca00af0b0 con 0x7f5ca00af4d0 2023-11-26T07:17:08.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca66a7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00b05a0 con 0x7f5ca006c710 2023-11-26T07:17:08.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca4c44700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ca00af4d0 0x7f5ca00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca4c44700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5ca00af4d0 0x7f5ca00afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41680/0 (socket says 172.21.15.83:41680) 2023-11-26T07:17:08.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca4c44700 1 -- 172.21.15.83:0/78299974 learned_addr learned my addr 172.21.15.83:0/78299974 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/78299974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ca006c710 0x7f5ca00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca4c44700 1 -- 172.21.15.83:0/78299974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ca006c710 msgr2=0x7f5ca00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/78299974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ca006c710 0x7f5ca00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca4c44700 1 -- 172.21.15.83:0/78299974 >> v1:172.21.15.203:6789/0 conn(0x7f5ca015e080 legacy=0x7f5ca006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca4c44700 1 -- 172.21.15.83:0/78299974 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca00ae710 con 0x7f5ca00af4d0 2023-11-26T07:17:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/78299974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ca006c710 0x7f5ca00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/78299974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca00af4d0 0x7f5ca00afb90 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009e80 tx=0x7f5c9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894071a0a6c85239 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5c9effd700 1 -- 172.21.15.83:0/78299974 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800e070 con 0x7f5ca00af4d0 2023-11-26T07:17:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5c9effd700 1 -- 172.21.15.83:0/78299974 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c98003a20 con 0x7f5ca00af4d0 2023-11-26T07:17:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.430+0000 7f5c9effd700 1 -- 172.21.15.83:0/78299974 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98003dd0 con 0x7f5ca00af4d0 2023-11-26T07:17:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.431+0000 7f5ca66a7700 1 -- 172.21.15.83:0/78299974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca00af4d0 msgr2=0x7f5ca00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.431+0000 7f5ca66a7700 1 --2- 172.21.15.83:0/78299974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca00af4d0 0x7f5ca00afb90 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009e80 tx=0x7f5c9800b330 comp rx=0 tx=0).stop 2023-11-26T07:17:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.431+0000 7f5ca66a7700 1 -- 172.21.15.83:0/78299974 shutdown_connections 2023-11-26T07:17:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.431+0000 7f5ca66a7700 1 --2- 172.21.15.83:0/78299974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca00af4d0 0x7f5ca00afb90 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.431+0000 7f5ca66a7700 1 --2- 172.21.15.83:0/78299974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ca006c710 0x7f5ca00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.431+0000 7f5ca66a7700 1 -- 172.21.15.83:0/78299974 >> 172.21.15.83:0/78299974 conn(0x7f5ca0071b90 msgr2=0x7f5ca0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.431+0000 7f5ca66a7700 1 -- 172.21.15.83:0/78299974 shutdown_connections 2023-11-26T07:17:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.431+0000 7f5ca66a7700 1 -- 172.21.15.83:0/78299974 wait complete. 2023-11-26T07:17:08.437 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.432+0000 7f5ca66a7700 1 Processor -- start 2023-11-26T07:17:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.432+0000 7f5ca66a7700 1 -- start start 2023-11-26T07:17:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca66a7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ca006c710 0x7f5ca015f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca66a7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ca00af4d0 0x7f5ca01f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca66a7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca015e080 0x7f5ca01f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca66a7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca0141f70 con 0x7f5ca015e080 2023-11-26T07:17:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca66a7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca01f4730 con 0x7f5ca006c710 2023-11-26T07:17:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca66a7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca01f3600 con 0x7f5ca00af4d0 2023-11-26T07:17:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca4c44700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca015e080 0x7f5ca01f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca4c44700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca015e080 0x7f5ca01f30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41686/0 (socket says 172.21.15.83:41686) 2023-11-26T07:17:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca4c44700 1 -- 172.21.15.83:0/2704247865 learned_addr learned my addr 172.21.15.83:0/2704247865 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ca006c710 0x7f5ca015f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ca00af4d0 0x7f5ca01f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca4c44700 1 -- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ca00af4d0 msgr2=0x7f5ca01f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ca00af4d0 0x7f5ca01f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca4c44700 1 -- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ca006c710 msgr2=0x7f5ca015f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ca006c710 0x7f5ca015f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.433+0000 7f5ca4c44700 1 -- 172.21.15.83:0/2704247865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c98009b30 con 0x7f5ca015e080 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.434+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca015e080 0x7f5ca01f30c0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800b730 tx=0x7f5c98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.434+0000 7f5c9cff9700 1 -- 172.21.15.83:0/2704247865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98004440 con 0x7f5ca015e080 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.434+0000 7f5c9cff9700 1 -- 172.21.15.83:0/2704247865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c980045c0 con 0x7f5ca015e080 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.434+0000 7f5ca66a7700 1 -- 172.21.15.83:0/2704247865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca0160040 con 0x7f5ca015e080 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.434+0000 7f5c9cff9700 1 -- 172.21.15.83:0/2704247865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98004970 con 0x7f5ca015e080 2023-11-26T07:17:08.439 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.434+0000 7f5ca66a7700 1 -- 172.21.15.83:0/2704247865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca0160600 con 0x7f5ca015e080 2023-11-26T07:17:08.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.436+0000 7f5c9cff9700 1 -- 172.21.15.83:0/2704247865 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f5c98012430 con 0x7f5ca015e080 2023-11-26T07:17:08.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.436+0000 7f5c9cff9700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5c8c06aff0 0x7f5c8c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.436+0000 7f5c9cff9700 1 -- 172.21.15.83:0/2704247865 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c9808add0 con 0x7f5ca015e080 2023-11-26T07:17:08.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.436+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5c8c06aff0 0x7f5c8c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.437+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5c8c06aff0 0x7f5c8c06d4b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ca01555f0 tx=0x7f5c90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.438+0000 7f5cb28a0b80 1 Processor -- start 2023-11-26T07:17:08.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.438+0000 7f5cb28a0b80 1 -- start start 2023-11-26T07:17:08.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d7c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797d5fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560797ca9770 con 0x560797d4bed0 2023-11-26T07:17:08.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560797c90750 con 0x560797d33490 2023-11-26T07:17:08.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x560797c8e070 con 0x560797d36080 2023-11-26T07:17:08.446 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c9f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c9f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41692/0 (socket says 172.21.15.83:41692) 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c9f7fe700 1 -- 172.21.15.83:0/975488968 learned_addr learned my addr 172.21.15.83:0/975488968 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/975488968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d7c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/975488968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797d5fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c9f7fe700 1 -- 172.21.15.83:0/975488968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 msgr2=0x560797d7c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/975488968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d7c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c9f7fe700 1 -- 172.21.15.83:0/975488968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 msgr2=0x560797d5fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/975488968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797d5fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c9f7fe700 1 -- 172.21.15.83:0/975488968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560797b065e0 con 0x560797d4bed0 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/975488968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7cd80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400ab60 tx=0x7f5c9400ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0978fcfda93f3a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c85ffb700 1 -- 172.21.15.83:0/975488968 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9400e990 con 0x560797d4bed0 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c85ffb700 1 -- 172.21.15.83:0/975488968 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9400c050 con 0x560797d4bed0 2023-11-26T07:17:08.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.439+0000 7f5c85ffb700 1 -- 172.21.15.83:0/975488968 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9400ed30 con 0x560797d4bed0 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.440+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/975488968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 msgr2=0x560797d7cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.440+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/975488968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7cd80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400ab60 tx=0x7f5c9400ae70 comp rx=0 tx=0).stop 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.440+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/975488968 shutdown_connections 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.440+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/975488968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797d5fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.440+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/975488968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d7c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.440+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/975488968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7cd80 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.440+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/975488968 >> 172.21.15.83:0/975488968 conn(0x560797c64310 msgr2=0x560797d7ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.440+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/975488968 shutdown_connections 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.440+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/975488968 wait complete. 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.441+0000 7f5cb28a0b80 1 -- start start 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.441+0000 7f5cb28a0b80 1 Processor -- start 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d33490 0x560797d5d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d5dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d4bed0 0x560797d68b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560797c89960 con 0x560797d33490 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560797c43510 con 0x560797d4bed0 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x560797c431b0 con 0x560797d36080 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5cb28a0b80 10 client.0 _ll_register_callbacks cb 0x560797b6ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c9ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d33490 0x560797d5d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c9ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d33490 0x560797d5d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41696/0 (socket says 172.21.15.83:41696) 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c9ffff700 1 -- 172.21.15.83:0/2497292758 learned_addr learned my addr 172.21.15.83:0/2497292758 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d4bed0 0x560797d68b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d5dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c9ffff700 1 -- 172.21.15.83:0/2497292758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 msgr2=0x560797d5dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d5dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c9ffff700 1 -- 172.21.15.83:0/2497292758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d4bed0 msgr2=0x560797d68b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.449 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d4bed0 0x560797d68b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c9ffff700 1 -- 172.21.15.83:0/2497292758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9400a7c0 con 0x560797d33490 2023-11-26T07:17:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d33490 0x560797d5d730 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5c90014d80 tx=0x7f5c90004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbc64ebb41b696e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c7effd700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:17:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c7effd700 1 -- 172.21.15.83:0/2497292758 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90014e60 con 0x560797d33490 2023-11-26T07:17:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c7effd700 1 -- 172.21.15.83:0/2497292758 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c90012e10 con 0x560797d33490 2023-11-26T07:17:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/2497292758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560797d28a40 con 0x560797d33490 2023-11-26T07:17:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5c7effd700 1 -- 172.21.15.83:0/2497292758 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c90020420 con 0x560797d33490 2023-11-26T07:17:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5cb28a0b80 10 client.7692 fetch_fsmap learned FSMap version 69 2023-11-26T07:17:08.450 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.442+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/2497292758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x560797ab2d90 con 0x560797d33490 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.443+0000 7f5c7effd700 1 -- 172.21.15.83:0/2497292758 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560797d28a40 con 0x560797d33490 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.443+0000 7f5c7effd700 1 -- 172.21.15.83:0/2497292758 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5c90020580 con 0x560797d33490 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.443+0000 7f5cb28a0b80 10 client.7692 fetch_fsmap finished waiting for FSMap version 69 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.443+0000 7f5cb28a0b80 10 client.7692 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.443+0000 7f5cb28a0b80 20 client.7692 populate_metadata read hostname 'smithi083' 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.443+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.83:6834/3327659486,v1:172.21.15.83:6835/3327659486] conn(0x560797d69610 0x560797d5c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.443+0000 7f5cb28a0b80 4 client.7692 mds_command: new command op to 7551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.443+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/2497292758 --> [v2:172.21.15.83:6834/3327659486,v1:172.21.15.83:6835/3327659486] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560797d04840 con 0x560797d69610 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.443+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.83:6834/3327659486,v1:172.21.15.83:6835/3327659486] conn(0x560797d69610 0x560797d5c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.443+0000 7f5c7effd700 10 client.7692 ms_handle_connect on v2:172.21.15.83:6834/3327659486 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.443+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.83:6834/3327659486,v1:172.21.15.83:6835/3327659486] conn(0x560797d69610 0x560797d5c3b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e19b6483beb718 server_cookie=1da3daec6a64adeb in_seq=0 out_seq=0 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5c7effd700 1 -- 172.21.15.83:0/2497292758 <== mds.0 v2:172.21.15.83:6834/3327659486 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560797d04840 con 0x560797d69610 2023-11-26T07:17:08.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5c7effd700 10 client.7692 handle_command_reply: tid=0 2023-11-26T07:17:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 1 client.7692 shutdown 2023-11-26T07:17:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/2497292758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d33490 msgr2=0x560797d5d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d33490 0x560797d5d730 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5c90014d80 tx=0x7f5c90004710 comp rx=0 tx=0).stop 2023-11-26T07:17:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/2497292758 shutdown_connections 2023-11-26T07:17:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d4bed0 0x560797d68b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d33490 0x560797d5d730 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.83:6834/3327659486,v1:172.21.15.83:6835/3327659486] conn(0x560797d69610 0x560797d5c3b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/2497292758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d5dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/2497292758 >> 172.21.15.83:0/2497292758 conn(0x560797c64310 msgr2=0x560797d7ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5c7effd700 0 client.7692 ms_handle_reset on v2:172.21.15.83:6834/3327659486 2023-11-26T07:17:08.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/2497292758 shutdown_connections 2023-11-26T07:17:08.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/2497292758 wait complete. 2023-11-26T07:17:08.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.444+0000 7f5cb28a0b80 20 client.7692 trim_cache size 0 max 16384 2023-11-26T07:17:08.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.462+0000 7f5cb28a0b80 1 Processor -- start 2023-11-26T07:17:08.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.462+0000 7f5cb28a0b80 1 -- start start 2023-11-26T07:17:08.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797d59860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d36080 0x560797d29ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d4bed0 0x560797d79f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560797b088d0 con 0x560797d36080 2023-11-26T07:17:08.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560797d7b1a0 con 0x560797d33490 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x560797d682f0 con 0x560797d4bed0 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c9f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d36080 0x560797d29ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c9f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d36080 0x560797d29ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41704/0 (socket says 172.21.15.83:41704) 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c9f7fe700 1 -- 172.21.15.83:0/1577704458 learned_addr learned my addr 172.21.15.83:0/1577704458 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/1577704458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d4bed0 0x560797d79f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/1577704458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797d59860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c9f7fe700 1 -- 172.21.15.83:0/1577704458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d4bed0 msgr2=0x560797d79f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/1577704458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d4bed0 0x560797d79f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c9f7fe700 1 -- 172.21.15.83:0/1577704458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 msgr2=0x560797d59860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/1577704458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797d59860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c9f7fe700 1 -- 172.21.15.83:0/1577704458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560797d32330 con 0x560797d36080 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/1577704458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d4bed0 0x560797d79f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/1577704458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797d59860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/1577704458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d36080 0x560797d29ea0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5c94010010 tx=0x7f5c94014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0d6fe00625cc74 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c867fc700 1 -- 172.21.15.83:0/1577704458 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94024d00 con 0x560797d36080 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c867fc700 1 -- 172.21.15.83:0/1577704458 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c94024e60 con 0x560797d36080 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.463+0000 7f5c867fc700 1 -- 172.21.15.83:0/1577704458 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c940243e0 con 0x560797d36080 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.464+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/1577704458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d36080 msgr2=0x560797d29ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.464+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/1577704458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d36080 0x560797d29ea0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5c94010010 tx=0x7f5c94014000 comp rx=0 tx=0).stop 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.464+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/1577704458 shutdown_connections 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.464+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/1577704458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d4bed0 0x560797d79f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.464+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/1577704458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797d59860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.464+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/1577704458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d36080 0x560797d29ea0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.464+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/1577704458 >> 172.21.15.83:0/1577704458 conn(0x560797c64310 msgr2=0x560797d30190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.464+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/1577704458 shutdown_connections 2023-11-26T07:17:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.464+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/1577704458 wait complete. 2023-11-26T07:17:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.464+0000 7f5cb28a0b80 1 -- start start 2023-11-26T07:17:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.465+0000 7f5cb28a0b80 1 Processor -- start 2023-11-26T07:17:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.465+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797de3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.465+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d7a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.465+0000 7f5cb28a0b80 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.465+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560797de3f20 con 0x560797d4bed0 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.465+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560797d2a050 con 0x560797d33490 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.465+0000 7f5cb28a0b80 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x560797d58f50 con 0x560797d36080 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.465+0000 7f5cb28a0b80 10 client.0 _ll_register_callbacks cb 0x560797b6ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5ca4c44700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5ca4c44700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41712/0 (socket says 172.21.15.83:41712) 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5ca4c44700 1 -- 172.21.15.83:0/3826812295 learned_addr learned my addr 172.21.15.83:0/3826812295 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797de3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5c9f7fe700 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d7a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5ca4c44700 1 -- 172.21.15.83:0/3826812295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 msgr2=0x560797d7a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d7a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5ca4c44700 1 -- 172.21.15.83:0/3826812295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 msgr2=0x560797de3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797de3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5ca4c44700 1 -- 172.21.15.83:0/3826812295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560797b065e0 con 0x560797d4bed0 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5ca4c44700 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7a640 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800bfd0 tx=0x7f5c9805db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a2e94954796929 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5c7effd700 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5c7effd700 1 -- 172.21.15.83:0/3826812295 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c980712e0 con 0x560797d4bed0 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5c7effd700 1 -- 172.21.15.83:0/3826812295 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c980092e0 con 0x560797d4bed0 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/3826812295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c9400c050 con 0x560797d4bed0 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5c7effd700 1 -- 172.21.15.83:0/3826812295 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c98059c10 con 0x560797d4bed0 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5cb28a0b80 10 client.7698 fetch_fsmap learned FSMap version 69 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/3826812295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x560797a9bc10 con 0x560797d4bed0 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5c7effd700 1 -- 172.21.15.83:0/3826812295 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9400c050 con 0x560797d4bed0 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5c7effd700 1 -- 172.21.15.83:0/3826812295 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5c98058ea0 con 0x560797d4bed0 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5cb28a0b80 10 client.7698 fetch_fsmap finished waiting for FSMap version 69 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5cb28a0b80 10 client.7698 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5cb28a0b80 20 client.7698 populate_metadata read hostname 'smithi083' 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.83:6834/3327659486,v1:172.21.15.83:6835/3327659486] conn(0x560797d58390 0x560797d58770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5cb28a0b80 4 client.7698 mds_command: new command op to 7551 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.466+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/3826812295 --> [v2:172.21.15.83:6834/3327659486,v1:172.21.15.83:6835/3327659486] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560797c4ff80 con 0x560797d58390 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.83:6834/3327659486,v1:172.21.15.83:6835/3327659486] conn(0x560797d58390 0x560797d58770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5c7effd700 10 client.7698 ms_handle_connect on v2:172.21.15.83:6834/3327659486 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5c9ffff700 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.83:6834/3327659486,v1:172.21.15.83:6835/3327659486] conn(0x560797d58390 0x560797d58770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72cb7ccd4fed7ef server_cookie=accc3b5af56b77be in_seq=0 out_seq=0 2023-11-26T07:17:08.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5c7effd700 1 -- 172.21.15.83:0/3826812295 <== mds.0 v2:172.21.15.83:6834/3327659486 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x560797c4ff80 con 0x560797d58390 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5c7effd700 10 client.7698 handle_command_reply: tid=0 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5cb28a0b80 1 client.7698 shutdown 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/3826812295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 msgr2=0x560797d7a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7a640 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800bfd0 tx=0x7f5c9805db00 comp rx=0 tx=0).stop 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/3826812295 shutdown_connections 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560797d4bed0 0x560797d7a640 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560797d33490 0x560797de3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.83:6834/3327659486,v1:172.21.15.83:6835/3327659486] conn(0x560797d58390 0x560797d58770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5cb28a0b80 1 --2- 172.21.15.83:0/3826812295 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560797d36080 0x560797d7a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5c7effd700 0 client.7698 ms_handle_reset on v2:172.21.15.83:6834/3327659486 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/3826812295 >> 172.21.15.83:0/3826812295 conn(0x560797c64310 msgr2=0x560797d7d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/3826812295 shutdown_connections 2023-11-26T07:17:08.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.467+0000 7f5cb28a0b80 1 -- 172.21.15.83:0/3826812295 wait complete. 2023-11-26T07:17:08.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5cb28a0b80 20 client.7698 trim_cache size 0 max 16384 2023-11-26T07:17:08.477 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "0aae1049-cc8c-4011-a01f-378673e57a96", 2023-11-26T07:17:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2023-11-26T07:17:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 7551, 2023-11-26T07:17:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2023-11-26T07:17:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2023-11-26T07:17:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2023-11-26T07:17:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 5.8853546080000001, 2023-11-26T07:17:08.477 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 69, 2023-11-26T07:17:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 102, 2023-11-26T07:17:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 102, 2023-11-26T07:17:08.478 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 11.535684309000001 2023-11-26T07:17:08.478 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 -- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5c8c06aff0 msgr2=0x7f5c8c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5c8c06aff0 0x7f5c8c06d4b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ca01555f0 tx=0x7f5c90008040 comp rx=0 tx=0).stop 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 -- 172.21.15.83:0/2704247865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca015e080 msgr2=0x7f5ca01f30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca015e080 0x7f5ca01f30c0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800b730 tx=0x7f5c98004ad0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 -- 172.21.15.83:0/2704247865 shutdown_connections 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5c8c06aff0 0x7f5c8c06d4b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca015e080 0x7f5ca01f30c0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ca006c710 0x7f5ca015f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 --2- 172.21.15.83:0/2704247865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ca00af4d0 0x7f5ca01f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 -- 172.21.15.83:0/2704247865 >> 172.21.15.83:0/2704247865 conn(0x7f5ca0071b90 msgr2=0x7f5ca006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 -- 172.21.15.83:0/2704247865 shutdown_connections 2023-11-26T07:17:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.468+0000 7f5ca66a7700 1 -- 172.21.15.83:0/2704247865 wait complete. 2023-11-26T07:17:08.481 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:17:08.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.600+0000 7f3fb0aaa700 1 Processor -- start 2023-11-26T07:17:08.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.600+0000 7f3fb0aaa700 1 -- start start 2023-11-26T07:17:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fb0aaa700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3fac15bdd0 0x7f3fac1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fb0aaa700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3fac0b3370 0x7f3fac0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fb0aaa700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fac0b2030 con 0x7f3fac0d1c70 2023-11-26T07:17:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fb0aaa700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac0b2f50 con 0x7f3fac0b3370 2023-11-26T07:17:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fb0aaa700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac0b22d0 con 0x7f3fac15bdd0 2023-11-26T07:17:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fa9d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3fac0b3370 0x7f3fac0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fa9d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3fac0b3370 0x7f3fac0b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41718/0 (socket says 172.21.15.83:41718) 2023-11-26T07:17:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3faad9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3fac0d1c70 0x7f3fac0ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33528/0 (socket says 172.21.15.83:33528) 2023-11-26T07:17:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fa9d9b700 1 -- 172.21.15.83:0/607863069 learned_addr learned my addr 172.21.15.83:0/607863069 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3faa59c700 1 --2- 172.21.15.83:0/607863069 >> v2:172.21.15.203:3300/0 conn(0x7f3fac15bdd0 0x7f3fac1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fa9d9b700 1 -- 172.21.15.83:0/607863069 >> v2:172.21.15.203:3300/0 conn(0x7f3fac15bdd0 msgr2=0x7f3fac1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fa9d9b700 1 --2- 172.21.15.83:0/607863069 >> v2:172.21.15.203:3300/0 conn(0x7f3fac15bdd0 0x7f3fac1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fa9d9b700 1 -- 172.21.15.83:0/607863069 >> v1:172.21.15.83:6789/0 conn(0x7f3fac0d1c70 legacy=0x7f3fac0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fa9d9b700 1 -- 172.21.15.83:0/607863069 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac0b25b0 con 0x7f3fac0b3370 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fa959a700 1 -- 172.21.15.83:0/607863069 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663391569 0 0) 0x7f3fac0b2030 con 0x7f3fac0d1c70 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.601+0000 7f3fa9d9b700 1 --2- 172.21.15.83:0/607863069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fac0b3370 0x7f3fac0b1af0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3fac06ec10 tx=0x7f3fa0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a14e3549a97b13 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.602+0000 7f3fa959a700 1 -- 172.21.15.83:0/607863069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa000b870 con 0x7f3fac0b3370 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.602+0000 7f3fa959a700 1 -- 172.21.15.83:0/607863069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa000b9d0 con 0x7f3fac0b3370 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.602+0000 7f3fa959a700 1 -- 172.21.15.83:0/607863069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa000bd60 con 0x7f3fac0b3370 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.602+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/607863069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fac0b3370 msgr2=0x7f3fac0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.602+0000 7f3fb0aaa700 1 --2- 172.21.15.83:0/607863069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fac0b3370 0x7f3fac0b1af0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3fac06ec10 tx=0x7f3fa0009bd0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.602+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/607863069 shutdown_connections 2023-11-26T07:17:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.602+0000 7f3fb0aaa700 1 --2- 172.21.15.83:0/607863069 >> v2:172.21.15.203:3300/0 conn(0x7f3fac15bdd0 0x7f3fac1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.602+0000 7f3fb0aaa700 1 --2- 172.21.15.83:0/607863069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fac0b3370 0x7f3fac0b1af0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.602+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/607863069 >> 172.21.15.83:0/607863069 conn(0x7f3fac0731a0 msgr2=0x7f3fac09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.602+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/607863069 shutdown_connections 2023-11-26T07:17:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.602+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/607863069 wait complete. 2023-11-26T07:17:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.604+0000 7f3fb0aaa700 1 Processor -- start 2023-11-26T07:17:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.604+0000 7f3fb0aaa700 1 -- start start 2023-11-26T07:17:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.604+0000 7f3fb0aaa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac0b3370 0x7f3fac0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.604+0000 7f3fb0aaa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fac0d1c70 0x7f3fac1e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.604+0000 7f3fb0aaa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fac15bdd0 0x7f3fac1e6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.604+0000 7f3fb0aaa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0cb8b0 con 0x7f3fac0d1c70 2023-11-26T07:17:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.604+0000 7f3fb0aaa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac150aa0 con 0x7f3fac0b3370 2023-11-26T07:17:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.604+0000 7f3fb0aaa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0caca0 con 0x7f3fac15bdd0 2023-11-26T07:17:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.604+0000 7f3fa9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fac0d1c70 0x7f3fac1e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.604+0000 7f3fa9d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fac0d1c70 0x7f3fac1e5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41724/0 (socket says 172.21.15.83:41724) 2023-11-26T07:17:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.604+0000 7f3fa9d9b700 1 -- 172.21.15.83:0/2843655124 learned_addr learned my addr 172.21.15.83:0/2843655124 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3faa59c700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac0b3370 0x7f3fac0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3faad9d700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fac15bdd0 0x7f3fac1e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3fa9d9b700 1 -- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fac15bdd0 msgr2=0x7f3fac1e6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3fa9d9b700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fac15bdd0 0x7f3fac1e6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3fa9d9b700 1 -- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac0b3370 msgr2=0x7f3fac0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3fa9d9b700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac0b3370 0x7f3fac0ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3fa9d9b700 1 -- 172.21.15.83:0/2843655124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa0009550 con 0x7f3fac0d1c70 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3faa59c700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac0b3370 0x7f3fac0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3faad9d700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fac15bdd0 0x7f3fac1e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3fa9d9b700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fac0d1c70 0x7f3fac1e5c40 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009c10 tx=0x7f3fa0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3f9affd700 1 -- 172.21.15.83:0/2843655124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0004310 con 0x7f3fac0d1c70 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3f9affd700 1 -- 172.21.15.83:0/2843655124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa001f070 con 0x7f3fac0d1c70 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3f9affd700 1 -- 172.21.15.83:0/2843655124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00036a0 con 0x7f3fac0d1c70 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/2843655124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac0caf80 con 0x7f3fac0d1c70 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.605+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/2843655124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac0cb540 con 0x7f3fac0d1c70 2023-11-26T07:17:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.606+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/2843655124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac0b4910 con 0x7f3fac0d1c70 2023-11-26T07:17:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.607+0000 7f3f9affd700 1 -- 172.21.15.83:0/2843655124 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f3fa0021030 con 0x7f3fac0d1c70 2023-11-26T07:17:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.607+0000 7f3f9affd700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3f8c06aff0 0x7f3f8c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.607+0000 7f3f9affd700 1 -- 172.21.15.83:0/2843655124 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fa008aa00 con 0x7f3fac0d1c70 2023-11-26T07:17:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.607+0000 7f3faa59c700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3f8c06aff0 0x7f3f8c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.608+0000 7f3faa59c700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3f8c06aff0 0x7f3f8c06d4b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c007a50 tx=0x7f3f9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.613 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.611+0000 7f3f9affd700 1 -- 172.21.15.83:0/2843655124 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3fac0b4910 con 0x7f3fac0d1c70 2023-11-26T07:17:08.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.844+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/2843655124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3fac0b4910 con 0x7f3fac0d1c70 2023-11-26T07:17:08.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.845+0000 7f3f9affd700 1 -- 172.21.15.83:0/2843655124 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3fac0b4910 con 0x7f3fac0d1c70 2023-11-26T07:17:08.846 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:17:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.846+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3f8c06aff0 msgr2=0x7f3f8c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.847+0000 7f3fb0aaa700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3f8c06aff0 0x7f3f8c06d4b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c007a50 tx=0x7f3f9c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.847+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/2843655124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fac0d1c70 msgr2=0x7f3fac1e5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.847+0000 7f3fb0aaa700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fac0d1c70 0x7f3fac1e5c40 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009c10 tx=0x7f3fa0004750 comp rx=0 tx=0).stop 2023-11-26T07:17:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.847+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/2843655124 shutdown_connections 2023-11-26T07:17:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.847+0000 7f3fb0aaa700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3f8c06aff0 0x7f3f8c06d4b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.847+0000 7f3fb0aaa700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fac15bdd0 0x7f3fac1e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.847+0000 7f3fb0aaa700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fac0b3370 0x7f3fac0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.847+0000 7f3fb0aaa700 1 --2- 172.21.15.83:0/2843655124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fac0d1c70 0x7f3fac1e5c40 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.847+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/2843655124 >> 172.21.15.83:0/2843655124 conn(0x7f3fac0731a0 msgr2=0x7f3fac0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.847+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/2843655124 shutdown_connections 2023-11-26T07:17:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:08.847+0000 7f3fb0aaa700 1 -- 172.21.15.83:0/2843655124 wait complete. 2023-11-26T07:17:08.860 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:17:08.860 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:17:08.860 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2023-11-26T07:17:08.860 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:17:08.860 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:17:08.860 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:17:08.860 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:17:08.861 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:17:08.861 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:17:08.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:08.861 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2023-11-26T07:17:08.878 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:17:08.878 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:08.879 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42057sec preferred_lft 42057sec 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::3a41:fab:adc5:5d6c/64 scope link noprefixroute 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::54b0:6eff:fe33:bb15/64 scope link 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether fe:46:b0:72:7b:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::fc46:b0ff:fe72:7b3f/64 scope link 2023-11-26T07:17:08.880 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:08.881 INFO:teuthology.orchestra.run.smithi083.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:17:08.881 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 7a:08:fd:1b:77:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:17:08.881 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::7808:fdff:fe1b:770c/64 scope link 2023-11-26T07:17:08.881 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:08.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:08.881 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:17:08.943 INFO:teuthology.orchestra.run.smithi083.stdout:1 2023-11-26T07:17:08.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:08.944 DEBUG:teuthology.orchestra.run.smithi083:> route 2023-11-26T07:17:09.004 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2023-11-26T07:17:09.004 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:17:09.004 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:17:09.004 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:17:09.004 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:17:09.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:09.005 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2023-11-26T07:17:09.005 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2023-11-26T07:17:09.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:17:09.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:17:09.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:17:09.005 DEBUG:teuthology.orchestra.run.smithi083:> ') 2023-11-26T07:17:09.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:09.108 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:17:09.125 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:17:09.125 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:17:09.125 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:17:09.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:09.125 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2023-11-26T07:17:09.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:09.191 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:17:09.248 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:17:09.248 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:09.248 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:17:09.304 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:17:09.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:09.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:09.389 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:09.389 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:09.416 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:09.416 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:09.417 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:17:09.417 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:17:09.417 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:17:09.417 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:17:09.417 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:17:09.417 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:17:09.494 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:09.494 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:09.494 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:17:09.494 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with new device syntax: 0@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:17:09.494 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:17:09.494 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:17:09.494 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:17:09.494 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:17:09.495 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:17:09.495 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:17:09.495 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:17:09.496 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:17:09.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:09.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:17:09.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:09.522 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:09.606 INFO:teuthology.orchestra.run.smithi083.stdout:client6790 2023-11-26T07:17:09.606 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:09.607 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:09.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:17:09.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.821+0000 7f5025042700 1 Processor -- start 2023-11-26T07:17:09.822 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.821+0000 7f5025042700 1 -- start start 2023-11-26T07:17:09.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f5025042700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f502015bdd0 0x7f50201604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f5025042700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50200b3370 0x7f50200b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f5025042700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50200b2030 con 0x7f50200d1c70 2023-11-26T07:17:09.826 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f5025042700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50200b2f50 con 0x7f502015bdd0 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f5025042700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50200b22d0 con 0x7f50200b3370 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501f59e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f50200d1c70 0x7f50200ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33546/0 (socket says 172.21.15.83:33546) 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501f59e700 1 -- 172.21.15.83:0/3669460511 learned_addr learned my addr 172.21.15.83:0/3669460511 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501ed9d700 1 --2- 172.21.15.83:0/3669460511 >> v2:172.21.15.203:3300/0 conn(0x7f502015bdd0 0x7f50201604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501e59c700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50200b3370 0x7f50200b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23250912 0 0) 0x7f50200b2030 con 0x7f50200d1c70 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50080035b0 con 0x7f50200d1c70 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3723858116 0 0) 0x7f50080035b0 con 0x7f50200d1c70 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50200b3370 msgr2=0x7f50200b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501dd9b700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50200b3370 0x7f50200b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 >> v2:172.21.15.203:3300/0 conn(0x7f502015bdd0 msgr2=0x7f50201604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:09.827 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501dd9b700 1 --2- 172.21.15.83:0/3669460511 >> v2:172.21.15.203:3300/0 conn(0x7f502015bdd0 0x7f50201604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50200b25b0 con 0x7f50200d1c70 2023-11-26T07:17:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.822+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f50180019f0 con 0x7f50200d1c70 2023-11-26T07:17:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 >> v1:172.21.15.83:6789/0 conn(0x7f50200d1c70 legacy=0x7f50200ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501dd9b700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 0x7f5008003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501dd9b700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5008004850 0x7f5008004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501dd9b700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50200b3370 0x7f5008005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200b2f50 con 0x7f5008003a00 2023-11-26T07:17:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200b22d0 con 0x7f5008004850 2023-11-26T07:17:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500800ba10 con 0x7f50200b3370 2023-11-26T07:17:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501ed9d700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 0x7f5008003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.828 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501f59e700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50200b3370 0x7f5008005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501e59c700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5008004850 0x7f5008004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501ed9d700 1 -- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50200b3370 msgr2=0x7f5008005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501ed9d700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50200b3370 0x7f5008005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501ed9d700 1 -- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5008004850 msgr2=0x7f5008004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501ed9d700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5008004850 0x7f5008004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501ed9d700 1 -- 172.21.15.83:0/3669460511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50080035b0 con 0x7f5008003a00 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501f59e700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50200b3370 0x7f5008005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501e59c700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5008004850 0x7f5008004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501ed9d700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 0x7f5008003e20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f5014009b60 tx=0x7f5008005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f62c0f4fb82a6e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501400e6c0 con 0x7f5008003a00 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501400e820 con 0x7f5008003a00 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.823+0000 7f501dd9b700 1 -- 172.21.15.83:0/3669460511 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501400eb90 con 0x7f5008003a00 2023-11-26T07:17:09.829 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.824+0000 7f5025042700 1 -- 172.21.15.83:0/3669460511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 msgr2=0x7f5008003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.824+0000 7f5025042700 1 --2- 172.21.15.83:0/3669460511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 0x7f5008003e20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f5014009b60 tx=0x7f5008005880 comp rx=0 tx=0).stop 2023-11-26T07:17:09.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.824+0000 7f501f59e700 1 -- 172.21.15.83:0/3669460511 reap_dead start 2023-11-26T07:17:09.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.824+0000 7f5025042700 1 -- 172.21.15.83:0/3669460511 shutdown_connections 2023-11-26T07:17:09.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.824+0000 7f5025042700 1 -- 172.21.15.83:0/3669460511 >> 172.21.15.83:0/3669460511 conn(0x7f50200731a0 msgr2=0x7f502009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:09.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.824+0000 7f5025042700 1 -- 172.21.15.83:0/3669460511 shutdown_connections 2023-11-26T07:17:09.830 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.824+0000 7f5025042700 1 -- 172.21.15.83:0/3669460511 wait complete. 2023-11-26T07:17:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.825+0000 7f5025042700 1 Processor -- start 2023-11-26T07:17:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.825+0000 7f5025042700 1 -- start start 2023-11-26T07:17:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.826+0000 7f5025042700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 0x7f50200c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.826+0000 7f5025042700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50201e7b60 0x7f50201f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.826+0000 7f5025042700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502015a8c0 0x7f502015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.826+0000 7f5025042700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502015b280 con 0x7f5008003a00 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.826+0000 7f5025042700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200cb340 con 0x7f50201e7b60 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.826+0000 7f5025042700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50200ca4e0 con 0x7f502015a8c0 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.826+0000 7f501ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 0x7f50200c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.826+0000 7f501ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 0x7f50200c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41746/0 (socket says 172.21.15.83:41746) 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.826+0000 7f501ed9d700 1 -- 172.21.15.83:0/2973078848 learned_addr learned my addr 172.21.15.83:0/2973078848 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f501ed9d700 1 -- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502015a8c0 msgr2=0x7f502015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f501e59c700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50201e7b60 0x7f50201f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f501ed9d700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502015a8c0 0x7f502015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f501ed9d700 1 -- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50201e7b60 msgr2=0x7f50201f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f501ed9d700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50201e7b60 0x7f50201f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f501ed9d700 1 -- 172.21.15.83:0/2973078848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50201f7320 con 0x7f5008003a00 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f501ed9d700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 0x7f50200c9fa0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f501400e4b0 tx=0x7f50140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f50077fe700 1 -- 172.21.15.83:0/2973078848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501400e6c0 con 0x7f5008003a00 2023-11-26T07:17:09.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f50077fe700 1 -- 172.21.15.83:0/2973078848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5014003ad0 con 0x7f5008003a00 2023-11-26T07:17:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f5025042700 1 -- 172.21.15.83:0/2973078848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50200ca730 con 0x7f5008003a00 2023-11-26T07:17:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f50077fe700 1 -- 172.21.15.83:0/2973078848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5014003e80 con 0x7f5008003a00 2023-11-26T07:17:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.827+0000 7f5025042700 1 -- 172.21.15.83:0/2973078848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50200cacf0 con 0x7f5008003a00 2023-11-26T07:17:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.828+0000 7f5025042700 1 -- 172.21.15.83:0/2973078848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5020067f60 con 0x7f5008003a00 2023-11-26T07:17:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.828+0000 7f50077fe700 1 -- 172.21.15.83:0/2973078848 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f5014013470 con 0x7f5008003a00 2023-11-26T07:17:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.829+0000 7f50077fe700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f500c06aff0 0x7f500c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.829+0000 7f50077fe700 1 -- 172.21.15.83:0/2973078848 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f501408af40 con 0x7f5008003a00 2023-11-26T07:17:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.829+0000 7f501e59c700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f500c06aff0 0x7f500c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.830+0000 7f501e59c700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f500c06aff0 0x7f500c06d4b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f50200c8b40 tx=0x7f5010008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:09.833+0000 7f50077fe700 1 -- 172.21.15.83:0/2973078848 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5014056090 con 0x7f5008003a00 2023-11-26T07:17:10.069 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.068+0000 7f5025042700 1 -- 172.21.15.83:0/2973078848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f50201e5400 con 0x7f5008003a00 2023-11-26T07:17:10.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.068+0000 7f50077fe700 1 -- 172.21.15.83:0/2973078848 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7f50140596b0 con 0x7f5008003a00 2023-11-26T07:17:10.070 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 -- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f500c06aff0 msgr2=0x7f500c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f500c06aff0 0x7f500c06d4b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f50200c8b40 tx=0x7f5010008040 comp rx=0 tx=0).stop 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 -- 172.21.15.83:0/2973078848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 msgr2=0x7f50200c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 0x7f50200c9fa0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f501400e4b0 tx=0x7f50140047c0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 -- 172.21.15.83:0/2973078848 shutdown_connections 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f500c06aff0 0x7f500c06d4b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f502015a8c0 0x7f502015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5008003a00 0x7f50200c9fa0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 --2- 172.21.15.83:0/2973078848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50201e7b60 0x7f50201f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 -- 172.21.15.83:0/2973078848 >> 172.21.15.83:0/2973078848 conn(0x7f50200731a0 msgr2=0x7f50200b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 -- 172.21.15.83:0/2973078848 shutdown_connections 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.070+0000 7f5025042700 1 -- 172.21.15.83:0/2973078848 wait complete. 2023-11-26T07:17:10.084 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:17:10.084 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:17:10.084 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2023-11-26T07:17:10.084 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:17:10.084 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:17:10.084 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:17:10.084 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:17:10.084 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:17:10.084 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:17:10.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:10.084 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2023-11-26T07:17:10.105 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:10.106 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:17:10.107 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:10.107 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-26T07:17:10.107 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42059sec preferred_lft 42059sec 2023-11-26T07:17:10.107 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::64b0:e04:3aa8:a78c/64 scope link noprefixroute 2023-11-26T07:17:10.107 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:10.107 INFO:teuthology.orchestra.run.smithi203.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:17:10.107 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:10.107 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:17:10.107 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:10.107 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::1092:a6ff:fe63:3da7/64 scope link 2023-11-26T07:17:10.107 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:10.108 INFO:teuthology.orchestra.run.smithi203.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:17:10.108 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether ba:8a:0c:60:2b:6f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:17:10.108 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::b88a:cff:fe60:2b6f/64 scope link 2023-11-26T07:17:10.108 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:10.108 INFO:teuthology.orchestra.run.smithi203.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:17:10.108 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether de:b3:89:ce:09:cd brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:17:10.108 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::dcb3:89ff:fece:9cd/64 scope link 2023-11-26T07:17:10.108 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:10.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:10.108 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:17:10.177 INFO:teuthology.orchestra.run.smithi203.stdout:1 2023-11-26T07:17:10.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:10.178 DEBUG:teuthology.orchestra.run.smithi203:> route 2023-11-26T07:17:10.239 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2023-11-26T07:17:10.239 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:17:10.239 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:17:10.240 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:17:10.240 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:17:10.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:10.240 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2023-11-26T07:17:10.240 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2023-11-26T07:17:10.241 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:17:10.241 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:17:10.241 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:17:10.241 DEBUG:teuthology.orchestra.run.smithi203:> ') 2023-11-26T07:17:10.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:10.350 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:17:10.367 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:17:10.367 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:17:10.368 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:17:10.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:10.368 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2023-11-26T07:17:10.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:10.434 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:17:10.491 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:17:10.491 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:10.491 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:17:10.547 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:17:10.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:10.547 DEBUG:teuthology.orchestra.run.smithi203:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:10.633 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:10.633 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:10.660 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:10.661 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:10.661 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:17:10.661 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:17:10.661 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:17:10.661 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:17:10.661 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:17:10.661 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:17:10.689 INFO:teuthology.orchestra.run.smithi203.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:10.689 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:10.689 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.203:3300,172.21.15.203:3301" 2023-11-26T07:17:10.689 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with new device syntax: 1@0aae1049-cc8c-4011-a01f-378673e57a96.cephfs=/ 2023-11-26T07:17:10.689 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.83:3300/172.21.15.203:3300/172.21.15.203:3301" will pass to kernel 2023-11-26T07:17:10.689 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: v2 mount support check returned 2 2023-11-26T07:17:10.689 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-26T07:17:10.689 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-26T07:17:10.690 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790" 2023-11-26T07:17:10.690 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.203:6789,172.21.15.203:6790:/ 2023-11-26T07:17:10.690 INFO:teuthology.orchestra.run.smithi203.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=0aae1049-cc8c-4011-a01f-378673e57a96" will pass to kernel 2023-11-26T07:17:10.692 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:17:10.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:10.692 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:17:10.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:10.721 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:10.793 INFO:teuthology.orchestra.run.smithi203.stdout:client6796 2023-11-26T07:17:10.793 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:17:10.794 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:10.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:17:10.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.982+0000 7faa41bcf700 1 Processor -- start 2023-11-26T07:17:10.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.982+0000 7faa41bcf700 1 -- start start 2023-11-26T07:17:10.989 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa41bcf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa3c0cf040 0x7faa3c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa41bcf700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faa3c0d21c0 0x7faa3c0af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa41bcf700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa3c0d01c0 con 0x7faa3c0702e0 2023-11-26T07:17:10.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa41bcf700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3c070910 con 0x7faa3c0d21c0 2023-11-26T07:17:10.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa41bcf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c070bf0 con 0x7faa3c0cf040 2023-11-26T07:17:10.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3affd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faa3c0702e0 0x7faa3c0d1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33562/0 (socket says 172.21.15.83:33562) 2023-11-26T07:17:10.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3affd700 1 -- 172.21.15.83:0/2236763505 learned_addr learned my addr 172.21.15.83:0/2236763505 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:10.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3b7fe700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa3c0cf040 0x7faa3c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3bfff700 1 --2- 172.21.15.83:0/2236763505 >> v2:172.21.15.203:3300/0 conn(0x7faa3c0d21c0 0x7faa3c0af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78919133 0 0) 0x7faa3c0d01c0 con 0x7faa3c0702e0 2023-11-26T07:17:10.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa240035b0 con 0x7faa3c0702e0 2023-11-26T07:17:10.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 146402960 0 0) 0x7faa240035b0 con 0x7faa3c0702e0 2023-11-26T07:17:10.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa3c0cf040 msgr2=0x7faa3c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:10.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3a7fc700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa3c0cf040 0x7faa3c06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 >> v2:172.21.15.203:3300/0 conn(0x7faa3c0d21c0 msgr2=0x7faa3c0af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:10.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3a7fc700 1 --2- 172.21.15.83:0/2236763505 >> v2:172.21.15.203:3300/0 conn(0x7faa3c0d21c0 0x7faa3c0af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c0aebb0 con 0x7faa3c0702e0 2023-11-26T07:17:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.983+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7faa2c001670 con 0x7faa3c0702e0 2023-11-26T07:17:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 >> v1:172.21.15.83:6789/0 conn(0x7faa3c0702e0 legacy=0x7faa3c0d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3a7fc700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa24003a00 0x7faa24003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3a7fc700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa24004850 0x7faa24004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3a7fc700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa3c0cf040 0x7faa24005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c070910 con 0x7faa24004850 2023-11-26T07:17:10.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c070bf0 con 0x7faa24003a00 2023-11-26T07:17:10.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2400ba10 con 0x7faa3c0cf040 2023-11-26T07:17:10.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3affd700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa24004850 0x7faa24004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3b7fe700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa24003a00 0x7faa24003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3bfff700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa3c0cf040 0x7faa24005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3affd700 1 -- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa3c0cf040 msgr2=0x7faa24005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:10.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3affd700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa3c0cf040 0x7faa24005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3affd700 1 -- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa24003a00 msgr2=0x7faa24003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:10.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3affd700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa24003a00 0x7faa24003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3affd700 1 -- 172.21.15.83:0/2236763505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa240035b0 con 0x7faa24004850 2023-11-26T07:17:10.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3bfff700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa3c0cf040 0x7faa24005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:10.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3affd700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa24004850 0x7faa24004cd0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00c370 tx=0x7faa24005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80483f158d2c8bd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:10.994 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c007c60 con 0x7faa24004850 2023-11-26T07:17:10.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c005960 con 0x7faa24004850 2023-11-26T07:17:10.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.984+0000 7faa3a7fc700 1 -- 172.21.15.83:0/2236763505 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c005d10 con 0x7faa24004850 2023-11-26T07:17:10.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.985+0000 7faa41bcf700 1 -- 172.21.15.83:0/2236763505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa24004850 msgr2=0x7faa24004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:10.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.985+0000 7faa41bcf700 1 --2- 172.21.15.83:0/2236763505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa24004850 0x7faa24004cd0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00c370 tx=0x7faa24005880 comp rx=0 tx=0).stop 2023-11-26T07:17:10.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.985+0000 7faa3bfff700 1 -- 172.21.15.83:0/2236763505 reap_dead start 2023-11-26T07:17:10.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.985+0000 7faa41bcf700 1 -- 172.21.15.83:0/2236763505 shutdown_connections 2023-11-26T07:17:10.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.985+0000 7faa41bcf700 1 -- 172.21.15.83:0/2236763505 >> 172.21.15.83:0/2236763505 conn(0x7faa3c09aa00 msgr2=0x7faa3c09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:10.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.985+0000 7faa41bcf700 1 -- 172.21.15.83:0/2236763505 shutdown_connections 2023-11-26T07:17:10.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.985+0000 7faa41bcf700 1 -- 172.21.15.83:0/2236763505 wait complete. 2023-11-26T07:17:10.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.986+0000 7faa41bcf700 1 Processor -- start 2023-11-26T07:17:10.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.986+0000 7faa41bcf700 1 -- start start 2023-11-26T07:17:10.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa41bcf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa24004850 0x7faa3c0c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa41bcf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa3c1564c0 0x7faa3c1f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa41bcf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa3c158370 0x7faa3c1f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa41bcf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c1557b0 con 0x7faa3c158370 2023-11-26T07:17:10.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa41bcf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c1f2360 con 0x7faa3c1564c0 2023-11-26T07:17:10.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa41bcf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c0c6500 con 0x7faa24004850 2023-11-26T07:17:10.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa3c158370 0x7faa3c1f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa3c158370 0x7faa3c1f4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41770/0 (socket says 172.21.15.83:41770) 2023-11-26T07:17:10.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3bfff700 1 -- 172.21.15.83:0/625223338 learned_addr learned my addr 172.21.15.83:0/625223338 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:10.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3b7fe700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa24004850 0x7faa3c0c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.997 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3affd700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa3c1564c0 0x7faa3c1f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3bfff700 1 -- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa24004850 msgr2=0x7faa3c0c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:10.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3bfff700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa24004850 0x7faa3c0c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3bfff700 1 -- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa3c1564c0 msgr2=0x7faa3c1f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:10.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3bfff700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa3c1564c0 0x7faa3c1f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3bfff700 1 -- 172.21.15.83:0/625223338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c0c67e0 con 0x7faa3c158370 2023-11-26T07:17:10.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3b7fe700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa24004850 0x7faa3c0c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:10.998 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3affd700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa3c1564c0 0x7faa3c1f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:10.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.987+0000 7faa3bfff700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa3c158370 0x7faa3c1f4730 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0cc1e0 tx=0x7faa340104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:10.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.988+0000 7faa23fff700 1 -- 172.21.15.83:0/625223338 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3401b070 con 0x7faa3c158370 2023-11-26T07:17:10.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.988+0000 7faa23fff700 1 -- 172.21.15.83:0/625223338 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3400f040 con 0x7faa3c158370 2023-11-26T07:17:10.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.988+0000 7faa41bcf700 1 -- 172.21.15.83:0/625223338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3c0c6ac0 con 0x7faa3c158370 2023-11-26T07:17:10.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.988+0000 7faa23fff700 1 -- 172.21.15.83:0/625223338 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3400e050 con 0x7faa3c158370 2023-11-26T07:17:10.999 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.988+0000 7faa41bcf700 1 -- 172.21.15.83:0/625223338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3c0c7080 con 0x7faa3c158370 2023-11-26T07:17:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.989+0000 7faa41bcf700 1 -- 172.21.15.83:0/625223338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3c0b2c10 con 0x7faa3c158370 2023-11-26T07:17:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.989+0000 7faa23fff700 1 -- 172.21.15.83:0/625223338 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7faa34003a20 con 0x7faa3c158370 2023-11-26T07:17:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.989+0000 7faa23fff700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faa2806aff0 0x7faa2806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:11.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.990+0000 7faa23fff700 1 -- 172.21.15.83:0/625223338 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa34089d40 con 0x7faa3c158370 2023-11-26T07:17:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.990+0000 7faa3b7fe700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faa2806aff0 0x7faa2806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.990+0000 7faa3b7fe700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faa2806aff0 0x7faa2806d4b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faa30006dd0 tx=0x7faa30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:10.994+0000 7faa23fff700 1 -- 172.21.15.83:0/625223338 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa34054b50 con 0x7faa3c158370 2023-11-26T07:17:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.230+0000 7faa41bcf700 1 -- 172.21.15.83:0/625223338 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faa3c067f60 con 0x7faa3c158370 2023-11-26T07:17:11.626 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.625+0000 7faa23fff700 1 -- 172.21.15.83:0/625223338 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 136+0+0 (secure 0 0 0) 0x7faa34058170 con 0x7faa3c158370 2023-11-26T07:17:11.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.627+0000 7faa41bcf700 1 -- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faa2806aff0 msgr2=0x7faa2806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.627+0000 7faa41bcf700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faa2806aff0 0x7faa2806d4b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faa30006dd0 tx=0x7faa30008040 comp rx=0 tx=0).stop 2023-11-26T07:17:11.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.627+0000 7faa41bcf700 1 -- 172.21.15.83:0/625223338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa3c158370 msgr2=0x7faa3c1f4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.627+0000 7faa41bcf700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa3c158370 0x7faa3c1f4730 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0cc1e0 tx=0x7faa340104a0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.627+0000 7faa41bcf700 1 -- 172.21.15.83:0/625223338 shutdown_connections 2023-11-26T07:17:11.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.627+0000 7faa41bcf700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faa2806aff0 0x7faa2806d4b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.627+0000 7faa41bcf700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa3c158370 0x7faa3c1f4730 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.627+0000 7faa41bcf700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa24004850 0x7faa3c0c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.627+0000 7faa41bcf700 1 --2- 172.21.15.83:0/625223338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa3c1564c0 0x7faa3c1f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.627+0000 7faa41bcf700 1 -- 172.21.15.83:0/625223338 >> 172.21.15.83:0/625223338 conn(0x7faa3c09aa00 msgr2=0x7faa3c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:11.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.627+0000 7faa41bcf700 1 -- 172.21.15.83:0/625223338 shutdown_connections 2023-11-26T07:17:11.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.628+0000 7faa41bcf700 1 -- 172.21.15.83:0/625223338 wait complete. 2023-11-26T07:17:11.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:17:11.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.761+0000 7fe1cb33e700 1 Processor -- start 2023-11-26T07:17:11.763 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.762+0000 7fe1cb33e700 1 -- start start 2023-11-26T07:17:11.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.762+0000 7fe1cb33e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe1c415f560 0x7fe1c40d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:11.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.762+0000 7fe1cb33e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe1c40d21f0 0x7fe1c40b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:11.765 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.762+0000 7fe1cb33e700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c40cd060 con 0x7fe1c40d0740 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.762+0000 7fe1cb33e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c415f940 con 0x7fe1c40d21f0 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.762+0000 7fe1cb33e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c40cdec0 con 0x7fe1c415f560 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1c98db700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe1c40d21f0 0x7fe1c40b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1c88d9700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe1c40d0740 0x7fe1c40d1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33582/0 (socket says 172.21.15.83:33582) 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1c88d9700 1 -- 172.21.15.83:0/3535718017 learned_addr learned my addr 172.21.15.83:0/3535718017 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1c90da700 1 --2- 172.21.15.83:0/3535718017 >> v2:172.21.15.203:3300/0 conn(0x7fe1c415f560 0x7fe1c40d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1c98db700 1 -- 172.21.15.83:0/3535718017 >> v2:172.21.15.203:3300/0 conn(0x7fe1c415f560 msgr2=0x7fe1c40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1c98db700 1 --2- 172.21.15.83:0/3535718017 >> v2:172.21.15.203:3300/0 conn(0x7fe1c415f560 0x7fe1c40d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1c98db700 1 -- 172.21.15.83:0/3535718017 >> v1:172.21.15.83:6789/0 conn(0x7fe1c40d0740 legacy=0x7fe1c40d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1c98db700 1 -- 172.21.15.83:0/3535718017 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c40b1420 con 0x7fe1c40d21f0 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1bbfff700 1 -- 172.21.15.83:0/3535718017 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1710558648 0 0) 0x7fe1c40cd060 con 0x7fe1c40d0740 2023-11-26T07:17:11.766 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1c98db700 1 --2- 172.21.15.83:0/3535718017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c40d21f0 0x7fe1c40b1b20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4072fb0 tx=0x7fe1bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13fcb9241a04763 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1bbfff700 1 -- 172.21.15.83:0/3535718017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00e070 con 0x7fe1c40d21f0 2023-11-26T07:17:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1bbfff700 1 -- 172.21.15.83:0/3535718017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc00bbd0 con 0x7fe1c40d21f0 2023-11-26T07:17:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.763+0000 7fe1bbfff700 1 -- 172.21.15.83:0/3535718017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc012430 con 0x7fe1c40d21f0 2023-11-26T07:17:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.764+0000 7fe1cb33e700 1 -- 172.21.15.83:0/3535718017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c40d21f0 msgr2=0x7fe1c40b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.764+0000 7fe1cb33e700 1 --2- 172.21.15.83:0/3535718017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c40d21f0 0x7fe1c40b1b20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4072fb0 tx=0x7fe1bc00b330 comp rx=0 tx=0).stop 2023-11-26T07:17:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.764+0000 7fe1cb33e700 1 -- 172.21.15.83:0/3535718017 shutdown_connections 2023-11-26T07:17:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.764+0000 7fe1cb33e700 1 --2- 172.21.15.83:0/3535718017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c40d21f0 0x7fe1c40b1b20 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.764+0000 7fe1cb33e700 1 --2- 172.21.15.83:0/3535718017 >> v2:172.21.15.203:3300/0 conn(0x7fe1c415f560 0x7fe1c40d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.764+0000 7fe1cb33e700 1 -- 172.21.15.83:0/3535718017 >> 172.21.15.83:0/3535718017 conn(0x7fe1c409aa00 msgr2=0x7fe1c409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.764+0000 7fe1cb33e700 1 -- 172.21.15.83:0/3535718017 shutdown_connections 2023-11-26T07:17:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.764+0000 7fe1cb33e700 1 -- 172.21.15.83:0/3535718017 wait complete. 2023-11-26T07:17:11.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.765+0000 7fe1cb33e700 1 Processor -- start 2023-11-26T07:17:11.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.765+0000 7fe1cb33e700 1 -- start start 2023-11-26T07:17:11.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1cb33e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1c40d0740 0x7fe1c4148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:11.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1cb33e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c40d21f0 0x7fe1c40c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:11.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1cb33e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1c415f560 0x7fe1c40ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:11.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1cb33e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4149460 con 0x7fe1c40d21f0 2023-11-26T07:17:11.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1cb33e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4149d10 con 0x7fe1c415f560 2023-11-26T07:17:11.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1cb33e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c41496e0 con 0x7fe1c40d0740 2023-11-26T07:17:11.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1c88d9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c40d21f0 0x7fe1c40c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:11.770 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1c88d9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c40d21f0 0x7fe1c40c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41780/0 (socket says 172.21.15.83:41780) 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1c88d9700 1 -- 172.21.15.83:0/4063977600 learned_addr learned my addr 172.21.15.83:0/4063977600 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1c98db700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1c415f560 0x7fe1c40ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1c90da700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1c40d0740 0x7fe1c4148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1c88d9700 1 -- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1c40d0740 msgr2=0x7fe1c4148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1c88d9700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1c40d0740 0x7fe1c4148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1c88d9700 1 -- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1c415f560 msgr2=0x7fe1c40ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1c88d9700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1c415f560 0x7fe1c40ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1c88d9700 1 -- 172.21.15.83:0/4063977600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc009a70 con 0x7fe1c40d21f0 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1c88d9700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c40d21f0 0x7fe1c40c7f60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c540 tx=0x7fe1c000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1b9ffb700 1 -- 172.21.15.83:0/4063977600 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0004c80 con 0x7fe1c40d21f0 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1b9ffb700 1 -- 172.21.15.83:0/4063977600 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c0004de0 con 0x7fe1c40d21f0 2023-11-26T07:17:11.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1cb33e700 1 -- 172.21.15.83:0/4063977600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c40c5b50 con 0x7fe1c40d21f0 2023-11-26T07:17:11.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.766+0000 7fe1b9ffb700 1 -- 172.21.15.83:0/4063977600 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0007310 con 0x7fe1c40d21f0 2023-11-26T07:17:11.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.767+0000 7fe1cb33e700 1 -- 172.21.15.83:0/4063977600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c41499a0 con 0x7fe1c40d21f0 2023-11-26T07:17:11.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.767+0000 7fe1cb33e700 1 -- 172.21.15.83:0/4063977600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c40cdb10 con 0x7fe1c40d21f0 2023-11-26T07:17:11.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.768+0000 7fe1b9ffb700 1 -- 172.21.15.83:0/4063977600 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fe1c0007470 con 0x7fe1c40d21f0 2023-11-26T07:17:11.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.768+0000 7fe1b9ffb700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe1a806aff0 0x7fe1a806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:11.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.768+0000 7fe1b9ffb700 1 -- 172.21.15.83:0/4063977600 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1c00894e0 con 0x7fe1c40d21f0 2023-11-26T07:17:11.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.769+0000 7fe1c90da700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe1a806aff0 0x7fe1a806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:11.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.769+0000 7fe1c90da700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe1a806aff0 0x7fe1a806d4b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe1c41f46f0 tx=0x7fe1b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:11.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.773+0000 7fe1b9ffb700 1 -- 172.21.15.83:0/4063977600 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe1c0054480 con 0x7fe1c40d21f0 2023-11-26T07:17:11.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.989+0000 7fe1cb33e700 1 -- 172.21.15.83:0/4063977600 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1c40cd870 con 0x7fe1a806aff0 2023-11-26T07:17:11.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.991+0000 7fe1b9ffb700 1 -- 172.21.15.83:0/4063977600 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe1b400b5c0 con 0x7fe1a806aff0 2023-11-26T07:17:11.993 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:17:11.993 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:17:11.993 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephfs" 2023-11-26T07:17:11.993 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:17:11.993 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:17:11.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.993+0000 7fe1cb33e700 1 -- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe1a806aff0 msgr2=0x7fe1a806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.993+0000 7fe1cb33e700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe1a806aff0 0x7fe1a806d4b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe1c41f46f0 tx=0x7fe1b4008040 comp rx=0 tx=0).stop 2023-11-26T07:17:11.995 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.993+0000 7fe1cb33e700 1 -- 172.21.15.83:0/4063977600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c40d21f0 msgr2=0x7fe1c40c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.993+0000 7fe1cb33e700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c40d21f0 0x7fe1c40c7f60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c540 tx=0x7fe1c000c900 comp rx=0 tx=0).stop 2023-11-26T07:17:11.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.993+0000 7fe1cb33e700 1 -- 172.21.15.83:0/4063977600 shutdown_connections 2023-11-26T07:17:11.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.993+0000 7fe1cb33e700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe1a806aff0 0x7fe1a806d4b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.993+0000 7fe1cb33e700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1c415f560 0x7fe1c40ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.993+0000 7fe1cb33e700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1c40d0740 0x7fe1c4148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.993+0000 7fe1cb33e700 1 --2- 172.21.15.83:0/4063977600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1c40d21f0 0x7fe1c40c7f60 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.994+0000 7fe1cb33e700 1 -- 172.21.15.83:0/4063977600 >> 172.21.15.83:0/4063977600 conn(0x7fe1c409aa00 msgr2=0x7fe1c40b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:11.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.994+0000 7fe1cb33e700 1 -- 172.21.15.83:0/4063977600 shutdown_connections 2023-11-26T07:17:11.996 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:11.994+0000 7fe1cb33e700 1 -- 172.21.15.83:0/4063977600 wait complete. 2023-11-26T07:17:12.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:17:12.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.127+0000 7fc1afade700 1 Processor -- start 2023-11-26T07:17:12.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.128+0000 7fc1afade700 1 -- start start 2023-11-26T07:17:12.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1afade700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a815bbf0 0x7fc1a815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:12.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1afade700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a815f490 con 0x7fc1a8071ac0 2023-11-26T07:17:12.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1afade700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a80b4c20 con 0x7fc1a80afae0 2023-11-26T07:17:12.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1afade700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a80b50b0 con 0x7fc1a815bbf0 2023-11-26T07:17:12.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ae07b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc1a8071ac0 0x7fc1a80ae250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33596/0 (socket says 172.21.15.83:33596) 2023-11-26T07:17:12.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ae07b700 1 -- 172.21.15.83:0/2664796772 learned_addr learned my addr 172.21.15.83:0/2664796772 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:12.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ad87a700 1 --2- 172.21.15.83:0/2664796772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a815bbf0 0x7fc1a815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:12.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ac878700 1 -- 172.21.15.83:0/2664796772 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677794062 0 0) 0x7fc1a815f490 con 0x7fc1a8071ac0 2023-11-26T07:17:12.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ac878700 1 -- 172.21.15.83:0/2664796772 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1940035b0 con 0x7fc1a8071ac0 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ac878700 1 -- 172.21.15.83:0/2664796772 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802705877 0 0) 0x7fc1a80b4c20 con 0x7fc1a80afae0 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ac878700 1 -- 172.21.15.83:0/2664796772 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a815f490 con 0x7fc1a80afae0 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ad87a700 1 -- 172.21.15.83:0/2664796772 >> v1:172.21.15.203:6789/0 conn(0x7fc1a80afae0 legacy=0x7fc1a80713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ac878700 1 -- 172.21.15.83:0/2664796772 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 889659632 0 0) 0x7fc1940035b0 con 0x7fc1a8071ac0 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ad87a700 1 -- 172.21.15.83:0/2664796772 >> v1:172.21.15.83:6789/0 conn(0x7fc1a8071ac0 legacy=0x7fc1a80ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ad87a700 1 -- 172.21.15.83:0/2664796772 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a80b4010 con 0x7fc1a815bbf0 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.129+0000 7fc1ad87a700 1 --2- 172.21.15.83:0/2664796772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a815bbf0 0x7fc1a815ef50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc198009870 tx=0x7fc198009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f189b19ac7b371b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.130+0000 7fc1ac878700 1 -- 172.21.15.83:0/2664796772 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00019f0 con 0x7fc1a815bbf0 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.130+0000 7fc1ac878700 1 -- 172.21.15.83:0/2664796772 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc198004030 con 0x7fc1a815bbf0 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.130+0000 7fc1ac878700 1 -- 172.21.15.83:0/2664796772 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980043c0 con 0x7fc1a815bbf0 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.130+0000 7fc1afade700 1 -- 172.21.15.83:0/2664796772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a815bbf0 msgr2=0x7fc1a815ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:12.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.130+0000 7fc1afade700 1 --2- 172.21.15.83:0/2664796772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a815bbf0 0x7fc1a815ef50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc198009870 tx=0x7fc198009b80 comp rx=0 tx=0).stop 2023-11-26T07:17:12.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.130+0000 7fc1afade700 1 -- 172.21.15.83:0/2664796772 shutdown_connections 2023-11-26T07:17:12.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.130+0000 7fc1afade700 1 --2- 172.21.15.83:0/2664796772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a815bbf0 0x7fc1a815ef50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.130+0000 7fc1afade700 1 -- 172.21.15.83:0/2664796772 >> 172.21.15.83:0/2664796772 conn(0x7fc1a8074170 msgr2=0x7fc1a806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:12.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.130+0000 7fc1afade700 1 -- 172.21.15.83:0/2664796772 shutdown_connections 2023-11-26T07:17:12.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.130+0000 7fc1afade700 1 -- 172.21.15.83:0/2664796772 wait complete. 2023-11-26T07:17:12.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.131+0000 7fc1afade700 1 Processor -- start 2023-11-26T07:17:12.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.132+0000 7fc1afade700 1 -- start start 2023-11-26T07:17:12.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.132+0000 7fc1afade700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1a8071ac0 0x7fc1a80b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:12.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.132+0000 7fc1afade700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a80afae0 0x7fc1a80cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:12.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.132+0000 7fc1afade700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1a815bbf0 0x7fc1a8145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:12.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.132+0000 7fc1afade700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a80b1040 con 0x7fc1a8071ac0 2023-11-26T07:17:12.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.132+0000 7fc1afade700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a80b0060 con 0x7fc1a815bbf0 2023-11-26T07:17:12.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.132+0000 7fc1afade700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a80ce0b0 con 0x7fc1a80afae0 2023-11-26T07:17:12.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.132+0000 7fc1ad87a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1a8071ac0 0x7fc1a80b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:12.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.132+0000 7fc1ad87a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1a8071ac0 0x7fc1a80b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41792/0 (socket says 172.21.15.83:41792) 2023-11-26T07:17:12.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.132+0000 7fc1ad87a700 1 -- 172.21.15.83:0/482171356 learned_addr learned my addr 172.21.15.83:0/482171356 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:12.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1ad079700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a80afae0 0x7fc1a80cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:12.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1ae07b700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1a815bbf0 0x7fc1a8145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:12.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1ad87a700 1 -- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a80afae0 msgr2=0x7fc1a80cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:12.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1ad87a700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a80afae0 0x7fc1a80cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1ad87a700 1 -- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1a815bbf0 msgr2=0x7fc1a8145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:12.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1ad87a700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1a815bbf0 0x7fc1a8145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1ad87a700 1 -- 172.21.15.83:0/482171356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1980094a0 con 0x7fc1a8071ac0 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1ad079700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a80afae0 0x7fc1a80cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1ae07b700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1a815bbf0 0x7fc1a8145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1ad87a700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1a8071ac0 0x7fc1a80b0b00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fc198000c00 tx=0x7fc198004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc19e7fc700 1 -- 172.21.15.83:0/482171356 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198003e30 con 0x7fc1a8071ac0 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc19e7fc700 1 -- 172.21.15.83:0/482171356 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19801f070 con 0x7fc1a8071ac0 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1afade700 1 -- 172.21.15.83:0/482171356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a80b0300 con 0x7fc1a8071ac0 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc19e7fc700 1 -- 172.21.15.83:0/482171356 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198011870 con 0x7fc1a8071ac0 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.133+0000 7fc1afade700 1 -- 172.21.15.83:0/482171356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a80cdda0 con 0x7fc1a8071ac0 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.134+0000 7fc1afade700 1 -- 172.21.15.83:0/482171356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a80b0950 con 0x7fc1a8071ac0 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.134+0000 7fc19e7fc700 1 -- 172.21.15.83:0/482171356 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fc198021030 con 0x7fc1a8071ac0 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.135+0000 7fc19e7fc700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc18c06aff0 0x7fc18c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.135+0000 7fc19e7fc700 1 -- 172.21.15.83:0/482171356 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc19808abe0 con 0x7fc1a8071ac0 2023-11-26T07:17:12.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.135+0000 7fc1ad079700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc18c06aff0 0x7fc18c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:12.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.136+0000 7fc1ad079700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc18c06aff0 0x7fc18c06d4b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc1a814ca60 tx=0x7fc1a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:12.140 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.139+0000 7fc19e7fc700 1 -- 172.21.15.83:0/482171356 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc198055d30 con 0x7fc1a8071ac0 2023-11-26T07:17:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.329+0000 7fc1afade700 1 -- 172.21.15.83:0/482171356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc1a8068fb0 con 0x7fc1a8071ac0 2023-11-26T07:17:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.330+0000 7fc19e7fc700 1 -- 172.21.15.83:0/482171356 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc198059350 con 0x7fc1a8071ac0 2023-11-26T07:17:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 -- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc18c06aff0 msgr2=0x7fc18c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc18c06aff0 0x7fc18c06d4b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc1a814ca60 tx=0x7fc1a4009040 comp rx=0 tx=0).stop 2023-11-26T07:17:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 -- 172.21.15.83:0/482171356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1a8071ac0 msgr2=0x7fc1a80b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1a8071ac0 0x7fc1a80b0b00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fc198000c00 tx=0x7fc198004750 comp rx=0 tx=0).stop 2023-11-26T07:17:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 -- 172.21.15.83:0/482171356 shutdown_connections 2023-11-26T07:17:12.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fc18c06aff0 0x7fc18c06d4b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1a815bbf0 0x7fc1a8145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1a8071ac0 0x7fc1a80b0b00 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 --2- 172.21.15.83:0/482171356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc1a80afae0 0x7fc1a80cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 -- 172.21.15.83:0/482171356 >> 172.21.15.83:0/482171356 conn(0x7fc1a8074170 msgr2=0x7fc1a81595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:12.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 -- 172.21.15.83:0/482171356 shutdown_connections 2023-11-26T07:17:12.335 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.332+0000 7fc1afade700 1 -- 172.21.15.83:0/482171356 wait complete. 2023-11-26T07:17:12.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:12.346 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070) 2023-11-26T07:17:12.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:12.378 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/fake) 2023-11-26T07:17:12.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:12.468 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-11-26T07:17:12.489 INFO:teuthology.orchestra.run.smithi083.stdout:cephfs_data 2023-11-26T07:17:12.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:12.489 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070) 2023-11-26T07:17:12.559 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:12.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol77760/.meta 2023-11-26T07:17:12.629 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol77760 2023-11-26T07:17:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.791+0000 7f2a98e1c700 1 Processor -- start 2023-11-26T07:17:12.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.791+0000 7f2a98e1c700 1 -- start start 2023-11-26T07:17:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a98e1c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a9415e080 0x7f2a9406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a98e1c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2a940ce430 0x7f2a940ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a98e1c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9406ba50 con 0x7f2a940b4310 2023-11-26T07:17:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a98e1c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a940d1200 con 0x7f2a940ce430 2023-11-26T07:17:12.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a98e1c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940cee20 con 0x7f2a9415e080 2023-11-26T07:17:12.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a92d9d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2a940ce430 0x7f2a940ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:12.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a9259c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a9415e080 0x7f2a9406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a91d9b700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2a940b4310 0x7f2a940d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:46458/0 (socket says 172.21.15.83:46458) 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a91d9b700 1 -- 172.21.15.83:0/2157387337 learned_addr learned my addr 172.21.15.83:0/2157387337 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a9159a700 1 -- 172.21.15.83:0/2157387337 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753062400 0 0) 0x7f2a9406ba50 con 0x7f2a940b4310 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a9159a700 1 -- 172.21.15.83:0/2157387337 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a7c0035b0 con 0x7f2a940b4310 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a92d9d700 1 -- 172.21.15.83:0/2157387337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a9415e080 msgr2=0x7f2a9406bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a92d9d700 1 --2- 172.21.15.83:0/2157387337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a9415e080 0x7f2a9406bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a92d9d700 1 -- 172.21.15.83:0/2157387337 >> v1:172.21.15.203:6789/0 conn(0x7f2a940b4310 legacy=0x7f2a940d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a92d9d700 1 -- 172.21.15.83:0/2157387337 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9406b120 con 0x7f2a940ce430 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.792+0000 7f2a9259c700 1 --2- 172.21.15.83:0/2157387337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a9415e080 0x7f2a9406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.793+0000 7f2a92d9d700 1 --2- 172.21.15.83:0/2157387337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a940ce430 0x7f2a940ce8e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009d90 tx=0x7f2a8c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ecbeb53c05e6386 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.793+0000 7f2a9159a700 1 -- 172.21.15.83:0/2157387337 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00e070 con 0x7f2a940ce430 2023-11-26T07:17:12.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.793+0000 7f2a9159a700 1 -- 172.21.15.83:0/2157387337 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c00bb20 con 0x7f2a940ce430 2023-11-26T07:17:12.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.793+0000 7f2a9159a700 1 -- 172.21.15.83:0/2157387337 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00be90 con 0x7f2a940ce430 2023-11-26T07:17:12.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.793+0000 7f2a98e1c700 1 -- 172.21.15.83:0/2157387337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a940ce430 msgr2=0x7f2a940ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:12.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.793+0000 7f2a98e1c700 1 --2- 172.21.15.83:0/2157387337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a940ce430 0x7f2a940ce8e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009d90 tx=0x7f2a8c00b330 comp rx=0 tx=0).stop 2023-11-26T07:17:12.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.793+0000 7f2a98e1c700 1 -- 172.21.15.83:0/2157387337 shutdown_connections 2023-11-26T07:17:12.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.793+0000 7f2a98e1c700 1 --2- 172.21.15.83:0/2157387337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a940ce430 0x7f2a940ce8e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.793+0000 7f2a98e1c700 1 --2- 172.21.15.83:0/2157387337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a9415e080 0x7f2a9406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.793+0000 7f2a98e1c700 1 -- 172.21.15.83:0/2157387337 >> 172.21.15.83:0/2157387337 conn(0x7f2a940b3780 msgr2=0x7f2a940b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:12.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.793+0000 7f2a98e1c700 1 -- 172.21.15.83:0/2157387337 shutdown_connections 2023-11-26T07:17:12.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.794+0000 7f2a98e1c700 1 -- 172.21.15.83:0/2157387337 wait complete. 2023-11-26T07:17:12.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.795+0000 7f2a98e1c700 1 Processor -- start 2023-11-26T07:17:12.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.795+0000 7f2a98e1c700 1 -- start start 2023-11-26T07:17:12.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.795+0000 7f2a98e1c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a940b4310 0x7f2a94160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a98e1c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a940ce430 0x7f2a940b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a98e1c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a9415e080 0x7f2a940c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a98e1c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940c56e0 con 0x7f2a9415e080 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a98e1c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940c59c0 con 0x7f2a940b4310 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a98e1c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940c5ca0 con 0x7f2a940ce430 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a92d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a9415e080 0x7f2a940c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a92d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a9415e080 0x7f2a940c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41810/0 (socket says 172.21.15.83:41810) 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a92d9d700 1 -- 172.21.15.83:0/4215907300 learned_addr learned my addr 172.21.15.83:0/4215907300 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a91d9b700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a940ce430 0x7f2a940b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a9259c700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a940b4310 0x7f2a94160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a92d9d700 1 -- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a940ce430 msgr2=0x7f2a940b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:12.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a92d9d700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a940ce430 0x7f2a940b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a92d9d700 1 -- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a940b4310 msgr2=0x7f2a94160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a92d9d700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a940b4310 0x7f2a94160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a92d9d700 1 -- 172.21.15.83:0/4215907300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a840094a0 con 0x7f2a9415e080 2023-11-26T07:17:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a92d9d700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a9415e080 0x7f2a940c6760 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00b370 tx=0x7f2a8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a82ffd700 1 -- 172.21.15.83:0/4215907300 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c0048c0 con 0x7f2a9415e080 2023-11-26T07:17:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a82ffd700 1 -- 172.21.15.83:0/4215907300 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c020070 con 0x7f2a9415e080 2023-11-26T07:17:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a82ffd700 1 -- 172.21.15.83:0/4215907300 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c0276c0 con 0x7f2a9415e080 2023-11-26T07:17:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a98e1c700 1 -- 172.21.15.83:0/4215907300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c009a70 con 0x7f2a9415e080 2023-11-26T07:17:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.796+0000 7f2a98e1c700 1 -- 172.21.15.83:0/4215907300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a94161220 con 0x7f2a9415e080 2023-11-26T07:17:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.797+0000 7f2a98e1c700 1 -- 172.21.15.83:0/4215907300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a94161410 con 0x7f2a9415e080 2023-11-26T07:17:12.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.798+0000 7f2a82ffd700 1 -- 172.21.15.83:0/4215907300 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f2a8c012430 con 0x7f2a9415e080 2023-11-26T07:17:12.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.798+0000 7f2a82ffd700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2a7406aff0 0x7f2a7406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:12.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.798+0000 7f2a82ffd700 1 -- 172.21.15.83:0/4215907300 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a8c022070 con 0x7f2a9415e080 2023-11-26T07:17:12.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.799+0000 7f2a9259c700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2a7406aff0 0x7f2a7406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:12.802 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.799+0000 7f2a9259c700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2a7406aff0 0x7f2a7406d4b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009cd0 tx=0x7f2a84009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:12.804 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:12.803+0000 7f2a82ffd700 1 -- 172.21.15.83:0/4215907300 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a94161410 con 0x7f2a9415e080 2023-11-26T07:17:13.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.028+0000 7f2a98e1c700 1 -- 172.21.15.83:0/4215907300 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol77760", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a9406b6e0 con 0x7f2a7406aff0 2023-11-26T07:17:13.035 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:13.033+0000 7f30f3727700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:13.035 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:13.033+0000 7f30f3727700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:13.035 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:13.033+0000 7f30f3727700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:13.035 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:13.033+0000 7f30f3727700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:13.035 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:13.033+0000 7f30f3727700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.074+0000 7f2a82ffd700 1 -- 172.21.15.83:0/4215907300 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7f2a84009920 con 0x7f2a7406aff0 2023-11-26T07:17:13.076 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070 2023-11-26T07:17:13.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 -- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2a7406aff0 msgr2=0x7f2a7406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2a7406aff0 0x7f2a7406d4b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009cd0 tx=0x7f2a84009210 comp rx=0 tx=0).stop 2023-11-26T07:17:13.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 -- 172.21.15.83:0/4215907300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a9415e080 msgr2=0x7f2a940c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a9415e080 0x7f2a940c6760 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00b370 tx=0x7f2a8c004ad0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 -- 172.21.15.83:0/4215907300 shutdown_connections 2023-11-26T07:17:13.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2a7406aff0 0x7f2a7406d4b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a9415e080 0x7f2a940c6760 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a940b4310 0x7f2a94160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 --2- 172.21.15.83:0/4215907300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a940ce430 0x7f2a940b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 -- 172.21.15.83:0/4215907300 >> 172.21.15.83:0/4215907300 conn(0x7f2a940b3780 msgr2=0x7f2a9406bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 -- 172.21.15.83:0/4215907300 shutdown_connections 2023-11-26T07:17:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.076+0000 7f2a98e1c700 1 -- 172.21.15.83:0/4215907300 wait complete. 2023-11-26T07:17:13.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2023-11-26T07:17:13.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.209+0000 7f4b77d99700 1 Processor -- start 2023-11-26T07:17:13.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.209+0000 7f4b77d99700 1 -- start start 2023-11-26T07:17:13.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b77d99700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4b7015e160 0x7f4b700b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b77d99700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b700afc90 0x7f4b70073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b77d99700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b700b3c60 con 0x7f4b700744d0 2023-11-26T07:17:13.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b77d99700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b700b4500 con 0x7f4b7015e160 2023-11-26T07:17:13.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b77d99700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b70074960 con 0x7f4b700afc90 2023-11-26T07:17:13.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b75334700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b700afc90 0x7f4b70073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b75b35700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4b7015e160 0x7f4b700b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b76336700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4b700744d0 0x7f4b7006d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:46476/0 (socket says 172.21.15.83:46476) 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b76336700 1 -- 172.21.15.83:0/1867798931 learned_addr learned my addr 172.21.15.83:0/1867798931 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b74b33700 1 -- 172.21.15.83:0/1867798931 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857376865 0 0) 0x7f4b700b3c60 con 0x7f4b700744d0 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b74b33700 1 -- 172.21.15.83:0/1867798931 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b5c0035b0 con 0x7f4b700744d0 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b75334700 1 -- 172.21.15.83:0/1867798931 >> v2:172.21.15.203:3300/0 conn(0x7f4b7015e160 msgr2=0x7f4b700b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b75334700 1 --2- 172.21.15.83:0/1867798931 >> v2:172.21.15.203:3300/0 conn(0x7f4b7015e160 0x7f4b700b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b75334700 1 -- 172.21.15.83:0/1867798931 >> v1:172.21.15.203:6789/0 conn(0x7f4b700744d0 legacy=0x7f4b7006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.210+0000 7f4b75334700 1 -- 172.21.15.83:0/1867798931 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700b58c0 con 0x7f4b700afc90 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.211+0000 7f4b75334700 1 --2- 172.21.15.83:0/1867798931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b700afc90 0x7f4b70073f90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009840 tx=0x7f4b60009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8c277b5eb7b8751 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.211+0000 7f4b74b33700 1 -- 172.21.15.83:0/1867798931 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60011830 con 0x7f4b700afc90 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.211+0000 7f4b74b33700 1 -- 172.21.15.83:0/1867798931 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b60011990 con 0x7f4b700afc90 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.211+0000 7f4b74b33700 1 -- 172.21.15.83:0/1867798931 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60011d00 con 0x7f4b700afc90 2023-11-26T07:17:13.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.211+0000 7f4b77d99700 1 -- 172.21.15.83:0/1867798931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b700afc90 msgr2=0x7f4b70073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.211+0000 7f4b77d99700 1 --2- 172.21.15.83:0/1867798931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b700afc90 0x7f4b70073f90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009840 tx=0x7f4b60009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:13.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.211+0000 7f4b77d99700 1 -- 172.21.15.83:0/1867798931 shutdown_connections 2023-11-26T07:17:13.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.211+0000 7f4b77d99700 1 --2- 172.21.15.83:0/1867798931 >> v2:172.21.15.203:3300/0 conn(0x7f4b7015e160 0x7f4b700b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.211+0000 7f4b77d99700 1 --2- 172.21.15.83:0/1867798931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b700afc90 0x7f4b70073f90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.211+0000 7f4b77d99700 1 -- 172.21.15.83:0/1867798931 >> 172.21.15.83:0/1867798931 conn(0x7f4b700d0bf0 msgr2=0x7f4b700d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:13.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.211+0000 7f4b77d99700 1 -- 172.21.15.83:0/1867798931 shutdown_connections 2023-11-26T07:17:13.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.212+0000 7f4b77d99700 1 -- 172.21.15.83:0/1867798931 wait complete. 2023-11-26T07:17:13.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.213+0000 7f4b77d99700 1 Processor -- start 2023-11-26T07:17:13.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.213+0000 7f4b77d99700 1 -- start start 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.213+0000 7f4b77d99700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700744d0 0x7f4b701f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.213+0000 7f4b77d99700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b700afc90 0x7f4b70162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.213+0000 7f4b77d99700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7015e160 0x7f4b700c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.213+0000 7f4b77d99700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b701f6dc0 con 0x7f4b700744d0 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.213+0000 7f4b77d99700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b701f57e0 con 0x7f4b700afc90 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.213+0000 7f4b77d99700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b701f5ac0 con 0x7f4b7015e160 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b75b35700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700744d0 0x7f4b701f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b75b35700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700744d0 0x7f4b701f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41820/0 (socket says 172.21.15.83:41820) 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b75b35700 1 -- 172.21.15.83:0/1165188111 learned_addr learned my addr 172.21.15.83:0/1165188111 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b75334700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b700afc90 0x7f4b70162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b76336700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7015e160 0x7f4b700c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b75b35700 1 -- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7015e160 msgr2=0x7f4b700c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b75b35700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7015e160 0x7f4b700c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b75b35700 1 -- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b700afc90 msgr2=0x7f4b70162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b75b35700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b700afc90 0x7f4b70162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b75b35700 1 -- 172.21.15.83:0/1165188111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600094a0 con 0x7f4b700744d0 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b75b35700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700744d0 0x7f4b701f6880 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00e870 tx=0x7f4b6c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b667fc700 1 -- 172.21.15.83:0/1165188111 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c009590 con 0x7f4b700744d0 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b667fc700 1 -- 172.21.15.83:0/1165188111 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c013680 con 0x7f4b700744d0 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b77d99700 1 -- 172.21.15.83:0/1165188111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b701f5da0 con 0x7f4b700744d0 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b667fc700 1 -- 172.21.15.83:0/1165188111 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0139f0 con 0x7f4b700744d0 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.214+0000 7f4b77d99700 1 -- 172.21.15.83:0/1165188111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b701f6360 con 0x7f4b700744d0 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.215+0000 7f4b77d99700 1 -- 172.21.15.83:0/1165188111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b70068fb0 con 0x7f4b700744d0 2023-11-26T07:17:13.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.216+0000 7f4b667fc700 1 -- 172.21.15.83:0/1165188111 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f4b6c013b50 con 0x7f4b700744d0 2023-11-26T07:17:13.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.216+0000 7f4b667fc700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4b5406aff0 0x7f4b5406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.216+0000 7f4b667fc700 1 -- 172.21.15.83:0/1165188111 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b6c089570 con 0x7f4b700744d0 2023-11-26T07:17:13.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.216+0000 7f4b75334700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4b5406aff0 0x7f4b5406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.217+0000 7f4b75334700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4b5406aff0 0x7f4b5406d4b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009fd0 tx=0x7f4b60008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:13.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.220+0000 7f4b667fc700 1 -- 172.21.15.83:0/1165188111 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b6c054510 con 0x7f4b700744d0 2023-11-26T07:17:13.440 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.438+0000 7f4b77d99700 1 -- 172.21.15.83:0/1165188111 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b700b4190 con 0x7f4b5406aff0 2023-11-26T07:17:13.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.443+0000 7f4b667fc700 1 -- 172.21.15.83:0/1165188111 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f4b60011b00 con 0x7f4b5406aff0 2023-11-26T07:17:13.444 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:17:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:17:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subvol77760" 2023-11-26T07:17:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:17:13.445 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:17:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 -- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4b5406aff0 msgr2=0x7f4b5406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4b5406aff0 0x7f4b5406d4b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009fd0 tx=0x7f4b60008030 comp rx=0 tx=0).stop 2023-11-26T07:17:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 -- 172.21.15.83:0/1165188111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700744d0 msgr2=0x7f4b701f6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700744d0 0x7f4b701f6880 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00e870 tx=0x7f4b6c00eb80 comp rx=0 tx=0).stop 2023-11-26T07:17:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 -- 172.21.15.83:0/1165188111 shutdown_connections 2023-11-26T07:17:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4b5406aff0 0x7f4b5406d4b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4b7015e160 0x7f4b700c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b700744d0 0x7f4b701f6880 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 --2- 172.21.15.83:0/1165188111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4b700afc90 0x7f4b70162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 -- 172.21.15.83:0/1165188111 >> 172.21.15.83:0/1165188111 conn(0x7f4b700d0bf0 msgr2=0x7f4b7015bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 -- 172.21.15.83:0/1165188111 shutdown_connections 2023-11-26T07:17:13.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.445+0000 7f4b77d99700 1 -- 172.21.15.83:0/1165188111 wait complete. 2023-11-26T07:17:13.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol77760 2023-11-26T07:17:13.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.579+0000 7fa253687700 1 Processor -- start 2023-11-26T07:17:13.581 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.580+0000 7fa253687700 1 -- start start 2023-11-26T07:17:13.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.580+0000 7fa253687700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa24c0b3bf0 0x7fa24c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.583 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa253687700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa24c0744d0 0x7fa24c06b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa253687700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa24c145ad0 con 0x7fa24c1456f0 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa253687700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa24c074960 con 0x7fa24c0744d0 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa253687700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c0b4070 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa251c24700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa24c0744d0 0x7fa24c06b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa251c24700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa24c0744d0 0x7fa24c06b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41832/0 (socket says 172.21.15.83:41832) 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa251c24700 1 -- 172.21.15.83:0/404321390 learned_addr learned my addr 172.21.15.83:0/404321390 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa250c22700 1 --2- 172.21.15.83:0/404321390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa24c0b3bf0 0x7fa24c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa251c24700 1 -- 172.21.15.83:0/404321390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa24c0b3bf0 msgr2=0x7fa24c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa251c24700 1 --2- 172.21.15.83:0/404321390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa24c0b3bf0 0x7fa24c073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa251c24700 1 -- 172.21.15.83:0/404321390 >> v1:172.21.15.203:6789/0 conn(0x7fa24c1456f0 legacy=0x7fa24c15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa251c24700 1 -- 172.21.15.83:0/404321390 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c15fd10 con 0x7fa24c0744d0 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa251c24700 1 --2- 172.21.15.83:0/404321390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa24c0744d0 0x7fa24c06b2d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fa244009e50 tx=0x7fa24400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7a6188b949ae8b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:13.584 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa243fff700 1 -- 172.21.15.83:0/404321390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24400e070 con 0x7fa24c0744d0 2023-11-26T07:17:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa243fff700 1 -- 172.21.15.83:0/404321390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa244003a20 con 0x7fa24c0744d0 2023-11-26T07:17:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.581+0000 7fa243fff700 1 -- 172.21.15.83:0/404321390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244003dd0 con 0x7fa24c0744d0 2023-11-26T07:17:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.582+0000 7fa253687700 1 -- 172.21.15.83:0/404321390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa24c0744d0 msgr2=0x7fa24c06b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.582+0000 7fa253687700 1 --2- 172.21.15.83:0/404321390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa24c0744d0 0x7fa24c06b2d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fa244009e50 tx=0x7fa24400b330 comp rx=0 tx=0).stop 2023-11-26T07:17:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.582+0000 7fa253687700 1 -- 172.21.15.83:0/404321390 shutdown_connections 2023-11-26T07:17:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.582+0000 7fa253687700 1 --2- 172.21.15.83:0/404321390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa24c0744d0 0x7fa24c06b2d0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.582+0000 7fa253687700 1 --2- 172.21.15.83:0/404321390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa24c0b3bf0 0x7fa24c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.582+0000 7fa253687700 1 -- 172.21.15.83:0/404321390 >> 172.21.15.83:0/404321390 conn(0x7fa24c0b28f0 msgr2=0x7fa24c0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.582+0000 7fa253687700 1 -- 172.21.15.83:0/404321390 shutdown_connections 2023-11-26T07:17:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.582+0000 7fa253687700 1 -- 172.21.15.83:0/404321390 wait complete. 2023-11-26T07:17:13.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.583+0000 7fa253687700 1 Processor -- start 2023-11-26T07:17:13.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.583+0000 7fa253687700 1 -- start start 2023-11-26T07:17:13.587 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa253687700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa24c0744d0 0x7fa24c13fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa253687700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa24c0b3bf0 0x7fa24c1f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa253687700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa24c1456f0 0x7fa24c13a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa253687700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c1f1fd0 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa253687700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c1f2280 con 0x7fa24c1456f0 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa253687700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c1f2560 con 0x7fa24c0744d0 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa250c22700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa24c0b3bf0 0x7fa24c1f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa250c22700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa24c0b3bf0 0x7fa24c1f1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41836/0 (socket says 172.21.15.83:41836) 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa250c22700 1 -- 172.21.15.83:0/3543526354 learned_addr learned my addr 172.21.15.83:0/3543526354 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa251423700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa24c0744d0 0x7fa24c13fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa251c24700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa24c1456f0 0x7fa24c13a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa250c22700 1 -- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa24c0744d0 msgr2=0x7fa24c13fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa250c22700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa24c0744d0 0x7fa24c13fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa250c22700 1 -- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa24c1456f0 msgr2=0x7fa24c13a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa250c22700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa24c1456f0 0x7fa24c13a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa250c22700 1 -- 172.21.15.83:0/3543526354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa244009b30 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.584+0000 7fa250c22700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa24c0b3bf0 0x7fa24c1f1a90 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa24800e810 tx=0x7fa24800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.585+0000 7fa241ffb700 1 -- 172.21.15.83:0/3543526354 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248013610 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.585+0000 7fa241ffb700 1 -- 172.21.15.83:0/3543526354 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa248013770 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.585+0000 7fa241ffb700 1 -- 172.21.15.83:0/3543526354 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248013b20 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.585+0000 7fa253687700 1 -- 172.21.15.83:0/3543526354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24c1f2840 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.585+0000 7fa253687700 1 -- 172.21.15.83:0/3543526354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24c13aa60 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.586+0000 7fa253687700 1 -- 172.21.15.83:0/3543526354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24c1f2a80 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.586+0000 7fa241ffb700 1 -- 172.21.15.83:0/3543526354 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fa248013c80 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.586+0000 7fa241ffb700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa23006aff0 0x7fa23006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.587+0000 7fa241ffb700 1 -- 172.21.15.83:0/3543526354 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa248089510 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.587+0000 7fa251423700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa23006aff0 0x7fa23006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.587+0000 7fa251423700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa23006aff0 0x7fa23006d4b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa24c15d900 tx=0x7fa23c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:13.592 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.591+0000 7fa241ffb700 1 -- 172.21.15.83:0/3543526354 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa2480543b0 con 0x7fa24c0b3bf0 2023-11-26T07:17:13.815 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.814+0000 7fa253687700 1 -- 172.21.15.83:0/3543526354 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol77760", "target": ["mon-mgr", ""]}) v1 -- 0x7fa24c15d970 con 0x7fa23006aff0 2023-11-26T07:17:13.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.839+0000 7fa241ffb700 1 -- 172.21.15.83:0/3543526354 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+669 (secure 0 0 0) 0x7fa23c00a3e0 con 0x7fa23006aff0 2023-11-26T07:17:13.840 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "atime": "2023-11-26 07:17:12", 2023-11-26T07:17:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_pcent": "undefined", 2023-11-26T07:17:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_quota": "infinite", 2023-11-26T07:17:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_used": 0, 2023-11-26T07:17:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "created_at": "2023-11-26 07:17:12", 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "ctime": "2023-11-26 07:17:13", 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "data_pool": "cephfs_data", 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "features": [ 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot-clone", 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot-autoprotect" 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 0, 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": 16877, 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_addrs": [ 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6789", 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.203:6789", 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.203:6790" 2023-11-26T07:17:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:17:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "mtime": "2023-11-26 07:17:12", 2023-11-26T07:17:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070", 2023-11-26T07:17:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_namespace": "", 2023-11-26T07:17:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "complete", 2023-11-26T07:17:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "subvolume", 2023-11-26T07:17:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "uid": 0 2023-11-26T07:17:13.842 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:13.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 -- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa23006aff0 msgr2=0x7fa23006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa23006aff0 0x7fa23006d4b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa24c15d900 tx=0x7fa23c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:13.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 -- 172.21.15.83:0/3543526354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa24c0b3bf0 msgr2=0x7fa24c1f1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa24c0b3bf0 0x7fa24c1f1a90 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa24800e810 tx=0x7fa24800ebd0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 -- 172.21.15.83:0/3543526354 shutdown_connections 2023-11-26T07:17:13.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa23006aff0 0x7fa23006d4b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa24c1456f0 0x7fa24c13a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa24c0744d0 0x7fa24c13fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 --2- 172.21.15.83:0/3543526354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa24c0b3bf0 0x7fa24c1f1a90 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 -- 172.21.15.83:0/3543526354 >> 172.21.15.83:0/3543526354 conn(0x7fa24c0b28f0 msgr2=0x7fa24c0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:13.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 -- 172.21.15.83:0/3543526354 shutdown_connections 2023-11-26T07:17:13.844 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.841+0000 7fa253687700 1 -- 172.21.15.83:0/3543526354 wait complete. 2023-11-26T07:17:13.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol77760 10485760 2023-11-26T07:17:13.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.977+0000 7ff872dfe700 1 Processor -- start 2023-11-26T07:17:13.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.977+0000 7ff872dfe700 1 -- start start 2023-11-26T07:17:13.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.978+0000 7ff872dfe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff86c155780 0x7ff86c15c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.978+0000 7ff872dfe700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff86c06dea0 con 0x7ff86c0cf100 2023-11-26T07:17:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.978+0000 7ff872dfe700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff86c06e140 con 0x7ff86c06eee0 2023-11-26T07:17:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.978+0000 7ff872dfe700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c06dbd0 con 0x7ff86c155780 2023-11-26T07:17:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.978+0000 7ff87139b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff86c0cf100 0x7ff86c15c9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33654/0 (socket says 172.21.15.83:33654) 2023-11-26T07:17:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.978+0000 7ff87139b700 1 -- 172.21.15.83:0/808160490 learned_addr learned my addr 172.21.15.83:0/808160490 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.978+0000 7ff870b9a700 1 --2- 172.21.15.83:0/808160490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff86c155780 0x7ff86c15c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff86b7fe700 1 -- 172.21.15.83:0/808160490 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687267815 0 0) 0x7ff86c06dea0 con 0x7ff86c0cf100 2023-11-26T07:17:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff86b7fe700 1 -- 172.21.15.83:0/808160490 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff85c0035b0 con 0x7ff86c0cf100 2023-11-26T07:17:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff86b7fe700 1 -- 172.21.15.83:0/808160490 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061756403 0 0) 0x7ff86c06e140 con 0x7ff86c06eee0 2023-11-26T07:17:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff86b7fe700 1 -- 172.21.15.83:0/808160490 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff86c06dea0 con 0x7ff86c06eee0 2023-11-26T07:17:13.981 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff870b9a700 1 -- 172.21.15.83:0/808160490 >> v1:172.21.15.203:6789/0 conn(0x7ff86c06eee0 legacy=0x7ff86c0ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff870b9a700 1 -- 172.21.15.83:0/808160490 >> v1:172.21.15.83:6789/0 conn(0x7ff86c0cf100 legacy=0x7ff86c15c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff86b7fe700 1 -- 172.21.15.83:0/808160490 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3857782082 0 0) 0x7ff85c0035b0 con 0x7ff86c0cf100 2023-11-26T07:17:13.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff870b9a700 1 -- 172.21.15.83:0/808160490 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c06e400 con 0x7ff86c155780 2023-11-26T07:17:13.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff870b9a700 1 --2- 172.21.15.83:0/808160490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff86c155780 0x7ff86c15c210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff860009870 tx=0x7ff860009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6f92cd8a5fd6eba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:13.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff86b7fe700 1 -- 172.21.15.83:0/808160490 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640019f0 con 0x7ff86c155780 2023-11-26T07:17:13.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff86b7fe700 1 -- 172.21.15.83:0/808160490 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff860004030 con 0x7ff86c155780 2023-11-26T07:17:13.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.979+0000 7ff86b7fe700 1 -- 172.21.15.83:0/808160490 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8600043a0 con 0x7ff86c155780 2023-11-26T07:17:13.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.980+0000 7ff872dfe700 1 -- 172.21.15.83:0/808160490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff86c155780 msgr2=0x7ff86c15c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.984 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.980+0000 7ff872dfe700 1 --2- 172.21.15.83:0/808160490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff86c155780 0x7ff86c15c210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff860009870 tx=0x7ff860009b80 comp rx=0 tx=0).stop 2023-11-26T07:17:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.980+0000 7ff872dfe700 1 -- 172.21.15.83:0/808160490 shutdown_connections 2023-11-26T07:17:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.980+0000 7ff872dfe700 1 --2- 172.21.15.83:0/808160490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff86c155780 0x7ff86c15c210 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.980+0000 7ff872dfe700 1 -- 172.21.15.83:0/808160490 >> 172.21.15.83:0/808160490 conn(0x7ff86c0d0af0 msgr2=0x7ff86c0d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.980+0000 7ff872dfe700 1 -- 172.21.15.83:0/808160490 shutdown_connections 2023-11-26T07:17:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.980+0000 7ff872dfe700 1 -- 172.21.15.83:0/808160490 wait complete. 2023-11-26T07:17:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.981+0000 7ff872dfe700 1 Processor -- start 2023-11-26T07:17:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.981+0000 7ff872dfe700 1 -- start start 2023-11-26T07:17:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff872dfe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff86c06eee0 0x7ff86c155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff872dfe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff86c0cf100 0x7ff86c1f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff872dfe700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff86c155780 0x7ff86c1f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.985 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff872dfe700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c15df30 con 0x7ff86c06eee0 2023-11-26T07:17:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff872dfe700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c1437e0 con 0x7ff86c0cf100 2023-11-26T07:17:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff872dfe700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c143280 con 0x7ff86c155780 2023-11-26T07:17:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff870b9a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff86c06eee0 0x7ff86c155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff870b9a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff86c06eee0 0x7ff86c155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41850/0 (socket says 172.21.15.83:41850) 2023-11-26T07:17:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff870b9a700 1 -- 172.21.15.83:0/3106037858 learned_addr learned my addr 172.21.15.83:0/3106037858 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff86bfff700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff86c0cf100 0x7ff86c1f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff87139b700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff86c155780 0x7ff86c1f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff870b9a700 1 -- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff86c155780 msgr2=0x7ff86c1f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff870b9a700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff86c155780 0x7ff86c1f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff870b9a700 1 -- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff86c0cf100 msgr2=0x7ff86c1f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:13.986 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff870b9a700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff86c0cf100 0x7ff86c1f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:13.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff870b9a700 1 -- 172.21.15.83:0/3106037858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8600094a0 con 0x7ff86c06eee0 2023-11-26T07:17:13.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.982+0000 7ff870b9a700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff86c06eee0 0x7ff86c155b60 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ff8600090b0 tx=0x7ff860004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:13.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.983+0000 7ff8697fa700 1 -- 172.21.15.83:0/3106037858 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff860003ad0 con 0x7ff86c06eee0 2023-11-26T07:17:13.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.983+0000 7ff8697fa700 1 -- 172.21.15.83:0/3106037858 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86001f070 con 0x7ff86c06eee0 2023-11-26T07:17:13.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.983+0000 7ff872dfe700 1 -- 172.21.15.83:0/3106037858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86c158610 con 0x7ff86c06eee0 2023-11-26T07:17:13.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.983+0000 7ff8697fa700 1 -- 172.21.15.83:0/3106037858 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff860003e90 con 0x7ff86c06eee0 2023-11-26T07:17:13.987 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.983+0000 7ff872dfe700 1 -- 172.21.15.83:0/3106037858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86c15dbc0 con 0x7ff86c06eee0 2023-11-26T07:17:13.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.983+0000 7ff872dfe700 1 -- 172.21.15.83:0/3106037858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86c067f60 con 0x7ff86c06eee0 2023-11-26T07:17:13.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.984+0000 7ff8697fa700 1 -- 172.21.15.83:0/3106037858 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7ff860021030 con 0x7ff86c06eee0 2023-11-26T07:17:13.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.984+0000 7ff8697fa700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff85006aff0 0x7ff85006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:13.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.984+0000 7ff8697fa700 1 -- 172.21.15.83:0/3106037858 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff86008a920 con 0x7ff86c06eee0 2023-11-26T07:17:13.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.985+0000 7ff86bfff700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff85006aff0 0x7ff85006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:13.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.985+0000 7ff86bfff700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff85006aff0 0x7ff85006d4b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff86c15eb10 tx=0x7ff858009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:13.990 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:13.989+0000 7ff8697fa700 1 -- 172.21.15.83:0/3106037858 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff860055950 con 0x7ff86c06eee0 2023-11-26T07:17:14.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.219+0000 7ff872dfe700 1 -- 172.21.15.83:0/3106037858 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol77760", "new_size": "10485760", "target": ["mon-mgr", ""]}) v1 -- 0x7ff86c15c750 con 0x7ff85006aff0 2023-11-26T07:17:14.247 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.246+0000 7ff8697fa700 1 -- 172.21.15.83:0/3106037858 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+128 (secure 0 0 0) 0x7ff858001550 con 0x7ff85006aff0 2023-11-26T07:17:14.247 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2023-11-26T07:17:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:17:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_used": 0 2023-11-26T07:17:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:17:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:17:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_quota": 10485760 2023-11-26T07:17:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2023-11-26T07:17:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:17:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_pcent": "0.00" 2023-11-26T07:17:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:17:14.248 INFO:teuthology.orchestra.run.smithi083.stdout:] 2023-11-26T07:17:14.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 -- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff85006aff0 msgr2=0x7ff85006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff85006aff0 0x7ff85006d4b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff86c15eb10 tx=0x7ff858009040 comp rx=0 tx=0).stop 2023-11-26T07:17:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 -- 172.21.15.83:0/3106037858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff86c06eee0 msgr2=0x7ff86c155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff86c06eee0 0x7ff86c155b60 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ff8600090b0 tx=0x7ff860004750 comp rx=0 tx=0).stop 2023-11-26T07:17:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 -- 172.21.15.83:0/3106037858 shutdown_connections 2023-11-26T07:17:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff85006aff0 0x7ff85006d4b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff86c155780 0x7ff86c1f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff86c06eee0 0x7ff86c155b60 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 --2- 172.21.15.83:0/3106037858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff86c0cf100 0x7ff86c1f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 -- 172.21.15.83:0/3106037858 >> 172.21.15.83:0/3106037858 conn(0x7ff86c0d0af0 msgr2=0x7ff86c0b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 -- 172.21.15.83:0/3106037858 shutdown_connections 2023-11-26T07:17:14.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.248+0000 7ff872dfe700 1 -- 172.21.15.83:0/3106037858 wait complete. 2023-11-26T07:17:14.261 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol77760 2023-11-26T07:17:14.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.382+0000 7f20a6c77700 1 Processor -- start 2023-11-26T07:17:14.383 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.382+0000 7f20a6c77700 1 -- start start 2023-11-26T07:17:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a6c77700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20a015bf50 0x7f20a00ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a6c77700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f20a00c6c30 0x7f20a00cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a6c77700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a00d0140 con 0x7f20a00ce830 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a6c77700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a015c330 con 0x7f20a015bf50 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a6c77700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a00cf3c0 con 0x7f20a00c6c30 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a4a13700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20a015bf50 0x7f20a00ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a4a13700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20a015bf50 0x7f20a00ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41856/0 (socket says 172.21.15.83:41856) 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a4a13700 1 -- 172.21.15.83:0/603516777 learned_addr learned my addr 172.21.15.83:0/603516777 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a5214700 1 --2- 172.21.15.83:0/603516777 >> v2:172.21.15.203:3300/0 conn(0x7f20a00c6c30 0x7f20a00cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a4a13700 1 -- 172.21.15.83:0/603516777 >> v2:172.21.15.203:3300/0 conn(0x7f20a00c6c30 msgr2=0x7f20a00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a4a13700 1 --2- 172.21.15.83:0/603516777 >> v2:172.21.15.203:3300/0 conn(0x7f20a00c6c30 0x7f20a00cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a4a13700 1 -- 172.21.15.83:0/603516777 >> v1:172.21.15.203:6789/0 conn(0x7f20a00ce830 legacy=0x7f20a00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a4a13700 1 -- 172.21.15.83:0/603516777 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00d03e0 con 0x7f20a015bf50 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f20a4a13700 1 --2- 172.21.15.83:0/603516777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20a015bf50 0x7f20a00ce2f0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f20a00c8c00 tx=0x7f2094009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17aa12a45eb3eafa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f209f7fe700 1 -- 172.21.15.83:0/603516777 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209400b870 con 0x7f20a015bf50 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f209f7fe700 1 -- 172.21.15.83:0/603516777 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209400b9d0 con 0x7f20a015bf50 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.383+0000 7f209f7fe700 1 -- 172.21.15.83:0/603516777 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209400bd60 con 0x7f20a015bf50 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.384+0000 7f20a6c77700 1 -- 172.21.15.83:0/603516777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20a015bf50 msgr2=0x7f20a00ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.384+0000 7f20a6c77700 1 --2- 172.21.15.83:0/603516777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20a015bf50 0x7f20a00ce2f0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f20a00c8c00 tx=0x7f2094009bd0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.384+0000 7f20a6c77700 1 -- 172.21.15.83:0/603516777 shutdown_connections 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.384+0000 7f20a6c77700 1 --2- 172.21.15.83:0/603516777 >> v2:172.21.15.203:3300/0 conn(0x7f20a00c6c30 0x7f20a00cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.384+0000 7f20a6c77700 1 --2- 172.21.15.83:0/603516777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20a015bf50 0x7f20a00ce2f0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.384+0000 7f20a6c77700 1 -- 172.21.15.83:0/603516777 >> 172.21.15.83:0/603516777 conn(0x7f20a00b3780 msgr2=0x7f20a00b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.384+0000 7f20a6c77700 1 -- 172.21.15.83:0/603516777 shutdown_connections 2023-11-26T07:17:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.384+0000 7f20a6c77700 1 -- 172.21.15.83:0/603516777 wait complete. 2023-11-26T07:17:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.385+0000 7f20a6c77700 1 Processor -- start 2023-11-26T07:17:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.385+0000 7f20a6c77700 1 -- start start 2023-11-26T07:17:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.386+0000 7f20a6c77700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20a00c6c30 0x7f20a00cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.386+0000 7f20a6c77700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20a00ce830 0x7f20a0149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.386+0000 7f20a6c77700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20a015bf50 0x7f20a01466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.386+0000 7f20a6c77700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a014a440 con 0x7f20a00ce830 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.386+0000 7f20a6c77700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01478d0 con 0x7f20a015bf50 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.386+0000 7f20a6c77700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0149e80 con 0x7f20a00c6c30 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.386+0000 7f209ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20a00ce830 0x7f20a0149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.386+0000 7f209ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20a00ce830 0x7f20a0149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41864/0 (socket says 172.21.15.83:41864) 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.386+0000 7f209ffff700 1 -- 172.21.15.83:0/1959200508 learned_addr learned my addr 172.21.15.83:0/1959200508 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.386+0000 7f20a5214700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20a015bf50 0x7f20a01466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.386+0000 7f20a4a13700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20a00c6c30 0x7f20a00cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.387+0000 7f209ffff700 1 -- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20a00c6c30 msgr2=0x7f20a00cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.387+0000 7f209ffff700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20a00c6c30 0x7f20a00cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.387+0000 7f209ffff700 1 -- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20a015bf50 msgr2=0x7f20a01466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.387+0000 7f209ffff700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20a015bf50 0x7f20a01466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.387+0000 7f209ffff700 1 -- 172.21.15.83:0/1959200508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2094009550 con 0x7f20a00ce830 2023-11-26T07:17:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.387+0000 7f209ffff700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20a00ce830 0x7f20a0149470 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f208c00c510 tx=0x7f208c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.387+0000 7f209d7fa700 1 -- 172.21.15.83:0/1959200508 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c0052d0 con 0x7f20a00ce830 2023-11-26T07:17:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.387+0000 7f209d7fa700 1 -- 172.21.15.83:0/1959200508 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c005430 con 0x7f20a00ce830 2023-11-26T07:17:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.387+0000 7f20a6c77700 1 -- 172.21.15.83:0/1959200508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a014a160 con 0x7f20a00ce830 2023-11-26T07:17:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.387+0000 7f209d7fa700 1 -- 172.21.15.83:0/1959200508 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c0038c0 con 0x7f20a00ce830 2023-11-26T07:17:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.387+0000 7f20a6c77700 1 -- 172.21.15.83:0/1959200508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a014d660 con 0x7f20a00ce830 2023-11-26T07:17:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.388+0000 7f20a6c77700 1 -- 172.21.15.83:0/1959200508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a014d820 con 0x7f20a00ce830 2023-11-26T07:17:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.388+0000 7f209d7fa700 1 -- 172.21.15.83:0/1959200508 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f208c003a20 con 0x7f20a00ce830 2023-11-26T07:17:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.389+0000 7f209d7fa700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f208406aff0 0x7f208406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.389+0000 7f209d7fa700 1 -- 172.21.15.83:0/1959200508 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f208c088a00 con 0x7f20a00ce830 2023-11-26T07:17:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.389+0000 7f20a4a13700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f208406aff0 0x7f208406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.390+0000 7f20a4a13700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f208406aff0 0x7f208406d4b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2094014040 tx=0x7f2094008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:14.395 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.393+0000 7f209d7fa700 1 -- 172.21.15.83:0/1959200508 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f208c053a30 con 0x7f20a00ce830 2023-11-26T07:17:14.617 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.616+0000 7f20a6c77700 1 -- 172.21.15.83:0/1959200508 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol77760", "target": ["mon-mgr", ""]}) v1 -- 0x7f20a00c5700 con 0x7f208406aff0 2023-11-26T07:17:14.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.639+0000 7f209d7fa700 1 -- 172.21.15.83:0/1959200508 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+662 (secure 0 0 0) 0x7f209400bb40 con 0x7f208406aff0 2023-11-26T07:17:14.640 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:14.640 INFO:teuthology.orchestra.run.smithi083.stdout: "atime": "2023-11-26 07:17:12", 2023-11-26T07:17:14.640 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_pcent": "0.00", 2023-11-26T07:17:14.640 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_quota": 10485760, 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_used": 0, 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "created_at": "2023-11-26 07:17:12", 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "ctime": "2023-11-26 07:17:14", 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "data_pool": "cephfs_data", 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "features": [ 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot-clone", 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot-autoprotect" 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 0, 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": 16877, 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_addrs": [ 2023-11-26T07:17:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6789", 2023-11-26T07:17:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.203:6789", 2023-11-26T07:17:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.203:6790" 2023-11-26T07:17:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:17:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: "mtime": "2023-11-26 07:17:12", 2023-11-26T07:17:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070", 2023-11-26T07:17:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_namespace": "", 2023-11-26T07:17:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "complete", 2023-11-26T07:17:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "subvolume", 2023-11-26T07:17:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: "uid": 0 2023-11-26T07:17:14.642 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:14.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 -- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f208406aff0 msgr2=0x7f208406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f208406aff0 0x7f208406d4b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2094014040 tx=0x7f2094008030 comp rx=0 tx=0).stop 2023-11-26T07:17:14.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 -- 172.21.15.83:0/1959200508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20a00ce830 msgr2=0x7f20a0149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.643 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20a00ce830 0x7f20a0149470 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f208c00c510 tx=0x7f208c00c820 comp rx=0 tx=0).stop 2023-11-26T07:17:14.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 -- 172.21.15.83:0/1959200508 shutdown_connections 2023-11-26T07:17:14.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f208406aff0 0x7f208406d4b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20a015bf50 0x7f20a01466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20a00c6c30 0x7f20a00cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 --2- 172.21.15.83:0/1959200508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20a00ce830 0x7f20a0149470 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 -- 172.21.15.83:0/1959200508 >> 172.21.15.83:0/1959200508 conn(0x7f20a00b3780 msgr2=0x7f20a0070620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:14.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 -- 172.21.15.83:0/1959200508 shutdown_connections 2023-11-26T07:17:14.644 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.642+0000 7f20a6c77700 1 -- 172.21.15.83:0/1959200508 wait complete. 2023-11-26T07:17:14.655 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol77760 --mode 777 --uid 1000 --gid 1000 2023-11-26T07:17:14.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.783+0000 7f3bc7dae700 1 Processor -- start 2023-11-26T07:17:14.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.784+0000 7f3bc7dae700 1 -- start start 2023-11-26T07:17:14.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.784+0000 7f3bc7dae700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3bc00cf610 0x7f3bc00add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.784+0000 7f3bc7dae700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bc015f9e0 0x7f3bc00b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.787 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.784+0000 7f3bc7dae700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc006b080 con 0x7f3bc0073f90 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.784+0000 7f3bc7dae700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc015c290 con 0x7f3bc015f9e0 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.784+0000 7f3bc7dae700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc00cf9f0 con 0x7f3bc00cf610 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.784+0000 7f3bc5349700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bc015f9e0 0x7f3bc00b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc634b700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f3bc0073f90 0x7f3bc006b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33682/0 (socket says 172.21.15.83:33682) 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc634b700 1 -- 172.21.15.83:0/1681900684 learned_addr learned my addr 172.21.15.83:0/1681900684 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc5349700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3bc015f9e0 0x7f3bc00b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41872/0 (socket says 172.21.15.83:41872) 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc5b4a700 1 --2- 172.21.15.83:0/1681900684 >> v2:172.21.15.203:3300/0 conn(0x7f3bc00cf610 0x7f3bc00add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc5349700 1 -- 172.21.15.83:0/1681900684 >> v2:172.21.15.203:3300/0 conn(0x7f3bc00cf610 msgr2=0x7f3bc00add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc5349700 1 --2- 172.21.15.83:0/1681900684 >> v2:172.21.15.203:3300/0 conn(0x7f3bc00cf610 0x7f3bc00add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc5349700 1 -- 172.21.15.83:0/1681900684 >> v1:172.21.15.83:6789/0 conn(0x7f3bc0073f90 legacy=0x7f3bc006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc5349700 1 -- 172.21.15.83:0/1681900684 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc015ff80 con 0x7f3bc015f9e0 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc4b48700 1 -- 172.21.15.83:0/1681900684 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562553204 0 0) 0x7f3bc006b080 con 0x7f3bc0073f90 2023-11-26T07:17:14.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc5b4a700 1 --2- 172.21.15.83:0/1681900684 >> v2:172.21.15.203:3300/0 conn(0x7f3bc00cf610 0x7f3bc00add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc5349700 1 --2- 172.21.15.83:0/1681900684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc015f9e0 0x7f3bc00b3c60 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009840 tx=0x7f3bb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c91f3d011f5a1bc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc4b48700 1 -- 172.21.15.83:0/1681900684 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0011830 con 0x7f3bc015f9e0 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc4b48700 1 -- 172.21.15.83:0/1681900684 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb0011990 con 0x7f3bc015f9e0 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.785+0000 7f3bc4b48700 1 -- 172.21.15.83:0/1681900684 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0011d20 con 0x7f3bc015f9e0 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.786+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1681900684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc015f9e0 msgr2=0x7f3bc00b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.786+0000 7f3bc7dae700 1 --2- 172.21.15.83:0/1681900684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc015f9e0 0x7f3bc00b3c60 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009840 tx=0x7f3bb0009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.786+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1681900684 shutdown_connections 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.786+0000 7f3bc7dae700 1 --2- 172.21.15.83:0/1681900684 >> v2:172.21.15.203:3300/0 conn(0x7f3bc00cf610 0x7f3bc00add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.786+0000 7f3bc7dae700 1 --2- 172.21.15.83:0/1681900684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc015f9e0 0x7f3bc00b3c60 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.786+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1681900684 >> 172.21.15.83:0/1681900684 conn(0x7f3bc00d0bf0 msgr2=0x7f3bc00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.786+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1681900684 shutdown_connections 2023-11-26T07:17:14.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.786+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1681900684 wait complete. 2023-11-26T07:17:14.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.787+0000 7f3bc7dae700 1 Processor -- start 2023-11-26T07:17:14.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.787+0000 7f3bc7dae700 1 -- start start 2023-11-26T07:17:14.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc7dae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc0073f90 0x7f3bc014a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc7dae700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc00cf610 0x7f3bc015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc7dae700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc015f9e0 0x7f3bc015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc7dae700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc006dbd0 con 0x7f3bc015f9e0 2023-11-26T07:17:14.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc7dae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc014ff00 con 0x7f3bc0073f90 2023-11-26T07:17:14.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc7dae700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc01501e0 con 0x7f3bc00cf610 2023-11-26T07:17:14.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc634b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc015f9e0 0x7f3bc015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc634b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc015f9e0 0x7f3bc015c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41880/0 (socket says 172.21.15.83:41880) 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc634b700 1 -- 172.21.15.83:0/1551892214 learned_addr learned my addr 172.21.15.83:0/1551892214 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc5349700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc00cf610 0x7f3bc015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc5b4a700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc0073f90 0x7f3bc014a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc634b700 1 -- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc00cf610 msgr2=0x7f3bc015c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc634b700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc00cf610 0x7f3bc015c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc634b700 1 -- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc0073f90 msgr2=0x7f3bc014a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc634b700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc0073f90 0x7f3bc014a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc634b700 1 -- 172.21.15.83:0/1551892214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb00094a0 con 0x7f3bc015f9e0 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc5349700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc00cf610 0x7f3bc015c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc5b4a700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc0073f90 0x7f3bc014a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc634b700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc015f9e0 0x7f3bc015c9a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00c410 tx=0x7f3bbc00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:14.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bb67fc700 1 -- 172.21.15.83:0/1551892214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc004c80 con 0x7f3bc015f9e0 2023-11-26T07:17:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bb67fc700 1 -- 172.21.15.83:0/1551892214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc004de0 con 0x7f3bc015f9e0 2023-11-26T07:17:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1551892214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc01504c0 con 0x7f3bc015f9e0 2023-11-26T07:17:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.788+0000 7f3bb67fc700 1 -- 172.21.15.83:0/1551892214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0072f0 con 0x7f3bc015f9e0 2023-11-26T07:17:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.789+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1551892214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc014ae30 con 0x7f3bc015f9e0 2023-11-26T07:17:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.789+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1551892214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc006d930 con 0x7f3bc015f9e0 2023-11-26T07:17:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.790+0000 7f3bb67fc700 1 -- 172.21.15.83:0/1551892214 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f3bbc007450 con 0x7f3bc015f9e0 2023-11-26T07:17:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.790+0000 7f3bb67fc700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ba406aff0 0x7f3ba406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.790+0000 7f3bb67fc700 1 -- 172.21.15.83:0/1551892214 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bbc089300 con 0x7f3bc015f9e0 2023-11-26T07:17:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.790+0000 7f3bc5b4a700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ba406aff0 0x7f3ba406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.791+0000 7f3bc5b4a700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ba406aff0 0x7f3ba406d4b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80094c0 tx=0x7f3bb800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:14.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:14.795+0000 7f3bb67fc700 1 -- 172.21.15.83:0/1551892214 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3bc006d930 con 0x7f3bc015f9e0 2023-11-26T07:17:15.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.014+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1551892214 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol77760", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) v1 -- 0x7f3bc0160820 con 0x7f3ba406aff0 2023-11-26T07:17:15.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.041+0000 7f3bb67fc700 1 -- 172.21.15.83:0/1551892214 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3bb800f040 con 0x7f3ba406aff0 2023-11-26T07:17:15.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ba406aff0 msgr2=0x7f3ba406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:15.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ba406aff0 0x7f3ba406d4b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80094c0 tx=0x7f3bb800b040 comp rx=0 tx=0).stop 2023-11-26T07:17:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1551892214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc015f9e0 msgr2=0x7f3bc015c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc015f9e0 0x7f3bc015c9a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00c410 tx=0x7f3bbc00c7d0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1551892214 shutdown_connections 2023-11-26T07:17:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3ba406aff0 0x7f3ba406d4b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bc015f9e0 0x7f3bc015c9a0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc0073f90 0x7f3bc014a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 --2- 172.21.15.83:0/1551892214 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc00cf610 0x7f3bc015c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1551892214 >> 172.21.15.83:0/1551892214 conn(0x7f3bc00d0bf0 msgr2=0x7f3bc015b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1551892214 shutdown_connections 2023-11-26T07:17:15.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.044+0000 7f3bc7dae700 1 -- 172.21.15.83:0/1551892214 wait complete. 2023-11-26T07:17:15.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol77760 2023-11-26T07:17:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.178+0000 7f95aefb0700 1 Processor -- start 2023-11-26T07:17:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.178+0000 7f95aefb0700 1 -- start start 2023-11-26T07:17:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95aefb0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f95a815e080 0x7f95a806bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95aefb0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a80ce430 0x7f95a80ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95aefb0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a806ba50 con 0x7f95a80b4310 2023-11-26T07:17:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95aefb0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a80d1200 con 0x7f95a815e080 2023-11-26T07:17:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95aefb0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a815e460 con 0x7f95a80ce430 2023-11-26T07:17:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95a7fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f95a80b4310 0x7f95a80d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33692/0 (socket says 172.21.15.83:33692) 2023-11-26T07:17:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95a7fff700 1 -- 172.21.15.83:0/2763978670 learned_addr learned my addr 172.21.15.83:0/2763978670 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648878397 0 0) 0x7f95a806ba50 con 0x7f95a80b4310 2023-11-26T07:17:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95980035b0 con 0x7f95a80b4310 2023-11-26T07:17:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95ad54d700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a80ce430 0x7f95a80ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95acd4c700 1 --2- 172.21.15.83:0/2763978670 >> v2:172.21.15.203:3300/0 conn(0x7f95a815e080 0x7f95a806bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 428826948 0 0) 0x7f95980035b0 con 0x7f95a80b4310 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a80ce430 msgr2=0x7f95a80ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95a77fe700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a80ce430 0x7f95a80ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 >> v2:172.21.15.203:3300/0 conn(0x7f95a815e080 msgr2=0x7f95a806bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95a77fe700 1 --2- 172.21.15.83:0/2763978670 >> v2:172.21.15.203:3300/0 conn(0x7f95a815e080 0x7f95a806bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.179+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a806b120 con 0x7f95a80b4310 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f9594001670 con 0x7f95a80b4310 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 >> v1:172.21.15.83:6789/0 conn(0x7f95a80b4310 legacy=0x7f95a80d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a77fe700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 0x7f9598003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a77fe700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a80ce430 0x7f9598004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a77fe700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9598004d90 0x7f9598005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80d1200 con 0x7f9598003a00 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959800ba10 con 0x7f9598004d90 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959800bb50 con 0x7f95a80ce430 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95acd4c700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 0x7f9598003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a7fff700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a80ce430 0x7f9598004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95ad54d700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9598004d90 0x7f9598005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95acd4c700 1 -- 172.21.15.83:0/2763978670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a80ce430 msgr2=0x7f9598004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95acd4c700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a80ce430 0x7f9598004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95acd4c700 1 -- 172.21.15.83:0/2763978670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9598004d90 msgr2=0x7f9598005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95acd4c700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9598004d90 0x7f9598005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95acd4c700 1 -- 172.21.15.83:0/2763978670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95980035b0 con 0x7f9598003a00 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95acd4c700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 0x7f9598003e20 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f959c009b60 tx=0x7f9598005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb56aebf629766f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00e6c0 con 0x7f9598003a00 2023-11-26T07:17:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00e820 con 0x7f9598003a00 2023-11-26T07:17:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.180+0000 7f95a77fe700 1 -- 172.21.15.83:0/2763978670 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00eb90 con 0x7f9598003a00 2023-11-26T07:17:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.181+0000 7f95aefb0700 1 -- 172.21.15.83:0/2763978670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 msgr2=0x7f9598003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.181+0000 7f95aefb0700 1 --2- 172.21.15.83:0/2763978670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 0x7f9598003e20 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f959c009b60 tx=0x7f9598005880 comp rx=0 tx=0).stop 2023-11-26T07:17:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.181+0000 7f95ad54d700 1 -- 172.21.15.83:0/2763978670 reap_dead start 2023-11-26T07:17:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.181+0000 7f95aefb0700 1 -- 172.21.15.83:0/2763978670 shutdown_connections 2023-11-26T07:17:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.181+0000 7f95aefb0700 1 -- 172.21.15.83:0/2763978670 >> 172.21.15.83:0/2763978670 conn(0x7f95a80b3780 msgr2=0x7f95a80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.181+0000 7f95aefb0700 1 -- 172.21.15.83:0/2763978670 shutdown_connections 2023-11-26T07:17:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.181+0000 7f95aefb0700 1 -- 172.21.15.83:0/2763978670 wait complete. 2023-11-26T07:17:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.182+0000 7f95aefb0700 1 Processor -- start 2023-11-26T07:17:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.183+0000 7f95aefb0700 1 -- start start 2023-11-26T07:17:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95aefb0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 0x7f95a80c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95aefb0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95a80b75b0 0x7f95a814cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95aefb0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a8157a40 0x7f95a80afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95aefb0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80c7630 con 0x7f9598003a00 2023-11-26T07:17:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95aefb0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80c6630 con 0x7f95a80b75b0 2023-11-26T07:17:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95aefb0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80c69b0 con 0x7f95a8157a40 2023-11-26T07:17:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95acd4c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 0x7f95a80c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95acd4c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 0x7f95a80c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41894/0 (socket says 172.21.15.83:41894) 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95acd4c700 1 -- 172.21.15.83:0/421577164 learned_addr learned my addr 172.21.15.83:0/421577164 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95ad54d700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a8157a40 0x7f95a80afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95a7fff700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95a80b75b0 0x7f95a814cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95acd4c700 1 -- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a8157a40 msgr2=0x7f95a80afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95acd4c700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a8157a40 0x7f95a80afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95acd4c700 1 -- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95a80b75b0 msgr2=0x7f95a814cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95acd4c700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95a80b75b0 0x7f95a814cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95acd4c700 1 -- 172.21.15.83:0/421577164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80c6c90 con 0x7f9598003a00 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95a7fff700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95a80b75b0 0x7f95a814cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95acd4c700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 0x7f95a80c60f0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f9598004730 tx=0x7f959c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95a57fa700 1 -- 172.21.15.83:0/421577164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00e6c0 con 0x7f9598003a00 2023-11-26T07:17:15.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95a57fa700 1 -- 172.21.15.83:0/421577164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c003ab0 con 0x7f9598003a00 2023-11-26T07:17:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95aefb0700 1 -- 172.21.15.83:0/421577164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a80c6f70 con 0x7f9598003a00 2023-11-26T07:17:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95a57fa700 1 -- 172.21.15.83:0/421577164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c003e60 con 0x7f9598003a00 2023-11-26T07:17:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.184+0000 7f95aefb0700 1 -- 172.21.15.83:0/421577164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a814d4d0 con 0x7f9598003a00 2023-11-26T07:17:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.185+0000 7f95aefb0700 1 -- 172.21.15.83:0/421577164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a8068fb0 con 0x7f9598003a00 2023-11-26T07:17:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.186+0000 7f95a57fa700 1 -- 172.21.15.83:0/421577164 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f959c013470 con 0x7f9598003a00 2023-11-26T07:17:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.186+0000 7f95a57fa700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f958c06b020 0x7f958c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.186+0000 7f95a57fa700 1 -- 172.21.15.83:0/421577164 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f959c08b1a0 con 0x7f9598003a00 2023-11-26T07:17:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.186+0000 7f95a7fff700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f958c06b020 0x7f958c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.187+0000 7f95a7fff700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f958c06b020 0x7f958c06d4e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9594000c00 tx=0x7f9594009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.190+0000 7f95a57fa700 1 -- 172.21.15.83:0/421577164 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f959c0562f0 con 0x7f9598003a00 2023-11-26T07:17:15.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.413+0000 7f95aefb0700 1 -- 172.21.15.83:0/421577164 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol77760", "target": ["mon-mgr", ""]}) v1 -- 0x7f95a806b6e0 con 0x7f958c06b020 2023-11-26T07:17:15.441 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.439+0000 7f95a57fa700 1 -- 172.21.15.83:0/421577164 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7f9594007d80 con 0x7f958c06b020 2023-11-26T07:17:15.441 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070 2023-11-26T07:17:15.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.441+0000 7f95aefb0700 1 -- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f958c06b020 msgr2=0x7f958c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:15.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.441+0000 7f95aefb0700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f958c06b020 0x7f958c06d4e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9594000c00 tx=0x7f9594009040 comp rx=0 tx=0).stop 2023-11-26T07:17:15.443 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.441+0000 7f95aefb0700 1 -- 172.21.15.83:0/421577164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 msgr2=0x7f95a80c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.441+0000 7f95aefb0700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 0x7f95a80c60f0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f9598004730 tx=0x7f959c0047c0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.442+0000 7f95aefb0700 1 -- 172.21.15.83:0/421577164 shutdown_connections 2023-11-26T07:17:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.442+0000 7f95aefb0700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f958c06b020 0x7f958c06d4e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.442+0000 7f95aefb0700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95a8157a40 0x7f95a80afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.442+0000 7f95aefb0700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9598003a00 0x7f95a80c60f0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.442+0000 7f95aefb0700 1 --2- 172.21.15.83:0/421577164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95a80b75b0 0x7f95a814cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.442+0000 7f95aefb0700 1 -- 172.21.15.83:0/421577164 >> 172.21.15.83:0/421577164 conn(0x7f95a80b3780 msgr2=0x7f95a80cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.442+0000 7f95aefb0700 1 -- 172.21.15.83:0/421577164 shutdown_connections 2023-11-26T07:17:15.444 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:15.442+0000 7f95aefb0700 1 -- 172.21.15.83:0/421577164 wait complete. 2023-11-26T07:17:15.455 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol77760 with 8 files each 1MB size under directory volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070 2023-11-26T07:17:15.455 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:15.455 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:15.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:15.472 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:17:15.541 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:17:15.541 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:17:15.541 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883881 s, 119 MB/s 2023-11-26T07:17:15.541 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:15.541 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:15.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:15.598 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:17:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:17:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:17:15.668 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108553 s, 96.6 MB/s 2023-11-26T07:17:15.669 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:15.669 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:15.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:15.726 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:17:15.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:17:15.792 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:17:15.792 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968045 s, 108 MB/s 2023-11-26T07:17:15.793 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:15.793 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:15.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:15.849 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:17:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:17:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:17:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139421 s, 75.2 MB/s 2023-11-26T07:17:15.921 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:15.921 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:15.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:15.978 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:17:16.047 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:17:16.047 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:17:16.047 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113241 s, 92.6 MB/s 2023-11-26T07:17:16.048 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:16.048 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:16.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:16.106 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:17:16.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:17:16.174 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:17:16.174 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107203 s, 97.8 MB/s 2023-11-26T07:17:16.175 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:16.175 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:16.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:16.232 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:17:16.300 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:17:16.300 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:17:16.300 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105652 s, 99.2 MB/s 2023-11-26T07:17:16.300 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:16.300 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:16.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:16.357 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:17:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2023-11-26T07:17:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2023-11-26T07:17:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116751 s, 89.8 MB/s 2023-11-26T07:17:16.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol77760 subvol_snap47320 2023-11-26T07:17:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.590+0000 7f6e80e89700 1 Processor -- start 2023-11-26T07:17:16.591 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.590+0000 7f6e80e89700 1 -- start start 2023-11-26T07:17:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.590+0000 7f6e80e89700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6e7c15df20 0x7f6e7c06c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e80e89700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6e7c162820 0x7f6e7c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:16.593 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e80e89700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e7c161ba0 0x7f6e7c162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e80e89700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c162570 con 0x7f6e7c162820 2023-11-26T07:17:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e80e89700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c06bed0 con 0x7f6e7c15df20 2023-11-26T07:17:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e80e89700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c06cfe0 con 0x7f6e7c161ba0 2023-11-26T07:17:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e79d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6e7c162820 0x7f6e7c06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e79d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6e7c162820 0x7f6e7c06caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41914/0 (socket says 172.21.15.83:41914) 2023-11-26T07:17:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e79d9b700 1 -- 172.21.15.83:0/3607796917 learned_addr learned my addr 172.21.15.83:0/3607796917 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e7a59c700 1 --2- 172.21.15.83:0/3607796917 >> v2:172.21.15.203:3300/0 conn(0x7f6e7c15df20 0x7f6e7c06c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e79d9b700 1 -- 172.21.15.83:0/3607796917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e7c161ba0 msgr2=0x7f6e7c162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e79d9b700 1 --2- 172.21.15.83:0/3607796917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e7c161ba0 0x7f6e7c162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e79d9b700 1 -- 172.21.15.83:0/3607796917 >> v2:172.21.15.203:3300/0 conn(0x7f6e7c15df20 msgr2=0x7f6e7c06c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:16.594 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e79d9b700 1 --2- 172.21.15.83:0/3607796917 >> v2:172.21.15.203:3300/0 conn(0x7f6e7c15df20 0x7f6e7c06c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e79d9b700 1 -- 172.21.15.83:0/3607796917 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c06bba0 con 0x7f6e7c162820 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e79d9b700 1 --2- 172.21.15.83:0/3607796917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e7c162820 0x7f6e7c06caa0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c06ee80 tx=0x7f6e64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdabbf063683069 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e7959a700 1 -- 172.21.15.83:0/3607796917 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6400b870 con 0x7f6e7c162820 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e7959a700 1 -- 172.21.15.83:0/3607796917 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6400b9d0 con 0x7f6e7c162820 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.591+0000 7f6e7959a700 1 -- 172.21.15.83:0/3607796917 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6400bd60 con 0x7f6e7c162820 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.592+0000 7f6e80e89700 1 -- 172.21.15.83:0/3607796917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e7c162820 msgr2=0x7f6e7c06caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.592+0000 7f6e80e89700 1 --2- 172.21.15.83:0/3607796917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e7c162820 0x7f6e7c06caa0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c06ee80 tx=0x7f6e64009bd0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.592+0000 7f6e80e89700 1 -- 172.21.15.83:0/3607796917 shutdown_connections 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.592+0000 7f6e80e89700 1 --2- 172.21.15.83:0/3607796917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e7c161ba0 0x7f6e7c162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.592+0000 7f6e80e89700 1 --2- 172.21.15.83:0/3607796917 >> v2:172.21.15.203:3300/0 conn(0x7f6e7c15df20 0x7f6e7c06c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.592+0000 7f6e80e89700 1 --2- 172.21.15.83:0/3607796917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e7c162820 0x7f6e7c06caa0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.592+0000 7f6e80e89700 1 -- 172.21.15.83:0/3607796917 >> 172.21.15.83:0/3607796917 conn(0x7f6e7c06dc80 msgr2=0x7f6e7c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:16.595 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.592+0000 7f6e80e89700 1 -- 172.21.15.83:0/3607796917 shutdown_connections 2023-11-26T07:17:16.596 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.592+0000 7f6e80e89700 1 -- 172.21.15.83:0/3607796917 wait complete. 2023-11-26T07:17:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.593+0000 7f6e80e89700 1 Processor -- start 2023-11-26T07:17:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.593+0000 7f6e80e89700 1 -- start start 2023-11-26T07:17:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e80e89700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e7c15df20 0x7f6e7c1f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e80e89700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e7c161ba0 0x7f6e7c1e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e80e89700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e7c162820 0x7f6e7c0b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:16.598 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e80e89700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c161180 con 0x7f6e7c161ba0 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e80e89700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c160500 con 0x7f6e7c162820 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e80e89700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1607e0 con 0x7f6e7c15df20 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e79d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e7c161ba0 0x7f6e7c1e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e79d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e7c161ba0 0x7f6e7c1e87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41920/0 (socket says 172.21.15.83:41920) 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e79d9b700 1 -- 172.21.15.83:0/1753997397 learned_addr learned my addr 172.21.15.83:0/1753997397 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e7ad9d700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e7c162820 0x7f6e7c0b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e7a59c700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e7c15df20 0x7f6e7c1f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e79d9b700 1 -- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e7c15df20 msgr2=0x7f6e7c1f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e79d9b700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e7c15df20 0x7f6e7c1f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e79d9b700 1 -- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e7c162820 msgr2=0x7f6e7c0b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e79d9b700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e7c162820 0x7f6e7c0b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.599 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e79d9b700 1 -- 172.21.15.83:0/1753997397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64009550 con 0x7f6e7c161ba0 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.594+0000 7f6e7ad9d700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e7c162820 0x7f6e7c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.595+0000 7f6e79d9b700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e7c161ba0 0x7f6e7c1e87c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6e64000c00 tx=0x7f6e64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.595+0000 7f6e72ffd700 1 -- 172.21.15.83:0/1753997397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e640036a0 con 0x7f6e7c161ba0 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.595+0000 7f6e72ffd700 1 -- 172.21.15.83:0/1753997397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e64003800 con 0x7f6e7c161ba0 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.595+0000 7f6e80e89700 1 -- 172.21.15.83:0/1753997397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c160ac0 con 0x7f6e7c161ba0 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.595+0000 7f6e72ffd700 1 -- 172.21.15.83:0/1753997397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64003b90 con 0x7f6e7c161ba0 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.595+0000 7f6e80e89700 1 -- 172.21.15.83:0/1753997397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c1e8d00 con 0x7f6e7c161ba0 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.596+0000 7f6e70ff9700 1 -- 172.21.15.83:0/1753997397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e540052e0 con 0x7f6e7c161ba0 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.596+0000 7f6e72ffd700 1 -- 172.21.15.83:0/1753997397 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f6e64003cf0 con 0x7f6e7c161ba0 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.596+0000 7f6e72ffd700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6e5c06b080 0x7f6e5c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.597+0000 7f6e72ffd700 1 -- 172.21.15.83:0/1753997397 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e6408ae00 con 0x7f6e7c161ba0 2023-11-26T07:17:16.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.597+0000 7f6e7a59c700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6e5c06b080 0x7f6e5c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:16.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.597+0000 7f6e7a59c700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6e5c06b080 0x7f6e5c06d540 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00b840 tx=0x7f6e6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:16.602 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.601+0000 7f6e72ffd700 1 -- 172.21.15.83:0/1753997397 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e64055cc0 con 0x7f6e7c161ba0 2023-11-26T07:17:16.831 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.830+0000 7f6e70ff9700 1 -- 172.21.15.83:0/1753997397 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol77760", "snap_name": "subvol_snap47320", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e54000c50 con 0x7f6e5c06b080 2023-11-26T07:17:16.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.862+0000 7f6e72ffd700 1 -- 172.21.15.83:0/1753997397 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6e54000c50 con 0x7f6e5c06b080 2023-11-26T07:17:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.864+0000 7f6e70ff9700 1 -- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6e5c06b080 msgr2=0x7f6e5c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.864+0000 7f6e70ff9700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6e5c06b080 0x7f6e5c06d540 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00b840 tx=0x7f6e6c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.864+0000 7f6e70ff9700 1 -- 172.21.15.83:0/1753997397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e7c161ba0 msgr2=0x7f6e7c1e87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.864+0000 7f6e70ff9700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e7c161ba0 0x7f6e7c1e87c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6e64000c00 tx=0x7f6e64004750 comp rx=0 tx=0).stop 2023-11-26T07:17:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.865+0000 7f6e70ff9700 1 -- 172.21.15.83:0/1753997397 shutdown_connections 2023-11-26T07:17:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.865+0000 7f6e70ff9700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6e5c06b080 0x7f6e5c06d540 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.865+0000 7f6e70ff9700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e7c162820 0x7f6e7c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.865+0000 7f6e70ff9700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e7c15df20 0x7f6e7c1f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.865+0000 7f6e70ff9700 1 --2- 172.21.15.83:0/1753997397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e7c161ba0 0x7f6e7c1e87c0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.865+0000 7f6e70ff9700 1 -- 172.21.15.83:0/1753997397 >> 172.21.15.83:0/1753997397 conn(0x7f6e7c06dc80 msgr2=0x7f6e7c0cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.865+0000 7f6e70ff9700 1 -- 172.21.15.83:0/1753997397 shutdown_connections 2023-11-26T07:17:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:16.865+0000 7f6e70ff9700 1 -- 172.21.15.83:0/1753997397 wait complete. 2023-11-26T07:17:16.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol77760 subvol_snap47320 subvol_clone67280 2023-11-26T07:17:17.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.003+0000 7f00faf79700 1 Processor -- start 2023-11-26T07:17:17.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.003+0000 7f00faf79700 1 -- start start 2023-11-26T07:17:17.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00faf79700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f40b5da0 0x7f00f40afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00faf79700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f40b3c60 con 0x7f00f40746a0 2023-11-26T07:17:17.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00faf79700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f40b58c0 con 0x7f00f415e160 2023-11-26T07:17:17.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00faf79700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40b4500 con 0x7f00f40b5da0 2023-11-26T07:17:17.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f3fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f00f40746a0 0x7f00f406d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33736/0 (socket says 172.21.15.83:33736) 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f3fff700 1 -- 172.21.15.83:0/3255456256 learned_addr learned my addr 172.21.15.83:0/3255456256 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f9516700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f40b5da0 0x7f00f40afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688742028 0 0) 0x7f00f40b3c60 con 0x7f00f40746a0 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e40035b0 con 0x7f00f40746a0 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1771860111 0 0) 0x7f00e40035b0 con 0x7f00f40746a0 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f40b5da0 msgr2=0x7f00f40afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f40b5da0 0x7f00f40afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 >> v1:172.21.15.203:6789/0 conn(0x7f00f415e160 legacy=0x7f00f4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f406bdc0 con 0x7f00f40746a0 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f00e0001670 con 0x7f00f40746a0 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 >> v1:172.21.15.83:6789/0 conn(0x7f00f40746a0 legacy=0x7f00f406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e4003a30 0x7f00e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e40043e0 0x7f00e40048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.004+0000 7f00f37fe700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f40b5da0 0x7f00e4005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40b4500 con 0x7f00e40043e0 2023-11-26T07:17:17.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e400ba10 con 0x7f00e4003a30 2023-11-26T07:17:17.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e400bbd0 con 0x7f00f40b5da0 2023-11-26T07:17:17.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f3fff700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e40043e0 0x7f00e40048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f8d15700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e4003a30 0x7f00e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f9516700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f40b5da0 0x7f00e4005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f3fff700 1 -- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f40b5da0 msgr2=0x7f00e4005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:17.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f3fff700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f40b5da0 0x7f00e4005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f3fff700 1 -- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e4003a30 msgr2=0x7f00e4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:17.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f3fff700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e4003a30 0x7f00e4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f3fff700 1 -- 172.21.15.83:0/3255456256 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e40035b0 con 0x7f00e40043e0 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f8d15700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e4003a30 0x7f00e4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f3fff700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e40043e0 0x7f00e40048b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f00e000c2f0 tx=0x7f00e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac34ed9ddce74781 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e000ce30 con 0x7f00e40043e0 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e0005960 con 0x7f00e40043e0 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.005+0000 7f00f37fe700 1 -- 172.21.15.83:0/3255456256 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0005d10 con 0x7f00e40043e0 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.006+0000 7f00faf79700 1 -- 172.21.15.83:0/3255456256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e40043e0 msgr2=0x7f00e40048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.006+0000 7f00faf79700 1 --2- 172.21.15.83:0/3255456256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00e40043e0 0x7f00e40048b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f00e000c2f0 tx=0x7f00e4005880 comp rx=0 tx=0).stop 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.006+0000 7f00f9516700 1 -- 172.21.15.83:0/3255456256 reap_dead start 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.006+0000 7f00faf79700 1 -- 172.21.15.83:0/3255456256 shutdown_connections 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.006+0000 7f00faf79700 1 -- 172.21.15.83:0/3255456256 >> 172.21.15.83:0/3255456256 conn(0x7f00f40d0bf0 msgr2=0x7f00f40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:17.011 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.006+0000 7f00faf79700 1 -- 172.21.15.83:0/3255456256 shutdown_connections 2023-11-26T07:17:17.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.006+0000 7f00faf79700 1 -- 172.21.15.83:0/3255456256 wait complete. 2023-11-26T07:17:17.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.007+0000 7f00faf79700 1 Processor -- start 2023-11-26T07:17:17.012 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.007+0000 7f00faf79700 1 -- start start 2023-11-26T07:17:17.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00faf79700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e40043e0 0x7f00f41f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00faf79700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00f4160600 0x7f00f4160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00faf79700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f415c040 0x7f00f415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00faf79700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f415ca00 con 0x7f00f4160600 2023-11-26T07:17:17.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00faf79700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40c6220 con 0x7f00e40043e0 2023-11-26T07:17:17.014 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00faf79700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f4161080 con 0x7f00f415c040 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00f4160600 0x7f00f4160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00f4160600 0x7f00f4160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41940/0 (socket says 172.21.15.83:41940) 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f3fff700 1 -- 172.21.15.83:0/916936505 learned_addr learned my addr 172.21.15.83:0/916936505 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f9516700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f415c040 0x7f00f415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f8d15700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e40043e0 0x7f00f41f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f3fff700 1 -- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f415c040 msgr2=0x7f00f415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f3fff700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f415c040 0x7f00f415c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f3fff700 1 -- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e40043e0 msgr2=0x7f00f41f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f3fff700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e40043e0 0x7f00f41f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f3fff700 1 -- 172.21.15.83:0/916936505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4161360 con 0x7f00f4160600 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f9516700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f415c040 0x7f00f415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:17.015 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f8d15700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e40043e0 0x7f00f41f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f3fff700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00f4160600 0x7f00f4160a80 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005d00 tx=0x7f00e0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.008+0000 7f00f17fa700 1 -- 172.21.15.83:0/916936505 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0001660 con 0x7f00f4160600 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.009+0000 7f00f17fa700 1 -- 172.21.15.83:0/916936505 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e0004a80 con 0x7f00f4160600 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.009+0000 7f00faf79700 1 -- 172.21.15.83:0/916936505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f40c5bd0 con 0x7f00f4160600 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.009+0000 7f00f17fa700 1 -- 172.21.15.83:0/916936505 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0004e30 con 0x7f00f4160600 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.009+0000 7f00faf79700 1 -- 172.21.15.83:0/916936505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f41f3900 con 0x7f00f4160600 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.010+0000 7f00faf79700 1 -- 172.21.15.83:0/916936505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f406f9e0 con 0x7f00f4160600 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.010+0000 7f00f17fa700 1 -- 172.21.15.83:0/916936505 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f00e00070d0 con 0x7f00f4160600 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.010+0000 7f00f17fa700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f00d806b080 0x7f00d806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.010+0000 7f00f17fa700 1 -- 172.21.15.83:0/916936505 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00e008ace0 con 0x7f00f4160600 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.011+0000 7f00f8d15700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f00d806b080 0x7f00d806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.016 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.011+0000 7f00f8d15700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f00d806b080 0x7f00d806d540 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f00e80011a0 tx=0x7f00e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:17.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.015+0000 7f00f17fa700 1 -- 172.21.15.83:0/916936505 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00e0055cc0 con 0x7f00f4160600 2023-11-26T07:17:17.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.242+0000 7f00faf79700 1 -- 172.21.15.83:0/916936505 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol77760", "snap_name": "subvol_snap47320", "target_sub_name": "subvol_clone67280", "target": ["mon-mgr", ""]}) v1 -- 0x7f00f415e540 con 0x7f00d806b080 2023-11-26T07:17:17.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.321+0000 7f00f17fa700 1 -- 172.21.15.83:0/916936505 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f00e8001370 con 0x7f00d806b080 2023-11-26T07:17:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 -- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f00d806b080 msgr2=0x7f00d806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f00d806b080 0x7f00d806d540 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f00e80011a0 tx=0x7f00e8009040 comp rx=0 tx=0).stop 2023-11-26T07:17:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 -- 172.21.15.83:0/916936505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00f4160600 msgr2=0x7f00f4160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00f4160600 0x7f00f4160a80 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005d00 tx=0x7f00e0001a00 comp rx=0 tx=0).stop 2023-11-26T07:17:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 -- 172.21.15.83:0/916936505 shutdown_connections 2023-11-26T07:17:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f00d806b080 0x7f00d806d540 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00f415c040 0x7f00f415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e40043e0 0x7f00f41f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 --2- 172.21.15.83:0/916936505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00f4160600 0x7f00f4160a80 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 -- 172.21.15.83:0/916936505 >> 172.21.15.83:0/916936505 conn(0x7f00f40d0bf0 msgr2=0x7f00f4073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 -- 172.21.15.83:0/916936505 shutdown_connections 2023-11-26T07:17:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.323+0000 7f00faf79700 1 -- 172.21.15.83:0/916936505 wait complete. 2023-11-26T07:17:17.327 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:17.325+0000 7f30f9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:17.327 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:17.325+0000 7f30f9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:17.327 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:17.325+0000 7f30f9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:17.327 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:17.325+0000 7f30f9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:17.327 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:17.325+0000 7f30f9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:17.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone67280 2023-11-26T07:17:17.341 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:17.339+0000 7f30f8731700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:17.341 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:17.339+0000 7f30f8731700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:17.341 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:17.339+0000 7f30f8731700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:17.341 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:17.339+0000 7f30f8731700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:17.342 INFO:tasks.ceph.mgr.x.smithi203.stderr:2023-11-26T07:17:17.339+0000 7f30f8731700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:17:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.474+0000 7f20c9c0e700 1 Processor -- start 2023-11-26T07:17:17.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.474+0000 7f20c9c0e700 1 -- start start 2023-11-26T07:17:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.475+0000 7f20c9c0e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f20c41587f0 0x7f20c406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.475+0000 7f20c9c0e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c40b4370 0x7f20c415b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.475+0000 7f20c9c0e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c406b530 con 0x7f20c415b370 2023-11-26T07:17:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.475+0000 7f20c9c0e700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c40ae5e0 con 0x7f20c41587f0 2023-11-26T07:17:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.475+0000 7f20c9c0e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c415be30 con 0x7f20c40b4370 2023-11-26T07:17:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c3fff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c40b4370 0x7f20c415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c3fff700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c40b4370 0x7f20c415b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.83:52408/0 (socket says 172.21.15.83:52408) 2023-11-26T07:17:17.478 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c37fe700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f20c41587f0 0x7f20c406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c3fff700 1 -- 172.21.15.83:0/1455926694 learned_addr learned my addr 172.21.15.83:0/1455926694 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c3fff700 1 -- 172.21.15.83:0/1455926694 >> v2:172.21.15.203:3300/0 conn(0x7f20c41587f0 msgr2=0x7f20c406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c3fff700 1 --2- 172.21.15.83:0/1455926694 >> v2:172.21.15.203:3300/0 conn(0x7f20c41587f0 0x7f20c406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c3fff700 1 -- 172.21.15.83:0/1455926694 >> v1:172.21.15.203:6789/0 conn(0x7f20c415b370 legacy=0x7f20c40b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c3fff700 1 -- 172.21.15.83:0/1455926694 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c415c110 con 0x7f20c40b4370 2023-11-26T07:17:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c27fc700 1 -- 172.21.15.83:0/1455926694 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343626822 0 0) 0x7f20c406b530 con 0x7f20c415b370 2023-11-26T07:17:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c37fe700 1 --2- 172.21.15.83:0/1455926694 >> v2:172.21.15.203:3300/0 conn(0x7f20c41587f0 0x7f20c406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c3fff700 1 --2- 172.21.15.83:0/1455926694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c40b4370 0x7f20c415b800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00d150 tx=0x7f20bc00d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b27a8f284ed47e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c27fc700 1 -- 172.21.15.83:0/1455926694 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc010070 con 0x7f20c40b4370 2023-11-26T07:17:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c27fc700 1 -- 172.21.15.83:0/1455926694 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc00dd10 con 0x7f20c40b4370 2023-11-26T07:17:17.479 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.476+0000 7f20c27fc700 1 -- 172.21.15.83:0/1455926694 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00b310 con 0x7f20c40b4370 2023-11-26T07:17:17.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.477+0000 7f20c9c0e700 1 -- 172.21.15.83:0/1455926694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c40b4370 msgr2=0x7f20c415b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:17.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.477+0000 7f20c9c0e700 1 --2- 172.21.15.83:0/1455926694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c40b4370 0x7f20c415b800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00d150 tx=0x7f20bc00d460 comp rx=0 tx=0).stop 2023-11-26T07:17:17.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.477+0000 7f20c9c0e700 1 -- 172.21.15.83:0/1455926694 shutdown_connections 2023-11-26T07:17:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.477+0000 7f20c9c0e700 1 --2- 172.21.15.83:0/1455926694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c40b4370 0x7f20c415b800 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.477+0000 7f20c9c0e700 1 --2- 172.21.15.83:0/1455926694 >> v2:172.21.15.203:3300/0 conn(0x7f20c41587f0 0x7f20c406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.477+0000 7f20c9c0e700 1 -- 172.21.15.83:0/1455926694 >> 172.21.15.83:0/1455926694 conn(0x7f20c40d0bf0 msgr2=0x7f20c40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.477+0000 7f20c9c0e700 1 -- 172.21.15.83:0/1455926694 shutdown_connections 2023-11-26T07:17:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.477+0000 7f20c9c0e700 1 -- 172.21.15.83:0/1455926694 wait complete. 2023-11-26T07:17:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.478+0000 7f20c9c0e700 1 Processor -- start 2023-11-26T07:17:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.479+0000 7f20c9c0e700 1 -- start start 2023-11-26T07:17:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.479+0000 7f20c9c0e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b4370 0x7f20c41448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.479+0000 7f20c9c0e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20c41587f0 0x7f20c409d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.479+0000 7f20c9c0e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c415b370 0x7f20c409d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.479+0000 7f20c9c0e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40c5de0 con 0x7f20c41587f0 2023-11-26T07:17:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.479+0000 7f20c9c0e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c41459c0 con 0x7f20c40b4370 2023-11-26T07:17:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.479+0000 7f20c9c0e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40b2460 con 0x7f20c415b370 2023-11-26T07:17:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.479+0000 7f20c2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20c41587f0 0x7f20c409d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.479+0000 7f20c2ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20c41587f0 0x7f20c409d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41954/0 (socket says 172.21.15.83:41954) 2023-11-26T07:17:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.479+0000 7f20c2ffd700 1 -- 172.21.15.83:0/2434698941 learned_addr learned my addr 172.21.15.83:0/2434698941 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.479+0000 7f20c37fe700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b4370 0x7f20c41448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20c3fff700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c415b370 0x7f20c409d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20c2ffd700 1 -- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c415b370 msgr2=0x7f20c409d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20c2ffd700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c415b370 0x7f20c409d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20c2ffd700 1 -- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b4370 msgr2=0x7f20c41448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:17.484 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20c2ffd700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b4370 0x7f20c41448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20c2ffd700 1 -- 172.21.15.83:0/2434698941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b40094a0 con 0x7f20c41587f0 2023-11-26T07:17:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20c37fe700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b4370 0x7f20c41448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20c2ffd700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20c41587f0 0x7f20c409d310 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f20b800c580 tx=0x7f20b800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20abfff700 1 -- 172.21.15.83:0/2434698941 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b80054e0 con 0x7f20c41587f0 2023-11-26T07:17:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20abfff700 1 -- 172.21.15.83:0/2434698941 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b8003680 con 0x7f20c41587f0 2023-11-26T07:17:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20c9c0e700 1 -- 172.21.15.83:0/2434698941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20bc00bd40 con 0x7f20c41587f0 2023-11-26T07:17:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20abfff700 1 -- 172.21.15.83:0/2434698941 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8003a30 con 0x7f20c41587f0 2023-11-26T07:17:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.480+0000 7f20c9c0e700 1 -- 172.21.15.83:0/2434698941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c4145370 con 0x7f20c41587f0 2023-11-26T07:17:17.485 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.481+0000 7f20c9c0e700 1 -- 172.21.15.83:0/2434698941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c4145b70 con 0x7f20c41587f0 2023-11-26T07:17:17.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.482+0000 7f20abfff700 1 -- 172.21.15.83:0/2434698941 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 89384+0+0 (secure 0 0 0) 0x7f20b8003b90 con 0x7f20c41587f0 2023-11-26T07:17:17.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.482+0000 7f20abfff700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f20b006b080 0x7f20b006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:17.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.482+0000 7f20abfff700 1 -- 172.21.15.83:0/2434698941 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20b8088bf0 con 0x7f20c41587f0 2023-11-26T07:17:17.486 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.482+0000 7f20c37fe700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f20b006b080 0x7f20b006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:17.487 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.483+0000 7f20c37fe700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f20b006b080 0x7f20b006d540 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f20b40114b0 tx=0x7f20b4009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:17.488 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.487+0000 7f20abfff700 1 -- 172.21.15.83:0/2434698941 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20c4145b70 con 0x7f20c41587f0 2023-11-26T07:17:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:17.739+0000 7f20c9c0e700 1 -- 172.21.15.83:0/2434698941 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone67280", "target": ["mon-mgr", ""]}) v1 -- 0x7f20c40aeb40 con 0x7f20b006b080 2023-11-26T07:17:19.672 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:19.669+0000 7f20abfff700 1 -- 172.21.15.83:0/2434698941 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f20b8053df0 con 0x7f20c41587f0 2023-11-26T07:17:21.947 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.945+0000 7f20abfff700 1 -- 172.21.15.83:0/2434698941 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f20b4011e10 con 0x7f20b006b080 2023-11-26T07:17:21.948 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: "status": { 2023-11-26T07:17:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "complete" 2023-11-26T07:17:21.948 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:17:21.948 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 -- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f20b006b080 msgr2=0x7f20b006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f20b006b080 0x7f20b006d540 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f20b40114b0 tx=0x7f20b4009210 comp rx=0 tx=0).stop 2023-11-26T07:17:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 -- 172.21.15.83:0/2434698941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20c41587f0 msgr2=0x7f20c409d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20c41587f0 0x7f20c409d310 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f20b800c580 tx=0x7f20b800c890 comp rx=0 tx=0).stop 2023-11-26T07:17:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 -- 172.21.15.83:0/2434698941 shutdown_connections 2023-11-26T07:17:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f20b006b080 0x7f20b006d540 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20c415b370 0x7f20c409d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b4370 0x7f20c41448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 --2- 172.21.15.83:0/2434698941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20c41587f0 0x7f20c409d310 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 -- 172.21.15.83:0/2434698941 >> 172.21.15.83:0/2434698941 conn(0x7f20c40d0bf0 msgr2=0x7f20c4071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 -- 172.21.15.83:0/2434698941 shutdown_connections 2023-11-26T07:17:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:21.948+0000 7f20c9c0e700 1 -- 172.21.15.83:0/2434698941 wait complete. 2023-11-26T07:17:21.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:21.966 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone67280/.meta) 2023-11-26T07:17:21.996 INFO:teuthology.orchestra.run.smithi083.stdout:[GLOBAL] 2023-11-26T07:17:21.996 INFO:teuthology.orchestra.run.smithi083.stdout:version = 2 2023-11-26T07:17:21.996 INFO:teuthology.orchestra.run.smithi083.stdout:type = clone 2023-11-26T07:17:21.996 INFO:teuthology.orchestra.run.smithi083.stdout:path = /volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb 2023-11-26T07:17:21.996 INFO:teuthology.orchestra.run.smithi083.stdout:state = complete 2023-11-26T07:17:21.996 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:21.996 INFO:teuthology.orchestra.run.smithi083.stdout:[source] 2023-11-26T07:17:21.997 INFO:teuthology.orchestra.run.smithi083.stdout:volume = cephfs 2023-11-26T07:17:21.997 INFO:teuthology.orchestra.run.smithi083.stdout:subvolume = subvol77760 2023-11-26T07:17:21.997 INFO:teuthology.orchestra.run.smithi083.stdout:snapshot = subvol_snap47320 2023-11-26T07:17:21.997 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:21.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol77760 2023-11-26T07:17:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.162+0000 7f45ce07f700 1 Processor -- start 2023-11-26T07:17:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.162+0000 7f45ce07f700 1 -- start start 2023-11-26T07:17:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45ce07f700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f45c815f9e0 0x7f45c80add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45ce07f700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f45c8073f90 0x7f45c8074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45ce07f700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c806b080 con 0x7f45c80cf610 2023-11-26T07:17:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45ce07f700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c8074960 con 0x7f45c815f9e0 2023-11-26T07:17:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45ce07f700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c815c290 con 0x7f45c8073f90 2023-11-26T07:17:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c6ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f45c815f9e0 0x7f45c80add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c77fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f45c80cf610 0x7f45c80b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33768/0 (socket says 172.21.15.83:33768) 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c6ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f45c815f9e0 0x7f45c80add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41962/0 (socket says 172.21.15.83:41962) 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c77fe700 1 -- 172.21.15.83:0/1016322 learned_addr learned my addr 172.21.15.83:0/1016322 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c6ffd700 1 -- 172.21.15.83:0/1016322 >> v2:172.21.15.203:3300/0 conn(0x7f45c8073f90 msgr2=0x7f45c8074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c7fff700 1 --2- 172.21.15.83:0/1016322 >> v2:172.21.15.203:3300/0 conn(0x7f45c8073f90 0x7f45c8074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c6ffd700 1 --2- 172.21.15.83:0/1016322 >> v2:172.21.15.203:3300/0 conn(0x7f45c8073f90 0x7f45c8074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c6ffd700 1 -- 172.21.15.83:0/1016322 >> v1:172.21.15.83:6789/0 conn(0x7f45c80cf610 legacy=0x7f45c80b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c6ffd700 1 -- 172.21.15.83:0/1016322 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c80cf9f0 con 0x7f45c815f9e0 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c67fc700 1 -- 172.21.15.83:0/1016322 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406663786 0 0) 0x7f45c806b080 con 0x7f45c80cf610 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c6ffd700 1 --2- 172.21.15.83:0/1016322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c815f9e0 0x7f45c80add40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009840 tx=0x7f45b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd81dc3ebb6a838 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.163+0000 7f45c67fc700 1 -- 172.21.15.83:0/1016322 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8011830 con 0x7f45c815f9e0 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.164+0000 7f45c67fc700 1 -- 172.21.15.83:0/1016322 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b8011990 con 0x7f45c815f9e0 2023-11-26T07:17:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.164+0000 7f45c67fc700 1 -- 172.21.15.83:0/1016322 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8011d20 con 0x7f45c815f9e0 2023-11-26T07:17:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.164+0000 7f45ce07f700 1 -- 172.21.15.83:0/1016322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c815f9e0 msgr2=0x7f45c80add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.164+0000 7f45ce07f700 1 --2- 172.21.15.83:0/1016322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c815f9e0 0x7f45c80add40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009840 tx=0x7f45b8009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.164+0000 7f45ce07f700 1 -- 172.21.15.83:0/1016322 shutdown_connections 2023-11-26T07:17:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.164+0000 7f45ce07f700 1 --2- 172.21.15.83:0/1016322 >> v2:172.21.15.203:3300/0 conn(0x7f45c8073f90 0x7f45c8074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.164+0000 7f45ce07f700 1 --2- 172.21.15.83:0/1016322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c815f9e0 0x7f45c80add40 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.164+0000 7f45ce07f700 1 -- 172.21.15.83:0/1016322 >> 172.21.15.83:0/1016322 conn(0x7f45c80d0bf0 msgr2=0x7f45c80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.164+0000 7f45ce07f700 1 -- 172.21.15.83:0/1016322 shutdown_connections 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.164+0000 7f45ce07f700 1 -- 172.21.15.83:0/1016322 wait complete. 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.165+0000 7f45ce07f700 1 Processor -- start 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.165+0000 7f45ce07f700 1 -- start start 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.166+0000 7f45ce07f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45c8073f90 0x7f45c814a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.166+0000 7f45ce07f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c80cf610 0x7f45c814d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.166+0000 7f45ce07f700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45c815f9e0 0x7f45c815e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.166+0000 7f45ce07f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c814dc30 con 0x7f45c80cf610 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.166+0000 7f45ce07f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c81f3b60 con 0x7f45c8073f90 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.166+0000 7f45ce07f700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c81f3e40 con 0x7f45c815f9e0 2023-11-26T07:17:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.166+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c80cf610 0x7f45c814d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.166+0000 7f45c6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c80cf610 0x7f45c814d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41968/0 (socket says 172.21.15.83:41968) 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.166+0000 7f45c6ffd700 1 -- 172.21.15.83:0/3007389849 learned_addr learned my addr 172.21.15.83:0/3007389849 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45c6ffd700 1 -- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45c815f9e0 msgr2=0x7f45c815e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45c77fe700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45c8073f90 0x7f45c814a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45c6ffd700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45c815f9e0 0x7f45c815e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45c6ffd700 1 -- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45c8073f90 msgr2=0x7f45c814a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45c6ffd700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45c8073f90 0x7f45c814a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45c6ffd700 1 -- 172.21.15.83:0/3007389849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b80094a0 con 0x7f45c80cf610 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45c6ffd700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c80cf610 0x7f45c814d6f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f45b80090b0 tx=0x7f45b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45affff700 1 -- 172.21.15.83:0/3007389849 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b80038a0 con 0x7f45c80cf610 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45affff700 1 -- 172.21.15.83:0/3007389849 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b801f070 con 0x7f45c80cf610 2023-11-26T07:17:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45ce07f700 1 -- 172.21.15.83:0/3007389849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c81f4120 con 0x7f45c80cf610 2023-11-26T07:17:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45affff700 1 -- 172.21.15.83:0/3007389849 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8003c40 con 0x7f45c80cf610 2023-11-26T07:17:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.167+0000 7f45ce07f700 1 -- 172.21.15.83:0/3007389849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c81f46e0 con 0x7f45c80cf610 2023-11-26T07:17:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.168+0000 7f45ce07f700 1 -- 172.21.15.83:0/3007389849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c814ad30 con 0x7f45c80cf610 2023-11-26T07:17:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.168+0000 7f45affff700 1 -- 172.21.15.83:0/3007389849 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f45b8026480 con 0x7f45c80cf610 2023-11-26T07:17:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.169+0000 7f45affff700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45b406b1a0 0x7f45b406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.169+0000 7f45affff700 1 -- 172.21.15.83:0/3007389849 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45b8021070 con 0x7f45c80cf610 2023-11-26T07:17:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.169+0000 7f45c77fe700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45b406b1a0 0x7f45b406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.170+0000 7f45c77fe700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45b406b1a0 0x7f45b406d660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000f60 tx=0x7f45bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.173+0000 7f45affff700 1 -- 172.21.15.83:0/3007389849 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45b8056290 con 0x7f45c80cf610 2023-11-26T07:17:22.398 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.395+0000 7f45ce07f700 1 -- 172.21.15.83:0/3007389849 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol77760", "target": ["mon-mgr", ""]}) v1 -- 0x7f45c8160540 con 0x7f45b406b1a0 2023-11-26T07:17:22.414 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.411+0000 7f45affff700 1 -- 172.21.15.83:0/3007389849 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7f45bc0013d0 con 0x7f45b406b1a0 2023-11-26T07:17:22.414 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070 2023-11-26T07:17:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.413+0000 7f45ce07f700 1 -- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45b406b1a0 msgr2=0x7f45b406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.413+0000 7f45ce07f700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45b406b1a0 0x7f45b406d660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000f60 tx=0x7f45bc009040 comp rx=0 tx=0).stop 2023-11-26T07:17:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.413+0000 7f45ce07f700 1 -- 172.21.15.83:0/3007389849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c80cf610 msgr2=0x7f45c814d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.416 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.413+0000 7f45ce07f700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c80cf610 0x7f45c814d6f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f45b80090b0 tx=0x7f45b8004750 comp rx=0 tx=0).stop 2023-11-26T07:17:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.413+0000 7f45ce07f700 1 -- 172.21.15.83:0/3007389849 shutdown_connections 2023-11-26T07:17:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.413+0000 7f45ce07f700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45b406b1a0 0x7f45b406d660 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.413+0000 7f45ce07f700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45c815f9e0 0x7f45c815e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.413+0000 7f45ce07f700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45c8073f90 0x7f45c814a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.413+0000 7f45ce07f700 1 --2- 172.21.15.83:0/3007389849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45c80cf610 0x7f45c814d6f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.413+0000 7f45ce07f700 1 -- 172.21.15.83:0/3007389849 >> 172.21.15.83:0/3007389849 conn(0x7f45c80d0bf0 msgr2=0x7f45c80b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.414+0000 7f45ce07f700 1 -- 172.21.15.83:0/3007389849 shutdown_connections 2023-11-26T07:17:22.417 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.414+0000 7f45ce07f700 1 -- 172.21.15.83:0/3007389849 wait complete. 2023-11-26T07:17:22.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone67280 2023-11-26T07:17:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.549+0000 7fe5c3571700 1 Processor -- start 2023-11-26T07:17:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.549+0000 7fe5c3571700 1 -- start start 2023-11-26T07:17:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c3571700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5bc0746a0 0x7fe5bc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c3571700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe5bc0afc90 0x7fe5bc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c3571700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5bc0b3c60 con 0x7fe5bc15e160 2023-11-26T07:17:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c3571700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5bc0b4500 con 0x7fe5bc0afc90 2023-11-26T07:17:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c3571700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc0b58c0 con 0x7fe5bc0746a0 2023-11-26T07:17:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c1b0e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe5bc0afc90 0x7fe5bc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c130d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe5bc15e160 0x7fe5bc073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33782/0 (socket says 172.21.15.83:33782) 2023-11-26T07:17:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c130d700 1 -- 172.21.15.83:0/1934000873 learned_addr learned my addr 172.21.15.83:0/1934000873 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c1b0e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe5bc0afc90 0x7fe5bc06d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41978/0 (socket says 172.21.15.83:41978) 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c1b0e700 1 -- 172.21.15.83:0/1934000873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5bc0746a0 msgr2=0x7fe5bc0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c0b0c700 1 --2- 172.21.15.83:0/1934000873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5bc0746a0 0x7fe5bc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5b3fff700 1 -- 172.21.15.83:0/1934000873 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509049143 0 0) 0x7fe5bc0b3c60 con 0x7fe5bc15e160 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c1b0e700 1 --2- 172.21.15.83:0/1934000873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5bc0746a0 0x7fe5bc0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c1b0e700 1 -- 172.21.15.83:0/1934000873 >> v1:172.21.15.83:6789/0 conn(0x7fe5bc15e160 legacy=0x7fe5bc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c1b0e700 1 -- 172.21.15.83:0/1934000873 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc0b0140 con 0x7fe5bc0afc90 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c0b0c700 1 --2- 172.21.15.83:0/1934000873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5bc0746a0 0x7fe5bc0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5c1b0e700 1 --2- 172.21.15.83:0/1934000873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5bc0afc90 0x7fe5bc06d930 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009d90 tx=0x7fe5b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8735e91cabe14df9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5b3fff700 1 -- 172.21.15.83:0/1934000873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400e070 con 0x7fe5bc0afc90 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.550+0000 7fe5b3fff700 1 -- 172.21.15.83:0/1934000873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b400bbd0 con 0x7fe5bc0afc90 2023-11-26T07:17:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.551+0000 7fe5b3fff700 1 -- 172.21.15.83:0/1934000873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4012430 con 0x7fe5bc0afc90 2023-11-26T07:17:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.551+0000 7fe5c3571700 1 -- 172.21.15.83:0/1934000873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5bc0afc90 msgr2=0x7fe5bc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.551+0000 7fe5c3571700 1 --2- 172.21.15.83:0/1934000873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5bc0afc90 0x7fe5bc06d930 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009d90 tx=0x7fe5b400b330 comp rx=0 tx=0).stop 2023-11-26T07:17:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.551+0000 7fe5c3571700 1 -- 172.21.15.83:0/1934000873 shutdown_connections 2023-11-26T07:17:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.551+0000 7fe5c3571700 1 --2- 172.21.15.83:0/1934000873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5bc0afc90 0x7fe5bc06d930 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.551+0000 7fe5c3571700 1 --2- 172.21.15.83:0/1934000873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5bc0746a0 0x7fe5bc0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.551+0000 7fe5c3571700 1 -- 172.21.15.83:0/1934000873 >> 172.21.15.83:0/1934000873 conn(0x7fe5bc0d0bf0 msgr2=0x7fe5bc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.551+0000 7fe5c3571700 1 -- 172.21.15.83:0/1934000873 shutdown_connections 2023-11-26T07:17:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.551+0000 7fe5c3571700 1 -- 172.21.15.83:0/1934000873 wait complete. 2023-11-26T07:17:22.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.552+0000 7fe5c3571700 1 Processor -- start 2023-11-26T07:17:22.558 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.552+0000 7fe5c3571700 1 -- start start 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c3571700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5bc0746a0 0x7fe5bc14faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c3571700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5bc0afc90 0x7fe5bc154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c3571700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5bc15e160 0x7fe5bc152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c3571700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc1542a0 con 0x7fe5bc0746a0 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c3571700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc1534a0 con 0x7fe5bc15e160 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c3571700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc15ebd0 con 0x7fe5bc0afc90 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c130d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5bc0746a0 0x7fe5bc14faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c130d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5bc0746a0 0x7fe5bc14faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41980/0 (socket says 172.21.15.83:41980) 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c130d700 1 -- 172.21.15.83:0/3570474662 learned_addr learned my addr 172.21.15.83:0/3570474662 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c1b0e700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5bc15e160 0x7fe5bc152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c130d700 1 -- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5bc0afc90 msgr2=0x7fe5bc154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.553+0000 7fe5c0b0c700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5bc0afc90 0x7fe5bc154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.554+0000 7fe5c130d700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5bc0afc90 0x7fe5bc154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.554+0000 7fe5c130d700 1 -- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5bc15e160 msgr2=0x7fe5bc152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.554+0000 7fe5c130d700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5bc15e160 0x7fe5bc152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.554+0000 7fe5c130d700 1 -- 172.21.15.83:0/3570474662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b4009a70 con 0x7fe5bc0746a0 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.554+0000 7fe5c1b0e700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5bc15e160 0x7fe5bc152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.554+0000 7fe5c130d700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5bc0746a0 0x7fe5bc14faf0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800c550 tx=0x7fe5b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.555+0000 7fe5b1ffb700 1 -- 172.21.15.83:0/3570474662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8004c80 con 0x7fe5bc0746a0 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.555+0000 7fe5b1ffb700 1 -- 172.21.15.83:0/3570474662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b8004e00 con 0x7fe5bc0746a0 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.555+0000 7fe5c3571700 1 -- 172.21.15.83:0/3570474662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5bc153720 con 0x7fe5bc0746a0 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.555+0000 7fe5b1ffb700 1 -- 172.21.15.83:0/3570474662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8007310 con 0x7fe5bc0746a0 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.555+0000 7fe5c3571700 1 -- 172.21.15.83:0/3570474662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5bc153ce0 con 0x7fe5bc0746a0 2023-11-26T07:17:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.556+0000 7fe5c3571700 1 -- 172.21.15.83:0/3570474662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5bc0b4c60 con 0x7fe5bc0746a0 2023-11-26T07:17:22.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.556+0000 7fe5b1ffb700 1 -- 172.21.15.83:0/3570474662 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fe5b8007470 con 0x7fe5bc0746a0 2023-11-26T07:17:22.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.557+0000 7fe5b1ffb700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe5a006b1a0 0x7fe5a006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:22.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.557+0000 7fe5b1ffb700 1 -- 172.21.15.83:0/3570474662 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5b8089880 con 0x7fe5bc0746a0 2023-11-26T07:17:22.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.557+0000 7fe5c0b0c700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe5a006b1a0 0x7fe5a006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:22.562 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.558+0000 7fe5c0b0c700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe5a006b1a0 0x7fe5a006d660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac006fd0 tx=0x7fe5ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.562+0000 7fe5b1ffb700 1 -- 172.21.15.83:0/3570474662 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5b8054500 con 0x7fe5bc0746a0 2023-11-26T07:17:22.786 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.784+0000 7fe5c3571700 1 -- 172.21.15.83:0/3570474662 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone67280", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5bc0b4190 con 0x7fe5a006b1a0 2023-11-26T07:17:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.792+0000 7fe5b1ffb700 1 -- 172.21.15.83:0/3570474662 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+73 (secure 0 0 0) 0x7fe5ac00c040 con 0x7fe5a006b1a0 2023-11-26T07:17:22.794 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb 2023-11-26T07:17:22.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 -- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe5a006b1a0 msgr2=0x7fe5a006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe5a006b1a0 0x7fe5a006d660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac006fd0 tx=0x7fe5ac008040 comp rx=0 tx=0).stop 2023-11-26T07:17:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 -- 172.21.15.83:0/3570474662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5bc0746a0 msgr2=0x7fe5bc14faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5bc0746a0 0x7fe5bc14faf0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800c550 tx=0x7fe5b800c910 comp rx=0 tx=0).stop 2023-11-26T07:17:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 -- 172.21.15.83:0/3570474662 shutdown_connections 2023-11-26T07:17:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe5a006b1a0 0x7fe5a006d660 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5bc15e160 0x7fe5bc152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5bc0746a0 0x7fe5bc14faf0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 --2- 172.21.15.83:0/3570474662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5bc0afc90 0x7fe5bc154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 -- 172.21.15.83:0/3570474662 >> 172.21.15.83:0/3570474662 conn(0x7fe5bc0d0bf0 msgr2=0x7fe5bc1623f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 -- 172.21.15.83:0/3570474662 shutdown_connections 2023-11-26T07:17:22.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:22.794+0000 7fe5c3571700 1 -- 172.21.15.83:0/3570474662 wait complete. 2023-11-26T07:17:22.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:22.808 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320) 2023-11-26T07:17:22.826 INFO:teuthology.orchestra.run.smithi083.stdout:9 2023-11-26T07:17:22.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:22.826 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb) 2023-11-26T07:17:22.883 INFO:teuthology.orchestra.run.smithi083.stdout:9 2023-11-26T07:17:22.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone67280 2023-11-26T07:17:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.045+0000 7f50848fb700 1 Processor -- start 2023-11-26T07:17:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.046+0000 7f50848fb700 1 -- start start 2023-11-26T07:17:23.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.046+0000 7f50848fb700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f508015bf50 0x7f50800ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.046+0000 7f50848fb700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50800c6c30 0x7f50800cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.046+0000 7f50848fb700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50800d0140 con 0x7f50800ce830 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.046+0000 7f50848fb700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508015c330 con 0x7f50800c6c30 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.046+0000 7f50848fb700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50800cf3c0 con 0x7f508015bf50 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507ed9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50800c6c30 0x7f50800cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507ed9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f50800c6c30 0x7f50800cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41992/0 (socket says 172.21.15.83:41992) 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507ed9d700 1 -- 172.21.15.83:0/2124721370 learned_addr learned my addr 172.21.15.83:0/2124721370 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507e59c700 1 --2- 172.21.15.83:0/2124721370 >> v2:172.21.15.203:3300/0 conn(0x7f508015bf50 0x7f50800ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507ed9d700 1 -- 172.21.15.83:0/2124721370 >> v2:172.21.15.203:3300/0 conn(0x7f508015bf50 msgr2=0x7f50800ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507ed9d700 1 --2- 172.21.15.83:0/2124721370 >> v2:172.21.15.203:3300/0 conn(0x7f508015bf50 0x7f50800ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507ed9d700 1 -- 172.21.15.83:0/2124721370 >> v1:172.21.15.203:6789/0 conn(0x7f50800ce830 legacy=0x7f50800cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:23.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507ed9d700 1 -- 172.21.15.83:0/2124721370 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50800d03e0 con 0x7f50800c6c30 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507ed9d700 1 --2- 172.21.15.83:0/2124721370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50800c6c30 0x7f50800cc6c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f50800715e0 tx=0x7f507800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e6c52f2ba01c9a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507d59a700 1 -- 172.21.15.83:0/2124721370 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507800e070 con 0x7f50800c6c30 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507d59a700 1 -- 172.21.15.83:0/2124721370 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5078003a20 con 0x7f50800c6c30 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.047+0000 7f507d59a700 1 -- 172.21.15.83:0/2124721370 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078003dd0 con 0x7f50800c6c30 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.048+0000 7f50848fb700 1 -- 172.21.15.83:0/2124721370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50800c6c30 msgr2=0x7f50800cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.048+0000 7f50848fb700 1 --2- 172.21.15.83:0/2124721370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50800c6c30 0x7f50800cc6c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f50800715e0 tx=0x7f507800b330 comp rx=0 tx=0).stop 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.048+0000 7f50848fb700 1 -- 172.21.15.83:0/2124721370 shutdown_connections 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.048+0000 7f50848fb700 1 --2- 172.21.15.83:0/2124721370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50800c6c30 0x7f50800cc6c0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.048+0000 7f50848fb700 1 --2- 172.21.15.83:0/2124721370 >> v2:172.21.15.203:3300/0 conn(0x7f508015bf50 0x7f50800ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.048+0000 7f50848fb700 1 -- 172.21.15.83:0/2124721370 >> 172.21.15.83:0/2124721370 conn(0x7f50800b3780 msgr2=0x7f50800b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.048+0000 7f50848fb700 1 -- 172.21.15.83:0/2124721370 shutdown_connections 2023-11-26T07:17:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.048+0000 7f50848fb700 1 -- 172.21.15.83:0/2124721370 wait complete. 2023-11-26T07:17:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.049+0000 7f50848fb700 1 Processor -- start 2023-11-26T07:17:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.049+0000 7f50848fb700 1 -- start start 2023-11-26T07:17:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f50848fb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50800c6c30 0x7f50801e67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:23.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f50848fb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50800ce830 0x7f50801e6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f50848fb700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508015bf50 0x7f50800b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f50848fb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50801e7570 con 0x7f50800ce830 2023-11-26T07:17:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f50848fb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50801e7850 con 0x7f50800c6c30 2023-11-26T07:17:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f50848fb700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50801e7b30 con 0x7f508015bf50 2023-11-26T07:17:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50800ce830 0x7f50801e6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50800ce830 0x7f50801e6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41996/0 (socket says 172.21.15.83:41996) 2023-11-26T07:17:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507dd9b700 1 -- 172.21.15.83:0/1664402693 learned_addr learned my addr 172.21.15.83:0/1664402693 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507e59c700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50800c6c30 0x7f50801e67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507ed9d700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508015bf50 0x7f50800b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507dd9b700 1 -- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508015bf50 msgr2=0x7f50800b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:23.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507dd9b700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508015bf50 0x7f50800b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507dd9b700 1 -- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50800c6c30 msgr2=0x7f50801e67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507dd9b700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50800c6c30 0x7f50801e67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507dd9b700 1 -- 172.21.15.83:0/1664402693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5078009b30 con 0x7f50800ce830 2023-11-26T07:17:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507e59c700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50800c6c30 0x7f50801e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.050+0000 7f507dd9b700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50800ce830 0x7f50801e6ce0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f507000c4d0 tx=0x7f507000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.051+0000 7f506effd700 1 -- 172.21.15.83:0/1664402693 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070011790 con 0x7f50800ce830 2023-11-26T07:17:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.051+0000 7f506effd700 1 -- 172.21.15.83:0/1664402693 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50700118f0 con 0x7f50800ce830 2023-11-26T07:17:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.051+0000 7f50848fb700 1 -- 172.21.15.83:0/1664402693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50801e7e10 con 0x7f50800ce830 2023-11-26T07:17:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.051+0000 7f506effd700 1 -- 172.21.15.83:0/1664402693 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070011ca0 con 0x7f50800ce830 2023-11-26T07:17:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.051+0000 7f50848fb700 1 -- 172.21.15.83:0/1664402693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50801534c0 con 0x7f50800ce830 2023-11-26T07:17:23.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.051+0000 7f50848fb700 1 -- 172.21.15.83:0/1664402693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5080068fb0 con 0x7f50800ce830 2023-11-26T07:17:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.052+0000 7f506effd700 1 -- 172.21.15.83:0/1664402693 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f507001c460 con 0x7f50800ce830 2023-11-26T07:17:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.052+0000 7f506effd700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f506006b1a0 0x7f506006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.052+0000 7f506effd700 1 -- 172.21.15.83:0/1664402693 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5070089870 con 0x7f50800ce830 2023-11-26T07:17:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.053+0000 7f507e59c700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f506006b1a0 0x7f506006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.053+0000 7f507e59c700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f506006b1a0 0x7f506006d660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5074007a50 tx=0x7f5074008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.057+0000 7f506effd700 1 -- 172.21.15.83:0/1664402693 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5070054660 con 0x7f50800ce830 2023-11-26T07:17:23.282 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.280+0000 7f50848fb700 1 -- 172.21.15.83:0/1664402693 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone67280", "target": ["mon-mgr", ""]}) v1 -- 0x7f50800c5700 con 0x7f506006b1a0 2023-11-26T07:17:23.292 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.290+0000 7f506effd700 1 -- 172.21.15.83:0/1664402693 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+707 (secure 0 0 0) 0x7f5074007260 con 0x7f506006b1a0 2023-11-26T07:17:23.292 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "atime": "2023-11-26 07:17:12", 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_pcent": "80.00", 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_quota": 10485760, 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "bytes_used": 8388608, 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "created_at": "2023-11-26 07:17:17", 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "ctime": "2023-11-26 07:17:17", 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "data_pool": "cephfs_data", 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "features": [ 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot-clone", 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot-autoprotect", 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot-retention" 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:17:23.293 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 1000, 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": 16895, 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: "mon_addrs": [ 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.83:6789", 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.203:6789", 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: "172.21.15.203:6790" 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: "mtime": "2023-11-26 07:17:16", 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb", 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_namespace": "", 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "complete", 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "clone", 2023-11-26T07:17:23.294 INFO:teuthology.orchestra.run.smithi083.stdout: "uid": 1000 2023-11-26T07:17:23.295 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 -- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f506006b1a0 msgr2=0x7f506006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f506006b1a0 0x7f506006d660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5074007a50 tx=0x7f5074008040 comp rx=0 tx=0).stop 2023-11-26T07:17:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 -- 172.21.15.83:0/1664402693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50800ce830 msgr2=0x7f50801e6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50800ce830 0x7f50801e6ce0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f507000c4d0 tx=0x7f507000c890 comp rx=0 tx=0).stop 2023-11-26T07:17:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 -- 172.21.15.83:0/1664402693 shutdown_connections 2023-11-26T07:17:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f506006b1a0 0x7f506006d660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f508015bf50 0x7f50800b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50800c6c30 0x7f50801e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:23.296 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 --2- 172.21.15.83:0/1664402693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50800ce830 0x7f50801e6ce0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 -- 172.21.15.83:0/1664402693 >> 172.21.15.83:0/1664402693 conn(0x7f50800b3780 msgr2=0x7f50800ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 -- 172.21.15.83:0/1664402693 shutdown_connections 2023-11-26T07:17:23.297 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:23.292+0000 7f50848fb700 1 -- 172.21.15.83:0/1664402693 wait complete. 2023-11-26T07:17:23.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.307 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320) 2023-11-26T07:17:23.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:17:23.324 INFO:teuthology.orchestra.run.smithi083.stderr:volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320: ceph.quota.max_bytes: No such attribute 2023-11-26T07:17:23.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.325 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320) 2023-11-26T07:17:23.381 INFO:teuthology.orchestra.run.smithi083.stdout:cephfs_data 2023-11-26T07:17:23.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.381 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320) 2023-11-26T07:17:23.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.438 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320) 2023-11-26T07:17:23.496 INFO:teuthology.orchestra.run.smithi083.stdout:volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320 2023-11-26T07:17:23.496 INFO:teuthology.orchestra.run.smithi083.stdout:volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.2 2023-11-26T07:17:23.496 INFO:teuthology.orchestra.run.smithi083.stdout:volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.5 2023-11-26T07:17:23.496 INFO:teuthology.orchestra.run.smithi083.stdout:volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.4 2023-11-26T07:17:23.496 INFO:teuthology.orchestra.run.smithi083.stdout:volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.7 2023-11-26T07:17:23.496 INFO:teuthology.orchestra.run.smithi083.stdout:volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.6 2023-11-26T07:17:23.496 INFO:teuthology.orchestra.run.smithi083.stdout:volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.0 2023-11-26T07:17:23.496 INFO:teuthology.orchestra.run.smithi083.stdout:volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.3 2023-11-26T07:17:23.496 INFO:teuthology.orchestra.run.smithi083.stdout:volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.1 2023-11-26T07:17:23.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.497 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320) 2023-11-26T07:17:23.552 INFO:teuthology.orchestra.run.smithi083.stdout:41ff 2023-11-26T07:17:23.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.553 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/) 2023-11-26T07:17:23.609 INFO:teuthology.orchestra.run.smithi083.stdout:41ff 2023-11-26T07:17:23.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.610 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320) 2023-11-26T07:17:23.667 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:23.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.667 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/) 2023-11-26T07:17:23.724 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:23.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.724 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320) 2023-11-26T07:17:23.781 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:23.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.781 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/) 2023-11-26T07:17:23.838 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:23.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.838 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320) 2023-11-26T07:17:23.894 INFO:teuthology.orchestra.run.smithi083.stdout:1700983036 2023-11-26T07:17:23.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.895 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/) 2023-11-26T07:17:23.952 INFO:teuthology.orchestra.run.smithi083.stdout:1700983036 2023-11-26T07:17:23.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:23.952 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.2) 2023-11-26T07:17:24.008 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:24.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.009 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.2) 2023-11-26T07:17:24.067 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:24.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.068 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.2) 2023-11-26T07:17:24.125 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:24.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.125 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.2) 2023-11-26T07:17:24.182 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:24.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.182 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.2) 2023-11-26T07:17:24.239 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:24.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.239 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.2) 2023-11-26T07:17:24.295 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:24.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.295 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.2) 2023-11-26T07:17:24.352 INFO:teuthology.orchestra.run.smithi083.stdout:1700983035 2023-11-26T07:17:24.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.353 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.2) 2023-11-26T07:17:24.409 INFO:teuthology.orchestra.run.smithi083.stdout:1700983035 2023-11-26T07:17:24.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.409 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.5) 2023-11-26T07:17:24.466 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:24.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.466 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.5) 2023-11-26T07:17:24.523 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:24.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.523 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.5) 2023-11-26T07:17:24.579 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:24.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.580 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.5) 2023-11-26T07:17:24.635 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:24.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.635 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.5) 2023-11-26T07:17:24.691 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:24.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.692 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.5) 2023-11-26T07:17:24.749 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:24.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.749 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.5) 2023-11-26T07:17:24.805 INFO:teuthology.orchestra.run.smithi083.stdout:1700983036 2023-11-26T07:17:24.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.806 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.5) 2023-11-26T07:17:24.863 INFO:teuthology.orchestra.run.smithi083.stdout:1700983036 2023-11-26T07:17:24.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.863 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.4) 2023-11-26T07:17:24.919 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:24.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.920 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.4) 2023-11-26T07:17:24.976 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:24.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:24.976 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.4) 2023-11-26T07:17:25.033 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:25.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.033 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.4) 2023-11-26T07:17:25.089 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:25.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.090 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.4) 2023-11-26T07:17:25.145 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:25.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.145 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.4) 2023-11-26T07:17:25.201 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:25.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.201 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.4) 2023-11-26T07:17:25.257 INFO:teuthology.orchestra.run.smithi083.stdout:1700983036 2023-11-26T07:17:25.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.258 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.4) 2023-11-26T07:17:25.313 INFO:teuthology.orchestra.run.smithi083.stdout:1700983036 2023-11-26T07:17:25.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.314 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.7) 2023-11-26T07:17:25.370 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:25.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.370 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.7) 2023-11-26T07:17:25.427 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:25.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.427 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.7) 2023-11-26T07:17:25.483 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:25.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.484 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.7) 2023-11-26T07:17:25.540 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:25.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.541 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.7) 2023-11-26T07:17:25.597 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:25.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.598 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.7) 2023-11-26T07:17:25.653 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:25.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.654 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.7) 2023-11-26T07:17:25.711 INFO:teuthology.orchestra.run.smithi083.stdout:1700983036 2023-11-26T07:17:25.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.711 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.7) 2023-11-26T07:17:25.768 INFO:teuthology.orchestra.run.smithi083.stdout:1700983036 2023-11-26T07:17:25.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.768 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.6) 2023-11-26T07:17:25.824 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:25.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.825 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.6) 2023-11-26T07:17:25.881 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:25.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.882 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.6) 2023-11-26T07:17:25.938 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:25.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.938 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.6) 2023-11-26T07:17:25.994 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:25.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:25.995 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.6) 2023-11-26T07:17:26.051 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:26.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.051 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.6) 2023-11-26T07:17:26.107 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:26.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.108 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.6) 2023-11-26T07:17:26.165 INFO:teuthology.orchestra.run.smithi083.stdout:1700983036 2023-11-26T07:17:26.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.166 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.6) 2023-11-26T07:17:26.222 INFO:teuthology.orchestra.run.smithi083.stdout:1700983036 2023-11-26T07:17:26.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.223 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.0) 2023-11-26T07:17:26.279 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:26.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.280 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.0) 2023-11-26T07:17:26.337 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:26.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.338 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.0) 2023-11-26T07:17:26.394 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:26.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.395 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.0) 2023-11-26T07:17:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:26.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.452 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.0) 2023-11-26T07:17:26.508 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:26.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.508 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.0) 2023-11-26T07:17:26.564 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:26.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.564 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.0) 2023-11-26T07:17:26.620 INFO:teuthology.orchestra.run.smithi083.stdout:1700983035 2023-11-26T07:17:26.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.621 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.0) 2023-11-26T07:17:26.678 INFO:teuthology.orchestra.run.smithi083.stdout:1700983035 2023-11-26T07:17:26.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.678 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.3) 2023-11-26T07:17:26.734 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:26.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.735 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.3) 2023-11-26T07:17:26.792 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:26.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.792 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.3) 2023-11-26T07:17:26.848 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:26.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.849 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.3) 2023-11-26T07:17:26.905 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:26.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.906 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.3) 2023-11-26T07:17:26.961 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:26.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:26.962 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.3) 2023-11-26T07:17:27.018 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:27.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:27.019 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.3) 2023-11-26T07:17:27.075 INFO:teuthology.orchestra.run.smithi083.stdout:1700983035 2023-11-26T07:17:27.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:27.075 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.3) 2023-11-26T07:17:27.131 INFO:teuthology.orchestra.run.smithi083.stdout:1700983035 2023-11-26T07:17:27.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:27.132 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.1) 2023-11-26T07:17:27.189 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:27.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:27.190 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.1) 2023-11-26T07:17:27.246 INFO:teuthology.orchestra.run.smithi083.stdout:81b4 2023-11-26T07:17:27.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:27.246 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.1) 2023-11-26T07:17:27.302 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:27.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:27.303 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.1) 2023-11-26T07:17:27.359 INFO:teuthology.orchestra.run.smithi083.stdout:1000 2023-11-26T07:17:27.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:27.360 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.1) 2023-11-26T07:17:27.417 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:27.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:27.417 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.1) 2023-11-26T07:17:27.451 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:17:27.460 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:17:27.473 INFO:teuthology.orchestra.run.smithi083.stdout:1265 2023-11-26T07:17:27.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:27.473 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol77760/261825a1-ca56-49c4-92ea-43e4b89b2070/.snap/subvol_snap47320/subvolume_file.1) 2023-11-26T07:17:27.561 INFO:teuthology.orchestra.run.smithi083.stdout:1700983035 2023-11-26T07:17:27.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:27.562 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone67280/0ccf39ab-446a-4ab8-8ed0-2b909d339fdb/subvolume_file.1) 2023-11-26T07:17:27.619 INFO:teuthology.orchestra.run.smithi083.stdout:1700983035 2023-11-26T07:17:27.619 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol77760 fake subvol_clone24571 2023-11-26T07:17:27.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.783+0000 7f044cb15700 1 Processor -- start 2023-11-26T07:17:27.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.783+0000 7f044cb15700 1 -- start start 2023-11-26T07:17:27.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.784+0000 7f044cb15700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f044815e160 0x7f04480b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:27.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.784+0000 7f044cb15700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04480b3c60 con 0x7f04480746a0 2023-11-26T07:17:27.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.784+0000 7f044cb15700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04480b58c0 con 0x7f04480afc90 2023-11-26T07:17:27.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.784+0000 7f044cb15700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04480b4500 con 0x7f044815e160 2023-11-26T07:17:27.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.784+0000 7f0446d9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f04480746a0 0x7f044806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33814/0 (socket says 172.21.15.83:33814) 2023-11-26T07:17:27.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.784+0000 7f0446d9d700 1 -- 172.21.15.83:0/3421747830 learned_addr learned my addr 172.21.15.83:0/3421747830 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:27.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044659c700 1 --2- 172.21.15.83:0/3421747830 >> v2:172.21.15.203:3300/0 conn(0x7f044815e160 0x7f04480b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:27.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638323612 0 0) 0x7f04480b3c60 con 0x7f04480746a0 2023-11-26T07:17:27.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04300035b0 con 0x7f04480746a0 2023-11-26T07:17:27.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2390157220 0 0) 0x7f04300035b0 con 0x7f04480746a0 2023-11-26T07:17:27.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 >> v2:172.21.15.203:3300/0 conn(0x7f044815e160 msgr2=0x7f04480b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:27.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 --2- 172.21.15.83:0/3421747830 >> v2:172.21.15.203:3300/0 conn(0x7f044815e160 0x7f04480b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:27.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 >> v1:172.21.15.203:6789/0 conn(0x7f04480afc90 legacy=0x7f0448073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:27.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04480b0140 con 0x7f04480746a0 2023-11-26T07:17:27.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f04400019f0 con 0x7f04480746a0 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 >> v1:172.21.15.83:6789/0 conn(0x7f04480746a0 legacy=0x7f044806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 --2- 172.21.15.83:0/3421747830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0430003a50 0x7f0430003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 --2- 172.21.15.83:0/3421747830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0430004850 0x7f0430004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 --2- 172.21.15.83:0/3421747830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0430005210 0x7f043000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04480b4500 con 0x7f0430005210 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043000c110 con 0x7f0430003a50 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043000c2d0 con 0x7f0430004850 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f0446d9d700 1 --2- 172.21.15.83:0/3421747830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0430005210 0x7f043000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f044659c700 1 --2- 172.21.15.83:0/3421747830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0430003a50 0x7f0430003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f0445d9b700 1 --2- 172.21.15.83:0/3421747830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0430004850 0x7f0430004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f0446d9d700 1 -- 172.21.15.83:0/3421747830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0430004850 msgr2=0x7f0430004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f0446d9d700 1 --2- 172.21.15.83:0/3421747830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0430004850 0x7f0430004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f0446d9d700 1 -- 172.21.15.83:0/3421747830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0430003a50 msgr2=0x7f0430003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f0446d9d700 1 --2- 172.21.15.83:0/3421747830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0430003a50 0x7f0430003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f0446d9d700 1 -- 172.21.15.83:0/3421747830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04300035b0 con 0x7f0430005210 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f0446d9d700 1 -- 172.21.15.83:0/3421747830 reap_dead start 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.785+0000 7f0446d9d700 1 --2- 172.21.15.83:0/3421747830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0430005210 0x7f043000ba10 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0448162dd0 tx=0x7f0430005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23590018c2c73823 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.786+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04400072d0 con 0x7f0430005210 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.786+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0440007450 con 0x7f0430005210 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.786+0000 7f044559a700 1 -- 172.21.15.83:0/3421747830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440007800 con 0x7f0430005210 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.786+0000 7f044cb15700 1 -- 172.21.15.83:0/3421747830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0430005210 msgr2=0x7f043000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.786+0000 7f044cb15700 1 --2- 172.21.15.83:0/3421747830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0430005210 0x7f043000ba10 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f0448162dd0 tx=0x7f0430005880 comp rx=0 tx=0).stop 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.786+0000 7f044cb15700 1 -- 172.21.15.83:0/3421747830 shutdown_connections 2023-11-26T07:17:27.797 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.786+0000 7f044cb15700 1 --2- 172.21.15.83:0/3421747830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0430005210 0x7f043000ba10 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:27.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.786+0000 7f044cb15700 1 -- 172.21.15.83:0/3421747830 >> 172.21.15.83:0/3421747830 conn(0x7f04480d0bf0 msgr2=0x7f04480d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:27.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.786+0000 7f044cb15700 1 -- 172.21.15.83:0/3421747830 shutdown_connections 2023-11-26T07:17:27.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.786+0000 7f044cb15700 1 -- 172.21.15.83:0/3421747830 wait complete. 2023-11-26T07:17:27.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.788+0000 7f044cb15700 1 Processor -- start 2023-11-26T07:17:27.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.788+0000 7f044cb15700 1 -- start start 2023-11-26T07:17:27.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.788+0000 7f044cb15700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0430005210 0x7f04481601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:27.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.788+0000 7f044cb15700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f044815f3a0 0x7f044815a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:27.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.788+0000 7f044cb15700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448156370 0x7f04481567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:27.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.788+0000 7f044cb15700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0448155f50 con 0x7f0448156370 2023-11-26T07:17:27.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.788+0000 7f044cb15700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0448160a30 con 0x7f0430005210 2023-11-26T07:17:27.798 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.788+0000 7f044cb15700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0448160d10 con 0x7f044815f3a0 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0446d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448156370 0x7f04481567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0446d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448156370 0x7f04481567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42018/0 (socket says 172.21.15.83:42018) 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0446d9d700 1 -- 172.21.15.83:0/3199840663 learned_addr learned my addr 172.21.15.83:0/3199840663 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f044659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0430005210 0x7f04481601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0445d9b700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f044815f3a0 0x7f044815a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0446d9d700 1 -- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f044815f3a0 msgr2=0x7f044815a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0446d9d700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f044815f3a0 0x7f044815a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0446d9d700 1 -- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0430005210 msgr2=0x7f04481601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0446d9d700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0430005210 0x7f04481601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0446d9d700 1 -- 172.21.15.83:0/3199840663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0448160ff0 con 0x7f0448156370 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0446d9d700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448156370 0x7f04481567f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f044000afd0 tx=0x7f04480b4470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:27.799 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0436ffd700 1 -- 172.21.15.83:0/3199840663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440025030 con 0x7f0448156370 2023-11-26T07:17:27.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0436ffd700 1 -- 172.21.15.83:0/3199840663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0440012600 con 0x7f0448156370 2023-11-26T07:17:27.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f0436ffd700 1 -- 172.21.15.83:0/3199840663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04400129b0 con 0x7f0448156370 2023-11-26T07:17:27.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f044cb15700 1 -- 172.21.15.83:0/3199840663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04481612d0 con 0x7f0448156370 2023-11-26T07:17:27.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.789+0000 7f044cb15700 1 -- 172.21.15.83:0/3199840663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044815aec0 con 0x7f0448156370 2023-11-26T07:17:27.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.790+0000 7f044cb15700 1 -- 172.21.15.83:0/3199840663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04480b5a80 con 0x7f0448156370 2023-11-26T07:17:27.800 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.791+0000 7f0436ffd700 1 -- 172.21.15.83:0/3199840663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f0440028070 con 0x7f0448156370 2023-11-26T07:17:27.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.791+0000 7f0436ffd700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f042806b1a0 0x7f042806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:27.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.791+0000 7f0436ffd700 1 -- 172.21.15.83:0/3199840663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0440026070 con 0x7f0448156370 2023-11-26T07:17:27.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.791+0000 7f044659c700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f042806b1a0 0x7f042806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:27.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.792+0000 7f044659c700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f042806b1a0 0x7f042806d660 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f043c009be0 tx=0x7f043c009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:27.801 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:27.796+0000 7f0436ffd700 1 -- 172.21.15.83:0/3199840663 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0440055610 con 0x7f0448156370 2023-11-26T07:17:28.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.025+0000 7f044cb15700 1 -- 172.21.15.83:0/3199840663 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol77760", "snap_name": "fake", "target_sub_name": "subvol_clone24571", "target": ["mon-mgr", ""]}) v1 -- 0x7f0448160e80 con 0x7f042806b1a0 2023-11-26T07:17:28.093 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.091+0000 7f0436ffd700 1 -- 172.21.15.83:0/3199840663 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0448160e80 con 0x7f042806b1a0 2023-11-26T07:17:28.095 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 -- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f042806b1a0 msgr2=0x7f042806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f042806b1a0 0x7f042806d660 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f043c009be0 tx=0x7f043c009210 comp rx=0 tx=0).stop 2023-11-26T07:17:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 -- 172.21.15.83:0/3199840663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448156370 msgr2=0x7f04481567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448156370 0x7f04481567f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f044000afd0 tx=0x7f04480b4470 comp rx=0 tx=0).stop 2023-11-26T07:17:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 -- 172.21.15.83:0/3199840663 shutdown_connections 2023-11-26T07:17:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f042806b1a0 0x7f042806d660 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0448156370 0x7f04481567f0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0430005210 0x7f04481601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 --2- 172.21.15.83:0/3199840663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f044815f3a0 0x7f044815a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 -- 172.21.15.83:0/3199840663 >> 172.21.15.83:0/3199840663 conn(0x7f04480d0bf0 msgr2=0x7f04480d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 -- 172.21.15.83:0/3199840663 shutdown_connections 2023-11-26T07:17:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.093+0000 7f044cb15700 1 -- 172.21.15.83:0/3199840663 wait complete. 2023-11-26T07:17:28.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone24571 2023-11-26T07:17:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.229+0000 7f4620b62700 1 Processor -- start 2023-11-26T07:17:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.229+0000 7f4620b62700 1 -- start start 2023-11-26T07:17:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4620b62700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f461c0b4310 0x7f461c06bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4620b62700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f461c0ce430 0x7f461c0ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4620b62700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c06ba50 con 0x7f461c15e080 2023-11-26T07:17:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4620b62700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461c0d1200 con 0x7f461c0b4310 2023-11-26T07:17:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4620b62700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c15e460 con 0x7f461c0ce430 2023-11-26T07:17:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4619d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f461c0b4310 0x7f461c06bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f461a59c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f461c15e080 0x7f461c0d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33830/0 (socket says 172.21.15.83:33830) 2023-11-26T07:17:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f461a59c700 1 -- 172.21.15.83:0/4264026120 learned_addr learned my addr 172.21.15.83:0/4264026120 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4619d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f461c0b4310 0x7f461c06bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42024/0 (socket says 172.21.15.83:42024) 2023-11-26T07:17:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f461ad9d700 1 --2- 172.21.15.83:0/4264026120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f461c0ce430 0x7f461c0ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4619d9b700 1 -- 172.21.15.83:0/4264026120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f461c0ce430 msgr2=0x7f461c0ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4619d9b700 1 --2- 172.21.15.83:0/4264026120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f461c0ce430 0x7f461c0ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4619d9b700 1 -- 172.21.15.83:0/4264026120 >> v1:172.21.15.83:6789/0 conn(0x7f461c15e080 legacy=0x7f461c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4619d9b700 1 -- 172.21.15.83:0/4264026120 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c06b120 con 0x7f461c0b4310 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f461959a700 1 -- 172.21.15.83:0/4264026120 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1659888884 0 0) 0x7f461c06ba50 con 0x7f461c15e080 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f461ad9d700 1 --2- 172.21.15.83:0/4264026120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f461c0ce430 0x7f461c0ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f4619d9b700 1 --2- 172.21.15.83:0/4264026120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c0b4310 0x7f461c06bee0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4610009840 tx=0x7f4610009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d599c8016fdc0e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.230+0000 7f461959a700 1 -- 172.21.15.83:0/4264026120 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610011830 con 0x7f461c0b4310 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.231+0000 7f461959a700 1 -- 172.21.15.83:0/4264026120 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4610011990 con 0x7f461c0b4310 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.231+0000 7f461959a700 1 -- 172.21.15.83:0/4264026120 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610011d20 con 0x7f461c0b4310 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.231+0000 7f4620b62700 1 -- 172.21.15.83:0/4264026120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c0b4310 msgr2=0x7f461c06bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.231+0000 7f4620b62700 1 --2- 172.21.15.83:0/4264026120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c0b4310 0x7f461c06bee0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4610009840 tx=0x7f4610009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.231+0000 7f4620b62700 1 -- 172.21.15.83:0/4264026120 shutdown_connections 2023-11-26T07:17:28.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.231+0000 7f4620b62700 1 --2- 172.21.15.83:0/4264026120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f461c0ce430 0x7f461c0ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.231+0000 7f4620b62700 1 --2- 172.21.15.83:0/4264026120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c0b4310 0x7f461c06bee0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.231+0000 7f4620b62700 1 -- 172.21.15.83:0/4264026120 >> 172.21.15.83:0/4264026120 conn(0x7f461c0b3780 msgr2=0x7f461c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:28.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.231+0000 7f4620b62700 1 -- 172.21.15.83:0/4264026120 shutdown_connections 2023-11-26T07:17:28.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.231+0000 7f4620b62700 1 -- 172.21.15.83:0/4264026120 wait complete. 2023-11-26T07:17:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.233+0000 7f4620b62700 1 Processor -- start 2023-11-26T07:17:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.233+0000 7f4620b62700 1 -- start start 2023-11-26T07:17:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.233+0000 7f4620b62700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f461c0b4310 0x7f461c160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.233+0000 7f4620b62700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c0ce430 0x7f461c158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.233+0000 7f4620b62700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f461c15e080 0x7f461c0c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.233+0000 7f4620b62700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0b6410 con 0x7f461c0ce430 2023-11-26T07:17:28.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.233+0000 7f4620b62700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c1613f0 con 0x7f461c15e080 2023-11-26T07:17:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.233+0000 7f4620b62700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0c6860 con 0x7f461c0b4310 2023-11-26T07:17:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.233+0000 7f4619d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c0ce430 0x7f461c158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.233+0000 7f4619d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c0ce430 0x7f461c158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42030/0 (socket says 172.21.15.83:42030) 2023-11-26T07:17:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.233+0000 7f4619d9b700 1 -- 172.21.15.83:0/4212074775 learned_addr learned my addr 172.21.15.83:0/4212074775 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f461a59c700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f461c0b4310 0x7f461c160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f461ad9d700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f461c15e080 0x7f461c0c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f4619d9b700 1 -- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f461c0b4310 msgr2=0x7f461c160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f4619d9b700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f461c0b4310 0x7f461c160eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f4619d9b700 1 -- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f461c15e080 msgr2=0x7f461c0c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f4619d9b700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f461c15e080 0x7f461c0c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:28.239 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f4619d9b700 1 -- 172.21.15.83:0/4212074775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100094a0 con 0x7f461c0ce430 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f461ad9d700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f461c15e080 0x7f461c0c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f4619d9b700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c0ce430 0x7f461c158020 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f4610004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f460affd700 1 -- 172.21.15.83:0/4212074775 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100045f0 con 0x7f461c0ce430 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f460affd700 1 -- 172.21.15.83:0/4212074775 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461001f070 con 0x7f461c0ce430 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f4620b62700 1 -- 172.21.15.83:0/4212074775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c0b6660 con 0x7f461c0ce430 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f460affd700 1 -- 172.21.15.83:0/4212074775 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610003ae0 con 0x7f461c0ce430 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.234+0000 7f4620b62700 1 -- 172.21.15.83:0/4212074775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c0b6c20 con 0x7f461c0ce430 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.235+0000 7f4620b62700 1 -- 172.21.15.83:0/4212074775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c0b6db0 con 0x7f461c0ce430 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.235+0000 7f460affd700 1 -- 172.21.15.83:0/4212074775 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f4610003c40 con 0x7f461c0ce430 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.236+0000 7f460affd700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45fc06b1a0 0x7f45fc06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.236+0000 7f460affd700 1 -- 172.21.15.83:0/4212074775 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4610021070 con 0x7f461c0ce430 2023-11-26T07:17:28.240 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.236+0000 7f461a59c700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45fc06b1a0 0x7f45fc06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:28.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.237+0000 7f461a59c700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45fc06b1a0 0x7f45fc06d660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f460c000f60 tx=0x7f460c007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:28.242 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.240+0000 7f460affd700 1 -- 172.21.15.83:0/4212074775 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4610056250 con 0x7f461c0ce430 2023-11-26T07:17:28.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:28.475+0000 7f4620b62700 1 -- 172.21.15.83:0/4212074775 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone24571", "target": ["mon-mgr", ""]}) v1 -- 0x7f461c06e010 con 0x7f45fc06b1a0 2023-11-26T07:17:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.976+0000 7f460affd700 1 -- 172.21.15.83:0/4212074775 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+153 (secure 0 0 0) 0x7f461c06e010 con 0x7f45fc06b1a0 2023-11-26T07:17:31.980 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "status": { 2023-11-26T07:17:31.980 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "pending", 2023-11-26T07:17:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "source": { 2023-11-26T07:17:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "volume": "cephfs", 2023-11-26T07:17:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "subvolume": "subvol77760", 2023-11-26T07:17:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot": "fake" 2023-11-26T07:17:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:17:31.981 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:17:31.981 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 -- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45fc06b1a0 msgr2=0x7f45fc06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45fc06b1a0 0x7f45fc06d660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f460c000f60 tx=0x7f460c007c90 comp rx=0 tx=0).stop 2023-11-26T07:17:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 -- 172.21.15.83:0/4212074775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c0ce430 msgr2=0x7f461c158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c0ce430 0x7f461c158020 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f4610004750 comp rx=0 tx=0).stop 2023-11-26T07:17:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 -- 172.21.15.83:0/4212074775 shutdown_connections 2023-11-26T07:17:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f45fc06b1a0 0x7f45fc06d660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f461c15e080 0x7f461c0c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f461c0b4310 0x7f461c160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 --2- 172.21.15.83:0/4212074775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f461c0ce430 0x7f461c158020 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 -- 172.21.15.83:0/4212074775 >> 172.21.15.83:0/4212074775 conn(0x7f461c0b3780 msgr2=0x7f461c0cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 -- 172.21.15.83:0/4212074775 shutdown_connections 2023-11-26T07:17:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:31.979+0000 7f4620b62700 1 -- 172.21.15.83:0/4212074775 wait complete. 2023-11-26T07:17:32.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone24571 2023-11-26T07:17:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.131+0000 7f2c6847c700 1 Processor -- start 2023-11-26T07:17:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.131+0000 7f2c6847c700 1 -- start start 2023-11-26T07:17:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.132+0000 7f2c6847c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c6015e160 0x7f2c600b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.132+0000 7f2c6847c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c600afc90 0x7f2c60073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.132+0000 7f2c6847c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c600744d0 0x7f2c6006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.132+0000 7f2c6847c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c600b4500 con 0x7f2c600744d0 2023-11-26T07:17:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.132+0000 7f2c6847c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c60074960 con 0x7f2c6015e160 2023-11-26T07:17:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.132+0000 7f2c6847c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600b3c60 con 0x7f2c600afc90 2023-11-26T07:17:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.132+0000 7f2c66a19700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c600744d0 0x7f2c6006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.132+0000 7f2c66a19700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2c600744d0 0x7f2c6006d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42038/0 (socket says 172.21.15.83:42038) 2023-11-26T07:17:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.132+0000 7f2c66a19700 1 -- 172.21.15.83:0/2251207951 learned_addr learned my addr 172.21.15.83:0/2251207951 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.132+0000 7f2c66a19700 1 -- 172.21.15.83:0/2251207951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c600afc90 msgr2=0x7f2c60073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c66a19700 1 --2- 172.21.15.83:0/2251207951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c600afc90 0x7f2c60073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.132+0000 7f2c66218700 1 --2- 172.21.15.83:0/2251207951 >> v2:172.21.15.203:3300/0 conn(0x7f2c6015e160 0x7f2c600b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c66a19700 1 -- 172.21.15.83:0/2251207951 >> v2:172.21.15.203:3300/0 conn(0x7f2c6015e160 msgr2=0x7f2c600b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c66a19700 1 --2- 172.21.15.83:0/2251207951 >> v2:172.21.15.203:3300/0 conn(0x7f2c6015e160 0x7f2c600b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c66a19700 1 -- 172.21.15.83:0/2251207951 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c600b58c0 con 0x7f2c600744d0 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c66218700 1 --2- 172.21.15.83:0/2251207951 >> v2:172.21.15.203:3300/0 conn(0x7f2c6015e160 0x7f2c600b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c66a19700 1 --2- 172.21.15.83:0/2251207951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600744d0 0x7f2c6006d930 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f2c6009efa0 tx=0x7f2c5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ecd124a37c27fb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c65216700 1 -- 172.21.15.83:0/2251207951 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800e070 con 0x7f2c600744d0 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c65216700 1 -- 172.21.15.83:0/2251207951 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5800b9c0 con 0x7f2c600744d0 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c65216700 1 -- 172.21.15.83:0/2251207951 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800bd70 con 0x7f2c600744d0 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c6847c700 1 -- 172.21.15.83:0/2251207951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600744d0 msgr2=0x7f2c6006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c6847c700 1 --2- 172.21.15.83:0/2251207951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600744d0 0x7f2c6006d930 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f2c6009efa0 tx=0x7f2c5800b330 comp rx=0 tx=0).stop 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c6847c700 1 -- 172.21.15.83:0/2251207951 shutdown_connections 2023-11-26T07:17:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c6847c700 1 --2- 172.21.15.83:0/2251207951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600744d0 0x7f2c6006d930 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c6847c700 1 --2- 172.21.15.83:0/2251207951 >> v2:172.21.15.203:3300/0 conn(0x7f2c6015e160 0x7f2c600b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c6847c700 1 --2- 172.21.15.83:0/2251207951 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c600afc90 0x7f2c60073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c6847c700 1 -- 172.21.15.83:0/2251207951 >> 172.21.15.83:0/2251207951 conn(0x7f2c600d0bf0 msgr2=0x7f2c600d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c6847c700 1 -- 172.21.15.83:0/2251207951 shutdown_connections 2023-11-26T07:17:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.133+0000 7f2c6847c700 1 -- 172.21.15.83:0/2251207951 wait complete. 2023-11-26T07:17:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.135+0000 7f2c6847c700 1 Processor -- start 2023-11-26T07:17:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.135+0000 7f2c6847c700 1 -- start start 2023-11-26T07:17:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.135+0000 7f2c6847c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c600744d0 0x7f2c601f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.135+0000 7f2c6847c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600afc90 0x7f2c601602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.135+0000 7f2c6847c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c6015e160 0x7f2c601524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.135+0000 7f2c6847c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60160800 con 0x7f2c600afc90 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.135+0000 7f2c6847c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60160aa0 con 0x7f2c6015e160 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.135+0000 7f2c6847c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60160d80 con 0x7f2c600744d0 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c65a17700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600afc90 0x7f2c601602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c65a17700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600afc90 0x7f2c601602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42042/0 (socket says 172.21.15.83:42042) 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c65a17700 1 -- 172.21.15.83:0/1737949384 learned_addr learned my addr 172.21.15.83:0/1737949384 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c66218700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c600744d0 0x7f2c601f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c66a19700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c6015e160 0x7f2c601524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c65a17700 1 -- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c600744d0 msgr2=0x7f2c601f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c65a17700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c600744d0 0x7f2c601f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c65a17700 1 -- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c6015e160 msgr2=0x7f2c601524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c65a17700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c6015e160 0x7f2c601524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c65a17700 1 -- 172.21.15.83:0/1737949384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c58009b30 con 0x7f2c600afc90 2023-11-26T07:17:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c65a17700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600afc90 0x7f2c601602c0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00b530 tx=0x7f2c5c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c56ffd700 1 -- 172.21.15.83:0/1737949384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c011830 con 0x7f2c600afc90 2023-11-26T07:17:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c56ffd700 1 -- 172.21.15.83:0/1737949384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c011990 con 0x7f2c600afc90 2023-11-26T07:17:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c6847c700 1 -- 172.21.15.83:0/1737949384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c60161060 con 0x7f2c600afc90 2023-11-26T07:17:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c56ffd700 1 -- 172.21.15.83:0/1737949384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c011d40 con 0x7f2c600afc90 2023-11-26T07:17:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.136+0000 7f2c6847c700 1 -- 172.21.15.83:0/1737949384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c60152a30 con 0x7f2c600afc90 2023-11-26T07:17:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.137+0000 7f2c6847c700 1 -- 172.21.15.83:0/1737949384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c60068fb0 con 0x7f2c600afc90 2023-11-26T07:17:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.138+0000 7f2c56ffd700 1 -- 172.21.15.83:0/1737949384 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f2c5c01a400 con 0x7f2c600afc90 2023-11-26T07:17:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.138+0000 7f2c56ffd700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2c4406b1a0 0x7f2c4406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.138+0000 7f2c56ffd700 1 -- 172.21.15.83:0/1737949384 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c5c089a30 con 0x7f2c600afc90 2023-11-26T07:17:33.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.138+0000 7f2c66218700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2c4406b1a0 0x7f2c4406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.144 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.139+0000 7f2c66218700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2c4406b1a0 0x7f2c4406d660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c50006fd0 tx=0x7f2c50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:33.145 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.143+0000 7f2c56ffd700 1 -- 172.21.15.83:0/1737949384 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c5c054850 con 0x7f2c600afc90 2023-11-26T07:17:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:33.373+0000 7f2c6847c700 1 -- 172.21.15.83:0/1737949384 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone24571", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c600b4190 con 0x7f2c4406b1a0 2023-11-26T07:17:36.946 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.943+0000 7f2c56ffd700 1 -- 172.21.15.83:0/1737949384 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f2c5000c040 con 0x7f2c4406b1a0 2023-11-26T07:17:36.946 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:36.946 INFO:teuthology.orchestra.run.smithi083.stdout: "status": { 2023-11-26T07:17:36.946 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "complete" 2023-11-26T07:17:36.946 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:17:36.946 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 -- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2c4406b1a0 msgr2=0x7f2c4406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2c4406b1a0 0x7f2c4406d660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c50006fd0 tx=0x7f2c50008040 comp rx=0 tx=0).stop 2023-11-26T07:17:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 -- 172.21.15.83:0/1737949384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600afc90 msgr2=0x7f2c601602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600afc90 0x7f2c601602c0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00b530 tx=0x7f2c5c00b840 comp rx=0 tx=0).stop 2023-11-26T07:17:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 -- 172.21.15.83:0/1737949384 shutdown_connections 2023-11-26T07:17:36.949 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2c4406b1a0 0x7f2c4406d660 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c6015e160 0x7f2c601524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c600744d0 0x7f2c601f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 --2- 172.21.15.83:0/1737949384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c600afc90 0x7f2c601602c0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 -- 172.21.15.83:0/1737949384 >> 172.21.15.83:0/1737949384 conn(0x7f2c600d0bf0 msgr2=0x7f2c6015d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 -- 172.21.15.83:0/1737949384 shutdown_connections 2023-11-26T07:17:36.950 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:36.946+0000 7f2c6847c700 1 -- 172.21.15.83:0/1737949384 wait complete. 2023-11-26T07:17:36.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:36.965 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone24571/.meta) 2023-11-26T07:17:36.994 INFO:teuthology.orchestra.run.smithi083.stdout:[GLOBAL] 2023-11-26T07:17:36.994 INFO:teuthology.orchestra.run.smithi083.stdout:version = 2 2023-11-26T07:17:36.995 INFO:teuthology.orchestra.run.smithi083.stdout:type = clone 2023-11-26T07:17:36.995 INFO:teuthology.orchestra.run.smithi083.stdout:path = /volumes/_nogroup/subvol_clone24571/ebfc75e7-0598-420d-9601-5cfea5232ca1 2023-11-26T07:17:36.995 INFO:teuthology.orchestra.run.smithi083.stdout:state = complete 2023-11-26T07:17:36.995 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:36.995 INFO:teuthology.orchestra.run.smithi083.stdout:[source] 2023-11-26T07:17:36.995 INFO:teuthology.orchestra.run.smithi083.stdout:volume = cephfs 2023-11-26T07:17:36.995 INFO:teuthology.orchestra.run.smithi083.stdout:subvolume = subvol77760 2023-11-26T07:17:36.995 INFO:teuthology.orchestra.run.smithi083.stdout:snapshot = fake 2023-11-26T07:17:36.995 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:36.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol77760 subvol_snap47320 2023-11-26T07:17:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.162+0000 7fe122585700 1 Processor -- start 2023-11-26T07:17:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.162+0000 7fe122585700 1 -- start start 2023-11-26T07:17:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.163+0000 7fe122585700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c0ae330 0x7fe11c165f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.163+0000 7fe122585700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11c0cf070 con 0x7fe11c0af3a0 2023-11-26T07:17:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.163+0000 7fe122585700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11c0b0e50 con 0x7fe11c0aefc0 2023-11-26T07:17:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.163+0000 7fe122585700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c15ea90 con 0x7fe11c0ae330 2023-11-26T07:17:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.163+0000 7fe11b7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe11c0af3a0 0x7fe11c0ce960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33858/0 (socket says 172.21.15.83:33858) 2023-11-26T07:17:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.163+0000 7fe11b7fe700 1 -- 172.21.15.83:0/1994693692 learned_addr learned my addr 172.21.15.83:0/1994693692 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868500541 0 0) 0x7fe11c0cf070 con 0x7fe11c0af3a0 2023-11-26T07:17:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1040035b0 con 0x7fe11c0af3a0 2023-11-26T07:17:37.169 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe120b22700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c0ae330 0x7fe11c165f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2382058842 0 0) 0x7fe1040035b0 con 0x7fe11c0af3a0 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c0ae330 msgr2=0x7fe11c165f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c0ae330 0x7fe11c165f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 >> v1:172.21.15.203:6789/0 conn(0x7fe11c0aefc0 legacy=0x7fe11c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c0b02b0 con 0x7fe11c0af3a0 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7fe10c001670 con 0x7fe11c0af3a0 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 >> v1:172.21.15.83:6789/0 conn(0x7fe11c0af3a0 legacy=0x7fe11c0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe104003a30 0x7fe104003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 0x7fe104004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c0ae330 0x7fe104005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c15ea90 con 0x7fe104004850 2023-11-26T07:17:37.170 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10400bac0 con 0x7fe104003a30 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10400bc80 con 0x7fe11c0ae330 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11b7fe700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 0x7fe104004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11bfff700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe104003a30 0x7fe104003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe120b22700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c0ae330 0x7fe104005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11b7fe700 1 -- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c0ae330 msgr2=0x7fe104005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11b7fe700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c0ae330 0x7fe104005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11b7fe700 1 -- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe104003a30 msgr2=0x7fe104003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11b7fe700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe104003a30 0x7fe104003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11b7fe700 1 -- 172.21.15.83:0/1994693692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1040035b0 con 0x7fe104004850 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11bfff700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe104003a30 0x7fe104003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe120b22700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c0ae330 0x7fe104005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.164+0000 7fe11b7fe700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 0x7fe104004cd0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe11c072200 tx=0x7fe104005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759a67cb81c0e433 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.171 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.165+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00cd50 con 0x7fe104004850 2023-11-26T07:17:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.165+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c007ba0 con 0x7fe104004850 2023-11-26T07:17:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.165+0000 7fe11affd700 1 -- 172.21.15.83:0/1994693692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c005b60 con 0x7fe104004850 2023-11-26T07:17:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.165+0000 7fe122585700 1 -- 172.21.15.83:0/1994693692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 msgr2=0x7fe104004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.165+0000 7fe122585700 1 --2- 172.21.15.83:0/1994693692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 0x7fe104004cd0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe11c072200 tx=0x7fe104005880 comp rx=0 tx=0).stop 2023-11-26T07:17:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.165+0000 7fe120b22700 1 -- 172.21.15.83:0/1994693692 reap_dead start 2023-11-26T07:17:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.165+0000 7fe122585700 1 -- 172.21.15.83:0/1994693692 shutdown_connections 2023-11-26T07:17:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.165+0000 7fe122585700 1 -- 172.21.15.83:0/1994693692 >> 172.21.15.83:0/1994693692 conn(0x7fe11c072710 msgr2=0x7fe11c072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.165+0000 7fe122585700 1 -- 172.21.15.83:0/1994693692 shutdown_connections 2023-11-26T07:17:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.165+0000 7fe122585700 1 -- 172.21.15.83:0/1994693692 wait complete. 2023-11-26T07:17:37.172 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.166+0000 7fe122585700 1 Processor -- start 2023-11-26T07:17:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.167+0000 7fe122585700 1 -- start start 2023-11-26T07:17:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.167+0000 7fe122585700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 0x7fe11c157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.167+0000 7fe122585700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c15ebf0 0x7fe11c15f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.167+0000 7fe122585700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe11c158a80 0x7fe11c158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.167+0000 7fe122585700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c15be00 con 0x7fe104004850 2023-11-26T07:17:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.167+0000 7fe122585700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c15f5b0 con 0x7fe11c158a80 2023-11-26T07:17:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.167+0000 7fe122585700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c15a3e0 con 0x7fe11c15ebf0 2023-11-26T07:17:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.167+0000 7fe11bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 0x7fe11c157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.167+0000 7fe11bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 0x7fe11c157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42060/0 (socket says 172.21.15.83:42060) 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.167+0000 7fe11bfff700 1 -- 172.21.15.83:0/1953954012 learned_addr learned my addr 172.21.15.83:0/1953954012 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe11b7fe700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c15ebf0 0x7fe11c15f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe120b22700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe11c158a80 0x7fe11c158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe11bfff700 1 -- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c15ebf0 msgr2=0x7fe11c15f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe11bfff700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c15ebf0 0x7fe11c15f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe11bfff700 1 -- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe11c158a80 msgr2=0x7fe11c158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe11bfff700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe11c158a80 0x7fe11c158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe11bfff700 1 -- 172.21.15.83:0/1953954012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c1597d0 con 0x7fe104004850 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe11b7fe700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c15ebf0 0x7fe11c15f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe120b22700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe11c158a80 0x7fe11c158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe11bfff700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 0x7fe11c157bf0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe11000e580 tx=0x7fe11000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.176 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe118ff9700 1 -- 172.21.15.83:0/1953954012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1100052e0 con 0x7fe104004850 2023-11-26T07:17:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe118ff9700 1 -- 172.21.15.83:0/1953954012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe110005440 con 0x7fe104004850 2023-11-26T07:17:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe122585700 1 -- 172.21.15.83:0/1953954012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11c159ab0 con 0x7fe104004850 2023-11-26T07:17:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe118ff9700 1 -- 172.21.15.83:0/1953954012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1100038a0 con 0x7fe104004850 2023-11-26T07:17:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.168+0000 7fe122585700 1 -- 172.21.15.83:0/1953954012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11c15a070 con 0x7fe104004850 2023-11-26T07:17:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.169+0000 7fe122585700 1 -- 172.21.15.83:0/1953954012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11c15a230 con 0x7fe104004850 2023-11-26T07:17:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.169+0000 7fe118ff9700 1 -- 172.21.15.83:0/1953954012 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fe110003a00 con 0x7fe104004850 2023-11-26T07:17:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.170+0000 7fe118ff9700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe10806b1d0 0x7fe10806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.170+0000 7fe118ff9700 1 -- 172.21.15.83:0/1953954012 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe110088be0 con 0x7fe104004850 2023-11-26T07:17:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.170+0000 7fe11b7fe700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe10806b1d0 0x7fe10806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.171+0000 7fe11b7fe700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe10806b1d0 0x7fe10806d690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe11c072200 tx=0x7fe10c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.174+0000 7fe118ff9700 1 -- 172.21.15.83:0/1953954012 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe110053a80 con 0x7fe104004850 2023-11-26T07:17:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.409+0000 7fe122585700 1 -- 172.21.15.83:0/1953954012 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol77760", "snap_name": "subvol_snap47320", "target": ["mon-mgr", ""]}) v1 -- 0x7fe11c0b15b0 con 0x7fe10806b1d0 2023-11-26T07:17:37.429 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.427+0000 7fe118ff9700 1 -- 172.21.15.83:0/1953954012 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+207 (secure 0 0 0) 0x7fe10c00cd10 con 0x7fe10806b1d0 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi083.stdout: "created_at": "2023-11-26 07:17:16.859132", 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi083.stdout: "data_pool": "cephfs_data", 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi083.stdout: "has_pending_clones": "yes", 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi083.stdout: "pending_clones": [ 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi083.stdout: { 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subvol_clone67280" 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi083.stdout: } 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.429+0000 7fe122585700 1 -- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe10806b1d0 msgr2=0x7fe10806d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.429+0000 7fe122585700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe10806b1d0 0x7fe10806d690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe11c072200 tx=0x7fe10c007000 comp rx=0 tx=0).stop 2023-11-26T07:17:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.429+0000 7fe122585700 1 -- 172.21.15.83:0/1953954012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 msgr2=0x7fe11c157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.429+0000 7fe122585700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 0x7fe11c157bf0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe11000e580 tx=0x7fe11000e940 comp rx=0 tx=0).stop 2023-11-26T07:17:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.429+0000 7fe122585700 1 -- 172.21.15.83:0/1953954012 shutdown_connections 2023-11-26T07:17:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.429+0000 7fe122585700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fe10806b1d0 0x7fe10806d690 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.429+0000 7fe122585700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe11c158a80 0x7fe11c158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.429+0000 7fe122585700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe104004850 0x7fe11c157bf0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.429+0000 7fe122585700 1 --2- 172.21.15.83:0/1953954012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe11c15ebf0 0x7fe11c15f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.429+0000 7fe122585700 1 -- 172.21.15.83:0/1953954012 >> 172.21.15.83:0/1953954012 conn(0x7fe11c072710 msgr2=0x7fe11c06e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:37.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.430+0000 7fe122585700 1 -- 172.21.15.83:0/1953954012 shutdown_connections 2023-11-26T07:17:37.433 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.430+0000 7fe122585700 1 -- 172.21.15.83:0/1953954012 wait complete. 2023-11-26T07:17:37.445 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:17:37.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:17:37.482 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:17:37.528 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:17:37.528 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:37.529 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:37.545 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:37.546 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:37.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:37.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.761+0000 7f85cbe3c700 1 Processor -- start 2023-11-26T07:17:37.764 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.762+0000 7f85cbe3c700 1 -- start start 2023-11-26T07:17:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85cbe3c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40adf00 0x7f85c406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85cbe3c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85c406c540 0x7f85c40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85cbe3c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c40af950 con 0x7f85c41446a0 2023-11-26T07:17:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85cbe3c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c4144a80 con 0x7f85c406c540 2023-11-26T07:17:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85cbe3c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c406c9b0 con 0x7f85c40adf00 2023-11-26T07:17:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85ca3d9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85c406c540 0x7f85c40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85ca3d9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f85c406c540 0x7f85c40b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42072/0 (socket says 172.21.15.83:42072) 2023-11-26T07:17:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85ca3d9700 1 -- 172.21.15.83:0/1834385593 learned_addr learned my addr 172.21.15.83:0/1834385593 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85c93d7700 1 --2- 172.21.15.83:0/1834385593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40adf00 0x7f85c406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.767 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85ca3d9700 1 -- 172.21.15.83:0/1834385593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40adf00 msgr2=0x7f85c406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85ca3d9700 1 --2- 172.21.15.83:0/1834385593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40adf00 0x7f85c406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85ca3d9700 1 -- 172.21.15.83:0/1834385593 >> v1:172.21.15.203:6789/0 conn(0x7f85c41446a0 legacy=0x7f85c40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85ca3d9700 1 -- 172.21.15.83:0/1834385593 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40afc30 con 0x7f85c406c540 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85ca3d9700 1 --2- 172.21.15.83:0/1834385593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c406c540 0x7f85c40b3d40 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009e80 tx=0x7f85bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa9642075354dad server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.763+0000 7f85c8bd6700 1 -- 172.21.15.83:0/1834385593 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc00e070 con 0x7f85c406c540 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.764+0000 7f85c8bd6700 1 -- 172.21.15.83:0/1834385593 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc00b980 con 0x7f85c406c540 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.764+0000 7f85c8bd6700 1 -- 172.21.15.83:0/1834385593 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc00bd30 con 0x7f85c406c540 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.764+0000 7f85cbe3c700 1 -- 172.21.15.83:0/1834385593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c406c540 msgr2=0x7f85c40b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.764+0000 7f85cbe3c700 1 --2- 172.21.15.83:0/1834385593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c406c540 0x7f85c40b3d40 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009e80 tx=0x7f85bc00b330 comp rx=0 tx=0).stop 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.764+0000 7f85cbe3c700 1 -- 172.21.15.83:0/1834385593 shutdown_connections 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.764+0000 7f85cbe3c700 1 --2- 172.21.15.83:0/1834385593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c406c540 0x7f85c40b3d40 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.768 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.764+0000 7f85cbe3c700 1 --2- 172.21.15.83:0/1834385593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c40adf00 0x7f85c406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.764+0000 7f85cbe3c700 1 -- 172.21.15.83:0/1834385593 >> 172.21.15.83:0/1834385593 conn(0x7f85c4071b90 msgr2=0x7f85c4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.764+0000 7f85cbe3c700 1 -- 172.21.15.83:0/1834385593 shutdown_connections 2023-11-26T07:17:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.764+0000 7f85cbe3c700 1 -- 172.21.15.83:0/1834385593 wait complete. 2023-11-26T07:17:37.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.766+0000 7f85cbe3c700 1 Processor -- start 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.766+0000 7f85cbe3c700 1 -- start start 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85cbe3c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c406c540 0x7f85c40ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85cbe3c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c40adf00 0x7f85c41515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85cbe3c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c41446a0 0x7f85c40cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85cbe3c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40ce8d0 con 0x7f85c40adf00 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85cbe3c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c41363b0 con 0x7f85c406c540 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85cbe3c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40ce5f0 con 0x7f85c41446a0 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85c93d7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c40adf00 0x7f85c41515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85c93d7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c40adf00 0x7f85c41515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42076/0 (socket says 172.21.15.83:42076) 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85c93d7700 1 -- 172.21.15.83:0/2733933021 learned_addr learned my addr 172.21.15.83:0/2733933021 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85c9bd8700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c406c540 0x7f85c40ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85ca3d9700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c41446a0 0x7f85c40cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85c93d7700 1 -- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c41446a0 msgr2=0x7f85c40cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85c93d7700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c41446a0 0x7f85c40cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85c93d7700 1 -- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c406c540 msgr2=0x7f85c40ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85c93d7700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c406c540 0x7f85c40ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85c93d7700 1 -- 172.21.15.83:0/2733933021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc009b30 con 0x7f85c40adf00 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85c93d7700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c40adf00 0x7f85c41515b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f85c000e7e0 tx=0x7f85c000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2733933021 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0013610 con 0x7f85c40adf00 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2733933021 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c0013770 con 0x7f85c40adf00 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85cbe3c700 1 -- 172.21.15.83:0/2733933021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c4136670 con 0x7f85c40adf00 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2733933021 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0013ae0 con 0x7f85c40adf00 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.767+0000 7f85cbe3c700 1 -- 172.21.15.83:0/2733933021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c4136c30 con 0x7f85c40adf00 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.768+0000 7f85cbe3c700 1 -- 172.21.15.83:0/2733933021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c4136fa0 con 0x7f85c40adf00 2023-11-26T07:17:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.769+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2733933021 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f85c0013c40 con 0x7f85c40adf00 2023-11-26T07:17:37.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.769+0000 7f85ba7fc700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f85a806b1a0 0x7f85a806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.769+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2733933021 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85c00897d0 con 0x7f85c40adf00 2023-11-26T07:17:37.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.769+0000 7f85c9bd8700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f85a806b1a0 0x7f85a806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.770+0000 7f85c9bd8700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f85a806b1a0 0x7f85a806d660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f85c415af20 tx=0x7f85b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.776 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.774+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2733933021 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85c0054430 con 0x7f85c40adf00 2023-11-26T07:17:37.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.970+0000 7f85cbe3c700 1 -- 172.21.15.83:0/2733933021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85c40753f0 con 0x7f85c40adf00 2023-11-26T07:17:37.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.971+0000 7f85ba7fc700 1 -- 172.21.15.83:0/2733933021 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16515 (secure 0 0 0) 0x7f85c0057a50 con 0x7f85c40adf00 2023-11-26T07:17:37.974 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:37.974 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":102,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:17:03.554036+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:00.194045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:00.926172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 -- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f85a806b1a0 msgr2=0x7f85a806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f85a806b1a0 0x7f85a806d660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f85c415af20 tx=0x7f85b4009040 comp rx=0 tx=0).stop 2023-11-26T07:17:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 -- 172.21.15.83:0/2733933021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c40adf00 msgr2=0x7f85c41515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c40adf00 0x7f85c41515b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f85c000e7e0 tx=0x7f85c000eba0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 -- 172.21.15.83:0/2733933021 shutdown_connections 2023-11-26T07:17:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f85a806b1a0 0x7f85a806d660 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c41446a0 0x7f85c40cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c406c540 0x7f85c40ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 --2- 172.21.15.83:0/2733933021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85c40adf00 0x7f85c41515b0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 -- 172.21.15.83:0/2733933021 >> 172.21.15.83:0/2733933021 conn(0x7f85c4071b90 msgr2=0x7f85c40b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:37.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 -- 172.21.15.83:0/2733933021 shutdown_connections 2023-11-26T07:17:37.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:37.973+0000 7f85cbe3c700 1 -- 172.21.15.83:0/2733933021 wait complete. 2023-11-26T07:17:37.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:00.194045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:00.926172+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:37.988 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:38.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.108+0000 7f5d3df11700 1 Processor -- start 2023-11-26T07:17:38.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.108+0000 7f5d3df11700 1 -- start start 2023-11-26T07:17:38.112 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d3df11700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5d380cc580 0x7f5d3806f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d3df11700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d38070820 con 0x7f5d3815f500 2023-11-26T07:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d3df11700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3806fb80 con 0x7f5d38070b40 2023-11-26T07:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d3df11700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d380d0260 con 0x7f5d380cc580 2023-11-26T07:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d37fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5d380cc580 0x7f5d3806f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d37fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5d380cc580 0x7f5d3806f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42086/0 (socket says 172.21.15.83:42086) 2023-11-26T07:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d377fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f5d3815f500 0x7f5d3806ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33890/0 (socket says 172.21.15.83:33890) 2023-11-26T07:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d37fff700 1 -- 172.21.15.83:0/3938605596 learned_addr learned my addr 172.21.15.83:0/3938605596 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d37fff700 1 -- 172.21.15.83:0/3938605596 >> v1:172.21.15.203:6789/0 conn(0x7f5d38070b40 legacy=0x7f5d380b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d37fff700 1 -- 172.21.15.83:0/3938605596 >> v1:172.21.15.83:6789/0 conn(0x7f5d3815f500 legacy=0x7f5d3806ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d37fff700 1 -- 172.21.15.83:0/3938605596 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d380d0540 con 0x7f5d380cc580 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d367fc700 1 -- 172.21.15.83:0/3938605596 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827032350 0 0) 0x7f5d38070820 con 0x7f5d3815f500 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d37fff700 1 --2- 172.21.15.83:0/3938605596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d380cc580 0x7f5d3806f480 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009e50 tx=0x7f5d3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e89f395eb8792ee server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d367fc700 1 -- 172.21.15.83:0/3938605596 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3000e070 con 0x7f5d380cc580 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d367fc700 1 -- 172.21.15.83:0/3938605596 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3000b9c0 con 0x7f5d380cc580 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.109+0000 7f5d367fc700 1 -- 172.21.15.83:0/3938605596 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3000bd70 con 0x7f5d380cc580 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.110+0000 7f5d3df11700 1 -- 172.21.15.83:0/3938605596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d380cc580 msgr2=0x7f5d3806f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.110+0000 7f5d3df11700 1 --2- 172.21.15.83:0/3938605596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d380cc580 0x7f5d3806f480 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009e50 tx=0x7f5d3000b330 comp rx=0 tx=0).stop 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.110+0000 7f5d3df11700 1 -- 172.21.15.83:0/3938605596 shutdown_connections 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.110+0000 7f5d3df11700 1 --2- 172.21.15.83:0/3938605596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d380cc580 0x7f5d3806f480 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.110+0000 7f5d3df11700 1 -- 172.21.15.83:0/3938605596 >> 172.21.15.83:0/3938605596 conn(0x7f5d380b3710 msgr2=0x7f5d380b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.110+0000 7f5d3df11700 1 -- 172.21.15.83:0/3938605596 shutdown_connections 2023-11-26T07:17:38.115 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.110+0000 7f5d3df11700 1 -- 172.21.15.83:0/3938605596 wait complete. 2023-11-26T07:17:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.111+0000 7f5d3df11700 1 Processor -- start 2023-11-26T07:17:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.111+0000 7f5d3df11700 1 -- start start 2023-11-26T07:17:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.112+0000 7f5d3df11700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d38070b40 0x7f5d381f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.112+0000 7f5d3df11700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d380cc580 0x7f5d380c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.112+0000 7f5d3df11700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d3815f500 0x7f5d381f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.112+0000 7f5d3df11700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d381f5000 con 0x7f5d380cc580 2023-11-26T07:17:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.112+0000 7f5d3df11700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d381f1720 con 0x7f5d38070b40 2023-11-26T07:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.112+0000 7f5d3df11700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d380b4ae0 con 0x7f5d3815f500 2023-11-26T07:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.112+0000 7f5d36ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d380cc580 0x7f5d380c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.112+0000 7f5d36ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d380cc580 0x7f5d380c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42090/0 (socket says 172.21.15.83:42090) 2023-11-26T07:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.112+0000 7f5d36ffd700 1 -- 172.21.15.83:0/2882609356 learned_addr learned my addr 172.21.15.83:0/2882609356 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d377fe700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d38070b40 0x7f5d381f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d37fff700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d3815f500 0x7f5d381f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d36ffd700 1 -- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d3815f500 msgr2=0x7f5d381f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d36ffd700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d3815f500 0x7f5d381f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d36ffd700 1 -- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d38070b40 msgr2=0x7f5d381f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d36ffd700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d38070b40 0x7f5d381f11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d36ffd700 1 -- 172.21.15.83:0/2882609356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d30009b30 con 0x7f5d380cc580 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d37fff700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d3815f500 0x7f5d381f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d36ffd700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d380cc580 0x7f5d380c94f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00c4e0 tx=0x7f5d2c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d1ffff700 1 -- 172.21.15.83:0/2882609356 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0052d0 con 0x7f5d380cc580 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d1ffff700 1 -- 172.21.15.83:0/2882609356 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c005430 con 0x7f5d380cc580 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d3df11700 1 -- 172.21.15.83:0/2882609356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3815fa60 con 0x7f5d380cc580 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d1ffff700 1 -- 172.21.15.83:0/2882609356 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0038c0 con 0x7f5d380cc580 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.113+0000 7f5d3df11700 1 -- 172.21.15.83:0/2882609356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d380b50a0 con 0x7f5d380cc580 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.114+0000 7f5d3df11700 1 -- 172.21.15.83:0/2882609356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d381f5170 con 0x7f5d380cc580 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.114+0000 7f5d1ffff700 1 -- 172.21.15.83:0/2882609356 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f5d2c003a20 con 0x7f5d380cc580 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.115+0000 7f5d1ffff700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5d2406b1a0 0x7f5d2406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.115+0000 7f5d1ffff700 1 -- 172.21.15.83:0/2882609356 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d2c088f00 con 0x7f5d380cc580 2023-11-26T07:17:38.119 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.115+0000 7f5d377fe700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5d2406b1a0 0x7f5d2406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.120 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.115+0000 7f5d377fe700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5d2406b1a0 0x7f5d2406d660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5d28000f60 tx=0x7f5d28007d90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.119+0000 7f5d1ffff700 1 -- 172.21.15.83:0/2882609356 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d2c053c60 con 0x7f5d380cc580 2023-11-26T07:17:38.310 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.308+0000 7f5d3df11700 1 -- 172.21.15.83:0/2882609356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d38068fb0 con 0x7f5d380cc580 2023-11-26T07:17:38.311 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.308+0000 7f5d1ffff700 1 -- 172.21.15.83:0/2882609356 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d2c0170a0 con 0x7f5d380cc580 2023-11-26T07:17:38.311 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:17:38.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 -- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5d2406b1a0 msgr2=0x7f5d2406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.313 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5d2406b1a0 0x7f5d2406d660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5d28000f60 tx=0x7f5d28007d90 comp rx=0 tx=0).stop 2023-11-26T07:17:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 -- 172.21.15.83:0/2882609356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d380cc580 msgr2=0x7f5d380c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d380cc580 0x7f5d380c94f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00c4e0 tx=0x7f5d2c00c8a0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 -- 172.21.15.83:0/2882609356 shutdown_connections 2023-11-26T07:17:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f5d2406b1a0 0x7f5d2406d660 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d3815f500 0x7f5d381f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d38070b40 0x7f5d381f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 --2- 172.21.15.83:0/2882609356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d380cc580 0x7f5d380c94f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 -- 172.21.15.83:0/2882609356 >> 172.21.15.83:0/2882609356 conn(0x7f5d380b3710 msgr2=0x7f5d3806ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 -- 172.21.15.83:0/2882609356 shutdown_connections 2023-11-26T07:17:38.314 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.311+0000 7f5d3df11700 1 -- 172.21.15.83:0/2882609356 wait complete. 2023-11-26T07:17:38.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:17:38.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.445+0000 7f62b8441700 1 Processor -- start 2023-11-26T07:17:38.448 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.445+0000 7f62b8441700 1 -- start start 2023-11-26T07:17:38.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.446+0000 7f62b8441700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b4e00 0x7f62b00b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.446+0000 7f62b8441700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f62b00cd840 0x7f62b00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.451 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.446+0000 7f62b8441700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b00b4970 con 0x7f62b015be20 2023-11-26T07:17:38.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.446+0000 7f62b8441700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b00ce210 con 0x7f62b00cd840 2023-11-26T07:17:38.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.446+0000 7f62b8441700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00cfed0 con 0x7f62b00b4e00 2023-11-26T07:17:38.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b61dd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f62b015be20 0x7f62b00af790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33904/0 (socket says 172.21.15.83:33904) 2023-11-26T07:17:38.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b61dd700 1 -- 172.21.15.83:0/3583301135 learned_addr learned my addr 172.21.15.83:0/3583301135 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:38.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171878783 0 0) 0x7f62b00b4970 con 0x7f62b015be20 2023-11-26T07:17:38.452 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b59dc700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b4e00 0x7f62b00b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f629c0035b0 con 0x7f62b015be20 2023-11-26T07:17:38.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b69de700 1 --2- 172.21.15.83:0/3583301135 >> v2:172.21.15.203:3300/0 conn(0x7f62b00cd840 0x7f62b00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 482076454 0 0) 0x7f629c0035b0 con 0x7f62b015be20 2023-11-26T07:17:38.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b4e00 msgr2=0x7f62b00b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:38.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b4e00 0x7f62b00b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 >> v2:172.21.15.203:3300/0 conn(0x7f62b00cd840 msgr2=0x7f62b00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:38.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 --2- 172.21.15.83:0/3583301135 >> v2:172.21.15.203:3300/0 conn(0x7f62b00cd840 0x7f62b00cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b00b4360 con 0x7f62b015be20 2023-11-26T07:17:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f62a0001670 con 0x7f62b015be20 2023-11-26T07:17:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b69de700 1 --2- 172.21.15.83:0/3583301135 >> v2:172.21.15.203:3300/0 conn(0x7f62b00cd840 0x7f62b00cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 >> v1:172.21.15.83:6789/0 conn(0x7f62b015be20 legacy=0x7f62b00af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c003a00 0x7f629c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b4e00 0x7f629c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629c004d90 0x7f629c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00ce210 con 0x7f629c004d90 2023-11-26T07:17:38.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00cfed0 con 0x7f629c003a00 2023-11-26T07:17:38.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c00ba10 con 0x7f62b00b4e00 2023-11-26T07:17:38.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b69de700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629c004d90 0x7f629c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b61dd700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c003a00 0x7f629c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.447+0000 7f62b59dc700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b4e00 0x7f629c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b69de700 1 -- 172.21.15.83:0/3583301135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b4e00 msgr2=0x7f629c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:38.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b69de700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b00b4e00 0x7f629c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b69de700 1 -- 172.21.15.83:0/3583301135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c003a00 msgr2=0x7f629c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:38.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b69de700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c003a00 0x7f629c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b69de700 1 -- 172.21.15.83:0/3583301135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c0035b0 con 0x7f629c004d90 2023-11-26T07:17:38.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b69de700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629c004d90 0x7f629c005210 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007410 tx=0x7f629c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218d4ca0010eabf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8014400 con 0x7f629c004d90 2023-11-26T07:17:38.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a8014560 con 0x7f629c004d90 2023-11-26T07:17:38.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b51db700 1 -- 172.21.15.83:0/3583301135 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a80148d0 con 0x7f629c004d90 2023-11-26T07:17:38.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b8441700 1 -- 172.21.15.83:0/3583301135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629c004d90 msgr2=0x7f629c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b8441700 1 --2- 172.21.15.83:0/3583301135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f629c004d90 0x7f629c005210 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007410 tx=0x7f629c005880 comp rx=0 tx=0).stop 2023-11-26T07:17:38.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b69de700 1 -- 172.21.15.83:0/3583301135 reap_dead start 2023-11-26T07:17:38.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b8441700 1 -- 172.21.15.83:0/3583301135 shutdown_connections 2023-11-26T07:17:38.457 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.448+0000 7f62b8441700 1 -- 172.21.15.83:0/3583301135 >> 172.21.15.83:0/3583301135 conn(0x7f62b00b2050 msgr2=0x7f62b00b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.449+0000 7f62b8441700 1 -- 172.21.15.83:0/3583301135 shutdown_connections 2023-11-26T07:17:38.458 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.449+0000 7f62b8441700 1 -- 172.21.15.83:0/3583301135 wait complete. 2023-11-26T07:17:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.450+0000 7f62b8441700 1 Processor -- start 2023-11-26T07:17:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.450+0000 7f62b8441700 1 -- start start 2023-11-26T07:17:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.450+0000 7f62b8441700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c004d90 0x7f62b0150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.450+0000 7f62b8441700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b014fdd0 0x7f62b014cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.450+0000 7f62b8441700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b006e700 0x7f62b015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.450+0000 7f62b8441700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b0150250 con 0x7f62b014fdd0 2023-11-26T07:17:38.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.450+0000 7f62b8441700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b014d320 con 0x7f629c004d90 2023-11-26T07:17:38.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.450+0000 7f62b8441700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b006d3c0 con 0x7f62b006e700 2023-11-26T07:17:38.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b59dc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b014fdd0 0x7f62b014cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b59dc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b014fdd0 0x7f62b014cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42110/0 (socket says 172.21.15.83:42110) 2023-11-26T07:17:38.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b59dc700 1 -- 172.21.15.83:0/3363473476 learned_addr learned my addr 172.21.15.83:0/3363473476 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:38.462 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b61dd700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c004d90 0x7f62b0150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b69de700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b006e700 0x7f62b015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b59dc700 1 -- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b006e700 msgr2=0x7f62b015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b59dc700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b006e700 0x7f62b015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b59dc700 1 -- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c004d90 msgr2=0x7f62b0150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b59dc700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c004d90 0x7f62b0150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b59dc700 1 -- 172.21.15.83:0/3363473476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b006d660 con 0x7f62b014fdd0 2023-11-26T07:17:38.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b59dc700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b014fdd0 0x7f62b014cde0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f62ac011aa0 tx=0x7f62ac011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62a6ffd700 1 -- 172.21.15.83:0/3363473476 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac011ee0 con 0x7f62b014fdd0 2023-11-26T07:17:38.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62a6ffd700 1 -- 172.21.15.83:0/3363473476 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac00f940 con 0x7f62b014fdd0 2023-11-26T07:17:38.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b8441700 1 -- 172.21.15.83:0/3363473476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b006d940 con 0x7f62b014fdd0 2023-11-26T07:17:38.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62a6ffd700 1 -- 172.21.15.83:0/3363473476 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00fcf0 con 0x7f62b014fdd0 2023-11-26T07:17:38.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.451+0000 7f62b8441700 1 -- 172.21.15.83:0/3363473476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b006df00 con 0x7f62b014fdd0 2023-11-26T07:17:38.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.452+0000 7f62b8441700 1 -- 172.21.15.83:0/3363473476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b0068d90 con 0x7f62b014fdd0 2023-11-26T07:17:38.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.453+0000 7f62a6ffd700 1 -- 172.21.15.83:0/3363473476 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f62ac014440 con 0x7f62b014fdd0 2023-11-26T07:17:38.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.453+0000 7f62a6ffd700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f629406b1a0 0x7f629406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.453+0000 7f62a6ffd700 1 -- 172.21.15.83:0/3363473476 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62ac08a050 con 0x7f62b014fdd0 2023-11-26T07:17:38.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.453+0000 7f62b61dd700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f629406b1a0 0x7f629406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.454+0000 7f62b61dd700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f629406b1a0 0x7f629406d660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62a0000c00 tx=0x7f62a0007980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.457+0000 7f62a6ffd700 1 -- 172.21.15.83:0/3363473476 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62ac054e70 con 0x7f62b014fdd0 2023-11-26T07:17:38.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.644+0000 7f62b8441700 1 -- 172.21.15.83:0/3363473476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62b006e550 con 0x7f62b014fdd0 2023-11-26T07:17:38.647 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.645+0000 7f62a6ffd700 1 -- 172.21.15.83:0/3363473476 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62ac022020 con 0x7f62b014fdd0 2023-11-26T07:17:38.647 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:17:38.649 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 -- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f629406b1a0 msgr2=0x7f629406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f629406b1a0 0x7f629406d660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62a0000c00 tx=0x7f62a0007980 comp rx=0 tx=0).stop 2023-11-26T07:17:38.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 -- 172.21.15.83:0/3363473476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b014fdd0 msgr2=0x7f62b014cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b014fdd0 0x7f62b014cde0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f62ac011aa0 tx=0x7f62ac011db0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 -- 172.21.15.83:0/3363473476 shutdown_connections 2023-11-26T07:17:38.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f629406b1a0 0x7f629406d660 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b006e700 0x7f62b015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f629c004d90 0x7f62b0150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 --2- 172.21.15.83:0/3363473476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b014fdd0 0x7f62b014cde0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 -- 172.21.15.83:0/3363473476 >> 172.21.15.83:0/3363473476 conn(0x7f62b00b2050 msgr2=0x7f62b0159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.650 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 -- 172.21.15.83:0/3363473476 shutdown_connections 2023-11-26T07:17:38.651 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.647+0000 7f62b8441700 1 -- 172.21.15.83:0/3363473476 wait complete. 2023-11-26T07:17:38.661 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:17:38.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.783+0000 7f53bac81700 1 Processor -- start 2023-11-26T07:17:38.785 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.783+0000 7f53bac81700 1 -- start start 2023-11-26T07:17:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.783+0000 7f53bac81700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f53b406fab0 0x7f53b40ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.783+0000 7f53bac81700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f53b40cf0b0 0x7f53b415e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.783+0000 7f53bac81700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b40b55a0 con 0x7f53b4151c30 2023-11-26T07:17:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.783+0000 7f53bac81700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b4062900 con 0x7f53b40cf0b0 2023-11-26T07:17:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.783+0000 7f53bac81700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b40b52c0 con 0x7f53b406fab0 2023-11-26T07:17:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b921e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f53b40cf0b0 0x7f53b415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b8a1d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f53b4151c30 0x7f53b40ce9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33924/0 (socket says 172.21.15.83:33924) 2023-11-26T07:17:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b8a1d700 1 -- 172.21.15.83:0/2698518135 learned_addr learned my addr 172.21.15.83:0/2698518135 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b3fff700 1 --2- 172.21.15.83:0/2698518135 >> v2:172.21.15.203:3300/0 conn(0x7f53b406fab0 0x7f53b40ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b921e700 1 -- 172.21.15.83:0/2698518135 >> v2:172.21.15.203:3300/0 conn(0x7f53b406fab0 msgr2=0x7f53b40ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:38.788 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b921e700 1 --2- 172.21.15.83:0/2698518135 >> v2:172.21.15.203:3300/0 conn(0x7f53b406fab0 0x7f53b40ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b921e700 1 -- 172.21.15.83:0/2698518135 >> v1:172.21.15.83:6789/0 conn(0x7f53b4151c30 legacy=0x7f53b40ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b921e700 1 -- 172.21.15.83:0/2698518135 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4062340 con 0x7f53b40cf0b0 2023-11-26T07:17:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b37fe700 1 -- 172.21.15.83:0/2698518135 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272252810 0 0) 0x7f53b40b55a0 con 0x7f53b4151c30 2023-11-26T07:17:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b921e700 1 --2- 172.21.15.83:0/2698518135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b40cf0b0 0x7f53b415e860 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f53b40b30e0 tx=0x7f53ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f833a3aa2bb15f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b37fe700 1 -- 172.21.15.83:0/2698518135 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac00e070 con 0x7f53b40cf0b0 2023-11-26T07:17:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b37fe700 1 -- 172.21.15.83:0/2698518135 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac00bbd0 con 0x7f53b40cf0b0 2023-11-26T07:17:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.784+0000 7f53b37fe700 1 -- 172.21.15.83:0/2698518135 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac012430 con 0x7f53b40cf0b0 2023-11-26T07:17:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.785+0000 7f53bac81700 1 -- 172.21.15.83:0/2698518135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b40cf0b0 msgr2=0x7f53b415e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.785+0000 7f53bac81700 1 --2- 172.21.15.83:0/2698518135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b40cf0b0 0x7f53b415e860 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f53b40b30e0 tx=0x7f53ac00b330 comp rx=0 tx=0).stop 2023-11-26T07:17:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.785+0000 7f53bac81700 1 -- 172.21.15.83:0/2698518135 shutdown_connections 2023-11-26T07:17:38.789 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.785+0000 7f53bac81700 1 --2- 172.21.15.83:0/2698518135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b40cf0b0 0x7f53b415e860 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.785+0000 7f53bac81700 1 --2- 172.21.15.83:0/2698518135 >> v2:172.21.15.203:3300/0 conn(0x7f53b406fab0 0x7f53b40ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.785+0000 7f53bac81700 1 -- 172.21.15.83:0/2698518135 >> 172.21.15.83:0/2698518135 conn(0x7f53b40b2050 msgr2=0x7f53b40b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.785+0000 7f53bac81700 1 -- 172.21.15.83:0/2698518135 shutdown_connections 2023-11-26T07:17:38.790 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.785+0000 7f53bac81700 1 -- 172.21.15.83:0/2698518135 wait complete. 2023-11-26T07:17:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.786+0000 7f53bac81700 1 Processor -- start 2023-11-26T07:17:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.786+0000 7f53bac81700 1 -- start start 2023-11-26T07:17:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53bac81700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b406fab0 0x7f53b414c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53bac81700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b40cf0b0 0x7f53b406df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53bac81700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b4151c30 0x7f53b406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53bac81700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b41e6940 con 0x7f53b40cf0b0 2023-11-26T07:17:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53bac81700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4155bb0 con 0x7f53b406fab0 2023-11-26T07:17:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53bac81700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4157750 con 0x7f53b4151c30 2023-11-26T07:17:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53b3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b40cf0b0 0x7f53b406df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53b3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b40cf0b0 0x7f53b406df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42124/0 (socket says 172.21.15.83:42124) 2023-11-26T07:17:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53b3fff700 1 -- 172.21.15.83:0/2721593344 learned_addr learned my addr 172.21.15.83:0/2721593344 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53b8a1d700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b406fab0 0x7f53b414c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53b921e700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b4151c30 0x7f53b406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53b3fff700 1 -- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b4151c30 msgr2=0x7f53b406e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53b3fff700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b4151c30 0x7f53b406e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53b3fff700 1 -- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b406fab0 msgr2=0x7f53b414c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53b3fff700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b406fab0 0x7f53b414c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53b3fff700 1 -- 172.21.15.83:0/2721593344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac009a70 con 0x7f53b40cf0b0 2023-11-26T07:17:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.787+0000 7f53b3fff700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b40cf0b0 0x7f53b406df60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f53a000d800 tx=0x7f53a000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.788+0000 7f53b17fa700 1 -- 172.21.15.83:0/2721593344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a00125f0 con 0x7f53b40cf0b0 2023-11-26T07:17:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.788+0000 7f53b17fa700 1 -- 172.21.15.83:0/2721593344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a0012750 con 0x7f53b40cf0b0 2023-11-26T07:17:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.788+0000 7f53bac81700 1 -- 172.21.15.83:0/2721593344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b4157420 con 0x7f53b40cf0b0 2023-11-26T07:17:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.788+0000 7f53b17fa700 1 -- 172.21.15.83:0/2721593344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0012b00 con 0x7f53b40cf0b0 2023-11-26T07:17:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.788+0000 7f53bac81700 1 -- 172.21.15.83:0/2721593344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b41e6380 con 0x7f53b40cf0b0 2023-11-26T07:17:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.789+0000 7f53bac81700 1 -- 172.21.15.83:0/2721593344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b4068d90 con 0x7f53b40cf0b0 2023-11-26T07:17:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.789+0000 7f53b17fa700 1 -- 172.21.15.83:0/2721593344 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f53a0012c60 con 0x7f53b40cf0b0 2023-11-26T07:17:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.789+0000 7f53b17fa700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f539806b1a0 0x7f539806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.790+0000 7f53b17fa700 1 -- 172.21.15.83:0/2721593344 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53a0089e30 con 0x7f53b40cf0b0 2023-11-26T07:17:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.790+0000 7f53b8a1d700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f539806b1a0 0x7f539806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.790+0000 7f53b8a1d700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f539806b1a0 0x7f539806d660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f53a8000f60 tx=0x7f53a8007be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.796 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.794+0000 7f53b17fa700 1 -- 172.21.15.83:0/2721593344 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53a0054b90 con 0x7f53b40cf0b0 2023-11-26T07:17:38.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.985+0000 7f53bac81700 1 -- 172.21.15.83:0/2721593344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f53b4068fb0 con 0x7f53b40cf0b0 2023-11-26T07:17:38.988 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.985+0000 7f53b17fa700 1 -- 172.21.15.83:0/2721593344 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7f53a00581b0 con 0x7f53b40cf0b0 2023-11-26T07:17:38.988 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:17:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 -- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f539806b1a0 msgr2=0x7f539806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f539806b1a0 0x7f539806d660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f53a8000f60 tx=0x7f53a8007be0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 -- 172.21.15.83:0/2721593344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b40cf0b0 msgr2=0x7f53b406df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.991 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b40cf0b0 0x7f53b406df60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f53a000d800 tx=0x7f53a000dbc0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 -- 172.21.15.83:0/2721593344 shutdown_connections 2023-11-26T07:17:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f539806b1a0 0x7f539806d660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53b4151c30 0x7f53b406e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.992 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53b406fab0 0x7f53b414c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 --2- 172.21.15.83:0/2721593344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53b40cf0b0 0x7f53b406df60 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 -- 172.21.15.83:0/2721593344 >> 172.21.15.83:0/2721593344 conn(0x7f53b40b2050 msgr2=0x7f53b40afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 -- 172.21.15.83:0/2721593344 shutdown_connections 2023-11-26T07:17:38.993 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:38.987+0000 7f53bac81700 1 -- 172.21.15.83:0/2721593344 wait complete. 2023-11-26T07:17:39.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:17:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.121+0000 7ff00a2d1700 1 Processor -- start 2023-11-26T07:17:39.123 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.121+0000 7ff00a2d1700 1 -- start start 2023-11-26T07:17:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff00a2d1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0041446a0 0x7ff0040b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff00a2d1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0040af950 con 0x7ff00406c710 2023-11-26T07:17:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff00a2d1700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0040afc30 con 0x7ff0040adf00 2023-11-26T07:17:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff00a2d1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff004144a80 con 0x7ff0041446a0 2023-11-26T07:17:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff00886e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7ff00406c710 0x7ff0040b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33940/0 (socket says 172.21.15.83:33940) 2023-11-26T07:17:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff00886e700 1 -- 172.21.15.83:0/3853976691 learned_addr learned my addr 172.21.15.83:0/3853976691 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345549254 0 0) 0x7ff0040af950 con 0x7ff00406c710 2023-11-26T07:17:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff003fff700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0041446a0 0x7ff0040b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefec0057c0 con 0x7ff00406c710 2023-11-26T07:17:39.127 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3389142253 0 0) 0x7fefec0057c0 con 0x7ff00406c710 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0041446a0 msgr2=0x7ff0040b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff002ffd700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0041446a0 0x7ff0040b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 >> v1:172.21.15.203:6789/0 conn(0x7ff0040adf00 legacy=0x7ff00406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0040aff10 con 0x7ff00406c710 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7feffc0019f0 con 0x7ff00406c710 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.122+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 >> v1:172.21.15.83:6789/0 conn(0x7ff00406c710 legacy=0x7ff0040b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff002ffd700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 0x7fefec006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff002ffd700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefec006a60 0x7fefec006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff002ffd700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0041446a0 0x7fefec007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff004144a80 con 0x7fefec005c40 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec00dcd0 con 0x7fefec006a60 2023-11-26T07:17:39.128 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefec00de90 con 0x7ff0041446a0 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff003fff700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 0x7fefec006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff0037fe700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefec006a60 0x7fefec006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff00886e700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0041446a0 0x7fefec007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff003fff700 1 -- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0041446a0 msgr2=0x7fefec007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff003fff700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff0041446a0 0x7fefec007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff003fff700 1 -- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefec006a60 msgr2=0x7fefec006ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff003fff700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefec006a60 0x7fefec006ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff003fff700 1 -- 172.21.15.83:0/3853976691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec0057c0 con 0x7fefec005c40 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff0037fe700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefec006a60 0x7fefec006ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff003fff700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 0x7fefec006060 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7feff8009b30 tx=0x7fefec007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7938c1b01586d176 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800e6c0 con 0x7fefec005c40 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff800e820 con 0x7fefec005c40 2023-11-26T07:17:39.129 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.123+0000 7ff002ffd700 1 -- 172.21.15.83:0/3853976691 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800eb90 con 0x7fefec005c40 2023-11-26T07:17:39.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.124+0000 7ff00a2d1700 1 -- 172.21.15.83:0/3853976691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 msgr2=0x7fefec006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.124+0000 7ff00a2d1700 1 --2- 172.21.15.83:0/3853976691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 0x7fefec006060 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7feff8009b30 tx=0x7fefec007a90 comp rx=0 tx=0).stop 2023-11-26T07:17:39.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.124+0000 7ff00886e700 1 -- 172.21.15.83:0/3853976691 reap_dead start 2023-11-26T07:17:39.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.124+0000 7ff00a2d1700 1 -- 172.21.15.83:0/3853976691 shutdown_connections 2023-11-26T07:17:39.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.124+0000 7ff00a2d1700 1 -- 172.21.15.83:0/3853976691 >> 172.21.15.83:0/3853976691 conn(0x7ff004071b90 msgr2=0x7ff004071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.124+0000 7ff00a2d1700 1 -- 172.21.15.83:0/3853976691 shutdown_connections 2023-11-26T07:17:39.130 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.124+0000 7ff00a2d1700 1 -- 172.21.15.83:0/3853976691 wait complete. 2023-11-26T07:17:39.132 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.125+0000 7ff00a2d1700 1 Processor -- start 2023-11-26T07:17:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.125+0000 7ff00a2d1700 1 -- start start 2023-11-26T07:17:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff00a2d1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 0x7ff00409b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff00a2d1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff00415a220 0x7ff004159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff00a2d1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff00415e980 0x7ff00415ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff00a2d1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00415f340 con 0x7fefec005c40 2023-11-26T07:17:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff00a2d1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00409d9f0 con 0x7ff00415a220 2023-11-26T07:17:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff00a2d1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00409b950 con 0x7ff00415e980 2023-11-26T07:17:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff003fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 0x7ff00409b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff003fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 0x7ff00409b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42142/0 (socket says 172.21.15.83:42142) 2023-11-26T07:17:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff003fff700 1 -- 172.21.15.83:0/2149895662 learned_addr learned my addr 172.21.15.83:0/2149895662 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:39.133 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff0037fe700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff00415a220 0x7ff004159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff003fff700 1 -- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff00415e980 msgr2=0x7ff00415ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff003fff700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff00415e980 0x7ff00415ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff003fff700 1 -- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff00415a220 msgr2=0x7ff004159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff003fff700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff00415a220 0x7ff004159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff003fff700 1 -- 172.21.15.83:0/2149895662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00409aa00 con 0x7fefec005c40 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff0037fe700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff00415a220 0x7ff004159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff003fff700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 0x7ff00409b410 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fefec0069c0 tx=0x7feff80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.126+0000 7ff000ff9700 1 -- 172.21.15.83:0/2149895662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800e6c0 con 0x7fefec005c40 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.127+0000 7ff000ff9700 1 -- 172.21.15.83:0/2149895662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff8003ad0 con 0x7fefec005c40 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.127+0000 7ff00a2d1700 1 -- 172.21.15.83:0/2149895662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00409acc0 con 0x7fefec005c40 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.127+0000 7ff000ff9700 1 -- 172.21.15.83:0/2149895662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8003e80 con 0x7fefec005c40 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.127+0000 7ff00a2d1700 1 -- 172.21.15.83:0/2149895662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00409dc90 con 0x7fefec005c40 2023-11-26T07:17:39.134 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.128+0000 7ff00a2d1700 1 -- 172.21.15.83:0/2149895662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00409de50 con 0x7fefec005c40 2023-11-26T07:17:39.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.128+0000 7ff000ff9700 1 -- 172.21.15.83:0/2149895662 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7feff8013470 con 0x7fefec005c40 2023-11-26T07:17:39.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.128+0000 7ff000ff9700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feff006b1d0 0x7feff006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.128+0000 7ff000ff9700 1 -- 172.21.15.83:0/2149895662 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feff808b290 con 0x7fefec005c40 2023-11-26T07:17:39.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.129+0000 7ff0037fe700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feff006b1d0 0x7feff006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.129+0000 7ff0037fe700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feff006b1d0 0x7feff006d690 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feff4007fd0 tx=0x7feff4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.135 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.133+0000 7ff000ff9700 1 -- 172.21.15.83:0/2149895662 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feff8056080 con 0x7fefec005c40 2023-11-26T07:17:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.340+0000 7ff00a2d1700 1 -- 172.21.15.83:0/2149895662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff0040753f0 con 0x7fefec005c40 2023-11-26T07:17:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.340+0000 7ff000ff9700 1 -- 172.21.15.83:0/2149895662 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) v1 ==== 80+0+0 (secure 0 0 0) 0x7feff80596a0 con 0x7fefec005c40 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.342+0000 7ff00a2d1700 1 -- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feff006b1d0 msgr2=0x7feff006d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.342+0000 7ff00a2d1700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feff006b1d0 0x7feff006d690 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feff4007fd0 tx=0x7feff4009040 comp rx=0 tx=0).stop 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.342+0000 7ff00a2d1700 1 -- 172.21.15.83:0/2149895662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 msgr2=0x7ff00409b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.342+0000 7ff00a2d1700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 0x7ff00409b410 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fefec0069c0 tx=0x7feff80047c0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.342+0000 7ff00a2d1700 1 -- 172.21.15.83:0/2149895662 shutdown_connections 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.342+0000 7ff00a2d1700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7feff006b1d0 0x7feff006d690 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.342+0000 7ff00a2d1700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff00415e980 0x7ff00415ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.342+0000 7ff00a2d1700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec005c40 0x7ff00409b410 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.342+0000 7ff00a2d1700 1 --2- 172.21.15.83:0/2149895662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff00415a220 0x7ff004159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.342+0000 7ff00a2d1700 1 -- 172.21.15.83:0/2149895662 >> 172.21.15.83:0/2149895662 conn(0x7ff004071b90 msgr2=0x7ff00406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.342+0000 7ff00a2d1700 1 -- 172.21.15.83:0/2149895662 shutdown_connections 2023-11-26T07:17:39.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.343+0000 7ff00a2d1700 1 -- 172.21.15.83:0/2149895662 wait complete. 2023-11-26T07:17:39.347 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:17:39.357 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:17:39.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:39.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:17:39.432 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:17:39.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:39.432 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:17:39.458 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:17:39.458 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:39.476 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:17:39.477 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:39.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:39.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.654+0000 7f0539ede700 1 Processor -- start 2023-11-26T07:17:39.656 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.654+0000 7f0539ede700 1 -- start start 2023-11-26T07:17:39.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f0539ede700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053406c710 0x7f05340b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f0539ede700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05340ae430 con 0x7f05340af4d0 2023-11-26T07:17:39.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f0539ede700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05340ae710 con 0x7f053415e080 2023-11-26T07:17:39.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f0539ede700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05340af0b0 con 0x7f053406c710 2023-11-26T07:17:39.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f0533fff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f05340af4d0 0x7f053406e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33960/0 (socket says 172.21.15.83:33960) 2023-11-26T07:17:39.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f0533fff700 1 -- 172.21.15.83:0/1470070175 learned_addr learned my addr 172.21.15.83:0/1470070175 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:39.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f0532ffd700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053406c710 0x7f05340b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.661 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375604249 0 0) 0x7f05340ae430 con 0x7f05340af4d0 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f051c0035b0 con 0x7f05340af4d0 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406275357 0 0) 0x7f05340ae710 con 0x7f053415e080 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05340ae430 con 0x7f053415e080 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1826617024 0 0) 0x7f051c0035b0 con 0x7f05340af4d0 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053406c710 msgr2=0x7f05340b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053406c710 0x7f05340b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 >> v1:172.21.15.203:6789/0 conn(0x7f053415e080 legacy=0x7f053406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340ae9f0 con 0x7f05340af4d0 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f052c0019f0 con 0x7f05340af4d0 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f0532ffd700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053406c710 0x7f05340b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 >> v1:172.21.15.83:6789/0 conn(0x7f05340af4d0 legacy=0x7f053406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.662 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 0x7f051c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f051c004850 0x7f051c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053406c710 0x7f051c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05340af0b0 con 0x7f051c003bc0 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c00ba40 con 0x7f051c004850 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051c00bc00 con 0x7f053406c710 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f05337fe700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 0x7f051c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.655+0000 7f0533fff700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053406c710 0x7f051c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f0532ffd700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f051c004850 0x7f051c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f05337fe700 1 -- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053406c710 msgr2=0x7f051c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f05337fe700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053406c710 0x7f051c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f05337fe700 1 -- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f051c004850 msgr2=0x7f051c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f05337fe700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f051c004850 0x7f051c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f05337fe700 1 -- 172.21.15.83:0/1470070175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05240094a0 con 0x7f051c003bc0 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f05337fe700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 0x7f051c003fc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f0528007fd0 tx=0x7f051c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86234222510ad8f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528001430 con 0x7f051c003bc0 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05280052d0 con 0x7f051c003bc0 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f05327fc700 1 -- 172.21.15.83:0/1470070175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528001d40 con 0x7f051c003bc0 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f0539ede700 1 -- 172.21.15.83:0/1470070175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 msgr2=0x7f051c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f0539ede700 1 --2- 172.21.15.83:0/1470070175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 0x7f051c003fc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f0528007fd0 tx=0x7f051c005880 comp rx=0 tx=0).stop 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.656+0000 7f0533fff700 1 -- 172.21.15.83:0/1470070175 reap_dead start 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.657+0000 7f0539ede700 1 -- 172.21.15.83:0/1470070175 shutdown_connections 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.657+0000 7f0539ede700 1 -- 172.21.15.83:0/1470070175 >> 172.21.15.83:0/1470070175 conn(0x7f0534071b90 msgr2=0x7f0534071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.657+0000 7f0539ede700 1 -- 172.21.15.83:0/1470070175 shutdown_connections 2023-11-26T07:17:39.664 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.657+0000 7f0539ede700 1 -- 172.21.15.83:0/1470070175 wait complete. 2023-11-26T07:17:39.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.658+0000 7f0539ede700 1 Processor -- start 2023-11-26T07:17:39.665 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.658+0000 7f0539ede700 1 -- start start 2023-11-26T07:17:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.659+0000 7f0539ede700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 0x7f053415b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.659+0000 7f0539ede700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f053415a530 0x7f053415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.659+0000 7f0539ede700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053415a990 0x7f053415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.659+0000 7f0539ede700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05341f3840 con 0x7f051c003bc0 2023-11-26T07:17:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.659+0000 7f0539ede700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053415b960 con 0x7f053415a530 2023-11-26T07:17:39.667 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.659+0000 7f0539ede700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05340ce670 con 0x7f053415a990 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.659+0000 7f05337fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 0x7f053415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.659+0000 7f05337fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 0x7f053415b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42160/0 (socket says 172.21.15.83:42160) 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.659+0000 7f05337fe700 1 -- 172.21.15.83:0/971000344 learned_addr learned my addr 172.21.15.83:0/971000344 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f0532ffd700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f053415a530 0x7f053415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f0533fff700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053415a990 0x7f053415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f05337fe700 1 -- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053415a990 msgr2=0x7f053415c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f05337fe700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053415a990 0x7f053415c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f05337fe700 1 -- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f053415a530 msgr2=0x7f053415bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f05337fe700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f053415a530 0x7f053415bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f05337fe700 1 -- 172.21.15.83:0/971000344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340ce950 con 0x7f051c003bc0 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f0532ffd700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f053415a530 0x7f053415bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:39.668 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f05337fe700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 0x7f053415b420 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f0528004000 tx=0x7f05280012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f051bfff700 1 -- 172.21.15.83:0/971000344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528001d40 con 0x7f051c003bc0 2023-11-26T07:17:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f051bfff700 1 -- 172.21.15.83:0/971000344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05280013d0 con 0x7f051c003bc0 2023-11-26T07:17:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f0539ede700 1 -- 172.21.15.83:0/971000344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05340cec30 con 0x7f051c003bc0 2023-11-26T07:17:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f051bfff700 1 -- 172.21.15.83:0/971000344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528007260 con 0x7f051c003bc0 2023-11-26T07:17:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.660+0000 7f0539ede700 1 -- 172.21.15.83:0/971000344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0534160480 con 0x7f051c003bc0 2023-11-26T07:17:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.661+0000 7f0539ede700 1 -- 172.21.15.83:0/971000344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05340cf150 con 0x7f051c003bc0 2023-11-26T07:17:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.662+0000 7f051bfff700 1 -- 172.21.15.83:0/971000344 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f05280073c0 con 0x7f051c003bc0 2023-11-26T07:17:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.662+0000 7f051bfff700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f052006b1a0 0x7f052006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.662+0000 7f051bfff700 1 -- 172.21.15.83:0/971000344 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f052808a980 con 0x7f051c003bc0 2023-11-26T07:17:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.662+0000 7f0532ffd700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f052006b1a0 0x7f052006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.669 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.663+0000 7f0532ffd700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f052006b1a0 0x7f052006d660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0524009df0 tx=0x7f0524008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.670 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.666+0000 7f051bfff700 1 -- 172.21.15.83:0/971000344 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0528055890 con 0x7f051c003bc0 2023-11-26T07:17:39.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.859+0000 7f0539ede700 1 -- 172.21.15.83:0/971000344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05340753f0 con 0x7f051c003bc0 2023-11-26T07:17:39.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.860+0000 7f051bfff700 1 -- 172.21.15.83:0/971000344 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16515 (secure 0 0 0) 0x7f0528017090 con 0x7f051c003bc0 2023-11-26T07:17:39.863 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:39.863 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":102,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:17:03.554036+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:00.194045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:00.926172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 -- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f052006b1a0 msgr2=0x7f052006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f052006b1a0 0x7f052006d660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0524009df0 tx=0x7f0524008040 comp rx=0 tx=0).stop 2023-11-26T07:17:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 -- 172.21.15.83:0/971000344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 msgr2=0x7f053415b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 0x7f053415b420 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f0528004000 tx=0x7f05280012d0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 -- 172.21.15.83:0/971000344 shutdown_connections 2023-11-26T07:17:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f052006b1a0 0x7f052006d660 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053415a990 0x7f053415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f051c003bc0 0x7f053415b420 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 --2- 172.21.15.83:0/971000344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f053415a530 0x7f053415bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 -- 172.21.15.83:0/971000344 >> 172.21.15.83:0/971000344 conn(0x7f0534071b90 msgr2=0x7f05340b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 -- 172.21.15.83:0/971000344 shutdown_connections 2023-11-26T07:17:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.863+0000 7f0539ede700 1 -- 172.21.15.83:0/971000344 wait complete. 2023-11-26T07:17:39.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:00.194045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:00.926172+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:39.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:40.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.997+0000 7f2ae66f6700 1 Processor -- start 2023-11-26T07:17:40.000 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.997+0000 7f2ae66f6700 1 -- start start 2023-11-26T07:17:40.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.998+0000 7f2ae66f6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ae00d45d0 0x7f2ae00b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.998+0000 7f2ae66f6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae00d39d0 0x7f2ae00b7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.998+0000 7f2ae66f6700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae00d2930 con 0x7f2ae015e140 2023-11-26T07:17:40.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.998+0000 7f2ae66f6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae015e520 con 0x7f2ae00d45d0 2023-11-26T07:17:40.002 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.998+0000 7f2ae66f6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae00d2c10 con 0x7f2ae00d39d0 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.998+0000 7f2adf7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ae00d45d0 0x7f2ae00b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.998+0000 7f2adffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2ae015e140 0x7f2ae00d3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33976/0 (socket says 172.21.15.83:33976) 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.998+0000 7f2adffff700 1 -- 172.21.15.83:0/2476033201 learned_addr learned my addr 172.21.15.83:0/2476033201 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.998+0000 7f2adf7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ae00d45d0 0x7f2ae00b70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42170/0 (socket says 172.21.15.83:42170) 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.998+0000 7f2ae4c93700 1 --2- 172.21.15.83:0/2476033201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae00d39d0 0x7f2ae00b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.999+0000 7f2adf7fe700 1 -- 172.21.15.83:0/2476033201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae00d39d0 msgr2=0x7f2ae00b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.999+0000 7f2adf7fe700 1 --2- 172.21.15.83:0/2476033201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae00d39d0 0x7f2ae00b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.999+0000 7f2adf7fe700 1 -- 172.21.15.83:0/2476033201 >> v1:172.21.15.83:6789/0 conn(0x7f2ae015e140 legacy=0x7f2ae00d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.999+0000 7f2adf7fe700 1 -- 172.21.15.83:0/2476033201 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae00d2ef0 con 0x7f2ae00d45d0 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.999+0000 7f2adeffd700 1 -- 172.21.15.83:0/2476033201 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666064162 0 0) 0x7f2ae00d2930 con 0x7f2ae015e140 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.999+0000 7f2adf7fe700 1 --2- 172.21.15.83:0/2476033201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae00d45d0 0x7f2ae00b70d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f2ae015e660 tx=0x7f2ad0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d6bfc15435e816 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.999+0000 7f2adeffd700 1 -- 172.21.15.83:0/2476033201 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0004020 con 0x7f2ae00d45d0 2023-11-26T07:17:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.999+0000 7f2adeffd700 1 -- 172.21.15.83:0/2476033201 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad0004180 con 0x7f2ae00d45d0 2023-11-26T07:17:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.999+0000 7f2adeffd700 1 -- 172.21.15.83:0/2476033201 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0004530 con 0x7f2ae00d45d0 2023-11-26T07:17:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.999+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2476033201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae00d45d0 msgr2=0x7f2ae00b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:39.999+0000 7f2ae66f6700 1 --2- 172.21.15.83:0/2476033201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae00d45d0 0x7f2ae00b70d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f2ae015e660 tx=0x7f2ad0009b20 comp rx=0 tx=0).stop 2023-11-26T07:17:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.000+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2476033201 shutdown_connections 2023-11-26T07:17:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.000+0000 7f2ae66f6700 1 --2- 172.21.15.83:0/2476033201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae00d39d0 0x7f2ae00b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.000+0000 7f2ae66f6700 1 --2- 172.21.15.83:0/2476033201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae00d45d0 0x7f2ae00b70d0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.000+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2476033201 >> 172.21.15.83:0/2476033201 conn(0x7f2ae006b410 msgr2=0x7f2ae006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.000+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2476033201 shutdown_connections 2023-11-26T07:17:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.000+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2476033201 wait complete. 2023-11-26T07:17:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.001+0000 7f2ae66f6700 1 Processor -- start 2023-11-26T07:17:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.001+0000 7f2ae66f6700 1 -- start start 2023-11-26T07:17:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.002+0000 7f2ae66f6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae00d39d0 0x7f2ae014f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.002+0000 7f2ae66f6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae00d45d0 0x7f2ae00ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.002+0000 7f2ae66f6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ae015e140 0x7f2ae00cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.002+0000 7f2ae66f6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae014f930 con 0x7f2ae00d45d0 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.002+0000 7f2ae66f6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae00cb7d0 con 0x7f2ae015e140 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.002+0000 7f2ae66f6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae00b5070 con 0x7f2ae00d39d0 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.002+0000 7f2adf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae00d45d0 0x7f2ae00ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.002+0000 7f2adf7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae00d45d0 0x7f2ae00ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42176/0 (socket says 172.21.15.83:42176) 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.002+0000 7f2adf7fe700 1 -- 172.21.15.83:0/2257330126 learned_addr learned my addr 172.21.15.83:0/2257330126 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.002+0000 7f2ae4c93700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ae015e140 0x7f2ae00cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2adf7fe700 1 -- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae00d39d0 msgr2=0x7f2ae014f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2adffff700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae00d39d0 0x7f2ae014f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2adf7fe700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae00d39d0 0x7f2ae014f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2adf7fe700 1 -- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ae015e140 msgr2=0x7f2ae00cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2adf7fe700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ae015e140 0x7f2ae00cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2adf7fe700 1 -- 172.21.15.83:0/2257330126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad00094a0 con 0x7f2ae00d45d0 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2ae4c93700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ae015e140 0x7f2ae00cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2adffff700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae00d39d0 0x7f2ae014f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2adf7fe700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae00d45d0 0x7f2ae00ccfe0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009b60 tx=0x7f2ad0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2adcff9700 1 -- 172.21.15.83:0/2257330126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0011e60 con 0x7f2ae00d45d0 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2adcff9700 1 -- 172.21.15.83:0/2257330126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad00036a0 con 0x7f2ae00d45d0 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2adcff9700 1 -- 172.21.15.83:0/2257330126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0003a30 con 0x7f2ae00d45d0 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2257330126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae00cb180 con 0x7f2ae00d45d0 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.003+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2257330126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae01f4690 con 0x7f2ae00d45d0 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.004+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2257330126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae00cafe0 con 0x7f2ae00d45d0 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.004+0000 7f2adcff9700 1 -- 172.21.15.83:0/2257330126 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f2ad0003b90 con 0x7f2ae00d45d0 2023-11-26T07:17:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.005+0000 7f2adcff9700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2acc06b1a0 0x7f2acc06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.005+0000 7f2adcff9700 1 -- 172.21.15.83:0/2257330126 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ad008b010 con 0x7f2ae00d45d0 2023-11-26T07:17:40.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.005+0000 7f2adffff700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2acc06b1a0 0x7f2acc06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.010 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.005+0000 7f2adffff700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2acc06b1a0 0x7f2acc06d660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4000f60 tx=0x7f2ad4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.009+0000 7f2adcff9700 1 -- 172.21.15.83:0/2257330126 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ae00cafe0 con 0x7f2ae00d45d0 2023-11-26T07:17:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.201+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2257330126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ae00cafe0 con 0x7f2ae00d45d0 2023-11-26T07:17:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.201+0000 7f2adcff9700 1 -- 172.21.15.83:0/2257330126 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ae00cafe0 con 0x7f2ae00d45d0 2023-11-26T07:17:40.204 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:17:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2acc06b1a0 msgr2=0x7f2acc06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2acc06b1a0 0x7f2acc06d660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4000f60 tx=0x7f2ad4009040 comp rx=0 tx=0).stop 2023-11-26T07:17:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2257330126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae00d45d0 msgr2=0x7f2ae00ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae00d45d0 0x7f2ae00ccfe0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009b60 tx=0x7f2ad0004750 comp rx=0 tx=0).stop 2023-11-26T07:17:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2257330126 shutdown_connections 2023-11-26T07:17:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2acc06b1a0 0x7f2acc06d660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ae015e140 0x7f2ae00cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2ae00d39d0 0x7f2ae014f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 --2- 172.21.15.83:0/2257330126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae00d45d0 0x7f2ae00ccfe0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2257330126 >> 172.21.15.83:0/2257330126 conn(0x7f2ae006b410 msgr2=0x7f2ae006b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:40.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2257330126 shutdown_connections 2023-11-26T07:17:40.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.204+0000 7f2ae66f6700 1 -- 172.21.15.83:0/2257330126 wait complete. 2023-11-26T07:17:40.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:17:40.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.338+0000 7f9d07c0a700 1 Processor -- start 2023-11-26T07:17:40.341 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.338+0000 7f9d07c0a700 1 -- start start 2023-11-26T07:17:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.339+0000 7f9d07c0a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9d0015e160 0x7f9d000b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.339+0000 7f9d07c0a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d000b3c60 con 0x7f9d000afc90 2023-11-26T07:17:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.339+0000 7f9d07c0a700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d000b58c0 con 0x7f9d000746a0 2023-11-26T07:17:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.339+0000 7f9d07c0a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d000b4500 con 0x7f9d0015e160 2023-11-26T07:17:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.339+0000 7f9d051a5700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9d000afc90 0x7f9d00073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:33990/0 (socket says 172.21.15.83:33990) 2023-11-26T07:17:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.339+0000 7f9d051a5700 1 -- 172.21.15.83:0/3210847626 learned_addr learned my addr 172.21.15.83:0/3210847626 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d059a6700 1 --2- 172.21.15.83:0/3210847626 >> v2:172.21.15.203:3300/0 conn(0x7f9d0015e160 0x7f9d000b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935181784 0 0) 0x7f9d000b3c60 con 0x7f9d000afc90 2023-11-26T07:17:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cec0035b0 con 0x7f9d000afc90 2023-11-26T07:17:40.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3064650208 0 0) 0x7f9cec0035b0 con 0x7f9d000afc90 2023-11-26T07:17:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 >> v2:172.21.15.203:3300/0 conn(0x7f9d0015e160 msgr2=0x7f9d000b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 --2- 172.21.15.83:0/3210847626 >> v2:172.21.15.203:3300/0 conn(0x7f9d0015e160 0x7f9d000b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 >> v1:172.21.15.203:6789/0 conn(0x7f9d000746a0 legacy=0x7f9d0006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d000b0140 con 0x7f9d000afc90 2023-11-26T07:17:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f9cfc001670 con 0x7f9d000afc90 2023-11-26T07:17:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 >> v1:172.21.15.83:6789/0 conn(0x7f9d000afc90 legacy=0x7f9d00073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cec003a00 0x7f9cec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cec004400 0x7f9cec0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cec0050d0 0x7f9cec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d000b4500 con 0x7f9cec003a00 2023-11-26T07:17:40.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec00c030 con 0x7f9cec004400 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec00c1f0 con 0x7f9cec0050d0 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d059a6700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cec003a00 0x7f9cec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d051a5700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cec004400 0x7f9cec0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d061a7700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cec0050d0 0x7f9cec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d059a6700 1 -- 172.21.15.83:0/3210847626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cec0050d0 msgr2=0x7f9cec00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d059a6700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cec0050d0 0x7f9cec00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d059a6700 1 -- 172.21.15.83:0/3210847626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cec004400 msgr2=0x7f9cec0048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d059a6700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cec004400 0x7f9cec0048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d059a6700 1 -- 172.21.15.83:0/3210847626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0035b0 con 0x7f9cec003a00 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d061a7700 1 -- 172.21.15.83:0/3210847626 reap_dead start 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.340+0000 7f9d051a5700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9cec004400 0x7f9cec0048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:40.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.341+0000 7f9d059a6700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cec003a00 0x7f9cec003de0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009b60 tx=0x7f9cec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f715bc618c748a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.341+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000e6c0 con 0x7f9cec003a00 2023-11-26T07:17:40.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.341+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf000e820 con 0x7f9cec003a00 2023-11-26T07:17:40.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.341+0000 7f9d049a4700 1 -- 172.21.15.83:0/3210847626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000eb90 con 0x7f9cec003a00 2023-11-26T07:17:40.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.341+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3210847626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cec003a00 msgr2=0x7f9cec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.341+0000 7f9d07c0a700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cec003a00 0x7f9cec003de0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0009b60 tx=0x7f9cec005880 comp rx=0 tx=0).stop 2023-11-26T07:17:40.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.341+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3210847626 shutdown_connections 2023-11-26T07:17:40.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.341+0000 7f9d07c0a700 1 --2- 172.21.15.83:0/3210847626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cec003a00 0x7f9cec003de0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.341+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3210847626 >> 172.21.15.83:0/3210847626 conn(0x7f9d000d0bf0 msgr2=0x7f9d000d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:40.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.341+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3210847626 shutdown_connections 2023-11-26T07:17:40.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.341+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3210847626 wait complete. 2023-11-26T07:17:40.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.342+0000 7f9d07c0a700 1 Processor -- start 2023-11-26T07:17:40.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.342+0000 7f9d07c0a700 1 -- start start 2023-11-26T07:17:40.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.343+0000 7f9d07c0a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cec003a00 0x7f9d0014e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.343+0000 7f9d07c0a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d001612d0 0x7f9d0015f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.343+0000 7f9d07c0a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015fcb0 0x7f9d00071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.343+0000 7f9d07c0a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00160e40 con 0x7f9d0015fcb0 2023-11-26T07:17:40.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.343+0000 7f9d07c0a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00160230 con 0x7f9d001612d0 2023-11-26T07:17:40.351 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.343+0000 7f9d07c0a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d00160510 con 0x7f9cec003a00 2023-11-26T07:17:40.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.343+0000 7f9d061a7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015fcb0 0x7f9d00071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.343+0000 7f9d061a7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015fcb0 0x7f9d00071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42196/0 (socket says 172.21.15.83:42196) 2023-11-26T07:17:40.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.343+0000 7f9d061a7700 1 -- 172.21.15.83:0/3425241394 learned_addr learned my addr 172.21.15.83:0/3425241394 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:40.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.343+0000 7f9d051a5700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d001612d0 0x7f9d0015f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.343+0000 7f9d059a6700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cec003a00 0x7f9d0014e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.344+0000 7f9d061a7700 1 -- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cec003a00 msgr2=0x7f9d0014e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.344+0000 7f9d061a7700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cec003a00 0x7f9d0014e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.344+0000 7f9d061a7700 1 -- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d001612d0 msgr2=0x7f9d0015f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.344+0000 7f9d061a7700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d001612d0 0x7f9d0015f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.344+0000 7f9d061a7700 1 -- 172.21.15.83:0/3425241394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d001607f0 con 0x7f9d0015fcb0 2023-11-26T07:17:40.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.344+0000 7f9d061a7700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015fcb0 0x7f9d00071bb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9d000d1b40 tx=0x7f9cf800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.344+0000 7f9cf67fc700 1 -- 172.21.15.83:0/3425241394 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8003c10 con 0x7f9d0015fcb0 2023-11-26T07:17:40.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.344+0000 7f9cf67fc700 1 -- 172.21.15.83:0/3425241394 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf8003d70 con 0x7f9d0015fcb0 2023-11-26T07:17:40.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.344+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3425241394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d00160ad0 con 0x7f9d0015fcb0 2023-11-26T07:17:40.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.344+0000 7f9cf67fc700 1 -- 172.21.15.83:0/3425241394 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8005310 con 0x7f9d0015fcb0 2023-11-26T07:17:40.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.344+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3425241394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d00158b20 con 0x7f9d0015fcb0 2023-11-26T07:17:40.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.345+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3425241394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d000b4640 con 0x7f9d0015fcb0 2023-11-26T07:17:40.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.345+0000 7f9cf67fc700 1 -- 172.21.15.83:0/3425241394 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f9cf8005470 con 0x7f9d0015fcb0 2023-11-26T07:17:40.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.346+0000 7f9cf67fc700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9ce406b1a0 0x7f9ce406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.346+0000 7f9cf67fc700 1 -- 172.21.15.83:0/3425241394 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cf8089af0 con 0x7f9d0015fcb0 2023-11-26T07:17:40.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.346+0000 7f9d059a6700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9ce406b1a0 0x7f9ce406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.346+0000 7f9d059a6700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9ce406b1a0 0x7f9ce406d660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00090b0 tx=0x7f9cf001f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.350+0000 7f9cf67fc700 1 -- 172.21.15.83:0/3425241394 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9cf8054850 con 0x7f9d0015fcb0 2023-11-26T07:17:40.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.540+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3425241394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d00068fb0 con 0x7f9d0015fcb0 2023-11-26T07:17:40.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.540+0000 7f9cf67fc700 1 -- 172.21.15.83:0/3425241394 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9cf8057e70 con 0x7f9d0015fcb0 2023-11-26T07:17:40.543 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:17:40.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9ce406b1a0 msgr2=0x7f9ce406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9ce406b1a0 0x7f9ce406d660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00090b0 tx=0x7f9cf001f040 comp rx=0 tx=0).stop 2023-11-26T07:17:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3425241394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015fcb0 msgr2=0x7f9d00071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015fcb0 0x7f9d00071bb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f9d000d1b40 tx=0x7f9cf800ee70 comp rx=0 tx=0).stop 2023-11-26T07:17:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3425241394 shutdown_connections 2023-11-26T07:17:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f9ce406b1a0 0x7f9ce406d660 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0015fcb0 0x7f9d00071bb0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9cec003a00 0x7f9d0014e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 --2- 172.21.15.83:0/3425241394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d001612d0 0x7f9d0015f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3425241394 >> 172.21.15.83:0/3425241394 conn(0x7f9d000d0bf0 msgr2=0x7f9d0006ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3425241394 shutdown_connections 2023-11-26T07:17:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.543+0000 7f9d07c0a700 1 -- 172.21.15.83:0/3425241394 wait complete. 2023-11-26T07:17:40.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:17:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.677+0000 7feff69c7700 1 Processor -- start 2023-11-26T07:17:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.677+0000 7feff69c7700 1 -- start start 2023-11-26T07:17:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.678+0000 7feff69c7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff015e160 0x7feff00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.678+0000 7feff69c7700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feff00afc90 0x7feff0073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.678+0000 7feff69c7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff00b3c60 con 0x7feff00744d0 2023-11-26T07:17:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.678+0000 7feff69c7700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff00b4500 con 0x7feff00afc90 2023-11-26T07:17:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.678+0000 7feff69c7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff0074960 con 0x7feff015e160 2023-11-26T07:17:40.686 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.678+0000 7fefef7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feff00afc90 0x7feff0073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.678+0000 7fefef7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feff00afc90 0x7feff0073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42200/0 (socket says 172.21.15.83:42200) 2023-11-26T07:17:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.678+0000 7fefef7fe700 1 -- 172.21.15.83:0/1437900488 learned_addr learned my addr 172.21.15.83:0/1437900488 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.679+0000 7fefeffff700 1 --2- 172.21.15.83:0/1437900488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff015e160 0x7feff00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.679+0000 7fefef7fe700 1 -- 172.21.15.83:0/1437900488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff015e160 msgr2=0x7feff00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.679+0000 7fefef7fe700 1 --2- 172.21.15.83:0/1437900488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff015e160 0x7feff00b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.679+0000 7fefef7fe700 1 -- 172.21.15.83:0/1437900488 >> v1:172.21.15.203:6789/0 conn(0x7feff00744d0 legacy=0x7feff006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.679+0000 7fefef7fe700 1 -- 172.21.15.83:0/1437900488 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff00b58c0 con 0x7feff00afc90 2023-11-26T07:17:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.679+0000 7fefeffff700 1 --2- 172.21.15.83:0/1437900488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff015e160 0x7feff00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.679+0000 7fefef7fe700 1 --2- 172.21.15.83:0/1437900488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00afc90 0x7feff0073f90 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fefe00098f0 tx=0x7fefe0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3d9e3405c7275c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.679+0000 7fefeeffd700 1 -- 172.21.15.83:0/1437900488 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0004230 con 0x7feff00afc90 2023-11-26T07:17:40.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.679+0000 7fefeeffd700 1 -- 172.21.15.83:0/1437900488 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0004390 con 0x7feff00afc90 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.679+0000 7fefeeffd700 1 -- 172.21.15.83:0/1437900488 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe00036a0 con 0x7feff00afc90 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.680+0000 7feff69c7700 1 -- 172.21.15.83:0/1437900488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00afc90 msgr2=0x7feff0073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.680+0000 7feff69c7700 1 --2- 172.21.15.83:0/1437900488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00afc90 0x7feff0073f90 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fefe00098f0 tx=0x7fefe0009c00 comp rx=0 tx=0).stop 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.680+0000 7feff69c7700 1 -- 172.21.15.83:0/1437900488 shutdown_connections 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.680+0000 7feff69c7700 1 --2- 172.21.15.83:0/1437900488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff015e160 0x7feff00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.680+0000 7feff69c7700 1 --2- 172.21.15.83:0/1437900488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00afc90 0x7feff0073f90 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.680+0000 7feff69c7700 1 -- 172.21.15.83:0/1437900488 >> 172.21.15.83:0/1437900488 conn(0x7feff00d0bf0 msgr2=0x7feff00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.680+0000 7feff69c7700 1 -- 172.21.15.83:0/1437900488 shutdown_connections 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.680+0000 7feff69c7700 1 -- 172.21.15.83:0/1437900488 wait complete. 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.681+0000 7feff69c7700 1 Processor -- start 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.681+0000 7feff69c7700 1 -- start start 2023-11-26T07:17:40.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7feff69c7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff00744d0 0x7feff014b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7feff69c7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00afc90 0x7feff00ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7feff69c7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff015e160 0x7feff00cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7feff69c7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00cb180 con 0x7feff00afc90 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7feff69c7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff014bf60 con 0x7feff015e160 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7feff69c7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff00c6cc0 con 0x7feff00744d0 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefef7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00afc90 0x7feff00ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefef7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00afc90 0x7feff00ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42208/0 (socket says 172.21.15.83:42208) 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefef7fe700 1 -- 172.21.15.83:0/2690409494 learned_addr learned my addr 172.21.15.83:0/2690409494 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefeffff700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff00744d0 0x7feff014b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7feff4f64700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff015e160 0x7feff00cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefef7fe700 1 -- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff00744d0 msgr2=0x7feff014b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefef7fe700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff00744d0 0x7feff014b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefef7fe700 1 -- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff015e160 msgr2=0x7feff00cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefef7fe700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff015e160 0x7feff00cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefef7fe700 1 -- 172.21.15.83:0/2690409494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe0009550 con 0x7feff00afc90 2023-11-26T07:17:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefeffff700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff00744d0 0x7feff014b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7feff4f64700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff015e160 0x7feff00cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefef7fe700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00afc90 0x7feff00ca700 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7feff0072ed0 tx=0x7fefe0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefecff9700 1 -- 172.21.15.83:0/2690409494 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0003e70 con 0x7feff00afc90 2023-11-26T07:17:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.682+0000 7fefecff9700 1 -- 172.21.15.83:0/2690409494 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe001f070 con 0x7feff00afc90 2023-11-26T07:17:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.683+0000 7feff69c7700 1 -- 172.21.15.83:0/2690409494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff00c6fa0 con 0x7feff00afc90 2023-11-26T07:17:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.683+0000 7fefecff9700 1 -- 172.21.15.83:0/2690409494 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe000b8c0 con 0x7feff00afc90 2023-11-26T07:17:40.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.683+0000 7feff69c7700 1 -- 172.21.15.83:0/2690409494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff00c7560 con 0x7feff00afc90 2023-11-26T07:17:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.683+0000 7feff69c7700 1 -- 172.21.15.83:0/2690409494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff00b54a0 con 0x7feff00afc90 2023-11-26T07:17:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.684+0000 7fefecff9700 1 -- 172.21.15.83:0/2690409494 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fefe0021030 con 0x7feff00afc90 2023-11-26T07:17:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.684+0000 7fefecff9700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fefdc06b1a0 0x7fefdc06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.685+0000 7fefecff9700 1 -- 172.21.15.83:0/2690409494 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefe008b150 con 0x7feff00afc90 2023-11-26T07:17:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.685+0000 7fefeffff700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fefdc06b1a0 0x7fefdc06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.685+0000 7fefeffff700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fefdc06b1a0 0x7fefdc06d660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fefe4008910 tx=0x7fefe4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.689+0000 7fefecff9700 1 -- 172.21.15.83:0/2690409494 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fefe0055e60 con 0x7feff00afc90 2023-11-26T07:17:40.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.880+0000 7feff69c7700 1 -- 172.21.15.83:0/2690409494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feff00b54a0 con 0x7feff00afc90 2023-11-26T07:17:40.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.880+0000 7fefecff9700 1 -- 172.21.15.83:0/2690409494 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7fefe0059480 con 0x7feff00afc90 2023-11-26T07:17:40.883 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:17:40.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 -- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fefdc06b1a0 msgr2=0x7fefdc06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fefdc06b1a0 0x7fefdc06d660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fefe4008910 tx=0x7fefe4008040 comp rx=0 tx=0).stop 2023-11-26T07:17:40.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 -- 172.21.15.83:0/2690409494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00afc90 msgr2=0x7feff00ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00afc90 0x7feff00ca700 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7feff0072ed0 tx=0x7fefe0004750 comp rx=0 tx=0).stop 2023-11-26T07:17:40.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 -- 172.21.15.83:0/2690409494 shutdown_connections 2023-11-26T07:17:40.885 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fefdc06b1a0 0x7fefdc06d660 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feff015e160 0x7feff00cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feff00744d0 0x7feff014b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 --2- 172.21.15.83:0/2690409494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feff00afc90 0x7feff00ca700 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 -- 172.21.15.83:0/2690409494 >> 172.21.15.83:0/2690409494 conn(0x7feff00d0bf0 msgr2=0x7feff00ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:40.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 -- 172.21.15.83:0/2690409494 shutdown_connections 2023-11-26T07:17:40.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:40.882+0000 7feff69c7700 1 -- 172.21.15.83:0/2690409494 wait complete. 2023-11-26T07:17:40.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:17:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.014+0000 7f74852d7700 1 Processor -- start 2023-11-26T07:17:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.015+0000 7f74852d7700 1 -- start start 2023-11-26T07:17:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.015+0000 7f74852d7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800b3370 0x7f74801604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.015+0000 7f74852d7700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74800b22d0 con 0x7f74800d1c70 2023-11-26T07:17:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.015+0000 7f74852d7700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74800b25b0 con 0x7f748015bdd0 2023-11-26T07:17:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.015+0000 7f74852d7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74800b2f50 con 0x7f74800b3370 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.015+0000 7f747f7fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f74800d1c70 0x7f7480165ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34026/0 (socket says 172.21.15.83:34026) 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.015+0000 7f747f7fe700 1 -- 172.21.15.83:0/794779185 learned_addr learned my addr 172.21.15.83:0/794779185 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747e7fc700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800b3370 0x7f74801604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393798987 0 0) 0x7f74800b22d0 con 0x7f74800d1c70 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f746c0035b0 con 0x7f74800d1c70 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 542642548 0 0) 0x7f746c0035b0 con 0x7f74800d1c70 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800b3370 msgr2=0x7f74801604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800b3370 0x7f74801604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 >> v1:172.21.15.203:6789/0 conn(0x7f748015bdd0 legacy=0x7f74800b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74800b2890 con 0x7f74800d1c70 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f74700019f0 con 0x7f74800d1c70 2023-11-26T07:17:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 >> v1:172.21.15.83:6789/0 conn(0x7f74800d1c70 legacy=0x7f7480165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f746c003a30 0x7f746c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800b3370 0x7f746c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f746c004d90 0x7f746c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800b2f50 con 0x7f746c004d90 2023-11-26T07:17:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c00ba10 con 0x7f746c003a30 2023-11-26T07:17:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c00bbd0 con 0x7f74800b3370 2023-11-26T07:17:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747f7fe700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f746c004d90 0x7f746c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747effd700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f746c003a30 0x7f746c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747e7fc700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800b3370 0x7f746c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747f7fe700 1 -- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800b3370 msgr2=0x7f746c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747f7fe700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800b3370 0x7f746c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747f7fe700 1 -- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f746c003a30 msgr2=0x7f746c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747f7fe700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f746c003a30 0x7f746c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747f7fe700 1 -- 172.21.15.83:0/794779185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c0035b0 con 0x7f746c004d90 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747effd700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f746c003a30 0x7f746c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.016+0000 7f747e7fc700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800b3370 0x7f746c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.017+0000 7f747f7fe700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f746c004d90 0x7f746c005210 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f747000a830 tx=0x7f746c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf467ca5bc6111e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.017+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470012600 con 0x7f746c004d90 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.017+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7470012780 con 0x7f746c004d90 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.017+0000 7f747dffb700 1 -- 172.21.15.83:0/794779185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470012b30 con 0x7f746c004d90 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.017+0000 7f74852d7700 1 -- 172.21.15.83:0/794779185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f746c004d90 msgr2=0x7f746c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.017+0000 7f74852d7700 1 --2- 172.21.15.83:0/794779185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f746c004d90 0x7f746c005210 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f747000a830 tx=0x7f746c005880 comp rx=0 tx=0).stop 2023-11-26T07:17:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.017+0000 7f747f7fe700 1 -- 172.21.15.83:0/794779185 reap_dead start 2023-11-26T07:17:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.017+0000 7f74852d7700 1 -- 172.21.15.83:0/794779185 shutdown_connections 2023-11-26T07:17:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.017+0000 7f74852d7700 1 -- 172.21.15.83:0/794779185 >> 172.21.15.83:0/794779185 conn(0x7f74800731a0 msgr2=0x7f748009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.017+0000 7f74852d7700 1 -- 172.21.15.83:0/794779185 shutdown_connections 2023-11-26T07:17:41.024 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.017+0000 7f74852d7700 1 -- 172.21.15.83:0/794779185 wait complete. 2023-11-26T07:17:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.019+0000 7f74852d7700 1 Processor -- start 2023-11-26T07:17:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.019+0000 7f74852d7700 1 -- start start 2023-11-26T07:17:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f74852d7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74800cd980 0x7f7480156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f74852d7700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800ca300 0x7f7480157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f74852d7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74800b63f0 0x7f74800b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f74852d7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800b6900 con 0x7f74800b63f0 2023-11-26T07:17:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f74852d7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800b6be0 con 0x7f74800cd980 2023-11-26T07:17:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f74852d7700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74800b6ec0 con 0x7f74800ca300 2023-11-26T07:17:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f747f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74800b63f0 0x7f74800b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.027 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f747f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74800b63f0 0x7f74800b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42230/0 (socket says 172.21.15.83:42230) 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f747f7fe700 1 -- 172.21.15.83:0/3483645810 learned_addr learned my addr 172.21.15.83:0/3483645810 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f747effd700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74800cd980 0x7f7480156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f747e7fc700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800ca300 0x7f7480157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f747f7fe700 1 -- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800ca300 msgr2=0x7f7480157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f747f7fe700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800ca300 0x7f7480157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f747f7fe700 1 -- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74800cd980 msgr2=0x7f7480156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f747f7fe700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74800cd980 0x7f7480156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f747f7fe700 1 -- 172.21.15.83:0/3483645810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74800b71a0 con 0x7f74800b63f0 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.020+0000 7f747f7fe700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74800b63f0 0x7f74800b48a0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7470012000 tx=0x7f7470001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.021+0000 7f74777fe700 1 -- 172.21.15.83:0/3483645810 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74700019b0 con 0x7f74800b63f0 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.021+0000 7f74777fe700 1 -- 172.21.15.83:0/3483645810 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7470012610 con 0x7f74800b63f0 2023-11-26T07:17:41.028 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.021+0000 7f74852d7700 1 -- 172.21.15.83:0/3483645810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74800b7480 con 0x7f74800b63f0 2023-11-26T07:17:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.021+0000 7f74777fe700 1 -- 172.21.15.83:0/3483645810 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747001e630 con 0x7f74800b63f0 2023-11-26T07:17:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.021+0000 7f74852d7700 1 -- 172.21.15.83:0/3483645810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74800b5c80 con 0x7f74800b63f0 2023-11-26T07:17:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.022+0000 7f74852d7700 1 -- 172.21.15.83:0/3483645810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7480067f60 con 0x7f74800b63f0 2023-11-26T07:17:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.022+0000 7f74777fe700 1 -- 172.21.15.83:0/3483645810 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f747001e790 con 0x7f74800b63f0 2023-11-26T07:17:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.022+0000 7f74777fe700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f746006b1a0 0x7f746006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.022+0000 7f74777fe700 1 -- 172.21.15.83:0/3483645810 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f747008b9f0 con 0x7f74800b63f0 2023-11-26T07:17:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.023+0000 7f747effd700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f746006b1a0 0x7f746006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.029 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.023+0000 7f747effd700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f746006b1a0 0x7f746006d660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f74801f52a0 tx=0x7f7478009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.030 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.027+0000 7f74777fe700 1 -- 172.21.15.83:0/3483645810 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7470056760 con 0x7f74800b63f0 2023-11-26T07:17:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.238+0000 7f74852d7700 1 -- 172.21.15.83:0/3483645810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f74800b7c60 con 0x7f74800b63f0 2023-11-26T07:17:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.239+0000 7f74777fe700 1 -- 172.21.15.83:0/3483645810 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7470059d80 con 0x7f74800b63f0 2023-11-26T07:17:41.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 -- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f746006b1a0 msgr2=0x7f746006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.243 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f746006b1a0 0x7f746006d660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f74801f52a0 tx=0x7f7478009040 comp rx=0 tx=0).stop 2023-11-26T07:17:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 -- 172.21.15.83:0/3483645810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74800b63f0 msgr2=0x7f74800b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74800b63f0 0x7f74800b48a0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7470012000 tx=0x7f7470001d80 comp rx=0 tx=0).stop 2023-11-26T07:17:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 -- 172.21.15.83:0/3483645810 shutdown_connections 2023-11-26T07:17:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f746006b1a0 0x7f746006d660 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74800b63f0 0x7f74800b48a0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74800cd980 0x7f7480156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 --2- 172.21.15.83:0/3483645810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74800ca300 0x7f7480157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 -- 172.21.15.83:0/3483645810 >> 172.21.15.83:0/3483645810 conn(0x7f74800731a0 msgr2=0x7f748015a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 -- 172.21.15.83:0/3483645810 shutdown_connections 2023-11-26T07:17:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.241+0000 7f74852d7700 1 -- 172.21.15.83:0/3483645810 wait complete. 2023-11-26T07:17:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:listed 0 entries 2023-11-26T07:17:41.255 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:17:41.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:41.255 DEBUG:teuthology.orchestra.run.smithi203:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:17:41.308 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:17:41.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:41.308 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:17:41.327 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:41.327 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:41.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:17:41.345 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:17:41.345 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:41.385 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:17:41.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:41.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.503+0000 7fa92d9c6700 1 Processor -- start 2023-11-26T07:17:41.506 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.503+0000 7fa92d9c6700 1 -- start start 2023-11-26T07:17:41.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa92d9c6700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa92815e0f0 0x7fa9280b9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa92d9c6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9280b2b60 0x7fa9280ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa92d9c6700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa9280cf0d0 0x7fa928161bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa92d9c6700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9280cf530 con 0x7fa9280cf0d0 2023-11-26T07:17:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa92d9c6700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92806bc00 con 0x7fa92815e0f0 2023-11-26T07:17:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa92d9c6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92815e4d0 con 0x7fa9280b2b60 2023-11-26T07:17:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa9277fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa9280cf0d0 0x7fa928161bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa9277fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa9280cf0d0 0x7fa928161bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42238/0 (socket says 172.21.15.83:42238) 2023-11-26T07:17:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa9277fe700 1 -- 172.21.15.83:0/3681594213 learned_addr learned my addr 172.21.15.83:0/3681594213 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa926ffd700 1 --2- 172.21.15.83:0/3681594213 >> v2:172.21.15.203:3300/0 conn(0x7fa92815e0f0 0x7fa9280b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa9277fe700 1 -- 172.21.15.83:0/3681594213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9280b2b60 msgr2=0x7fa9280ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa9267fc700 1 --2- 172.21.15.83:0/3681594213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9280b2b60 0x7fa9280ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa9277fe700 1 --2- 172.21.15.83:0/3681594213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9280b2b60 0x7fa9280ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa9277fe700 1 -- 172.21.15.83:0/3681594213 >> v2:172.21.15.203:3300/0 conn(0x7fa92815e0f0 msgr2=0x7fa9280b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa9277fe700 1 --2- 172.21.15.83:0/3681594213 >> v2:172.21.15.203:3300/0 conn(0x7fa92815e0f0 0x7fa9280b9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa9277fe700 1 -- 172.21.15.83:0/3681594213 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92806aff0 con 0x7fa9280cf0d0 2023-11-26T07:17:41.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa926ffd700 1 --2- 172.21.15.83:0/3681594213 >> v2:172.21.15.203:3300/0 conn(0x7fa92815e0f0 0x7fa9280b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:41.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa9277fe700 1 --2- 172.21.15.83:0/3681594213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9280cf0d0 0x7fa928161bc0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa92808c270 tx=0x7fa92000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1943cdfb0001571f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa925ffb700 1 -- 172.21.15.83:0/3681594213 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92000e070 con 0x7fa9280cf0d0 2023-11-26T07:17:41.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa925ffb700 1 -- 172.21.15.83:0/3681594213 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92000b9c0 con 0x7fa9280cf0d0 2023-11-26T07:17:41.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.504+0000 7fa925ffb700 1 -- 172.21.15.83:0/3681594213 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92000bd70 con 0x7fa9280cf0d0 2023-11-26T07:17:41.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.505+0000 7fa92d9c6700 1 -- 172.21.15.83:0/3681594213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9280cf0d0 msgr2=0x7fa928161bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.505+0000 7fa92d9c6700 1 --2- 172.21.15.83:0/3681594213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9280cf0d0 0x7fa928161bc0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa92808c270 tx=0x7fa92000b330 comp rx=0 tx=0).stop 2023-11-26T07:17:41.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.505+0000 7fa92d9c6700 1 -- 172.21.15.83:0/3681594213 shutdown_connections 2023-11-26T07:17:41.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.505+0000 7fa92d9c6700 1 --2- 172.21.15.83:0/3681594213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9280cf0d0 0x7fa928161bc0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.505+0000 7fa92d9c6700 1 --2- 172.21.15.83:0/3681594213 >> v2:172.21.15.203:3300/0 conn(0x7fa92815e0f0 0x7fa9280b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.505+0000 7fa92d9c6700 1 --2- 172.21.15.83:0/3681594213 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9280b2b60 0x7fa9280ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.505+0000 7fa92d9c6700 1 -- 172.21.15.83:0/3681594213 >> 172.21.15.83:0/3681594213 conn(0x7fa9280b5ce0 msgr2=0x7fa9280b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.505+0000 7fa92d9c6700 1 -- 172.21.15.83:0/3681594213 shutdown_connections 2023-11-26T07:17:41.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.505+0000 7fa92d9c6700 1 -- 172.21.15.83:0/3681594213 wait complete. 2023-11-26T07:17:41.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.506+0000 7fa92d9c6700 1 Processor -- start 2023-11-26T07:17:41.511 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.506+0000 7fa92d9c6700 1 -- start start 2023-11-26T07:17:41.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa92d9c6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9280b2b60 0x7fa9280d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa92d9c6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9280cf0d0 0x7fa9280b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa92d9c6700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa92815e0f0 0x7fa92815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa92d9c6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280c84b0 con 0x7fa9280cf0d0 2023-11-26T07:17:41.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa92d9c6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281f7470 con 0x7fa9280b2b60 2023-11-26T07:17:41.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa92d9c6700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281f7750 con 0x7fa92815e0f0 2023-11-26T07:17:41.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa9267fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9280cf0d0 0x7fa9280b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa9267fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9280cf0d0 0x7fa9280b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42242/0 (socket says 172.21.15.83:42242) 2023-11-26T07:17:41.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa9267fc700 1 -- 172.21.15.83:0/415701817 learned_addr learned my addr 172.21.15.83:0/415701817 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:41.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa926ffd700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9280b2b60 0x7fa9280d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa9267fc700 1 -- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa92815e0f0 msgr2=0x7fa92815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa9267fc700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa92815e0f0 0x7fa92815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa9267fc700 1 -- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9280b2b60 msgr2=0x7fa9280d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa9267fc700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9280b2b60 0x7fa9280d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa9267fc700 1 -- 172.21.15.83:0/415701817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa920009b30 con 0x7fa9280cf0d0 2023-11-26T07:17:41.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.507+0000 7fa926ffd700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9280b2b60 0x7fa9280d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:41.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.508+0000 7fa9267fc700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9280cf0d0 0x7fa9280b4010 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00c7f0 tx=0x7fa91c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.508+0000 7fa90ffff700 1 -- 172.21.15.83:0/415701817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c011800 con 0x7fa9280cf0d0 2023-11-26T07:17:41.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.508+0000 7fa90ffff700 1 -- 172.21.15.83:0/415701817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c011960 con 0x7fa9280cf0d0 2023-11-26T07:17:41.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.508+0000 7fa90ffff700 1 -- 172.21.15.83:0/415701817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c011d10 con 0x7fa9280cf0d0 2023-11-26T07:17:41.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.508+0000 7fa92d9c6700 1 -- 172.21.15.83:0/415701817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9281f7a30 con 0x7fa9280cf0d0 2023-11-26T07:17:41.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.508+0000 7fa92d9c6700 1 -- 172.21.15.83:0/415701817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9280d2910 con 0x7fa9280cf0d0 2023-11-26T07:17:41.516 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.509+0000 7fa92d9c6700 1 -- 172.21.15.83:0/415701817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92815e610 con 0x7fa9280cf0d0 2023-11-26T07:17:41.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.509+0000 7fa90ffff700 1 -- 172.21.15.83:0/415701817 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fa91c018360 con 0x7fa9280cf0d0 2023-11-26T07:17:41.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.510+0000 7fa90ffff700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa91406b1a0 0x7fa91406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.510+0000 7fa90ffff700 1 -- 172.21.15.83:0/415701817 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa91c0896d0 con 0x7fa9280cf0d0 2023-11-26T07:17:41.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.510+0000 7fa926ffd700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa91406b1a0 0x7fa91406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.510+0000 7fa926ffd700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa91406b1a0 0x7fa91406d660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa9180089b0 tx=0x7fa918008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.514+0000 7fa90ffff700 1 -- 172.21.15.83:0/415701817 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa91c054320 con 0x7fa9280cf0d0 2023-11-26T07:17:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.747+0000 7fa92d9c6700 1 -- 172.21.15.83:0/415701817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa92815e7c0 con 0x7fa9280cf0d0 2023-11-26T07:17:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.747+0000 7fa90ffff700 1 -- 172.21.15.83:0/415701817 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fa92815e7c0 con 0x7fa9280cf0d0 2023-11-26T07:17:41.751 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:41.751 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/136760159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":136760159},{"type":"v1","addr":"172.21.15.203:6834","nonce":136760159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3723980912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":3723980912},{"type":"v1","addr":"172.21.15.203:6835","nonce":3723980912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3900154869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3900154869},{"type":"v1","addr":"172.21.15.83:6833","nonce":3900154869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:01.974687+0000","modified":"2023-11-26T07:17:04.985222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7551":{"gid":7551,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3327659486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3327659486},{"type":"v1","addr":"172.21.15.83:6835","nonce":3327659486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:17:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 -- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa91406b1a0 msgr2=0x7fa91406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa91406b1a0 0x7fa91406d660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa9180089b0 tx=0x7fa918008040 comp rx=0 tx=0).stop 2023-11-26T07:17:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 -- 172.21.15.83:0/415701817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9280cf0d0 msgr2=0x7fa9280b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9280cf0d0 0x7fa9280b4010 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00c7f0 tx=0x7fa91c00cbb0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 -- 172.21.15.83:0/415701817 shutdown_connections 2023-11-26T07:17:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa91406b1a0 0x7fa91406d660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa92815e0f0 0x7fa92815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9280b2b60 0x7fa9280d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 --2- 172.21.15.83:0/415701817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9280cf0d0 0x7fa9280b4010 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 -- 172.21.15.83:0/415701817 >> 172.21.15.83:0/415701817 conn(0x7fa9280b5ce0 msgr2=0x7fa9280b60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 -- 172.21.15.83:0/415701817 shutdown_connections 2023-11-26T07:17:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.750+0000 7fa92d9c6700 1 -- 172.21.15.83:0/415701817 wait complete. 2023-11-26T07:17:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 70 2023-11-26T07:17:41.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.881+0000 7f7f66d10700 1 Processor -- start 2023-11-26T07:17:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.882+0000 7f7f66d10700 1 -- start start 2023-11-26T07:17:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.882+0000 7f7f66d10700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7f60158670 0x7f7f600622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.882+0000 7f7f66d10700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60158a50 0x7f7f600c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.882+0000 7f7f66d10700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f600cfff0 con 0x7f7f60162a30 2023-11-26T07:17:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.882+0000 7f7f66d10700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f600cfd10 con 0x7f7f60158670 2023-11-26T07:17:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.882+0000 7f7f66d10700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f60161f50 con 0x7f7f60158a50 2023-11-26T07:17:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f652ad700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7f60162a30 0x7f7f600d0230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34060/0 (socket says 172.21.15.83:34060) 2023-11-26T07:17:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f652ad700 1 -- 172.21.15.83:0/1648747060 learned_addr learned my addr 172.21.15.83:0/1648747060 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f5ffff700 1 --2- 172.21.15.83:0/1648747060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60158a50 0x7f7f600c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f64aac700 1 --2- 172.21.15.83:0/1648747060 >> v2:172.21.15.203:3300/0 conn(0x7f7f60158670 0x7f7f600622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f5f7fe700 1 -- 172.21.15.83:0/1648747060 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290304622 0 0) 0x7f7f600cfff0 con 0x7f7f60162a30 2023-11-26T07:17:41.887 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f5f7fe700 1 -- 172.21.15.83:0/1648747060 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f500035b0 con 0x7f7f60162a30 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f5ffff700 1 -- 172.21.15.83:0/1648747060 >> v2:172.21.15.203:3300/0 conn(0x7f7f60158670 msgr2=0x7f7f600622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f5ffff700 1 --2- 172.21.15.83:0/1648747060 >> v2:172.21.15.203:3300/0 conn(0x7f7f60158670 0x7f7f600622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f5ffff700 1 -- 172.21.15.83:0/1648747060 >> v1:172.21.15.83:6789/0 conn(0x7f7f60162a30 legacy=0x7f7f600d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f5ffff700 1 -- 172.21.15.83:0/1648747060 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f60162230 con 0x7f7f60158a50 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f64aac700 1 --2- 172.21.15.83:0/1648747060 >> v2:172.21.15.203:3300/0 conn(0x7f7f60158670 0x7f7f600622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f5ffff700 1 --2- 172.21.15.83:0/1648747060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60158a50 0x7f7f600c98b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009840 tx=0x7f7f4c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c082d2e297cede2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f5f7fe700 1 -- 172.21.15.83:0/1648747060 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c011830 con 0x7f7f60158a50 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f5f7fe700 1 -- 172.21.15.83:0/1648747060 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c011990 con 0x7f7f60158a50 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.883+0000 7f7f5f7fe700 1 -- 172.21.15.83:0/1648747060 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c011d20 con 0x7f7f60158a50 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.884+0000 7f7f66d10700 1 -- 172.21.15.83:0/1648747060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60158a50 msgr2=0x7f7f600c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.884+0000 7f7f66d10700 1 --2- 172.21.15.83:0/1648747060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60158a50 0x7f7f600c98b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009840 tx=0x7f7f4c009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.884+0000 7f7f66d10700 1 -- 172.21.15.83:0/1648747060 shutdown_connections 2023-11-26T07:17:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.884+0000 7f7f66d10700 1 --2- 172.21.15.83:0/1648747060 >> v2:172.21.15.203:3300/0 conn(0x7f7f60158670 0x7f7f600622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.884+0000 7f7f66d10700 1 --2- 172.21.15.83:0/1648747060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60158a50 0x7f7f600c98b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.884+0000 7f7f66d10700 1 -- 172.21.15.83:0/1648747060 >> 172.21.15.83:0/1648747060 conn(0x7f7f600b57b0 msgr2=0x7f7f600b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.884+0000 7f7f66d10700 1 -- 172.21.15.83:0/1648747060 shutdown_connections 2023-11-26T07:17:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.884+0000 7f7f66d10700 1 -- 172.21.15.83:0/1648747060 wait complete. 2023-11-26T07:17:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.885+0000 7f7f66d10700 1 Processor -- start 2023-11-26T07:17:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.885+0000 7f7f66d10700 1 -- start start 2023-11-26T07:17:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f66d10700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f60158670 0x7f7f60155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f66d10700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f60158a50 0x7f7f60160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f66d10700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60162a30 0x7f7f600b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f66d10700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600ad590 con 0x7f7f60158670 2023-11-26T07:17:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f66d10700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f60160b30 con 0x7f7f60158a50 2023-11-26T07:17:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f66d10700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f60160e10 con 0x7f7f60162a30 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f64aac700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f60158670 0x7f7f60155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f64aac700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f60158670 0x7f7f60155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42254/0 (socket says 172.21.15.83:42254) 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f64aac700 1 -- 172.21.15.83:0/2471823662 learned_addr learned my addr 172.21.15.83:0/2471823662 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f5ffff700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f60158a50 0x7f7f60160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f652ad700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60162a30 0x7f7f600b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f64aac700 1 -- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60162a30 msgr2=0x7f7f600b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f64aac700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60162a30 0x7f7f600b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f64aac700 1 -- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f60158a50 msgr2=0x7f7f60160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f64aac700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f60158a50 0x7f7f60160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f64aac700 1 -- 172.21.15.83:0/2471823662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f540094a0 con 0x7f7f60158670 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.886+0000 7f7f652ad700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60162a30 0x7f7f600b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.887+0000 7f7f64aac700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f60158670 0x7f7f60155790 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400ebe0 tx=0x7f7f54009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.887+0000 7f7f5d7fa700 1 -- 172.21.15.83:0/2471823662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400c910 con 0x7f7f60158670 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.887+0000 7f7f66d10700 1 -- 172.21.15.83:0/2471823662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c0094a0 con 0x7f7f60158670 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.887+0000 7f7f5d7fa700 1 -- 172.21.15.83:0/2471823662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5400ca70 con 0x7f7f60158670 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.887+0000 7f7f5d7fa700 1 -- 172.21.15.83:0/2471823662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400cde0 con 0x7f7f60158670 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.887+0000 7f7f66d10700 1 -- 172.21.15.83:0/2471823662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f600ada80 con 0x7f7f60158670 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.888+0000 7f7f66d10700 1 -- 172.21.15.83:0/2471823662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f600addf0 con 0x7f7f60158670 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.888+0000 7f7f5d7fa700 1 -- 172.21.15.83:0/2471823662 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f7f54013470 con 0x7f7f60158670 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.889+0000 7f7f5d7fa700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7f4406b1a0 0x7f7f4406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.889+0000 7f7f5d7fa700 1 -- 172.21.15.83:0/2471823662 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f54089c00 con 0x7f7f60158670 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.889+0000 7f7f5ffff700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7f4406b1a0 0x7f7f4406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.889+0000 7f7f5ffff700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7f4406b1a0 0x7f7f4406d660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000c00 tx=0x7f7f4c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:41.893+0000 7f7f5d7fa700 1 -- 172.21.15.83:0/2471823662 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f54054970 con 0x7f7f60158670 2023-11-26T07:17:42.088 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.085+0000 7f7f66d10700 1 -- 172.21.15.83:0/2471823662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f60067f60 con 0x7f7f60158670 2023-11-26T07:17:42.089 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.086+0000 7f7f5d7fa700 1 -- 172.21.15.83:0/2471823662 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16515 (secure 0 0 0) 0x7f7f54057f90 con 0x7f7f60158670 2023-11-26T07:17:42.089 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:42.089 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":102,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:17:03.554036+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:00.194045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:00.926172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:42.091 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 -- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7f4406b1a0 msgr2=0x7f7f4406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7f4406b1a0 0x7f7f4406d660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c000c00 tx=0x7f7f4c008030 comp rx=0 tx=0).stop 2023-11-26T07:17:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 -- 172.21.15.83:0/2471823662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f60158670 msgr2=0x7f7f60155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f60158670 0x7f7f60155790 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400ebe0 tx=0x7f7f54009ec0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 -- 172.21.15.83:0/2471823662 shutdown_connections 2023-11-26T07:17:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7f4406b1a0 0x7f7f4406d660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f60162a30 0x7f7f600b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f60158670 0x7f7f60155790 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 --2- 172.21.15.83:0/2471823662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f60158a50 0x7f7f60160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 -- 172.21.15.83:0/2471823662 >> 172.21.15.83:0/2471823662 conn(0x7f7f600b57b0 msgr2=0x7f7f6006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 -- 172.21.15.83:0/2471823662 shutdown_connections 2023-11-26T07:17:42.092 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.089+0000 7f7f66d10700 1 -- 172.21.15.83:0/2471823662 wait complete. 2023-11-26T07:17:42.103 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:11:16.349108+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:00.194045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:00.926172+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:42.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.224+0000 7f6c6dbcf700 1 Processor -- start 2023-11-26T07:17:42.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.224+0000 7f6c6dbcf700 1 -- start start 2023-11-26T07:17:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.225+0000 7f6c6dbcf700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6c68144760 0x7f6c680b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.225+0000 7f6c6dbcf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c6806c000 0x7f6c6806c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.225+0000 7f6c6dbcf700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c680b4300 con 0x7f6c680b4cd0 2023-11-26T07:17:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.225+0000 7f6c6dbcf700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6806c950 con 0x7f6c6806c000 2023-11-26T07:17:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.225+0000 7f6c6dbcf700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c68144b40 con 0x7f6c68144760 2023-11-26T07:17:42.229 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.225+0000 7f6c67fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c6806c000 0x7f6c6806c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.225+0000 7f6c67fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6c6806c000 0x7f6c6806c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42266/0 (socket says 172.21.15.83:42266) 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.225+0000 7f6c67fff700 1 -- 172.21.15.83:0/3578561493 learned_addr learned my addr 172.21.15.83:0/3578561493 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.226+0000 7f6c677fe700 1 --2- 172.21.15.83:0/3578561493 >> v2:172.21.15.203:3300/0 conn(0x7f6c68144760 0x7f6c680b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.226+0000 7f6c67fff700 1 -- 172.21.15.83:0/3578561493 >> v2:172.21.15.203:3300/0 conn(0x7f6c68144760 msgr2=0x7f6c680b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.226+0000 7f6c67fff700 1 --2- 172.21.15.83:0/3578561493 >> v2:172.21.15.203:3300/0 conn(0x7f6c68144760 0x7f6c680b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.226+0000 7f6c67fff700 1 -- 172.21.15.83:0/3578561493 >> v1:172.21.15.203:6789/0 conn(0x7f6c680b4cd0 legacy=0x7f6c6806db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.226+0000 7f6c67fff700 1 -- 172.21.15.83:0/3578561493 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6815ec80 con 0x7f6c6806c000 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.226+0000 7f6c67fff700 1 --2- 172.21.15.83:0/3578561493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c6806c000 0x7f6c6806c410 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6c68068c60 tx=0x7f6c6000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f2b72fb7e73c76 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.226+0000 7f6c667fc700 1 -- 172.21.15.83:0/3578561493 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60010070 con 0x7f6c6806c000 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.226+0000 7f6c667fc700 1 -- 172.21.15.83:0/3578561493 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6000dd90 con 0x7f6c6806c000 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.226+0000 7f6c667fc700 1 -- 172.21.15.83:0/3578561493 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000b310 con 0x7f6c6806c000 2023-11-26T07:17:42.230 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.226+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3578561493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c6806c000 msgr2=0x7f6c6806c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.226+0000 7f6c6dbcf700 1 --2- 172.21.15.83:0/3578561493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c6806c000 0x7f6c6806c410 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f6c68068c60 tx=0x7f6c6000d430 comp rx=0 tx=0).stop 2023-11-26T07:17:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.227+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3578561493 shutdown_connections 2023-11-26T07:17:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.227+0000 7f6c6dbcf700 1 --2- 172.21.15.83:0/3578561493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c6806c000 0x7f6c6806c410 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.227+0000 7f6c6dbcf700 1 --2- 172.21.15.83:0/3578561493 >> v2:172.21.15.203:3300/0 conn(0x7f6c68144760 0x7f6c680b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.227+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3578561493 >> 172.21.15.83:0/3578561493 conn(0x7f6c68071a70 msgr2=0x7f6c68071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.227+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3578561493 shutdown_connections 2023-11-26T07:17:42.231 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.227+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3578561493 wait complete. 2023-11-26T07:17:42.233 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.228+0000 7f6c6dbcf700 1 Processor -- start 2023-11-26T07:17:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.228+0000 7f6c6dbcf700 1 -- start start 2023-11-26T07:17:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c6dbcf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c6806c000 0x7f6c680cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c6dbcf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680b4cd0 0x7f6c680ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c6dbcf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c68144760 0x7f6c6815df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c6dbcf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6815e470 con 0x7f6c680b4cd0 2023-11-26T07:17:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c6dbcf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680cf960 con 0x7f6c6806c000 2023-11-26T07:17:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c6dbcf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680ced50 con 0x7f6c68144760 2023-11-26T07:17:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c66ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680b4cd0 0x7f6c680ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.234 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c66ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680b4cd0 0x7f6c680ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42270/0 (socket says 172.21.15.83:42270) 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c66ffd700 1 -- 172.21.15.83:0/3487012841 learned_addr learned my addr 172.21.15.83:0/3487012841 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c677fe700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c6806c000 0x7f6c680cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c67fff700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c68144760 0x7f6c6815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c66ffd700 1 -- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c68144760 msgr2=0x7f6c6815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c66ffd700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c68144760 0x7f6c6815df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c66ffd700 1 -- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c6806c000 msgr2=0x7f6c680cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c66ffd700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c6806c000 0x7f6c680cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c66ffd700 1 -- 172.21.15.83:0/3487012841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6000bd40 con 0x7f6c680b4cd0 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c66ffd700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680b4cd0 0x7f6c680ce700 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00c4e0 tx=0x7f6c5c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c4ffff700 1 -- 172.21.15.83:0/3487012841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0052d0 con 0x7f6c680b4cd0 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c4ffff700 1 -- 172.21.15.83:0/3487012841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c005430 con 0x7f6c680b4cd0 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c4ffff700 1 -- 172.21.15.83:0/3487012841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0038c0 con 0x7f6c680b4cd0 2023-11-26T07:17:42.235 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.229+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3487012841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c680cf030 con 0x7f6c680b4cd0 2023-11-26T07:17:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.230+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3487012841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c680cf5f0 con 0x7f6c680b4cd0 2023-11-26T07:17:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.230+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3487012841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c680b6ba0 con 0x7f6c680b4cd0 2023-11-26T07:17:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.231+0000 7f6c4ffff700 1 -- 172.21.15.83:0/3487012841 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f6c5c003a20 con 0x7f6c680b4cd0 2023-11-26T07:17:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.231+0000 7f6c4ffff700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c5406b1a0 0x7f6c5406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.231+0000 7f6c4ffff700 1 -- 172.21.15.83:0/3487012841 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c5c088db0 con 0x7f6c680b4cd0 2023-11-26T07:17:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.231+0000 7f6c677fe700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c5406b1a0 0x7f6c5406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.236 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.232+0000 7f6c677fe700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c5406b1a0 0x7f6c5406d660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c58006b20 tx=0x7f6c58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.238 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.236+0000 7f6c4ffff700 1 -- 172.21.15.83:0/3487012841 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c5c053b10 con 0x7f6c680b4cd0 2023-11-26T07:17:42.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.425+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3487012841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c680cfaa0 con 0x7f6c680b4cd0 2023-11-26T07:17:42.428 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.426+0000 7f6c4ffff700 1 -- 172.21.15.83:0/3487012841 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c680cfaa0 con 0x7f6c680b4cd0 2023-11-26T07:17:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2023-11-26T07:17:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c5406b1a0 msgr2=0x7f6c5406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c5406b1a0 0x7f6c5406d660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c58006b20 tx=0x7f6c58008040 comp rx=0 tx=0).stop 2023-11-26T07:17:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3487012841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680b4cd0 msgr2=0x7f6c680ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680b4cd0 0x7f6c680ce700 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00c4e0 tx=0x7f6c5c00c7f0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3487012841 shutdown_connections 2023-11-26T07:17:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c5406b1a0 0x7f6c5406d660 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c68144760 0x7f6c6815df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.431 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c6806c000 0x7f6c680cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 --2- 172.21.15.83:0/3487012841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c680b4cd0 0x7f6c680ce700 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3487012841 >> 172.21.15.83:0/3487012841 conn(0x7f6c68071a70 msgr2=0x7f6c6806db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3487012841 shutdown_connections 2023-11-26T07:17:42.432 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.428+0000 7f6c6dbcf700 1 -- 172.21.15.83:0/3487012841 wait complete. 2023-11-26T07:17:42.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:17:42.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.563+0000 7fb4b6162700 1 Processor -- start 2023-11-26T07:17:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.563+0000 7fb4b6162700 1 -- start start 2023-11-26T07:17:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4b6162700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b0161d20 0x7fb4b0162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4b6162700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4b00995e0 0x7fb4b0099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4b6162700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b0162640 con 0x7fb4b006e3a0 2023-11-26T07:17:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4b6162700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b00d0700 con 0x7fb4b00995e0 2023-11-26T07:17:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4b6162700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00ce2e0 con 0x7fb4b0161d20 2023-11-26T07:17:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4aeffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4b00995e0 0x7fb4b0099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4aeffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb4b00995e0 0x7fb4b0099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42278/0 (socket says 172.21.15.83:42278) 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4aeffd700 1 -- 172.21.15.83:0/923153797 learned_addr learned my addr 172.21.15.83:0/923153797 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4af7fe700 1 --2- 172.21.15.83:0/923153797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b0161d20 0x7fb4b0162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4aeffd700 1 -- 172.21.15.83:0/923153797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b0161d20 msgr2=0x7fb4b0162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4aeffd700 1 --2- 172.21.15.83:0/923153797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b0161d20 0x7fb4b0162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4aeffd700 1 -- 172.21.15.83:0/923153797 >> v1:172.21.15.203:6789/0 conn(0x7fb4b006e3a0 legacy=0x7fb4b006e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4aeffd700 1 -- 172.21.15.83:0/923153797 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b006f2d0 con 0x7fb4b00995e0 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4aeffd700 1 --2- 172.21.15.83:0/923153797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b00995e0 0x7fb4b0099a40 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00098f0 tx=0x7fb4a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea45f96d8714b1a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4ae7fc700 1 -- 172.21.15.83:0/923153797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a000b8a0 con 0x7fb4b00995e0 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4ae7fc700 1 -- 172.21.15.83:0/923153797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a000ba00 con 0x7fb4b00995e0 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.564+0000 7fb4ae7fc700 1 -- 172.21.15.83:0/923153797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a000bd90 con 0x7fb4b00995e0 2023-11-26T07:17:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.565+0000 7fb4b6162700 1 -- 172.21.15.83:0/923153797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b00995e0 msgr2=0x7fb4b0099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.565+0000 7fb4b6162700 1 --2- 172.21.15.83:0/923153797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b00995e0 0x7fb4b0099a40 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00098f0 tx=0x7fb4a0009c00 comp rx=0 tx=0).stop 2023-11-26T07:17:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.565+0000 7fb4b6162700 1 -- 172.21.15.83:0/923153797 shutdown_connections 2023-11-26T07:17:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.565+0000 7fb4b6162700 1 --2- 172.21.15.83:0/923153797 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b0161d20 0x7fb4b0162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.565+0000 7fb4b6162700 1 --2- 172.21.15.83:0/923153797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b00995e0 0x7fb4b0099a40 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.565+0000 7fb4b6162700 1 -- 172.21.15.83:0/923153797 >> 172.21.15.83:0/923153797 conn(0x7fb4b00726e0 msgr2=0x7fb4b0072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.565+0000 7fb4b6162700 1 -- 172.21.15.83:0/923153797 shutdown_connections 2023-11-26T07:17:42.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.565+0000 7fb4b6162700 1 -- 172.21.15.83:0/923153797 wait complete. 2023-11-26T07:17:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.566+0000 7fb4b6162700 1 Processor -- start 2023-11-26T07:17:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.567+0000 7fb4b6162700 1 -- start start 2023-11-26T07:17:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4b6162700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b006e3a0 0x7fb4b0159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4b6162700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b00995e0 0x7fb4b00c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4b6162700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b0161d20 0x7fb4b015a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4b6162700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c73d0 con 0x7fb4b00995e0 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4b6162700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c8f20 con 0x7fb4b006e3a0 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4b6162700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c91c0 con 0x7fb4b0161d20 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4aeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b00995e0 0x7fb4b00c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4aeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b00995e0 0x7fb4b00c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42284/0 (socket says 172.21.15.83:42284) 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4aeffd700 1 -- 172.21.15.83:0/1517288802 learned_addr learned my addr 172.21.15.83:0/1517288802 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4af7fe700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b006e3a0 0x7fb4b0159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4affff700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b0161d20 0x7fb4b015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4aeffd700 1 -- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b0161d20 msgr2=0x7fb4b015a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4aeffd700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b0161d20 0x7fb4b015a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4aeffd700 1 -- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b006e3a0 msgr2=0x7fb4b0159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.573 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4aeffd700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b006e3a0 0x7fb4b0159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4aeffd700 1 -- 172.21.15.83:0/1517288802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a0009550 con 0x7fb4b00995e0 2023-11-26T07:17:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.568+0000 7fb4aeffd700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b00995e0 0x7fb4b00c89e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0000c00 tx=0x7fb4a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.569+0000 7fb497fff700 1 -- 172.21.15.83:0/1517288802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0004380 con 0x7fb4b00995e0 2023-11-26T07:17:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.569+0000 7fb497fff700 1 -- 172.21.15.83:0/1517288802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a001f070 con 0x7fb4b00995e0 2023-11-26T07:17:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.569+0000 7fb4b6162700 1 -- 172.21.15.83:0/1517288802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b00c7670 con 0x7fb4b00995e0 2023-11-26T07:17:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.569+0000 7fb497fff700 1 -- 172.21.15.83:0/1517288802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00036a0 con 0x7fb4b00995e0 2023-11-26T07:17:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.569+0000 7fb4b6162700 1 -- 172.21.15.83:0/1517288802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b00c94a0 con 0x7fb4b00995e0 2023-11-26T07:17:42.574 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.570+0000 7fb4b6162700 1 -- 172.21.15.83:0/1517288802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b00c9630 con 0x7fb4b00995e0 2023-11-26T07:17:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.570+0000 7fb497fff700 1 -- 172.21.15.83:0/1517288802 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fb4a0021030 con 0x7fb4b00995e0 2023-11-26T07:17:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.571+0000 7fb497fff700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb49c06b1a0 0x7fb49c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.571+0000 7fb497fff700 1 -- 172.21.15.83:0/1517288802 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4a008af90 con 0x7fb4b00995e0 2023-11-26T07:17:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.571+0000 7fb4af7fe700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb49c06b1a0 0x7fb49c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.575 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.571+0000 7fb4af7fe700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb49c06b1a0 0x7fb49c06d660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4006b20 tx=0x7fb4a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.578 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.576+0000 7fb497fff700 1 -- 172.21.15.83:0/1517288802 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4a0055ea0 con 0x7fb4b00995e0 2023-11-26T07:17:42.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.767+0000 7fb4b6162700 1 -- 172.21.15.83:0/1517288802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4b00c7b90 con 0x7fb4b00995e0 2023-11-26T07:17:42.771 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.768+0000 7fb497fff700 1 -- 172.21.15.83:0/1517288802 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4b00c7b90 con 0x7fb4b00995e0 2023-11-26T07:17:42.772 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2023-11-26T07:17:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 -- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb49c06b1a0 msgr2=0x7fb49c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb49c06b1a0 0x7fb49c06d660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4006b20 tx=0x7fb4a4008040 comp rx=0 tx=0).stop 2023-11-26T07:17:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 -- 172.21.15.83:0/1517288802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b00995e0 msgr2=0x7fb4b00c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b00995e0 0x7fb4b00c89e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0000c00 tx=0x7fb4a0004750 comp rx=0 tx=0).stop 2023-11-26T07:17:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 -- 172.21.15.83:0/1517288802 shutdown_connections 2023-11-26T07:17:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fb49c06b1a0 0x7fb49c06d660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b0161d20 0x7fb4b015a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b006e3a0 0x7fb4b0159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.773 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 --2- 172.21.15.83:0/1517288802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b00995e0 0x7fb4b00c89e0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 -- 172.21.15.83:0/1517288802 >> 172.21.15.83:0/1517288802 conn(0x7fb4b00726e0 msgr2=0x7fb4b00b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:42.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 -- 172.21.15.83:0/1517288802 shutdown_connections 2023-11-26T07:17:42.774 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.770+0000 7fb4b6162700 1 -- 172.21.15.83:0/1517288802 wait complete. 2023-11-26T07:17:42.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:17:42.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.905+0000 7f775882a700 1 Processor -- start 2023-11-26T07:17:42.908 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.905+0000 7f775882a700 1 -- start start 2023-11-26T07:17:42.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775882a700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f775415bf50 0x7f77540ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775882a700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f77540ce830 0x7f77540cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.910 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775882a700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77540d0140 con 0x7f77540cf1f0 2023-11-26T07:17:42.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775882a700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f775415c330 con 0x7f775415bf50 2023-11-26T07:17:42.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775882a700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77540d03e0 con 0x7f77540ce830 2023-11-26T07:17:42.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775259c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f775415bf50 0x7f77540ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f7752d9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f77540cf1f0 0x7f77540c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34102/0 (socket says 172.21.15.83:34102) 2023-11-26T07:17:42.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775259c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f775415bf50 0x7f77540ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42290/0 (socket says 172.21.15.83:42290) 2023-11-26T07:17:42.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f7752d9d700 1 -- 172.21.15.83:0/3567982697 learned_addr learned my addr 172.21.15.83:0/3567982697 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:42.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f7751d9b700 1 --2- 172.21.15.83:0/3567982697 >> v2:172.21.15.203:3300/0 conn(0x7f77540ce830 0x7f77540cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775259c700 1 -- 172.21.15.83:0/3567982697 >> v2:172.21.15.203:3300/0 conn(0x7f77540ce830 msgr2=0x7f77540cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:42.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775259c700 1 --2- 172.21.15.83:0/3567982697 >> v2:172.21.15.203:3300/0 conn(0x7f77540ce830 0x7f77540cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775259c700 1 -- 172.21.15.83:0/3567982697 >> v1:172.21.15.83:6789/0 conn(0x7f77540cf1f0 legacy=0x7f77540c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.911 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775259c700 1 -- 172.21.15.83:0/3567982697 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77540d06c0 con 0x7f775415bf50 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775159a700 1 -- 172.21.15.83:0/3567982697 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945401047 0 0) 0x7f77540d0140 con 0x7f77540cf1f0 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.906+0000 7f775259c700 1 --2- 172.21.15.83:0/3567982697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f775415bf50 0x7f77540ce2f0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7754072420 tx=0x7f7748009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc7957ce932faf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.907+0000 7f775159a700 1 -- 172.21.15.83:0/3567982697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748011830 con 0x7f775415bf50 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.907+0000 7f775159a700 1 -- 172.21.15.83:0/3567982697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7748011990 con 0x7f775415bf50 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.907+0000 7f775159a700 1 -- 172.21.15.83:0/3567982697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748011d20 con 0x7f775415bf50 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.907+0000 7f775882a700 1 -- 172.21.15.83:0/3567982697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f775415bf50 msgr2=0x7f77540ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.907+0000 7f775882a700 1 --2- 172.21.15.83:0/3567982697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f775415bf50 0x7f77540ce2f0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7754072420 tx=0x7f7748009b20 comp rx=0 tx=0).stop 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.907+0000 7f775882a700 1 -- 172.21.15.83:0/3567982697 shutdown_connections 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.907+0000 7f775882a700 1 --2- 172.21.15.83:0/3567982697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f775415bf50 0x7f77540ce2f0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.907+0000 7f775882a700 1 --2- 172.21.15.83:0/3567982697 >> v2:172.21.15.203:3300/0 conn(0x7f77540ce830 0x7f77540cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.907+0000 7f775882a700 1 -- 172.21.15.83:0/3567982697 >> 172.21.15.83:0/3567982697 conn(0x7f77540b3780 msgr2=0x7f77540b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:42.912 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.907+0000 7f775882a700 1 -- 172.21.15.83:0/3567982697 shutdown_connections 2023-11-26T07:17:42.913 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.907+0000 7f775882a700 1 -- 172.21.15.83:0/3567982697 wait complete. 2023-11-26T07:17:42.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.909+0000 7f775882a700 1 Processor -- start 2023-11-26T07:17:42.915 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.909+0000 7f775882a700 1 -- start start 2023-11-26T07:17:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.909+0000 7f775882a700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77540ce830 0x7f77540c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.909+0000 7f775882a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77540cf1f0 0x7f77540c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.909+0000 7f775882a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f775415bf50 0x7f77540b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.909+0000 7f775882a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77540b6210 con 0x7f775415bf50 2023-11-26T07:17:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.909+0000 7f775882a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77540b64f0 con 0x7f77540cf1f0 2023-11-26T07:17:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.909+0000 7f775882a700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77540b67d0 con 0x7f77540ce830 2023-11-26T07:17:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.909+0000 7f7752d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f775415bf50 0x7f77540b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7752d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f775415bf50 0x7f77540b72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42300/0 (socket says 172.21.15.83:42300) 2023-11-26T07:17:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7752d9d700 1 -- 172.21.15.83:0/2236239307 learned_addr learned my addr 172.21.15.83:0/2236239307 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7751d9b700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77540cf1f0 0x7f77540c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.916 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f775259c700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77540ce830 0x7f77540c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7752d9d700 1 -- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77540ce830 msgr2=0x7f77540c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7752d9d700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77540ce830 0x7f77540c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7752d9d700 1 -- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77540cf1f0 msgr2=0x7f77540c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7752d9d700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77540cf1f0 0x7f77540c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7752d9d700 1 -- 172.21.15.83:0/2236239307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77480094a0 con 0x7f775415bf50 2023-11-26T07:17:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7752d9d700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f775415bf50 0x7f77540b72b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f774c00cbe0 tx=0x7f774c00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7742ffd700 1 -- 172.21.15.83:0/2236239307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c00aaa0 con 0x7f775415bf50 2023-11-26T07:17:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7742ffd700 1 -- 172.21.15.83:0/2236239307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c00ac20 con 0x7f775415bf50 2023-11-26T07:17:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f775882a700 1 -- 172.21.15.83:0/2236239307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77540b6ab0 con 0x7f775415bf50 2023-11-26T07:17:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f7742ffd700 1 -- 172.21.15.83:0/2236239307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c0070d0 con 0x7f775415bf50 2023-11-26T07:17:42.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.910+0000 7f775882a700 1 -- 172.21.15.83:0/2236239307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77541565d0 con 0x7f775415bf50 2023-11-26T07:17:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.911+0000 7f775882a700 1 -- 172.21.15.83:0/2236239307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77540d0090 con 0x7f775415bf50 2023-11-26T07:17:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.911+0000 7f7742ffd700 1 -- 172.21.15.83:0/2236239307 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f774c007230 con 0x7f775415bf50 2023-11-26T07:17:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.912+0000 7f7742ffd700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f773406b1a0 0x7f773406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.912+0000 7f7742ffd700 1 -- 172.21.15.83:0/2236239307 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f774c089730 con 0x7f775415bf50 2023-11-26T07:17:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.912+0000 7f775259c700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f773406b1a0 0x7f773406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.913+0000 7f775259c700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f773406b1a0 0x7f773406d660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f77480090b0 tx=0x7f774801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.918 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:42.916+0000 7f7742ffd700 1 -- 172.21.15.83:0/2236239307 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f774c0544d0 con 0x7f775415bf50 2023-11-26T07:17:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.107+0000 7f775882a700 1 -- 172.21.15.83:0/2236239307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7754068fb0 con 0x7f775415bf50 2023-11-26T07:17:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.108+0000 7f7742ffd700 1 -- 172.21.15.83:0/2236239307 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7f774c0180a0 con 0x7f775415bf50 2023-11-26T07:17:43.111 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2023-11-26T07:17:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 -- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f773406b1a0 msgr2=0x7f773406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f773406b1a0 0x7f773406d660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f77480090b0 tx=0x7f774801c040 comp rx=0 tx=0).stop 2023-11-26T07:17:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 -- 172.21.15.83:0/2236239307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f775415bf50 msgr2=0x7f77540b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f775415bf50 0x7f77540b72b0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f774c00cbe0 tx=0x7f774c00cef0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 -- 172.21.15.83:0/2236239307 shutdown_connections 2023-11-26T07:17:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f773406b1a0 0x7f773406d660 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f775415bf50 0x7f77540b72b0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77540ce830 0x7f77540c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 --2- 172.21.15.83:0/2236239307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77540cf1f0 0x7f77540c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 -- 172.21.15.83:0/2236239307 >> 172.21.15.83:0/2236239307 conn(0x7f77540b3780 msgr2=0x7f775406b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 -- 172.21.15.83:0/2236239307 shutdown_connections 2023-11-26T07:17:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.110+0000 7f775882a700 1 -- 172.21.15.83:0/2236239307 wait complete. 2023-11-26T07:17:43.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.243+0000 7f56d1f6e700 1 Processor -- start 2023-11-26T07:17:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.243+0000 7f56d1f6e700 1 -- start start 2023-11-26T07:17:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.244+0000 7f56d1f6e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f56cc1446a0 0x7f56cc0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.244+0000 7f56d1f6e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56cc06c710 0x7f56cc0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.244+0000 7f56d1f6e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc0af950 con 0x7f56cc0adf00 2023-11-26T07:17:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.244+0000 7f56d1f6e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc144a80 con 0x7f56cc1446a0 2023-11-26T07:17:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56d1f6e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0afc30 con 0x7f56cc06c710 2023-11-26T07:17:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56cb7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f56cc1446a0 0x7f56cc0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56cb7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f56cc1446a0 0x7f56cc0b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42304/0 (socket says 172.21.15.83:42304) 2023-11-26T07:17:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56cb7fe700 1 -- 172.21.15.83:0/1988301402 learned_addr learned my addr 172.21.15.83:0/1988301402 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56cbfff700 1 --2- 172.21.15.83:0/1988301402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56cc06c710 0x7f56cc0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56cb7fe700 1 -- 172.21.15.83:0/1988301402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56cc06c710 msgr2=0x7f56cc0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56cb7fe700 1 --2- 172.21.15.83:0/1988301402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56cc06c710 0x7f56cc0b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56cb7fe700 1 -- 172.21.15.83:0/1988301402 >> v1:172.21.15.203:6789/0 conn(0x7f56cc0adf00 legacy=0x7f56cc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56cb7fe700 1 -- 172.21.15.83:0/1988301402 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc0aff10 con 0x7f56cc1446a0 2023-11-26T07:17:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56cb7fe700 1 --2- 172.21.15.83:0/1988301402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56cc1446a0 0x7f56cc0b5ae0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00bb90 tx=0x7f56bc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9e81e1424b106a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56caffd700 1 -- 172.21.15.83:0/1988301402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00dd50 con 0x7f56cc1446a0 2023-11-26T07:17:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56caffd700 1 -- 172.21.15.83:0/1988301402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc010470 con 0x7f56cc1446a0 2023-11-26T07:17:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.245+0000 7f56caffd700 1 -- 172.21.15.83:0/1988301402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc010800 con 0x7f56cc1446a0 2023-11-26T07:17:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.246+0000 7f56d1f6e700 1 -- 172.21.15.83:0/1988301402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56cc1446a0 msgr2=0x7f56cc0b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.246+0000 7f56d1f6e700 1 --2- 172.21.15.83:0/1988301402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56cc1446a0 0x7f56cc0b5ae0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00bb90 tx=0x7f56bc00bea0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.246+0000 7f56d1f6e700 1 -- 172.21.15.83:0/1988301402 shutdown_connections 2023-11-26T07:17:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.246+0000 7f56d1f6e700 1 --2- 172.21.15.83:0/1988301402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56cc06c710 0x7f56cc0b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.246+0000 7f56d1f6e700 1 --2- 172.21.15.83:0/1988301402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56cc1446a0 0x7f56cc0b5ae0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.246+0000 7f56d1f6e700 1 -- 172.21.15.83:0/1988301402 >> 172.21.15.83:0/1988301402 conn(0x7f56cc071b90 msgr2=0x7f56cc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.246+0000 7f56d1f6e700 1 -- 172.21.15.83:0/1988301402 shutdown_connections 2023-11-26T07:17:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.246+0000 7f56d1f6e700 1 -- 172.21.15.83:0/1988301402 wait complete. 2023-11-26T07:17:43.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.247+0000 7f56d1f6e700 1 Processor -- start 2023-11-26T07:17:43.253 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.247+0000 7f56d1f6e700 1 -- start start 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56d1f6e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56cc06c710 0x7f56cc0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56d1f6e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56cc0adf00 0x7f56cc140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56d1f6e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56cc1446a0 0x7f56cc15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56d1f6e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1421c0 con 0x7f56cc1446a0 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56d1f6e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc141540 con 0x7f56cc06c710 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56d1f6e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc141820 con 0x7f56cc0adf00 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56cbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56cc1446a0 0x7f56cc15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56cbfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56cc1446a0 0x7f56cc15a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42314/0 (socket says 172.21.15.83:42314) 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56cbfff700 1 -- 172.21.15.83:0/2078411222 learned_addr learned my addr 172.21.15.83:0/2078411222 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56cb7fe700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56cc06c710 0x7f56cc0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56c2ffd700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56cc0adf00 0x7f56cc140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56cbfff700 1 -- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56cc0adf00 msgr2=0x7f56cc140af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.254 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56cbfff700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56cc0adf00 0x7f56cc140af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56cbfff700 1 -- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56cc06c710 msgr2=0x7f56cc0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56cbfff700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56cc06c710 0x7f56cc0b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.248+0000 7f56cbfff700 1 -- 172.21.15.83:0/2078411222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc00b7f0 con 0x7f56cc1446a0 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.249+0000 7f56cbfff700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56cc1446a0 0x7f56cc15a1f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f56c4007e40 tx=0x7f56c400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.249+0000 7f56c8ff9700 1 -- 172.21.15.83:0/2078411222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c400ed30 con 0x7f56cc1446a0 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.249+0000 7f56c8ff9700 1 -- 172.21.15.83:0/2078411222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4014420 con 0x7f56cc1446a0 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.249+0000 7f56d1f6e700 1 -- 172.21.15.83:0/2078411222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc141b00 con 0x7f56cc1446a0 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.249+0000 7f56c8ff9700 1 -- 172.21.15.83:0/2078411222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40147d0 con 0x7f56cc1446a0 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.249+0000 7f56d1f6e700 1 -- 172.21.15.83:0/2078411222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc1400a0 con 0x7f56cc1446a0 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.250+0000 7f56d1f6e700 1 -- 172.21.15.83:0/2078411222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc0753f0 con 0x7f56cc1446a0 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.250+0000 7f56c8ff9700 1 -- 172.21.15.83:0/2078411222 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f56c4014930 con 0x7f56cc1446a0 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.251+0000 7f56c8ff9700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f56ac06b1a0 0x7f56ac06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.255 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.251+0000 7f56c8ff9700 1 -- 172.21.15.83:0/2078411222 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56c408a010 con 0x7f56cc1446a0 2023-11-26T07:17:43.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.251+0000 7f56cb7fe700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f56ac06b1a0 0x7f56ac06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.256 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.252+0000 7f56cb7fe700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f56ac06b1a0 0x7f56ac06d660 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00bee0 tx=0x7f56bc013000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.258 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.255+0000 7f56c8ff9700 1 -- 172.21.15.83:0/2078411222 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56c4054e00 con 0x7f56cc1446a0 2023-11-26T07:17:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.492+0000 7f56d1f6e700 1 -- 172.21.15.83:0/2078411222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56cc140230 con 0x7f56cc1446a0 2023-11-26T07:17:43.495 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.492+0000 7f56c8ff9700 1 -- 172.21.15.83:0/2078411222 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f56c4058420 con 0x7f56cc1446a0 2023-11-26T07:17:43.496 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:43.496 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/136760159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":136760159},{"type":"v1","addr":"172.21.15.203:6834","nonce":136760159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3723980912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":3723980912},{"type":"v1","addr":"172.21.15.203:6835","nonce":3723980912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3900154869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3900154869},{"type":"v1","addr":"172.21.15.83:6833","nonce":3900154869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:01.974687+0000","modified":"2023-11-26T07:17:04.985222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7551":{"gid":7551,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3327659486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3327659486},{"type":"v1","addr":"172.21.15.83:6835","nonce":3327659486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:17:43.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 -- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f56ac06b1a0 msgr2=0x7f56ac06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f56ac06b1a0 0x7f56ac06d660 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00bee0 tx=0x7f56bc013000 comp rx=0 tx=0).stop 2023-11-26T07:17:43.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 -- 172.21.15.83:0/2078411222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56cc1446a0 msgr2=0x7f56cc15a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56cc1446a0 0x7f56cc15a1f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f56c4007e40 tx=0x7f56c400e3c0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 -- 172.21.15.83:0/2078411222 shutdown_connections 2023-11-26T07:17:43.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f56ac06b1a0 0x7f56ac06d660 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56cc1446a0 0x7f56cc15a1f0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56cc06c710 0x7f56cc0b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 --2- 172.21.15.83:0/2078411222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56cc0adf00 0x7f56cc140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.498 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 -- 172.21.15.83:0/2078411222 >> 172.21.15.83:0/2078411222 conn(0x7f56cc071b90 msgr2=0x7f56cc0b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 -- 172.21.15.83:0/2078411222 shutdown_connections 2023-11-26T07:17:43.499 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.495+0000 7f56d1f6e700 1 -- 172.21.15.83:0/2078411222 wait complete. 2023-11-26T07:17:43.499 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 70 2023-11-26T07:17:43.510 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:43.629 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.627+0000 7f629fa7c700 1 Processor -- start 2023-11-26T07:17:43.630 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.627+0000 7f629fa7c700 1 -- start start 2023-11-26T07:17:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.628+0000 7f629fa7c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f629815e080 0x7f62980b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.628+0000 7f629fa7c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f629806c710 0x7f62980afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.628+0000 7f629fa7c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62980ae430 con 0x7f62980af4d0 2023-11-26T07:17:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.628+0000 7f629fa7c700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62980af0b0 con 0x7f629815e080 2023-11-26T07:17:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.628+0000 7f629fa7c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62980b05a0 con 0x7f629806c710 2023-11-26T07:17:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.628+0000 7f629d017700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f62980af4d0 0x7f629806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34126/0 (socket says 172.21.15.83:34126) 2023-11-26T07:17:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.628+0000 7f629d017700 1 -- 172.21.15.83:0/1684790464 learned_addr learned my addr 172.21.15.83:0/1684790464 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.628+0000 7f629d818700 1 --2- 172.21.15.83:0/1684790464 >> v2:172.21.15.203:3300/0 conn(0x7f629815e080 0x7f62980b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.628+0000 7f629e019700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f629806c710 0x7f62980afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.628+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237271723 0 0) 0x7f62980ae430 con 0x7f62980af4d0 2023-11-26T07:17:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.628+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62840035b0 con 0x7f62980af4d0 2023-11-26T07:17:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4067616656 0 0) 0x7f62840035b0 con 0x7f62980af4d0 2023-11-26T07:17:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f629806c710 msgr2=0x7f62980afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f629806c710 0x7f62980afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 >> v2:172.21.15.203:3300/0 conn(0x7f629815e080 msgr2=0x7f62980b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 --2- 172.21.15.83:0/1684790464 >> v2:172.21.15.203:3300/0 conn(0x7f629815e080 0x7f62980b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62980ae710 con 0x7f62980af4d0 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f6294001670 con 0x7f62980af4d0 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 >> v1:172.21.15.83:6789/0 conn(0x7f62980af4d0 legacy=0x7f629806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 0x7f6284003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6284004850 0x7f6284004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f629806c710 0x7f6284005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980af0b0 con 0x7f6284003a30 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980b05a0 con 0x7f6284004850 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628400ba10 con 0x7f629806c710 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629d818700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 0x7f6284003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629e019700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f629806c710 0x7f6284005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629d017700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6284004850 0x7f6284004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629d818700 1 -- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f629806c710 msgr2=0x7f6284005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629d818700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f629806c710 0x7f6284005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629d818700 1 -- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6284004850 msgr2=0x7f6284004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629d818700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6284004850 0x7f6284004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629d818700 1 -- 172.21.15.83:0/1684790464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62840035b0 con 0x7f6284003a30 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629e019700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f629806c710 0x7f6284005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629d017700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6284004850 0x7f6284004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629d818700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 0x7f6284003e50 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6288009b60 tx=0x7f6284005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594628b0b51bb856 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800e6c0 con 0x7f6284003a30 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628800e820 con 0x7f6284003a30 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.629+0000 7f629c816700 1 -- 172.21.15.83:0/1684790464 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800eb90 con 0x7f6284003a30 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.630+0000 7f629fa7c700 1 -- 172.21.15.83:0/1684790464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 msgr2=0x7f6284003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.630+0000 7f629fa7c700 1 --2- 172.21.15.83:0/1684790464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 0x7f6284003e50 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6288009b60 tx=0x7f6284005880 comp rx=0 tx=0).stop 2023-11-26T07:17:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.630+0000 7f629e019700 1 -- 172.21.15.83:0/1684790464 reap_dead start 2023-11-26T07:17:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.630+0000 7f629fa7c700 1 -- 172.21.15.83:0/1684790464 shutdown_connections 2023-11-26T07:17:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.630+0000 7f629fa7c700 1 -- 172.21.15.83:0/1684790464 >> 172.21.15.83:0/1684790464 conn(0x7f6298071b90 msgr2=0x7f6298071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.630+0000 7f629fa7c700 1 -- 172.21.15.83:0/1684790464 shutdown_connections 2023-11-26T07:17:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.630+0000 7f629fa7c700 1 -- 172.21.15.83:0/1684790464 wait complete. 2023-11-26T07:17:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.631+0000 7f629fa7c700 1 Processor -- start 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.631+0000 7f629fa7c700 1 -- start start 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.632+0000 7f629fa7c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 0x7f62981f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.632+0000 7f629fa7c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6298161060 0x7f6298151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.632+0000 7f629fa7c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6298159560 0x7f62981599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.632+0000 7f629fa7c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6298159f30 con 0x7f6284003a30 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.632+0000 7f629fa7c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62981f4830 con 0x7f6298159560 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.632+0000 7f629fa7c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62981588f0 con 0x7f6298161060 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.632+0000 7f629d818700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 0x7f62981f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.632+0000 7f629d818700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 0x7f62981f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42330/0 (socket says 172.21.15.83:42330) 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.632+0000 7f629d818700 1 -- 172.21.15.83:0/198861870 learned_addr learned my addr 172.21.15.83:0/198861870 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.632+0000 7f629e019700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6298159560 0x7f62981599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.632+0000 7f629d017700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6298161060 0x7f6298151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f629d818700 1 -- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6298161060 msgr2=0x7f6298151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f629d818700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6298161060 0x7f6298151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f629d818700 1 -- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6298159560 msgr2=0x7f62981599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f629d818700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6298159560 0x7f62981599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f629d818700 1 -- 172.21.15.83:0/198861870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6298157850 con 0x7f6284003a30 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f629d017700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6298161060 0x7f6298151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f629d818700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 0x7f62981f42f0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f62880090b0 tx=0x7f62880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f628e7fc700 1 -- 172.21.15.83:0/198861870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800e6c0 con 0x7f6284003a30 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f628e7fc700 1 -- 172.21.15.83:0/198861870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288003ad0 con 0x7f6284003a30 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f628e7fc700 1 -- 172.21.15.83:0/198861870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288003e80 con 0x7f6284003a30 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f629fa7c700 1 -- 172.21.15.83:0/198861870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6298157b30 con 0x7f6284003a30 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.633+0000 7f629fa7c700 1 -- 172.21.15.83:0/198861870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62981580f0 con 0x7f6284003a30 2023-11-26T07:17:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.634+0000 7f629fa7c700 1 -- 172.21.15.83:0/198861870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62980753f0 con 0x7f6284003a30 2023-11-26T07:17:43.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.634+0000 7f628e7fc700 1 -- 172.21.15.83:0/198861870 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f6288013470 con 0x7f6284003a30 2023-11-26T07:17:43.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.635+0000 7f628e7fc700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f627c06b1a0 0x7f627c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.635+0000 7f628e7fc700 1 -- 172.21.15.83:0/198861870 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f628808b400 con 0x7f6284003a30 2023-11-26T07:17:43.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.635+0000 7f629d017700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f627c06b1a0 0x7f627c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.635+0000 7f629d017700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f627c06b1a0 0x7f627c06d660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f62980af950 tx=0x7f6294009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.642 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.639+0000 7f628e7fc700 1 -- 172.21.15.83:0/198861870 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6288056310 con 0x7f6284003a30 2023-11-26T07:17:43.833 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.830+0000 7f629fa7c700 1 -- 172.21.15.83:0/198861870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6298067f60 con 0x7f6284003a30 2023-11-26T07:17:43.834 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.831+0000 7f628e7fc700 1 -- 172.21.15.83:0/198861870 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16515 (secure 0 0 0) 0x7f628805ee60 con 0x7f6284003a30 2023-11-26T07:17:43.834 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:43.834 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":102,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:17:03.554036+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:00.194045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:00.926172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:43.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.833+0000 7f629fa7c700 1 -- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f627c06b1a0 msgr2=0x7f627c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.836 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.833+0000 7f629fa7c700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f627c06b1a0 0x7f627c06d660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f62980af950 tx=0x7f6294009040 comp rx=0 tx=0).stop 2023-11-26T07:17:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.833+0000 7f629fa7c700 1 -- 172.21.15.83:0/198861870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 msgr2=0x7f62981f42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.833+0000 7f629fa7c700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 0x7f62981f42f0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f62880090b0 tx=0x7f62880047c0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.833+0000 7f629fa7c700 1 -- 172.21.15.83:0/198861870 shutdown_connections 2023-11-26T07:17:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.833+0000 7f629fa7c700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f627c06b1a0 0x7f627c06d660 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.833+0000 7f629fa7c700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6298159560 0x7f62981599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.833+0000 7f629fa7c700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284003a30 0x7f62981f42f0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.833+0000 7f629fa7c700 1 --2- 172.21.15.83:0/198861870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6298161060 0x7f6298151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.833+0000 7f629fa7c700 1 -- 172.21.15.83:0/198861870 >> 172.21.15.83:0/198861870 conn(0x7f6298071b90 msgr2=0x7f62980b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.834+0000 7f629fa7c700 1 -- 172.21.15.83:0/198861870 shutdown_connections 2023-11-26T07:17:43.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.834+0000 7f629fa7c700 1 -- 172.21.15.83:0/198861870 wait complete. 2023-11-26T07:17:43.848 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:17:43.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.965+0000 7f21309d1700 1 Processor -- start 2023-11-26T07:17:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.965+0000 7f21309d1700 1 -- start start 2023-11-26T07:17:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f21309d1700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f212c158670 0x7f212c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f21309d1700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f212c158a50 0x7f212c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f21309d1700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f212c0cfff0 con 0x7f212c162a30 2023-11-26T07:17:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f21309d1700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212c0cfd10 con 0x7f212c158670 2023-11-26T07:17:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f21309d1700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212c161f50 con 0x7f212c158a50 2023-11-26T07:17:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f212a59c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f212c158670 0x7f212c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f212ad9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f212c162a30 0x7f212c0d0230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34150/0 (socket says 172.21.15.83:34150) 2023-11-26T07:17:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f212ad9d700 1 -- 172.21.15.83:0/356189489 learned_addr learned my addr 172.21.15.83:0/356189489 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f2129d9b700 1 --2- 172.21.15.83:0/356189489 >> v2:172.21.15.203:3300/0 conn(0x7f212c158a50 0x7f212c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f212959a700 1 -- 172.21.15.83:0/356189489 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590562378 0 0) 0x7f212c0cfff0 con 0x7f212c162a30 2023-11-26T07:17:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f212959a700 1 -- 172.21.15.83:0/356189489 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21140035b0 con 0x7f212c162a30 2023-11-26T07:17:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f212a59c700 1 -- 172.21.15.83:0/356189489 >> v2:172.21.15.203:3300/0 conn(0x7f212c158a50 msgr2=0x7f212c0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f212a59c700 1 --2- 172.21.15.83:0/356189489 >> v2:172.21.15.203:3300/0 conn(0x7f212c158a50 0x7f212c0c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f212a59c700 1 -- 172.21.15.83:0/356189489 >> v1:172.21.15.83:6789/0 conn(0x7f212c162a30 legacy=0x7f212c0d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.966+0000 7f212a59c700 1 -- 172.21.15.83:0/356189489 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c162230 con 0x7f212c158670 2023-11-26T07:17:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f212a59c700 1 --2- 172.21.15.83:0/356189489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c158670 0x7f212c0622a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f212c0ca730 tx=0x7f2120009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59512147557c9762 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f212959a700 1 -- 172.21.15.83:0/356189489 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120011830 con 0x7f212c158670 2023-11-26T07:17:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f212959a700 1 -- 172.21.15.83:0/356189489 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2120011990 con 0x7f212c158670 2023-11-26T07:17:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f212959a700 1 -- 172.21.15.83:0/356189489 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120011d20 con 0x7f212c158670 2023-11-26T07:17:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f21309d1700 1 -- 172.21.15.83:0/356189489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c158670 msgr2=0x7f212c0622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f21309d1700 1 --2- 172.21.15.83:0/356189489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c158670 0x7f212c0622a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f212c0ca730 tx=0x7f2120009b20 comp rx=0 tx=0).stop 2023-11-26T07:17:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f21309d1700 1 -- 172.21.15.83:0/356189489 shutdown_connections 2023-11-26T07:17:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f21309d1700 1 --2- 172.21.15.83:0/356189489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c158670 0x7f212c0622a0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f21309d1700 1 --2- 172.21.15.83:0/356189489 >> v2:172.21.15.203:3300/0 conn(0x7f212c158a50 0x7f212c0c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f21309d1700 1 -- 172.21.15.83:0/356189489 >> 172.21.15.83:0/356189489 conn(0x7f212c0b57b0 msgr2=0x7f212c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f21309d1700 1 -- 172.21.15.83:0/356189489 shutdown_connections 2023-11-26T07:17:43.973 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.967+0000 7f21309d1700 1 -- 172.21.15.83:0/356189489 wait complete. 2023-11-26T07:17:43.975 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.969+0000 7f21309d1700 1 Processor -- start 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.969+0000 7f21309d1700 1 -- start start 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.969+0000 7f21309d1700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f212c158670 0x7f212c09dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.969+0000 7f21309d1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f212c158a50 0x7f212c0ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.969+0000 7f21309d1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c162a30 0x7f212c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f21309d1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c09e150 con 0x7f212c162a30 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f21309d1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c15f600 con 0x7f212c158a50 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f21309d1700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212c15f8e0 con 0x7f212c158670 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f212ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c162a30 0x7f212c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f212ad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c162a30 0x7f212c1606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42348/0 (socket says 172.21.15.83:42348) 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f212ad9d700 1 -- 172.21.15.83:0/3860920789 learned_addr learned my addr 172.21.15.83:0/3860920789 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f212a59c700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f212c158670 0x7f212c09dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.976 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f2129d9b700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f212c158a50 0x7f212c0ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f212ad9d700 1 -- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f212c158670 msgr2=0x7f212c09dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f212ad9d700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f212c158670 0x7f212c09dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f212ad9d700 1 -- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f212c158a50 msgr2=0x7f212c0ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f212ad9d700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f212c158a50 0x7f212c0ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f212ad9d700 1 -- 172.21.15.83:0/3860920789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21200094a0 con 0x7f212c162a30 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f212ad9d700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c162a30 0x7f212c1606a0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f212400caa0 tx=0x7f212400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f211affd700 1 -- 172.21.15.83:0/3860920789 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212400aa80 con 0x7f212c162a30 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f211affd700 1 -- 172.21.15.83:0/3860920789 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212400abe0 con 0x7f212c162a30 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f21309d1700 1 -- 172.21.15.83:0/3860920789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212c15fbc0 con 0x7f212c162a30 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f211affd700 1 -- 172.21.15.83:0/3860920789 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124012400 con 0x7f212c162a30 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.970+0000 7f21309d1700 1 -- 172.21.15.83:0/3860920789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212c160180 con 0x7f212c162a30 2023-11-26T07:17:43.977 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.971+0000 7f21309d1700 1 -- 172.21.15.83:0/3860920789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212c067f60 con 0x7f212c162a30 2023-11-26T07:17:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.972+0000 7f211affd700 1 -- 172.21.15.83:0/3860920789 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f212401b3b0 con 0x7f212c162a30 2023-11-26T07:17:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.972+0000 7f211affd700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f210c06b1a0 0x7f210c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.972+0000 7f211affd700 1 -- 172.21.15.83:0/3860920789 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21240896f0 con 0x7f212c162a30 2023-11-26T07:17:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.972+0000 7f212a59c700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f210c06b1a0 0x7f210c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.978 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.973+0000 7f212a59c700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f210c06b1a0 0x7f210c06d660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f21200090b0 tx=0x7f2120008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.979 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:43.977+0000 7f211affd700 1 -- 172.21.15.83:0/3860920789 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21240544e0 con 0x7f212c162a30 2023-11-26T07:17:44.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.211+0000 7f21309d1700 1 -- 172.21.15.83:0/3860920789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f212c074a20 con 0x7f212c162a30 2023-11-26T07:17:44.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.211+0000 7f211affd700 1 -- 172.21.15.83:0/3860920789 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f2124057b00 con 0x7f212c162a30 2023-11-26T07:17:44.214 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:44.214 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/136760159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":136760159},{"type":"v1","addr":"172.21.15.203:6834","nonce":136760159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3723980912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":3723980912},{"type":"v1","addr":"172.21.15.203:6835","nonce":3723980912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3900154869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3900154869},{"type":"v1","addr":"172.21.15.83:6833","nonce":3900154869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:01.974687+0000","modified":"2023-11-26T07:17:04.985222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7551":{"gid":7551,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3327659486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3327659486},{"type":"v1","addr":"172.21.15.83:6835","nonce":3327659486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:17:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 -- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f210c06b1a0 msgr2=0x7f210c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f210c06b1a0 0x7f210c06d660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f21200090b0 tx=0x7f2120008030 comp rx=0 tx=0).stop 2023-11-26T07:17:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 -- 172.21.15.83:0/3860920789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c162a30 msgr2=0x7f212c1606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c162a30 0x7f212c1606a0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f212400caa0 tx=0x7f212400ce60 comp rx=0 tx=0).stop 2023-11-26T07:17:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 -- 172.21.15.83:0/3860920789 shutdown_connections 2023-11-26T07:17:44.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f210c06b1a0 0x7f210c06d660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c162a30 0x7f212c1606a0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f212c158670 0x7f212c09dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 --2- 172.21.15.83:0/3860920789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f212c158a50 0x7f212c0ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 -- 172.21.15.83:0/3860920789 >> 172.21.15.83:0/3860920789 conn(0x7f212c0b57b0 msgr2=0x7f212c0b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:44.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 -- 172.21.15.83:0/3860920789 shutdown_connections 2023-11-26T07:17:44.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.213+0000 7f21309d1700 1 -- 172.21.15.83:0/3860920789 wait complete. 2023-11-26T07:17:44.217 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 70 2023-11-26T07:17:44.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:44.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.347+0000 7f626581e700 1 Processor -- start 2023-11-26T07:17:44.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.347+0000 7f626581e700 1 -- start start 2023-11-26T07:17:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f626581e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62600d0870 0x7f62600d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f626581e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62600b6ef0 0x7f62600b7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f626581e700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62600d1210 con 0x7f626015e1e0 2023-11-26T07:17:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f626581e700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62600d3150 con 0x7f62600b6ef0 2023-11-26T07:17:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f626581e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626015e5c0 con 0x7f62600d0870 2023-11-26T07:17:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f625f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62600b6ef0 0x7f62600b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f625f7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62600b6ef0 0x7f62600b7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42356/0 (socket says 172.21.15.83:42356) 2023-11-26T07:17:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f625f7fe700 1 -- 172.21.15.83:0/1398428908 learned_addr learned my addr 172.21.15.83:0/1398428908 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f625f7fe700 1 -- 172.21.15.83:0/1398428908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62600d0870 msgr2=0x7f62600d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:44.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f625f7fe700 1 --2- 172.21.15.83:0/1398428908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62600d0870 0x7f62600d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f625f7fe700 1 -- 172.21.15.83:0/1398428908 >> v1:172.21.15.203:6789/0 conn(0x7f626015e1e0 legacy=0x7f62600d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f625f7fe700 1 -- 172.21.15.83:0/1398428908 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62600d20b0 con 0x7f62600b6ef0 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.348+0000 7f625f7fe700 1 --2- 172.21.15.83:0/1398428908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62600b6ef0 0x7f62600b7380 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6258009e50 tx=0x7f625800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cf8130531bed0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.349+0000 7f625dffb700 1 -- 172.21.15.83:0/1398428908 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625800e070 con 0x7f62600b6ef0 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.349+0000 7f625dffb700 1 -- 172.21.15.83:0/1398428908 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6258003a20 con 0x7f62600b6ef0 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.349+0000 7f625dffb700 1 -- 172.21.15.83:0/1398428908 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258003dd0 con 0x7f62600b6ef0 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.349+0000 7f626581e700 1 -- 172.21.15.83:0/1398428908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62600b6ef0 msgr2=0x7f62600b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.349+0000 7f626581e700 1 --2- 172.21.15.83:0/1398428908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62600b6ef0 0x7f62600b7380 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6258009e50 tx=0x7f625800b330 comp rx=0 tx=0).stop 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.349+0000 7f626581e700 1 -- 172.21.15.83:0/1398428908 shutdown_connections 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.349+0000 7f626581e700 1 --2- 172.21.15.83:0/1398428908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62600b6ef0 0x7f62600b7380 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.349+0000 7f626581e700 1 --2- 172.21.15.83:0/1398428908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62600d0870 0x7f62600d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.349+0000 7f626581e700 1 -- 172.21.15.83:0/1398428908 >> 172.21.15.83:0/1398428908 conn(0x7f62600b5b50 msgr2=0x7f62600b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:44.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.349+0000 7f626581e700 1 -- 172.21.15.83:0/1398428908 shutdown_connections 2023-11-26T07:17:44.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.349+0000 7f626581e700 1 -- 172.21.15.83:0/1398428908 wait complete. 2023-11-26T07:17:44.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.350+0000 7f626581e700 1 Processor -- start 2023-11-26T07:17:44.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f626581e700 1 -- start start 2023-11-26T07:17:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f626581e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62600b6ef0 0x7f62600cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f626581e700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62600d0870 0x7f62600cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f626581e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626015e1e0 0x7f6260154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f626581e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6260161540 con 0x7f626015e1e0 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f626581e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6260155c90 con 0x7f62600b6ef0 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f626581e700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6260155f70 con 0x7f62600d0870 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f625f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626015e1e0 0x7f6260154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f625f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626015e1e0 0x7f6260154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42362/0 (socket says 172.21.15.83:42362) 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f625f7fe700 1 -- 172.21.15.83:0/86189821 learned_addr learned my addr 172.21.15.83:0/86189821 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f625effd700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62600b6ef0 0x7f62600cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.351+0000 7f625e7fc700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62600d0870 0x7f62600cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f625f7fe700 1 -- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62600d0870 msgr2=0x7f62600cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f625f7fe700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62600d0870 0x7f62600cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f625f7fe700 1 -- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62600b6ef0 msgr2=0x7f62600cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f625f7fe700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62600b6ef0 0x7f62600cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f625f7fe700 1 -- 172.21.15.83:0/86189821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6258009b30 con 0x7f626015e1e0 2023-11-26T07:17:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f625effd700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62600b6ef0 0x7f62600cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f625f7fe700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626015e1e0 0x7f6260154fb0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f626008c570 tx=0x7f6258004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f6247fff700 1 -- 172.21.15.83:0/86189821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62580092e0 con 0x7f626015e1e0 2023-11-26T07:17:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f6247fff700 1 -- 172.21.15.83:0/86189821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6258004240 con 0x7f626015e1e0 2023-11-26T07:17:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f626581e700 1 -- 172.21.15.83:0/86189821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6260156250 con 0x7f626015e1e0 2023-11-26T07:17:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f6247fff700 1 -- 172.21.15.83:0/86189821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258004610 con 0x7f626015e1e0 2023-11-26T07:17:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.352+0000 7f626581e700 1 -- 172.21.15.83:0/86189821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6260156810 con 0x7f626015e1e0 2023-11-26T07:17:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.353+0000 7f626581e700 1 -- 172.21.15.83:0/86189821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62601569a0 con 0x7f626015e1e0 2023-11-26T07:17:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.353+0000 7f6247fff700 1 -- 172.21.15.83:0/86189821 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f6258019070 con 0x7f626015e1e0 2023-11-26T07:17:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.354+0000 7f6247fff700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f624c06b1a0 0x7f624c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.354+0000 7f6247fff700 1 -- 172.21.15.83:0/86189821 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f625808ab50 con 0x7f626015e1e0 2023-11-26T07:17:44.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.354+0000 7f625effd700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f624c06b1a0 0x7f624c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.354+0000 7f625effd700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f624c06b1a0 0x7f624c06d660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6254009930 tx=0x7f6254008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.358+0000 7f6247fff700 1 -- 172.21.15.83:0/86189821 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6258055710 con 0x7f626015e1e0 2023-11-26T07:17:44.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.595+0000 7f626581e700 1 -- 172.21.15.83:0/86189821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62600751e0 con 0x7f626015e1e0 2023-11-26T07:17:44.597 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.595+0000 7f6247fff700 1 -- 172.21.15.83:0/86189821 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f6258017020 con 0x7f626015e1e0 2023-11-26T07:17:44.598 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:44.598 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/136760159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":136760159},{"type":"v1","addr":"172.21.15.203:6834","nonce":136760159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3723980912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":3723980912},{"type":"v1","addr":"172.21.15.203:6835","nonce":3723980912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3900154869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3900154869},{"type":"v1","addr":"172.21.15.83:6833","nonce":3900154869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:01.974687+0000","modified":"2023-11-26T07:17:04.985222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7551":{"gid":7551,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3327659486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3327659486},{"type":"v1","addr":"172.21.15.83:6835","nonce":3327659486}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:17:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 -- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f624c06b1a0 msgr2=0x7f624c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f624c06b1a0 0x7f624c06d660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6254009930 tx=0x7f6254008040 comp rx=0 tx=0).stop 2023-11-26T07:17:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 -- 172.21.15.83:0/86189821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626015e1e0 msgr2=0x7f6260154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626015e1e0 0x7f6260154fb0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f626008c570 tx=0x7f6258004ad0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 -- 172.21.15.83:0/86189821 shutdown_connections 2023-11-26T07:17:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f624c06b1a0 0x7f624c06d660 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f626015e1e0 0x7f6260154fb0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62600b6ef0 0x7f62600cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 --2- 172.21.15.83:0/86189821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62600d0870 0x7f62600cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 -- 172.21.15.83:0/86189821 >> 172.21.15.83:0/86189821 conn(0x7f62600b5b50 msgr2=0x7f62600d4120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 -- 172.21.15.83:0/86189821 shutdown_connections 2023-11-26T07:17:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.597+0000 7f626581e700 1 -- 172.21.15.83:0/86189821 wait complete. 2023-11-26T07:17:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 70 2023-11-26T07:17:44.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.729+0000 7f47ebd08700 1 Processor -- start 2023-11-26T07:17:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.729+0000 7f47ebd08700 1 -- start start 2023-11-26T07:17:44.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.729+0000 7f47ebd08700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f47e406c710 0x7f47e40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.729+0000 7f47ebd08700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e40af4d0 0x7f47e40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.729+0000 7f47ebd08700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e40ae430 con 0x7f47e415e080 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.729+0000 7f47ebd08700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e40af0b0 con 0x7f47e406c710 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.729+0000 7f47ebd08700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e40b05a0 con 0x7f47e40af4d0 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47e92a3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f47e406c710 0x7f47e40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47e92a3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f47e406c710 0x7f47e40b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42368/0 (socket says 172.21.15.83:42368) 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47e92a3700 1 -- 172.21.15.83:0/1693178128 learned_addr learned my addr 172.21.15.83:0/1693178128 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47ea2a5700 1 --2- 172.21.15.83:0/1693178128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e40af4d0 0x7f47e40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47e92a3700 1 -- 172.21.15.83:0/1693178128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e40af4d0 msgr2=0x7f47e40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47e92a3700 1 --2- 172.21.15.83:0/1693178128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e40af4d0 0x7f47e40afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47e92a3700 1 -- 172.21.15.83:0/1693178128 >> v1:172.21.15.203:6789/0 conn(0x7f47e415e080 legacy=0x7f47e406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47e92a3700 1 -- 172.21.15.83:0/1693178128 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e40ae710 con 0x7f47e406c710 2023-11-26T07:17:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47e92a3700 1 --2- 172.21.15.83:0/1693178128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e406c710 0x7f47e40b5050 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f47e00098f0 tx=0x7f47e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd0954c66b30473 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47e8aa2700 1 -- 172.21.15.83:0/1693178128 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000b8a0 con 0x7f47e406c710 2023-11-26T07:17:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47e8aa2700 1 -- 172.21.15.83:0/1693178128 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e000ba00 con 0x7f47e406c710 2023-11-26T07:17:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.730+0000 7f47e8aa2700 1 -- 172.21.15.83:0/1693178128 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000bd90 con 0x7f47e406c710 2023-11-26T07:17:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.731+0000 7f47ebd08700 1 -- 172.21.15.83:0/1693178128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e406c710 msgr2=0x7f47e40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.731+0000 7f47ebd08700 1 --2- 172.21.15.83:0/1693178128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e406c710 0x7f47e40b5050 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f47e00098f0 tx=0x7f47e0009c00 comp rx=0 tx=0).stop 2023-11-26T07:17:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.731+0000 7f47ebd08700 1 -- 172.21.15.83:0/1693178128 shutdown_connections 2023-11-26T07:17:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.731+0000 7f47ebd08700 1 --2- 172.21.15.83:0/1693178128 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e40af4d0 0x7f47e40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.731+0000 7f47ebd08700 1 --2- 172.21.15.83:0/1693178128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e406c710 0x7f47e40b5050 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.731+0000 7f47ebd08700 1 -- 172.21.15.83:0/1693178128 >> 172.21.15.83:0/1693178128 conn(0x7f47e4071b90 msgr2=0x7f47e4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.731+0000 7f47ebd08700 1 -- 172.21.15.83:0/1693178128 shutdown_connections 2023-11-26T07:17:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.731+0000 7f47ebd08700 1 -- 172.21.15.83:0/1693178128 wait complete. 2023-11-26T07:17:44.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.732+0000 7f47ebd08700 1 Processor -- start 2023-11-26T07:17:44.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.732+0000 7f47ebd08700 1 -- start start 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47ebd08700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e406c710 0x7f47e41f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47ebd08700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e40af4d0 0x7f47e40b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47ebd08700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e415e080 0x7f47e4144650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47ebd08700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e41f7400 con 0x7f47e40af4d0 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47ebd08700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e41f76e0 con 0x7f47e406c710 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47ebd08700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e41f79c0 con 0x7f47e415e080 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47e92a3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e40af4d0 0x7f47e40b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47e92a3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e40af4d0 0x7f47e40b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42374/0 (socket says 172.21.15.83:42374) 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47e92a3700 1 -- 172.21.15.83:0/157600004 learned_addr learned my addr 172.21.15.83:0/157600004 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47e9aa4700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e406c710 0x7f47e41f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47ea2a5700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e415e080 0x7f47e4144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47e92a3700 1 -- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e415e080 msgr2=0x7f47e4144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47e92a3700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e415e080 0x7f47e4144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47e92a3700 1 -- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e406c710 msgr2=0x7f47e41f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47e92a3700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e406c710 0x7f47e41f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.733+0000 7f47e92a3700 1 -- 172.21.15.83:0/157600004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0009550 con 0x7f47e40af4d0 2023-11-26T07:17:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.734+0000 7f47e92a3700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e40af4d0 0x7f47e40b4300 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f47e0000c00 tx=0x7f47e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.735+0000 7f47da7fc700 1 -- 172.21.15.83:0/157600004 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0004300 con 0x7f47e40af4d0 2023-11-26T07:17:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.735+0000 7f47da7fc700 1 -- 172.21.15.83:0/157600004 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e001f070 con 0x7f47e40af4d0 2023-11-26T07:17:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.735+0000 7f47ebd08700 1 -- 172.21.15.83:0/157600004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e4144130 con 0x7f47e40af4d0 2023-11-26T07:17:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.735+0000 7f47da7fc700 1 -- 172.21.15.83:0/157600004 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e00036a0 con 0x7f47e40af4d0 2023-11-26T07:17:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.735+0000 7f47ebd08700 1 -- 172.21.15.83:0/157600004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e40b4840 con 0x7f47e40af4d0 2023-11-26T07:17:44.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.736+0000 7f47ebd08700 1 -- 172.21.15.83:0/157600004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e40b4a30 con 0x7f47e40af4d0 2023-11-26T07:17:44.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.736+0000 7f47da7fc700 1 -- 172.21.15.83:0/157600004 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f47e0021030 con 0x7f47e40af4d0 2023-11-26T07:17:44.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.736+0000 7f47da7fc700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f47c806b1a0 0x7f47c806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.740 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.737+0000 7f47da7fc700 1 -- 172.21.15.83:0/157600004 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47e008ad00 con 0x7f47e40af4d0 2023-11-26T07:17:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.737+0000 7f47e9aa4700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f47c806b1a0 0x7f47c806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.737+0000 7f47e9aa4700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f47c806b1a0 0x7f47c806d660 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f47d4007fd0 tx=0x7f47d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.743+0000 7f47da7fc700 1 -- 172.21.15.83:0/157600004 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47e0055a70 con 0x7f47e40af4d0 2023-11-26T07:17:44.937 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.935+0000 7f47ebd08700 1 -- 172.21.15.83:0/157600004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47e40753f0 con 0x7f47e40af4d0 2023-11-26T07:17:44.938 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.936+0000 7f47da7fc700 1 -- 172.21.15.83:0/157600004 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16515 (secure 0 0 0) 0x7f47e0059090 con 0x7f47e40af4d0 2023-11-26T07:17:44.938 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:44.939 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":102,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:17:03.554036+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:00.194045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:00.926172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:44.940 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 -- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f47c806b1a0 msgr2=0x7f47c806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f47c806b1a0 0x7f47c806d660 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f47d4007fd0 tx=0x7f47d4009040 comp rx=0 tx=0).stop 2023-11-26T07:17:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 -- 172.21.15.83:0/157600004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e40af4d0 msgr2=0x7f47e40b4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e40af4d0 0x7f47e40b4300 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f47e0000c00 tx=0x7f47e0004750 comp rx=0 tx=0).stop 2023-11-26T07:17:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 -- 172.21.15.83:0/157600004 shutdown_connections 2023-11-26T07:17:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f47c806b1a0 0x7f47c806d660 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47e415e080 0x7f47e4144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e406c710 0x7f47e41f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 --2- 172.21.15.83:0/157600004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e40af4d0 0x7f47e40b4300 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 -- 172.21.15.83:0/157600004 >> 172.21.15.83:0/157600004 conn(0x7f47e4071b90 msgr2=0x7f47e40b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 -- 172.21.15.83:0/157600004 shutdown_connections 2023-11-26T07:17:44.941 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:44.938+0000 7f47ebd08700 1 -- 172.21.15.83:0/157600004 wait complete. 2023-11-26T07:17:44.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:17:45.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f51787e9700 1 Processor -- start 2023-11-26T07:17:45.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f51787e9700 1 -- start start 2023-11-26T07:17:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f51787e9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f517015e080 0x7f51700b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f51787e9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f517006c710 0x7f51700afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f51787e9700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51700ae430 con 0x7f51700af4d0 2023-11-26T07:17:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f51787e9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51700af0b0 con 0x7f517006c710 2023-11-26T07:17:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f51787e9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51700b05a0 con 0x7f517015e080 2023-11-26T07:17:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f5176d86700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f517006c710 0x7f51700afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f5176d86700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f517006c710 0x7f51700afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42382/0 (socket says 172.21.15.83:42382) 2023-11-26T07:17:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f5175d84700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f51700af4d0 0x7f517006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34188/0 (socket says 172.21.15.83:34188) 2023-11-26T07:17:45.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f5176d86700 1 -- 172.21.15.83:0/2315765176 learned_addr learned my addr 172.21.15.83:0/2315765176 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f5176585700 1 --2- 172.21.15.83:0/2315765176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f517015e080 0x7f51700b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f5176d86700 1 -- 172.21.15.83:0/2315765176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f517015e080 msgr2=0x7f51700b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f5176d86700 1 --2- 172.21.15.83:0/2315765176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f517015e080 0x7f51700b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f5176d86700 1 -- 172.21.15.83:0/2315765176 >> v1:172.21.15.83:6789/0 conn(0x7f51700af4d0 legacy=0x7f517006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f5175583700 1 -- 172.21.15.83:0/2315765176 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1260096187 0 0) 0x7f51700ae430 con 0x7f51700af4d0 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f5176d86700 1 -- 172.21.15.83:0/2315765176 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51700ae710 con 0x7f517006c710 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.072+0000 7f5176d86700 1 --2- 172.21.15.83:0/2315765176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f517006c710 0x7f51700afb90 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5168009e80 tx=0x7f516800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2326a7b2f1372b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.073+0000 7f5175583700 1 -- 172.21.15.83:0/2315765176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516800e070 con 0x7f517006c710 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.073+0000 7f5175583700 1 -- 172.21.15.83:0/2315765176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516800b8d0 con 0x7f517006c710 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.073+0000 7f5175583700 1 -- 172.21.15.83:0/2315765176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516800bc80 con 0x7f517006c710 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.073+0000 7f51787e9700 1 -- 172.21.15.83:0/2315765176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f517006c710 msgr2=0x7f51700afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.073+0000 7f51787e9700 1 --2- 172.21.15.83:0/2315765176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f517006c710 0x7f51700afb90 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5168009e80 tx=0x7f516800b330 comp rx=0 tx=0).stop 2023-11-26T07:17:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.073+0000 7f51787e9700 1 -- 172.21.15.83:0/2315765176 shutdown_connections 2023-11-26T07:17:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.073+0000 7f51787e9700 1 --2- 172.21.15.83:0/2315765176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f517006c710 0x7f51700afb90 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.073+0000 7f51787e9700 1 --2- 172.21.15.83:0/2315765176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f517015e080 0x7f51700b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.073+0000 7f51787e9700 1 -- 172.21.15.83:0/2315765176 >> 172.21.15.83:0/2315765176 conn(0x7f5170071b90 msgr2=0x7f5170071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.073+0000 7f51787e9700 1 -- 172.21.15.83:0/2315765176 shutdown_connections 2023-11-26T07:17:45.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.073+0000 7f51787e9700 1 -- 172.21.15.83:0/2315765176 wait complete. 2023-11-26T07:17:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.074+0000 7f51787e9700 1 Processor -- start 2023-11-26T07:17:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.075+0000 7f51787e9700 1 -- start start 2023-11-26T07:17:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.075+0000 7f51787e9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f517006c710 0x7f5170150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.075+0000 7f51787e9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51700af4d0 0x7f51701545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.075+0000 7f51787e9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f517015e080 0x7f51700c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.075+0000 7f51787e9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51701f7a90 con 0x7f517006c710 2023-11-26T07:17:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.075+0000 7f51787e9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5170154b10 con 0x7f51700af4d0 2023-11-26T07:17:45.081 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.075+0000 7f51787e9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51701f6f10 con 0x7f517015e080 2023-11-26T07:17:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.075+0000 7f5176585700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f517006c710 0x7f5170150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.075+0000 7f5176585700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f517006c710 0x7f5170150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42386/0 (socket says 172.21.15.83:42386) 2023-11-26T07:17:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.075+0000 7f5176585700 1 -- 172.21.15.83:0/871633330 learned_addr learned my addr 172.21.15.83:0/871633330 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5176d86700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f517015e080 0x7f51700c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5175d84700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51700af4d0 0x7f51701545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5176585700 1 -- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f517015e080 msgr2=0x7f51700c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5176585700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f517015e080 0x7f51700c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5176585700 1 -- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51700af4d0 msgr2=0x7f51701545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5176585700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51700af4d0 0x7f51701545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5176585700 1 -- 172.21.15.83:0/871633330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5168009b30 con 0x7f517006c710 2023-11-26T07:17:45.082 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5175d84700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51700af4d0 0x7f51701545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5176585700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f517006c710 0x7f5170150a60 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f516c00e870 tx=0x7f516c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5166ffd700 1 -- 172.21.15.83:0/871633330 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c009590 con 0x7f517006c710 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5166ffd700 1 -- 172.21.15.83:0/871633330 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c00ee70 con 0x7f517006c710 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f51787e9700 1 -- 172.21.15.83:0/871633330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51701f71f0 con 0x7f517006c710 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f5166ffd700 1 -- 172.21.15.83:0/871633330 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c00ca50 con 0x7f517006c710 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.076+0000 7f51787e9700 1 -- 172.21.15.83:0/871633330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51701f77b0 con 0x7f517006c710 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.077+0000 7f51787e9700 1 -- 172.21.15.83:0/871633330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51701574f0 con 0x7f517006c710 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.077+0000 7f5166ffd700 1 -- 172.21.15.83:0/871633330 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f516c00cbb0 con 0x7f517006c710 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.078+0000 7f5166ffd700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f515406b1a0 0x7f515406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.078+0000 7f5166ffd700 1 -- 172.21.15.83:0/871633330 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f516c089920 con 0x7f517006c710 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.078+0000 7f5175d84700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f515406b1a0 0x7f515406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.083 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.079+0000 7f5175d84700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f515406b1a0 0x7f515406d660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f51701f6a50 tx=0x7f5160007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.085 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.083+0000 7f5166ffd700 1 -- 172.21.15.83:0/871633330 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f516c054710 con 0x7f517006c710 2023-11-26T07:17:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.314+0000 7f51787e9700 1 -- 172.21.15.83:0/871633330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f51700753f0 con 0x7f517006c710 2023-11-26T07:17:45.316 INFO:tasks.ceph.mon.a.smithi083.stderr:2023-11-26T07:17:45.314+0000 7fcd25823700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:17:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.315+0000 7f5166ffd700 1 -- 172.21.15.83:0/871633330 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) v1 ==== 161+0+0 (secure 0 0 0) 0x7f516c021090 con 0x7f517006c710 2023-11-26T07:17:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.318+0000 7f51787e9700 1 -- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f515406b1a0 msgr2=0x7f515406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.318+0000 7f51787e9700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f515406b1a0 0x7f515406d660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f51701f6a50 tx=0x7f5160007ca0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.318+0000 7f51787e9700 1 -- 172.21.15.83:0/871633330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f517006c710 msgr2=0x7f5170150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.318+0000 7f51787e9700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f517006c710 0x7f5170150a60 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f516c00e870 tx=0x7f516c00ec30 comp rx=0 tx=0).stop 2023-11-26T07:17:45.321 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.319+0000 7f51787e9700 1 -- 172.21.15.83:0/871633330 shutdown_connections 2023-11-26T07:17:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.319+0000 7f51787e9700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f515406b1a0 0x7f515406d660 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.319+0000 7f51787e9700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f517015e080 0x7f51700c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.319+0000 7f51787e9700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f517006c710 0x7f5170150a60 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.319+0000 7f51787e9700 1 --2- 172.21.15.83:0/871633330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51700af4d0 0x7f51701545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.319+0000 7f51787e9700 1 -- 172.21.15.83:0/871633330 >> 172.21.15.83:0/871633330 conn(0x7f5170071b90 msgr2=0x7f517006e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.319+0000 7f51787e9700 1 -- 172.21.15.83:0/871633330 shutdown_connections 2023-11-26T07:17:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.319+0000 7f51787e9700 1 -- 172.21.15.83:0/871633330 wait complete. 2023-11-26T07:17:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:17:45.334 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:17:45.360 INFO:tasks.ceph.mds.a.smithi083.stdout:starting mds.a at 2023-11-26T07:17:45.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.454+0000 7f2d17afc700 1 Processor -- start 2023-11-26T07:17:45.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.454+0000 7f2d17afc700 1 -- start start 2023-11-26T07:17:45.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d17afc700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2d1015e160 0x7f2d100b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d17afc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d100746a0 0x7f2d1006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d17afc700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d100b3c60 con 0x7f2d100afc90 2023-11-26T07:17:45.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d17afc700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100b4500 con 0x7f2d100746a0 2023-11-26T07:17:45.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d17afc700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100b58c0 con 0x7f2d1015e160 2023-11-26T07:17:45.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d16099700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2d100746a0 0x7f2d1006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d15097700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f2d100afc90 0x7f2d10073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34214/0 (socket says 172.21.15.83:34214) 2023-11-26T07:17:45.459 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d15097700 1 -- 172.21.15.83:0/1694251638 learned_addr learned my addr 172.21.15.83:0/1694251638 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d15898700 1 --2- 172.21.15.83:0/1694251638 >> v2:172.21.15.203:3300/0 conn(0x7f2d1015e160 0x7f2d100b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d16099700 1 -- 172.21.15.83:0/1694251638 >> v2:172.21.15.203:3300/0 conn(0x7f2d1015e160 msgr2=0x7f2d100b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d16099700 1 --2- 172.21.15.83:0/1694251638 >> v2:172.21.15.203:3300/0 conn(0x7f2d1015e160 0x7f2d100b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d16099700 1 -- 172.21.15.83:0/1694251638 >> v1:172.21.15.83:6789/0 conn(0x7f2d100afc90 legacy=0x7f2d10073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d16099700 1 -- 172.21.15.83:0/1694251638 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100b0140 con 0x7f2d100746a0 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d14896700 1 -- 172.21.15.83:0/1694251638 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458440590 0 0) 0x7f2d100b3c60 con 0x7f2d100afc90 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.455+0000 7f2d16099700 1 --2- 172.21.15.83:0/1694251638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d100746a0 0x7f2d1006d930 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f2d1009efa0 tx=0x7f2d0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd97dc5783c2276 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.456+0000 7f2d14896700 1 -- 172.21.15.83:0/1694251638 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0800e070 con 0x7f2d100746a0 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.456+0000 7f2d14896700 1 -- 172.21.15.83:0/1694251638 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0800bbd0 con 0x7f2d100746a0 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.456+0000 7f2d14896700 1 -- 172.21.15.83:0/1694251638 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08012430 con 0x7f2d100746a0 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.456+0000 7f2d17afc700 1 -- 172.21.15.83:0/1694251638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d100746a0 msgr2=0x7f2d1006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.460 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.456+0000 7f2d17afc700 1 --2- 172.21.15.83:0/1694251638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d100746a0 0x7f2d1006d930 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f2d1009efa0 tx=0x7f2d0800b330 comp rx=0 tx=0).stop 2023-11-26T07:17:45.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.456+0000 7f2d17afc700 1 -- 172.21.15.83:0/1694251638 shutdown_connections 2023-11-26T07:17:45.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.456+0000 7f2d17afc700 1 --2- 172.21.15.83:0/1694251638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d100746a0 0x7f2d1006d930 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.456+0000 7f2d17afc700 1 --2- 172.21.15.83:0/1694251638 >> v2:172.21.15.203:3300/0 conn(0x7f2d1015e160 0x7f2d100b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.456+0000 7f2d17afc700 1 -- 172.21.15.83:0/1694251638 >> 172.21.15.83:0/1694251638 conn(0x7f2d100d0bf0 msgr2=0x7f2d100d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:45.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.456+0000 7f2d17afc700 1 -- 172.21.15.83:0/1694251638 shutdown_connections 2023-11-26T07:17:45.461 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.456+0000 7f2d17afc700 1 -- 172.21.15.83:0/1694251638 wait complete. 2023-11-26T07:17:45.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.457+0000 7f2d17afc700 1 Processor -- start 2023-11-26T07:17:45.463 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.457+0000 7f2d17afc700 1 -- start start 2023-11-26T07:17:45.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.458+0000 7f2d17afc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d100746a0 0x7f2d101f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.458+0000 7f2d17afc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d100afc90 0x7f2d101f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.458+0000 7f2d17afc700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d1015e160 0x7f2d101f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.458+0000 7f2d17afc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1014b210 con 0x7f2d100afc90 2023-11-26T07:17:45.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.458+0000 7f2d17afc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d101f3990 con 0x7f2d100746a0 2023-11-26T07:17:45.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.458+0000 7f2d17afc700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100c75f0 con 0x7f2d1015e160 2023-11-26T07:17:45.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.458+0000 7f2d15097700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d100afc90 0x7f2d101f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.458+0000 7f2d15097700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d100afc90 0x7f2d101f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42412/0 (socket says 172.21.15.83:42412) 2023-11-26T07:17:45.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.458+0000 7f2d15097700 1 -- 172.21.15.83:0/2741951941 learned_addr learned my addr 172.21.15.83:0/2741951941 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:45.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.458+0000 7f2d16099700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d1015e160 0x7f2d101f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.464 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.458+0000 7f2d15898700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d100746a0 0x7f2d101f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.459+0000 7f2d15097700 1 -- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d1015e160 msgr2=0x7f2d101f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.459+0000 7f2d15097700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d1015e160 0x7f2d101f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.459+0000 7f2d15097700 1 -- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d100746a0 msgr2=0x7f2d101f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.459+0000 7f2d15097700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d100746a0 0x7f2d101f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.459+0000 7f2d15097700 1 -- 172.21.15.83:0/2741951941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08009a70 con 0x7f2d100afc90 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.459+0000 7f2d15097700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d100afc90 0x7f2d101f7110 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2d000011e0 tx=0x7f2d0000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.459+0000 7f2d067fc700 1 -- 172.21.15.83:0/2741951941 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00004c80 con 0x7f2d100afc90 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.459+0000 7f2d067fc700 1 -- 172.21.15.83:0/2741951941 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d00004e00 con 0x7f2d100afc90 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.459+0000 7f2d17afc700 1 -- 172.21.15.83:0/2741951941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1014b540 con 0x7f2d100afc90 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.459+0000 7f2d067fc700 1 -- 172.21.15.83:0/2741951941 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00007310 con 0x7f2d100afc90 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.459+0000 7f2d17afc700 1 -- 172.21.15.83:0/2741951941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1014bb00 con 0x7f2d100afc90 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.460+0000 7f2d17afc700 1 -- 172.21.15.83:0/2741951941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1014bc90 con 0x7f2d100afc90 2023-11-26T07:17:45.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.460+0000 7f2d067fc700 1 -- 172.21.15.83:0/2741951941 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f2d00007470 con 0x7f2d100afc90 2023-11-26T07:17:45.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.461+0000 7f2d067fc700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2cf406b1a0 0x7f2cf406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.461+0000 7f2d067fc700 1 -- 172.21.15.83:0/2741951941 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2d00089830 con 0x7f2d100afc90 2023-11-26T07:17:45.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.461+0000 7f2d15898700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2cf406b1a0 0x7f2cf406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.466 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.463+0000 7f2d15898700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2cf406b1a0 0x7f2cf406d660 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00b9b0 tx=0x7f2d0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.467 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.466+0000 7f2d067fc700 1 -- 172.21.15.83:0/2741951941 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d00054530 con 0x7f2d100afc90 2023-11-26T07:17:45.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:45.698+0000 7f2d17afc700 1 -- 172.21.15.83:0/2741951941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2d10068fb0 con 0x7f2d100afc90 2023-11-26T07:17:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.318+0000 7f2d067fc700 1 -- 172.21.15.83:0/2741951941 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2d00057b50 con 0x7f2d100afc90 2023-11-26T07:17:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 -- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2cf406b1a0 msgr2=0x7f2cf406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2cf406b1a0 0x7f2cf406d660 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00b9b0 tx=0x7f2d0c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 -- 172.21.15.83:0/2741951941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d100afc90 msgr2=0x7f2d101f7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d100afc90 0x7f2d101f7110 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2d000011e0 tx=0x7f2d0000cae0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 -- 172.21.15.83:0/2741951941 shutdown_connections 2023-11-26T07:17:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f2cf406b1a0 0x7f2cf406d660 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d1015e160 0x7f2d101f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d100746a0 0x7f2d101f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 --2- 172.21.15.83:0/2741951941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d100afc90 0x7f2d101f7110 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 -- 172.21.15.83:0/2741951941 >> 172.21.15.83:0/2741951941 conn(0x7f2d100d0bf0 msgr2=0x7f2d1006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 -- 172.21.15.83:0/2741951941 shutdown_connections 2023-11-26T07:17:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.321+0000 7f2d17afc700 1 -- 172.21.15.83:0/2741951941 wait complete. 2023-11-26T07:17:46.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:17:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.503+0000 7f1621db9700 1 Processor -- start 2023-11-26T07:17:46.505 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.504+0000 7f1621db9700 1 -- start start 2023-11-26T07:17:46.507 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.504+0000 7f1621db9700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f161c145620 0x7f161c06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.504+0000 7f1621db9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f161c0cd810 0x7f161c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.504+0000 7f1621db9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f161c073f90 0x7f161c074400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.504+0000 7f1621db9700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161c074940 con 0x7f161c073f90 2023-11-26T07:17:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.504+0000 7f1621db9700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161c0cf530 con 0x7f161c145620 2023-11-26T07:17:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.504+0000 7f1621db9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c06b8f0 con 0x7f161c0cd810 2023-11-26T07:17:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f161c073f90 0x7f161c074400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161bfff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f161c073f90 0x7f161c074400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42424/0 (socket says 172.21.15.83:42424) 2023-11-26T07:17:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161bfff700 1 -- 172.21.15.83:0/1419782357 learned_addr learned my addr 172.21.15.83:0/1419782357 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161b7fe700 1 --2- 172.21.15.83:0/1419782357 >> v2:172.21.15.203:3300/0 conn(0x7f161c145620 0x7f161c06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.508 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161affd700 1 --2- 172.21.15.83:0/1419782357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f161c0cd810 0x7f161c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161bfff700 1 -- 172.21.15.83:0/1419782357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f161c0cd810 msgr2=0x7f161c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161bfff700 1 --2- 172.21.15.83:0/1419782357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f161c0cd810 0x7f161c06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161bfff700 1 -- 172.21.15.83:0/1419782357 >> v2:172.21.15.203:3300/0 conn(0x7f161c145620 msgr2=0x7f161c06dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161bfff700 1 --2- 172.21.15.83:0/1419782357 >> v2:172.21.15.203:3300/0 conn(0x7f161c145620 0x7f161c06dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161bfff700 1 -- 172.21.15.83:0/1419782357 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c06bbd0 con 0x7f161c073f90 2023-11-26T07:17:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161bfff700 1 --2- 172.21.15.83:0/1419782357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161c073f90 0x7f161c074400 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f161c075070 tx=0x7f161400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117b031200216cc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161a7fc700 1 -- 172.21.15.83:0/1419782357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400e070 con 0x7f161c073f90 2023-11-26T07:17:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161a7fc700 1 -- 172.21.15.83:0/1419782357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161400b9c0 con 0x7f161c073f90 2023-11-26T07:17:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.505+0000 7f161a7fc700 1 -- 172.21.15.83:0/1419782357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400bd70 con 0x7f161c073f90 2023-11-26T07:17:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.506+0000 7f1621db9700 1 -- 172.21.15.83:0/1419782357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161c073f90 msgr2=0x7f161c074400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.509 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.506+0000 7f1621db9700 1 --2- 172.21.15.83:0/1419782357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161c073f90 0x7f161c074400 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f161c075070 tx=0x7f161400b330 comp rx=0 tx=0).stop 2023-11-26T07:17:46.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.506+0000 7f1621db9700 1 -- 172.21.15.83:0/1419782357 shutdown_connections 2023-11-26T07:17:46.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.506+0000 7f1621db9700 1 --2- 172.21.15.83:0/1419782357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161c073f90 0x7f161c074400 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.506+0000 7f1621db9700 1 --2- 172.21.15.83:0/1419782357 >> v2:172.21.15.203:3300/0 conn(0x7f161c145620 0x7f161c06dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.506+0000 7f1621db9700 1 --2- 172.21.15.83:0/1419782357 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f161c0cd810 0x7f161c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.506+0000 7f1621db9700 1 -- 172.21.15.83:0/1419782357 >> 172.21.15.83:0/1419782357 conn(0x7f161c0d0b40 msgr2=0x7f161c0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:46.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.506+0000 7f1621db9700 1 -- 172.21.15.83:0/1419782357 shutdown_connections 2023-11-26T07:17:46.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.506+0000 7f1621db9700 1 -- 172.21.15.83:0/1419782357 wait complete. 2023-11-26T07:17:46.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.507+0000 7f1621db9700 1 Processor -- start 2023-11-26T07:17:46.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.507+0000 7f1621db9700 1 -- start start 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f1621db9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161c073f90 0x7f161c139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f1621db9700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f161c0cd810 0x7f161c13cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f1621db9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f161c145620 0x7f161c159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f1621db9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c140500 con 0x7f161c073f90 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f1621db9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c139aa0 con 0x7f161c145620 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f1621db9700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c1f59f0 con 0x7f161c0cd810 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f161b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161c073f90 0x7f161c139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f161b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161c073f90 0x7f161c139560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42426/0 (socket says 172.21.15.83:42426) 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f161b7fe700 1 -- 172.21.15.83:0/3713716876 learned_addr learned my addr 172.21.15.83:0/3713716876 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f161affd700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f161c0cd810 0x7f161c13cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f161bfff700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f161c145620 0x7f161c159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f161b7fe700 1 -- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f161c0cd810 msgr2=0x7f161c13cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f161b7fe700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f161c0cd810 0x7f161c13cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f161b7fe700 1 -- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f161c145620 msgr2=0x7f161c159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f161b7fe700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f161c145620 0x7f161c159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f161b7fe700 1 -- 172.21.15.83:0/3713716876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614009b30 con 0x7f161c073f90 2023-11-26T07:17:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.508+0000 7f161b7fe700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161c073f90 0x7f161c139560 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f161000d830 tx=0x7f161000dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.509+0000 7f1603fff700 1 -- 172.21.15.83:0/3713716876 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16100125f0 con 0x7f161c073f90 2023-11-26T07:17:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.509+0000 7f1603fff700 1 -- 172.21.15.83:0/3713716876 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1610012750 con 0x7f161c073f90 2023-11-26T07:17:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.509+0000 7f1603fff700 1 -- 172.21.15.83:0/3713716876 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1610012b00 con 0x7f161c073f90 2023-11-26T07:17:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.509+0000 7f1621db9700 1 -- 172.21.15.83:0/3713716876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c1f5710 con 0x7f161c073f90 2023-11-26T07:17:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.509+0000 7f1621db9700 1 -- 172.21.15.83:0/3713716876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c13c3a0 con 0x7f161c073f90 2023-11-26T07:17:46.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.510+0000 7f1621db9700 1 -- 172.21.15.83:0/3713716876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161c067f60 con 0x7f161c073f90 2023-11-26T07:17:46.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.510+0000 7f1603fff700 1 -- 172.21.15.83:0/3713716876 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f1610012c60 con 0x7f161c073f90 2023-11-26T07:17:46.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.511+0000 7f1603fff700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f160806b1d0 0x7f160806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.511+0000 7f1603fff700 1 -- 172.21.15.83:0/3713716876 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1610089d10 con 0x7f161c073f90 2023-11-26T07:17:46.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.511+0000 7f161affd700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f160806b1d0 0x7f160806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.511+0000 7f161affd700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f160806b1d0 0x7f160806d690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f161c0b2190 tx=0x7f160c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.517 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.515+0000 7f1603fff700 1 -- 172.21.15.83:0/3713716876 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1610054aa0 con 0x7f161c073f90 2023-11-26T07:17:46.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:46.705+0000 7f1621db9700 1 -- 172.21.15.83:0/3713716876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f161c0d05f0 con 0x7f161c073f90 2023-11-26T07:17:47.326 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.325+0000 7f1603fff700 1 -- 172.21.15.83:0/3713716876 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) v1 ==== 181+0+0 (secure 0 0 0) 0x7f16100580c0 con 0x7f161c073f90 2023-11-26T07:17:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.327+0000 7f1621db9700 1 -- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f160806b1d0 msgr2=0x7f160806d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.327+0000 7f1621db9700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f160806b1d0 0x7f160806d690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f161c0b2190 tx=0x7f160c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.327+0000 7f1621db9700 1 -- 172.21.15.83:0/3713716876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161c073f90 msgr2=0x7f161c139560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.327+0000 7f1621db9700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161c073f90 0x7f161c139560 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f161000d830 tx=0x7f161000dbf0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.327+0000 7f1621db9700 1 -- 172.21.15.83:0/3713716876 shutdown_connections 2023-11-26T07:17:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.327+0000 7f1621db9700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f160806b1d0 0x7f160806d690 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.327+0000 7f1621db9700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f161c145620 0x7f161c159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.327+0000 7f1621db9700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f161c073f90 0x7f161c139560 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.327+0000 7f1621db9700 1 --2- 172.21.15.83:0/3713716876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f161c0cd810 0x7f161c13cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.327+0000 7f1621db9700 1 -- 172.21.15.83:0/3713716876 >> 172.21.15.83:0/3713716876 conn(0x7f161c0d0b40 msgr2=0x7f161c0b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.327+0000 7f1621db9700 1 -- 172.21.15.83:0/3713716876 shutdown_connections 2023-11-26T07:17:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.328+0000 7f1621db9700 1 -- 172.21.15.83:0/3713716876 wait complete. 2023-11-26T07:17:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:17:47.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:17:47.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.469+0000 7f194da83700 1 Processor -- start 2023-11-26T07:17:47.470 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.469+0000 7f194da83700 1 -- start start 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f194da83700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f194806e9c0 0x7f19480b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f194da83700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194806ee50 con 0x7f19480746a0 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f194da83700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19480b05c0 con 0x7f19480b01e0 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f194da83700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19481470a0 con 0x7f194806e9c0 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f1946ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f19480746a0 0x7f194806aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34242/0 (socket says 172.21.15.83:34242) 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f1946ffd700 1 -- 172.21.15.83:0/312693614 learned_addr learned my addr 172.21.15.83:0/312693614 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847808634 0 0) 0x7f194806ee50 con 0x7f19480746a0 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19300035b0 con 0x7f19480746a0 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f1947fff700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f194806e9c0 0x7f19480b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1308122005 0 0) 0x7f19300035b0 con 0x7f19480746a0 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f194806e9c0 msgr2=0x7f19480b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f194806e9c0 0x7f19480b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 >> v1:172.21.15.203:6789/0 conn(0x7f19480b01e0 legacy=0x7f1948073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19480b4250 con 0x7f19480746a0 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f193c001670 con 0x7f19480746a0 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 >> v1:172.21.15.83:6789/0 conn(0x7f19480746a0 legacy=0x7f194806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1930003a30 0x7f1930003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1930004850 0x7f1930004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f194806e9c0 0x7f1930005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.475 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19481470a0 con 0x7f1930004850 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193000bb40 con 0x7f1930003a30 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.470+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193000bd00 con 0x7f194806e9c0 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f1946ffd700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1930004850 0x7f1930004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f19477fe700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1930003a30 0x7f1930003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f1947fff700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f194806e9c0 0x7f1930005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f1946ffd700 1 -- 172.21.15.83:0/312693614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f194806e9c0 msgr2=0x7f1930005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f1946ffd700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f194806e9c0 0x7f1930005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f1946ffd700 1 -- 172.21.15.83:0/312693614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1930003a30 msgr2=0x7f1930003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f1946ffd700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1930003a30 0x7f1930003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f1946ffd700 1 -- 172.21.15.83:0/312693614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19300035b0 con 0x7f1930004850 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f1946ffd700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1930004850 0x7f1930004cd0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c270 tx=0x7f1930005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed2902e08a68ebf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.476 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c007c10 con 0x7f1930004850 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c00cd90 con 0x7f1930004850 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.471+0000 7f19467fc700 1 -- 172.21.15.83:0/312693614 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c005ba0 con 0x7f1930004850 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.472+0000 7f194da83700 1 -- 172.21.15.83:0/312693614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1930004850 msgr2=0x7f1930004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.472+0000 7f194da83700 1 --2- 172.21.15.83:0/312693614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1930004850 0x7f1930004cd0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c270 tx=0x7f1930005880 comp rx=0 tx=0).stop 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.472+0000 7f1947fff700 1 -- 172.21.15.83:0/312693614 reap_dead start 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.472+0000 7f194da83700 1 -- 172.21.15.83:0/312693614 shutdown_connections 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.472+0000 7f194da83700 1 -- 172.21.15.83:0/312693614 >> 172.21.15.83:0/312693614 conn(0x7f19480d0b40 msgr2=0x7f19480d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.472+0000 7f194da83700 1 -- 172.21.15.83:0/312693614 shutdown_connections 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.472+0000 7f194da83700 1 -- 172.21.15.83:0/312693614 wait complete. 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.473+0000 7f194da83700 1 Processor -- start 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.473+0000 7f194da83700 1 -- start start 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.474+0000 7f194da83700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1930004850 0x7f19480cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.474+0000 7f194da83700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19480c84e0 0x7f19480c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.474+0000 7f194da83700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1948160af0 0x7f1948160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.474+0000 7f194da83700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1948133fe0 con 0x7f19480c84e0 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.474+0000 7f194da83700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19481614b0 con 0x7f1930004850 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.474+0000 7f194da83700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19480c8f80 con 0x7f1948160af0 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.474+0000 7f1946ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19480c84e0 0x7f19480c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.474+0000 7f1946ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19480c84e0 0x7f19480c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42446/0 (socket says 172.21.15.83:42446) 2023-11-26T07:17:47.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.474+0000 7f1946ffd700 1 -- 172.21.15.83:0/2478950883 learned_addr learned my addr 172.21.15.83:0/2478950883 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:47.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f19477fe700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1930004850 0x7f19480cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f1947fff700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1948160af0 0x7f1948160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f1946ffd700 1 -- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1948160af0 msgr2=0x7f1948160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f1946ffd700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1948160af0 0x7f1948160f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f1946ffd700 1 -- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1930004850 msgr2=0x7f19480cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f1946ffd700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1930004850 0x7f19480cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f1946ffd700 1 -- 172.21.15.83:0/2478950883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19480c9260 con 0x7f19480c84e0 2023-11-26T07:17:47.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f1946ffd700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19480c84e0 0x7f19480c8960 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c860 tx=0x7f19480b0750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f192ffff700 1 -- 172.21.15.83:0/2478950883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c001640 con 0x7f19480c84e0 2023-11-26T07:17:47.481 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f192ffff700 1 -- 172.21.15.83:0/2478950883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c0070d0 con 0x7f19480c84e0 2023-11-26T07:17:47.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f194da83700 1 -- 172.21.15.83:0/2478950883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19480c9540 con 0x7f19480c84e0 2023-11-26T07:17:47.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f192ffff700 1 -- 172.21.15.83:0/2478950883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c0074a0 con 0x7f19480c84e0 2023-11-26T07:17:47.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.475+0000 7f194da83700 1 -- 172.21.15.83:0/2478950883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1948144320 con 0x7f19480c84e0 2023-11-26T07:17:47.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.476+0000 7f194da83700 1 -- 172.21.15.83:0/2478950883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19480c9780 con 0x7f19480c84e0 2023-11-26T07:17:47.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.476+0000 7f192ffff700 1 -- 172.21.15.83:0/2478950883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f193c019030 con 0x7f19480c84e0 2023-11-26T07:17:47.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.477+0000 7f192ffff700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f193406b1a0 0x7f193406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.477+0000 7f192ffff700 1 -- 172.21.15.83:0/2478950883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f193c08a620 con 0x7f19480c84e0 2023-11-26T07:17:47.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.477+0000 7f19477fe700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f193406b1a0 0x7f193406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.482 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.477+0000 7f19477fe700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f193406b1a0 0x7f193406d660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1938007fd0 tx=0x7f1938009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.483 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.481+0000 7f192ffff700 1 -- 172.21.15.83:0/2478950883 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19480c9780 con 0x7f19480c84e0 2023-11-26T07:17:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:47.671+0000 7f194da83700 1 -- 172.21.15.83:0/2478950883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f19480c9780 con 0x7f19480c84e0 2023-11-26T07:17:48.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.563+0000 7f192ffff700 1 -- 172.21.15.83:0/2478950883 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) v1 ==== 169+0+0 (secure 0 0 0) 0x7f19480c9780 con 0x7f19480c84e0 2023-11-26T07:17:48.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.567+0000 7f194da83700 1 -- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f193406b1a0 msgr2=0x7f193406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.567+0000 7f194da83700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f193406b1a0 0x7f193406d660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1938007fd0 tx=0x7f1938009040 comp rx=0 tx=0).stop 2023-11-26T07:17:48.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.567+0000 7f194da83700 1 -- 172.21.15.83:0/2478950883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19480c84e0 msgr2=0x7f19480c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.567+0000 7f194da83700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19480c84e0 0x7f19480c8960 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c860 tx=0x7f19480b0750 comp rx=0 tx=0).stop 2023-11-26T07:17:48.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.568+0000 7f194da83700 1 -- 172.21.15.83:0/2478950883 shutdown_connections 2023-11-26T07:17:48.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.568+0000 7f194da83700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f193406b1a0 0x7f193406d660 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.568+0000 7f194da83700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1948160af0 0x7f1948160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.568+0000 7f194da83700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1930004850 0x7f19480cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.568+0000 7f194da83700 1 --2- 172.21.15.83:0/2478950883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19480c84e0 0x7f19480c8960 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.568+0000 7f194da83700 1 -- 172.21.15.83:0/2478950883 >> 172.21.15.83:0/2478950883 conn(0x7f19480d0b40 msgr2=0x7f1948071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:48.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.568+0000 7f194da83700 1 -- 172.21.15.83:0/2478950883 shutdown_connections 2023-11-26T07:17:48.570 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.568+0000 7f194da83700 1 -- 172.21.15.83:0/2478950883 wait complete. 2023-11-26T07:17:48.572 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2023-11-26T07:17:48.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:17:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.712+0000 7eff0683c700 1 Processor -- start 2023-11-26T07:17:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.712+0000 7eff0683c700 1 -- start start 2023-11-26T07:17:48.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.713+0000 7eff0683c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff0015e290 0x7eff000ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:48.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.713+0000 7eff0683c700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0006e400 con 0x7eff000aedc0 2023-11-26T07:17:48.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.713+0000 7eff0683c700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0006e630 con 0x7eff000ad870 2023-11-26T07:17:48.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.713+0000 7eff0683c700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0006fde0 con 0x7eff0015e290 2023-11-26T07:17:48.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.713+0000 7efeff7fe700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff0015e290 0x7eff000ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.713+0000 7efefffff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7eff000aedc0 0x7eff000af1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34262/0 (socket says 172.21.15.83:34262) 2023-11-26T07:17:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.713+0000 7efefffff700 1 -- 172.21.15.83:0/3920130904 learned_addr learned my addr 172.21.15.83:0/3920130904 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.713+0000 7efeff7fe700 1 -- 172.21.15.83:0/3920130904 >> v1:172.21.15.203:6789/0 conn(0x7eff000ad870 legacy=0x7eff0006f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.713+0000 7efefeffd700 1 -- 172.21.15.83:0/3920130904 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365096136 0 0) 0x7eff0006e400 con 0x7eff000aedc0 2023-11-26T07:17:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7efeff7fe700 1 -- 172.21.15.83:0/3920130904 >> v1:172.21.15.83:6789/0 conn(0x7eff000aedc0 legacy=0x7eff000af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7efeff7fe700 1 -- 172.21.15.83:0/3920130904 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0006e910 con 0x7eff0015e290 2023-11-26T07:17:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7efeff7fe700 1 --2- 172.21.15.83:0/3920130904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff0015e290 0x7eff000ae790 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7efef0009840 tx=0x7efef0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0023c2505701a05 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7efefeffd700 1 -- 172.21.15.83:0/3920130904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0004230 con 0x7eff0015e290 2023-11-26T07:17:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7efefeffd700 1 -- 172.21.15.83:0/3920130904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef0004390 con 0x7eff0015e290 2023-11-26T07:17:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7efefeffd700 1 -- 172.21.15.83:0/3920130904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef00036a0 con 0x7eff0015e290 2023-11-26T07:17:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7eff0683c700 1 -- 172.21.15.83:0/3920130904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff0015e290 msgr2=0x7eff000ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7eff0683c700 1 --2- 172.21.15.83:0/3920130904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff0015e290 0x7eff000ae790 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7efef0009840 tx=0x7efef0009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7eff0683c700 1 -- 172.21.15.83:0/3920130904 shutdown_connections 2023-11-26T07:17:48.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7eff0683c700 1 --2- 172.21.15.83:0/3920130904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff0015e290 0x7eff000ae790 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7eff0683c700 1 -- 172.21.15.83:0/3920130904 >> 172.21.15.83:0/3920130904 conn(0x7eff000b2050 msgr2=0x7eff000b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:48.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7eff0683c700 1 -- 172.21.15.83:0/3920130904 shutdown_connections 2023-11-26T07:17:48.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.714+0000 7eff0683c700 1 -- 172.21.15.83:0/3920130904 wait complete. 2023-11-26T07:17:48.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.716+0000 7eff0683c700 1 Processor -- start 2023-11-26T07:17:48.718 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.716+0000 7eff0683c700 1 -- start start 2023-11-26T07:17:48.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.716+0000 7eff0683c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff000ad870 0x7eff001519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:48.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.716+0000 7eff0683c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff000aedc0 0x7eff0014b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:48.720 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7eff0683c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0015e290 0x7eff000b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7eff0683c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0014bb00 con 0x7eff000ad870 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7eff0683c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff000ccd30 con 0x7eff000aedc0 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7eff0683c700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff001602b0 con 0x7eff0015e290 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff000ad870 0x7eff001519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff000ad870 0x7eff001519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42460/0 (socket says 172.21.15.83:42460) 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefffff700 1 -- 172.21.15.83:0/3069928893 learned_addr learned my addr 172.21.15.83:0/3069928893 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efeff7fe700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff000aedc0 0x7eff0014b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7eff04dd9700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0015e290 0x7eff000b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefffff700 1 -- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0015e290 msgr2=0x7eff000b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefffff700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0015e290 0x7eff000b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefffff700 1 -- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff000aedc0 msgr2=0x7eff0014b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.721 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefffff700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff000aedc0 0x7eff0014b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefffff700 1 -- 172.21.15.83:0/3069928893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef00094a0 con 0x7eff000ad870 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7eff04dd9700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0015e290 0x7eff000b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefffff700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff000ad870 0x7eff001519b0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7efee80011e0 tx=0x7efee800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefcff9700 1 -- 172.21.15.83:0/3069928893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8004c80 con 0x7eff000ad870 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefcff9700 1 -- 172.21.15.83:0/3069928893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee8004e00 con 0x7eff000ad870 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7efefcff9700 1 -- 172.21.15.83:0/3069928893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8007310 con 0x7eff000ad870 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7eff0683c700 1 -- 172.21.15.83:0/3069928893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff00160550 con 0x7eff000ad870 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.717+0000 7eff0683c700 1 -- 172.21.15.83:0/3069928893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff00160b10 con 0x7eff000ad870 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.718+0000 7eff0683c700 1 -- 172.21.15.83:0/3069928893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff00068d90 con 0x7eff000ad870 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.719+0000 7efefcff9700 1 -- 172.21.15.83:0/3069928893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7efee8007470 con 0x7eff000ad870 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.719+0000 7efefcff9700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efee006b1a0 0x7efee006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.719+0000 7efefcff9700 1 -- 172.21.15.83:0/3069928893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efee8089560 con 0x7eff000ad870 2023-11-26T07:17:48.722 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.719+0000 7efeff7fe700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efee006b1a0 0x7efee006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.720+0000 7efeff7fe700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efee006b1a0 0x7efee006d660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efef00090b0 tx=0x7efef0013040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.723+0000 7efefcff9700 1 -- 172.21.15.83:0/3069928893 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efee80545c0 con 0x7eff000ad870 2023-11-26T07:17:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.916+0000 7eff0683c700 1 -- 172.21.15.83:0/3069928893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7eff00068fb0 con 0x7eff000ad870 2023-11-26T07:17:48.921 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.920+0000 7efefcff9700 1 -- 172.21.15.83:0/3069928893 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7efee8057be0 con 0x7eff000ad870 2023-11-26T07:17:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 -- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efee006b1a0 msgr2=0x7efee006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.923 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efee006b1a0 0x7efee006d660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efef00090b0 tx=0x7efef0013040 comp rx=0 tx=0).stop 2023-11-26T07:17:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 -- 172.21.15.83:0/3069928893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff000ad870 msgr2=0x7eff001519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff000ad870 0x7eff001519b0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7efee80011e0 tx=0x7efee800cae0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 -- 172.21.15.83:0/3069928893 shutdown_connections 2023-11-26T07:17:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efee006b1a0 0x7efee006d660 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0015e290 0x7eff000b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff000ad870 0x7eff001519b0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 --2- 172.21.15.83:0/3069928893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff000aedc0 0x7eff0014b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 -- 172.21.15.83:0/3069928893 >> 172.21.15.83:0/3069928893 conn(0x7eff000b2050 msgr2=0x7eff00159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 -- 172.21.15.83:0/3069928893 shutdown_connections 2023-11-26T07:17:48.924 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:48.922+0000 7eff0683c700 1 -- 172.21.15.83:0/3069928893 wait complete. 2023-11-26T07:17:48.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2023-11-26T07:17:49.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.054+0000 7f7a409b0700 1 Processor -- start 2023-11-26T07:17:49.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.054+0000 7f7a409b0700 1 -- start start 2023-11-26T07:17:49.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.055+0000 7f7a409b0700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7a3c15e080 0x7f7a3c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:49.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.055+0000 7f7a409b0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3c0af4d0 0x7f7a3c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.055+0000 7f7a409b0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3c0ae430 con 0x7f7a3c06c000 2023-11-26T07:17:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.055+0000 7f7a409b0700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c0af0b0 con 0x7f7a3c15e080 2023-11-26T07:17:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.055+0000 7f7a409b0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c0b05a0 con 0x7f7a3c0af4d0 2023-11-26T07:17:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a39d9b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3c0af4d0 0x7f7a3c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a3a59c700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7a3c15e080 0x7f7a3c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a3ad9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f7a3c06c000 0x7f7a3c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:47130/0 (socket says 172.21.15.83:47130) 2023-11-26T07:17:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a3ad9d700 1 -- 172.21.15.83:0/1940365492 learned_addr learned my addr 172.21.15.83:0/1940365492 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a39d9b700 1 -- 172.21.15.83:0/1940365492 >> v2:172.21.15.203:3300/0 conn(0x7f7a3c15e080 msgr2=0x7f7a3c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a39d9b700 1 --2- 172.21.15.83:0/1940365492 >> v2:172.21.15.203:3300/0 conn(0x7f7a3c15e080 0x7f7a3c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a39d9b700 1 -- 172.21.15.83:0/1940365492 >> v1:172.21.15.203:6789/0 conn(0x7f7a3c06c000 legacy=0x7f7a3c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a39d9b700 1 -- 172.21.15.83:0/1940365492 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c0ae710 con 0x7f7a3c0af4d0 2023-11-26T07:17:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a3a59c700 1 --2- 172.21.15.83:0/1940365492 >> v2:172.21.15.203:3300/0 conn(0x7f7a3c15e080 0x7f7a3c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a39d9b700 1 --2- 172.21.15.83:0/1940365492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3c0af4d0 0x7f7a3c0afb90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009840 tx=0x7f7a2c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe9829f475781576 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a3959a700 1 -- 172.21.15.83:0/1940365492 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c011830 con 0x7f7a3c0af4d0 2023-11-26T07:17:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a3959a700 1 -- 172.21.15.83:0/1940365492 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c011990 con 0x7f7a3c0af4d0 2023-11-26T07:17:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.056+0000 7f7a3959a700 1 -- 172.21.15.83:0/1940365492 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c011d20 con 0x7f7a3c0af4d0 2023-11-26T07:17:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.057+0000 7f7a409b0700 1 -- 172.21.15.83:0/1940365492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3c0af4d0 msgr2=0x7f7a3c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.057+0000 7f7a409b0700 1 --2- 172.21.15.83:0/1940365492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3c0af4d0 0x7f7a3c0afb90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009840 tx=0x7f7a2c009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.057+0000 7f7a409b0700 1 -- 172.21.15.83:0/1940365492 shutdown_connections 2023-11-26T07:17:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.057+0000 7f7a409b0700 1 --2- 172.21.15.83:0/1940365492 >> v2:172.21.15.203:3300/0 conn(0x7f7a3c15e080 0x7f7a3c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.057+0000 7f7a409b0700 1 --2- 172.21.15.83:0/1940365492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3c0af4d0 0x7f7a3c0afb90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.057+0000 7f7a409b0700 1 -- 172.21.15.83:0/1940365492 >> 172.21.15.83:0/1940365492 conn(0x7f7a3c071b90 msgr2=0x7f7a3c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.057+0000 7f7a409b0700 1 -- 172.21.15.83:0/1940365492 shutdown_connections 2023-11-26T07:17:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.057+0000 7f7a409b0700 1 -- 172.21.15.83:0/1940365492 wait complete. 2023-11-26T07:17:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.058+0000 7f7a409b0700 1 Processor -- start 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a409b0700 1 -- start start 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a409b0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a3c06c000 0x7f7a3c1464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a409b0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a3c0af4d0 0x7f7a3c157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a409b0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3c15e080 0x7f7a3c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a409b0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c154eb0 con 0x7f7a3c06c000 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a409b0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c158860 con 0x7f7a3c0af4d0 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a409b0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c146a00 con 0x7f7a3c15e080 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a3a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a3c06c000 0x7f7a3c1464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a3a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a3c06c000 0x7f7a3c1464c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42474/0 (socket says 172.21.15.83:42474) 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a3a59c700 1 -- 172.21.15.83:0/2679143751 learned_addr learned my addr 172.21.15.83:0/2679143751 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a39d9b700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a3c0af4d0 0x7f7a3c157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.059+0000 7f7a3ad9d700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3c15e080 0x7f7a3c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.060+0000 7f7a3a59c700 1 -- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3c15e080 msgr2=0x7f7a3c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.060+0000 7f7a3a59c700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3c15e080 0x7f7a3c159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.060+0000 7f7a3a59c700 1 -- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a3c0af4d0 msgr2=0x7f7a3c157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.060+0000 7f7a3a59c700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a3c0af4d0 0x7f7a3c157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.060+0000 7f7a3a59c700 1 -- 172.21.15.83:0/2679143751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300094a0 con 0x7f7a3c06c000 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.060+0000 7f7a3a59c700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a3c06c000 0x7f7a3c1464c0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000ec10 tx=0x7f7a3000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.060+0000 7f7a2affd700 1 -- 172.21.15.83:0/2679143751 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30009760 con 0x7f7a3c06c000 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.060+0000 7f7a2affd700 1 -- 172.21.15.83:0/2679143751 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3000c850 con 0x7f7a3c06c000 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.060+0000 7f7a409b0700 1 -- 172.21.15.83:0/2679143751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2c0094a0 con 0x7f7a3c06c000 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.060+0000 7f7a2affd700 1 -- 172.21.15.83:0/2679143751 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000cbe0 con 0x7f7a3c06c000 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.060+0000 7f7a409b0700 1 -- 172.21.15.83:0/2679143751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c155090 con 0x7f7a3c06c000 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.061+0000 7f7a409b0700 1 -- 172.21.15.83:0/2679143751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3c0753f0 con 0x7f7a3c06c000 2023-11-26T07:17:49.064 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.061+0000 7f7a2affd700 1 -- 172.21.15.83:0/2679143751 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f7a3000cd40 con 0x7f7a3c06c000 2023-11-26T07:17:49.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.062+0000 7f7a2affd700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7a1c06b1d0 0x7f7a1c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:49.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.062+0000 7f7a2affd700 1 -- 172.21.15.83:0/2679143751 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7a300897d0 con 0x7f7a3c06c000 2023-11-26T07:17:49.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.062+0000 7f7a39d9b700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7a1c06b1d0 0x7f7a1c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:49.065 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.062+0000 7f7a39d9b700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7a1c06b1d0 0x7f7a1c06d690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0090b0 tx=0x7f7a2c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.066+0000 7f7a2affd700 1 -- 172.21.15.83:0/2679143751 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a300546e0 con 0x7f7a3c06c000 2023-11-26T07:17:49.339 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.337+0000 7f7a409b0700 1 -- 172.21.15.83:0/2679143751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7f7a3c067f60 con 0x7f7a3c06c000 2023-11-26T07:17:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.564+0000 7f7a2affd700 1 -- 172.21.15.83:0/2679143751 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v346) v1 ==== 149+0+0 (secure 0 0 0) 0x7f7a30057d00 con 0x7f7a3c06c000 2023-11-26T07:17:49.568 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 -- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7a1c06b1d0 msgr2=0x7f7a1c06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7a1c06b1d0 0x7f7a1c06d690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0090b0 tx=0x7f7a2c008030 comp rx=0 tx=0).stop 2023-11-26T07:17:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 -- 172.21.15.83:0/2679143751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a3c06c000 msgr2=0x7f7a3c1464c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a3c06c000 0x7f7a3c1464c0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000ec10 tx=0x7f7a3000ef20 comp rx=0 tx=0).stop 2023-11-26T07:17:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 -- 172.21.15.83:0/2679143751 shutdown_connections 2023-11-26T07:17:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f7a1c06b1d0 0x7f7a1c06d690 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a3c15e080 0x7f7a3c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a3c06c000 0x7f7a3c1464c0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 --2- 172.21.15.83:0/2679143751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a3c0af4d0 0x7f7a3c157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 -- 172.21.15.83:0/2679143751 >> 172.21.15.83:0/2679143751 conn(0x7f7a3c071b90 msgr2=0x7f7a3c0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 -- 172.21.15.83:0/2679143751 shutdown_connections 2023-11-26T07:17:49.569 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:49.567+0000 7f7a409b0700 1 -- 172.21.15.83:0/2679143751 wait complete. 2023-11-26T07:17:49.582 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... FAIL 2023-11-26T07:17:49.583 INFO:tasks.cephfs_test_runner: 2023-11-26T07:17:49.583 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-26T07:17:49.583 INFO:tasks.cephfs_test_runner:FAIL: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:17:49.583 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... 2023-11-26T07:17:49.583 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-26T07:17:49.583 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-26T07:17:49.583 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs/test_volumes.py", line 7763, in test_subvolume_no_upgrade_v1_sanity 2023-11-26T07:17:49.583 INFO:tasks.cephfs_test_runner: self.assertEqual(snap_info["has_pending_clones"], "no") 2023-11-26T07:17:49.583 INFO:tasks.cephfs_test_runner:AssertionError: 'yes' != 'no' 2023-11-26T07:17:49.583 INFO:tasks.cephfs_test_runner:- yes 2023-11-26T07:17:49.584 INFO:tasks.cephfs_test_runner:+ no 2023-11-26T07:17:49.584 INFO:tasks.cephfs_test_runner: 2023-11-26T07:17:49.584 INFO:tasks.cephfs_test_runner: 2023-11-26T07:17:49.584 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-26T07:17:49.584 INFO:tasks.cephfs_test_runner:Ran 6 tests in 352.641s 2023-11-26T07:17:49.584 INFO:tasks.cephfs_test_runner: 2023-11-26T07:17:49.584 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2023-11-26T07:17:49.584 INFO:tasks.cephfs_test_runner: 2023-11-26T07:17:49.584 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-26T07:17:49.584 INFO:tasks.cephfs_test_runner:FAIL: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:17:49.584 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... 2023-11-26T07:17:49.585 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-26T07:17:49.585 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-26T07:17:49.585 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs/test_volumes.py", line 7763, in test_subvolume_no_upgrade_v1_sanity 2023-11-26T07:17:49.585 INFO:tasks.cephfs_test_runner: self.assertEqual(snap_info["has_pending_clones"], "no") 2023-11-26T07:17:49.585 INFO:tasks.cephfs_test_runner:AssertionError: 'yes' != 'no' 2023-11-26T07:17:49.585 INFO:tasks.cephfs_test_runner:- yes 2023-11-26T07:17:49.585 INFO:tasks.cephfs_test_runner:+ no 2023-11-26T07:17:49.585 INFO:tasks.cephfs_test_runner: 2023-11-26T07:17:49.585 INFO:tasks.cephfs_test_runner: 2023-11-26T07:17:49.585 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:17:49.679 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d790d891731748d79c13c7ba8ff278d1 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:17:49.681 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-26T07:17:49.719 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-26T07:17:49.774 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-26T07:17:49.774 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:49.775 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:49.795 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:17:49.796 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2023-11-26T07:17:49.796 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:49.852 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14766 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-26T07:17:49.852 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:17:49.852 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:49.873 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:17:49.873 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-11-26T07:17:49.873 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:49.933 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284188k,nr_inodes=4071047,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2446 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-26T07:17:49.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:49.933 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:17:49.950 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:17:49.950 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:17:49.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:49.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:17:50.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:17:50.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2023-11-26T07:17:50.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.250 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2023-11-26T07:17:50.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.266 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2023-11-26T07:17:50.336 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2023-11-26T07:17:50.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:17:50.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.338 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:17:50.358 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:17:50.359 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:17:50.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.359 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:17:50.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.444 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:17:50.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.532 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2023-11-26T07:17:50.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.618 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2023-11-26T07:17:50.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.635 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2023-11-26T07:17:50.705 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2023-11-26T07:17:50.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:17:50.706 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-26T07:17:50.716 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-26T07:17:50.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:50.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.836+0000 7ff41d606700 1 Processor -- start 2023-11-26T07:17:50.837 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.836+0000 7ff41d606700 1 -- start start 2023-11-26T07:17:50.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff41d606700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff41815a3b0 0x7ff4180b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff41d606700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff41815deb0 0x7ff41815e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff41d606700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41815a790 con 0x7ff4180cf360 2023-11-26T07:17:50.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff41d606700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4180cf8b0 con 0x7ff41815a3b0 2023-11-26T07:17:50.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff41d606700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4180b4c40 con 0x7ff41815deb0 2023-11-26T07:17:50.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff416ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff41815a3b0 0x7ff4180b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.840 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff416ffd700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff41815a3b0 0x7ff4180b3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42482/0 (socket says 172.21.15.83:42482) 2023-11-26T07:17:50.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff416ffd700 1 -- 172.21.15.83:0/2073168054 learned_addr learned my addr 172.21.15.83:0/2073168054 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:50.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff4177fe700 1 --2- 172.21.15.83:0/2073168054 >> v2:172.21.15.203:3300/0 conn(0x7ff41815deb0 0x7ff41815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff416ffd700 1 -- 172.21.15.83:0/2073168054 >> v2:172.21.15.203:3300/0 conn(0x7ff41815deb0 msgr2=0x7ff41815e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:50.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff416ffd700 1 --2- 172.21.15.83:0/2073168054 >> v2:172.21.15.203:3300/0 conn(0x7ff41815deb0 0x7ff41815e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff416ffd700 1 -- 172.21.15.83:0/2073168054 >> v1:172.21.15.203:6789/0 conn(0x7ff4180cf360 legacy=0x7ff4180b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff416ffd700 1 -- 172.21.15.83:0/2073168054 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff418070c60 con 0x7ff41815a3b0 2023-11-26T07:17:50.841 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff416ffd700 1 --2- 172.21.15.83:0/2073168054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff41815a3b0 0x7ff4180b3320 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ff418074fd0 tx=0x7ff40800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a70a419e8896c59 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:50.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff415ffb700 1 -- 172.21.15.83:0/2073168054 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40800dc70 con 0x7ff41815a3b0 2023-11-26T07:17:50.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff415ffb700 1 -- 172.21.15.83:0/2073168054 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40800ddd0 con 0x7ff41815a3b0 2023-11-26T07:17:50.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.837+0000 7ff415ffb700 1 -- 172.21.15.83:0/2073168054 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408010690 con 0x7ff41815a3b0 2023-11-26T07:17:50.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.838+0000 7ff41d606700 1 -- 172.21.15.83:0/2073168054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff41815a3b0 msgr2=0x7ff4180b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.838+0000 7ff41d606700 1 --2- 172.21.15.83:0/2073168054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff41815a3b0 0x7ff4180b3320 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ff418074fd0 tx=0x7ff40800be70 comp rx=0 tx=0).stop 2023-11-26T07:17:50.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.838+0000 7ff41d606700 1 -- 172.21.15.83:0/2073168054 shutdown_connections 2023-11-26T07:17:50.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.838+0000 7ff41d606700 1 --2- 172.21.15.83:0/2073168054 >> v2:172.21.15.203:3300/0 conn(0x7ff41815deb0 0x7ff41815e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.838+0000 7ff41d606700 1 --2- 172.21.15.83:0/2073168054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff41815a3b0 0x7ff4180b3320 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.838+0000 7ff41d606700 1 -- 172.21.15.83:0/2073168054 >> 172.21.15.83:0/2073168054 conn(0x7ff4180b1740 msgr2=0x7ff4180b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:50.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.838+0000 7ff41d606700 1 -- 172.21.15.83:0/2073168054 shutdown_connections 2023-11-26T07:17:50.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.838+0000 7ff41d606700 1 -- 172.21.15.83:0/2073168054 wait complete. 2023-11-26T07:17:50.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.839+0000 7ff41d606700 1 Processor -- start 2023-11-26T07:17:50.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.839+0000 7ff41d606700 1 -- start start 2023-11-26T07:17:50.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.840+0000 7ff41d606700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4180cf360 0x7ff41814b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.840+0000 7ff41d606700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41815a3b0 0x7ff418147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.840+0000 7ff41d606700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff41815deb0 0x7ff41813c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.840+0000 7ff41d606700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4181495b0 con 0x7ff4180cf360 2023-11-26T07:17:50.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.840+0000 7ff41d606700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff418148270 con 0x7ff41815a3b0 2023-11-26T07:17:50.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.840+0000 7ff41d606700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff418148510 con 0x7ff41815deb0 2023-11-26T07:17:50.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.840+0000 7ff416ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4180cf360 0x7ff41814b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.840+0000 7ff416ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4180cf360 0x7ff41814b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42488/0 (socket says 172.21.15.83:42488) 2023-11-26T07:17:50.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.840+0000 7ff416ffd700 1 -- 172.21.15.83:0/3487812890 learned_addr learned my addr 172.21.15.83:0/3487812890 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:50.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.840+0000 7ff4177fe700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff41815deb0 0x7ff41813c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.840+0000 7ff4167fc700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41815a3b0 0x7ff418147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff416ffd700 1 -- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff41815deb0 msgr2=0x7ff41813c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff416ffd700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff41815deb0 0x7ff41813c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff416ffd700 1 -- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41815a3b0 msgr2=0x7ff418147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff416ffd700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41815a3b0 0x7ff418147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff416ffd700 1 -- 172.21.15.83:0/3487812890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40800b7f0 con 0x7ff4180cf360 2023-11-26T07:17:50.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff4177fe700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff41815deb0 0x7ff41813c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:50.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff4167fc700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41815a3b0 0x7ff418147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:50.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff416ffd700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4180cf360 0x7ff41814b370 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ff40800beb0 tx=0x7ff408004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:50.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff3ff7fe700 1 -- 172.21.15.83:0/3487812890 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40801f070 con 0x7ff4180cf360 2023-11-26T07:17:50.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff3ff7fe700 1 -- 172.21.15.83:0/3487812890 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4080039f0 con 0x7ff4180cf360 2023-11-26T07:17:50.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff41d606700 1 -- 172.21.15.83:0/3487812890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4181487f0 con 0x7ff4180cf360 2023-11-26T07:17:50.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff3ff7fe700 1 -- 172.21.15.83:0/3487812890 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408003d80 con 0x7ff4180cf360 2023-11-26T07:17:50.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.841+0000 7ff41d606700 1 -- 172.21.15.83:0/3487812890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff418148db0 con 0x7ff4180cf360 2023-11-26T07:17:50.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.842+0000 7ff41d606700 1 -- 172.21.15.83:0/3487812890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4180751e0 con 0x7ff4180cf360 2023-11-26T07:17:50.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.842+0000 7ff3ff7fe700 1 -- 172.21.15.83:0/3487812890 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7ff408019400 con 0x7ff4180cf360 2023-11-26T07:17:50.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.843+0000 7ff3ff7fe700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff40406b1a0 0x7ff40406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:50.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.843+0000 7ff3ff7fe700 1 -- 172.21.15.83:0/3487812890 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff40808cbe0 con 0x7ff4180cf360 2023-11-26T07:17:50.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.843+0000 7ff4167fc700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff40406b1a0 0x7ff40406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:50.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.843+0000 7ff4167fc700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff40406b1a0 0x7ff40406d660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff418147c50 tx=0x7ff40c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:50.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:50.847+0000 7ff3ff7fe700 1 -- 172.21.15.83:0/3487812890 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff408057b40 con 0x7ff4180cf360 2023-11-26T07:17:51.036 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.035+0000 7ff41d606700 1 -- 172.21.15.83:0/3487812890 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff418071570 con 0x7ff40406b1a0 2023-11-26T07:17:51.039 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.038+0000 7ff3ff7fe700 1 -- 172.21.15.83:0/3487812890 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50722 (secure 0 0 0) 0x7ff40c0050d0 con 0x7ff40406b1a0 2023-11-26T07:17:51.040 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:51.041 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 -- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff40406b1a0 msgr2=0x7ff40406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff40406b1a0 0x7ff40406d660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff418147c50 tx=0x7ff40c009040 comp rx=0 tx=0).stop 2023-11-26T07:17:51.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 -- 172.21.15.83:0/3487812890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4180cf360 msgr2=0x7ff41814b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4180cf360 0x7ff41814b370 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ff40800beb0 tx=0x7ff408004750 comp rx=0 tx=0).stop 2023-11-26T07:17:51.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 -- 172.21.15.83:0/3487812890 shutdown_connections 2023-11-26T07:17:51.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff40406b1a0 0x7ff40406d660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff41815deb0 0x7ff41813c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.042 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4180cf360 0x7ff41814b370 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 --2- 172.21.15.83:0/3487812890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff41815a3b0 0x7ff418147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 -- 172.21.15.83:0/3487812890 >> 172.21.15.83:0/3487812890 conn(0x7ff4180b1740 msgr2=0x7ff4180ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 -- 172.21.15.83:0/3487812890 shutdown_connections 2023-11-26T07:17:51.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.040+0000 7ff41d606700 1 -- 172.21.15.83:0/3487812890 wait complete. 2023-11-26T07:17:51.043 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:17:51.053 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-11-26T07:17:50.332195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":285976,"kb_used_data":44872,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754688744,"statfs":{"total":773094113280,"available":772801273856,"internally_reserved":0,"allocated":45948928,"data_stored":40119192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17229834,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-80,"num_read_kb":-8238,"num_write":-177,"num_write_kb":-16903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001293"},"pg_stats":[{"pgid":"1.0","version":"71'128","reported_seq":289,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:21.293408+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:17:21.293408+0000","last_peered":"2023-11-26T07:17:21.293408+0000","last_clean":"2023-11-26T07:17:21.293408+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:17:21.293408+0000","last_undegraded":"2023-11-26T07:17:21.293408+0000","last_fullsized":"2023-11-26T07:17:21.293408+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39176,"kb_used_data":7996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332664,"statfs":{"total":96636764160,"available":96596647936,"internally_reserved":0,"allocated":8187904,"data_stored":7445543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":0.84099999999999997,"15min":0.86099999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42499999999999999,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.78600000000000003,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.752}]},{"osd":1,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.36299999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69499999999999995,"5min":0.88100000000000001,"15min":1.048},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59799999999999998,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.315},"max":{"1min":0.753,"5min":0.81899999999999995,"15min":0.89600000000000002},"last":0.78000000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79800000000000004,"5min":0.877,"15min":0.92000000000000004},"last":0.67700000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.33600000000000002},"max":{"1min":0.77100000000000002,"5min":0.91300000000000003,"15min":1.0629999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.73999999999999999,"5min":0.91400000000000003,"15min":1.0960000000000001},"last":0.76500000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0269999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69099999999999995,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.627}]},{"osd":5,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72499999999999998,"5min":0.996,"15min":0.996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81100000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.48299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80000000000000004,"5min":1.006,"15min":1.361},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.751,"5min":1.0409999999999999,"15min":1.1899999999999999},"last":0.85299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":4316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":4419584,"data_stored":3702167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54300000000000004},"min":{"1min":0.34599999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.018,"5min":1.159,"15min":1.159},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75800000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77100000000000002,"5min":0.79900000000000004,"15min":0.88900000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.54100000000000004,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":1.0209999999999999},"last":0.58199999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38300000000000001,"5min":0.376,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":0.79200000000000004,"15min":0.96199999999999997},"last":0.441},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.85099999999999998},"last":0.52500000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.55800000000000005},"min":{"1min":0.46800000000000003,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":0.76200000000000001,"5min":4.8689999999999998,"15min":4.8689999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74099999999999999,"5min":0.82599999999999996,"15min":1.1950000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.379}]},{"osd":5,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.36699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.67400000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.49299999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.432},{"interface":"front","average":{"1min":0.63,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.501,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71899999999999997,"5min":0.81999999999999995,"15min":0.83099999999999996},"last":0.45800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36524,"kb_used_data":5348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335316,"statfs":{"total":96636764160,"available":96599363584,"internally_reserved":0,"allocated":5476352,"data_stored":4751403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.497,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.34899999999999998},"min":{"1min":0.26500000000000001,"5min":0.20000000000000001,"15min":0.19700000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.35399999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.33900000000000002},"min":{"1min":0.247,"5min":0.22500000000000001,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28100000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.29999999999999999,"5min":0.252,"15min":0.246},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.378},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.26700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47999999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.32500000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":1.167,"15min":1.167},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.80700000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.5}]},{"osd":5,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.879,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.48499999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.48499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83199999999999996,"5min":0.91700000000000004,"15min":1.129},"last":0.437},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76600000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.46400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.79200000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.46300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40640,"kb_used_data":9460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331200,"statfs":{"total":96636764160,"available":96595148800,"internally_reserved":0,"allocated":9687040,"data_stored":8945279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.217,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.51900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.215},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.36899999999999999,"15min":0.36699999999999999},"min":{"1min":0.25600000000000001,"5min":0.23999999999999999,"15min":0.23300000000000001},"max":{"1min":0.50900000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39000000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":0.64300000000000002,"15min":0.65700000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.37,"15min":0.35199999999999998},"min":{"1min":0.246,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":0.48299999999999998,"5min":1.202,"15min":1.202},"last":0.58399999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.39900000000000002},"min":{"1min":0.27900000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52200000000000002,"5min":0.75,"15min":0.75},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41899999999999998,"15min":0.433},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.53600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53500000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":1.419,"15min":1.419},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64000000000000001,"15min":0.66300000000000003},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":1,"15min":1.0009999999999999},"last":0.38600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59699999999999998,"15min":0.627},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81499999999999995,"5min":1.046,"15min":1.046},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.629},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.746,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81699999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81699999999999995,"5min":1.387,"15min":1.387},"last":0.63}]},{"osd":7,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.626},"min":{"1min":0.41599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76700000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.627,"15min":0.65900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":1.087,"15min":1.087},"last":0.41699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35496,"kb_used_data":4320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336344,"statfs":{"total":96636764160,"available":96600416256,"internally_reserved":0,"allocated":4423680,"data_stored":3702901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.36499999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.59199999999999997,"5min":0.627,"15min":0.627},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.248,"5min":0.186,"15min":0.186},"max":{"1min":0.51500000000000001,"5min":0.70099999999999996,"15min":0.84599999999999997},"last":0.35699999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38700000000000001},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52000000000000002,"5min":0.58799999999999997,"15min":0.65700000000000003},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38500000000000001,"15min":0.37},"min":{"1min":0.252,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.308}]},{"osd":3,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36299999999999999,"15min":0.36899999999999999},"min":{"1min":0.254,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.58799999999999997,"15min":0.68700000000000006},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.42199999999999999},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.76100000000000001},"last":0.29499999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.65600000000000003},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81799999999999995,"5min":0.85799999999999998,"15min":2.2160000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80600000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.61599999999999999},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82899999999999996,"5min":0.90000000000000002,"15min":1.075},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.0309999999999999,"15min":1.2190000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90800000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89400000000000002,"5min":1.004,"15min":1.03},"last":0.55000000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92600000000000005,"5min":0.94899999999999995,"15min":1.1919999999999999},"last":0.63},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.62},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.40999999999999998},"max":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":1.1120000000000001},"last":0.57699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36968,"kb_used_data":5792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334872,"statfs":{"total":96636764160,"available":96598908928,"internally_reserved":0,"allocated":5931008,"data_stored":5203060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.35299999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.188},"max":{"1min":0.55700000000000005,"5min":0.629,"15min":1.284},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.371,"15min":0.45400000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57799999999999996,"5min":0.63600000000000001,"15min":2.8460000000000001},"last":0.32400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.34100000000000003,"15min":0.48299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53500000000000003,"5min":0.61799999999999999,"15min":4.9109999999999996},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.39100000000000001,"15min":0.47299999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.65800000000000003,"5min":0.69399999999999995,"15min":2.8940000000000001},"last":0.40200000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41999999999999998,"15min":0.48599999999999999},"min":{"1min":0.27900000000000003,"5min":0.224,"15min":0.224},"max":{"1min":0.626,"5min":0.626,"15min":2.9790000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38300000000000001,"15min":0.41699999999999998},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.23100000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":1.3440000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.66000000000000003},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":2.9500000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60899999999999999,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.405},"last":0.621}]},{"osd":5,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":0.629},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.377},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":4.8129999999999997},"last":0.72199999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61699999999999999,"15min":0.66000000000000003},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.4319999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.75700000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":4.883},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.77200000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":4.8499999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":2.923},"last":0.64800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35512,"kb_used_data":4332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336328,"statfs":{"total":96636764160,"available":96600399872,"internally_reserved":0,"allocated":4435968,"data_stored":3708633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.65600000000000003,"5min":0.79400000000000004,"15min":2.9510000000000001},"last":0.498},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54600000000000004,"15min":0.53100000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70899999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57699999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55900000000000005,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68700000000000006,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.64200000000000002},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.91100000000000003,"15min":2.9820000000000002},"last":0.32400000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77400000000000002,"5min":2.113,"15min":2.113},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.65700000000000003},"min":{"1min":0.40400000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":3.0030000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.54700000000000004},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.33300000000000002},"max":{"1min":0.67600000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.58499999999999996},"min":{"1min":0.41899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85199999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80900000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.35899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78100000000000003,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.41199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.84799999999999998},"last":0.53300000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82999999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.55300000000000005,"5min":0.315,"15min":0.315},"max":{"1min":0.91400000000000003,"5min":1.01,"15min":1.01},"last":0.63100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30328,"kb_used_data":3308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341512,"statfs":{"total":96636764160,"available":96605708288,"internally_reserved":0,"allocated":3387392,"data_stored":2660206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.29799999999999999,"15min":0.27500000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96199999999999997,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.36399999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79400000000000004,"5min":1.091,"15min":1.091},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.44}]},{"osd":2,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.42399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.54200000000000004},"min":{"1min":0.44400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53700000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.503},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61499999999999999,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52300000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.35599999999999998,"15min":0.311},"max":{"1min":1.044,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0980000000000001,"5min":1.161,"15min":1.161},"last":0.59499999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.35199999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0129999999999999,"5min":1.196,"15min":1.196},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.628,"15min":0.64500000000000002},"min":{"1min":0.40699999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0309999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:51.055 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:51.174 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.173+0000 7fadfe443700 1 Processor -- start 2023-11-26T07:17:51.175 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.173+0000 7fadfe443700 1 -- start start 2023-11-26T07:17:51.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.174+0000 7fadfe443700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fadf806c710 0x7fadf8165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.174+0000 7fadfe443700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf80af950 con 0x7fadf80adf00 2023-11-26T07:17:51.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.174+0000 7fadfe443700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf80afc30 con 0x7fadf81446a0 2023-11-26T07:17:51.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.174+0000 7fadfe443700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf8144a80 con 0x7fadf806c710 2023-11-26T07:17:51.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.174+0000 7fadfc9e0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fadf806c710 0x7fadf8165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.174+0000 7fadf77fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fadf80adf00 0x7fadf806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34306/0 (socket says 172.21.15.83:34306) 2023-11-26T07:17:51.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.174+0000 7fadf77fe700 1 -- 172.21.15.83:0/906079533 learned_addr learned my addr 172.21.15.83:0/906079533 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:51.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.174+0000 7fadfc9e0700 1 -- 172.21.15.83:0/906079533 >> v1:172.21.15.203:6789/0 conn(0x7fadf81446a0 legacy=0x7fadf80b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:51.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.174+0000 7fadfc9e0700 1 -- 172.21.15.83:0/906079533 >> v1:172.21.15.83:6789/0 conn(0x7fadf80adf00 legacy=0x7fadf806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.174+0000 7fadfc9e0700 1 -- 172.21.15.83:0/906079533 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80aff10 con 0x7fadf806c710 2023-11-26T07:17:51.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.174+0000 7fadf6ffd700 1 -- 172.21.15.83:0/906079533 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072594378 0 0) 0x7fadf80af950 con 0x7fadf80adf00 2023-11-26T07:17:51.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.175+0000 7fadfc9e0700 1 --2- 172.21.15.83:0/906079533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf806c710 0x7fadf8165b70 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009d90 tx=0x7fadf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc33d32a6c0c9806 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.175+0000 7fadf6ffd700 1 -- 172.21.15.83:0/906079533 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf000e070 con 0x7fadf806c710 2023-11-26T07:17:51.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.175+0000 7fadf6ffd700 1 -- 172.21.15.83:0/906079533 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf000bb90 con 0x7fadf806c710 2023-11-26T07:17:51.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.175+0000 7fadf6ffd700 1 -- 172.21.15.83:0/906079533 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0003a20 con 0x7fadf806c710 2023-11-26T07:17:51.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.175+0000 7fadfe443700 1 -- 172.21.15.83:0/906079533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf806c710 msgr2=0x7fadf8165b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.175+0000 7fadfe443700 1 --2- 172.21.15.83:0/906079533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf806c710 0x7fadf8165b70 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009d90 tx=0x7fadf000b330 comp rx=0 tx=0).stop 2023-11-26T07:17:51.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.175+0000 7fadfe443700 1 -- 172.21.15.83:0/906079533 shutdown_connections 2023-11-26T07:17:51.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.175+0000 7fadfe443700 1 --2- 172.21.15.83:0/906079533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf806c710 0x7fadf8165b70 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.175+0000 7fadfe443700 1 -- 172.21.15.83:0/906079533 >> 172.21.15.83:0/906079533 conn(0x7fadf8071b90 msgr2=0x7fadf8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.175+0000 7fadfe443700 1 -- 172.21.15.83:0/906079533 shutdown_connections 2023-11-26T07:17:51.179 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.175+0000 7fadfe443700 1 -- 172.21.15.83:0/906079533 wait complete. 2023-11-26T07:17:51.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.177+0000 7fadfe443700 1 Processor -- start 2023-11-26T07:17:51.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.177+0000 7fadfe443700 1 -- start start 2023-11-26T07:17:51.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.177+0000 7fadfe443700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf806c710 0x7fadf8138c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.177+0000 7fadfe443700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadf80adf00 0x7fadf80ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.177+0000 7fadfe443700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81446a0 0x7fadf81556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.177+0000 7fadfe443700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf8155c10 con 0x7fadf81446a0 2023-11-26T07:17:51.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.177+0000 7fadfe443700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf8141c90 con 0x7fadf80adf00 2023-11-26T07:17:51.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.177+0000 7fadfe443700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf8142810 con 0x7fadf806c710 2023-11-26T07:17:51.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadfc9e0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81446a0 0x7fadf81556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadfc9e0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81446a0 0x7fadf81556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42506/0 (socket says 172.21.15.83:42506) 2023-11-26T07:17:51.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadfc9e0700 1 -- 172.21.15.83:0/3578442937 learned_addr learned my addr 172.21.15.83:0/3578442937 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadf77fe700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadf80adf00 0x7fadf80ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadf7fff700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf806c710 0x7fadf8138c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadfc9e0700 1 -- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf806c710 msgr2=0x7fadf8138c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadfc9e0700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf806c710 0x7fadf8138c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadfc9e0700 1 -- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadf80adf00 msgr2=0x7fadf80ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadfc9e0700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadf80adf00 0x7fadf80ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadfc9e0700 1 -- 172.21.15.83:0/3578442937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf0009a70 con 0x7fadf81446a0 2023-11-26T07:17:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadfc9e0700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81446a0 0x7fadf81556d0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009470 tx=0x7fadf0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.184 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadf4ff9700 1 -- 172.21.15.83:0/3578442937 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0012630 con 0x7fadf81446a0 2023-11-26T07:17:51.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadf4ff9700 1 -- 172.21.15.83:0/3578442937 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0012790 con 0x7fadf81446a0 2023-11-26T07:17:51.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadfe443700 1 -- 172.21.15.83:0/3578442937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf8141ee0 con 0x7fadf81446a0 2023-11-26T07:17:51.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadf4ff9700 1 -- 172.21.15.83:0/3578442937 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0012b20 con 0x7fadf81446a0 2023-11-26T07:17:51.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.178+0000 7fadfe443700 1 -- 172.21.15.83:0/3578442937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf81424a0 con 0x7fadf81446a0 2023-11-26T07:17:51.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.179+0000 7fadfe443700 1 -- 172.21.15.83:0/3578442937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf8142660 con 0x7fadf81446a0 2023-11-26T07:17:51.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.180+0000 7fadf4ff9700 1 -- 172.21.15.83:0/3578442937 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fadf0012c80 con 0x7fadf81446a0 2023-11-26T07:17:51.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.180+0000 7fadf4ff9700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fade406b1a0 0x7fade406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.185 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.180+0000 7fadf4ff9700 1 -- 172.21.15.83:0/3578442937 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fadf008bd90 con 0x7fadf81446a0 2023-11-26T07:17:51.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.180+0000 7fadf7fff700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fade406b1a0 0x7fade406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.181+0000 7fadf7fff700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fade406b1a0 0x7fade406d660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fadec009960 tx=0x7fadec0090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.185+0000 7fadf4ff9700 1 -- 172.21.15.83:0/3578442937 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadf008f030 con 0x7fadf81446a0 2023-11-26T07:17:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.369+0000 7fadfe443700 1 -- 172.21.15.83:0/3578442937 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fadf80b04d0 con 0x7fade406b1a0 2023-11-26T07:17:51.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.372+0000 7fadf4ff9700 1 -- 172.21.15.83:0/3578442937 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50722 (secure 0 0 0) 0x7fadec009e30 con 0x7fade406b1a0 2023-11-26T07:17:51.374 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:51.375 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 -- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fade406b1a0 msgr2=0x7fade406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fade406b1a0 0x7fade406d660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fadec009960 tx=0x7fadec0090e0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 -- 172.21.15.83:0/3578442937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81446a0 msgr2=0x7fadf81556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81446a0 0x7fadf81556d0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009470 tx=0x7fadf0004ad0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 -- 172.21.15.83:0/3578442937 shutdown_connections 2023-11-26T07:17:51.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fade406b1a0 0x7fade406d660 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81446a0 0x7fadf81556d0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf806c710 0x7fadf8138c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 --2- 172.21.15.83:0/3578442937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadf80adf00 0x7fadf80ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 -- 172.21.15.83:0/3578442937 >> 172.21.15.83:0/3578442937 conn(0x7fadf8071b90 msgr2=0x7fadf8073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 -- 172.21.15.83:0/3578442937 shutdown_connections 2023-11-26T07:17:51.377 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.374+0000 7fadfe443700 1 -- 172.21.15.83:0/3578442937 wait complete. 2023-11-26T07:17:51.377 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:17:51.387 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-11-26T07:17:50.332195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":285976,"kb_used_data":44872,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754688744,"statfs":{"total":773094113280,"available":772801273856,"internally_reserved":0,"allocated":45948928,"data_stored":40119192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17229834,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-80,"num_read_kb":-8238,"num_write":-177,"num_write_kb":-16903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001293"},"pg_stats":[{"pgid":"1.0","version":"71'128","reported_seq":289,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:21.293408+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:17:21.293408+0000","last_peered":"2023-11-26T07:17:21.293408+0000","last_clean":"2023-11-26T07:17:21.293408+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:17:21.293408+0000","last_undegraded":"2023-11-26T07:17:21.293408+0000","last_fullsized":"2023-11-26T07:17:21.293408+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39176,"kb_used_data":7996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332664,"statfs":{"total":96636764160,"available":96596647936,"internally_reserved":0,"allocated":8187904,"data_stored":7445543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":0.84099999999999997,"15min":0.86099999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42499999999999999,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.78600000000000003,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.752}]},{"osd":1,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.36299999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69499999999999995,"5min":0.88100000000000001,"15min":1.048},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59799999999999998,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.315},"max":{"1min":0.753,"5min":0.81899999999999995,"15min":0.89600000000000002},"last":0.78000000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79800000000000004,"5min":0.877,"15min":0.92000000000000004},"last":0.67700000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.33600000000000002},"max":{"1min":0.77100000000000002,"5min":0.91300000000000003,"15min":1.0629999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.73999999999999999,"5min":0.91400000000000003,"15min":1.0960000000000001},"last":0.76500000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0269999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69099999999999995,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.627}]},{"osd":5,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72499999999999998,"5min":0.996,"15min":0.996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81100000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.48299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80000000000000004,"5min":1.006,"15min":1.361},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.751,"5min":1.0409999999999999,"15min":1.1899999999999999},"last":0.85299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":4316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":4419584,"data_stored":3702167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54300000000000004},"min":{"1min":0.34599999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.018,"5min":1.159,"15min":1.159},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75800000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77100000000000002,"5min":0.79900000000000004,"15min":0.88900000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.54100000000000004,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":1.0209999999999999},"last":0.58199999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38300000000000001,"5min":0.376,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":0.79200000000000004,"15min":0.96199999999999997},"last":0.441},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.85099999999999998},"last":0.52500000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.55800000000000005},"min":{"1min":0.46800000000000003,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":0.76200000000000001,"5min":4.8689999999999998,"15min":4.8689999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74099999999999999,"5min":0.82599999999999996,"15min":1.1950000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.379}]},{"osd":5,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.36699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.67400000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.49299999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.432},{"interface":"front","average":{"1min":0.63,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.501,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71899999999999997,"5min":0.81999999999999995,"15min":0.83099999999999996},"last":0.45800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36524,"kb_used_data":5348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335316,"statfs":{"total":96636764160,"available":96599363584,"internally_reserved":0,"allocated":5476352,"data_stored":4751403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.497,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.34899999999999998},"min":{"1min":0.26500000000000001,"5min":0.20000000000000001,"15min":0.19700000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.35399999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.33900000000000002},"min":{"1min":0.247,"5min":0.22500000000000001,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28100000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.29999999999999999,"5min":0.252,"15min":0.246},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.378},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.26700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47999999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.32500000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":1.167,"15min":1.167},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.80700000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.5}]},{"osd":5,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.879,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.48499999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.48499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83199999999999996,"5min":0.91700000000000004,"15min":1.129},"last":0.437},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76600000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.46400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.79200000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.46300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40640,"kb_used_data":9460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331200,"statfs":{"total":96636764160,"available":96595148800,"internally_reserved":0,"allocated":9687040,"data_stored":8945279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.217,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.51900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.215},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.36899999999999999,"15min":0.36699999999999999},"min":{"1min":0.25600000000000001,"5min":0.23999999999999999,"15min":0.23300000000000001},"max":{"1min":0.50900000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39000000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":0.64300000000000002,"15min":0.65700000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.37,"15min":0.35199999999999998},"min":{"1min":0.246,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":0.48299999999999998,"5min":1.202,"15min":1.202},"last":0.58399999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.39900000000000002},"min":{"1min":0.27900000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52200000000000002,"5min":0.75,"15min":0.75},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41899999999999998,"15min":0.433},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.53600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53500000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":1.419,"15min":1.419},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64000000000000001,"15min":0.66300000000000003},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":1,"15min":1.0009999999999999},"last":0.38600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59699999999999998,"15min":0.627},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81499999999999995,"5min":1.046,"15min":1.046},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.629},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.746,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81699999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81699999999999995,"5min":1.387,"15min":1.387},"last":0.63}]},{"osd":7,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.626},"min":{"1min":0.41599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76700000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.627,"15min":0.65900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":1.087,"15min":1.087},"last":0.41699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35496,"kb_used_data":4320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336344,"statfs":{"total":96636764160,"available":96600416256,"internally_reserved":0,"allocated":4423680,"data_stored":3702901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.36499999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.59199999999999997,"5min":0.627,"15min":0.627},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.248,"5min":0.186,"15min":0.186},"max":{"1min":0.51500000000000001,"5min":0.70099999999999996,"15min":0.84599999999999997},"last":0.35699999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38700000000000001},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52000000000000002,"5min":0.58799999999999997,"15min":0.65700000000000003},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38500000000000001,"15min":0.37},"min":{"1min":0.252,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.308}]},{"osd":3,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36299999999999999,"15min":0.36899999999999999},"min":{"1min":0.254,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.58799999999999997,"15min":0.68700000000000006},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.42199999999999999},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.76100000000000001},"last":0.29499999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.65600000000000003},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81799999999999995,"5min":0.85799999999999998,"15min":2.2160000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80600000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.61599999999999999},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82899999999999996,"5min":0.90000000000000002,"15min":1.075},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.0309999999999999,"15min":1.2190000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90800000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89400000000000002,"5min":1.004,"15min":1.03},"last":0.55000000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92600000000000005,"5min":0.94899999999999995,"15min":1.1919999999999999},"last":0.63},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.62},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.40999999999999998},"max":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":1.1120000000000001},"last":0.57699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36968,"kb_used_data":5792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334872,"statfs":{"total":96636764160,"available":96598908928,"internally_reserved":0,"allocated":5931008,"data_stored":5203060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.35299999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.188},"max":{"1min":0.55700000000000005,"5min":0.629,"15min":1.284},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.371,"15min":0.45400000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57799999999999996,"5min":0.63600000000000001,"15min":2.8460000000000001},"last":0.32400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.34100000000000003,"15min":0.48299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53500000000000003,"5min":0.61799999999999999,"15min":4.9109999999999996},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.39100000000000001,"15min":0.47299999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.65800000000000003,"5min":0.69399999999999995,"15min":2.8940000000000001},"last":0.40200000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41999999999999998,"15min":0.48599999999999999},"min":{"1min":0.27900000000000003,"5min":0.224,"15min":0.224},"max":{"1min":0.626,"5min":0.626,"15min":2.9790000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38300000000000001,"15min":0.41699999999999998},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.23100000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":1.3440000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.66000000000000003},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":2.9500000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60899999999999999,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.405},"last":0.621}]},{"osd":5,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":0.629},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.377},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":4.8129999999999997},"last":0.72199999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61699999999999999,"15min":0.66000000000000003},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.4319999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.75700000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":4.883},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.77200000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":4.8499999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":2.923},"last":0.64800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35512,"kb_used_data":4332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336328,"statfs":{"total":96636764160,"available":96600399872,"internally_reserved":0,"allocated":4435968,"data_stored":3708633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.65600000000000003,"5min":0.79400000000000004,"15min":2.9510000000000001},"last":0.498},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54600000000000004,"15min":0.53100000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70899999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57699999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55900000000000005,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68700000000000006,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.64200000000000002},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.91100000000000003,"15min":2.9820000000000002},"last":0.32400000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77400000000000002,"5min":2.113,"15min":2.113},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.65700000000000003},"min":{"1min":0.40400000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":3.0030000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.54700000000000004},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.33300000000000002},"max":{"1min":0.67600000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.58499999999999996},"min":{"1min":0.41899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85199999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80900000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.35899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78100000000000003,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.41199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.84799999999999998},"last":0.53300000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82999999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.55300000000000005,"5min":0.315,"15min":0.315},"max":{"1min":0.91400000000000003,"5min":1.01,"15min":1.01},"last":0.63100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30328,"kb_used_data":3308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341512,"statfs":{"total":96636764160,"available":96605708288,"internally_reserved":0,"allocated":3387392,"data_stored":2660206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.29799999999999999,"15min":0.27500000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96199999999999997,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.36399999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79400000000000004,"5min":1.091,"15min":1.091},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.44}]},{"osd":2,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.42399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.54200000000000004},"min":{"1min":0.44400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53700000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.503},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61499999999999999,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52300000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.35599999999999998,"15min":0.311},"max":{"1min":1.044,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0980000000000001,"5min":1.161,"15min":1.161},"last":0.59499999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.35199999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0129999999999999,"5min":1.196,"15min":1.196},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.628,"15min":0.64500000000000002},"min":{"1min":0.40699999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0309999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:51.389 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-26T07:17:51.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.509+0000 7fa529a13700 1 Processor -- start 2023-11-26T07:17:51.510 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.509+0000 7fa529a13700 1 -- start start 2023-11-26T07:17:51.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.509+0000 7fa529a13700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa52406ed40 0x7fa5240afa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.509+0000 7fa529a13700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa524073f90 0x7fa5240743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.509+0000 7fa529a13700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5240748e0 con 0x7fa52415deb0 2023-11-26T07:17:51.512 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.509+0000 7fa529a13700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5240affa0 con 0x7fa52406ed40 2023-11-26T07:17:51.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.509+0000 7fa529a13700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52406dca0 con 0x7fa524073f90 2023-11-26T07:17:51.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.509+0000 7fa5227fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa52406ed40 0x7fa5240afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa522ffd700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa52415deb0 0x7fa5240b5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34318/0 (socket says 172.21.15.83:34318) 2023-11-26T07:17:51.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa522ffd700 1 -- 172.21.15.83:0/3960461132 learned_addr learned my addr 172.21.15.83:0/3960461132 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:51.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa5237fe700 1 --2- 172.21.15.83:0/3960461132 >> v2:172.21.15.203:3300/0 conn(0x7fa524073f90 0x7fa5240743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa5227fc700 1 -- 172.21.15.83:0/3960461132 >> v2:172.21.15.203:3300/0 conn(0x7fa524073f90 msgr2=0x7fa5240743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:51.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa5227fc700 1 --2- 172.21.15.83:0/3960461132 >> v2:172.21.15.203:3300/0 conn(0x7fa524073f90 0x7fa5240743a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.513 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa5227fc700 1 -- 172.21.15.83:0/3960461132 >> v1:172.21.15.83:6789/0 conn(0x7fa52415deb0 legacy=0x7fa5240b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa5227fc700 1 -- 172.21.15.83:0/3960461132 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52406df80 con 0x7fa52406ed40 2023-11-26T07:17:51.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa5237fe700 1 --2- 172.21.15.83:0/3960461132 >> v2:172.21.15.203:3300/0 conn(0x7fa524073f90 0x7fa5240743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:51.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa5227fc700 1 --2- 172.21.15.83:0/3960461132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa52406ed40 0x7fa5240afa60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa5240b5190 tx=0x7fa514009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ff006c26f12c8e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa521ffb700 1 -- 172.21.15.83:0/3960461132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514011830 con 0x7fa52406ed40 2023-11-26T07:17:51.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa521ffb700 1 -- 172.21.15.83:0/3960461132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa514011990 con 0x7fa52406ed40 2023-11-26T07:17:51.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.510+0000 7fa521ffb700 1 -- 172.21.15.83:0/3960461132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514011d20 con 0x7fa52406ed40 2023-11-26T07:17:51.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.511+0000 7fa529a13700 1 -- 172.21.15.83:0/3960461132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa52406ed40 msgr2=0x7fa5240afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.514 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.511+0000 7fa529a13700 1 --2- 172.21.15.83:0/3960461132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa52406ed40 0x7fa5240afa60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa5240b5190 tx=0x7fa514009b20 comp rx=0 tx=0).stop 2023-11-26T07:17:51.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.511+0000 7fa529a13700 1 -- 172.21.15.83:0/3960461132 shutdown_connections 2023-11-26T07:17:51.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.511+0000 7fa529a13700 1 --2- 172.21.15.83:0/3960461132 >> v2:172.21.15.203:3300/0 conn(0x7fa524073f90 0x7fa5240743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.511+0000 7fa529a13700 1 --2- 172.21.15.83:0/3960461132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa52406ed40 0x7fa5240afa60 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.511+0000 7fa529a13700 1 -- 172.21.15.83:0/3960461132 >> 172.21.15.83:0/3960461132 conn(0x7fa5240d0ba0 msgr2=0x7fa5240d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.511+0000 7fa529a13700 1 -- 172.21.15.83:0/3960461132 shutdown_connections 2023-11-26T07:17:51.515 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.511+0000 7fa529a13700 1 -- 172.21.15.83:0/3960461132 wait complete. 2023-11-26T07:17:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.512+0000 7fa529a13700 1 Processor -- start 2023-11-26T07:17:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.512+0000 7fa529a13700 1 -- start start 2023-11-26T07:17:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa529a13700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa52406ed40 0x7fa5241486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa529a13700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524073f90 0x7fa524160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa529a13700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa52415deb0 0x7fa524160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa529a13700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52415e880 con 0x7fa524073f90 2023-11-26T07:17:51.518 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa529a13700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa524148c10 con 0x7fa52406ed40 2023-11-26T07:17:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa529a13700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa524154990 con 0x7fa52415deb0 2023-11-26T07:17:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa5227fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524073f90 0x7fa524160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa5227fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524073f90 0x7fa524160ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42518/0 (socket says 172.21.15.83:42518) 2023-11-26T07:17:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa5227fc700 1 -- 172.21.15.83:0/3427557572 learned_addr learned my addr 172.21.15.83:0/3427557572 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa522ffd700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa52406ed40 0x7fa5241486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa5227fc700 1 -- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa52415deb0 msgr2=0x7fa524160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa5227fc700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa52415deb0 0x7fa524160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa5227fc700 1 -- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa52406ed40 msgr2=0x7fa5241486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.519 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa5227fc700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa52406ed40 0x7fa5241486d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa5227fc700 1 -- 172.21.15.83:0/3427557572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5140094a0 con 0x7fa524073f90 2023-11-26T07:17:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa5227fc700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524073f90 0x7fa524160ab0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fa514009b60 tx=0x7fa514004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa50bfff700 1 -- 172.21.15.83:0/3427557572 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5140036a0 con 0x7fa524073f90 2023-11-26T07:17:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa50bfff700 1 -- 172.21.15.83:0/3427557572 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51401e070 con 0x7fa524073f90 2023-11-26T07:17:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa50bfff700 1 -- 172.21.15.83:0/3427557572 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514003a60 con 0x7fa524073f90 2023-11-26T07:17:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.513+0000 7fa529a13700 1 -- 172.21.15.83:0/3427557572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5241540f0 con 0x7fa524073f90 2023-11-26T07:17:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.514+0000 7fa529a13700 1 -- 172.21.15.83:0/3427557572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5241546b0 con 0x7fa524073f90 2023-11-26T07:17:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.514+0000 7fa529a13700 1 -- 172.21.15.83:0/3427557572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa524068480 con 0x7fa524073f90 2023-11-26T07:17:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.515+0000 7fa50bfff700 1 -- 172.21.15.83:0/3427557572 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fa514003bc0 con 0x7fa524073f90 2023-11-26T07:17:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.515+0000 7fa50bfff700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa51006b1a0 0x7fa51006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.515+0000 7fa50bfff700 1 -- 172.21.15.83:0/3427557572 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa51408a8a0 con 0x7fa524073f90 2023-11-26T07:17:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.515+0000 7fa522ffd700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa51006b1a0 0x7fa51006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.516+0000 7fa522ffd700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa51006b1a0 0x7fa51006d660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa518005910 tx=0x7fa518009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.520+0000 7fa50bfff700 1 -- 172.21.15.83:0/3427557572 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa514055870 con 0x7fa524073f90 2023-11-26T07:17:51.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.708+0000 7fa529a13700 1 -- 172.21.15.83:0/3427557572 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa52406e820 con 0x7fa51006b1a0 2023-11-26T07:17:51.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.710+0000 7fa50bfff700 1 -- 172.21.15.83:0/3427557572 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50722 (secure 0 0 0) 0x7fa5180013f0 con 0x7fa51006b1a0 2023-11-26T07:17:51.712 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 -- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa51006b1a0 msgr2=0x7fa51006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa51006b1a0 0x7fa51006d660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa518005910 tx=0x7fa518009040 comp rx=0 tx=0).stop 2023-11-26T07:17:51.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 -- 172.21.15.83:0/3427557572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524073f90 msgr2=0x7fa524160ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524073f90 0x7fa524160ab0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fa514009b60 tx=0x7fa514004750 comp rx=0 tx=0).stop 2023-11-26T07:17:51.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 -- 172.21.15.83:0/3427557572 shutdown_connections 2023-11-26T07:17:51.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fa51006b1a0 0x7fa51006d660 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa52415deb0 0x7fa524160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa52406ed40 0x7fa5241486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 --2- 172.21.15.83:0/3427557572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524073f90 0x7fa524160ab0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 -- 172.21.15.83:0/3427557572 >> 172.21.15.83:0/3427557572 conn(0x7fa5240d0ba0 msgr2=0x7fa5240ad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 -- 172.21.15.83:0/3427557572 shutdown_connections 2023-11-26T07:17:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.712+0000 7fa529a13700 1 -- 172.21.15.83:0/3427557572 wait complete. 2023-11-26T07:17:51.715 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:17:51.725 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-11-26T07:17:50.332195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":285976,"kb_used_data":44872,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754688744,"statfs":{"total":773094113280,"available":772801273856,"internally_reserved":0,"allocated":45948928,"data_stored":40119192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17229834,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-80,"num_read_kb":-8238,"num_write":-177,"num_write_kb":-16903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001293"},"pg_stats":[{"pgid":"1.0","version":"71'128","reported_seq":289,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:17:21.293408+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:17:21.293408+0000","last_peered":"2023-11-26T07:17:21.293408+0000","last_clean":"2023-11-26T07:17:21.293408+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:17:21.293408+0000","last_undegraded":"2023-11-26T07:17:21.293408+0000","last_fullsized":"2023-11-26T07:17:21.293408+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39176,"kb_used_data":7996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332664,"statfs":{"total":96636764160,"available":96596647936,"internally_reserved":0,"allocated":8187904,"data_stored":7445543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":0.84099999999999997,"15min":0.86099999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42499999999999999,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.78600000000000003,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.752}]},{"osd":1,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.36299999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69499999999999995,"5min":0.88100000000000001,"15min":1.048},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59799999999999998,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.315},"max":{"1min":0.753,"5min":0.81899999999999995,"15min":0.89600000000000002},"last":0.78000000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79800000000000004,"5min":0.877,"15min":0.92000000000000004},"last":0.67700000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.33600000000000002},"max":{"1min":0.77100000000000002,"5min":0.91300000000000003,"15min":1.0629999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.73999999999999999,"5min":0.91400000000000003,"15min":1.0960000000000001},"last":0.76500000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0269999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69099999999999995,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.627}]},{"osd":5,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72499999999999998,"5min":0.996,"15min":0.996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81100000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.48299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80000000000000004,"5min":1.006,"15min":1.361},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.751,"5min":1.0409999999999999,"15min":1.1899999999999999},"last":0.85299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":4316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":4419584,"data_stored":3702167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54300000000000004},"min":{"1min":0.34599999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.018,"5min":1.159,"15min":1.159},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75800000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77100000000000002,"5min":0.79900000000000004,"15min":0.88900000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.54100000000000004,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":1.0209999999999999},"last":0.58199999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38300000000000001,"5min":0.376,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":0.79200000000000004,"15min":0.96199999999999997},"last":0.441},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.85099999999999998},"last":0.52500000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.55800000000000005},"min":{"1min":0.46800000000000003,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":0.76200000000000001,"5min":4.8689999999999998,"15min":4.8689999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74099999999999999,"5min":0.82599999999999996,"15min":1.1950000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.379}]},{"osd":5,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.36699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.67400000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.49299999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.432},{"interface":"front","average":{"1min":0.63,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.501,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71899999999999997,"5min":0.81999999999999995,"15min":0.83099999999999996},"last":0.45800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36524,"kb_used_data":5348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335316,"statfs":{"total":96636764160,"available":96599363584,"internally_reserved":0,"allocated":5476352,"data_stored":4751403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.497,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.34899999999999998},"min":{"1min":0.26500000000000001,"5min":0.20000000000000001,"15min":0.19700000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.35399999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.33900000000000002},"min":{"1min":0.247,"5min":0.22500000000000001,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28100000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.29999999999999999,"5min":0.252,"15min":0.246},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.378},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.26700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47999999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.32500000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":1.167,"15min":1.167},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.80700000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.5}]},{"osd":5,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.879,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.48499999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.48499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83199999999999996,"5min":0.91700000000000004,"15min":1.129},"last":0.437},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76600000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.46400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.79200000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.46300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40640,"kb_used_data":9460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331200,"statfs":{"total":96636764160,"available":96595148800,"internally_reserved":0,"allocated":9687040,"data_stored":8945279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.217,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.51900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.215},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.36899999999999999,"15min":0.36699999999999999},"min":{"1min":0.25600000000000001,"5min":0.23999999999999999,"15min":0.23300000000000001},"max":{"1min":0.50900000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39000000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":0.64300000000000002,"15min":0.65700000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.37,"15min":0.35199999999999998},"min":{"1min":0.246,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":0.48299999999999998,"5min":1.202,"15min":1.202},"last":0.58399999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.39900000000000002},"min":{"1min":0.27900000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52200000000000002,"5min":0.75,"15min":0.75},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41899999999999998,"15min":0.433},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.53600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.53500000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":1.419,"15min":1.419},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64000000000000001,"15min":0.66300000000000003},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":1,"15min":1.0009999999999999},"last":0.38600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59699999999999998,"15min":0.627},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81499999999999995,"5min":1.046,"15min":1.046},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.629},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.746,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81699999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81699999999999995,"5min":1.387,"15min":1.387},"last":0.63}]},{"osd":7,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.626},"min":{"1min":0.41599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76700000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.627,"15min":0.65900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":1.087,"15min":1.087},"last":0.41699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35496,"kb_used_data":4320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336344,"statfs":{"total":96636764160,"available":96600416256,"internally_reserved":0,"allocated":4423680,"data_stored":3702901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.36499999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.59199999999999997,"5min":0.627,"15min":0.627},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.248,"5min":0.186,"15min":0.186},"max":{"1min":0.51500000000000001,"5min":0.70099999999999996,"15min":0.84599999999999997},"last":0.35699999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38700000000000001},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52000000000000002,"5min":0.58799999999999997,"15min":0.65700000000000003},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38500000000000001,"15min":0.37},"min":{"1min":0.252,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.308}]},{"osd":3,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36299999999999999,"15min":0.36899999999999999},"min":{"1min":0.254,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.58799999999999997,"15min":0.68700000000000006},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.42199999999999999},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.76100000000000001},"last":0.29499999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.65600000000000003},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81799999999999995,"5min":0.85799999999999998,"15min":2.2160000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80600000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.61599999999999999},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82899999999999996,"5min":0.90000000000000002,"15min":1.075},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.0309999999999999,"15min":1.2190000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90800000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89400000000000002,"5min":1.004,"15min":1.03},"last":0.55000000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92600000000000005,"5min":0.94899999999999995,"15min":1.1919999999999999},"last":0.63},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.62},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.40999999999999998},"max":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":1.1120000000000001},"last":0.57699999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36968,"kb_used_data":5792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334872,"statfs":{"total":96636764160,"available":96598908928,"internally_reserved":0,"allocated":5931008,"data_stored":5203060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.35299999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.188},"max":{"1min":0.55700000000000005,"5min":0.629,"15min":1.284},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.371,"15min":0.45400000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57799999999999996,"5min":0.63600000000000001,"15min":2.8460000000000001},"last":0.32400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.34100000000000003,"15min":0.48299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53500000000000003,"5min":0.61799999999999999,"15min":4.9109999999999996},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.39100000000000001,"15min":0.47299999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.65800000000000003,"5min":0.69399999999999995,"15min":2.8940000000000001},"last":0.40200000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41999999999999998,"15min":0.48599999999999999},"min":{"1min":0.27900000000000003,"5min":0.224,"15min":0.224},"max":{"1min":0.626,"5min":0.626,"15min":2.9790000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38300000000000001,"15min":0.41699999999999998},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.23100000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":1.3440000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.66000000000000003},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":2.9500000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60899999999999999,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.405},"last":0.621}]},{"osd":5,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":0.629},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.377},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":4.8129999999999997},"last":0.72199999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61699999999999999,"15min":0.66000000000000003},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.4319999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.75700000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":4.883},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.77200000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":4.8499999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":2.923},"last":0.64800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35512,"kb_used_data":4332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336328,"statfs":{"total":96636764160,"available":96600399872,"internally_reserved":0,"allocated":4435968,"data_stored":3708633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.65600000000000003,"5min":0.79400000000000004,"15min":2.9510000000000001},"last":0.498},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54600000000000004,"15min":0.53100000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70899999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57699999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55900000000000005,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68700000000000006,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.64200000000000002},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.91100000000000003,"15min":2.9820000000000002},"last":0.32400000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77400000000000002,"5min":2.113,"15min":2.113},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.65700000000000003},"min":{"1min":0.40400000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":3.0030000000000001},"last":0.42399999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.54700000000000004},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.33300000000000002},"max":{"1min":0.67600000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.58499999999999996},"min":{"1min":0.41899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85199999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80900000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.35899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78100000000000003,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.64700000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.41199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.84799999999999998},"last":0.53300000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82999999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.55300000000000005,"5min":0.315,"15min":0.315},"max":{"1min":0.91400000000000003,"5min":1.01,"15min":1.01},"last":0.63100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30328,"kb_used_data":3308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341512,"statfs":{"total":96636764160,"available":96605708288,"internally_reserved":0,"allocated":3387392,"data_stored":2660206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.29799999999999999,"15min":0.27500000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96199999999999997,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.36399999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79400000000000004,"5min":1.091,"15min":1.091},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.44}]},{"osd":2,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.42399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.54200000000000004},"min":{"1min":0.44400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53700000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.503},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61499999999999999,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.52300000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.35599999999999998,"15min":0.311},"max":{"1min":1.044,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0980000000000001,"5min":1.161,"15min":1.161},"last":0.59499999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.35199999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0129999999999999,"5min":1.196,"15min":1.196},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.628,"15min":0.64500000000000002},"min":{"1min":0.40699999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0309999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:51.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:51.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.845+0000 7f4a42e2d700 1 Processor -- start 2023-11-26T07:17:51.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.845+0000 7f4a42e2d700 1 -- start start 2023-11-26T07:17:51.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a42e2d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4a3c152cc0 0x7f4a3c06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a42e2d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4a3c0d0630 0x7f4a3c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a42e2d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3c15c4b0 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a42e2d700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3c15c790 con 0x7f4a3c152cc0 2023-11-26T07:17:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a42e2d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3c15ca70 con 0x7f4a3c0d0630 2023-11-26T07:17:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a40bc9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4a3c152cc0 0x7f4a3c06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a40bc9700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4a3c152cc0 0x7f4a3c06c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42526/0 (socket says 172.21.15.83:42526) 2023-11-26T07:17:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a3bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f4a3c0b49a0 0x7f4a3c0cff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34332/0 (socket says 172.21.15.83:34332) 2023-11-26T07:17:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a40bc9700 1 -- 172.21.15.83:0/700667361 learned_addr learned my addr 172.21.15.83:0/700667361 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:51.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a413ca700 1 --2- 172.21.15.83:0/700667361 >> v2:172.21.15.203:3300/0 conn(0x7f4a3c0d0630 0x7f4a3c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a40bc9700 1 -- 172.21.15.83:0/700667361 >> v2:172.21.15.203:3300/0 conn(0x7f4a3c0d0630 msgr2=0x7f4a3c06c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a40bc9700 1 --2- 172.21.15.83:0/700667361 >> v2:172.21.15.203:3300/0 conn(0x7f4a3c0d0630 0x7f4a3c06c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a40bc9700 1 -- 172.21.15.83:0/700667361 >> v1:172.21.15.83:6789/0 conn(0x7f4a3c0b49a0 legacy=0x7f4a3c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a40bc9700 1 -- 172.21.15.83:0/700667361 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c15cd50 con 0x7f4a3c152cc0 2023-11-26T07:17:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a3b7fe700 1 -- 172.21.15.83:0/700667361 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87896053 0 0) 0x7f4a3c15c4b0 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a40bc9700 1 --2- 172.21.15.83:0/700667361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3c152cc0 0x7f4a3c06c000 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0b4e80 tx=0x7f4a30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320d5c09cdf66f88 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.846+0000 7f4a3b7fe700 1 -- 172.21.15.83:0/700667361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000b870 con 0x7f4a3c152cc0 2023-11-26T07:17:51.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.847+0000 7f4a3b7fe700 1 -- 172.21.15.83:0/700667361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000b9d0 con 0x7f4a3c152cc0 2023-11-26T07:17:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.847+0000 7f4a3b7fe700 1 -- 172.21.15.83:0/700667361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000bd60 con 0x7f4a3c152cc0 2023-11-26T07:17:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.847+0000 7f4a42e2d700 1 -- 172.21.15.83:0/700667361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3c152cc0 msgr2=0x7f4a3c06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.847+0000 7f4a42e2d700 1 --2- 172.21.15.83:0/700667361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3c152cc0 0x7f4a3c06c000 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0b4e80 tx=0x7f4a30009bd0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.847+0000 7f4a42e2d700 1 -- 172.21.15.83:0/700667361 shutdown_connections 2023-11-26T07:17:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.847+0000 7f4a42e2d700 1 --2- 172.21.15.83:0/700667361 >> v2:172.21.15.203:3300/0 conn(0x7f4a3c0d0630 0x7f4a3c06c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.847+0000 7f4a42e2d700 1 --2- 172.21.15.83:0/700667361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3c152cc0 0x7f4a3c06c000 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.847+0000 7f4a42e2d700 1 -- 172.21.15.83:0/700667361 >> 172.21.15.83:0/700667361 conn(0x7f4a3c071b90 msgr2=0x7f4a3c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.847+0000 7f4a42e2d700 1 -- 172.21.15.83:0/700667361 shutdown_connections 2023-11-26T07:17:51.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.847+0000 7f4a42e2d700 1 -- 172.21.15.83:0/700667361 wait complete. 2023-11-26T07:17:51.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.848+0000 7f4a42e2d700 1 Processor -- start 2023-11-26T07:17:51.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.848+0000 7f4a42e2d700 1 -- start start 2023-11-26T07:17:51.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.849+0000 7f4a42e2d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3c0b49a0 0x7f4a3c156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.849+0000 7f4a42e2d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a3c0d0630 0x7f4a3c158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.849+0000 7f4a42e2d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3c152cc0 0x7f4a3c158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.849+0000 7f4a42e2d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c153680 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.849+0000 7f4a42e2d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c153c40 con 0x7f4a3c152cc0 2023-11-26T07:17:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.849+0000 7f4a42e2d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c159080 con 0x7f4a3c0d0630 2023-11-26T07:17:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.849+0000 7f4a40bc9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3c0b49a0 0x7f4a3c156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.849+0000 7f4a40bc9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3c0b49a0 0x7f4a3c156c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42532/0 (socket says 172.21.15.83:42532) 2023-11-26T07:17:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.849+0000 7f4a40bc9700 1 -- 172.21.15.83:0/1033957868 learned_addr learned my addr 172.21.15.83:0/1033957868 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.849+0000 7f4a413ca700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3c152cc0 0x7f4a3c158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.849+0000 7f4a3bfff700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a3c0d0630 0x7f4a3c158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a40bc9700 1 -- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a3c0d0630 msgr2=0x7f4a3c158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a40bc9700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a3c0d0630 0x7f4a3c158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a40bc9700 1 -- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3c152cc0 msgr2=0x7f4a3c158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a40bc9700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3c152cc0 0x7f4a3c158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a40bc9700 1 -- 172.21.15.83:0/1033957868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30009550 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a413ca700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3c152cc0 0x7f4a3c158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a40bc9700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3c0b49a0 0x7f4a3c156c80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4a30000c00 tx=0x7f4a30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a397fa700 1 -- 172.21.15.83:0/1033957868 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30004520 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a397fa700 1 -- 172.21.15.83:0/1033957868 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3001e070 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a42e2d700 1 -- 172.21.15.83:0/1033957868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c1571c0 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a397fa700 1 -- 172.21.15.83:0/1033957868 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a300038a0 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.850+0000 7f4a42e2d700 1 -- 172.21.15.83:0/1033957868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c15ee60 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.851+0000 7f4a42e2d700 1 -- 172.21.15.83:0/1033957868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3c153f20 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.851+0000 7f4a397fa700 1 -- 172.21.15.83:0/1033957868 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f4a30003a00 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.852+0000 7f4a397fa700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4a2006b1a0 0x7f4a2006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.852+0000 7f4a397fa700 1 -- 172.21.15.83:0/1033957868 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4a30020070 con 0x7f4a3c0b49a0 2023-11-26T07:17:51.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.852+0000 7f4a3bfff700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4a2006b1a0 0x7f4a2006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.852+0000 7f4a3bfff700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4a2006b1a0 0x7f4a2006d660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0d0150 tx=0x7f4a28007be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:51.856+0000 7f4a397fa700 1 -- 172.21.15.83:0/1033957868 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a30055d80 con 0x7f4a3c0b49a0 2023-11-26T07:17:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.047+0000 7f4a42e2d700 1 -- 172.21.15.83:0/1033957868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a3c0753f0 con 0x7f4a3c0b49a0 2023-11-26T07:17:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.048+0000 7f4a397fa700 1 -- 172.21.15.83:0/1033957868 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+12984 (secure 0 0 0) 0x7f4a30015020 con 0x7f4a3c0b49a0 2023-11-26T07:17:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:52.050 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":105,"fsid":"0aae1049-cc8c-4011-a01f-378673e57a96","created":"2023-11-26T07:11:02.736202+0000","modified":"2023-11-26T07:17:48.562011+0000","last_up_change":"2023-11-26T07:11:14.365020+0000","last_in_change":"2023-11-26T07:11:05.232194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:11:16.349108+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2941c9d0-0912-44e6-bce1-c2c8749bd713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6801","nonce":3940814281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6803","nonce":3940814281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6807","nonce":3940814281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3940814281},{"type":"v1","addr":"172.21.15.83:6805","nonce":3940814281}]},"public_addr":"172.21.15.83:6801/3940814281","cluster_addr":"172.21.15.83:6803/3940814281","heartbeat_back_addr":"172.21.15.83:6807/3940814281","heartbeat_front_addr":"172.21.15.83:6805/3940814281","state":["exists","up"]},{"osd":1,"uuid":"7c65463e-79cb-4487-aec7-5ef4fd9b7520","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6809","nonce":438950548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6811","nonce":438950548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6815","nonce":438950548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":438950548},{"type":"v1","addr":"172.21.15.83:6813","nonce":438950548}]},"public_addr":"172.21.15.83:6809/438950548","cluster_addr":"172.21.15.83:6811/438950548","heartbeat_back_addr":"172.21.15.83:6815/438950548","heartbeat_front_addr":"172.21.15.83:6813/438950548","state":["exists","up"]},{"osd":2,"uuid":"f8c63f2d-9fb4-4b12-ae19-3445c43fc9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6820","nonce":913771869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6824","nonce":913771869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6831","nonce":913771869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":913771869},{"type":"v1","addr":"172.21.15.83:6828","nonce":913771869}]},"public_addr":"172.21.15.83:6820/913771869","cluster_addr":"172.21.15.83:6824/913771869","heartbeat_back_addr":"172.21.15.83:6831/913771869","heartbeat_front_addr":"172.21.15.83:6828/913771869","state":["exists","up"]},{"osd":3,"uuid":"10dbc082-3b8c-46a8-81c2-e1e471c78783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6817","nonce":1108190663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6819","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6821","nonce":1108190663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6827","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6829","nonce":1108190663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6823","nonce":1108190663},{"type":"v1","addr":"172.21.15.83:6825","nonce":1108190663}]},"public_addr":"172.21.15.83:6817/1108190663","cluster_addr":"172.21.15.83:6821/1108190663","heartbeat_back_addr":"172.21.15.83:6829/1108190663","heartbeat_front_addr":"172.21.15.83:6825/1108190663","state":["exists","up"]},{"osd":4,"uuid":"777669fb-d21e-4bff-b20a-514e5803d4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6809","nonce":1845411432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6811","nonce":1845411432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6815","nonce":1845411432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1845411432},{"type":"v1","addr":"172.21.15.203:6813","nonce":1845411432}]},"public_addr":"172.21.15.203:6809/1845411432","cluster_addr":"172.21.15.203:6811/1845411432","heartbeat_back_addr":"172.21.15.203:6815/1845411432","heartbeat_front_addr":"172.21.15.203:6813/1845411432","state":["exists","up"]},{"osd":5,"uuid":"75d30c7f-9ed3-45e8-be57-a79c37c2f1f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6801","nonce":2181588544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6803","nonce":2181588544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6807","nonce":2181588544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":2181588544},{"type":"v1","addr":"172.21.15.203:6805","nonce":2181588544}]},"public_addr":"172.21.15.203:6801/2181588544","cluster_addr":"172.21.15.203:6803/2181588544","heartbeat_back_addr":"172.21.15.203:6807/2181588544","heartbeat_front_addr":"172.21.15.203:6805/2181588544","state":["exists","up"]},{"osd":6,"uuid":"b639b30a-d697-4550-8610-09f074b23c40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6825","nonce":2898154732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6827","nonce":2898154732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6831","nonce":2898154732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2898154732},{"type":"v1","addr":"172.21.15.203:6829","nonce":2898154732}]},"public_addr":"172.21.15.203:6825/2898154732","cluster_addr":"172.21.15.203:6827/2898154732","heartbeat_back_addr":"172.21.15.203:6831/2898154732","heartbeat_front_addr":"172.21.15.203:6829/2898154732","state":["exists","up"]},{"osd":7,"uuid":"e1479b74-f049-477c-97c9-101df7533c7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6817","nonce":2979691223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6819","nonce":2979691223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6823","nonce":2979691223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":2979691223},{"type":"v1","addr":"172.21.15.203:6821","nonce":2979691223}]},"public_addr":"172.21.15.203:6817/2979691223","cluster_addr":"172.21.15.203:6819/2979691223","heartbeat_back_addr":"172.21.15.203:6823/2979691223","heartbeat_front_addr":"172.21.15.203:6821/2979691223","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.340875+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.399247+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.466067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.464319+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.434784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.422342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.453536+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:11:10.503586+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6835/3327659486":"2023-11-27T07:17:45.315683+0000","172.21.15.83:6834/3327659486":"2023-11-27T07:17:45.315683+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 -- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4a2006b1a0 msgr2=0x7f4a2006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4a2006b1a0 0x7f4a2006d660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0d0150 tx=0x7f4a28007be0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 -- 172.21.15.83:0/1033957868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3c0b49a0 msgr2=0x7f4a3c156c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3c0b49a0 0x7f4a3c156c80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4a30000c00 tx=0x7f4a30004750 comp rx=0 tx=0).stop 2023-11-26T07:17:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 -- 172.21.15.83:0/1033957868 shutdown_connections 2023-11-26T07:17:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f4a2006b1a0 0x7f4a2006d660 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a3c152cc0 0x7f4a3c158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3c0b49a0 0x7f4a3c156c80 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 --2- 172.21.15.83:0/1033957868 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a3c0d0630 0x7f4a3c158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:52.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 -- 172.21.15.83:0/1033957868 >> 172.21.15.83:0/1033957868 conn(0x7f4a3c071b90 msgr2=0x7f4a3c151020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:52.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 -- 172.21.15.83:0/1033957868 shutdown_connections 2023-11-26T07:17:52.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:52.050+0000 7f4a42e2d700 1 -- 172.21.15.83:0/1033957868 wait complete. 2023-11-26T07:17:53.065 INFO:tasks.ceph:Scrubbing osd.0 2023-11-26T07:17:53.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:17:53.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.200+0000 7f1e2c25d700 1 Processor -- start 2023-11-26T07:17:53.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.201+0000 7f1e2c25d700 1 -- start start 2023-11-26T07:17:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.201+0000 7f1e2c25d700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1e2415e0c0 0x7f1e240cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.201+0000 7f1e2c25d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24071d20 0x7f1e240b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.201+0000 7f1e2c25d700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e240d29e0 con 0x7f1e240d3a40 2023-11-26T07:17:53.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.201+0000 7f1e2c25d700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e240d2c80 con 0x7f1e2415e0c0 2023-11-26T07:17:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.201+0000 7f1e2c25d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240d2f60 con 0x7f1e24071d20 2023-11-26T07:17:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e297f8700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1e240d3a40 0x7f1e24071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34348/0 (socket says 172.21.15.83:34348) 2023-11-26T07:17:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e297f8700 1 -- 172.21.15.83:0/1395462506 learned_addr learned my addr 172.21.15.83:0/1395462506 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e2a7fa700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24071d20 0x7f1e240b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e29ff9700 1 --2- 172.21.15.83:0/1395462506 >> v2:172.21.15.203:3300/0 conn(0x7f1e2415e0c0 0x7f1e240cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132215957 0 0) 0x7f1e240d29e0 con 0x7f1e240d3a40 2023-11-26T07:17:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e100035b0 con 0x7f1e240d3a40 2023-11-26T07:17:53.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1110053810 0 0) 0x7f1e100035b0 con 0x7f1e240d3a40 2023-11-26T07:17:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24071d20 msgr2=0x7f1e240b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24071d20 0x7f1e240b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 >> v2:172.21.15.203:3300/0 conn(0x7f1e2415e0c0 msgr2=0x7f1e240cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 --2- 172.21.15.83:0/1395462506 >> v2:172.21.15.203:3300/0 conn(0x7f1e2415e0c0 0x7f1e240cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e240d3240 con 0x7f1e240d3a40 2023-11-26T07:17:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f1e20001670 con 0x7f1e240d3a40 2023-11-26T07:17:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 >> v1:172.21.15.83:6789/0 conn(0x7f1e240d3a40 legacy=0x7f1e24071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10003a00 0x7f1e10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e10004850 0x7f1e10004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24071d20 0x7f1e10005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240d2c80 con 0x7f1e10003a00 2023-11-26T07:17:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240d2f60 con 0x7f1e10004850 2023-11-26T07:17:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1000ba40 con 0x7f1e24071d20 2023-11-26T07:17:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e29ff9700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10003a00 0x7f1e10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e2a7fa700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24071d20 0x7f1e10005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.202+0000 7f1e297f8700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e10004850 0x7f1e10004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e29ff9700 1 -- 172.21.15.83:0/1395462506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24071d20 msgr2=0x7f1e10005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e29ff9700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24071d20 0x7f1e10005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e29ff9700 1 -- 172.21.15.83:0/1395462506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e10004850 msgr2=0x7f1e10004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e29ff9700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e10004850 0x7f1e10004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e29ff9700 1 -- 172.21.15.83:0/1395462506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e100035b0 con 0x7f1e10003a00 2023-11-26T07:17:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e29ff9700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10003a00 0x7f1e10003e20 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009b60 tx=0x7f1e10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e563becc5cab05 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400e6c0 con 0x7f1e10003a00 2023-11-26T07:17:53.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1400e820 con 0x7f1e10003a00 2023-11-26T07:17:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e28ff7700 1 -- 172.21.15.83:0/1395462506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400eb90 con 0x7f1e10003a00 2023-11-26T07:17:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1395462506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10003a00 msgr2=0x7f1e10003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e2c25d700 1 --2- 172.21.15.83:0/1395462506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10003a00 0x7f1e10003e20 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009b60 tx=0x7f1e10005880 comp rx=0 tx=0).stop 2023-11-26T07:17:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.203+0000 7f1e2a7fa700 1 -- 172.21.15.83:0/1395462506 reap_dead start 2023-11-26T07:17:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.204+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1395462506 shutdown_connections 2023-11-26T07:17:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.204+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1395462506 >> 172.21.15.83:0/1395462506 conn(0x7f1e2406b620 msgr2=0x7f1e240d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.204+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1395462506 shutdown_connections 2023-11-26T07:17:53.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.204+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1395462506 wait complete. 2023-11-26T07:17:53.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.205+0000 7f1e2c25d700 1 Processor -- start 2023-11-26T07:17:53.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.205+0000 7f1e2c25d700 1 -- start start 2023-11-26T07:17:53.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.205+0000 7f1e2c25d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10003a00 0x7f1e24154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e2c25d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e24147c30 0x7f1e240cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e2c25d700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24157870 0x7f1e24147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e2c25d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e241f3c40 con 0x7f1e10003a00 2023-11-26T07:17:53.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e2c25d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240cf590 con 0x7f1e24147c30 2023-11-26T07:17:53.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e2c25d700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e24145010 con 0x7f1e24157870 2023-11-26T07:17:53.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e29ff9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10003a00 0x7f1e24154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e29ff9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10003a00 0x7f1e24154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42550/0 (socket says 172.21.15.83:42550) 2023-11-26T07:17:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e29ff9700 1 -- 172.21.15.83:0/1983387034 learned_addr learned my addr 172.21.15.83:0/1983387034 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e297f8700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e24147c30 0x7f1e240cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e2a7fa700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24157870 0x7f1e24147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e29ff9700 1 -- 172.21.15.83:0/1983387034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24157870 msgr2=0x7f1e24147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e29ff9700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1e24157870 0x7f1e24147000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e29ff9700 1 -- 172.21.15.83:0/1983387034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e24147c30 msgr2=0x7f1e240cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e29ff9700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e24147c30 0x7f1e240cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e29ff9700 1 -- 172.21.15.83:0/1983387034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2415e620 con 0x7f1e10003a00 2023-11-26T07:17:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e29ff9700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10003a00 0x7f1e24154ed0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1e140090b0 tx=0x7f1e140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e1a7fc700 1 -- 172.21.15.83:0/1983387034 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400e6c0 con 0x7f1e10003a00 2023-11-26T07:17:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e1a7fc700 1 -- 172.21.15.83:0/1983387034 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e14003ad0 con 0x7f1e10003a00 2023-11-26T07:17:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1983387034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e241f3c40 con 0x7f1e10003a00 2023-11-26T07:17:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e1a7fc700 1 -- 172.21.15.83:0/1983387034 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14003e80 con 0x7f1e10003a00 2023-11-26T07:17:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.206+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1983387034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e241455d0 con 0x7f1e10003a00 2023-11-26T07:17:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.207+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1983387034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e2415e4a0 con 0x7f1e10003a00 2023-11-26T07:17:53.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.208+0000 7f1e1a7fc700 1 -- 172.21.15.83:0/1983387034 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f1e14013470 con 0x7f1e10003a00 2023-11-26T07:17:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.208+0000 7f1e1a7fc700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1e0806b1a0 0x7f1e0806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.208+0000 7f1e1a7fc700 1 -- 172.21.15.83:0/1983387034 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1e1408b060 con 0x7f1e10003a00 2023-11-26T07:17:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.208+0000 7f1e1a7fc700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] conn(0x7f1e08071000 0x7f1e08073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.208+0000 7f1e1a7fc700 1 -- 172.21.15.83:0/1983387034 --> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e08073af0 con 0x7f1e08071000 2023-11-26T07:17:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.208+0000 7f1e297f8700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1e0806b1a0 0x7f1e0806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.208+0000 7f1e1a7fc700 1 -- 172.21.15.83:0/1983387034 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e1408d6a0 con 0x7f1e10003a00 2023-11-26T07:17:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.208+0000 7f1e2a7fa700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] conn(0x7f1e08071000 0x7f1e08073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.209+0000 7f1e2a7fa700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] conn(0x7f1e08071000 0x7f1e08073420 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.209+0000 7f1e297f8700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1e0806b1a0 0x7f1e0806d660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1e20000c00 tx=0x7f1e20007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.209+0000 7f1e1a7fc700 1 -- 172.21.15.83:0/1983387034 <== osd.0 v2:172.21.15.83:6800/3940814281 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1e08073af0 con 0x7f1e08071000 2023-11-26T07:17:53.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.224+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1983387034 --> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1e240b9c50 con 0x7f1e08071000 2023-11-26T07:17:53.226 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.224+0000 7f1e1a7fc700 1 -- 172.21.15.83:0/1983387034 <== osd.0 v2:172.21.15.83:6800/3940814281 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1e1c00f040 con 0x7f1e08071000 2023-11-26T07:17:53.226 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:53.226 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:17:53.226 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:53.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.225+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1983387034 >> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] conn(0x7f1e08071000 msgr2=0x7f1e08073420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.225+0000 7f1e2c25d700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.83:6800/3940814281,v1:172.21.15.83:6801/3940814281] conn(0x7f1e08071000 0x7f1e08073420 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.227 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.225+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1983387034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1e0806b1a0 msgr2=0x7f1e0806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.225+0000 7f1e2c25d700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f1e0806b1a0 0x7f1e0806d660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1e20000c00 tx=0x7f1e20007040 comp rx=0 tx=0).stop 2023-11-26T07:17:53.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.225+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1983387034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10003a00 msgr2=0x7f1e24154ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.225+0000 7f1e2c25d700 1 --2- 172.21.15.83:0/1983387034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e10003a00 0x7f1e24154ed0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1e140090b0 tx=0x7f1e140047c0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.225+0000 7f1e2a7fa700 1 -- 172.21.15.83:0/1983387034 reap_dead start 2023-11-26T07:17:53.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.225+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1983387034 shutdown_connections 2023-11-26T07:17:53.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.225+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1983387034 >> 172.21.15.83:0/1983387034 conn(0x7f1e2406b620 msgr2=0x7f1e240ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:53.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.225+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1983387034 shutdown_connections 2023-11-26T07:17:53.228 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.225+0000 7f1e2c25d700 1 -- 172.21.15.83:0/1983387034 wait complete. 2023-11-26T07:17:53.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-26T07:17:53.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.357+0000 7f3991aa8700 1 Processor -- start 2023-11-26T07:17:53.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.357+0000 7f3991aa8700 1 -- start start 2023-11-26T07:17:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.358+0000 7f3991aa8700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f398c15e080 0x7f398c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.358+0000 7f3991aa8700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c0ae430 con 0x7f398c06c710 2023-11-26T07:17:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.358+0000 7f3991aa8700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c0ae710 con 0x7f398c0af4d0 2023-11-26T07:17:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.358+0000 7f3991aa8700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c0af0b0 con 0x7f398c15e080 2023-11-26T07:17:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.358+0000 7f398bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f398c06c710 0x7f398c06e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34372/0 (socket says 172.21.15.83:34372) 2023-11-26T07:17:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.358+0000 7f398bfff700 1 -- 172.21.15.83:0/3152340242 learned_addr learned my addr 172.21.15.83:0/3152340242 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:53.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.358+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612331047 0 0) 0x7f398c0ae430 con 0x7f398c06c710 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398b7fe700 1 --2- 172.21.15.83:0/3152340242 >> v2:172.21.15.203:3300/0 conn(0x7f398c15e080 0x7f398c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39740035b0 con 0x7f398c06c710 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2250491166 0 0) 0x7f39740035b0 con 0x7f398c06c710 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 >> v2:172.21.15.203:3300/0 conn(0x7f398c15e080 msgr2=0x7f398c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 --2- 172.21.15.83:0/3152340242 >> v2:172.21.15.203:3300/0 conn(0x7f398c15e080 0x7f398c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 >> v1:172.21.15.203:6789/0 conn(0x7f398c0af4d0 legacy=0x7f398c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0ae9f0 con 0x7f398c06c710 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f39840019f0 con 0x7f398c06c710 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 >> v1:172.21.15.83:6789/0 conn(0x7f398c06c710 legacy=0x7f398c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3974003a00 0x7f3974003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 0x7f3974004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3974005210 0x7f397400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0af0b0 con 0x7f3974004850 2023-11-26T07:17:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397400c0b0 con 0x7f3974005210 2023-11-26T07:17:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397400c270 con 0x7f3974003a00 2023-11-26T07:17:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398affd700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 0x7f3974004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.365 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398b7fe700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3974003a00 0x7f3974003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398bfff700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3974005210 0x7f397400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398affd700 1 -- 172.21.15.83:0/3152340242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3974003a00 msgr2=0x7f3974003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398affd700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3974003a00 0x7f3974003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398affd700 1 -- 172.21.15.83:0/3152340242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3974005210 msgr2=0x7f397400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398affd700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3974005210 0x7f397400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398affd700 1 -- 172.21.15.83:0/3152340242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39740035b0 con 0x7f3974004850 2023-11-26T07:17:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398b7fe700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3974003a00 0x7f3974003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:53.366 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398bfff700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3974005210 0x7f397400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398bfff700 1 -- 172.21.15.83:0/3152340242 reap_dead start 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398affd700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 0x7f3974004cd0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f398c0af950 tx=0x7f3974005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73913c4039d22aef server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.359+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c0052d0 con 0x7f3974004850 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.360+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c005430 con 0x7f3974004850 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.360+0000 7f398a7fc700 1 -- 172.21.15.83:0/3152340242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c0038c0 con 0x7f3974004850 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.360+0000 7f3991aa8700 1 -- 172.21.15.83:0/3152340242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 msgr2=0x7f3974004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.360+0000 7f3991aa8700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 0x7f3974004cd0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f398c0af950 tx=0x7f3974005880 comp rx=0 tx=0).stop 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.360+0000 7f3991aa8700 1 -- 172.21.15.83:0/3152340242 shutdown_connections 2023-11-26T07:17:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.360+0000 7f3991aa8700 1 --2- 172.21.15.83:0/3152340242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 0x7f3974004cd0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.360+0000 7f3991aa8700 1 -- 172.21.15.83:0/3152340242 >> 172.21.15.83:0/3152340242 conn(0x7f398c071b90 msgr2=0x7f398c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.360+0000 7f3991aa8700 1 -- 172.21.15.83:0/3152340242 shutdown_connections 2023-11-26T07:17:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.360+0000 7f3991aa8700 1 -- 172.21.15.83:0/3152340242 wait complete. 2023-11-26T07:17:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.361+0000 7f3991aa8700 1 Processor -- start 2023-11-26T07:17:53.368 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f3991aa8700 1 -- start start 2023-11-26T07:17:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f3991aa8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 0x7f398c159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.370 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f3991aa8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f398c142170 0x7f398c144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f3991aa8700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f398c1453a0 0x7f398c144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f3991aa8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c158e50 con 0x7f3974004850 2023-11-26T07:17:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f3991aa8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0cdd10 con 0x7f398c142170 2023-11-26T07:17:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f3991aa8700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0cd750 con 0x7f398c1453a0 2023-11-26T07:17:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f398b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 0x7f398c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f398b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 0x7f398c159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42572/0 (socket says 172.21.15.83:42572) 2023-11-26T07:17:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f398b7fe700 1 -- 172.21.15.83:0/4060245589 learned_addr learned my addr 172.21.15.83:0/4060245589 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:53.371 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f398affd700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f398c142170 0x7f398c144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.362+0000 7f398bfff700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f398c1453a0 0x7f398c144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.363+0000 7f398b7fe700 1 -- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f398c1453a0 msgr2=0x7f398c144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.363+0000 7f398b7fe700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f398c1453a0 0x7f398c144c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.363+0000 7f398b7fe700 1 -- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f398c142170 msgr2=0x7f398c144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.363+0000 7f398b7fe700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f398c142170 0x7f398c144730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.363+0000 7f398b7fe700 1 -- 172.21.15.83:0/4060245589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0cda30 con 0x7f3974004850 2023-11-26T07:17:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.363+0000 7f398b7fe700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 0x7f398c159610 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f3980011590 tx=0x7f39800118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.372 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.363+0000 7f3973fff700 1 -- 172.21.15.83:0/4060245589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980004220 con 0x7f3974004850 2023-11-26T07:17:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.363+0000 7f3973fff700 1 -- 172.21.15.83:0/4060245589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3980004380 con 0x7f3974004850 2023-11-26T07:17:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.363+0000 7f3991aa8700 1 -- 172.21.15.83:0/4060245589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c1590f0 con 0x7f3974004850 2023-11-26T07:17:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.363+0000 7f3973fff700 1 -- 172.21.15.83:0/4060245589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980011ea0 con 0x7f3974004850 2023-11-26T07:17:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.363+0000 7f3991aa8700 1 -- 172.21.15.83:0/4060245589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c142700 con 0x7f3974004850 2023-11-26T07:17:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.364+0000 7f3991aa8700 1 -- 172.21.15.83:0/4060245589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c0753f0 con 0x7f3974004850 2023-11-26T07:17:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.364+0000 7f3973fff700 1 -- 172.21.15.83:0/4060245589 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f39800036a0 con 0x7f3974004850 2023-11-26T07:17:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.365+0000 7f3973fff700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f397806b1d0 0x7f397806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.365+0000 7f3973fff700 1 -- 172.21.15.83:0/4060245589 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f39800898a0 con 0x7f3974004850 2023-11-26T07:17:53.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.365+0000 7f398affd700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f397806b1d0 0x7f397806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.365+0000 7f398affd700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f397806b1d0 0x7f397806d690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f397c015040 tx=0x7f397c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.374 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.369+0000 7f3973fff700 1 -- 172.21.15.83:0/4060245589 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f39800548c0 con 0x7f3974004850 2023-11-26T07:17:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.560+0000 7f3991aa8700 1 -- 172.21.15.83:0/4060245589 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f398c0b05a0 con 0x7f397806b1d0 2023-11-26T07:17:53.561 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.560+0000 7f3973fff700 1 -- 172.21.15.83:0/4060245589 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f397c0051d0 con 0x7f397806b1d0 2023-11-26T07:17:53.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.562+0000 7f3991aa8700 1 -- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f397806b1d0 msgr2=0x7f397806d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.562+0000 7f3991aa8700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f397806b1d0 0x7f397806d690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f397c015040 tx=0x7f397c009030 comp rx=0 tx=0).stop 2023-11-26T07:17:53.564 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.562+0000 7f3991aa8700 1 -- 172.21.15.83:0/4060245589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 msgr2=0x7f398c159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.562+0000 7f3991aa8700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 0x7f398c159610 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f3980011590 tx=0x7f39800118a0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.562+0000 7f3991aa8700 1 -- 172.21.15.83:0/4060245589 shutdown_connections 2023-11-26T07:17:53.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.562+0000 7f3991aa8700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f397806b1d0 0x7f397806d690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.562+0000 7f3991aa8700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f398c1453a0 0x7f398c144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.562+0000 7f3991aa8700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3974004850 0x7f398c159610 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.562+0000 7f3991aa8700 1 --2- 172.21.15.83:0/4060245589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f398c142170 0x7f398c144730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.565 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.563+0000 7f3991aa8700 1 -- 172.21.15.83:0/4060245589 >> 172.21.15.83:0/4060245589 conn(0x7f398c071b90 msgr2=0x7f398c0cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:53.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.563+0000 7f3991aa8700 1 -- 172.21.15.83:0/4060245589 shutdown_connections 2023-11-26T07:17:53.566 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.563+0000 7f3991aa8700 1 -- 172.21.15.83:0/4060245589 wait complete. 2023-11-26T07:17:53.566 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 0 to deep-scrub 2023-11-26T07:17:53.576 INFO:tasks.ceph:Scrubbing osd.1 2023-11-26T07:17:53.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:17:53.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.699+0000 7f13b5225700 1 Processor -- start 2023-11-26T07:17:53.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.699+0000 7f13b5225700 1 -- start start 2023-11-26T07:17:53.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.699+0000 7f13b5225700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13b00d1ab0 0x7f13b00d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.699+0000 7f13b5225700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13b00af3b0 0x7f13b00af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.699+0000 7f13b5225700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b00d01c0 con 0x7f13b00cf040 2023-11-26T07:17:53.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.699+0000 7f13b5225700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b00d2470 con 0x7f13b00af3b0 2023-11-26T07:17:53.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.699+0000 7f13b5225700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0070910 con 0x7f13b00d1ab0 2023-11-26T07:17:53.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.700+0000 7f13a659c700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13b00d1ab0 0x7f13b00d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.700+0000 7f13af59e700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13b00af3b0 0x7f13b00af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.700+0000 7f13aed9d700 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f13b00cf040 0x7f13b006fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.83:47238/0 (socket says 172.21.15.83:47238) 2023-11-26T07:17:53.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.700+0000 7f13aed9d700 1 -- 172.21.15.83:0/549887082 learned_addr learned my addr 172.21.15.83:0/549887082 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:53.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.700+0000 7f13af59e700 1 -- 172.21.15.83:0/549887082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13b00d1ab0 msgr2=0x7f13b00d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.700+0000 7f13af59e700 1 --2- 172.21.15.83:0/549887082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13b00d1ab0 0x7f13b00d1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.700+0000 7f13af59e700 1 -- 172.21.15.83:0/549887082 >> v1:172.21.15.203:6789/0 conn(0x7f13b00cf040 legacy=0x7f13b006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.700+0000 7f13af59e700 1 -- 172.21.15.83:0/549887082 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b0070bf0 con 0x7f13b00af3b0 2023-11-26T07:17:53.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.700+0000 7f13a659c700 1 --2- 172.21.15.83:0/549887082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13b00d1ab0 0x7f13b00d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.700+0000 7f13af59e700 1 --2- 172.21.15.83:0/549887082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b00af3b0 0x7f13b00af800 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009840 tx=0x7f13a8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f014fe632441fa04 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.701+0000 7f13ae59c700 1 -- 172.21.15.83:0/549887082 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8011830 con 0x7f13b00af3b0 2023-11-26T07:17:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.701+0000 7f13ae59c700 1 -- 172.21.15.83:0/549887082 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a8011990 con 0x7f13b00af3b0 2023-11-26T07:17:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.701+0000 7f13ae59c700 1 -- 172.21.15.83:0/549887082 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8011d20 con 0x7f13b00af3b0 2023-11-26T07:17:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.701+0000 7f13b5225700 1 -- 172.21.15.83:0/549887082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b00af3b0 msgr2=0x7f13b00af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.701+0000 7f13b5225700 1 --2- 172.21.15.83:0/549887082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b00af3b0 0x7f13b00af800 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009840 tx=0x7f13a8009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.701+0000 7f13b5225700 1 -- 172.21.15.83:0/549887082 shutdown_connections 2023-11-26T07:17:53.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.701+0000 7f13b5225700 1 --2- 172.21.15.83:0/549887082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b00af3b0 0x7f13b00af800 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.701+0000 7f13b5225700 1 --2- 172.21.15.83:0/549887082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13b00d1ab0 0x7f13b00d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.701+0000 7f13b5225700 1 -- 172.21.15.83:0/549887082 >> 172.21.15.83:0/549887082 conn(0x7f13b009aa00 msgr2=0x7f13b009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:53.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.701+0000 7f13b5225700 1 -- 172.21.15.83:0/549887082 shutdown_connections 2023-11-26T07:17:53.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.701+0000 7f13b5225700 1 -- 172.21.15.83:0/549887082 wait complete. 2023-11-26T07:17:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.703+0000 7f13b5225700 1 Processor -- start 2023-11-26T07:17:53.709 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.703+0000 7f13b5225700 1 -- start start 2023-11-26T07:17:53.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.703+0000 7f13b5225700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b00af3b0 0x7f13b00c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.703+0000 7f13b5225700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b00cf040 0x7f13b00b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.703+0000 7f13b5225700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13b00d1ab0 0x7f13b006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.703+0000 7f13b5225700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b00c9320 con 0x7f13b00cf040 2023-11-26T07:17:53.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.703+0000 7f13b5225700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b00b7670 con 0x7f13b00af3b0 2023-11-26T07:17:53.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.703+0000 7f13b5225700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b00b7950 con 0x7f13b00d1ab0 2023-11-26T07:17:53.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b00cf040 0x7f13b00b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b00cf040 0x7f13b00b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42588/0 (socket says 172.21.15.83:42588) 2023-11-26T07:17:53.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a659c700 1 -- 172.21.15.83:0/2597260004 learned_addr learned my addr 172.21.15.83:0/2597260004 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:53.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13aed9d700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b00af3b0 0x7f13b00c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13af59e700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13b00d1ab0 0x7f13b006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a659c700 1 -- 172.21.15.83:0/2597260004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13b00d1ab0 msgr2=0x7f13b006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a659c700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13b00d1ab0 0x7f13b006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a659c700 1 -- 172.21.15.83:0/2597260004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b00af3b0 msgr2=0x7f13b00c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a659c700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13b00af3b0 0x7f13b00c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a659c700 1 -- 172.21.15.83:0/2597260004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13980094a0 con 0x7f13b00cf040 2023-11-26T07:17:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a659c700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b00cf040 0x7f13b00b49b0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f139800ebe0 tx=0x7f139800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a7fff700 1 -- 172.21.15.83:0/2597260004 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139800c850 con 0x7f13b00cf040 2023-11-26T07:17:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a7fff700 1 -- 172.21.15.83:0/2597260004 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139800c9b0 con 0x7f13b00cf040 2023-11-26T07:17:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13b5225700 1 -- 172.21.15.83:0/2597260004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a80094a0 con 0x7f13b00cf040 2023-11-26T07:17:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13a7fff700 1 -- 172.21.15.83:0/2597260004 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139800cd40 con 0x7f13b00cf040 2023-11-26T07:17:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.704+0000 7f13b5225700 1 -- 172.21.15.83:0/2597260004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b00c97e0 con 0x7f13b00cf040 2023-11-26T07:17:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.705+0000 7f13b5225700 1 -- 172.21.15.83:0/2597260004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13b00cf7b0 con 0x7f13b00cf040 2023-11-26T07:17:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.706+0000 7f13a7fff700 1 -- 172.21.15.83:0/2597260004 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f1398013470 con 0x7f13b00cf040 2023-11-26T07:17:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.706+0000 7f13a7fff700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f139006b1a0 0x7f139006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.706+0000 7f13a7fff700 1 -- 172.21.15.83:0/2597260004 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1398089b30 con 0x7f13b00cf040 2023-11-26T07:17:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.706+0000 7f13a7fff700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] conn(0x7f1390071000 0x7f1390073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.706+0000 7f13aed9d700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f139006b1a0 0x7f139006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.706+0000 7f13a7fff700 1 -- 172.21.15.83:0/2597260004 --> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1390073af0 con 0x7f1390071000 2023-11-26T07:17:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.706+0000 7f13a7fff700 1 -- 172.21.15.83:0/2597260004 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1398089f20 con 0x7f13b00cf040 2023-11-26T07:17:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.706+0000 7f13af59e700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] conn(0x7f1390071000 0x7f1390073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.707+0000 7f13af59e700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] conn(0x7f1390071000 0x7f1390073420 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.707+0000 7f13aed9d700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f139006b1a0 0x7f139006d660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13a0005c90 tx=0x7f13a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.707+0000 7f13a7fff700 1 -- 172.21.15.83:0/2597260004 <== osd.1 v2:172.21.15.83:6808/438950548 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1390073af0 con 0x7f1390071000 2023-11-26T07:17:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.722+0000 7f13b5225700 1 -- 172.21.15.83:0/2597260004 --> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f13b00cf420 con 0x7f1390071000 2023-11-26T07:17:53.724 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.722+0000 7f13a7fff700 1 -- 172.21.15.83:0/2597260004 <== osd.1 v2:172.21.15.83:6808/438950548 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f13a8011b00 con 0x7f1390071000 2023-11-26T07:17:53.724 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:53.724 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:17:53.724 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:53.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.722+0000 7f13b5225700 1 -- 172.21.15.83:0/2597260004 >> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] conn(0x7f1390071000 msgr2=0x7f1390073420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.725 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.722+0000 7f13b5225700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.83:6808/438950548,v1:172.21.15.83:6809/438950548] conn(0x7f1390071000 0x7f1390073420 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.723+0000 7f13b5225700 1 -- 172.21.15.83:0/2597260004 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f139006b1a0 msgr2=0x7f139006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.723+0000 7f13b5225700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f139006b1a0 0x7f139006d660 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13a0005c90 tx=0x7f13a0009040 comp rx=0 tx=0).stop 2023-11-26T07:17:53.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.723+0000 7f13b5225700 1 -- 172.21.15.83:0/2597260004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b00cf040 msgr2=0x7f13b00b49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.723+0000 7f13b5225700 1 --2- 172.21.15.83:0/2597260004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13b00cf040 0x7f13b00b49b0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f139800ebe0 tx=0x7f139800eef0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.723+0000 7f13af59e700 1 -- 172.21.15.83:0/2597260004 reap_dead start 2023-11-26T07:17:53.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.723+0000 7f13b5225700 1 -- 172.21.15.83:0/2597260004 shutdown_connections 2023-11-26T07:17:53.726 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.723+0000 7f13b5225700 1 -- 172.21.15.83:0/2597260004 >> 172.21.15.83:0/2597260004 conn(0x7f13b009aa00 msgr2=0x7f13b00cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:53.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.723+0000 7f13b5225700 1 -- 172.21.15.83:0/2597260004 shutdown_connections 2023-11-26T07:17:53.727 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.723+0000 7f13b5225700 1 -- 172.21.15.83:0/2597260004 wait complete. 2023-11-26T07:17:53.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-26T07:17:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.855+0000 7f72504bb700 1 Processor -- start 2023-11-26T07:17:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.855+0000 7f72504bb700 1 -- start start 2023-11-26T07:17:53.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.856+0000 7f72504bb700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72481446a0 0x7f72480b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.856+0000 7f72504bb700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72480af950 con 0x7f724806c710 2023-11-26T07:17:53.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.856+0000 7f72504bb700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72480afc30 con 0x7f72480adf00 2023-11-26T07:17:53.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.856+0000 7f72504bb700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7248144a80 con 0x7f72481446a0 2023-11-26T07:17:53.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.856+0000 7f724ea58700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f724806c710 0x7f72480b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34408/0 (socket says 172.21.15.83:34408) 2023-11-26T07:17:53.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724ea58700 1 -- 172.21.15.83:0/230917636 learned_addr learned my addr 172.21.15.83:0/230917636 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:53.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724e257700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72481446a0 0x7f72480b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.861 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153843536 0 0) 0x7f72480af950 con 0x7f724806c710 2023-11-26T07:17:53.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72340035b0 con 0x7f724806c710 2023-11-26T07:17:53.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2547364748 0 0) 0x7f72340035b0 con 0x7f724806c710 2023-11-26T07:17:53.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72481446a0 msgr2=0x7f72480b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72481446a0 0x7f72480b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 >> v1:172.21.15.203:6789/0 conn(0x7f72480adf00 legacy=0x7f724806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480aff10 con 0x7f724806c710 2023-11-26T07:17:53.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f72400019f0 con 0x7f724806c710 2023-11-26T07:17:53.862 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 >> v1:172.21.15.83:6789/0 conn(0x7f724806c710 legacy=0x7f72480b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72481446a0 0x7f7234003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7234003f70 0x7f7234004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7234004d90 0x7f7234005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7248144a80 con 0x7f7234004d90 2023-11-26T07:17:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723400ba70 con 0x7f7234003f70 2023-11-26T07:17:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723400bc30 con 0x7f72481446a0 2023-11-26T07:17:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724e257700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72481446a0 0x7f7234003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724ea58700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7234004d90 0x7f7234005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.857+0000 7f724da56700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7234003f70 0x7f7234004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f724ea58700 1 -- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72481446a0 msgr2=0x7f7234003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f724ea58700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72481446a0 0x7f7234003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f724ea58700 1 -- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7234003f70 msgr2=0x7f7234004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f724ea58700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7234003f70 0x7f7234004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f724ea58700 1 -- 172.21.15.83:0/230917636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72340035b0 con 0x7f7234004d90 2023-11-26T07:17:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f724e257700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72481446a0 0x7f7234003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f724ea58700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7234004d90 0x7f7234005210 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f724000c740 tx=0x7f7234005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fdd3c26cbf34ff server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724000c230 con 0x7f7234004d90 2023-11-26T07:17:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240014640 con 0x7f7234004d90 2023-11-26T07:17:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f724d255700 1 -- 172.21.15.83:0/230917636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72400149f0 con 0x7f7234004d90 2023-11-26T07:17:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f72504bb700 1 -- 172.21.15.83:0/230917636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7234004d90 msgr2=0x7f7234005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f72504bb700 1 --2- 172.21.15.83:0/230917636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7234004d90 0x7f7234005210 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f724000c740 tx=0x7f7234005880 comp rx=0 tx=0).stop 2023-11-26T07:17:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f724ea58700 1 -- 172.21.15.83:0/230917636 reap_dead start 2023-11-26T07:17:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.858+0000 7f72504bb700 1 -- 172.21.15.83:0/230917636 shutdown_connections 2023-11-26T07:17:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.859+0000 7f72504bb700 1 -- 172.21.15.83:0/230917636 >> 172.21.15.83:0/230917636 conn(0x7f7248071b90 msgr2=0x7f7248071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.859+0000 7f72504bb700 1 -- 172.21.15.83:0/230917636 shutdown_connections 2023-11-26T07:17:53.866 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.859+0000 7f72504bb700 1 -- 172.21.15.83:0/230917636 wait complete. 2023-11-26T07:17:53.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.860+0000 7f72504bb700 1 Processor -- start 2023-11-26T07:17:53.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.860+0000 7f72504bb700 1 -- start start 2023-11-26T07:17:53.868 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.860+0000 7f72504bb700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7234004d90 0x7f724813ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.860+0000 7f72504bb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72481f4550 0x7f72480cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.860+0000 7f72504bb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72481f6260 0x7f72481f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.860+0000 7f72504bb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72480caa40 con 0x7f72481f6260 2023-11-26T07:17:53.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.860+0000 7f72504bb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72480cccc0 con 0x7f72481f4550 2023-11-26T07:17:53.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f72504bb700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72481f49d0 con 0x7f7234004d90 2023-11-26T07:17:53.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f724ea58700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72481f6260 0x7f72481f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.869 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f724ea58700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72481f6260 0x7f72481f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42612/0 (socket says 172.21.15.83:42612) 2023-11-26T07:17:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f724ea58700 1 -- 172.21.15.83:0/4139116286 learned_addr learned my addr 172.21.15.83:0/4139116286 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f724da56700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72481f4550 0x7f72480cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f724e257700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7234004d90 0x7f724813ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f724ea58700 1 -- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7234004d90 msgr2=0x7f724813ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f724ea58700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7234004d90 0x7f724813ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f724ea58700 1 -- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72481f4550 msgr2=0x7f72480cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f724ea58700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72481f4550 0x7f72480cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f724ea58700 1 -- 172.21.15.83:0/4139116286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480ccef0 con 0x7f72481f6260 2023-11-26T07:17:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f724ea58700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72481f6260 0x7f72481f6a60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f7240015000 tx=0x7f7240001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f723effd700 1 -- 172.21.15.83:0/4139116286 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72400011c0 con 0x7f72481f6260 2023-11-26T07:17:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f723effd700 1 -- 172.21.15.83:0/4139116286 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240022070 con 0x7f72481f6260 2023-11-26T07:17:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f72504bb700 1 -- 172.21.15.83:0/4139116286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72480ca0a0 con 0x7f72481f6260 2023-11-26T07:17:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f723effd700 1 -- 172.21.15.83:0/4139116286 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240006110 con 0x7f72481f6260 2023-11-26T07:17:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.861+0000 7f72504bb700 1 -- 172.21.15.83:0/4139116286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72480ca660 con 0x7f72481f6260 2023-11-26T07:17:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.862+0000 7f72504bb700 1 -- 172.21.15.83:0/4139116286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72480753f0 con 0x7f72481f6260 2023-11-26T07:17:53.871 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.863+0000 7f723effd700 1 -- 172.21.15.83:0/4139116286 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f7240006480 con 0x7f72481f6260 2023-11-26T07:17:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.863+0000 7f723effd700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f722c06b1a0 0x7f722c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.863+0000 7f723effd700 1 -- 172.21.15.83:0/4139116286 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f724008b660 con 0x7f72481f6260 2023-11-26T07:17:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.863+0000 7f724e257700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f722c06b1a0 0x7f722c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.864+0000 7f724e257700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f722c06b1a0 0x7f722c06d660 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7238009c60 tx=0x7f7238009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:53.868+0000 7f723effd700 1 -- 172.21.15.83:0/4139116286 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7240056870 con 0x7f72481f6260 2023-11-26T07:17:54.055 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.054+0000 7f72504bb700 1 -- 172.21.15.83:0/4139116286 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f72480b04d0 con 0x7f722c06b1a0 2023-11-26T07:17:54.056 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.055+0000 7f723effd700 1 -- 172.21.15.83:0/4139116286 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7238019070 con 0x7f722c06b1a0 2023-11-26T07:17:54.058 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 -- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f722c06b1a0 msgr2=0x7f722c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f722c06b1a0 0x7f722c06d660 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7238009c60 tx=0x7f7238009210 comp rx=0 tx=0).stop 2023-11-26T07:17:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 -- 172.21.15.83:0/4139116286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72481f6260 msgr2=0x7f72481f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72481f6260 0x7f72481f6a60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f7240015000 tx=0x7f7240001d80 comp rx=0 tx=0).stop 2023-11-26T07:17:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 -- 172.21.15.83:0/4139116286 shutdown_connections 2023-11-26T07:17:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f722c06b1a0 0x7f722c06d660 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72481f6260 0x7f72481f6a60 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7234004d90 0x7f724813ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 --2- 172.21.15.83:0/4139116286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72481f4550 0x7f72480cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 -- 172.21.15.83:0/4139116286 >> 172.21.15.83:0/4139116286 conn(0x7f7248071b90 msgr2=0x7f724806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 -- 172.21.15.83:0/4139116286 shutdown_connections 2023-11-26T07:17:54.060 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.057+0000 7f72504bb700 1 -- 172.21.15.83:0/4139116286 wait complete. 2023-11-26T07:17:54.060 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 1 to deep-scrub 2023-11-26T07:17:54.070 INFO:tasks.ceph:Scrubbing osd.2 2023-11-26T07:17:54.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:17:54.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.193+0000 7ff22c0cf700 1 Processor -- start 2023-11-26T07:17:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.193+0000 7ff22c0cf700 1 -- start start 2023-11-26T07:17:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22c0cf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2240b01e0 0x7ff22406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22c0cf700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff2240746a0 0x7ff2240b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22c0cf700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2240b5e40 con 0x7ff2240b59c0 2023-11-26T07:17:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22c0cf700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2241470a0 con 0x7ff2240746a0 2023-11-26T07:17:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22c0cf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2240b05c0 con 0x7ff2240b01e0 2023-11-26T07:17:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22a66c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff2240746a0 0x7ff2240b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22a66c700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff2240746a0 0x7ff2240b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42620/0 (socket says 172.21.15.83:42620) 2023-11-26T07:17:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22a66c700 1 -- 172.21.15.83:0/1781694833 learned_addr learned my addr 172.21.15.83:0/1781694833 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff229e6b700 1 --2- 172.21.15.83:0/1781694833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2240b01e0 0x7ff22406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22a66c700 1 -- 172.21.15.83:0/1781694833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2240b01e0 msgr2=0x7ff22406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22a66c700 1 --2- 172.21.15.83:0/1781694833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2240b01e0 0x7ff22406e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22a66c700 1 -- 172.21.15.83:0/1781694833 >> v1:172.21.15.203:6789/0 conn(0x7ff2240b59c0 legacy=0x7ff224073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.194+0000 7ff22a66c700 1 -- 172.21.15.83:0/1781694833 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2240b4250 con 0x7ff2240746a0 2023-11-26T07:17:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff22a66c700 1 --2- 172.21.15.83:0/1781694833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2240746a0 0x7ff2240b3b50 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009e50 tx=0x7ff21c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea1733878a3a617 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff228e69700 1 -- 172.21.15.83:0/1781694833 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00e070 con 0x7ff2240746a0 2023-11-26T07:17:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff228e69700 1 -- 172.21.15.83:0/1781694833 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c00b9c0 con 0x7ff2240746a0 2023-11-26T07:17:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff228e69700 1 -- 172.21.15.83:0/1781694833 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00bd70 con 0x7ff2240746a0 2023-11-26T07:17:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff22c0cf700 1 -- 172.21.15.83:0/1781694833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2240746a0 msgr2=0x7ff2240b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff22c0cf700 1 --2- 172.21.15.83:0/1781694833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2240746a0 0x7ff2240b3b50 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009e50 tx=0x7ff21c00b330 comp rx=0 tx=0).stop 2023-11-26T07:17:54.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff22c0cf700 1 -- 172.21.15.83:0/1781694833 shutdown_connections 2023-11-26T07:17:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff22c0cf700 1 --2- 172.21.15.83:0/1781694833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2240746a0 0x7ff2240b3b50 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff22c0cf700 1 --2- 172.21.15.83:0/1781694833 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2240b01e0 0x7ff22406e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff22c0cf700 1 -- 172.21.15.83:0/1781694833 >> 172.21.15.83:0/1781694833 conn(0x7ff2240d0b40 msgr2=0x7ff2240d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff22c0cf700 1 -- 172.21.15.83:0/1781694833 shutdown_connections 2023-11-26T07:17:54.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.195+0000 7ff22c0cf700 1 -- 172.21.15.83:0/1781694833 wait complete. 2023-11-26T07:17:54.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.196+0000 7ff22c0cf700 1 Processor -- start 2023-11-26T07:17:54.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff22c0cf700 1 -- start start 2023-11-26T07:17:54.203 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff22c0cf700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2240746a0 0x7ff224142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff22c0cf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2240b01e0 0x7ff2240cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff22c0cf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2240b59c0 0x7ff224144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff22c0cf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2241f79d0 con 0x7ff2240b59c0 2023-11-26T07:17:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff22c0cf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2241f7cb0 con 0x7ff2240b01e0 2023-11-26T07:17:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff22c0cf700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2241f7f90 con 0x7ff2240746a0 2023-11-26T07:17:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff22a66c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2240b59c0 0x7ff224144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff22a66c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2240b59c0 0x7ff224144260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42626/0 (socket says 172.21.15.83:42626) 2023-11-26T07:17:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff22a66c700 1 -- 172.21.15.83:0/3835701499 learned_addr learned my addr 172.21.15.83:0/3835701499 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff22966a700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2240b01e0 0x7ff2240cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.197+0000 7ff229e6b700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2240746a0 0x7ff224142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff22a66c700 1 -- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2240746a0 msgr2=0x7ff224142510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff22a66c700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2240746a0 0x7ff224142510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff22a66c700 1 -- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2240b01e0 msgr2=0x7ff2240cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff22a66c700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2240b01e0 0x7ff2240cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff22a66c700 1 -- 172.21.15.83:0/3835701499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c009b30 con 0x7ff2240b59c0 2023-11-26T07:17:54.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff229e6b700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2240746a0 0x7ff224142510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:54.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff22966a700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2240b01e0 0x7ff2240cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:54.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff22a66c700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2240b59c0 0x7ff224144260 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7ff224072d10 tx=0x7ff21c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff21a7fc700 1 -- 172.21.15.83:0/3835701499 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c004960 con 0x7ff2240b59c0 2023-11-26T07:17:54.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff21a7fc700 1 -- 172.21.15.83:0/3835701499 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c0092e0 con 0x7ff2240b59c0 2023-11-26T07:17:54.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff22c0cf700 1 -- 172.21.15.83:0/3835701499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2241f8270 con 0x7ff2240b59c0 2023-11-26T07:17:54.206 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff21a7fc700 1 -- 172.21.15.83:0/3835701499 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c01c660 con 0x7ff2240b59c0 2023-11-26T07:17:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.198+0000 7ff22c0cf700 1 -- 172.21.15.83:0/3835701499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2240cffd0 con 0x7ff2240b59c0 2023-11-26T07:17:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.199+0000 7ff22c0cf700 1 -- 172.21.15.83:0/3835701499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff2240686e0 con 0x7ff2240b59c0 2023-11-26T07:17:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.199+0000 7ff21a7fc700 1 -- 172.21.15.83:0/3835701499 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7ff21c01c7c0 con 0x7ff2240b59c0 2023-11-26T07:17:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.200+0000 7ff21a7fc700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff20806b1a0 0x7ff20806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.200+0000 7ff21a7fc700 1 -- 172.21.15.83:0/3835701499 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff21c08b560 con 0x7ff2240b59c0 2023-11-26T07:17:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.200+0000 7ff21a7fc700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] conn(0x7ff208070e90 0x7ff2080732b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.200+0000 7ff21a7fc700 1 -- 172.21.15.83:0/3835701499 --> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff208073980 con 0x7ff208070e90 2023-11-26T07:17:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.200+0000 7ff21a7fc700 1 -- 172.21.15.83:0/3835701499 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff21c08dba0 con 0x7ff2240b59c0 2023-11-26T07:17:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.200+0000 7ff229e6b700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff20806b1a0 0x7ff20806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.200+0000 7ff22966a700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] conn(0x7ff208070e90 0x7ff2080732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.200+0000 7ff22966a700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] conn(0x7ff208070e90 0x7ff2080732b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.200+0000 7ff229e6b700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff20806b1a0 0x7ff20806d660 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff2140099f0 tx=0x7ff214008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.201+0000 7ff21a7fc700 1 -- 172.21.15.83:0/3835701499 <== osd.2 v2:172.21.15.83:6818/913771869 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff208073980 con 0x7ff208070e90 2023-11-26T07:17:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22c0cf700 1 -- 172.21.15.83:0/3835701499 --> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff22406e1c0 con 0x7ff208070e90 2023-11-26T07:17:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff21a7fc700 1 -- 172.21.15.83:0/3835701499 <== osd.2 v2:172.21.15.83:6818/913771869 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff2200050e0 con 0x7ff208070e90 2023-11-26T07:17:54.218 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:54.218 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:17:54.218 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:54.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22c0cf700 1 -- 172.21.15.83:0/3835701499 >> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] conn(0x7ff208070e90 msgr2=0x7ff2080732b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22c0cf700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.83:6818/913771869,v1:172.21.15.83:6820/913771869] conn(0x7ff208070e90 0x7ff2080732b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22c0cf700 1 -- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff20806b1a0 msgr2=0x7ff20806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22c0cf700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7ff20806b1a0 0x7ff20806d660 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff2140099f0 tx=0x7ff214008040 comp rx=0 tx=0).stop 2023-11-26T07:17:54.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22c0cf700 1 -- 172.21.15.83:0/3835701499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2240b59c0 msgr2=0x7ff224144260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22c0cf700 1 --2- 172.21.15.83:0/3835701499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2240b59c0 0x7ff224144260 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7ff224072d10 tx=0x7ff21c004ad0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22a66c700 1 -- 172.21.15.83:0/3835701499 reap_dead start 2023-11-26T07:17:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22c0cf700 1 -- 172.21.15.83:0/3835701499 shutdown_connections 2023-11-26T07:17:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22c0cf700 1 -- 172.21.15.83:0/3835701499 >> 172.21.15.83:0/3835701499 conn(0x7ff2240d0b40 msgr2=0x7ff224073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22c0cf700 1 -- 172.21.15.83:0/3835701499 shutdown_connections 2023-11-26T07:17:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.216+0000 7ff22c0cf700 1 -- 172.21.15.83:0/3835701499 wait complete. 2023-11-26T07:17:54.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-26T07:17:54.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.349+0000 7f6343129700 1 Processor -- start 2023-11-26T07:17:54.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.349+0000 7f6343129700 1 -- start start 2023-11-26T07:17:54.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f6343129700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f633c0adf00 0x7f633c06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f6343129700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f633c06c540 0x7f633c0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f6343129700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c0af950 con 0x7f633c1446a0 2023-11-26T07:17:54.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f6343129700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633c144a80 con 0x7f633c0adf00 2023-11-26T07:17:54.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f6343129700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c06c9c0 con 0x7f633c06c540 2023-11-26T07:17:54.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f6340ec5700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f633c1446a0 0x7f633c0b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34440/0 (socket says 172.21.15.83:34440) 2023-11-26T07:17:54.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f6340ec5700 1 -- 172.21.15.83:0/2545769559 learned_addr learned my addr 172.21.15.83:0/2545769559 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:54.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f633bfff700 1 --2- 172.21.15.83:0/2545769559 >> v2:172.21.15.203:3300/0 conn(0x7f633c0adf00 0x7f633c06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f63416c6700 1 --2- 172.21.15.83:0/2545769559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f633c06c540 0x7f633c0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f633b7fe700 1 -- 172.21.15.83:0/2545769559 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732678209 0 0) 0x7f633c0af950 con 0x7f633c1446a0 2023-11-26T07:17:54.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f633b7fe700 1 -- 172.21.15.83:0/2545769559 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f632c0035b0 con 0x7f633c1446a0 2023-11-26T07:17:54.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f633bfff700 1 -- 172.21.15.83:0/2545769559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f633c06c540 msgr2=0x7f633c0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:54.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f633bfff700 1 --2- 172.21.15.83:0/2545769559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f633c06c540 0x7f633c0b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f633bfff700 1 -- 172.21.15.83:0/2545769559 >> v1:172.21.15.83:6789/0 conn(0x7f633c1446a0 legacy=0x7f633c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f633b7fe700 1 -- 172.21.15.83:0/2545769559 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 615433404 0 0) 0x7f632c0035b0 con 0x7f633c1446a0 2023-11-26T07:17:54.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.350+0000 7f633bfff700 1 -- 172.21.15.83:0/2545769559 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0afc30 con 0x7f633c0adf00 2023-11-26T07:17:54.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.351+0000 7f633bfff700 1 --2- 172.21.15.83:0/2545769559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f633c0adf00 0x7f633c06c000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6328009840 tx=0x7f6328009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b2c18beeb6941bc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.351+0000 7f633b7fe700 1 -- 172.21.15.83:0/2545769559 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330001670 con 0x7f633c0adf00 2023-11-26T07:17:54.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.351+0000 7f633b7fe700 1 -- 172.21.15.83:0/2545769559 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6328004030 con 0x7f633c0adf00 2023-11-26T07:17:54.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.351+0000 7f633b7fe700 1 -- 172.21.15.83:0/2545769559 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280043a0 con 0x7f633c0adf00 2023-11-26T07:17:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.351+0000 7f6343129700 1 -- 172.21.15.83:0/2545769559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f633c0adf00 msgr2=0x7f633c06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.358 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.351+0000 7f6343129700 1 --2- 172.21.15.83:0/2545769559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f633c0adf00 0x7f633c06c000 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6328009840 tx=0x7f6328009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.351+0000 7f6343129700 1 -- 172.21.15.83:0/2545769559 shutdown_connections 2023-11-26T07:17:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.351+0000 7f6343129700 1 --2- 172.21.15.83:0/2545769559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f633c06c540 0x7f633c0b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.351+0000 7f6343129700 1 --2- 172.21.15.83:0/2545769559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f633c0adf00 0x7f633c06c000 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.351+0000 7f6343129700 1 -- 172.21.15.83:0/2545769559 >> 172.21.15.83:0/2545769559 conn(0x7f633c071b90 msgr2=0x7f633c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.352+0000 7f6343129700 1 -- 172.21.15.83:0/2545769559 shutdown_connections 2023-11-26T07:17:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.352+0000 7f6343129700 1 -- 172.21.15.83:0/2545769559 wait complete. 2023-11-26T07:17:54.359 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.353+0000 7f6343129700 1 Processor -- start 2023-11-26T07:17:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.353+0000 7f6343129700 1 -- start start 2023-11-26T07:17:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f6343129700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f633c06c540 0x7f633c1f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f6343129700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f633c0adf00 0x7f633c1392b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f6343129700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f633c1446a0 0x7f633c1397f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f6343129700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c13aa80 con 0x7f633c1446a0 2023-11-26T07:17:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f6343129700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c139e70 con 0x7f633c06c540 2023-11-26T07:17:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f6343129700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c13a150 con 0x7f633c0adf00 2023-11-26T07:17:54.360 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f63416c6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f633c1446a0 0x7f633c1397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f63416c6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f633c1446a0 0x7f633c1397f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42642/0 (socket says 172.21.15.83:42642) 2023-11-26T07:17:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f63416c6700 1 -- 172.21.15.83:0/3330412364 learned_addr learned my addr 172.21.15.83:0/3330412364 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f6340ec5700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f633c06c540 0x7f633c1f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f633bfff700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f633c0adf00 0x7f633c1392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.354+0000 7f63416c6700 1 -- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f633c0adf00 msgr2=0x7f633c1392b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.355+0000 7f63416c6700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f633c0adf00 0x7f633c1392b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.355+0000 7f63416c6700 1 -- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f633c06c540 msgr2=0x7f633c1f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.361 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.355+0000 7f63416c6700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f633c06c540 0x7f633c1f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.355+0000 7f63416c6700 1 -- 172.21.15.83:0/3330412364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280094a0 con 0x7f633c1446a0 2023-11-26T07:17:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.355+0000 7f6340ec5700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f633c06c540 0x7f633c1f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.355+0000 7f63416c6700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f633c1446a0 0x7f633c1397f0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f633400efc0 tx=0x7f633400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.355+0000 7f63397fa700 1 -- 172.21.15.83:0/3330412364 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633400cd40 con 0x7f633c1446a0 2023-11-26T07:17:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.355+0000 7f63397fa700 1 -- 172.21.15.83:0/3330412364 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633400f040 con 0x7f633c1446a0 2023-11-26T07:17:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.355+0000 7f6343129700 1 -- 172.21.15.83:0/3330412364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c13a430 con 0x7f633c1446a0 2023-11-26T07:17:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.355+0000 7f63397fa700 1 -- 172.21.15.83:0/3330412364 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340146c0 con 0x7f633c1446a0 2023-11-26T07:17:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.355+0000 7f6343129700 1 -- 172.21.15.83:0/3330412364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c1f4f90 con 0x7f633c1446a0 2023-11-26T07:17:54.362 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.356+0000 7f6343129700 1 -- 172.21.15.83:0/3330412364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c0753f0 con 0x7f633c1446a0 2023-11-26T07:17:54.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.356+0000 7f63397fa700 1 -- 172.21.15.83:0/3330412364 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f6334014820 con 0x7f633c1446a0 2023-11-26T07:17:54.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.356+0000 7f63397fa700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f632006b1a0 0x7f632006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.357+0000 7f63397fa700 1 -- 172.21.15.83:0/3330412364 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f633408a0f0 con 0x7f633c1446a0 2023-11-26T07:17:54.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.357+0000 7f6340ec5700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f632006b1a0 0x7f632006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.357+0000 7f6340ec5700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f632006b1a0 0x7f632006d660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f63300011e0 tx=0x7f6330007ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.363 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.361+0000 7f63397fa700 1 -- 172.21.15.83:0/3330412364 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6334055090 con 0x7f633c1446a0 2023-11-26T07:17:54.551 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.550+0000 7f6343129700 1 -- 172.21.15.83:0/3330412364 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f633c0b04d0 con 0x7f632006b1a0 2023-11-26T07:17:54.552 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.550+0000 7f63397fa700 1 -- 172.21.15.83:0/3330412364 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6330005de0 con 0x7f632006b1a0 2023-11-26T07:17:54.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.552+0000 7f6343129700 1 -- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f632006b1a0 msgr2=0x7f632006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.552+0000 7f6343129700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f632006b1a0 0x7f632006d660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f63300011e0 tx=0x7f6330007ac0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.552+0000 7f6343129700 1 -- 172.21.15.83:0/3330412364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f633c1446a0 msgr2=0x7f633c1397f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.552+0000 7f6343129700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f633c1446a0 0x7f633c1397f0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f633400efc0 tx=0x7f633400c3a0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.552+0000 7f6343129700 1 -- 172.21.15.83:0/3330412364 shutdown_connections 2023-11-26T07:17:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.552+0000 7f6343129700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f632006b1a0 0x7f632006d660 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.552+0000 7f6343129700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f633c1446a0 0x7f633c1397f0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.552+0000 7f6343129700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f633c06c540 0x7f633c1f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.553+0000 7f6343129700 1 --2- 172.21.15.83:0/3330412364 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f633c0adf00 0x7f633c1392b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.553+0000 7f6343129700 1 -- 172.21.15.83:0/3330412364 >> 172.21.15.83:0/3330412364 conn(0x7f633c071b90 msgr2=0x7f633c15e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.553+0000 7f6343129700 1 -- 172.21.15.83:0/3330412364 shutdown_connections 2023-11-26T07:17:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.553+0000 7f6343129700 1 -- 172.21.15.83:0/3330412364 wait complete. 2023-11-26T07:17:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 2 to deep-scrub 2023-11-26T07:17:54.566 INFO:tasks.ceph:Scrubbing osd.3 2023-11-26T07:17:54.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:17:54.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.689+0000 7f0328904700 1 Processor -- start 2023-11-26T07:17:54.691 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.689+0000 7f0328904700 1 -- start start 2023-11-26T07:17:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.690+0000 7f0328904700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03240cf040 0x7f032406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.690+0000 7f0328904700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f03240702e0 0x7f03240d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.690+0000 7f0328904700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03240d01c0 con 0x7f03240d1ff0 2023-11-26T07:17:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.690+0000 7f0328904700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0324070910 con 0x7f03240702e0 2023-11-26T07:17:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.690+0000 7f0328904700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0324070bf0 con 0x7f03240cf040 2023-11-26T07:17:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.690+0000 7f0321d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f03240702e0 0x7f03240d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.690+0000 7f0322d9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f03240d1ff0 0x7f032415f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34458/0 (socket says 172.21.15.83:34458) 2023-11-26T07:17:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.690+0000 7f0322d9d700 1 -- 172.21.15.83:0/2987429195 learned_addr learned my addr 172.21.15.83:0/2987429195 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.691+0000 7f032259c700 1 --2- 172.21.15.83:0/2987429195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03240cf040 0x7f032406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.691+0000 7f0321d9b700 1 -- 172.21.15.83:0/2987429195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03240cf040 msgr2=0x7f032406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.691+0000 7f0321d9b700 1 --2- 172.21.15.83:0/2987429195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03240cf040 0x7f032406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.691+0000 7f0321d9b700 1 -- 172.21.15.83:0/2987429195 >> v1:172.21.15.83:6789/0 conn(0x7f03240d1ff0 legacy=0x7f032415f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.691+0000 7f0321d9b700 1 -- 172.21.15.83:0/2987429195 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03240aebb0 con 0x7f03240702e0 2023-11-26T07:17:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.691+0000 7f032159a700 1 -- 172.21.15.83:0/2987429195 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850656236 0 0) 0x7f03240d01c0 con 0x7f03240d1ff0 2023-11-26T07:17:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.691+0000 7f0321d9b700 1 --2- 172.21.15.83:0/2987429195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03240702e0 0x7f03240d1ab0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f0314009840 tx=0x7f0314009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9077c0c52aff server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.691+0000 7f032159a700 1 -- 172.21.15.83:0/2987429195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314011830 con 0x7f03240702e0 2023-11-26T07:17:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.691+0000 7f032159a700 1 -- 172.21.15.83:0/2987429195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0314011990 con 0x7f03240702e0 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.691+0000 7f032159a700 1 -- 172.21.15.83:0/2987429195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314011d20 con 0x7f03240702e0 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.692+0000 7f0328904700 1 -- 172.21.15.83:0/2987429195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03240702e0 msgr2=0x7f03240d1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.692+0000 7f0328904700 1 --2- 172.21.15.83:0/2987429195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03240702e0 0x7f03240d1ab0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f0314009840 tx=0x7f0314009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.692+0000 7f0328904700 1 -- 172.21.15.83:0/2987429195 shutdown_connections 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.692+0000 7f0328904700 1 --2- 172.21.15.83:0/2987429195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03240cf040 0x7f032406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.692+0000 7f0328904700 1 --2- 172.21.15.83:0/2987429195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03240702e0 0x7f03240d1ab0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.692+0000 7f0328904700 1 -- 172.21.15.83:0/2987429195 >> 172.21.15.83:0/2987429195 conn(0x7f032409aa00 msgr2=0x7f032409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.692+0000 7f0328904700 1 -- 172.21.15.83:0/2987429195 shutdown_connections 2023-11-26T07:17:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.692+0000 7f0328904700 1 -- 172.21.15.83:0/2987429195 wait complete. 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.693+0000 7f0328904700 1 Processor -- start 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.693+0000 7f0328904700 1 -- start start 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0328904700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03240702e0 0x7f03240b7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0328904700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03240cf040 0x7f0324150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0328904700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03240d1ff0 0x7f03240c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0328904700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03240b6d10 con 0x7f03240d1ff0 2023-11-26T07:17:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0328904700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03240b7670 con 0x7f03240cf040 2023-11-26T07:17:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0328904700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03240b5db0 con 0x7f03240702e0 2023-11-26T07:17:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0322d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03240d1ff0 0x7f03240c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0322d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03240d1ff0 0x7f03240c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42656/0 (socket says 172.21.15.83:42656) 2023-11-26T07:17:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0322d9d700 1 -- 172.21.15.83:0/1909524890 learned_addr learned my addr 172.21.15.83:0/1909524890 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0321d9b700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03240cf040 0x7f0324150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f032259c700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03240702e0 0x7f03240b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0322d9d700 1 -- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03240702e0 msgr2=0x7f03240b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0322d9d700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03240702e0 0x7f03240b7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0322d9d700 1 -- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03240cf040 msgr2=0x7f0324150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0322d9d700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03240cf040 0x7f0324150080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0322d9d700 1 -- 172.21.15.83:0/1909524890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03140094a0 con 0x7f03240d1ff0 2023-11-26T07:17:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f032259c700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03240702e0 0x7f03240b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0322d9d700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03240d1ff0 0x7f03240c8cd0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f031c00cba0 tx=0x7f031c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.694+0000 7f0312ffd700 1 -- 172.21.15.83:0/1909524890 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c00ab80 con 0x7f03240d1ff0 2023-11-26T07:17:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.695+0000 7f0312ffd700 1 -- 172.21.15.83:0/1909524890 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c00ad00 con 0x7f03240d1ff0 2023-11-26T07:17:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.695+0000 7f0328904700 1 -- 172.21.15.83:0/1909524890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03240b6090 con 0x7f03240d1ff0 2023-11-26T07:17:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.695+0000 7f0312ffd700 1 -- 172.21.15.83:0/1909524890 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c007310 con 0x7f03240d1ff0 2023-11-26T07:17:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.695+0000 7f0328904700 1 -- 172.21.15.83:0/1909524890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03240b6650 con 0x7f03240d1ff0 2023-11-26T07:17:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.695+0000 7f0328904700 1 -- 172.21.15.83:0/1909524890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f03240cf420 con 0x7f03240d1ff0 2023-11-26T07:17:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.696+0000 7f0312ffd700 1 -- 172.21.15.83:0/1909524890 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f031c007470 con 0x7f03240d1ff0 2023-11-26T07:17:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.696+0000 7f0312ffd700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f030406b1a0 0x7f030406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.696+0000 7f0312ffd700 1 -- 172.21.15.83:0/1909524890 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f031c089b10 con 0x7f03240d1ff0 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.697+0000 7f0312ffd700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] conn(0x7f0304071000 0x7f0304073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.697+0000 7f0312ffd700 1 -- 172.21.15.83:0/1909524890 --> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0304073af0 con 0x7f0304071000 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.697+0000 7f0312ffd700 1 -- 172.21.15.83:0/1909524890 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f031c089f00 con 0x7f03240d1ff0 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.697+0000 7f032259c700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f030406b1a0 0x7f030406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.697+0000 7f0321d9b700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] conn(0x7f0304071000 0x7f0304073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.697+0000 7f0321d9b700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] conn(0x7f0304071000 0x7f0304073420 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.697+0000 7f032259c700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f030406b1a0 0x7f030406d660 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0318009a00 tx=0x7f0318008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.698+0000 7f0312ffd700 1 -- 172.21.15.83:0/1909524890 <== osd.3 v2:172.21.15.83:6816/1108190663 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0304073af0 con 0x7f0304071000 2023-11-26T07:17:54.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.712+0000 7f0328904700 1 -- 172.21.15.83:0/1909524890 --> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f03240d2440 con 0x7f0304071000 2023-11-26T07:17:54.714 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0312ffd700 1 -- 172.21.15.83:0/1909524890 <== osd.3 v2:172.21.15.83:6816/1108190663 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0314021040 con 0x7f0304071000 2023-11-26T07:17:54.714 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:54.715 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:17:54.715 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:54.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0328904700 1 -- 172.21.15.83:0/1909524890 >> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] conn(0x7f0304071000 msgr2=0x7f0304073420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0328904700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.83:6816/1108190663,v1:172.21.15.83:6817/1108190663] conn(0x7f0304071000 0x7f0304073420 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0328904700 1 -- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f030406b1a0 msgr2=0x7f030406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0328904700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f030406b1a0 0x7f030406d660 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0318009a00 tx=0x7f0318008040 comp rx=0 tx=0).stop 2023-11-26T07:17:54.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0328904700 1 -- 172.21.15.83:0/1909524890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03240d1ff0 msgr2=0x7f03240c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0328904700 1 --2- 172.21.15.83:0/1909524890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03240d1ff0 0x7f03240c8cd0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f031c00cba0 tx=0x7f031c00cf60 comp rx=0 tx=0).stop 2023-11-26T07:17:54.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0322d9d700 1 -- 172.21.15.83:0/1909524890 reap_dead start 2023-11-26T07:17:54.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0328904700 1 -- 172.21.15.83:0/1909524890 shutdown_connections 2023-11-26T07:17:54.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0328904700 1 -- 172.21.15.83:0/1909524890 >> 172.21.15.83:0/1909524890 conn(0x7f032409aa00 msgr2=0x7f03240cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0328904700 1 -- 172.21.15.83:0/1909524890 shutdown_connections 2023-11-26T07:17:54.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.713+0000 7f0328904700 1 -- 172.21.15.83:0/1909524890 wait complete. 2023-11-26T07:17:54.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-26T07:17:54.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.846+0000 7f6cac857700 1 Processor -- start 2023-11-26T07:17:54.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.846+0000 7f6cac857700 1 -- start start 2023-11-26T07:17:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.847+0000 7f6cac857700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6ca806c710 0x7f6ca80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.847+0000 7f6cac857700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca80ae430 con 0x7f6ca80af4d0 2023-11-26T07:17:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.847+0000 7f6cac857700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca80ae710 con 0x7f6ca815e080 2023-11-26T07:17:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.847+0000 7f6cac857700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca80af0b0 con 0x7f6ca806c710 2023-11-26T07:17:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.847+0000 7f6ca5d9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6ca806c710 0x7f6ca80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.847+0000 7f6ca6d9d700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f6ca80af4d0 0x7f6ca806e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34474/0 (socket says 172.21.15.83:34474) 2023-11-26T07:17:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.847+0000 7f6ca6d9d700 1 -- 172.21.15.83:0/4077736850 learned_addr learned my addr 172.21.15.83:0/4077736850 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.847+0000 7f6ca5d9b700 1 -- 172.21.15.83:0/4077736850 >> v1:172.21.15.203:6789/0 conn(0x7f6ca815e080 legacy=0x7f6ca806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.847+0000 7f6ca5d9b700 1 -- 172.21.15.83:0/4077736850 >> v1:172.21.15.83:6789/0 conn(0x7f6ca80af4d0 legacy=0x7f6ca806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.847+0000 7f6ca559a700 1 -- 172.21.15.83:0/4077736850 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330398410 0 0) 0x7f6ca80ae430 con 0x7f6ca80af4d0 2023-11-26T07:17:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.847+0000 7f6ca5d9b700 1 -- 172.21.15.83:0/4077736850 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca80ae9f0 con 0x7f6ca806c710 2023-11-26T07:17:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.848+0000 7f6ca5d9b700 1 --2- 172.21.15.83:0/4077736850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca806c710 0x7f6ca80b5050 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009840 tx=0x7f6c98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6451ec0eeba5e88 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.848+0000 7f6ca559a700 1 -- 172.21.15.83:0/4077736850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98004230 con 0x7f6ca806c710 2023-11-26T07:17:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.848+0000 7f6ca559a700 1 -- 172.21.15.83:0/4077736850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c98004390 con 0x7f6ca806c710 2023-11-26T07:17:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.848+0000 7f6ca559a700 1 -- 172.21.15.83:0/4077736850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980036a0 con 0x7f6ca806c710 2023-11-26T07:17:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.848+0000 7f6cac857700 1 -- 172.21.15.83:0/4077736850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca806c710 msgr2=0x7f6ca80b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.848+0000 7f6cac857700 1 --2- 172.21.15.83:0/4077736850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca806c710 0x7f6ca80b5050 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009840 tx=0x7f6c98009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.848+0000 7f6cac857700 1 -- 172.21.15.83:0/4077736850 shutdown_connections 2023-11-26T07:17:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.848+0000 7f6cac857700 1 --2- 172.21.15.83:0/4077736850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca806c710 0x7f6ca80b5050 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.848+0000 7f6cac857700 1 -- 172.21.15.83:0/4077736850 >> 172.21.15.83:0/4077736850 conn(0x7f6ca8071b90 msgr2=0x7f6ca8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.848+0000 7f6cac857700 1 -- 172.21.15.83:0/4077736850 shutdown_connections 2023-11-26T07:17:54.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.848+0000 7f6cac857700 1 -- 172.21.15.83:0/4077736850 wait complete. 2023-11-26T07:17:54.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.850+0000 7f6cac857700 1 Processor -- start 2023-11-26T07:17:54.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.850+0000 7f6cac857700 1 -- start start 2023-11-26T07:17:54.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.850+0000 7f6cac857700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca806c710 0x7f6ca80ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.850+0000 7f6cac857700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca80af4d0 0x7f6ca8147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.850+0000 7f6cac857700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca815e080 0x7f6ca8144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.850+0000 7f6cac857700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca80cea50 con 0x7f6ca80af4d0 2023-11-26T07:17:54.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.850+0000 7f6cac857700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca8147c80 con 0x7f6ca815e080 2023-11-26T07:17:54.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.850+0000 7f6cac857700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca80cde40 con 0x7f6ca806c710 2023-11-26T07:17:54.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca5d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca80af4d0 0x7f6ca8147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca5d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca80af4d0 0x7f6ca8147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42670/0 (socket says 172.21.15.83:42670) 2023-11-26T07:17:54.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca5d9b700 1 -- 172.21.15.83:0/3171387771 learned_addr learned my addr 172.21.15.83:0/3171387771 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:54.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca659c700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca806c710 0x7f6ca80ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca6d9d700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca815e080 0x7f6ca8144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca5d9b700 1 -- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca806c710 msgr2=0x7f6ca80ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca5d9b700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca806c710 0x7f6ca80ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca5d9b700 1 -- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca815e080 msgr2=0x7f6ca8144430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca5d9b700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca815e080 0x7f6ca8144430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca5d9b700 1 -- 172.21.15.83:0/3171387771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c980094a0 con 0x7f6ca80af4d0 2023-11-26T07:17:54.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca6d9d700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca815e080 0x7f6ca8144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:54.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6ca5d9b700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca80af4d0 0x7f6ca8147740 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f6c980090b0 tx=0x7f6c98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6c96ffd700 1 -- 172.21.15.83:0/3171387771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98011830 con 0x7f6ca80af4d0 2023-11-26T07:17:54.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6c96ffd700 1 -- 172.21.15.83:0/3171387771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9801e070 con 0x7f6ca80af4d0 2023-11-26T07:17:54.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6cac857700 1 -- 172.21.15.83:0/3171387771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca80ce120 con 0x7f6ca80af4d0 2023-11-26T07:17:54.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6c96ffd700 1 -- 172.21.15.83:0/3171387771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98011bb0 con 0x7f6ca80af4d0 2023-11-26T07:17:54.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.851+0000 7f6cac857700 1 -- 172.21.15.83:0/3171387771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca80ce6e0 con 0x7f6ca80af4d0 2023-11-26T07:17:54.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.852+0000 7f6cac857700 1 -- 172.21.15.83:0/3171387771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca80753f0 con 0x7f6ca80af4d0 2023-11-26T07:17:54.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.853+0000 7f6c96ffd700 1 -- 172.21.15.83:0/3171387771 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f6c98011d10 con 0x7f6ca80af4d0 2023-11-26T07:17:54.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.853+0000 7f6c96ffd700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c8806b1a0 0x7f6c8806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.853+0000 7f6c96ffd700 1 -- 172.21.15.83:0/3171387771 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6c9808b280 con 0x7f6ca80af4d0 2023-11-26T07:17:54.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.853+0000 7f6ca659c700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c8806b1a0 0x7f6c8806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.854+0000 7f6ca659c700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c8806b1a0 0x7f6c8806d660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009870 tx=0x7f6c9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:54.857+0000 7f6c96ffd700 1 -- 172.21.15.83:0/3171387771 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c98056470 con 0x7f6ca80af4d0 2023-11-26T07:17:55.049 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.048+0000 7f6cac857700 1 -- 172.21.15.83:0/3171387771 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ca80b05a0 con 0x7f6c8806b1a0 2023-11-26T07:17:55.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.049+0000 7f6c96ffd700 1 -- 172.21.15.83:0/3171387771 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6c9c014430 con 0x7f6c8806b1a0 2023-11-26T07:17:55.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 -- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c8806b1a0 msgr2=0x7f6c8806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.052 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c8806b1a0 0x7f6c8806d660 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009870 tx=0x7f6c9c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 -- 172.21.15.83:0/3171387771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca80af4d0 msgr2=0x7f6ca8147740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca80af4d0 0x7f6ca8147740 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f6c980090b0 tx=0x7f6c98004750 comp rx=0 tx=0).stop 2023-11-26T07:17:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 -- 172.21.15.83:0/3171387771 shutdown_connections 2023-11-26T07:17:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f6c8806b1a0 0x7f6c8806d660 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ca815e080 0x7f6ca8144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ca806c710 0x7f6ca80ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 --2- 172.21.15.83:0/3171387771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca80af4d0 0x7f6ca8147740 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.053 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 -- 172.21.15.83:0/3171387771 >> 172.21.15.83:0/3171387771 conn(0x7f6ca8071b90 msgr2=0x7f6ca806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 -- 172.21.15.83:0/3171387771 shutdown_connections 2023-11-26T07:17:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.051+0000 7f6cac857700 1 -- 172.21.15.83:0/3171387771 wait complete. 2023-11-26T07:17:55.054 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 3 to deep-scrub 2023-11-26T07:17:55.064 INFO:tasks.ceph:Scrubbing osd.4 2023-11-26T07:17:55.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:17:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.186+0000 7f0f24f82700 1 Processor -- start 2023-11-26T07:17:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.187+0000 7f0f24f82700 1 -- start start 2023-11-26T07:17:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.187+0000 7f0f24f82700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2015e020 0x7f0f200b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.187+0000 7f0f24f82700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f200b1720 0x7f0f200cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.187+0000 7f0f24f82700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f200b1ba0 con 0x7f0f200cf910 2023-11-26T07:17:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.187+0000 7f0f24f82700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f20161c80 con 0x7f0f200b1720 2023-11-26T07:17:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.187+0000 7f0f24f82700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20161f40 con 0x7f0f2015e020 2023-11-26T07:17:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1dd9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f200b1720 0x7f0f200cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1dd9b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0f200b1720 0x7f0f200cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42676/0 (socket says 172.21.15.83:42676) 2023-11-26T07:17:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1dd9b700 1 -- 172.21.15.83:0/3269792192 learned_addr learned my addr 172.21.15.83:0/3269792192 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1e59c700 1 --2- 172.21.15.83:0/3269792192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2015e020 0x7f0f200b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1dd9b700 1 -- 172.21.15.83:0/3269792192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2015e020 msgr2=0x7f0f200b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1dd9b700 1 --2- 172.21.15.83:0/3269792192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2015e020 0x7f0f200b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1dd9b700 1 -- 172.21.15.83:0/3269792192 >> v1:172.21.15.203:6789/0 conn(0x7f0f200cf910 legacy=0x7f0f2015e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1dd9b700 1 -- 172.21.15.83:0/3269792192 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f20162220 con 0x7f0f200b1720 2023-11-26T07:17:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1e59c700 1 --2- 172.21.15.83:0/3269792192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2015e020 0x7f0f200b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1dd9b700 1 --2- 172.21.15.83:0/3269792192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f200b1720 0x7f0f200cf3d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f0f100098f0 tx=0x7f0f10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9ab16b6b83064c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1d59a700 1 -- 172.21.15.83:0/3269792192 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1000b8a0 con 0x7f0f200b1720 2023-11-26T07:17:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1d59a700 1 -- 172.21.15.83:0/3269792192 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1000ba00 con 0x7f0f200b1720 2023-11-26T07:17:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.188+0000 7f0f1d59a700 1 -- 172.21.15.83:0/3269792192 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1000bd90 con 0x7f0f200b1720 2023-11-26T07:17:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.189+0000 7f0f24f82700 1 -- 172.21.15.83:0/3269792192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f200b1720 msgr2=0x7f0f200cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.189+0000 7f0f24f82700 1 --2- 172.21.15.83:0/3269792192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f200b1720 0x7f0f200cf3d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f0f100098f0 tx=0x7f0f10009c00 comp rx=0 tx=0).stop 2023-11-26T07:17:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.189+0000 7f0f24f82700 1 -- 172.21.15.83:0/3269792192 shutdown_connections 2023-11-26T07:17:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.189+0000 7f0f24f82700 1 --2- 172.21.15.83:0/3269792192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2015e020 0x7f0f200b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.189+0000 7f0f24f82700 1 --2- 172.21.15.83:0/3269792192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f200b1720 0x7f0f200cf3d0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.189+0000 7f0f24f82700 1 -- 172.21.15.83:0/3269792192 >> 172.21.15.83:0/3269792192 conn(0x7f0f20072710 msgr2=0x7f0f20072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.189+0000 7f0f24f82700 1 -- 172.21.15.83:0/3269792192 shutdown_connections 2023-11-26T07:17:55.194 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.189+0000 7f0f24f82700 1 -- 172.21.15.83:0/3269792192 wait complete. 2023-11-26T07:17:55.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.190+0000 7f0f24f82700 1 Processor -- start 2023-11-26T07:17:55.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.190+0000 7f0f24f82700 1 -- start start 2023-11-26T07:17:55.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f24f82700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f200b1720 0x7f0f2015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f24f82700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f200cf910 0x7f0f2015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f24f82700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f2015e020 0x7f0f2015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f24f82700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201608f0 con 0x7f0f200b1720 2023-11-26T07:17:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f24f82700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20160050 con 0x7f0f2015e020 2023-11-26T07:17:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f24f82700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20160330 con 0x7f0f200cf910 2023-11-26T07:17:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f200b1720 0x7f0f2015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f200b1720 0x7f0f2015a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42682/0 (socket says 172.21.15.83:42682) 2023-11-26T07:17:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1e59c700 1 -- 172.21.15.83:0/3130444237 learned_addr learned my addr 172.21.15.83:0/3130444237 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1ed9d700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f2015e020 0x7f0f2015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1dd9b700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f200cf910 0x7f0f2015f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1e59c700 1 -- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f200cf910 msgr2=0x7f0f2015f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1e59c700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f200cf910 0x7f0f2015f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1e59c700 1 -- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f2015e020 msgr2=0x7f0f2015fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1e59c700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f2015e020 0x7f0f2015fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1e59c700 1 -- 172.21.15.83:0/3130444237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f10009550 con 0x7f0f200b1720 2023-11-26T07:17:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1dd9b700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f200cf910 0x7f0f2015f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.191+0000 7f0f1ed9d700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f2015e020 0x7f0f2015fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.192+0000 7f0f1e59c700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f200b1720 0x7f0f2015a210 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400e810 tx=0x7f0f1400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.192+0000 7f0f0effd700 1 -- 172.21.15.83:0/3130444237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400c020 con 0x7f0f200b1720 2023-11-26T07:17:55.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.192+0000 7f0f0effd700 1 -- 172.21.15.83:0/3130444237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f14004250 con 0x7f0f200b1720 2023-11-26T07:17:55.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.192+0000 7f0f24f82700 1 -- 172.21.15.83:0/3130444237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f20160610 con 0x7f0f200b1720 2023-11-26T07:17:55.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.192+0000 7f0f0effd700 1 -- 172.21.15.83:0/3130444237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f140036a0 con 0x7f0f200b1720 2023-11-26T07:17:55.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.192+0000 7f0f24f82700 1 -- 172.21.15.83:0/3130444237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2015a750 con 0x7f0f200b1720 2023-11-26T07:17:55.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.192+0000 7f0f24f82700 1 -- 172.21.15.83:0/3130444237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f200d09c0 con 0x7f0f200b1720 2023-11-26T07:17:55.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.193+0000 7f0f0effd700 1 -- 172.21.15.83:0/3130444237 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f0f14003800 con 0x7f0f200b1720 2023-11-26T07:17:55.200 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.193+0000 7f0f0effd700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f0006b1a0 0x7f0f0006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.194+0000 7f0f0effd700 1 -- 172.21.15.83:0/3130444237 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0f140892c0 con 0x7f0f200b1720 2023-11-26T07:17:55.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.194+0000 7f0f0effd700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] conn(0x7f0f00071000 0x7f0f00073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.194+0000 7f0f1dd9b700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f0006b1a0 0x7f0f0006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.194+0000 7f0f0effd700 1 -- 172.21.15.83:0/3130444237 --> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f00073af0 con 0x7f0f00071000 2023-11-26T07:17:55.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.194+0000 7f0f0effd700 1 -- 172.21.15.83:0/3130444237 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f1408b900 con 0x7f0f200b1720 2023-11-26T07:17:55.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.194+0000 7f0f1ed9d700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] conn(0x7f0f00071000 0x7f0f00073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.201 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.194+0000 7f0f1dd9b700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f0006b1a0 0x7f0f0006d660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009c40 tx=0x7f0f10008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.194+0000 7f0f1ed9d700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] conn(0x7f0f00071000 0x7f0f00073420 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.202 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.195+0000 7f0f0effd700 1 -- 172.21.15.83:0/3130444237 <== osd.4 v2:172.21.15.203:6808/1845411432 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0f00073af0 con 0x7f0f00071000 2023-11-26T07:17:55.211 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.210+0000 7f0f24f82700 1 -- 172.21.15.83:0/3130444237 --> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0f201627e0 con 0x7f0f00071000 2023-11-26T07:17:55.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f0effd700 1 -- 172.21.15.83:0/3130444237 <== osd.4 v2:172.21.15.203:6808/1845411432 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0f18003a00 con 0x7f0f00071000 2023-11-26T07:17:55.212 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:55.212 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:17:55.212 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:55.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f24f82700 1 -- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] conn(0x7f0f00071000 msgr2=0x7f0f00073420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f24f82700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:6808/1845411432,v1:172.21.15.203:6809/1845411432] conn(0x7f0f00071000 0x7f0f00073420 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f24f82700 1 -- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f0006b1a0 msgr2=0x7f0f0006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f24f82700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f0f0006b1a0 0x7f0f0006d660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009c40 tx=0x7f0f10008030 comp rx=0 tx=0).stop 2023-11-26T07:17:55.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f24f82700 1 -- 172.21.15.83:0/3130444237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f200b1720 msgr2=0x7f0f2015a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f24f82700 1 --2- 172.21.15.83:0/3130444237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f200b1720 0x7f0f2015a210 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400e810 tx=0x7f0f1400ebd0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f1ed9d700 1 -- 172.21.15.83:0/3130444237 reap_dead start 2023-11-26T07:17:55.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f24f82700 1 -- 172.21.15.83:0/3130444237 shutdown_connections 2023-11-26T07:17:55.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f24f82700 1 -- 172.21.15.83:0/3130444237 >> 172.21.15.83:0/3130444237 conn(0x7f0f20072710 msgr2=0x7f0f200b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f24f82700 1 -- 172.21.15.83:0/3130444237 shutdown_connections 2023-11-26T07:17:55.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.211+0000 7f0f24f82700 1 -- 172.21.15.83:0/3130444237 wait complete. 2023-11-26T07:17:55.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-26T07:17:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.344+0000 7f77804b5700 1 Processor -- start 2023-11-26T07:17:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.344+0000 7f77804b5700 1 -- start start 2023-11-26T07:17:55.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f77804b5700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f777806c710 0x7f77780b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f77804b5700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77780ae430 con 0x7f77780af4d0 2023-11-26T07:17:55.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f77804b5700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77780ae710 con 0x7f777815e080 2023-11-26T07:17:55.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f77804b5700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77780af0b0 con 0x7f777806c710 2023-11-26T07:17:55.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f777da50700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f777806c710 0x7f77780b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f777ea52700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f77780af4d0 0x7f777806e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34504/0 (socket says 172.21.15.83:34504) 2023-11-26T07:17:55.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f777ea52700 1 -- 172.21.15.83:0/1242334480 learned_addr learned my addr 172.21.15.83:0/1242334480 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:55.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f777da50700 1 -- 172.21.15.83:0/1242334480 >> v1:172.21.15.203:6789/0 conn(0x7f777815e080 legacy=0x7f777806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f777da50700 1 -- 172.21.15.83:0/1242334480 >> v1:172.21.15.83:6789/0 conn(0x7f77780af4d0 legacy=0x7f777806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f777d24f700 1 -- 172.21.15.83:0/1242334480 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701549622 0 0) 0x7f77780ae430 con 0x7f77780af4d0 2023-11-26T07:17:55.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f777da50700 1 -- 172.21.15.83:0/1242334480 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77780ae9f0 con 0x7f777806c710 2023-11-26T07:17:55.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.345+0000 7f777da50700 1 --2- 172.21.15.83:0/1242334480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777806c710 0x7f77780b5050 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7768009840 tx=0x7f7768009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1945f20d99153bfa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.346+0000 7f777d24f700 1 -- 172.21.15.83:0/1242334480 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768011830 con 0x7f777806c710 2023-11-26T07:17:55.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.346+0000 7f777d24f700 1 -- 172.21.15.83:0/1242334480 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7768011990 con 0x7f777806c710 2023-11-26T07:17:55.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.346+0000 7f777d24f700 1 -- 172.21.15.83:0/1242334480 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768011d20 con 0x7f777806c710 2023-11-26T07:17:55.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.346+0000 7f77804b5700 1 -- 172.21.15.83:0/1242334480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777806c710 msgr2=0x7f77780b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.346+0000 7f77804b5700 1 --2- 172.21.15.83:0/1242334480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777806c710 0x7f77780b5050 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7768009840 tx=0x7f7768009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:55.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.346+0000 7f77804b5700 1 -- 172.21.15.83:0/1242334480 shutdown_connections 2023-11-26T07:17:55.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.346+0000 7f77804b5700 1 --2- 172.21.15.83:0/1242334480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777806c710 0x7f77780b5050 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.346+0000 7f77804b5700 1 -- 172.21.15.83:0/1242334480 >> 172.21.15.83:0/1242334480 conn(0x7f7778071b90 msgr2=0x7f7778071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.346+0000 7f77804b5700 1 -- 172.21.15.83:0/1242334480 shutdown_connections 2023-11-26T07:17:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.346+0000 7f77804b5700 1 -- 172.21.15.83:0/1242334480 wait complete. 2023-11-26T07:17:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.347+0000 7f77804b5700 1 Processor -- start 2023-11-26T07:17:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.347+0000 7f77804b5700 1 -- start start 2023-11-26T07:17:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f77804b5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777806c710 0x7f7778146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f77804b5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77780af4d0 0x7f7778146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f77804b5700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f777815e080 0x7f77781551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f77804b5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7778155c60 con 0x7f777806c710 2023-11-26T07:17:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f77804b5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77781532c0 con 0x7f77780af4d0 2023-11-26T07:17:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f77804b5700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777815f090 con 0x7f777815e080 2023-11-26T07:17:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f777e251700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777806c710 0x7f7778146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f777e251700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777806c710 0x7f7778146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42698/0 (socket says 172.21.15.83:42698) 2023-11-26T07:17:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f777e251700 1 -- 172.21.15.83:0/1123538587 learned_addr learned my addr 172.21.15.83:0/1123538587 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f777da50700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77780af4d0 0x7f7778146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f777ea52700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f777815e080 0x7f77781551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f777e251700 1 -- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f777815e080 msgr2=0x7f77781551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f777e251700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f777815e080 0x7f77781551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f777e251700 1 -- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77780af4d0 msgr2=0x7f7778146470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.348+0000 7f777e251700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77780af4d0 0x7f7778146470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.349+0000 7f777e251700 1 -- 172.21.15.83:0/1123538587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77680094a0 con 0x7f777806c710 2023-11-26T07:17:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.349+0000 7f777da50700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77780af4d0 0x7f7778146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.349+0000 7f777e251700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777806c710 0x7f7778146f30 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f777400c450 tx=0x7f777400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.349+0000 7f776effd700 1 -- 172.21.15.83:0/1123538587 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740052d0 con 0x7f777806c710 2023-11-26T07:17:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.349+0000 7f776effd700 1 -- 172.21.15.83:0/1123538587 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7774005430 con 0x7f777806c710 2023-11-26T07:17:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.349+0000 7f776effd700 1 -- 172.21.15.83:0/1123538587 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740038c0 con 0x7f777806c710 2023-11-26T07:17:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.349+0000 7f77804b5700 1 -- 172.21.15.83:0/1123538587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777815f370 con 0x7f777806c710 2023-11-26T07:17:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.349+0000 7f77804b5700 1 -- 172.21.15.83:0/1123538587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777815f930 con 0x7f777806c710 2023-11-26T07:17:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.350+0000 7f77804b5700 1 -- 172.21.15.83:0/1123538587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7778155ed0 con 0x7f777806c710 2023-11-26T07:17:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.350+0000 7f776effd700 1 -- 172.21.15.83:0/1123538587 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f7774003a20 con 0x7f777806c710 2023-11-26T07:17:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.351+0000 7f776effd700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f775c06b1a0 0x7f775c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.351+0000 7f776effd700 1 -- 172.21.15.83:0/1123538587 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7774088a80 con 0x7f777806c710 2023-11-26T07:17:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.351+0000 7f777da50700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f775c06b1a0 0x7f775c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.351+0000 7f777da50700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f775c06b1a0 0x7f775c06d660 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f77780690b0 tx=0x7f776801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.357 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.355+0000 7f776effd700 1 -- 172.21.15.83:0/1123538587 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f777408ae60 con 0x7f777806c710 2023-11-26T07:17:55.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.548+0000 7f77804b5700 1 -- 172.21.15.83:0/1123538587 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f77780b05a0 con 0x7f775c06b1a0 2023-11-26T07:17:55.550 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.549+0000 7f776effd700 1 -- 172.21.15.83:0/1123538587 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7768011cb0 con 0x7f775c06b1a0 2023-11-26T07:17:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 -- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f775c06b1a0 msgr2=0x7f775c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f775c06b1a0 0x7f775c06d660 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f77780690b0 tx=0x7f776801c040 comp rx=0 tx=0).stop 2023-11-26T07:17:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 -- 172.21.15.83:0/1123538587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777806c710 msgr2=0x7f7778146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777806c710 0x7f7778146f30 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f777400c450 tx=0x7f777400c810 comp rx=0 tx=0).stop 2023-11-26T07:17:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 -- 172.21.15.83:0/1123538587 shutdown_connections 2023-11-26T07:17:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f775c06b1a0 0x7f775c06d660 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f777815e080 0x7f77781551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777806c710 0x7f7778146f30 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 --2- 172.21.15.83:0/1123538587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f77780af4d0 0x7f7778146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 -- 172.21.15.83:0/1123538587 >> 172.21.15.83:0/1123538587 conn(0x7f7778071b90 msgr2=0x7f777806f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 -- 172.21.15.83:0/1123538587 shutdown_connections 2023-11-26T07:17:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.551+0000 7f77804b5700 1 -- 172.21.15.83:0/1123538587 wait complete. 2023-11-26T07:17:55.555 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 4 to deep-scrub 2023-11-26T07:17:55.565 INFO:tasks.ceph:Scrubbing osd.5 2023-11-26T07:17:55.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:17:55.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.688+0000 7f513ad24700 1 Processor -- start 2023-11-26T07:17:55.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.688+0000 7f513ad24700 1 -- start start 2023-11-26T07:17:55.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.689+0000 7f513ad24700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51340746a0 0x7f513406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.689+0000 7f513ad24700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51340b59c0 0x7f51340b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.689+0000 7f513ad24700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51340b5e50 con 0x7f51340b01e0 2023-11-26T07:17:55.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.689+0000 7f513ad24700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51341470a0 con 0x7f51340746a0 2023-11-26T07:17:55.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.689+0000 7f513ad24700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51340b05c0 con 0x7f51340b59c0 2023-11-26T07:17:55.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.689+0000 7f5138ac0700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f51340b01e0 0x7f5134073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34514/0 (socket says 172.21.15.83:34514) 2023-11-26T07:17:55.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.689+0000 7f5138ac0700 1 -- 172.21.15.83:0/331611328 learned_addr learned my addr 172.21.15.83:0/331611328 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:55.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.689+0000 7f5133fff700 1 --2- 172.21.15.83:0/331611328 >> v2:172.21.15.203:3300/0 conn(0x7f51340746a0 0x7f513406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.689+0000 7f51392c1700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51340b59c0 0x7f51340b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.689+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106867433 0 0) 0x7f5128001110 con 0x7f51340b01e0 2023-11-26T07:17:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.689+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51240035b0 con 0x7f51340b01e0 2023-11-26T07:17:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2134964446 0 0) 0x7f51240035b0 con 0x7f51340b01e0 2023-11-26T07:17:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51340b59c0 msgr2=0x7f51340b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51340b59c0 0x7f51340b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 >> v2:172.21.15.203:3300/0 conn(0x7f51340746a0 msgr2=0x7f513406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 --2- 172.21.15.83:0/331611328 >> v2:172.21.15.203:3300/0 conn(0x7f51340746a0 0x7f513406e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51340b4250 con 0x7f51340b01e0 2023-11-26T07:17:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7f5128001800 con 0x7f51340b01e0 2023-11-26T07:17:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 >> v1:172.21.15.83:6789/0 conn(0x7f51340b01e0 legacy=0x7f5134073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124003a00 0x7f5124003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5124004850 0x7f5124004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51340b59c0 0x7f5124005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51341470a0 con 0x7f5124003a00 2023-11-26T07:17:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51340b05c0 con 0x7f5124004850 2023-11-26T07:17:55.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512400ba10 con 0x7f51340b59c0 2023-11-26T07:17:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f5138ac0700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124003a00 0x7f5124003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f5133fff700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5124004850 0x7f5124004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51392c1700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51340b59c0 0x7f5124005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f5138ac0700 1 -- 172.21.15.83:0/331611328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51340b59c0 msgr2=0x7f5124005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f5138ac0700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51340b59c0 0x7f5124005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f5138ac0700 1 -- 172.21.15.83:0/331611328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5124004850 msgr2=0x7f5124004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f5138ac0700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5124004850 0x7f5124004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f5138ac0700 1 -- 172.21.15.83:0/331611328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240035b0 con 0x7f5124003a00 2023-11-26T07:17:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f5138ac0700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124003a00 0x7f5124003e20 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f512800c320 tx=0x7f5124005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733082de9eff6e37 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800ce20 con 0x7f5124003a00 2023-11-26T07:17:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5128005960 con 0x7f5124003a00 2023-11-26T07:17:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.690+0000 7f51337fe700 1 -- 172.21.15.83:0/331611328 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128005d10 con 0x7f5124003a00 2023-11-26T07:17:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.691+0000 7f513ad24700 1 -- 172.21.15.83:0/331611328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124003a00 msgr2=0x7f5124003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.691+0000 7f513ad24700 1 --2- 172.21.15.83:0/331611328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124003a00 0x7f5124003e20 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f512800c320 tx=0x7f5124005880 comp rx=0 tx=0).stop 2023-11-26T07:17:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.691+0000 7f51392c1700 1 -- 172.21.15.83:0/331611328 reap_dead start 2023-11-26T07:17:55.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.691+0000 7f513ad24700 1 -- 172.21.15.83:0/331611328 shutdown_connections 2023-11-26T07:17:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.691+0000 7f513ad24700 1 -- 172.21.15.83:0/331611328 >> 172.21.15.83:0/331611328 conn(0x7f51340d0b40 msgr2=0x7f51340d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.691+0000 7f513ad24700 1 -- 172.21.15.83:0/331611328 shutdown_connections 2023-11-26T07:17:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.691+0000 7f513ad24700 1 -- 172.21.15.83:0/331611328 wait complete. 2023-11-26T07:17:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.692+0000 7f513ad24700 1 Processor -- start 2023-11-26T07:17:55.699 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.692+0000 7f513ad24700 1 -- start start 2023-11-26T07:17:55.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f513ad24700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124003a00 0x7f51340aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f513ad24700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51341e9ae0 0x7f51340c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f513ad24700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f513415d190 0x7f513415d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f513ad24700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51340c7040 con 0x7f5124003a00 2023-11-26T07:17:55.702 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f513ad24700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51340c6430 con 0x7f513415d190 2023-11-26T07:17:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f513ad24700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51340c6710 con 0x7f51341e9ae0 2023-11-26T07:17:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f5138ac0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124003a00 0x7f51340aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f5138ac0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124003a00 0x7f51340aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42718/0 (socket says 172.21.15.83:42718) 2023-11-26T07:17:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f5138ac0700 1 -- 172.21.15.83:0/2247462040 learned_addr learned my addr 172.21.15.83:0/2247462040 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f51392c1700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f513415d190 0x7f513415d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f5133fff700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51341e9ae0 0x7f51340c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f5138ac0700 1 -- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51341e9ae0 msgr2=0x7f51340c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f5138ac0700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51341e9ae0 0x7f51340c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.703 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f5138ac0700 1 -- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f513415d190 msgr2=0x7f513415d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f5138ac0700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f513415d190 0x7f513415d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f5138ac0700 1 -- 172.21.15.83:0/2247462040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51340c69f0 con 0x7f5124003a00 2023-11-26T07:17:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.693+0000 7f51392c1700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f513415d190 0x7f513415d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.694+0000 7f5138ac0700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124003a00 0x7f51340aec60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f5128008fd0 tx=0x7f5128000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.694+0000 7f51317fa700 1 -- 172.21.15.83:0/2247462040 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800ce90 con 0x7f5124003a00 2023-11-26T07:17:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.694+0000 7f51317fa700 1 -- 172.21.15.83:0/2247462040 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5128001110 con 0x7f5124003a00 2023-11-26T07:17:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.694+0000 7f513ad24700 1 -- 172.21.15.83:0/2247462040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51340c6cd0 con 0x7f5124003a00 2023-11-26T07:17:55.704 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.694+0000 7f51317fa700 1 -- 172.21.15.83:0/2247462040 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128004cc0 con 0x7f5124003a00 2023-11-26T07:17:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.694+0000 7f513ad24700 1 -- 172.21.15.83:0/2247462040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51340c74d0 con 0x7f5124003a00 2023-11-26T07:17:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.695+0000 7f513ad24700 1 -- 172.21.15.83:0/2247462040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f513406e1c0 con 0x7f5124003a00 2023-11-26T07:17:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.695+0000 7f51317fa700 1 -- 172.21.15.83:0/2247462040 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f5128005040 con 0x7f5124003a00 2023-11-26T07:17:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.695+0000 7f51317fa700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f511806b1a0 0x7f511806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.696+0000 7f51317fa700 1 -- 172.21.15.83:0/2247462040 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f512808aa80 con 0x7f5124003a00 2023-11-26T07:17:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.696+0000 7f51317fa700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] conn(0x7f5118071000 0x7f5118073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.696+0000 7f51317fa700 1 -- 172.21.15.83:0/2247462040 --> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5118073af0 con 0x7f5118071000 2023-11-26T07:17:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.696+0000 7f51317fa700 1 -- 172.21.15.83:0/2247462040 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f512808f2a0 con 0x7f5124003a00 2023-11-26T07:17:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.696+0000 7f5133fff700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f511806b1a0 0x7f511806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.696+0000 7f51392c1700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] conn(0x7f5118071000 0x7f5118073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.696+0000 7f5133fff700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f511806b1a0 0x7f511806d660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f51341e9ee0 tx=0x7f5120008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.696+0000 7f51392c1700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] conn(0x7f5118071000 0x7f5118073420 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.706 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.697+0000 7f51317fa700 1 -- 172.21.15.83:0/2247462040 <== osd.5 v2:172.21.15.203:6800/2181588544 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5118073af0 con 0x7f5118071000 2023-11-26T07:17:55.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.712+0000 7f513ad24700 1 -- 172.21.15.83:0/2247462040 --> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f513406dee0 con 0x7f5118071000 2023-11-26T07:17:55.713 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.712+0000 7f51317fa700 1 -- 172.21.15.83:0/2247462040 <== osd.5 v2:172.21.15.203:6800/2181588544 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f512c00f040 con 0x7f5118071000 2023-11-26T07:17:55.714 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:55.714 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:17:55.714 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:55.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.713+0000 7f513ad24700 1 -- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] conn(0x7f5118071000 msgr2=0x7f5118073420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.713+0000 7f513ad24700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:6800/2181588544,v1:172.21.15.203:6801/2181588544] conn(0x7f5118071000 0x7f5118073420 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.715 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.713+0000 7f513ad24700 1 -- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f511806b1a0 msgr2=0x7f511806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.713+0000 7f513ad24700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f511806b1a0 0x7f511806d660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f51341e9ee0 tx=0x7f5120008040 comp rx=0 tx=0).stop 2023-11-26T07:17:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.713+0000 7f513ad24700 1 -- 172.21.15.83:0/2247462040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124003a00 msgr2=0x7f51340aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.713+0000 7f513ad24700 1 --2- 172.21.15.83:0/2247462040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5124003a00 0x7f51340aec60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f5128008fd0 tx=0x7f5128000fe0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.713+0000 7f51392c1700 1 -- 172.21.15.83:0/2247462040 reap_dead start 2023-11-26T07:17:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.713+0000 7f513ad24700 1 -- 172.21.15.83:0/2247462040 shutdown_connections 2023-11-26T07:17:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.713+0000 7f513ad24700 1 -- 172.21.15.83:0/2247462040 >> 172.21.15.83:0/2247462040 conn(0x7f51340d0b40 msgr2=0x7f5134071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.716 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.713+0000 7f513ad24700 1 -- 172.21.15.83:0/2247462040 shutdown_connections 2023-11-26T07:17:55.717 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.713+0000 7f513ad24700 1 -- 172.21.15.83:0/2247462040 wait complete. 2023-11-26T07:17:55.726 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-26T07:17:55.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.846+0000 7f22bd0fa700 1 Processor -- start 2023-11-26T07:17:55.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.847+0000 7f22bd0fa700 1 -- start start 2023-11-26T07:17:55.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.847+0000 7f22bd0fa700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22b81446a0 0x7f22b80b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.850 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.847+0000 7f22bd0fa700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22b80adf00 0x7f22b806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.847+0000 7f22bd0fa700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b80af950 con 0x7f22b806c540 2023-11-26T07:17:55.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.847+0000 7f22bd0fa700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b806c9f0 con 0x7f22b81446a0 2023-11-26T07:17:55.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.847+0000 7f22bd0fa700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b8144a80 con 0x7f22b80adf00 2023-11-26T07:17:55.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.847+0000 7f22b6d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22b81446a0 0x7f22b80b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22b6d9d700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22b81446a0 0x7f22b80b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42728/0 (socket says 172.21.15.83:42728) 2023-11-26T07:17:55.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22b6d9d700 1 -- 172.21.15.83:0/1857119882 learned_addr learned my addr 172.21.15.83:0/1857119882 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:55.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22b6d9d700 1 -- 172.21.15.83:0/1857119882 >> v2:172.21.15.203:3300/0 conn(0x7f22b80adf00 msgr2=0x7f22b806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22b6d9d700 1 --2- 172.21.15.83:0/1857119882 >> v2:172.21.15.203:3300/0 conn(0x7f22b80adf00 0x7f22b806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22b6d9d700 1 -- 172.21.15.83:0/1857119882 >> v1:172.21.15.203:6789/0 conn(0x7f22b806c540 legacy=0x7f22b806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22b6d9d700 1 -- 172.21.15.83:0/1857119882 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b80afc30 con 0x7f22b81446a0 2023-11-26T07:17:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22b6d9d700 1 --2- 172.21.15.83:0/1857119882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22b81446a0 0x7f22b80b5ae0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f22b809f030 tx=0x7f22ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859b12927956aa70 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22b5d9b700 1 -- 172.21.15.83:0/1857119882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac004230 con 0x7f22b81446a0 2023-11-26T07:17:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22b5d9b700 1 -- 172.21.15.83:0/1857119882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac004390 con 0x7f22b81446a0 2023-11-26T07:17:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22b5d9b700 1 -- 172.21.15.83:0/1857119882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0036a0 con 0x7f22b81446a0 2023-11-26T07:17:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22bd0fa700 1 -- 172.21.15.83:0/1857119882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22b81446a0 msgr2=0x7f22b80b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.848+0000 7f22bd0fa700 1 --2- 172.21.15.83:0/1857119882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22b81446a0 0x7f22b80b5ae0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f22b809f030 tx=0x7f22ac009c00 comp rx=0 tx=0).stop 2023-11-26T07:17:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.849+0000 7f22bd0fa700 1 -- 172.21.15.83:0/1857119882 shutdown_connections 2023-11-26T07:17:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.849+0000 7f22bd0fa700 1 --2- 172.21.15.83:0/1857119882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22b81446a0 0x7f22b80b5ae0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.849+0000 7f22bd0fa700 1 --2- 172.21.15.83:0/1857119882 >> v2:172.21.15.203:3300/0 conn(0x7f22b80adf00 0x7f22b806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.849+0000 7f22bd0fa700 1 -- 172.21.15.83:0/1857119882 >> 172.21.15.83:0/1857119882 conn(0x7f22b8071b90 msgr2=0x7f22b8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.849+0000 7f22bd0fa700 1 -- 172.21.15.83:0/1857119882 shutdown_connections 2023-11-26T07:17:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.849+0000 7f22bd0fa700 1 -- 172.21.15.83:0/1857119882 wait complete. 2023-11-26T07:17:55.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.850+0000 7f22bd0fa700 1 Processor -- start 2023-11-26T07:17:55.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.850+0000 7f22bd0fa700 1 -- start start 2023-11-26T07:17:55.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.851+0000 7f22bd0fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22b806c540 0x7f22b815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.856 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.851+0000 7f22bd0fa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22b80adf00 0x7f22b809d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.851+0000 7f22bd0fa700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22b81446a0 0x7f22b8142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.851+0000 7f22bd0fa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b809b330 con 0x7f22b80adf00 2023-11-26T07:17:55.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.851+0000 7f22bd0fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b809e420 con 0x7f22b806c540 2023-11-26T07:17:55.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.851+0000 7f22bd0fa700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b809aa00 con 0x7f22b81446a0 2023-11-26T07:17:55.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.851+0000 7f22b659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22b80adf00 0x7f22b809d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22b659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22b80adf00 0x7f22b809d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42736/0 (socket says 172.21.15.83:42736) 2023-11-26T07:17:55.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22b659c700 1 -- 172.21.15.83:0/23234090 learned_addr learned my addr 172.21.15.83:0/23234090 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:55.857 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22b6d9d700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22b806c540 0x7f22b815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22b759e700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22b81446a0 0x7f22b8142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22b659c700 1 -- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22b81446a0 msgr2=0x7f22b8142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22b659c700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22b81446a0 0x7f22b8142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22b659c700 1 -- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22b806c540 msgr2=0x7f22b815a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22b659c700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22b806c540 0x7f22b815a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22b659c700 1 -- 172.21.15.83:0/23234090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac009550 con 0x7f22b80adf00 2023-11-26T07:17:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22b659c700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22b80adf00 0x7f22b809d9d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f22a800b520 tx=0x7f22a800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f229f7fe700 1 -- 172.21.15.83:0/23234090 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8011830 con 0x7f22b80adf00 2023-11-26T07:17:55.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f229f7fe700 1 -- 172.21.15.83:0/23234090 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a8011990 con 0x7f22b80adf00 2023-11-26T07:17:55.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22bd0fa700 1 -- 172.21.15.83:0/23234090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b809ace0 con 0x7f22b80adf00 2023-11-26T07:17:55.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f229f7fe700 1 -- 172.21.15.83:0/23234090 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8011d40 con 0x7f22b80adf00 2023-11-26T07:17:55.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.852+0000 7f22bd0fa700 1 -- 172.21.15.83:0/23234090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b815aa10 con 0x7f22b80adf00 2023-11-26T07:17:55.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.853+0000 7f22bd0fa700 1 -- 172.21.15.83:0/23234090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b80753f0 con 0x7f22b80adf00 2023-11-26T07:17:55.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.854+0000 7f229f7fe700 1 -- 172.21.15.83:0/23234090 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f22a801a400 con 0x7f22b80adf00 2023-11-26T07:17:55.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.854+0000 7f229f7fe700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f22a406b1a0 0x7f22a406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.859 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.854+0000 7f229f7fe700 1 -- 172.21.15.83:0/23234090 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f22a8089620 con 0x7f22b80adf00 2023-11-26T07:17:55.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.854+0000 7f22b6d9d700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f22a406b1a0 0x7f22a406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.855+0000 7f22b6d9d700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f22a406b1a0 0x7f22a406d660 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f22ac000c00 tx=0x7f22ac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.860 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:55.858+0000 7f229f7fe700 1 -- 172.21.15.83:0/23234090 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f22a8054680 con 0x7f22b80adf00 2023-11-26T07:17:56.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.046+0000 7f22bd0fa700 1 -- 172.21.15.83:0/23234090 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f22b806dc50 con 0x7f22a406b1a0 2023-11-26T07:17:56.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.046+0000 7f229f7fe700 1 -- 172.21.15.83:0/23234090 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f22b806dc50 con 0x7f22a406b1a0 2023-11-26T07:17:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 -- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f22a406b1a0 msgr2=0x7f22a406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f22a406b1a0 0x7f22a406d660 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f22ac000c00 tx=0x7f22ac008030 comp rx=0 tx=0).stop 2023-11-26T07:17:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 -- 172.21.15.83:0/23234090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22b80adf00 msgr2=0x7f22b809d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.050 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22b80adf00 0x7f22b809d9d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f22a800b520 tx=0x7f22a800b830 comp rx=0 tx=0).stop 2023-11-26T07:17:56.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 -- 172.21.15.83:0/23234090 shutdown_connections 2023-11-26T07:17:56.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f22a406b1a0 0x7f22a406d660 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22b81446a0 0x7f22b8142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22b806c540 0x7f22b815a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 --2- 172.21.15.83:0/23234090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22b80adf00 0x7f22b809d9d0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 -- 172.21.15.83:0/23234090 >> 172.21.15.83:0/23234090 conn(0x7f22b8071b90 msgr2=0x7f22b815d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 -- 172.21.15.83:0/23234090 shutdown_connections 2023-11-26T07:17:56.051 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.048+0000 7f22bd0fa700 1 -- 172.21.15.83:0/23234090 wait complete. 2023-11-26T07:17:56.052 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 5 to deep-scrub 2023-11-26T07:17:56.062 INFO:tasks.ceph:Scrubbing osd.6 2023-11-26T07:17:56.062 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:17:56.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.185+0000 7f3f73c91700 1 Processor -- start 2023-11-26T07:17:56.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.185+0000 7f3f73c91700 1 -- start start 2023-11-26T07:17:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f73c91700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f6c15e020 0x7f3f6c0b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f73c91700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3f6c0cfae0 0x7f3f6c0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f73c91700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6c0b1ba0 con 0x7f3f6c0b1720 2023-11-26T07:17:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f73c91700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f6c0cff70 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f73c91700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c161c80 con 0x7f3f6c15e020 2023-11-26T07:17:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f7222e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3f6c0cfae0 0x7f3f6c0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f7222e700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3f6c0cfae0 0x7f3f6c0af3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42746/0 (socket says 172.21.15.83:42746) 2023-11-26T07:17:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f7222e700 1 -- 172.21.15.83:0/3453018165 learned_addr learned my addr 172.21.15.83:0/3453018165 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f71a2d700 1 --2- 172.21.15.83:0/3453018165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f6c15e020 0x7f3f6c0b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f7222e700 1 -- 172.21.15.83:0/3453018165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f6c15e020 msgr2=0x7f3f6c0b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f7222e700 1 --2- 172.21.15.83:0/3453018165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f6c15e020 0x7f3f6c0b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f7222e700 1 -- 172.21.15.83:0/3453018165 >> v1:172.21.15.203:6789/0 conn(0x7f3f6c0b1720 legacy=0x7f3f6c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f7222e700 1 -- 172.21.15.83:0/3453018165 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c161f40 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f7222e700 1 --2- 172.21.15.83:0/3453018165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f6c0cfae0 0x7f3f6c0af3a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009e50 tx=0x7f3f6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec90ffc994d1d18a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.186+0000 7f3f70a2b700 1 -- 172.21.15.83:0/3453018165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6400e070 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.187+0000 7f3f70a2b700 1 -- 172.21.15.83:0/3453018165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6400b910 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.187+0000 7f3f70a2b700 1 -- 172.21.15.83:0/3453018165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6400bcc0 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.187+0000 7f3f73c91700 1 -- 172.21.15.83:0/3453018165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f6c0cfae0 msgr2=0x7f3f6c0af3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.187+0000 7f3f73c91700 1 --2- 172.21.15.83:0/3453018165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f6c0cfae0 0x7f3f6c0af3a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009e50 tx=0x7f3f6400b330 comp rx=0 tx=0).stop 2023-11-26T07:17:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.187+0000 7f3f73c91700 1 -- 172.21.15.83:0/3453018165 shutdown_connections 2023-11-26T07:17:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.187+0000 7f3f73c91700 1 --2- 172.21.15.83:0/3453018165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f6c0cfae0 0x7f3f6c0af3a0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.187+0000 7f3f73c91700 1 --2- 172.21.15.83:0/3453018165 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f6c15e020 0x7f3f6c0b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.187+0000 7f3f73c91700 1 -- 172.21.15.83:0/3453018165 >> 172.21.15.83:0/3453018165 conn(0x7f3f6c072710 msgr2=0x7f3f6c072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.187+0000 7f3f73c91700 1 -- 172.21.15.83:0/3453018165 shutdown_connections 2023-11-26T07:17:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.187+0000 7f3f73c91700 1 -- 172.21.15.83:0/3453018165 wait complete. 2023-11-26T07:17:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.188+0000 7f3f73c91700 1 Processor -- start 2023-11-26T07:17:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.188+0000 7f3f73c91700 1 -- start start 2023-11-26T07:17:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f73c91700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f6c0b1720 0x7f3f6c099790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f73c91700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f6c0cfae0 0x7f3f6c0c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f73c91700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f6c15e020 0x7f3f6c14e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f73c91700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c0c6c40 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f73c91700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c14e160 con 0x7f3f6c0b1720 2023-11-26T07:17:56.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f73c91700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c0c6960 con 0x7f3f6c15e020 2023-11-26T07:17:56.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f7122c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f6c0cfae0 0x7f3f6c0c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f7122c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f6c0cfae0 0x7f3f6c0c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42750/0 (socket says 172.21.15.83:42750) 2023-11-26T07:17:56.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f7122c700 1 -- 172.21.15.83:0/1189865260 learned_addr learned my addr 172.21.15.83:0/1189865260 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:56.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f71a2d700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f6c0b1720 0x7f3f6c099790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f7222e700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f6c15e020 0x7f3f6c14e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.196 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f7122c700 1 -- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f6c15e020 msgr2=0x7f3f6c14e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f7122c700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f6c15e020 0x7f3f6c14e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f7122c700 1 -- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f6c0b1720 msgr2=0x7f3f6c099790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f7122c700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f6c0b1720 0x7f3f6c099790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.189+0000 7f3f7122c700 1 -- 172.21.15.83:0/1189865260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f64009b30 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.190+0000 7f3f7122c700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f6c0cfae0 0x7f3f6c0c5f70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00c510 tx=0x7f3f5c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.190+0000 7f3f627fc700 1 -- 172.21.15.83:0/1189865260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0052d0 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.190+0000 7f3f627fc700 1 -- 172.21.15.83:0/1189865260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5c005430 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.197 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.190+0000 7f3f73c91700 1 -- 172.21.15.83:0/1189865260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6c14db10 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.190+0000 7f3f627fc700 1 -- 172.21.15.83:0/1189865260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0038c0 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.190+0000 7f3f73c91700 1 -- 172.21.15.83:0/1189865260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6c0c51d0 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.190+0000 7f3f73c91700 1 -- 172.21.15.83:0/1189865260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3f6c0ceda0 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.191+0000 7f3f627fc700 1 -- 172.21.15.83:0/1189865260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f3f5c003a20 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.192+0000 7f3f627fc700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3f5006b1a0 0x7f3f5006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.192+0000 7f3f627fc700 1 -- 172.21.15.83:0/1189865260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3f5c0891e0 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.192+0000 7f3f71a2d700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3f5006b1a0 0x7f3f5006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.198 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.192+0000 7f3f627fc700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] conn(0x7f3f50071000 0x7f3f50073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.192+0000 7f3f627fc700 1 -- 172.21.15.83:0/1189865260 --> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f50073af0 con 0x7f3f50071000 2023-11-26T07:17:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.192+0000 7f3f627fc700 1 -- 172.21.15.83:0/1189865260 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f5c089550 con 0x7f3f6c0cfae0 2023-11-26T07:17:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.192+0000 7f3f7222e700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] conn(0x7f3f50071000 0x7f3f50073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.193+0000 7f3f71a2d700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3f5006b1a0 0x7f3f5006d660 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009b40 tx=0x7f3f68009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.193+0000 7f3f7222e700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] conn(0x7f3f50071000 0x7f3f50073420 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.194+0000 7f3f627fc700 1 -- 172.21.15.83:0/1189865260 <== osd.6 v2:172.21.15.203:6824/2898154732 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3f50073af0 con 0x7f3f50071000 2023-11-26T07:17:56.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.208+0000 7f3f73c91700 1 -- 172.21.15.83:0/1189865260 --> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3f6c06cf90 con 0x7f3f50071000 2023-11-26T07:17:56.210 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f627fc700 1 -- 172.21.15.83:0/1189865260 <== osd.6 v2:172.21.15.203:6824/2898154732 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3f6c06cf90 con 0x7f3f50071000 2023-11-26T07:17:56.210 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:56.210 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:17:56.210 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f73c91700 1 -- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] conn(0x7f3f50071000 msgr2=0x7f3f50073420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f73c91700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:6824/2898154732,v1:172.21.15.203:6825/2898154732] conn(0x7f3f50071000 0x7f3f50073420 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f73c91700 1 -- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3f5006b1a0 msgr2=0x7f3f5006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f73c91700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3f5006b1a0 0x7f3f5006d660 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009b40 tx=0x7f3f68009210 comp rx=0 tx=0).stop 2023-11-26T07:17:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f73c91700 1 -- 172.21.15.83:0/1189865260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f6c0cfae0 msgr2=0x7f3f6c0c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f73c91700 1 --2- 172.21.15.83:0/1189865260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f6c0cfae0 0x7f3f6c0c5f70 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00c510 tx=0x7f3f5c00c820 comp rx=0 tx=0).stop 2023-11-26T07:17:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f7222e700 1 -- 172.21.15.83:0/1189865260 reap_dead start 2023-11-26T07:17:56.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f73c91700 1 -- 172.21.15.83:0/1189865260 shutdown_connections 2023-11-26T07:17:56.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f73c91700 1 -- 172.21.15.83:0/1189865260 >> 172.21.15.83:0/1189865260 conn(0x7f3f6c072710 msgr2=0x7f3f6c1605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f73c91700 1 -- 172.21.15.83:0/1189865260 shutdown_connections 2023-11-26T07:17:56.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.209+0000 7f3f73c91700 1 -- 172.21.15.83:0/1189865260 wait complete. 2023-11-26T07:17:56.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-26T07:17:56.343 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.342+0000 7fcb2a132700 1 Processor -- start 2023-11-26T07:17:56.344 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.342+0000 7fcb2a132700 1 -- start start 2023-11-26T07:17:56.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.343+0000 7fcb2a132700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2406c710 0x7fcb240b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.346 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.343+0000 7fcb2a132700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcb240af4d0 0x7fcb240afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.343+0000 7fcb2a132700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb240ae430 con 0x7fcb2415e080 2023-11-26T07:17:56.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.343+0000 7fcb2a132700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb240af0b0 con 0x7fcb240af4d0 2023-11-26T07:17:56.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.343+0000 7fcb2a132700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb240b05a0 con 0x7fcb2406c710 2023-11-26T07:17:56.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.343+0000 7fcb237fe700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fcb2415e080 0x7fcb2406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34566/0 (socket says 172.21.15.83:34566) 2023-11-26T07:17:56.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.343+0000 7fcb237fe700 1 -- 172.21.15.83:0/4130907006 learned_addr learned my addr 172.21.15.83:0/4130907006 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:56.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.343+0000 7fcb23fff700 1 --2- 172.21.15.83:0/4130907006 >> v2:172.21.15.203:3300/0 conn(0x7fcb240af4d0 0x7fcb240afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.343+0000 7fcb22ffd700 1 --2- 172.21.15.83:0/4130907006 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2406c710 0x7fcb240b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.347 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.343+0000 7fcb227fc700 1 -- 172.21.15.83:0/4130907006 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385398336 0 0) 0x7fcb240ae430 con 0x7fcb2415e080 2023-11-26T07:17:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.343+0000 7fcb227fc700 1 -- 172.21.15.83:0/4130907006 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb0c0035b0 con 0x7fcb2415e080 2023-11-26T07:17:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.344+0000 7fcb227fc700 1 -- 172.21.15.83:0/4130907006 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2876502872 0 0) 0x7fcb0c0035b0 con 0x7fcb2415e080 2023-11-26T07:17:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.344+0000 7fcb23fff700 1 -- 172.21.15.83:0/4130907006 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2406c710 msgr2=0x7fcb240b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.344+0000 7fcb23fff700 1 --2- 172.21.15.83:0/4130907006 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2406c710 0x7fcb240b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.344+0000 7fcb23fff700 1 -- 172.21.15.83:0/4130907006 >> v1:172.21.15.83:6789/0 conn(0x7fcb2415e080 legacy=0x7fcb2406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.344+0000 7fcb23fff700 1 -- 172.21.15.83:0/4130907006 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb240ae710 con 0x7fcb240af4d0 2023-11-26T07:17:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.344+0000 7fcb23fff700 1 --2- 172.21.15.83:0/4130907006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb240af4d0 0x7fcb240afb90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009dc0 tx=0x7fcb1c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc9e20cff086104 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.348 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.344+0000 7fcb227fc700 1 -- 172.21.15.83:0/4130907006 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c00e070 con 0x7fcb240af4d0 2023-11-26T07:17:56.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.344+0000 7fcb227fc700 1 -- 172.21.15.83:0/4130907006 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1c003a20 con 0x7fcb240af4d0 2023-11-26T07:17:56.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.344+0000 7fcb227fc700 1 -- 172.21.15.83:0/4130907006 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18001670 con 0x7fcb240af4d0 2023-11-26T07:17:56.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.345+0000 7fcb2a132700 1 -- 172.21.15.83:0/4130907006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb240af4d0 msgr2=0x7fcb240afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.345+0000 7fcb2a132700 1 --2- 172.21.15.83:0/4130907006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb240af4d0 0x7fcb240afb90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009dc0 tx=0x7fcb1c00b330 comp rx=0 tx=0).stop 2023-11-26T07:17:56.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.345+0000 7fcb2a132700 1 -- 172.21.15.83:0/4130907006 shutdown_connections 2023-11-26T07:17:56.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.345+0000 7fcb2a132700 1 --2- 172.21.15.83:0/4130907006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb240af4d0 0x7fcb240afb90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.345+0000 7fcb2a132700 1 --2- 172.21.15.83:0/4130907006 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2406c710 0x7fcb240b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.349 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.345+0000 7fcb2a132700 1 -- 172.21.15.83:0/4130907006 >> 172.21.15.83:0/4130907006 conn(0x7fcb24071b90 msgr2=0x7fcb24071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.345+0000 7fcb2a132700 1 -- 172.21.15.83:0/4130907006 shutdown_connections 2023-11-26T07:17:56.350 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.345+0000 7fcb2a132700 1 -- 172.21.15.83:0/4130907006 wait complete. 2023-11-26T07:17:56.352 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.346+0000 7fcb2a132700 1 Processor -- start 2023-11-26T07:17:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.346+0000 7fcb2a132700 1 -- start start 2023-11-26T07:17:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb2a132700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2406c710 0x7fcb240cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb2a132700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb240af4d0 0x7fcb2414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb2a132700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2415e080 0x7fcb24158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb2a132700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb240cdb60 con 0x7fcb2406c710 2023-11-26T07:17:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb2a132700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb240cde00 con 0x7fcb240af4d0 2023-11-26T07:17:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb2a132700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb240ce0e0 con 0x7fcb2415e080 2023-11-26T07:17:56.353 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb237fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2406c710 0x7fcb240cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb237fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2406c710 0x7fcb240cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42764/0 (socket says 172.21.15.83:42764) 2023-11-26T07:17:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb237fe700 1 -- 172.21.15.83:0/2585417343 learned_addr learned my addr 172.21.15.83:0/2585417343 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb23fff700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2415e080 0x7fcb24158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb22ffd700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb240af4d0 0x7fcb2414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb237fe700 1 -- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2415e080 msgr2=0x7fcb24158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb237fe700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2415e080 0x7fcb24158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.354 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb237fe700 1 -- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb240af4d0 msgr2=0x7fcb2414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb237fe700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb240af4d0 0x7fcb2414a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb237fe700 1 -- 172.21.15.83:0/2585417343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c009a70 con 0x7fcb2406c710 2023-11-26T07:17:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb237fe700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2406c710 0x7fcb240cd620 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcb180017a0 tx=0x7fcb1800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb0bfff700 1 -- 172.21.15.83:0/2585417343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18004c80 con 0x7fcb2406c710 2023-11-26T07:17:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb0bfff700 1 -- 172.21.15.83:0/2585417343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb18004e20 con 0x7fcb2406c710 2023-11-26T07:17:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb2a132700 1 -- 172.21.15.83:0/2585417343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb240ce3c0 con 0x7fcb2406c710 2023-11-26T07:17:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.347+0000 7fcb0bfff700 1 -- 172.21.15.83:0/2585417343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18007310 con 0x7fcb2406c710 2023-11-26T07:17:56.355 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.348+0000 7fcb2a132700 1 -- 172.21.15.83:0/2585417343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb240ce980 con 0x7fcb2406c710 2023-11-26T07:17:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.348+0000 7fcb2a132700 1 -- 172.21.15.83:0/2585417343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb240753f0 con 0x7fcb2406c710 2023-11-26T07:17:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.349+0000 7fcb0bfff700 1 -- 172.21.15.83:0/2585417343 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fcb18007470 con 0x7fcb2406c710 2023-11-26T07:17:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.349+0000 7fcb0bfff700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcb1006b1a0 0x7fcb1006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.349+0000 7fcb0bfff700 1 -- 172.21.15.83:0/2585417343 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcb18089530 con 0x7fcb2406c710 2023-11-26T07:17:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.349+0000 7fcb22ffd700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcb1006b1a0 0x7fcb1006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.350+0000 7fcb22ffd700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcb1006b1a0 0x7fcb1006d660 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcb240cf530 tx=0x7fcb14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.356 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.353+0000 7fcb0bfff700 1 -- 172.21.15.83:0/2585417343 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb180545a0 con 0x7fcb2406c710 2023-11-26T07:17:56.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.543+0000 7fcb2a132700 1 -- 172.21.15.83:0/2585417343 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb240cd470 con 0x7fcb1006b1a0 2023-11-26T07:17:56.545 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.544+0000 7fcb0bfff700 1 -- 172.21.15.83:0/2585417343 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcb240cd470 con 0x7fcb1006b1a0 2023-11-26T07:17:56.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 -- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcb1006b1a0 msgr2=0x7fcb1006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcb1006b1a0 0x7fcb1006d660 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcb240cf530 tx=0x7fcb14008040 comp rx=0 tx=0).stop 2023-11-26T07:17:56.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 -- 172.21.15.83:0/2585417343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2406c710 msgr2=0x7fcb240cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2406c710 0x7fcb240cd620 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcb180017a0 tx=0x7fcb1800cae0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 -- 172.21.15.83:0/2585417343 shutdown_connections 2023-11-26T07:17:56.548 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7fcb1006b1a0 0x7fcb1006d660 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcb2415e080 0x7fcb24158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2406c710 0x7fcb240cd620 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 --2- 172.21.15.83:0/2585417343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcb240af4d0 0x7fcb2414a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 -- 172.21.15.83:0/2585417343 >> 172.21.15.83:0/2585417343 conn(0x7fcb24071b90 msgr2=0x7fcb2406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 -- 172.21.15.83:0/2585417343 shutdown_connections 2023-11-26T07:17:56.549 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.546+0000 7fcb2a132700 1 -- 172.21.15.83:0/2585417343 wait complete. 2023-11-26T07:17:56.550 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 6 to deep-scrub 2023-11-26T07:17:56.560 INFO:tasks.ceph:Scrubbing osd.7 2023-11-26T07:17:56.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:17:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.683+0000 7faa639d3700 1 Processor -- start 2023-11-26T07:17:56.685 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.683+0000 7faa639d3700 1 -- start start 2023-11-26T07:17:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.684+0000 7faa639d3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa5c0b1020 0x7faa5c0b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.684+0000 7faa639d3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa5c0624b0 0x7faa5c15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.687 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.684+0000 7faa639d3700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5c0d0ec0 con 0x7faa5c0cc820 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.684+0000 7faa639d3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5c0d0380 con 0x7faa5c0624b0 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.684+0000 7faa639d3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c0b1940 con 0x7faa5c0b1020 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.684+0000 7faa61f70700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa5c0624b0 0x7faa5c15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.684+0000 7faa61f70700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7faa5c0624b0 0x7faa5c15c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42776/0 (socket says 172.21.15.83:42776) 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.684+0000 7faa60f6e700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7faa5c0cc820 0x7faa5c15fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34580/0 (socket says 172.21.15.83:34580) 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.684+0000 7faa61f70700 1 -- 172.21.15.83:0/847243558 learned_addr learned my addr 172.21.15.83:0/847243558 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa6176f700 1 --2- 172.21.15.83:0/847243558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa5c0b1020 0x7faa5c0b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.688 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa61f70700 1 -- 172.21.15.83:0/847243558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa5c0b1020 msgr2=0x7faa5c0b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa61f70700 1 --2- 172.21.15.83:0/847243558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa5c0b1020 0x7faa5c0b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa61f70700 1 -- 172.21.15.83:0/847243558 >> v1:172.21.15.83:6789/0 conn(0x7faa5c0cc820 legacy=0x7faa5c15fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa53fff700 1 -- 172.21.15.83:0/847243558 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080491605 0 0) 0x7faa5c0d0ec0 con 0x7faa5c0cc820 2023-11-26T07:17:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa61f70700 1 -- 172.21.15.83:0/847243558 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c0d0620 con 0x7faa5c0624b0 2023-11-26T07:17:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa61f70700 1 --2- 172.21.15.83:0/847243558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5c0624b0 0x7faa5c15c7f0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7faa54009e50 tx=0x7faa5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5469207a32053bb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa53fff700 1 -- 172.21.15.83:0/847243558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5400e070 con 0x7faa5c0624b0 2023-11-26T07:17:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa53fff700 1 -- 172.21.15.83:0/847243558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5400b9c0 con 0x7faa5c0624b0 2023-11-26T07:17:56.689 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa53fff700 1 -- 172.21.15.83:0/847243558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5400bd70 con 0x7faa5c0624b0 2023-11-26T07:17:56.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa639d3700 1 -- 172.21.15.83:0/847243558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5c0624b0 msgr2=0x7faa5c15c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa639d3700 1 --2- 172.21.15.83:0/847243558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5c0624b0 0x7faa5c15c7f0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7faa54009e50 tx=0x7faa5400b330 comp rx=0 tx=0).stop 2023-11-26T07:17:56.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa639d3700 1 -- 172.21.15.83:0/847243558 shutdown_connections 2023-11-26T07:17:56.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa639d3700 1 --2- 172.21.15.83:0/847243558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5c0624b0 0x7faa5c15c7f0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa639d3700 1 --2- 172.21.15.83:0/847243558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa5c0b1020 0x7faa5c0b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.685+0000 7faa639d3700 1 -- 172.21.15.83:0/847243558 >> 172.21.15.83:0/847243558 conn(0x7faa5c072710 msgr2=0x7faa5c072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.686+0000 7faa639d3700 1 -- 172.21.15.83:0/847243558 shutdown_connections 2023-11-26T07:17:56.690 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.686+0000 7faa639d3700 1 -- 172.21.15.83:0/847243558 wait complete. 2023-11-26T07:17:56.693 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.687+0000 7faa639d3700 1 Processor -- start 2023-11-26T07:17:56.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.687+0000 7faa639d3700 1 -- start start 2023-11-26T07:17:56.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.687+0000 7faa639d3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5c0624b0 0x7faa5c0c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.687+0000 7faa639d3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa5c0b1020 0x7faa5c153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.687+0000 7faa639d3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa5c0cc820 0x7faa5c1f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.687+0000 7faa639d3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c0adb80 con 0x7faa5c0624b0 2023-11-26T07:17:56.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.687+0000 7faa639d3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c159710 con 0x7faa5c0cc820 2023-11-26T07:17:56.694 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.687+0000 7faa639d3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c0addd0 con 0x7faa5c0b1020 2023-11-26T07:17:56.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa6176f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5c0624b0 0x7faa5c0c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa6176f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5c0624b0 0x7faa5c0c3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42778/0 (socket says 172.21.15.83:42778) 2023-11-26T07:17:56.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa6176f700 1 -- 172.21.15.83:0/1760525189 learned_addr learned my addr 172.21.15.83:0/1760525189 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:56.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa61f70700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa5c0cc820 0x7faa5c1f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa60f6e700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa5c0b1020 0x7faa5c153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa6176f700 1 -- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa5c0b1020 msgr2=0x7faa5c153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa6176f700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa5c0b1020 0x7faa5c153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.695 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa6176f700 1 -- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa5c0cc820 msgr2=0x7faa5c1f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa6176f700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa5c0cc820 0x7faa5c1f4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa6176f700 1 -- 172.21.15.83:0/1760525189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa54009b30 con 0x7faa5c0624b0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa6176f700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5c0624b0 0x7faa5c0c3d60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00e810 tx=0x7faa4c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa51ffb700 1 -- 172.21.15.83:0/1760525189 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c013610 con 0x7faa5c0624b0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa51ffb700 1 -- 172.21.15.83:0/1760525189 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c013770 con 0x7faa5c0624b0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa639d3700 1 -- 172.21.15.83:0/1760525189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c0ae0b0 con 0x7faa5c0624b0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa51ffb700 1 -- 172.21.15.83:0/1760525189 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c013b20 con 0x7faa5c0624b0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.688+0000 7faa639d3700 1 -- 172.21.15.83:0/1760525189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c0c42a0 con 0x7faa5c0624b0 2023-11-26T07:17:56.696 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.689+0000 7faa639d3700 1 -- 172.21.15.83:0/1760525189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa5c0c3b70 con 0x7faa5c0624b0 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.690+0000 7faa51ffb700 1 -- 172.21.15.83:0/1760525189 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7faa4c013c80 con 0x7faa5c0624b0 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.690+0000 7faa51ffb700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faa4006b1a0 0x7faa4006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.690+0000 7faa51ffb700 1 -- 172.21.15.83:0/1760525189 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faa4c0895b0 con 0x7faa5c0624b0 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.690+0000 7faa51ffb700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] conn(0x7faa40071000 0x7faa40073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.690+0000 7faa51ffb700 1 -- 172.21.15.83:0/1760525189 --> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa40073af0 con 0x7faa40071000 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.690+0000 7faa51ffb700 1 -- 172.21.15.83:0/1760525189 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa4c08bbf0 con 0x7faa5c0624b0 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.690+0000 7faa60f6e700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faa4006b1a0 0x7faa4006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.697 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.690+0000 7faa61f70700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] conn(0x7faa40071000 0x7faa40073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.691+0000 7faa61f70700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] conn(0x7faa40071000 0x7faa40073420 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.691+0000 7faa60f6e700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faa4006b1a0 0x7faa4006d660 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faa5c155a30 tx=0x7faa58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.698 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.692+0000 7faa51ffb700 1 -- 172.21.15.83:0/1760525189 <== osd.7 v2:172.21.15.203:6816/2979691223 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7faa40073af0 con 0x7faa40071000 2023-11-26T07:17:56.707 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.706+0000 7faa639d3700 1 -- 172.21.15.83:0/1760525189 --> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faa5c0d0be0 con 0x7faa40071000 2023-11-26T07:17:56.708 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.707+0000 7faa51ffb700 1 -- 172.21.15.83:0/1760525189 <== osd.7 v2:172.21.15.203:6816/2979691223 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7faa540092e0 con 0x7faa40071000 2023-11-26T07:17:56.708 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2023-11-26T07:17:56.709 INFO:teuthology.orchestra.run.smithi083.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:17:56.709 INFO:teuthology.orchestra.run.smithi083.stdout:} 2023-11-26T07:17:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.707+0000 7faa639d3700 1 -- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] conn(0x7faa40071000 msgr2=0x7faa40073420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.707+0000 7faa639d3700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:6816/2979691223,v1:172.21.15.203:6817/2979691223] conn(0x7faa40071000 0x7faa40073420 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.707+0000 7faa639d3700 1 -- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faa4006b1a0 msgr2=0x7faa4006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.707+0000 7faa639d3700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7faa4006b1a0 0x7faa4006d660 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faa5c155a30 tx=0x7faa58009040 comp rx=0 tx=0).stop 2023-11-26T07:17:56.710 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.708+0000 7faa639d3700 1 -- 172.21.15.83:0/1760525189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5c0624b0 msgr2=0x7faa5c0c3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.708+0000 7faa639d3700 1 --2- 172.21.15.83:0/1760525189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa5c0624b0 0x7faa5c0c3d60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00e810 tx=0x7faa4c00ebd0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.708+0000 7faa61f70700 1 -- 172.21.15.83:0/1760525189 reap_dead start 2023-11-26T07:17:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.708+0000 7faa639d3700 1 -- 172.21.15.83:0/1760525189 shutdown_connections 2023-11-26T07:17:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.708+0000 7faa639d3700 1 -- 172.21.15.83:0/1760525189 >> 172.21.15.83:0/1760525189 conn(0x7faa5c072710 msgr2=0x7faa5c0b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.708+0000 7faa639d3700 1 -- 172.21.15.83:0/1760525189 shutdown_connections 2023-11-26T07:17:56.711 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.708+0000 7faa639d3700 1 -- 172.21.15.83:0/1760525189 wait complete. 2023-11-26T07:17:56.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-26T07:17:56.842 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.841+0000 7f3aba0ce700 1 Processor -- start 2023-11-26T07:17:56.843 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.841+0000 7f3aba0ce700 1 -- start start 2023-11-26T07:17:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3aba0ce700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3ab415e080 0x7f3ab40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3aba0ce700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab40af4d0 0x7f3ab40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3aba0ce700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3ab406c000 0x7f3ab406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3aba0ce700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab40af0b0 con 0x7f3ab406c000 2023-11-26T07:17:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3aba0ce700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab406c9f0 con 0x7f3ab415e080 2023-11-26T07:17:56.845 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3aba0ce700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab40b05a0 con 0x7f3ab40af4d0 2023-11-26T07:17:56.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab3fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3ab406c000 0x7f3ab406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab3fff700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3ab406c000 0x7f3ab406c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42792/0 (socket says 172.21.15.83:42792) 2023-11-26T07:17:56.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab3fff700 1 -- 172.21.15.83:0/335074622 learned_addr learned my addr 172.21.15.83:0/335074622 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:56.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab37fe700 1 --2- 172.21.15.83:0/335074622 >> v2:172.21.15.203:3300/0 conn(0x7f3ab415e080 0x7f3ab40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab2ffd700 1 --2- 172.21.15.83:0/335074622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab40af4d0 0x7f3ab40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab3fff700 1 -- 172.21.15.83:0/335074622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab40af4d0 msgr2=0x7f3ab40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab3fff700 1 --2- 172.21.15.83:0/335074622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab40af4d0 0x7f3ab40afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.846 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab3fff700 1 -- 172.21.15.83:0/335074622 >> v2:172.21.15.203:3300/0 conn(0x7f3ab415e080 msgr2=0x7f3ab40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab3fff700 1 --2- 172.21.15.83:0/335074622 >> v2:172.21.15.203:3300/0 conn(0x7f3ab415e080 0x7f3ab40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab3fff700 1 -- 172.21.15.83:0/335074622 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab40ae430 con 0x7f3ab406c000 2023-11-26T07:17:56.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab2ffd700 1 --2- 172.21.15.83:0/335074622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab40af4d0 0x7f3ab40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:56.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab37fe700 1 --2- 172.21.15.83:0/335074622 >> v2:172.21.15.203:3300/0 conn(0x7f3ab415e080 0x7f3ab40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:56.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab3fff700 1 --2- 172.21.15.83:0/335074622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab406c000 0x7f3ab406c4b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4072a90 tx=0x7f3aac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d4f0c2dac4cf3f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.842+0000 7f3ab27fc700 1 -- 172.21.15.83:0/335074622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00e070 con 0x7f3ab406c000 2023-11-26T07:17:56.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.843+0000 7f3ab27fc700 1 -- 172.21.15.83:0/335074622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac00b980 con 0x7f3ab406c000 2023-11-26T07:17:56.847 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.843+0000 7f3ab27fc700 1 -- 172.21.15.83:0/335074622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00bd30 con 0x7f3ab406c000 2023-11-26T07:17:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.843+0000 7f3aba0ce700 1 -- 172.21.15.83:0/335074622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab406c000 msgr2=0x7f3ab406c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.843+0000 7f3aba0ce700 1 --2- 172.21.15.83:0/335074622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab406c000 0x7f3ab406c4b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4072a90 tx=0x7f3aac00b330 comp rx=0 tx=0).stop 2023-11-26T07:17:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.843+0000 7f3aba0ce700 1 -- 172.21.15.83:0/335074622 shutdown_connections 2023-11-26T07:17:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.843+0000 7f3aba0ce700 1 --2- 172.21.15.83:0/335074622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab406c000 0x7f3ab406c4b0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.843+0000 7f3aba0ce700 1 --2- 172.21.15.83:0/335074622 >> v2:172.21.15.203:3300/0 conn(0x7f3ab415e080 0x7f3ab40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.843+0000 7f3aba0ce700 1 --2- 172.21.15.83:0/335074622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab40af4d0 0x7f3ab40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.843+0000 7f3aba0ce700 1 -- 172.21.15.83:0/335074622 >> 172.21.15.83:0/335074622 conn(0x7f3ab4071b90 msgr2=0x7f3ab4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.848 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.843+0000 7f3aba0ce700 1 -- 172.21.15.83:0/335074622 shutdown_connections 2023-11-26T07:17:56.849 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.843+0000 7f3aba0ce700 1 -- 172.21.15.83:0/335074622 wait complete. 2023-11-26T07:17:56.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.844+0000 7f3aba0ce700 1 Processor -- start 2023-11-26T07:17:56.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.844+0000 7f3aba0ce700 1 -- start start 2023-11-26T07:17:56.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.845+0000 7f3aba0ce700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab406c000 0x7f3ab414d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.851 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.845+0000 7f3aba0ce700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab40af4d0 0x7f3ab40b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.845+0000 7f3aba0ce700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab415e080 0x7f3ab40b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.845+0000 7f3aba0ce700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab414db90 con 0x7f3ab406c000 2023-11-26T07:17:56.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.845+0000 7f3aba0ce700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab41582e0 con 0x7f3ab40af4d0 2023-11-26T07:17:56.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.845+0000 7f3aba0ce700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab41576d0 con 0x7f3ab415e080 2023-11-26T07:17:56.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.845+0000 7f3ab37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab406c000 0x7f3ab414d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.845+0000 7f3ab37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab406c000 0x7f3ab414d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42794/0 (socket says 172.21.15.83:42794) 2023-11-26T07:17:56.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.845+0000 7f3ab37fe700 1 -- 172.21.15.83:0/3063281739 learned_addr learned my addr 172.21.15.83:0/3063281739 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:56.852 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3ab2ffd700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab40af4d0 0x7f3ab40b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3ab3fff700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab415e080 0x7f3ab40b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3ab37fe700 1 -- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab415e080 msgr2=0x7f3ab40b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3ab37fe700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab415e080 0x7f3ab40b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3ab37fe700 1 -- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab40af4d0 msgr2=0x7f3ab40b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3ab37fe700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab40af4d0 0x7f3ab40b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3ab37fe700 1 -- 172.21.15.83:0/3063281739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac009b30 con 0x7f3ab406c000 2023-11-26T07:17:56.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3ab37fe700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab406c000 0x7f3ab414d650 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800c7f0 tx=0x7f3aa800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.853 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3a9bfff700 1 -- 172.21.15.83:0/3063281739 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8011800 con 0x7f3ab406c000 2023-11-26T07:17:56.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3a9bfff700 1 -- 172.21.15.83:0/3063281739 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa8011960 con 0x7f3ab406c000 2023-11-26T07:17:56.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3aba0ce700 1 -- 172.21.15.83:0/3063281739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab41579b0 con 0x7f3ab406c000 2023-11-26T07:17:56.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3a9bfff700 1 -- 172.21.15.83:0/3063281739 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8011d10 con 0x7f3ab406c000 2023-11-26T07:17:56.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.846+0000 7f3aba0ce700 1 -- 172.21.15.83:0/3063281739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab4157f70 con 0x7f3ab406c000 2023-11-26T07:17:56.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.847+0000 7f3aba0ce700 1 -- 172.21.15.83:0/3063281739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab40753f0 con 0x7f3ab406c000 2023-11-26T07:17:56.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.847+0000 7f3a9bfff700 1 -- 172.21.15.83:0/3063281739 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f3aa8018360 con 0x7f3ab406c000 2023-11-26T07:17:56.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.848+0000 7f3a9bfff700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3aa006b1a0 0x7f3aa006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.854 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.848+0000 7f3a9bfff700 1 -- 172.21.15.83:0/3063281739 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3aa8089850 con 0x7f3ab406c000 2023-11-26T07:17:56.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.848+0000 7f3ab2ffd700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3aa006b1a0 0x7f3aa006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.848+0000 7f3ab2ffd700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3aa006b1a0 0x7f3aa006d660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4006fd0 tx=0x7f3aa4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.855 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:56.852+0000 7f3a9bfff700 1 -- 172.21.15.83:0/3063281739 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3aa80547a0 con 0x7f3ab406c000 2023-11-26T07:17:57.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.041+0000 7f3aba0ce700 1 -- 172.21.15.83:0/3063281739 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ab40ae9f0 con 0x7f3aa006b1a0 2023-11-26T07:17:57.043 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.042+0000 7f3a9bfff700 1 -- 172.21.15.83:0/3063281739 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3aa400c040 con 0x7f3aa006b1a0 2023-11-26T07:17:57.045 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 -- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3aa006b1a0 msgr2=0x7f3aa006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3aa006b1a0 0x7f3aa006d660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4006fd0 tx=0x7f3aa4008040 comp rx=0 tx=0).stop 2023-11-26T07:17:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 -- 172.21.15.83:0/3063281739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab406c000 msgr2=0x7f3ab414d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab406c000 0x7f3ab414d650 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800c7f0 tx=0x7f3aa800cbb0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 -- 172.21.15.83:0/3063281739 shutdown_connections 2023-11-26T07:17:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f3aa006b1a0 0x7f3aa006d660 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ab415e080 0x7f3ab40b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab406c000 0x7f3ab414d650 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 --2- 172.21.15.83:0/3063281739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab40af4d0 0x7f3ab40b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 -- 172.21.15.83:0/3063281739 >> 172.21.15.83:0/3063281739 conn(0x7f3ab4071b90 msgr2=0x7f3ab4160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:57.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 -- 172.21.15.83:0/3063281739 shutdown_connections 2023-11-26T07:17:57.047 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.044+0000 7f3aba0ce700 1 -- 172.21.15.83:0/3063281739 wait complete. 2023-11-26T07:17:57.047 INFO:teuthology.orchestra.run.smithi083.stderr:instructed osd(s) 7 to deep-scrub 2023-11-26T07:17:57.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:57.177 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.176+0000 7f34346b0700 1 Processor -- start 2023-11-26T07:17:57.178 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.176+0000 7f34346b0700 1 -- start start 2023-11-26T07:17:57.180 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.177+0000 7f34346b0700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f342c0ce140 0x7f342c15f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f34346b0700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c1414c0 con 0x7f342c15af10 2023-11-26T07:17:57.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f34346b0700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0ce9f0 con 0x7f342c15f630 2023-11-26T07:17:57.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f34346b0700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c15c300 con 0x7f342c0ce140 2023-11-26T07:17:57.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f3431c4b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f342c0ce140 0x7f342c15f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f343244c700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f342c15af10 0x7f342c15e9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34610/0 (socket says 172.21.15.83:34610) 2023-11-26T07:17:57.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f343244c700 1 -- 172.21.15.83:0/3993844550 learned_addr learned my addr 172.21.15.83:0/3993844550 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:17:57.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f3431c4b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f342c0ce140 0x7f342c15f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42802/0 (socket says 172.21.15.83:42802) 2023-11-26T07:17:57.181 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f3431c4b700 1 -- 172.21.15.83:0/3993844550 >> v1:172.21.15.203:6789/0 conn(0x7f342c15f630 legacy=0x7f342c06b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f3431c4b700 1 -- 172.21.15.83:0/3993844550 >> v1:172.21.15.83:6789/0 conn(0x7f342c15af10 legacy=0x7f342c15e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f3431c4b700 1 -- 172.21.15.83:0/3993844550 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0cecd0 con 0x7f342c0ce140 2023-11-26T07:17:57.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f343144a700 1 -- 172.21.15.83:0/3993844550 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779848195 0 0) 0x7f342c1414c0 con 0x7f342c15af10 2023-11-26T07:17:57.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f3431c4b700 1 --2- 172.21.15.83:0/3993844550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f342c0ce140 0x7f342c15f0f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f341c009840 tx=0x7f341c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776c1fa6d9755e04 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f343144a700 1 -- 172.21.15.83:0/3993844550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c011830 con 0x7f342c0ce140 2023-11-26T07:17:57.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f343144a700 1 -- 172.21.15.83:0/3993844550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c011990 con 0x7f342c0ce140 2023-11-26T07:17:57.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.178+0000 7f343144a700 1 -- 172.21.15.83:0/3993844550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c011d20 con 0x7f342c0ce140 2023-11-26T07:17:57.182 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.179+0000 7f34346b0700 1 -- 172.21.15.83:0/3993844550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f342c0ce140 msgr2=0x7f342c15f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.179+0000 7f34346b0700 1 --2- 172.21.15.83:0/3993844550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f342c0ce140 0x7f342c15f0f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f341c009840 tx=0x7f341c009b50 comp rx=0 tx=0).stop 2023-11-26T07:17:57.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.179+0000 7f34346b0700 1 -- 172.21.15.83:0/3993844550 shutdown_connections 2023-11-26T07:17:57.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.179+0000 7f34346b0700 1 --2- 172.21.15.83:0/3993844550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f342c0ce140 0x7f342c15f0f0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.179+0000 7f34346b0700 1 -- 172.21.15.83:0/3993844550 >> 172.21.15.83:0/3993844550 conn(0x7f342c0cd570 msgr2=0x7f342c0cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:57.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.179+0000 7f34346b0700 1 -- 172.21.15.83:0/3993844550 shutdown_connections 2023-11-26T07:17:57.183 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.179+0000 7f34346b0700 1 -- 172.21.15.83:0/3993844550 wait complete. 2023-11-26T07:17:57.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.180+0000 7f34346b0700 1 Processor -- start 2023-11-26T07:17:57.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.181+0000 7f34346b0700 1 -- start start 2023-11-26T07:17:57.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.181+0000 7f34346b0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c0ce140 0x7f342c13d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.181+0000 7f34346b0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f342c15af10 0x7f342c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.186 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.181+0000 7f34346b0700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f342c13fb10 0x7f342c141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.181+0000 7f34346b0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c13ffa0 con 0x7f342c15af10 2023-11-26T07:17:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.181+0000 7f34346b0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1f3250 con 0x7f342c0ce140 2023-11-26T07:17:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.181+0000 7f34346b0700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c142ac0 con 0x7f342c13fb10 2023-11-26T07:17:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3431c4b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f342c15af10 0x7f342c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3431c4b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f342c15af10 0x7f342c13f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42810/0 (socket says 172.21.15.83:42810) 2023-11-26T07:17:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3431c4b700 1 -- 172.21.15.83:0/1000717844 learned_addr learned my addr 172.21.15.83:0/1000717844 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:17:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f343244c700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c0ce140 0x7f342c13d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.187 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3432c4d700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f342c13fb10 0x7f342c141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3431c4b700 1 -- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f342c13fb10 msgr2=0x7f342c141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3431c4b700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f342c13fb10 0x7f342c141d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3431c4b700 1 -- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c0ce140 msgr2=0x7f342c13d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3431c4b700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c0ce140 0x7f342c13d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3431c4b700 1 -- 172.21.15.83:0/1000717844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c0094a0 con 0x7f342c15af10 2023-11-26T07:17:57.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3431c4b700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f342c15af10 0x7f342c13f5d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f341c0090b0 tx=0x7f341c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.188 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3422ffd700 1 -- 172.21.15.83:0/1000717844 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c01a7e0 con 0x7f342c15af10 2023-11-26T07:17:57.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3422ffd700 1 -- 172.21.15.83:0/1000717844 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c01a940 con 0x7f342c15af10 2023-11-26T07:17:57.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f34346b0700 1 -- 172.21.15.83:0/1000717844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c142da0 con 0x7f342c15af10 2023-11-26T07:17:57.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f3422ffd700 1 -- 172.21.15.83:0/1000717844 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c01acb0 con 0x7f342c15af10 2023-11-26T07:17:57.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.182+0000 7f34346b0700 1 -- 172.21.15.83:0/1000717844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c0b4340 con 0x7f342c15af10 2023-11-26T07:17:57.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.183+0000 7f34346b0700 1 -- 172.21.15.83:0/1000717844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c067f60 con 0x7f342c15af10 2023-11-26T07:17:57.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.183+0000 7f3422ffd700 1 -- 172.21.15.83:0/1000717844 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f341c022070 con 0x7f342c15af10 2023-11-26T07:17:57.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.184+0000 7f3422ffd700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f341006b1a0 0x7f341006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.189 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.184+0000 7f3422ffd700 1 -- 172.21.15.83:0/1000717844 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f341c08a8b0 con 0x7f342c15af10 2023-11-26T07:17:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.184+0000 7f343244c700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f341006b1a0 0x7f341006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.185+0000 7f343244c700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f341006b1a0 0x7f341006d660 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3428007b10 tx=0x7f3428007a60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.190 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.188+0000 7f3422ffd700 1 -- 172.21.15.83:0/1000717844 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f341c055ac0 con 0x7f342c15af10 2023-11-26T07:17:57.373 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.372+0000 7f34346b0700 1 -- 172.21.15.83:0/1000717844 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f342c0cefb0 con 0x7f341006b1a0 2023-11-26T07:17:57.376 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.375+0000 7f3422ffd700 1 -- 172.21.15.83:0/1000717844 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50627 (secure 0 0 0) 0x7f3428001330 con 0x7f341006b1a0 2023-11-26T07:17:57.377 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:17:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 -- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f341006b1a0 msgr2=0x7f341006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f341006b1a0 0x7f341006d660 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3428007b10 tx=0x7f3428007a60 comp rx=0 tx=0).stop 2023-11-26T07:17:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 -- 172.21.15.83:0/1000717844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f342c15af10 msgr2=0x7f342c13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f342c15af10 0x7f342c13f5d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f341c0090b0 tx=0x7f341c004750 comp rx=0 tx=0).stop 2023-11-26T07:17:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 -- 172.21.15.83:0/1000717844 shutdown_connections 2023-11-26T07:17:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f341006b1a0 0x7f341006d660 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f342c13fb10 0x7f342c141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f342c0ce140 0x7f342c13d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 --2- 172.21.15.83:0/1000717844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f342c15af10 0x7f342c13f5d0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 -- 172.21.15.83:0/1000717844 >> 172.21.15.83:0/1000717844 conn(0x7f342c0cd570 msgr2=0x7f342c06e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:57.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 -- 172.21.15.83:0/1000717844 shutdown_connections 2023-11-26T07:17:57.380 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:17:57.377+0000 7f34346b0700 1 -- 172.21.15.83:0/1000717844 wait complete. 2023-11-26T07:17:57.380 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:17:57.390 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-11-26T07:17:56.333426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":256728,"kb_used_data":11432,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754717992,"statfs":{"total":773094113280,"available":772831223808,"internally_reserved":0,"allocated":11706368,"data_stored":5772880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16777869,"num_objects":-19,"num_object_clones":0,"num_object_copies":-38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-19,"num_whiteouts":0,"num_read":-75,"num_read_kb":-8233,"num_write":-31,"num_write_kb":-16393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.004345"},"pg_stats":[{"pgid":"1.0","version":"71'128","reported_seq":291,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-11-26T07:17:47.333936+0000","last_change":"2023-11-26T07:11:17.461899+0000","last_active":"2023-11-26T07:17:47.333936+0000","last_peered":"2023-11-26T07:17:47.333936+0000","last_clean":"2023-11-26T07:17:47.333936+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:17:47.333936+0000","last_undegraded":"2023-11-26T07:17:47.333936+0000","last_fullsized":"2023-11-26T07:17:47.333936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:11:16.368704+0000","last_clean_scrub_stamp":"2023-11-26T07:11:16.368704+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:38:51.691801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1908736,"data_stored":1164386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":0.84099999999999997,"15min":0.86099999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42499999999999999,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.78600000000000003,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.36299999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69499999999999995,"5min":0.88100000000000001,"15min":1.048},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59799999999999998,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.315},"max":{"1min":0.753,"5min":0.81899999999999995,"15min":0.89600000000000002},"last":0.42999999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79800000000000004,"5min":0.877,"15min":0.92000000000000004},"last":0.58799999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.33600000000000002},"max":{"1min":0.77100000000000002,"5min":0.91300000000000003,"15min":1.0629999999999999},"last":0.629},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.73999999999999999,"5min":0.91400000000000003,"15min":1.0960000000000001},"last":0.67000000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0269999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69099999999999995,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72399999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72499999999999998,"5min":0.996,"15min":0.996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81100000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.65400000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80000000000000004,"5min":1.006,"15min":1.361},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.751,"5min":1.0409999999999999,"15min":1.1899999999999999},"last":0.68300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28304,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343536,"statfs":{"total":96636764160,"available":96607780864,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54300000000000004},"min":{"1min":0.34599999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.018,"5min":1.159,"15min":1.159},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75800000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.41799999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77100000000000002,"5min":0.79900000000000004,"15min":0.88900000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.54100000000000004,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":1.0209999999999999},"last":0.47199999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38300000000000001,"5min":0.376,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":0.79200000000000004,"15min":0.96199999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.85099999999999998},"last":0.54200000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.55800000000000005},"min":{"1min":0.46800000000000003,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":0.76200000000000001,"5min":4.8689999999999998,"15min":4.8689999999999998},"last":0.503},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74099999999999999,"5min":0.82599999999999996,"15min":1.1950000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.64700000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.36699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.67400000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.63},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69399999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.501,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71899999999999997,"5min":0.81999999999999995,"15min":0.83099999999999996},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32464,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.497,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.34899999999999998},"min":{"1min":0.26500000000000001,"5min":0.20000000000000001,"15min":0.19700000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.32200000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.33900000000000002},"min":{"1min":0.247,"5min":0.22500000000000001,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28100000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.309}]},{"osd":3,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.29999999999999999,"5min":0.252,"15min":0.246},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.26700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47999999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.372}]},{"osd":4,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":1.167,"15min":1.167},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.80700000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.46600000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.879,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.441}]},{"osd":6,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.48499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83199999999999996,"5min":0.91700000000000004,"15min":1.129},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.48799999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76600000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.46400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.79200000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1908736,"data_stored":1164386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.217,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.51900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.247},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.36899999999999999,"15min":0.36699999999999999},"min":{"1min":0.25600000000000001,"5min":0.23999999999999999,"15min":0.23300000000000001},"max":{"1min":0.50900000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.29699999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39000000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":0.64300000000000002,"15min":0.65700000000000003},"last":0.375},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.37,"15min":0.35199999999999998},"min":{"1min":0.246,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":0.48299999999999998,"5min":1.202,"15min":1.202},"last":0.31900000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.39900000000000002},"min":{"1min":0.27900000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52200000000000002,"5min":0.75,"15min":0.75},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41899999999999998,"15min":0.433},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.53600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34899999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":1.419,"15min":1.419},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64000000000000001,"15min":0.66300000000000003},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":1,"15min":1.0009999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59699999999999998,"15min":0.627},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81499999999999995,"5min":1.046,"15min":1.046},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.629},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.746,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.55600000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81699999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81699999999999995,"5min":1.387,"15min":1.387},"last":0.65400000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.626},"min":{"1min":0.41599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76700000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.627,"15min":0.65900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":1.087,"15min":1.087},"last":0.53900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.36499999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.59199999999999997,"5min":0.627,"15min":0.627},"last":0.37},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.248,"5min":0.186,"15min":0.186},"max":{"1min":0.51500000000000001,"5min":0.70099999999999996,"15min":0.84599999999999997},"last":0.248}]},{"osd":1,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38700000000000001},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52000000000000002,"5min":0.58799999999999997,"15min":0.65700000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38500000000000001,"15min":0.37},"min":{"1min":0.252,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.318}]},{"osd":3,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36299999999999999,"15min":0.36899999999999999},"min":{"1min":0.254,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.58799999999999997,"15min":0.68700000000000006},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.42199999999999999},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.76100000000000001},"last":0.34100000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.65600000000000003},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81799999999999995,"5min":0.85799999999999998,"15min":2.2160000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80600000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.65200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.61599999999999999},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82899999999999996,"5min":0.90000000000000002,"15min":1.075},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.0309999999999999,"15min":1.2190000000000001},"last":0.55800000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90800000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89400000000000002,"5min":1.004,"15min":1.03},"last":0.63400000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92600000000000005,"5min":0.94899999999999995,"15min":1.1919999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.62},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.40999999999999998},"max":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":1.1120000000000001},"last":0.622}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32464,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.35299999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.188},"max":{"1min":0.55700000000000005,"5min":0.629,"15min":1.284},"last":0.309},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.371,"15min":0.45400000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57799999999999996,"5min":0.63600000000000001,"15min":2.8460000000000001},"last":0.28100000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.34100000000000003,"15min":0.48299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53500000000000003,"5min":0.61799999999999999,"15min":4.9109999999999996},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.39100000000000001,"15min":0.47299999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.65800000000000003,"5min":0.69399999999999995,"15min":2.8940000000000001},"last":0.33400000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41999999999999998,"15min":0.48599999999999999},"min":{"1min":0.27900000000000003,"5min":0.224,"15min":0.224},"max":{"1min":0.626,"5min":0.626,"15min":2.9790000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38300000000000001,"15min":0.41699999999999998},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.23100000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":1.3440000000000001},"last":0.35099999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.66000000000000003},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":2.9500000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60899999999999999,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.405},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":0.629},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.377},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":4.8129999999999997},"last":0.622}]},{"osd":6,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61699999999999999,"15min":0.66000000000000003},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.4319999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.75700000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":4.883},"last":0.59399999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.77200000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":4.8499999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":2.923},"last":0.53300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.65600000000000003,"5min":0.79400000000000004,"15min":2.9510000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54600000000000004,"15min":0.53100000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70899999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.501}]},{"osd":1,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55900000000000005,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68700000000000006,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.64200000000000002},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.91100000000000003,"15min":2.9820000000000002},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77400000000000002,"5min":2.113,"15min":2.113},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.65700000000000003},"min":{"1min":0.40400000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":3.0030000000000001},"last":0.40999999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.54700000000000004},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.33300000000000002},"max":{"1min":0.67600000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.58499999999999996},"min":{"1min":0.41899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85199999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52300000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80900000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.35899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78100000000000003,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.41199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.439},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.84799999999999998},"last":0.58899999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82999999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.55300000000000005,"5min":0.315,"15min":0.315},"max":{"1min":0.91400000000000003,"5min":1.01,"15min":1.01},"last":0.57399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.29799999999999999,"15min":0.27500000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96199999999999997,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79400000000000004,"5min":1.091,"15min":1.091},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45100000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.42399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.54200000000000004},"min":{"1min":0.44400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.40300000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.433},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61499999999999999,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.629}]},{"osd":4,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.35599999999999998,"15min":0.311},"max":{"1min":1.044,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0980000000000001,"5min":1.161,"15min":1.161},"last":0.38200000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.35199999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54700000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0129999999999999,"5min":1.196,"15min":1.196},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.628,"15min":0.64500000000000002},"min":{"1min":0.40699999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0309999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.495}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:57.393 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-26T07:11:16.368704+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=26, tm_hour=7, tm_min=11, tm_sec=16, tm_wday=6, tm_yday=330, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=26, tm_hour=7, tm_min=17, tm_sec=52, tm_wday=6, tm_yday=330, tm_isdst=0) 2023-11-26T07:17:57.393 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-26T07:17:57.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:17:57.526 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:17.395 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:17.530 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.528+0000 7f187570b700 1 Processor -- start 2023-11-26T07:18:17.531 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.528+0000 7f187570b700 1 -- start start 2023-11-26T07:18:17.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f187570b700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18700cf360 0x7f18700b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f187570b700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f187015deb0 0x7f187015e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.533 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f187570b700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187015a790 con 0x7f187015a3b0 2023-11-26T07:18:17.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f187570b700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18700cf8b0 con 0x7f18700cf360 2023-11-26T07:18:17.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f187570b700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18700b4c40 con 0x7f187015deb0 2023-11-26T07:18:17.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f186e7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18700cf360 0x7f18700b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f186e7fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18700cf360 0x7f18700b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42816/0 (socket says 172.21.15.83:42816) 2023-11-26T07:18:17.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f186e7fc700 1 -- 172.21.15.83:0/1348109208 learned_addr learned my addr 172.21.15.83:0/1348109208 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:18:17.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f186f7fe700 1 --2- 172.21.15.83:0/1348109208 >> v2:172.21.15.203:3300/0 conn(0x7f187015deb0 0x7f187015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.534 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f186e7fc700 1 -- 172.21.15.83:0/1348109208 >> v2:172.21.15.203:3300/0 conn(0x7f187015deb0 msgr2=0x7f187015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:17.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f186e7fc700 1 --2- 172.21.15.83:0/1348109208 >> v2:172.21.15.203:3300/0 conn(0x7f187015deb0 0x7f187015e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f186e7fc700 1 -- 172.21.15.83:0/1348109208 >> v1:172.21.15.203:6789/0 conn(0x7f187015a3b0 legacy=0x7f18700b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f186e7fc700 1 -- 172.21.15.83:0/1348109208 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1870070c60 con 0x7f18700cf360 2023-11-26T07:18:17.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.529+0000 7f186f7fe700 1 --2- 172.21.15.83:0/1348109208 >> v2:172.21.15.203:3300/0 conn(0x7f187015deb0 0x7f187015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:17.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f186e7fc700 1 --2- 172.21.15.83:0/1348109208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18700cf360 0x7f18700b3a30 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1870074fd0 tx=0x7f1860009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a8c571f5d311db server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f186dffb700 1 -- 172.21.15.83:0/1348109208 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186000b870 con 0x7f18700cf360 2023-11-26T07:18:17.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f186dffb700 1 -- 172.21.15.83:0/1348109208 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186000b9d0 con 0x7f18700cf360 2023-11-26T07:18:17.535 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f186dffb700 1 -- 172.21.15.83:0/1348109208 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186000bd60 con 0x7f18700cf360 2023-11-26T07:18:17.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f187570b700 1 -- 172.21.15.83:0/1348109208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18700cf360 msgr2=0x7f18700b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f187570b700 1 --2- 172.21.15.83:0/1348109208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18700cf360 0x7f18700b3a30 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1870074fd0 tx=0x7f1860009bd0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f187570b700 1 -- 172.21.15.83:0/1348109208 shutdown_connections 2023-11-26T07:18:17.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f187570b700 1 --2- 172.21.15.83:0/1348109208 >> v2:172.21.15.203:3300/0 conn(0x7f187015deb0 0x7f187015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f187570b700 1 --2- 172.21.15.83:0/1348109208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18700cf360 0x7f18700b3a30 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f187570b700 1 -- 172.21.15.83:0/1348109208 >> 172.21.15.83:0/1348109208 conn(0x7f18700b1740 msgr2=0x7f18700b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:17.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f187570b700 1 -- 172.21.15.83:0/1348109208 shutdown_connections 2023-11-26T07:18:17.536 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.530+0000 7f187570b700 1 -- 172.21.15.83:0/1348109208 wait complete. 2023-11-26T07:18:17.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.532+0000 7f187570b700 1 Processor -- start 2023-11-26T07:18:17.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.532+0000 7f187570b700 1 -- start start 2023-11-26T07:18:17.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f187570b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18700cf360 0x7f187014b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.539 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f187570b700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f187015a3b0 0x7f18701f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f187570b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f187015deb0 0x7f18701f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f187570b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187014bdc0 con 0x7f18700cf360 2023-11-26T07:18:17.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f187570b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187014b600 con 0x7f187015deb0 2023-11-26T07:18:17.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f187570b700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187014c010 con 0x7f187015a3b0 2023-11-26T07:18:17.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f186effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18700cf360 0x7f187014b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f186effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18700cf360 0x7f187014b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42822/0 (socket says 172.21.15.83:42822) 2023-11-26T07:18:17.540 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f186effd700 1 -- 172.21.15.83:0/1951800312 learned_addr learned my addr 172.21.15.83:0/1951800312 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:18:17.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f186e7fc700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f187015a3b0 0x7f18701f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f186f7fe700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f187015deb0 0x7f18701f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f186effd700 1 -- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f187015a3b0 msgr2=0x7f18701f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f186effd700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f187015a3b0 0x7f18701f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f186effd700 1 -- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f187015deb0 msgr2=0x7f18701f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f186effd700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f187015deb0 0x7f18701f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f186effd700 1 -- 172.21.15.83:0/1951800312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1860009550 con 0x7f18700cf360 2023-11-26T07:18:17.541 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f186effd700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18700cf360 0x7f187014b0c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f186400c450 tx=0x7f186400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f18577fe700 1 -- 172.21.15.83:0/1951800312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18640052d0 con 0x7f18700cf360 2023-11-26T07:18:17.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f18577fe700 1 -- 172.21.15.83:0/1951800312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1864005430 con 0x7f18700cf360 2023-11-26T07:18:17.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f187570b700 1 -- 172.21.15.83:0/1951800312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187014c2f0 con 0x7f18700cf360 2023-11-26T07:18:17.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f18577fe700 1 -- 172.21.15.83:0/1951800312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18640038c0 con 0x7f18700cf360 2023-11-26T07:18:17.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.533+0000 7f187570b700 1 -- 172.21.15.83:0/1951800312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187014c8b0 con 0x7f18700cf360 2023-11-26T07:18:17.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.535+0000 7f187570b700 1 -- 172.21.15.83:0/1951800312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18700751e0 con 0x7f18700cf360 2023-11-26T07:18:17.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.535+0000 7f18577fe700 1 -- 172.21.15.83:0/1951800312 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f1864003a20 con 0x7f18700cf360 2023-11-26T07:18:17.542 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.535+0000 7f18577fe700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185c06b1a0 0x7f185c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.535+0000 7f18577fe700 1 -- 172.21.15.83:0/1951800312 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f18640889f0 con 0x7f18700cf360 2023-11-26T07:18:17.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.535+0000 7f186e7fc700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185c06b1a0 0x7f185c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.536+0000 7f186e7fc700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185c06b1a0 0x7f185c06d660 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1870074fd0 tx=0x7f1860008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.543 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.540+0000 7f18577fe700 1 -- 172.21.15.83:0/1951800312 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1864053ae0 con 0x7f18700cf360 2023-11-26T07:18:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.728+0000 7f187570b700 1 -- 172.21.15.83:0/1951800312 --> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1870071570 con 0x7f185c06b1a0 2023-11-26T07:18:17.733 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.731+0000 7f18577fe700 1 -- 172.21.15.83:0/1951800312 <== mgr.4105 v2:172.21.15.203:6836/866214038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50674 (secure 0 0 0) 0x7f186000bb40 con 0x7f185c06b1a0 2023-11-26T07:18:17.734 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:18:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 -- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185c06b1a0 msgr2=0x7f185c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185c06b1a0 0x7f185c06d660 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1870074fd0 tx=0x7f1860008030 comp rx=0 tx=0).stop 2023-11-26T07:18:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 -- 172.21.15.83:0/1951800312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18700cf360 msgr2=0x7f187014b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18700cf360 0x7f187014b0c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f186400c450 tx=0x7f186400c810 comp rx=0 tx=0).stop 2023-11-26T07:18:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 -- 172.21.15.83:0/1951800312 shutdown_connections 2023-11-26T07:18:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f185c06b1a0 0x7f185c06d660 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f187015deb0 0x7f18701f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18700cf360 0x7f187014b0c0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 --2- 172.21.15.83:0/1951800312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f187015a3b0 0x7f18701f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 -- 172.21.15.83:0/1951800312 >> 172.21.15.83:0/1951800312 conn(0x7f18700b1740 msgr2=0x7f187015d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 -- 172.21.15.83:0/1951800312 shutdown_connections 2023-11-26T07:18:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.733+0000 7f187570b700 1 -- 172.21.15.83:0/1951800312 wait complete. 2023-11-26T07:18:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2023-11-26T07:18:17.747 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2023-11-26T07:18:16.337754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":256716,"kb_used_data":11432,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754718004,"statfs":{"total":773094113280,"available":772831236096,"internally_reserved":0,"allocated":11706368,"data_stored":5772880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002533"},"pg_stats":[{"pgid":"1.0","version":"105'130","reported_seq":302,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-11-26T07:17:57.907617+0000","last_change":"2023-11-26T07:17:57.907656+0000","last_active":"2023-11-26T07:17:57.907617+0000","last_peered":"2023-11-26T07:17:57.907617+0000","last_clean":"2023-11-26T07:17:57.907617+0000","last_became_active":"2023-11-26T07:11:17.460837+0000","last_became_peered":"2023-11-26T07:11:17.460837+0000","last_unstale":"2023-11-26T07:17:57.907617+0000","last_undegraded":"2023-11-26T07:17:57.907617+0000","last_fullsized":"2023-11-26T07:17:57.907617+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'130","last_scrub_stamp":"2023-11-26T07:17:57.907512+0000","last_deep_scrub":"105'130","last_deep_scrub_stamp":"2023-11-26T07:17:57.907512+0000","last_clean_scrub_stamp":"2023-11-26T07:17:57.907512+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:12:14.626050+0000","scrub_duration":0.0076847219999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574937,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1908736,"data_stored":1164386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82899999999999996,"5min":0.84099999999999997,"15min":0.86099999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42499999999999999,"5min":0.307,"15min":0.29599999999999999},"max":{"1min":0.78600000000000003,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.73199999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.36299999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69499999999999995,"5min":0.88100000000000001,"15min":1.048},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.59799999999999998,"15min":0.55800000000000005},"min":{"1min":0.40600000000000003,"5min":0.38200000000000001,"15min":0.315},"max":{"1min":0.753,"5min":0.81899999999999995,"15min":0.89600000000000002},"last":0.68899999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.79800000000000004,"5min":0.877,"15min":0.92000000000000004},"last":0.75800000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.33600000000000002},"max":{"1min":0.77100000000000002,"5min":0.91300000000000003,"15min":1.0629999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.73999999999999999,"5min":0.91400000000000003,"15min":1.0960000000000001},"last":0.59199999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0269999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.69099999999999995,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.55000000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72499999999999998,"5min":0.996,"15min":0.996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81100000000000005,"5min":0.89800000000000002,"15min":1.171},"last":0.60699999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80000000000000004,"5min":1.006,"15min":1.361},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.751,"5min":1.0409999999999999,"15min":1.1899999999999999},"last":0.621}]}]},{"osd":6,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28304,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343536,"statfs":{"total":96636764160,"available":96607780864,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56799999999999995,"15min":0.54300000000000004},"min":{"1min":0.34599999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.018,"5min":1.159,"15min":1.159},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.75800000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.50900000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77100000000000002,"5min":0.79900000000000004,"15min":0.88900000000000001},"last":0.496},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.54100000000000004,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":1.0209999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.38300000000000001,"5min":0.376,"15min":0.33200000000000002},"max":{"1min":0.751,"5min":0.79200000000000004,"15min":0.96199999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.85099999999999998},"last":0.45900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.624,"15min":0.55800000000000005},"min":{"1min":0.46800000000000003,"5min":0.35099999999999998,"15min":0.34699999999999998},"max":{"1min":0.76200000000000001,"5min":4.8689999999999998,"15min":4.8689999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.74099999999999999,"5min":0.82599999999999996,"15min":1.1950000000000001},"last":0.433}]},{"osd":4,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.34300000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.36699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.67400000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.373},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.41699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.47399999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81100000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.501,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.71899999999999997,"5min":0.81999999999999995,"15min":0.83099999999999996},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32464,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.23000000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":0.497,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.34899999999999998},"min":{"1min":0.26500000000000001,"5min":0.20000000000000001,"15min":0.19700000000000001},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.47599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.35599999999999998,"15min":0.33900000000000002},"min":{"1min":0.247,"5min":0.22500000000000001,"15min":0.214},"max":{"1min":0.52400000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28100000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.51400000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.39400000000000002},"min":{"1min":0.29999999999999999,"5min":0.252,"15min":0.246},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.39300000000000002},"min":{"1min":0.26700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.47999999999999998,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.53000000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60299999999999998,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.88900000000000001,"5min":1.167,"15min":1.167},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40500000000000003},"max":{"1min":0.80700000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.89400000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.879,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79900000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.92600000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.48499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83199999999999996,"5min":0.91700000000000004,"15min":1.129},"last":0.878},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.46300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76600000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.46400000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.79200000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.74299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574937,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1908736,"data_stored":1164386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34599999999999997,"15min":0.35499999999999998},"min":{"1min":0.217,"5min":0.17699999999999999,"15min":0.17699999999999999},"max":{"1min":0.51900000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.307},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.36899999999999999,"15min":0.36699999999999999},"min":{"1min":0.25600000000000001,"5min":0.23999999999999999,"15min":0.23300000000000001},"max":{"1min":0.50900000000000001,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.375}]},{"osd":2,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.39000000000000001,"15min":0.38900000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.23699999999999999},"max":{"1min":0.46400000000000002,"5min":0.64300000000000002,"15min":0.65700000000000003},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.37,"15min":0.35199999999999998},"min":{"1min":0.246,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":0.48299999999999998,"5min":1.202,"15min":1.202},"last":0.39800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.39900000000000002},"min":{"1min":0.27900000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52200000000000002,"5min":0.75,"15min":0.75},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41899999999999998,"15min":0.433},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.53600000000000003,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.435}]},{"osd":4,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":1.419,"15min":1.419},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64000000000000001,"15min":0.66300000000000003},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.79800000000000004,"5min":1,"15min":1.0009999999999999},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59699999999999998,"15min":0.627},"min":{"1min":0.38200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81499999999999995,"5min":1.046,"15min":1.046},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.629},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.746,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.65700000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81699999999999995,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59599999999999997,"15min":0.61399999999999999},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81699999999999995,"5min":1.387,"15min":1.387},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61299999999999999,"15min":0.626},"min":{"1min":0.41599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76700000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.627,"15min":0.65900000000000003},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":1.087,"15min":1.087},"last":0.66900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.36499999999999999},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.59199999999999997,"5min":0.627,"15min":0.627},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.248,"5min":0.186,"15min":0.186},"max":{"1min":0.51500000000000001,"5min":0.70099999999999996,"15min":0.84599999999999997},"last":0.52000000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.374,"15min":0.38700000000000001},"min":{"1min":0.254,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52000000000000002,"5min":0.58799999999999997,"15min":0.65700000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38500000000000001,"15min":0.37},"min":{"1min":0.252,"5min":0.245,"15min":0.245},"max":{"1min":0.55900000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.58799999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36299999999999999,"15min":0.36899999999999999},"min":{"1min":0.254,"5min":0.216,"15min":0.216},"max":{"1min":0.53400000000000003,"5min":0.58799999999999997,"15min":0.68700000000000006},"last":0.499},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.42199999999999999},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.622,"5min":0.69699999999999995,"15min":0.76100000000000001},"last":0.61199999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.65600000000000003},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81799999999999995,"5min":0.85799999999999998,"15min":2.2160000000000002},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80600000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.91100000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59599999999999997,"15min":0.61599999999999999},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82899999999999996,"5min":0.90000000000000002,"15min":1.075},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":1.0309999999999999,"15min":1.2190000000000001},"last":0.81599999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.435,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90800000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89400000000000002,"5min":1.004,"15min":1.03},"last":0.86099999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92600000000000005,"5min":0.94899999999999995,"15min":1.1919999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.62},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.40999999999999998},"max":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":1.1120000000000001},"last":0.876}]}]},{"osd":3,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32464,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.35299999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.188},"max":{"1min":0.55700000000000005,"5min":0.629,"15min":1.284},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.371,"15min":0.45400000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57799999999999996,"5min":0.63600000000000001,"15min":2.8460000000000001},"last":0.42899999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.34100000000000003,"15min":0.48299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53500000000000003,"5min":0.61799999999999999,"15min":4.9109999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.39100000000000001,"15min":0.47299999999999998},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.65800000000000003,"5min":0.69399999999999995,"15min":2.8940000000000001},"last":0.36199999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.41999999999999998,"15min":0.48599999999999999},"min":{"1min":0.27900000000000003,"5min":0.224,"15min":0.224},"max":{"1min":0.626,"5min":0.626,"15min":2.9790000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38300000000000001,"15min":0.41699999999999998},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.23100000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":1.3440000000000001},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.66000000000000003},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":2.9500000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60899999999999999,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.405},"last":0.753}]},{"osd":5,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":0.629},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.377},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":4.8129999999999997},"last":0.78300000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61699999999999999,"15min":0.66000000000000003},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.4319999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.75700000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":4.883},"last":0.68400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.77200000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":4.8499999999999996},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":2.923},"last":0.70499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.65600000000000003,"5min":0.79400000000000004,"15min":2.9510000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54600000000000004,"15min":0.53100000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70899999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55900000000000005,"15min":0.52100000000000002},"min":{"1min":0.40600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68700000000000006,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.64200000000000002},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.91100000000000003,"15min":2.9820000000000002},"last":0.60999999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77400000000000002,"5min":2.113,"15min":2.113},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.65700000000000003},"min":{"1min":0.40400000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":3.0030000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57499999999999996,"15min":0.54700000000000004},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.33300000000000002},"max":{"1min":0.67600000000000005,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60899999999999999,"15min":0.58499999999999996},"min":{"1min":0.41899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85199999999999998,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.44700000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80900000000000005,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55800000000000005,"15min":0.56299999999999994},"min":{"1min":0.35899999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78100000000000003,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.42499999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.41199999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.373},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.84799999999999998},"last":0.747}]},{"osd":7,"last update":"Sun Nov 26 07:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58899999999999997,"15min":0.59299999999999997},"min":{"1min":0.38700000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82999999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.55300000000000005,"5min":0.315,"15min":0.315},"max":{"1min":0.91400000000000003,"5min":1.01,"15min":1.01},"last":0.49099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":574018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.29799999999999999,"15min":0.27500000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.54700000000000004},"min":{"1min":0.39800000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96199999999999997,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.56599999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79400000000000004,"5min":1.091,"15min":1.091},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.38}]},{"osd":2,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.42399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.54200000000000004},"min":{"1min":0.44400000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58399999999999996,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61499999999999999,"15min":0.57999999999999996},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.57899999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.35599999999999998,"15min":0.311},"max":{"1min":1.044,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.443},{"interface":"front","average":{"1min":0.623,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.432,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0980000000000001,"5min":1.161,"15min":1.161},"last":0.52100000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.35199999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.48899999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.38200000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0129999999999999,"5min":1.196,"15min":1.196},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.628,"15min":0.64500000000000002},"min":{"1min":0.40699999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0309999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.503}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:18:17.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-26T07:18:17.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.867+0000 7f229d3d3700 1 Processor -- start 2023-11-26T07:18:17.870 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.867+0000 7f229d3d3700 1 -- start start 2023-11-26T07:18:17.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f229d3d3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229815e140 0x7f22980d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f229d3d3700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22980d39d0 0x7f22980d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.872 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f229d3d3700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22980b98c0 0x7f22980b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f229d3d3700 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f229815e520 con 0x7f22980d39d0 2023-11-26T07:18:17.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f229d3d3700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22980d2930 con 0x7f22980b98c0 2023-11-26T07:18:17.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f229d3d3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22980d2c10 con 0x7f229815e140 2023-11-26T07:18:17.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f22967fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22980d39d0 0x7f22980d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f22967fc700 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22980d39d0 0x7f22980d4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42832/0 (socket says 172.21.15.83:42832) 2023-11-26T07:18:17.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f22967fc700 1 -- 172.21.15.83:0/2400293232 learned_addr learned my addr 172.21.15.83:0/2400293232 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:18:17.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f22977fe700 1 --2- 172.21.15.83:0/2400293232 >> v2:172.21.15.203:3300/0 conn(0x7f22980b98c0 0x7f22980b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f2296ffd700 1 --2- 172.21.15.83:0/2400293232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229815e140 0x7f22980d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.873 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f22967fc700 1 -- 172.21.15.83:0/2400293232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229815e140 msgr2=0x7f22980d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:17.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f22967fc700 1 --2- 172.21.15.83:0/2400293232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229815e140 0x7f22980d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f22967fc700 1 -- 172.21.15.83:0/2400293232 >> v2:172.21.15.203:3300/0 conn(0x7f22980b98c0 msgr2=0x7f22980b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:17.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f22967fc700 1 --2- 172.21.15.83:0/2400293232 >> v2:172.21.15.203:3300/0 conn(0x7f22980b98c0 0x7f22980b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f22967fc700 1 -- 172.21.15.83:0/2400293232 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22980d2ef0 con 0x7f22980d39d0 2023-11-26T07:18:17.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.868+0000 7f22967fc700 1 --2- 172.21.15.83:0/2400293232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22980d39d0 0x7f22980d4400 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f22980d4f70 tx=0x7f2288009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08bec33c253f761 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f2295ffb700 1 -- 172.21.15.83:0/2400293232 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800b870 con 0x7f22980d39d0 2023-11-26T07:18:17.874 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f2295ffb700 1 -- 172.21.15.83:0/2400293232 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228800b9d0 con 0x7f22980d39d0 2023-11-26T07:18:17.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f2295ffb700 1 -- 172.21.15.83:0/2400293232 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800bd60 con 0x7f22980d39d0 2023-11-26T07:18:17.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f229d3d3700 1 -- 172.21.15.83:0/2400293232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22980d39d0 msgr2=0x7f22980d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f229d3d3700 1 --2- 172.21.15.83:0/2400293232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22980d39d0 0x7f22980d4400 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f22980d4f70 tx=0x7f2288009bd0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f229d3d3700 1 -- 172.21.15.83:0/2400293232 shutdown_connections 2023-11-26T07:18:17.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f229d3d3700 1 --2- 172.21.15.83:0/2400293232 >> v2:172.21.15.203:3300/0 conn(0x7f22980b98c0 0x7f22980b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f229d3d3700 1 --2- 172.21.15.83:0/2400293232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f229815e140 0x7f22980d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f229d3d3700 1 --2- 172.21.15.83:0/2400293232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22980d39d0 0x7f22980d4400 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.875 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f229d3d3700 1 -- 172.21.15.83:0/2400293232 >> 172.21.15.83:0/2400293232 conn(0x7f229806b410 msgr2=0x7f229806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:17.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f229d3d3700 1 -- 172.21.15.83:0/2400293232 shutdown_connections 2023-11-26T07:18:17.876 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.869+0000 7f229d3d3700 1 -- 172.21.15.83:0/2400293232 wait complete. 2023-11-26T07:18:17.878 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.871+0000 7f229d3d3700 1 Processor -- start 2023-11-26T07:18:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.871+0000 7f229d3d3700 1 -- start start 2023-11-26T07:18:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.871+0000 7f229d3d3700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22980b98c0 0x7f229815c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.871+0000 7f229d3d3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22980d39d0 0x7f22980b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.871+0000 7f229d3d3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f229815e140 0x7f22980b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.871+0000 7f229d3d3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22980c8630 con 0x7f229815e140 2023-11-26T07:18:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.871+0000 7f229d3d3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22980c7cd0 con 0x7f22980d39d0 2023-11-26T07:18:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.871+0000 7f229d3d3700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22980c7f70 con 0x7f22980b98c0 2023-11-26T07:18:17.879 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.871+0000 7f22977fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f229815e140 0x7f22980b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.871+0000 7f22977fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f229815e140 0x7f22980b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42840/0 (socket says 172.21.15.83:42840) 2023-11-26T07:18:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.871+0000 7f22977fe700 1 -- 172.21.15.83:0/3519039331 learned_addr learned my addr 172.21.15.83:0/3519039331 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:18:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f22967fc700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22980d39d0 0x7f22980b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f2296ffd700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22980b98c0 0x7f229815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f22977fe700 1 -- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22980b98c0 msgr2=0x7f229815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f22977fe700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22980b98c0 0x7f229815c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f22977fe700 1 -- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22980d39d0 msgr2=0x7f22980b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f22977fe700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22980d39d0 0x7f22980b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f22977fe700 1 -- 172.21.15.83:0/3519039331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288009550 con 0x7f229815e140 2023-11-26T07:18:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f2296ffd700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22980b98c0 0x7f229815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f22977fe700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f229815e140 0x7f22980b5100 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f2290007e30 tx=0x7f229000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f227f7fe700 1 -- 172.21.15.83:0/3519039331 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229000ede0 con 0x7f229815e140 2023-11-26T07:18:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f227f7fe700 1 -- 172.21.15.83:0/3519039331 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2290014420 con 0x7f229815e140 2023-11-26T07:18:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f229d3d3700 1 -- 172.21.15.83:0/3519039331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22980c8250 con 0x7f229815e140 2023-11-26T07:18:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f227f7fe700 1 -- 172.21.15.83:0/3519039331 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22900147d0 con 0x7f229815e140 2023-11-26T07:18:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.872+0000 7f229d3d3700 1 -- 172.21.15.83:0/3519039331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22980b5640 con 0x7f229815e140 2023-11-26T07:18:17.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.873+0000 7f229d3d3700 1 -- 172.21.15.83:0/3519039331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2298068670 con 0x7f229815e140 2023-11-26T07:18:17.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.874+0000 7f227f7fe700 1 -- 172.21.15.83:0/3519039331 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f2290014930 con 0x7f229815e140 2023-11-26T07:18:17.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.874+0000 7f227f7fe700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f228406b1a0 0x7f228406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.874+0000 7f227f7fe700 1 -- 172.21.15.83:0/3519039331 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2290089f70 con 0x7f229815e140 2023-11-26T07:18:17.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.874+0000 7f2296ffd700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f228406b1a0 0x7f228406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.882 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.875+0000 7f2296ffd700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f228406b1a0 0x7f228406d660 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f228c009820 tx=0x7f228c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.883 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:17.878+0000 7f227f7fe700 1 -- 172.21.15.83:0/3519039331 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2290054f40 con 0x7f229815e140 2023-11-26T07:18:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.068+0000 7f229d3d3700 1 -- 172.21.15.83:0/3519039331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f22980d22f0 con 0x7f229815e140 2023-11-26T07:18:18.073 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.071+0000 7f227f7fe700 1 -- 172.21.15.83:0/3519039331 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f2290058560 con 0x7f229815e140 2023-11-26T07:18:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.071+0000 7f227f7fe700 1 -- 172.21.15.83:0/3519039331 <== mon.0 v2:172.21.15.83:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2290009980 con 0x7f229815e140 2023-11-26T07:18:18.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.073+0000 7f229d3d3700 1 -- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f228406b1a0 msgr2=0x7f228406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.073+0000 7f229d3d3700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f228406b1a0 0x7f228406d660 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f228c009820 tx=0x7f228c008040 comp rx=0 tx=0).stop 2023-11-26T07:18:18.076 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.073+0000 7f229d3d3700 1 -- 172.21.15.83:0/3519039331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f229815e140 msgr2=0x7f22980b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.073+0000 7f229d3d3700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f229815e140 0x7f22980b5100 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f2290007e30 tx=0x7f229000e3c0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.073+0000 7f229d3d3700 1 -- 172.21.15.83:0/3519039331 shutdown_connections 2023-11-26T07:18:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.073+0000 7f229d3d3700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7f228406b1a0 0x7f228406d660 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.073+0000 7f229d3d3700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f229815e140 0x7f22980b5100 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.073+0000 7f229d3d3700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22980b98c0 0x7f229815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.073+0000 7f229d3d3700 1 --2- 172.21.15.83:0/3519039331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22980d39d0 0x7f22980b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.073+0000 7f229d3d3700 1 -- 172.21.15.83:0/3519039331 >> 172.21.15.83:0/3519039331 conn(0x7f229806b410 msgr2=0x7f229815d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.074+0000 7f229d3d3700 1 -- 172.21.15.83:0/3519039331 shutdown_connections 2023-11-26T07:18:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.074+0000 7f229d3d3700 1 -- 172.21.15.83:0/3519039331 wait complete. 2023-11-26T07:18:18.088 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:18:18.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:18.208 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.206+0000 7efc22601700 1 Processor -- start 2023-11-26T07:18:18.209 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.206+0000 7efc22601700 1 -- start start 2023-11-26T07:18:18.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc22601700 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efc1c0b4350 0x7efc1c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc22601700 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc1c0b4820 con 0x7efc1c0b59f0 2023-11-26T07:18:18.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc22601700 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc1c0b4b00 con 0x7efc1c0b5dd0 2023-11-26T07:18:18.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc22601700 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc1c0ae9b0 con 0x7efc1c0b4350 2023-11-26T07:18:18.213 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1bfff700 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7efc1c0b59f0 0x7efc1c06db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:34652/0 (socket says 172.21.15.83:34652) 2023-11-26T07:18:18.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1bfff700 1 -- 172.21.15.83:0/2801361088 learned_addr learned my addr 172.21.15.83:0/2801361088 (peer_addr_for_me v1:172.21.15.83:0/0) 2023-11-26T07:18:18.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 <== mon.0 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025370861 0 0) 0x7efc1c0b4820 con 0x7efc1c0b59f0 2023-11-26T07:18:18.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc20b9e700 1 --2- 172.21.15.83:0/2801361088 >> v2:172.21.15.203:3300/0 conn(0x7efc1c0b4350 0x7efc1c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc040035b0 con 0x7efc1c0b59f0 2023-11-26T07:18:18.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 <== mon.0 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1261713165 0 0) 0x7efc040035b0 con 0x7efc1c0b59f0 2023-11-26T07:18:18.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 >> v2:172.21.15.203:3300/0 conn(0x7efc1c0b4350 msgr2=0x7efc1c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:18.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1affd700 1 --2- 172.21.15.83:0/2801361088 >> v2:172.21.15.203:3300/0 conn(0x7efc1c0b4350 0x7efc1c0af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.214 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 >> v1:172.21.15.203:6789/0 conn(0x7efc1c0b5dd0 legacy=0x7efc1c0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0aec90 con 0x7efc1c0b59f0 2023-11-26T07:18:18.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 <== mon.0 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4019073588 0 0) 0x7efc10001670 con 0x7efc1c0b59f0 2023-11-26T07:18:18.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 >> v1:172.21.15.83:6789/0 conn(0x7efc1c0b59f0 legacy=0x7efc1c06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1affd700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04003a00 0x7efc04003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.207+0000 7efc1affd700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc04004850 0x7efc04004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.215 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc1affd700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc04005210 0x7efc0400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c0ae9b0 con 0x7efc04005210 2023-11-26T07:18:18.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0400c1b0 con 0x7efc04003a00 2023-11-26T07:18:18.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0400c370 con 0x7efc04004850 2023-11-26T07:18:18.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc20b9e700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc04005210 0x7efc0400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc1bfff700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04003a00 0x7efc04003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc1b7fe700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc04004850 0x7efc04004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc20b9e700 1 -- 172.21.15.83:0/2801361088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc04004850 msgr2=0x7efc04004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:18.216 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc20b9e700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc04004850 0x7efc04004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc20b9e700 1 -- 172.21.15.83:0/2801361088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04003a00 msgr2=0x7efc04003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:18.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc20b9e700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04003a00 0x7efc04003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc20b9e700 1 -- 172.21.15.83:0/2801361088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc040035b0 con 0x7efc04005210 2023-11-26T07:18:18.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc1b7fe700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc04004850 0x7efc04004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:18.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc20b9e700 1 -- 172.21.15.83:0/2801361088 reap_dead start 2023-11-26T07:18:18.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc20b9e700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc04005210 0x7efc0400ba10 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7efc14009fc0 tx=0x7efc04005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d85ec7970bcf19 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14003a20 con 0x7efc04005210 2023-11-26T07:18:18.217 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7efc1400f040 con 0x7efc04005210 2023-11-26T07:18:18.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.208+0000 7efc1affd700 1 -- 172.21.15.83:0/2801361088 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14003e00 con 0x7efc04005210 2023-11-26T07:18:18.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.209+0000 7efc22601700 1 -- 172.21.15.83:0/2801361088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc04005210 msgr2=0x7efc0400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.209+0000 7efc22601700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc04005210 0x7efc0400ba10 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7efc14009fc0 tx=0x7efc04005880 comp rx=0 tx=0).stop 2023-11-26T07:18:18.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.209+0000 7efc22601700 1 -- 172.21.15.83:0/2801361088 shutdown_connections 2023-11-26T07:18:18.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.209+0000 7efc22601700 1 --2- 172.21.15.83:0/2801361088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc04005210 0x7efc0400ba10 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.209+0000 7efc22601700 1 -- 172.21.15.83:0/2801361088 >> 172.21.15.83:0/2801361088 conn(0x7efc1c072990 msgr2=0x7efc1c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.218 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.209+0000 7efc22601700 1 -- 172.21.15.83:0/2801361088 shutdown_connections 2023-11-26T07:18:18.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.209+0000 7efc22601700 1 -- 172.21.15.83:0/2801361088 wait complete. 2023-11-26T07:18:18.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.210+0000 7efc22601700 1 Processor -- start 2023-11-26T07:18:18.219 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.210+0000 7efc22601700 1 -- start start 2023-11-26T07:18:18.221 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc22601700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04005210 0x7efc1c0ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc22601700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc1c0ccec0 0x7efc1c1434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc22601700 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc1c152df0 0x7efc1c0cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc22601700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c0cd340 con 0x7efc1c0ccec0 2023-11-26T07:18:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc22601700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c143a30 con 0x7efc04005210 2023-11-26T07:18:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc22601700 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c0ce160 con 0x7efc1c152df0 2023-11-26T07:18:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc1b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc1c0ccec0 0x7efc1c1434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc1b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc1c0ccec0 0x7efc1c1434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42858/0 (socket says 172.21.15.83:42858) 2023-11-26T07:18:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc1b7fe700 1 -- 172.21.15.83:0/835301022 learned_addr learned my addr 172.21.15.83:0/835301022 (peer_addr_for_me v2:172.21.15.83:0/0) 2023-11-26T07:18:18.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc1bfff700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04005210 0x7efc1c0ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc20b9e700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc1c152df0 0x7efc1c0cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc1b7fe700 1 -- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc1c152df0 msgr2=0x7efc1c0cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc1b7fe700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc1c152df0 0x7efc1c0cece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc1b7fe700 1 -- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04005210 msgr2=0x7efc1c0ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc1b7fe700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04005210 0x7efc1c0ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.211+0000 7efc1b7fe700 1 -- 172.21.15.83:0/835301022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0cd550 con 0x7efc1c0ccec0 2023-11-26T07:18:18.223 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.212+0000 7efc1b7fe700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc1c0ccec0 0x7efc1c1434f0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7efc0c000c00 tx=0x7efc0c00e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.212+0000 7efc18ff9700 1 -- 172.21.15.83:0/835301022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c0052e0 con 0x7efc1c0ccec0 2023-11-26T07:18:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.212+0000 7efc18ff9700 1 -- 172.21.15.83:0/835301022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7efc0c005440 con 0x7efc1c0ccec0 2023-11-26T07:18:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.212+0000 7efc22601700 1 -- 172.21.15.83:0/835301022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1c0cd830 con 0x7efc1c0ccec0 2023-11-26T07:18:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.212+0000 7efc18ff9700 1 -- 172.21.15.83:0/835301022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c0038c0 con 0x7efc1c0ccec0 2023-11-26T07:18:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.212+0000 7efc22601700 1 -- 172.21.15.83:0/835301022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1c0cddf0 con 0x7efc1c0ccec0 2023-11-26T07:18:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.213+0000 7efc22601700 1 -- 172.21.15.83:0/835301022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1c0ce440 con 0x7efc1c0ccec0 2023-11-26T07:18:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.213+0000 7efc18ff9700 1 -- 172.21.15.83:0/835301022 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) v1 ==== 89486+0+0 (secure 0 0 0) 0x7efc0c003a20 con 0x7efc1c0ccec0 2023-11-26T07:18:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.213+0000 7efc18ff9700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efc0806b1a0 0x7efc0806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.214+0000 7efc18ff9700 1 -- 172.21.15.83:0/835301022 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efc0c088eb0 con 0x7efc1c0ccec0 2023-11-26T07:18:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.214+0000 7efc1bfff700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efc0806b1a0 0x7efc0806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.214+0000 7efc1bfff700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efc0806b1a0 0x7efc0806d660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc10001eb0 tx=0x7efc10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.218+0000 7efc18ff9700 1 -- 172.21.15.83:0/835301022 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc0c053ea0 con 0x7efc1c0ccec0 2023-11-26T07:18:18.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.450+0000 7efc22601700 1 -- 172.21.15.83:0/835301022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc1c0ce2a0 con 0x7efc1c0ccec0 2023-11-26T07:18:18.453 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.450+0000 7efc18ff9700 1 -- 172.21.15.83:0/835301022 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+3702 (secure 0 0 0) 0x7efc1c0ce2a0 con 0x7efc1c0ccec0 2023-11-26T07:18:18.453 INFO:teuthology.orchestra.run.smithi083.stdout: 2023-11-26T07:18:18.454 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6834/136760159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":136760159},{"type":"v1","addr":"172.21.15.203:6834","nonce":136760159}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":6691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3723980912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6833","nonce":3723980912},{"type":"v1","addr":"172.21.15.203:6835","nonce":3723980912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7527,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/3900154869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3900154869},{"type":"v1","addr":"172.21.15.83:6833","nonce":3900154869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8004,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2371515513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":2371515513},{"type":"v1","addr":"172.21.15.83:6835","nonce":2371515513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[]} 2023-11-26T07:18:18.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 -- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efc0806b1a0 msgr2=0x7efc0806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efc0806b1a0 0x7efc0806d660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc10001eb0 tx=0x7efc10009040 comp rx=0 tx=0).stop 2023-11-26T07:18:18.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 -- 172.21.15.83:0/835301022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc1c0ccec0 msgr2=0x7efc1c1434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc1c0ccec0 0x7efc1c1434f0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7efc0c000c00 tx=0x7efc0c00e830 comp rx=0 tx=0).stop 2023-11-26T07:18:18.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 -- 172.21.15.83:0/835301022 shutdown_connections 2023-11-26T07:18:18.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:6836/866214038,v1:172.21.15.203:6837/866214038] conn(0x7efc0806b1a0 0x7efc0806d660 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.455 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc1c152df0 0x7efc1c0cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc04005210 0x7efc1c0ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 --2- 172.21.15.83:0/835301022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc1c0ccec0 0x7efc1c1434f0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 -- 172.21.15.83:0/835301022 >> 172.21.15.83:0/835301022 conn(0x7efc1c072990 msgr2=0x7efc1c159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 -- 172.21.15.83:0/835301022 shutdown_connections 2023-11-26T07:18:18.456 INFO:teuthology.orchestra.run.smithi083.stderr:2023-11-26T07:18:18.452+0000 7efc22601700 1 -- 172.21.15.83:0/835301022 wait complete. 2023-11-26T07:18:18.457 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 72 2023-11-26T07:18:18.467 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-26T07:18:18.467 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-26T07:18:18.468 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:18:18.468 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:18.476 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:18:18.476 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:18:18.476 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:18.524 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:18:18.525 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:18:18.525 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:18.537 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:18:18.537 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:18:18.537 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:18.588 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:18:18.588 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-26T07:18:18.588 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-26T07:18:18.588 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:18.619 INFO:tasks.ceph.osd.0:Stopped 2023-11-26T07:18:18.619 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-26T07:18:18.619 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:18.698 INFO:tasks.ceph.osd.1:Stopped 2023-11-26T07:18:18.698 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-26T07:18:18.698 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:18.772 INFO:tasks.ceph.osd.2:Stopped 2023-11-26T07:18:18.773 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-26T07:18:18.773 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:18.850 INFO:tasks.ceph.osd.3:Stopped 2023-11-26T07:18:18.851 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-26T07:18:18.851 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:18.888 INFO:tasks.ceph.osd.4:Stopped 2023-11-26T07:18:18.889 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-26T07:18:18.889 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:18.970 INFO:tasks.ceph.osd.5:Stopped 2023-11-26T07:18:18.971 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-26T07:18:18.971 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:19.043 INFO:tasks.ceph.osd.6:Stopped 2023-11-26T07:18:19.044 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-26T07:18:19.044 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:19.123 INFO:tasks.ceph.osd.7:Stopped 2023-11-26T07:18:19.123 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-26T07:18:19.123 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-26T07:18:19.123 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:19.147 INFO:tasks.ceph.mgr.y:Stopped 2023-11-26T07:18:19.148 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-26T07:18:19.148 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:19.191 INFO:tasks.ceph.mgr.x:Stopped 2023-11-26T07:18:19.191 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-26T07:18:19.191 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-26T07:18:19.191 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:19.202 INFO:tasks.ceph.mon.a:Stopped 2023-11-26T07:18:19.202 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-26T07:18:19.203 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:19.243 INFO:tasks.ceph.mon.b:Stopped 2023-11-26T07:18:19.243 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-26T07:18:19.244 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:19.295 INFO:tasks.ceph.mon.c:Stopped 2023-11-26T07:18:19.295 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-26T07:18:19.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-26T07:18:19.328 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:18:19.328 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-26T07:18:19.594 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:18:19.595 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-26T07:18:19.682 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:18:19.682 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-26T07:18:19.757 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:18:19.758 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-26T07:18:19.829 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:18:19.829 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-26T07:18:20.025 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:18:20.026 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-26T07:18:20.125 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:18:20.125 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-26T07:18:20.223 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:18:20.224 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-26T07:18:20.303 INFO:tasks.ceph:Archiving mon data... 2023-11-26T07:18:20.303 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353/data/mon.a.tgz 2023-11-26T07:18:20.304 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2023-11-26T07:18:20.321 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.VYu82bgNUl 2023-11-26T07:18:20.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.VYu82bgNUl 2023-11-26T07:18:20.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.VYu82bgNUl 2023-11-26T07:18:20.604 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.VYu82bgNUl is 779KB 2023-11-26T07:18:20.707 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.VYu82bgNUl 2023-11-26T07:18:20.724 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353/data/mon.b.tgz 2023-11-26T07:18:20.724 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2023-11-26T07:18:20.742 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.NV8h4fJQV3 2023-11-26T07:18:20.742 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.NV8h4fJQV3 2023-11-26T07:18:20.961 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.NV8h4fJQV3 2023-11-26T07:18:21.046 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.NV8h4fJQV3 is 800KB 2023-11-26T07:18:21.147 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.NV8h4fJQV3 2023-11-26T07:18:21.163 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353/data/mon.c.tgz 2023-11-26T07:18:21.164 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2023-11-26T07:18:21.219 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.SqcUzgofJN 2023-11-26T07:18:21.219 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.SqcUzgofJN 2023-11-26T07:18:21.411 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.SqcUzgofJN 2023-11-26T07:18:21.498 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.SqcUzgofJN is 800KB 2023-11-26T07:18:21.573 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.SqcUzgofJN 2023-11-26T07:18:21.589 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-26T07:18:21.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-26T07:18:21.591 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-26T07:18:21.655 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-26T07:18:21.655 DEBUG:teuthology.orchestra.run.smithi083:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-26T07:18:21.661 DEBUG:teuthology.orchestra.run.smithi203:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-26T07:18:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-26T07:18:21.727 INFO:teuthology.orchestra.run.smithi203.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-26T07:18:21.733 INFO:tasks.ceph:Archiving crash dumps... 2023-11-26T07:18:21.734 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353/remote/smithi083/crash 2023-11-26T07:18:21.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-26T07:18:21.762 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353/remote/smithi203/crash 2023-11-26T07:18:21.763 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-26T07:18:21.798 INFO:tasks.ceph:Shutting down logrotate 2023-11-26T07:18:21.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:21.806 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:21.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:21.924 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:21.950 INFO:tasks.ceph:Compressing logs... 2023-11-26T07:18:21.951 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-26T07:18:21.966 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-26T07:18:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138184.log 2023-11-26T07:18:21.991 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110552.log 2023-11-26T07:18:21.992 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.138184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126997.log 2023-11-26T07:18:22.033 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103533.log 2023-11-26T07:18:22.033 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.110295.log 2023-11-26T07:18:22.033 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.103533.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-26T07:18:22.033 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.110295.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103533.log.gz 2023-11-26T07:18:22.034 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-26T07:18:22.034 INFO:teuthology.orchestra.run.smithi203.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.110295.log.gz 2023-11-26T07:18:22.034 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.112321.log 2023-11-26T07:18:22.034 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103396.log 2023-11-26T07:18:22.034 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.112321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.110287.log 2023-11-26T07:18:22.034 INFO:teuthology.orchestra.run.smithi203.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.112321.log.gz 2023-11-26T07:18:22.034 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.103396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103349.log 2023-11-26T07:18:22.035 INFO:teuthology.orchestra.run.smithi203.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103396.log.gz 2023-11-26T07:18:22.035 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.110287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.110287.log.gz 2023-11-26T07:18:22.035 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.110906.log 2023-11-26T07:18:22.035 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-26T07:18:22.035 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.103349.log: /var/log/ceph/ceph-client.3.110906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.111705.log 2023-11-26T07:18:22.035 INFO:teuthology.orchestra.run.smithi203.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.110906.log.gz 2023-11-26T07:18:22.035 INFO:teuthology.orchestra.run.smithi203.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103349.log.gz 2023-11-26T07:18:22.035 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.115000.log 2023-11-26T07:18:22.035 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.111705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111705.log.gz 2023-11-26T07:18:22.036 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.113030.log 2023-11-26T07:18:22.036 INFO:teuthology.orchestra.run.smithi203.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-26T07:18:22.036 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103676.log 2023-11-26T07:18:22.036 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.115000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.115000.log.gz 2023-11-26T07:18:22.036 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.113030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.113030.log.gz 2023-11-26T07:18:22.036 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103578.log 2023-11-26T07:18:22.036 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.103676.log: -- /var/log/ceph/ceph-client.1.111701.log 2023-11-26T07:18:22.036 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.103578.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103676.log.gz 2023-11-26T07:18:22.037 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-26T07:18:22.037 INFO:teuthology.orchestra.run.smithi203.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.103578.log.gz 2023-11-26T07:18:22.037 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.111701.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-26T07:18:22.037 INFO:teuthology.orchestra.run.smithi203.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111701.log.gz 2023-11-26T07:18:22.037 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.114366.log 2023-11-26T07:18:22.037 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-26T07:18:22.037 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.114366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.114366.log.gz 2023-11-26T07:18:22.037 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.113622.log 2023-11-26T07:18:22.038 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-26T07:18:22.038 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.113622.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.113622.log.gz 2023-11-26T07:18:22.038 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mgr.x.log: 93.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-26T07:18:22.038 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.114362.log 2023-11-26T07:18:22.038 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103488.log 2023-11-26T07:18:22.038 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.114362.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.114362.log.gz 2023-11-26T07:18:22.038 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.103488.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103488.log.gz 2023-11-26T07:18:22.039 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-26T07:18:22.039 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.115004.log 2023-11-26T07:18:22.039 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.1.115004.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.115004.log.gz 2023-11-26T07:18:22.039 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.110902.log 2023-11-26T07:18:22.039 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.3.110902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.110902.log.gz 2023-11-26T07:18:22.039 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103623.log 2023-11-26T07:18:22.039 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.103623.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103623.log.gz 2023-11-26T07:18:22.039 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-26T07:18:22.041 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.112317.log 2023-11-26T07:18:22.041 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.112317.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.112317.log.gz 2023-11-26T07:18:22.041 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-26T07:18:22.041 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.113034.log 2023-11-26T07:18:22.059 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.113034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.113034.log.gz 2023-11-26T07:18:22.059 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-26T07:18:22.070 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103441.log 2023-11-26T07:18:22.073 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110552.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.138184.log.gz 2023-11-26T07:18:22.073 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123029.log 2023-11-26T07:18:22.073 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110552.log.gz 2023-11-26T07:18:22.073 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.126997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111886.log 2023-11-26T07:18:22.073 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123029.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126997.log.gz 2023-11-26T07:18:22.073 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129650.log 2023-11-26T07:18:22.073 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111886.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123029.log.gz 2023-11-26T07:18:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111495.log 2023-11-26T07:18:22.074 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111886.log.gz 2023-11-26T07:18:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.129650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134708.log 2023-11-26T07:18:22.074 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129650.log.gz 2023-11-26T07:18:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124969.log 2023-11-26T07:18:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111495.log: /var/log/ceph/ceph-client.admin.134708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125107.log 2023-11-26T07:18:22.074 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111495.log.gz 2023-11-26T07:18:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124969.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134708.log.gz 2023-11-26T07:18:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138138.log 2023-11-26T07:18:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125107.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124969.log.gz 2023-11-26T07:18:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121672.log 2023-11-26T07:18:22.075 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125107.log.gz 2023-11-26T07:18:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.138138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129558.log 2023-11-26T07:18:22.075 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138138.log.gz 2023-11-26T07:18:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112070.log 2023-11-26T07:18:22.075 INFO:teuthology.orchestra.run.smithi083.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121672.log.gz 2023-11-26T07:18:22.076 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.129558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134313.log 2023-11-26T07:18:22.076 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.112070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142283.log 2023-11-26T07:18:22.076 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129558.log.gz 2023-11-26T07:18:22.076 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.134313.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112070.log.gz 2023-11-26T07:18:22.076 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133531.log 2023-11-26T07:18:22.076 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.134313.log.gz 2023-11-26T07:18:22.076 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128860.log 2023-11-26T07:18:22.076 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142283.log: /var/log/ceph/ceph-client.admin.133531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125761.log 2023-11-26T07:18:22.077 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142283.log.gz 2023-11-26T07:18:22.077 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128860.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.133531.log.gz 2023-11-26T07:18:22.077 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123470.log 2023-11-26T07:18:22.077 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125761.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.128860.log.gz 2023-11-26T07:18:22.077 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131932.log 2023-11-26T07:18:22.077 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123470.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125761.log.gz 2023-11-26T07:18:22.077 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113036.log 2023-11-26T07:18:22.077 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.131932.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123470.log.gz 2023-11-26T07:18:22.078 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122537.log 2023-11-26T07:18:22.078 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.131932.log.gz 2023-11-26T07:18:22.078 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.113036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124766.log 2023-11-26T07:18:22.078 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113036.log.gz 2023-11-26T07:18:22.078 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117864.log 2023-11-26T07:18:22.078 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141193.log 2023-11-26T07:18:22.078 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122537.log.gz 2023-11-26T07:18:22.079 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117864.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124766.log.gz 2023-11-26T07:18:22.079 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133853.log 2023-11-26T07:18:22.079 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141193.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.117864.log.gz 2023-11-26T07:18:22.079 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114007.log 2023-11-26T07:18:22.079 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133853.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141193.log.gz 2023-11-26T07:18:22.079 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140652.log 2023-11-26T07:18:22.079 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.133853.log.gz 2023-11-26T07:18:22.079 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142421.log 2023-11-26T07:18:22.080 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114007.log: /var/log/ceph/ceph-client.admin.140652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116697.log 2023-11-26T07:18:22.080 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114007.log.gz 2023-11-26T07:18:22.080 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142421.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.140652.log.gz 2023-11-26T07:18:22.080 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121834.log 2023-11-26T07:18:22.080 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.142421.log.gz 2023-11-26T07:18:22.080 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142881.log 2023-11-26T07:18:22.080 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116697.log.gz 2023-11-26T07:18:22.080 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118599.log 2023-11-26T07:18:22.081 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121834.log.gz 2023-11-26T07:18:22.081 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114422.log 2023-11-26T07:18:22.081 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.118599.log: 87.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.142881.log.gz--verbose 2023-11-26T07:18:22.081 INFO:teuthology.orchestra.run.smithi083.stderr:-- /var/log/ceph/ceph-client.admin.129420.log 2023-11-26T07:18:22.081 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114422.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118599.log.gz 2023-11-26T07:18:22.081 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126538.log 2023-11-26T07:18:22.081 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.114422.log.gz 2023-11-26T07:18:22.082 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123330.log 2023-11-26T07:18:22.082 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.129420.log: /var/log/ceph/ceph-client.admin.126538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122071.log 2023-11-26T07:18:22.082 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.129420.log.gz 2023-11-26T07:18:22.082 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143279.log 2023-11-26T07:18:22.082 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123330.log.gz 2023-11-26T07:18:22.082 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.122071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120893.log 2023-11-26T07:18:22.082 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.122071.log.gz 2023-11-26T07:18:22.082 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.126538.log.gz 2023-11-26T07:18:22.083 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.143279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137250.log 2023-11-26T07:18:22.083 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120893.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143279.log.gz 2023-11-26T07:18:22.083 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138841.log 2023-11-26T07:18:22.083 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120893.log.gz 2023-11-26T07:18:22.083 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117567.log 2023-11-26T07:18:22.083 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.137250.log: /var/log/ceph/ceph-client.admin.138841.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-26T07:18:22.083 INFO:teuthology.orchestra.run.smithi083.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.137250.log.gz 2023-11-26T07:18:22.084 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.138841.log.gz 2023-11-26T07:18:22.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134664.log 2023-11-26T07:18:22.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.2.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.117567.log.gzgzip 2023-11-26T07:18:22.084 INFO:teuthology.orchestra.run.smithi083.stderr:-5 --verbose -- /var/log/ceph/ceph-client.0.130569.log 2023-11-26T07:18:22.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.134664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124628.log 2023-11-26T07:18:22.084 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.134664.log.gz 2023-11-26T07:18:22.084 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.130569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.130569.log.gz 2023-11-26T07:18:22.084 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133761.log 2023-11-26T07:18:22.085 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131672.log 2023-11-26T07:18:22.085 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124628.log: /var/log/ceph/ceph-client.admin.133761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103229.log 2023-11-26T07:18:22.085 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124628.log.gz 2023-11-26T07:18:22.085 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.131672.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.133761.log.gz 2023-11-26T07:18:22.085 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119595.log 2023-11-26T07:18:22.085 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141626.log 2023-11-26T07:18:22.085 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.103229.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.131672.log.gz 2023-11-26T07:18:22.085 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141764.log 2023-11-26T07:18:22.086 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103229.log.gz 2023-11-26T07:18:22.086 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141626.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119595.log.gzgzip 2023-11-26T07:18:22.086 INFO:teuthology.orchestra.run.smithi083.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.111380.log 2023-11-26T07:18:22.086 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.141626.log.gz 2023-11-26T07:18:22.086 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143165.log 2023-11-26T07:18:22.086 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141764.log.gz 2023-11-26T07:18:22.086 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116559.log 2023-11-26T07:18:22.086 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.143165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.137649.log 2023-11-26T07:18:22.087 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111380.log.gz 2023-11-26T07:18:22.087 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134620.log 2023-11-26T07:18:22.087 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.143165.log.gz 2023-11-26T07:18:22.087 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116559.log.gz 2023-11-26T07:18:22.087 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.137649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114660.log 2023-11-26T07:18:22.087 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.137649.log.gz 2023-11-26T07:18:22.087 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.134620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128814.log 2023-11-26T07:18:22.087 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114660.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.134620.log.gz 2023-11-26T07:18:22.088 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124876.log 2023-11-26T07:18:22.088 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142329.log 2023-11-26T07:18:22.088 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114660.log.gz 2023-11-26T07:18:22.088 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128814.log.gz 2023-11-26T07:18:22.088 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130721.log 2023-11-26T07:18:22.088 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124876.log.gz 2023-11-26T07:18:22.088 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115448.log 2023-11-26T07:18:22.089 INFO:teuthology.orchestra.run.smithi083.stderr: 87.3%/var/log/ceph/ceph-client.admin.130721.log: -- replaced with /var/log/ceph/ceph-client.admin.142329.log.gz 2023-11-26T07:18:22.089 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125809.log 2023-11-26T07:18:22.089 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.115448.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130721.log.gz 2023-11-26T07:18:22.089 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126492.log 2023-11-26T07:18:22.089 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125809.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115448.log.gz 2023-11-26T07:18:22.089 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134359.log 2023-11-26T07:18:22.089 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125809.log.gz 2023-11-26T07:18:22.089 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.126492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128611.log 2023-11-26T07:18:22.090 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.134359.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.126492.log.gz 2023-11-26T07:18:22.090 INFO:teuthology.orchestra.run.smithi083.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.137342.log 2023-11-26T07:18:22.090 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128611.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134359.log.gz 2023-11-26T07:18:22.090 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125153.log 2023-11-26T07:18:22.090 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.128611.log.gz 2023-11-26T07:18:22.090 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.137342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136638.log 2023-11-26T07:18:22.090 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142375.log 2023-11-26T07:18:22.090 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136638.log: 91.3% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125153.log.gz -- replaced with /var/log/ceph/ceph-client.admin.137342.log.gz 2023-11-26T07:18:22.091 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-11-26T07:18:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134975.log 2023-11-26T07:18:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120846.log 2023-11-26T07:18:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142375.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136638.log.gz 2023-11-26T07:18:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.134975.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-26T07:18:22.091 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142375.log.gz 2023-11-26T07:18:22.091 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134975.log.gz 2023-11-26T07:18:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129466.log 2023-11-26T07:18:22.091 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117103.log 2023-11-26T07:18:22.092 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.120846.log.gz 2023-11-26T07:18:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.129466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138973.log 2023-11-26T07:18:22.092 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129466.log.gz 2023-11-26T07:18:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116789.log 2023-11-26T07:18:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132085.log 2023-11-26T07:18:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.138973.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.117103.log.gz 2023-11-26T07:18:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116789.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138973.log.gz 2023-11-26T07:18:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122809.log 2023-11-26T07:18:22.093 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116789.log.gz 2023-11-26T07:18:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142013.log 2023-11-26T07:18:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132085.log: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-26T07:18:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125963.log 2023-11-26T07:18:22.093 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132085.log.gz 2023-11-26T07:18:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142013.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122809.log.gz 2023-11-26T07:18:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103670.log 2023-11-26T07:18:22.093 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7%/var/log/ceph/ceph-client.admin.125963.log: -- replaced with /var/log/ceph/ceph-client.admin.142013.log.gz 2023-11-26T07:18:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137801.log 2023-11-26T07:18:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.103670.log: gzip 89.1%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.125963.log.gz 2023-11-26T07:18:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:-- /var/log/ceph/ceph-client.admin.135726.log 2023-11-26T07:18:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.137801.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103670.log.gz 2023-11-26T07:18:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109837.log 2023-11-26T07:18:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129940.log 2023-11-26T07:18:22.094 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.137801.log.gz 2023-11-26T07:18:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.109837.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135726.log.gz 2023-11-26T07:18:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114237.log 2023-11-26T07:18:22.095 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4%/var/log/ceph/ceph-client.admin.129940.log: -- replaced with /var/log/ceph/ceph-client.admin.109837.log.gz 2023-11-26T07:18:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111065.log 2023-11-26T07:18:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114237.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129940.log.gz 2023-11-26T07:18:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136094.log 2023-11-26T07:18:22.095 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114237.log.gz 2023-11-26T07:18:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141013.log 2023-11-26T07:18:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136094.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111065.log.gzgzip 2023-11-26T07:18:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.110368.log 2023-11-26T07:18:22.096 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136094.log.gz 2023-11-26T07:18:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127885.log 2023-11-26T07:18:22.096 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141013.log.gz 2023-11-26T07:18:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135496.log 2023-11-26T07:18:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.127885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142605.log 2023-11-26T07:18:22.096 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110368.log.gz 2023-11-26T07:18:22.096 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127885.log.gz 2023-11-26T07:18:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117818.log 2023-11-26T07:18:22.097 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.135496.log.gz 2023-11-26T07:18:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115689.log 2023-11-26T07:18:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117818.log: gzip 88.7%-5 -- replaced with /var/log/ceph/ceph-client.admin.142605.log.gz--verbose 2023-11-26T07:18:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:-- /var/log/ceph/ceph-client.admin.120379.log 2023-11-26T07:18:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.115689.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117818.log.gz 2023-11-26T07:18:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110184.log 2023-11-26T07:18:22.097 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115689.log.gz 2023-11-26T07:18:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123709.log 2023-11-26T07:18:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110184.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120379.log.gz 2023-11-26T07:18:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140829.log 2023-11-26T07:18:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134472.log 2023-11-26T07:18:22.098 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110184.log.gz 2023-11-26T07:18:22.098 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6%/var/log/ceph/ceph-client.admin.140829.log: -- replaced with /var/log/ceph/ceph-client.admin.123709.log.gz 2023-11-26T07:18:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140921.log 2023-11-26T07:18:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.134472.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140829.log.gz 2023-11-26T07:18:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113915.log 2023-11-26T07:18:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.140921.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134472.log.gz 2023-11-26T07:18:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115402.log 2023-11-26T07:18:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.113915.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140921.log.gz 2023-11-26T07:18:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122067.log 2023-11-26T07:18:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.115402.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113915.log.gz 2023-11-26T07:18:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128906.log 2023-11-26T07:18:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.122067.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.115402.log.gz-5 2023-11-26T07:18:22.099 INFO:teuthology.orchestra.run.smithi083.stderr:--verbose -- /var/log/ceph/ceph-client.admin.116329.log 2023-11-26T07:18:22.100 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.122067.log.gz 2023-11-26T07:18:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.130565.log 2023-11-26T07:18:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116329.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128906.log.gz 2023-11-26T07:18:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131081.log 2023-11-26T07:18:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.130565.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116329.log.gz 2023-11-26T07:18:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122583.log 2023-11-26T07:18:22.100 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.130565.log.gz 2023-11-26T07:18:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.131081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117425.log 2023-11-26T07:18:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117056.log 2023-11-26T07:18:22.101 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.131081.log.gz 2023-11-26T07:18:22.101 INFO:teuthology.orchestra.run.smithi083.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.122583.log.gz 2023-11-26T07:18:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130124.log 2023-11-26T07:18:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117056.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117425.log.gz 2023-11-26T07:18:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129136.log 2023-11-26T07:18:22.101 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128657.log 2023-11-26T07:18:22.101 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117056.log.gz 2023-11-26T07:18:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.129136.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.130124.log.gz 2023-11-26T07:18:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131556.log 2023-11-26T07:18:22.102 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129136.log.gz 2023-11-26T07:18:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142789.log 2023-11-26T07:18:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.131556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116191.log 2023-11-26T07:18:22.102 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.128657.log.gz 2023-11-26T07:18:22.102 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131556.log.gz 2023-11-26T07:18:22.102 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124674.log 2023-11-26T07:18:22.103 INFO:teuthology.orchestra.run.smithi083.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.142789.log.gz 2023-11-26T07:18:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122971.log 2023-11-26T07:18:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124674.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116191.log.gz 2023-11-26T07:18:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133969.log 2023-11-26T07:18:22.103 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124674.log.gz/var/log/ceph/ceph-client.admin.122971.log: 2023-11-26T07:18:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125871.log 2023-11-26T07:18:22.103 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122971.log.gz 2023-11-26T07:18:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110751.log 2023-11-26T07:18:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125871.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133969.log.gz 2023-11-26T07:18:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-26T07:18:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110751.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.125871.log.gz 2023-11-26T07:18:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142697.log 2023-11-26T07:18:22.104 INFO:teuthology.orchestra.run.smithi083.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110751.log.gz 2023-11-26T07:18:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128703.log 2023-11-26T07:18:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120579.log 2023-11-26T07:18:22.104 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142697.log.gz 2023-11-26T07:18:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.134206.log 2023-11-26T07:18:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143073.log 2023-11-26T07:18:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120579.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.128703.log.gz 2023-11-26T07:18:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.134206.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120579.log.gz 2023-11-26T07:18:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127485.log 2023-11-26T07:18:22.105 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.134206.log.gz 2023-11-26T07:18:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.143073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116099.log 2023-11-26T07:18:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.127485.log: gzip 87.8%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.143073.log.gz-- 2023-11-26T07:18:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141396.log 2023-11-26T07:18:22.106 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127485.log.gz 2023-11-26T07:18:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130897.log 2023-11-26T07:18:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141396.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116099.log.gz 2023-11-26T07:18:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112300.log 2023-11-26T07:18:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130897.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141396.log.gz 2023-11-26T07:18:22.106 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121488.log 2023-11-26T07:18:22.106 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130897.log.gz 2023-11-26T07:18:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.112300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114376.log 2023-11-26T07:18:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109791.log 2023-11-26T07:18:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121488.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112300.log.gz 2023-11-26T07:18:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114376.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121488.log.gz 2023-11-26T07:18:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136972.log 2023-11-26T07:18:22.107 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114376.log.gz 2023-11-26T07:18:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.109791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115130.log 2023-11-26T07:18:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126608.log 2023-11-26T07:18:22.108 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.109791.log.gz 2023-11-26T07:18:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.115130.log: 88.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.136972.log.gz--verbose 2023-11-26T07:18:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:-- /var/log/ceph/ceph-client.admin.141580.log 2023-11-26T07:18:22.108 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115130.log.gz 2023-11-26T07:18:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.126608.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-26T07:18:22.108 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126608.log.gz 2023-11-26T07:18:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115961.log 2023-11-26T07:18:22.108 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141580.log: /var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128952.log 2023-11-26T07:18:22.109 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.141580.log.gz 2023-11-26T07:18:22.109 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.115961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127393.log 2023-11-26T07:18:22.109 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115961.log.gz 2023-11-26T07:18:22.109 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128952.log: /var/log/ceph/ceph-client.admin.127393.log: gzip 88.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.128952.log.gz-- 2023-11-26T07:18:22.109 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125667.log 2023-11-26T07:18:22.109 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119530.log 2023-11-26T07:18:22.109 INFO:teuthology.orchestra.run.smithi083.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.127393.log.gz 2023-11-26T07:18:22.109 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133393.log 2023-11-26T07:18:22.110 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125667.log.gz 2023-11-26T07:18:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132506.log 2023-11-26T07:18:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:gzip /var/log/ceph/ceph-client.admin.133393.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.130767.log 2023-11-26T07:18:22.110 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119530.log.gz 2023-11-26T07:18:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132506.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.133393.log.gz 2023-11-26T07:18:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127199.log 2023-11-26T07:18:22.110 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.132506.log.gz 2023-11-26T07:18:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122445.log 2023-11-26T07:18:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.127199.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130767.log.gz 2023-11-26T07:18:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122445.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127199.log.gz 2023-11-26T07:18:22.111 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122445.log.gz 2023-11-26T07:18:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110756.log 2023-11-26T07:18:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131223.log 2023-11-26T07:18:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114145.log 2023-11-26T07:18:22.111 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110756.log.gz 2023-11-26T07:18:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.131223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114614.log 2023-11-26T07:18:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125917.log 2023-11-26T07:18:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114145.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.131223.log.gz 2023-11-26T07:18:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114614.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114145.log.gz 2023-11-26T07:18:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125917.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114614.log.gz 2023-11-26T07:18:22.112 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125917.log.gz 2023-11-26T07:18:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126354.log 2023-11-26T07:18:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137020.log 2023-11-26T07:18:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.126354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110138.log 2023-11-26T07:18:22.113 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.126354.log.gz 2023-11-26T07:18:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.137020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129846.log 2023-11-26T07:18:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128222.log 2023-11-26T07:18:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110138.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137020.log.gz 2023-11-26T07:18:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.129846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125620.log 2023-11-26T07:18:22.113 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.110138.log.gz 2023-11-26T07:18:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128222.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.129846.log.gz 2023-11-26T07:18:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118858.log 2023-11-26T07:18:22.114 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128222.log.gz 2023-11-26T07:18:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111028.log 2023-11-26T07:18:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.118858.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125620.log.gzgzip 2023-11-26T07:18:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.111334.log 2023-11-26T07:18:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111028.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118858.log.gz 2023-11-26T07:18:22.114 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111028.log.gz 2023-11-26T07:18:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111334.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111334.log.gz 2023-11-26T07:18:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122491.log 2023-11-26T07:18:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111123.log 2023-11-26T07:18:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123849.log 2023-11-26T07:18:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111123.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122491.log.gz 2023-11-26T07:18:22.115 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111123.log.gz 2023-11-26T07:18:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123849.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123849.log.gz 2023-11-26T07:18:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142059.log 2023-11-26T07:18:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117379.log 2023-11-26T07:18:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142059.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-26T07:18:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117379.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142059.log.gz 2023-11-26T07:18:22.116 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117379.log.gz 2023-11-26T07:18:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109883.log 2023-11-26T07:18:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126124.log 2023-11-26T07:18:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.109883.log: /var/log/ceph/ceph-client.admin.126124.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109883.log.gz 2023-11-26T07:18:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125353.log 2023-11-26T07:18:22.116 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126124.log.gz 2023-11-26T07:18:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134929.log 2023-11-26T07:18:22.117 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133250.log 2023-11-26T07:18:22.117 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.134929.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125353.log.gz 2023-11-26T07:18:22.117 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133250.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134929.log.gz 2023-11-26T07:18:22.117 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.133250.log.gz 2023-11-26T07:18:22.117 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137412.log 2023-11-26T07:18:22.117 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125574.log 2023-11-26T07:18:22.117 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.137412.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137412.log.gz 2023-11-26T07:18:22.117 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125574.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125574.log.gz 2023-11-26T07:18:22.117 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124812.log 2023-11-26T07:18:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131035.log 2023-11-26T07:18:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124812.log: /var/log/ceph/ceph-client.admin.131035.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124812.log.gz 2023-11-26T07:18:22.118 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131035.log.gz 2023-11-26T07:18:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111794.log 2023-11-26T07:18:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123400.log 2023-11-26T07:18:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142559.log 2023-11-26T07:18:22.118 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111794.log.gz 2023-11-26T07:18:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123400.log: /var/log/ceph/ceph-client.admin.142559.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.123400.log.gz 2023-11-26T07:18:22.119 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142559.log.gz 2023-11-26T07:18:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112116.log 2023-11-26T07:18:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121534.log 2023-11-26T07:18:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.112116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121189.log 2023-11-26T07:18:22.119 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112116.log.gz 2023-11-26T07:18:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121534.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121534.log.gz 2023-11-26T07:18:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121189.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121189.log.gz 2023-11-26T07:18:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132690.log 2023-11-26T07:18:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129892.log 2023-11-26T07:18:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132690.log: /var/log/ceph/ceph-client.admin.129892.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132690.log.gz 2023-11-26T07:18:22.120 INFO:teuthology.orchestra.run.smithi083.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.129892.log.gz 2023-11-26T07:18:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119166.log 2023-11-26T07:18:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112024.log 2023-11-26T07:18:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119166.log: /var/log/ceph/ceph-client.admin.112024.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.119166.log.gz 2023-11-26T07:18:22.120 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112024.log.gz 2023-11-26T07:18:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135155.log 2023-11-26T07:18:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128519.log 2023-11-26T07:18:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135155.log: /var/log/ceph/ceph-client.admin.128519.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135155.log.gz 2023-11-26T07:18:22.121 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128519.log.gz 2023-11-26T07:18:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122901.log 2023-11-26T07:18:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116743.log 2023-11-26T07:18:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109698.log 2023-11-26T07:18:22.121 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122901.log.gz/var/log/ceph/ceph-client.admin.116743.log: 2023-11-26T07:18:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.109698.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116743.log.gz 2023-11-26T07:18:22.121 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109698.log.gz 2023-11-26T07:18:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119825.log 2023-11-26T07:18:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132387.log 2023-11-26T07:18:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129044.log 2023-11-26T07:18:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132387.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.119825.log.gz 2023-11-26T07:18:22.122 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132387.log.gz 2023-11-26T07:18:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.129044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132037.log 2023-11-26T07:18:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129698.log 2023-11-26T07:18:22.122 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129044.log.gz 2023-11-26T07:18:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141350.log 2023-11-26T07:18:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.129698.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.132037.log.gz 2023-11-26T07:18:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141350.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129698.log.gz 2023-11-26T07:18:22.123 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141350.log.gz 2023-11-26T07:18:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110752.log 2023-11-26T07:18:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135956.log 2023-11-26T07:18:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110752.log: /var/log/ceph/ceph-client.admin.135956.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110752.log.gz 2023-11-26T07:18:22.123 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135956.log.gz 2023-11-26T07:18:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131315.log 2023-11-26T07:18:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133715.log 2023-11-26T07:18:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.131315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125245.log 2023-11-26T07:18:22.124 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.131315.log.gz 2023-11-26T07:18:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133715.log: /var/log/ceph/ceph-client.admin.125245.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133715.log.gz 2023-11-26T07:18:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114773.log 2023-11-26T07:18:22.124 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125245.log.gz 2023-11-26T07:18:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136826.log 2023-11-26T07:18:22.124 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122315.log 2023-11-26T07:18:22.125 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.114773.log.gz 2023-11-26T07:18:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136826.log: /var/log/ceph/ceph-client.admin.122315.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136826.log.gz 2023-11-26T07:18:22.125 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122315.log.gz 2023-11-26T07:18:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133485.log 2023-11-26T07:18:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119438.log 2023-11-26T07:18:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133485.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.133485.log.gz 2023-11-26T07:18:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119438.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119438.log.gz 2023-11-26T07:18:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128767.log 2023-11-26T07:18:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129512.log 2023-11-26T07:18:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128767.log: /var/log/ceph/ceph-client.admin.129512.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128767.log.gz 2023-11-26T07:18:22.126 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129512.log.gz 2023-11-26T07:18:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110230.log 2023-11-26T07:18:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117241.log 2023-11-26T07:18:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142467.log 2023-11-26T07:18:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117241.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110230.log.gz 2023-11-26T07:18:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142467.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117241.log.gz 2023-11-26T07:18:22.126 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142467.log.gz 2023-11-26T07:18:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121764.log 2023-11-26T07:18:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119733.log 2023-11-26T07:18:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121764.log: /var/log/ceph/ceph-client.admin.119733.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119733.log.gz 2023-11-26T07:18:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125061.log 2023-11-26T07:18:22.127 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.121764.log.gz 2023-11-26T07:18:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111064.log 2023-11-26T07:18:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110046.log 2023-11-26T07:18:22.127 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111064.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125061.log.gz 2023-11-26T07:18:22.127 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111064.log.gz 2023-11-26T07:18:22.128 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110046.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110046.log.gz 2023-11-26T07:18:22.128 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120471.log 2023-11-26T07:18:22.128 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141534.log 2023-11-26T07:18:22.128 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120471.log: /var/log/ceph/ceph-client.admin.141534.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120471.log.gz 2023-11-26T07:18:22.128 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141534.log.gz 2023-11-26T07:18:22.128 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109745.log 2023-11-26T07:18:22.128 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137204.log 2023-11-26T07:18:22.128 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.109745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116513.log 2023-11-26T07:18:22.128 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109745.log.gz 2023-11-26T07:18:22.129 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.137204.log: /var/log/ceph/ceph-client.admin.116513.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137204.log.gz 2023-11-26T07:18:22.129 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.116513.log.gz 2023-11-26T07:18:22.129 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119392.log 2023-11-26T07:18:22.129 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124923.log 2023-11-26T07:18:22.129 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.126845.log 2023-11-26T07:18:22.129 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119392.log.gz 2023-11-26T07:18:22.129 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124923.log: 88.1%/var/log/ceph/ceph-client.0.126845.log: -- replaced with /var/log/ceph/ceph-client.admin.124923.log.gz 2023-11-26T07:18:22.129 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.126845.log.gz 2023-11-26T07:18:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-26T07:18:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117473.log 2023-11-26T07:18:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.117473.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117473.log.gz 2023-11-26T07:18:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111081.log 2023-11-26T07:18:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137755.log 2023-11-26T07:18:22.131 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.103441.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103441.log.gz 2023-11-26T07:18:22.131 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.113626.log 2023-11-26T07:18:22.131 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.1.113626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.113626.log.gz 2023-11-26T07:18:22.132 INFO:teuthology.orchestra.run.smithi203.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-26T07:18:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111081.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111081.log.gz 2023-11-26T07:18:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.137755.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.137755.log.gz 2023-11-26T07:18:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125713.log 2023-11-26T07:18:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138046.log 2023-11-26T07:18:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125713.log: /var/log/ceph/ceph-client.admin.138046.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125713.log.gz 2023-11-26T07:18:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124150.log 2023-11-26T07:18:22.160 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138046.log.gz 2023-11-26T07:18:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140967.log 2023-11-26T07:18:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124150.log: 88.8%/var/log/ceph/ceph-client.admin.140967.log: -- replaced with /var/log/ceph/ceph-client.admin.124150.log.gz 2023-11-26T07:18:22.161 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140967.log.gz 2023-11-26T07:18:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119258.log 2023-11-26T07:18:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126262.log 2023-11-26T07:18:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119258.log: /var/log/ceph/ceph-client.admin.126262.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119258.log.gz 2023-11-26T07:18:22.161 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126262.log.gz 2023-11-26T07:18:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110758.log 2023-11-26T07:18:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136232.log 2023-11-26T07:18:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110758.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110758.log.gz 2023-11-26T07:18:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136232.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136232.log.gz 2023-11-26T07:18:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141239.log 2023-11-26T07:18:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135588.log 2023-11-26T07:18:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141239.log: /var/log/ceph/ceph-client.admin.135588.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141239.log.gz 2023-11-26T07:18:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119120.log 2023-11-26T07:18:22.162 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135588.log.gz 2023-11-26T07:18:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128565.log 2023-11-26T07:18:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119120.log: 88.7%/var/log/ceph/ceph-client.admin.128565.log: -- replaced with /var/log/ceph/ceph-client.admin.119120.log.gz 2023-11-26T07:18:22.163 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128565.log.gz 2023-11-26T07:18:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142935.log 2023-11-26T07:18:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131361.log 2023-11-26T07:18:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142935.log: /var/log/ceph/ceph-client.admin.131361.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.142935.log.gz 2023-11-26T07:18:22.163 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.131361.log.gz 2023-11-26T07:18:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116007.log 2023-11-26T07:18:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116421.log 2023-11-26T07:18:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116007.log: /var/log/ceph/ceph-client.admin.116421.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116007.log.gz 2023-11-26T07:18:22.164 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116421.log.gz 2023-11-26T07:18:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126308.log 2023-11-26T07:18:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116283.log 2023-11-26T07:18:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.126308.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126308.log.gz 2023-11-26T07:18:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116283.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116283.log.gz 2023-11-26T07:18:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138276.log 2023-11-26T07:18:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118966.log 2023-11-26T07:18:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.138276.log: /var/log/ceph/ceph-client.admin.118966.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138276.log.gz 2023-11-26T07:18:22.165 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118966.log.gz 2023-11-26T07:18:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142651.log 2023-11-26T07:18:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127747.log 2023-11-26T07:18:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142651.log: 88.5%/var/log/ceph/ceph-client.admin.127747.log: -- replaced with /var/log/ceph/ceph-client.admin.142651.log.gz 2023-11-26T07:18:22.165 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127747.log.gz 2023-11-26T07:18:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122399.log 2023-11-26T07:18:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134837.log 2023-11-26T07:18:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122399.log: /var/log/ceph/ceph-client.admin.134837.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122399.log.gz 2023-11-26T07:18:22.166 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134837.log.gz 2023-11-26T07:18:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130332.log 2023-11-26T07:18:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129334.log 2023-11-26T07:18:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130332.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130332.log.gz 2023-11-26T07:18:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.129334.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129334.log.gz 2023-11-26T07:18:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120333.log 2023-11-26T07:18:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136546.log 2023-11-26T07:18:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120333.log: /var/log/ceph/ceph-client.admin.136546.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120333.log.gz 2023-11-26T07:18:22.167 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136546.log.gz 2023-11-26T07:18:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120193.log 2023-11-26T07:18:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117751.log 2023-11-26T07:18:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120193.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.120193.log.gz 2023-11-26T07:18:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117751.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117751.log.gz 2023-11-26T07:18:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133296.log 2023-11-26T07:18:22.168 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131169.log 2023-11-26T07:18:22.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133296.log: /var/log/ceph/ceph-client.admin.131169.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.133296.log.gz 2023-11-26T07:18:22.168 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.131169.log.gz 2023-11-26T07:18:22.168 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130813.log 2023-11-26T07:18:22.168 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142835.log 2023-11-26T07:18:22.168 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130813.log: /var/log/ceph/ceph-client.admin.142835.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130813.log.gz 2023-11-26T07:18:22.168 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142835.log.gz 2023-11-26T07:18:22.168 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120147.log 2023-11-26T07:18:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141967.log 2023-11-26T07:18:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120147.log: /var/log/ceph/ceph-client.admin.141967.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120147.log.gz 2023-11-26T07:18:22.169 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141967.log.gz 2023-11-26T07:18:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111932.log 2023-11-26T07:18:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120009.log 2023-11-26T07:18:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111932.log: /var/log/ceph/ceph-client.admin.120009.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.111932.log.gz 2023-11-26T07:18:22.169 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120009.log.gz 2023-11-26T07:18:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131626.log 2023-11-26T07:18:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135818.log 2023-11-26T07:18:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.131626.log: /var/log/ceph/ceph-client.admin.135818.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131626.log.gz 2023-11-26T07:18:22.170 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135818.log.gz 2023-11-26T07:18:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126078.log 2023-11-26T07:18:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109362.log 2023-11-26T07:18:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.126078.log: /var/log/ceph/ceph-client.admin.109362.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.126078.log.gz 2023-11-26T07:18:22.170 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109362.log.gz 2023-11-26T07:18:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141147.log 2023-11-26T07:18:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124536.log 2023-11-26T07:18:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141147.log: /var/log/ceph/ceph-client.admin.124536.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141147.log.gz 2023-11-26T07:18:22.171 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.124536.log.gz 2023-11-26T07:18:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131718.log 2023-11-26T07:18:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137112.log 2023-11-26T07:18:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.131718.log: /var/log/ceph/ceph-client.admin.137112.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.131718.log.gz 2023-11-26T07:18:22.171 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137112.log.gz 2023-11-26T07:18:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128427.log 2023-11-26T07:18:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121143.log 2023-11-26T07:18:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128427.log: /var/log/ceph/ceph-client.admin.121143.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128427.log.gz 2023-11-26T07:18:22.172 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121143.log.gz 2023-11-26T07:18:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-26T07:18:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135864.log 2023-11-26T07:18:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115540.log 2023-11-26T07:18:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135864.log: /var/log/ceph/ceph-client.admin.115540.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135864.log.gz 2023-11-26T07:18:22.173 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115540.log.gz 2023-11-26T07:18:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121304.log 2023-11-26T07:18:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121304.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.121304.log.gz 2023-11-26T07:18:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132552.log 2023-11-26T07:18:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132552.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-26T07:18:22.173 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.132552.log.gz 2023-11-26T07:18:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110276.log 2023-11-26T07:18:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111748.log 2023-11-26T07:18:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110276.log: /var/log/ceph/ceph-client.admin.111748.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.110276.log.gz 2023-11-26T07:18:22.174 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111748.log.gz 2023-11-26T07:18:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111379.log 2023-11-26T07:18:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135450.log 2023-11-26T07:18:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111379.log: /var/log/ceph/ceph-client.admin.135450.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111379.log.gz 2023-11-26T07:18:22.174 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.135450.log.gz 2023-11-26T07:18:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132457.log 2023-11-26T07:18:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117659.log 2023-11-26T07:18:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132457.log: /var/log/ceph/ceph-client.admin.117659.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.132457.log.gz 2023-11-26T07:18:22.175 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117659.log.gz 2023-11-26T07:18:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112392.log 2023-11-26T07:18:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136880.log 2023-11-26T07:18:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.112392.log: /var/log/ceph/ceph-client.admin.136880.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136880.log.gz 2023-11-26T07:18:22.175 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.112392.log.gz 2023-11-26T07:18:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123919.log 2023-11-26T07:18:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130032.log 2023-11-26T07:18:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123919.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123919.log.gz 2023-11-26T07:18:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130032.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.130032.log.gz 2023-11-26T07:18:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135910.log 2023-11-26T07:18:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141442.log 2023-11-26T07:18:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135910.log: /var/log/ceph/ceph-client.admin.141442.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135910.log.gz 2023-11-26T07:18:22.176 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141442.log.gz 2023-11-26T07:18:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113693.log 2023-11-26T07:18:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137066.log 2023-11-26T07:18:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.2.113693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113693.log.gz 2023-11-26T07:18:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.137066.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137066.log.gz 2023-11-26T07:18:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142981.log 2023-11-26T07:18:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138092.log 2023-11-26T07:18:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142981.log: /var/log/ceph/ceph-client.admin.138092.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142981.log.gz 2023-11-26T07:18:22.177 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138092.log.gz 2023-11-26T07:18:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110322.log 2023-11-26T07:18:22.178 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114706.log 2023-11-26T07:18:22.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110322.log: 88.8%/var/log/ceph/ceph-client.admin.114706.log: -- replaced with /var/log/ceph/ceph-client.admin.110322.log.gz 2023-11-26T07:18:22.178 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114706.log.gz 2023-11-26T07:18:22.178 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117613.log 2023-11-26T07:18:22.178 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117287.log 2023-11-26T07:18:22.178 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117613.log: /var/log/ceph/ceph-client.admin.117287.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117613.log.gz 2023-11-26T07:18:22.178 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117287.log.gz 2023-11-26T07:18:22.178 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119963.log 2023-11-26T07:18:22.179 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114519.log 2023-11-26T07:18:22.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119963.log: /var/log/ceph/ceph-client.admin.114519.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119963.log.gz 2023-11-26T07:18:22.179 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114519.log.gz 2023-11-26T07:18:22.179 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125015.log 2023-11-26T07:18:22.179 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126400.log 2023-11-26T07:18:22.179 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125015.log: /var/log/ceph/ceph-client.admin.126400.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125015.log.gz 2023-11-26T07:18:22.179 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.126400.log.gz 2023-11-26T07:18:22.179 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114283.log 2023-11-26T07:18:22.180 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130078.log 2023-11-26T07:18:22.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114283.log: /var/log/ceph/ceph-client.admin.130078.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114283.log.gz 2023-11-26T07:18:22.180 INFO:teuthology.orchestra.run.smithi083.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.130078.log.gz 2023-11-26T07:18:22.180 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124080.log 2023-11-26T07:18:22.180 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114819.log 2023-11-26T07:18:22.180 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124080.log: /var/log/ceph/ceph-client.admin.114819.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124080.log.gz 2023-11-26T07:18:22.180 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114819.log.gz 2023-11-26T07:18:22.180 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117010.log 2023-11-26T07:18:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118048.log 2023-11-26T07:18:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117010.log: /var/log/ceph/ceph-client.admin.118048.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117010.log.gz 2023-11-26T07:18:22.181 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.118048.log.gz 2023-11-26T07:18:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111702.log 2023-11-26T07:18:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116237.log 2023-11-26T07:18:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111702.log: /var/log/ceph/ceph-client.admin.116237.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111702.log.gz 2023-11-26T07:18:22.181 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116237.log.gz 2023-11-26T07:18:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141921.log 2023-11-26T07:18:22.182 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133026.log 2023-11-26T07:18:22.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141921.log: /var/log/ceph/ceph-client.admin.133026.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141921.log.gz 2023-11-26T07:18:22.182 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.133026.log.gz 2023-11-26T07:18:22.182 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136140.log 2023-11-26T07:18:22.182 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138726.log 2023-11-26T07:18:22.182 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136140.log: /var/log/ceph/ceph-client.admin.138726.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.136140.log.gz 2023-11-26T07:18:22.182 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138726.log.gz 2023-11-26T07:18:22.182 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140606.log 2023-11-26T07:18:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120939.log 2023-11-26T07:18:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.140606.log: /var/log/ceph/ceph-client.admin.120939.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140606.log.gz 2023-11-26T07:18:22.183 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120939.log.gz 2023-11-26T07:18:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128473.log 2023-11-26T07:18:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116605.log 2023-11-26T07:18:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128473.log: /var/log/ceph/ceph-client.admin.116605.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128473.log.gz 2023-11-26T07:18:22.183 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116605.log.gz 2023-11-26T07:18:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117705.log 2023-11-26T07:18:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138680.log 2023-11-26T07:18:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117705.log: /var/log/ceph/ceph-client.admin.138680.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117705.log.gz 2023-11-26T07:18:22.184 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138680.log.gz 2023-11-26T07:18:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123169.log 2023-11-26T07:18:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143211.log 2023-11-26T07:18:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132782.log 2023-11-26T07:18:22.184 INFO:teuthology.orchestra.run.smithi083.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123169.log.gz 2023-11-26T07:18:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.143211.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.143211.log.gz 2023-11-26T07:18:22.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132782.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132782.log.gz 2023-11-26T07:18:22.185 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133158.log 2023-11-26T07:18:22.185 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126032.log 2023-11-26T07:18:22.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133158.log: /var/log/ceph/ceph-client.admin.126032.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.133158.log.gz 2023-11-26T07:18:22.185 INFO:teuthology.orchestra.run.smithi083.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.126032.log.gz 2023-11-26T07:18:22.185 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120287.log 2023-11-26T07:18:22.185 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143323.log 2023-11-26T07:18:22.185 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120287.log: /var/log/ceph/ceph-client.admin.143323.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120287.log.gz 2023-11-26T07:18:22.186 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.143323.log.gz 2023-11-26T07:18:22.186 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128268.log 2023-11-26T07:18:22.186 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115314.log 2023-11-26T07:18:22.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128268.log: /var/log/ceph/ceph-client.admin.115314.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128268.log.gz 2023-11-26T07:18:22.186 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.115314.log.gz 2023-11-26T07:18:22.186 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127701.log 2023-11-26T07:18:22.186 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128314.log 2023-11-26T07:18:22.186 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.127701.log: /var/log/ceph/ceph-client.admin.128314.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127701.log.gz 2023-11-26T07:18:22.187 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128314.log.gz 2023-11-26T07:18:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136186.log 2023-11-26T07:18:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131407.log 2023-11-26T07:18:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136186.log: /var/log/ceph/ceph-client.admin.131407.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136186.log.gz 2023-11-26T07:18:22.187 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131407.log.gz 2023-11-26T07:18:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115494.log 2023-11-26T07:18:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128360.log 2023-11-26T07:18:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.115494.log: /var/log/ceph/ceph-client.admin.128360.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115494.log.gz 2023-11-26T07:18:22.188 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128360.log.gz 2023-11-26T07:18:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111443.log 2023-11-26T07:18:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120669.log 2023-11-26T07:18:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111443.log: /var/log/ceph/ceph-client.admin.120669.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111443.log.gz 2023-11-26T07:18:22.188 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120669.log.gz 2023-11-26T07:18:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117910.log 2023-11-26T07:18:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112162.log 2023-11-26T07:18:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117910.log: /var/log/ceph/ceph-client.admin.112162.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117910.log.gz 2023-11-26T07:18:22.189 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112162.log.gz 2023-11-26T07:18:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124352.log 2023-11-26T07:18:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140721.log 2023-11-26T07:18:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124352.log: /var/log/ceph/ceph-client.admin.140721.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124352.log.gz 2023-11-26T07:18:22.189 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140721.log.gz 2023-11-26T07:18:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.96938.log 2023-11-26T07:18:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109998.log 2023-11-26T07:18:22.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.tmp-client.admin.96938.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.96938.log.gz 2023-11-26T07:18:22.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.109998.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109998.log.gz 2023-11-26T07:18:22.190 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136459.log 2023-11-26T07:18:22.190 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129090.log 2023-11-26T07:18:22.190 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136459.log: /var/log/ceph/ceph-client.admin.129090.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136459.log.gz 2023-11-26T07:18:22.190 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.129090.log.gz 2023-11-26T07:18:22.190 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122269.log 2023-11-26T07:18:22.190 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129986.log 2023-11-26T07:18:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122269.log: /var/log/ceph/ceph-client.admin.129986.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.122269.log.gz 2023-11-26T07:18:22.191 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129986.log.gz 2023-11-26T07:18:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111840.log 2023-11-26T07:18:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138230.log 2023-11-26T07:18:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111840.log: /var/log/ceph/ceph-client.admin.138230.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111840.log.gz 2023-11-26T07:18:22.191 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138230.log.gz 2023-11-26T07:18:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120101.log 2023-11-26T07:18:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128111.log 2023-11-26T07:18:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120101.log: /var/log/ceph/ceph-client.admin.128111.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120101.log.gz 2023-11-26T07:18:22.192 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128111.log.gz 2023-11-26T07:18:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135109.log 2023-11-26T07:18:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120425.log 2023-11-26T07:18:22.193 INFO:teuthology.orchestra.run.smithi203.stderr: 95.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-26T07:18:22.202 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135109.log: /var/log/ceph/ceph-client.admin.120425.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135109.log.gz 2023-11-26T07:18:22.203 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120425.log.gz 2023-11-26T07:18:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120055.log 2023-11-26T07:18:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135404.log 2023-11-26T07:18:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120055.log: /var/log/ceph/ceph-client.admin.135404.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120055.log.gz 2023-11-26T07:18:22.203 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.135404.log.gz 2023-11-26T07:18:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126216.log 2023-11-26T07:18:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134883.log 2023-11-26T07:18:22.203 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.126216.log: /var/log/ceph/ceph-client.admin.134883.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126216.log.gz 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134883.log.gz 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110748.log 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118553.log 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110748.log: /var/log/ceph/ceph-client.admin.118553.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110748.log.gz 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118553.log.gz 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116053.log 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116053.log: /var/log/ceph/ceph-osd.1.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116053.log.gz 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118461.log 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.118461.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118461.log.gz 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117333.log 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117333.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117333.log.gz 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122177.log 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122177.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122177.log.gz 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117521.log 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117521.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117521.log.gz 2023-11-26T07:18:22.206 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.134202.log 2023-11-26T07:18:22.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.134202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.134202.log.gz 2023-11-26T07:18:22.206 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132133.log 2023-11-26T07:18:22.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132133.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.132133.log.gz 2023-11-26T07:18:22.206 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121626.log 2023-11-26T07:18:22.206 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121626.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.121626.log.gz 2023-11-26T07:18:22.206 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113028.log 2023-11-26T07:18:22.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.113028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113028.log.gz 2023-11-26T07:18:22.207 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132644.log 2023-11-26T07:18:22.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132644.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.132644.log.gz 2023-11-26T07:18:22.207 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136684.log 2023-11-26T07:18:22.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136684.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.136684.log.gz 2023-11-26T07:18:22.207 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136341.log 2023-11-26T07:18:22.207 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129604.log 2023-11-26T07:18:22.207 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136341.log: /var/log/ceph/ceph-client.admin.129604.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.136341.log.gz 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi083.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.129604.log.gz 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119687.log 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141718.log 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119687.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119687.log.gz 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141718.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141718.log.gz 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136926.log 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122855.log 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136926.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.136926.log.gz 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122855.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122855.log.gz 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121258.log 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121258.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121258.log.gz 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121396.log 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121396.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121396.log.gz 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130170.log 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130170.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130170.log.gz 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129746.log 2023-11-26T07:18:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.129746.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129746.log.gz 2023-11-26T07:18:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130675.log 2023-11-26T07:18:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123989.log 2023-11-26T07:18:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130675.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130675.log.gz 2023-11-26T07:18:22.210 INFO:teuthology.orchestra.run.smithi083.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-26T07:18:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123989.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123989.log.gz 2023-11-26T07:18:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133439.log 2023-11-26T07:18:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137296.log 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133439.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.133439.log.gz 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.137296.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.137296.log.gz 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130943.log 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112346.log 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130943.log: /var/log/ceph/ceph-client.admin.112346.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130943.log.gz 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112346.log.gz 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127793.log 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127973.log 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.127793.log: /var/log/ceph/ceph-client.admin.127973.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127793.log.gz 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127973.log.gz 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132184.log 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118507.log 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132184.log: /var/log/ceph/ceph-client.admin.118507.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132184.log.gz 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118507.log.gz 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126446.log 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124444.log 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.126446.log: /var/log/ceph/ceph-client.admin.124444.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126446.log.gz 2023-11-26T07:18:22.213 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.124444.log.gz 2023-11-26T07:18:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122671.log 2023-11-26T07:18:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110506.log 2023-11-26T07:18:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122671.log: /var/log/ceph/ceph-client.admin.110506.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.122671.log.gz 2023-11-26T07:18:22.213 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110506.log.gz 2023-11-26T07:18:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132736.log 2023-11-26T07:18:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133344.log 2023-11-26T07:18:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132736.log: /var/log/ceph/ceph-client.admin.133344.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.132736.log.gz 2023-11-26T07:18:22.216 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.133344.log.gz 2023-11-26T07:18:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118730.log 2023-11-26T07:18:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115268.log 2023-11-26T07:18:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.118730.log: /var/log/ceph/ceph-client.admin.115268.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118730.log.gz 2023-11-26T07:18:22.216 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115268.log.gz 2023-11-26T07:18:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138887.log 2023-11-26T07:18:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141872.log 2023-11-26T07:18:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.138887.log: /var/log/ceph/ceph-client.admin.141872.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138887.log.gz 2023-11-26T07:18:22.217 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141872.log.gz 2023-11-26T07:18:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133112.log 2023-11-26T07:18:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141285.log 2023-11-26T07:18:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133112.log: /var/log/ceph/ceph-client.admin.141285.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.133112.log.gz 2023-11-26T07:18:22.217 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141285.log.gz 2023-11-26T07:18:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132287.log 2023-11-26T07:18:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142513.log 2023-11-26T07:18:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132287.log: /var/log/ceph/ceph-client.admin.142513.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132287.log.gz 2023-11-26T07:18:22.218 INFO:teuthology.orchestra.run.smithi083.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142513.log.gz 2023-11-26T07:18:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133899.log 2023-11-26T07:18:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119212.log 2023-11-26T07:18:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133899.log: /var/log/ceph/ceph-client.admin.119212.log: 91.0% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133899.log.gz -- replaced with /var/log/ceph/ceph-client.admin.119212.log.gz 2023-11-26T07:18:22.219 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-11-26T07:18:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115176.log 2023-11-26T07:18:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116467.log 2023-11-26T07:18:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.115176.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115176.log.gz 2023-11-26T07:18:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127839.log 2023-11-26T07:18:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116467.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116467.log.gz 2023-11-26T07:18:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.127839.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.127839.log.gz 2023-11-26T07:18:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114911.log 2023-11-26T07:18:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133577.log 2023-11-26T07:18:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114911.log: /var/log/ceph/ceph-client.admin.133577.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114911.log.gz 2023-11-26T07:18:22.223 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.133577.log.gz 2023-11-26T07:18:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118351.log 2023-11-26T07:18:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141101.log 2023-11-26T07:18:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.118351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124220.log 2023-11-26T07:18:22.223 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118351.log.gz 2023-11-26T07:18:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141101.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141101.log.gz 2023-11-26T07:18:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128157.log 2023-11-26T07:18:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124220.log: /var/log/ceph/ceph-client.admin.128157.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124220.log.gz 2023-11-26T07:18:22.224 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.128157.log.gz 2023-11-26T07:18:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135201.log 2023-11-26T07:18:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127043.log 2023-11-26T07:18:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135201.log: /var/log/ceph/ceph-client.admin.127043.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135201.log.gz 2023-11-26T07:18:22.224 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127043.log.gz 2023-11-26T07:18:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127325.log 2023-11-26T07:18:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124582.log 2023-11-26T07:18:22.225 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.127325.log: /var/log/ceph/ceph-client.admin.124582.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127325.log.gz 2023-11-26T07:18:22.225 INFO:teuthology.orchestra.run.smithi203.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-26T07:18:22.228 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.124582.log.gz 2023-11-26T07:18:22.228 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115222.log 2023-11-26T07:18:22.228 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111077.log 2023-11-26T07:18:22.228 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.115222.log: /var/log/ceph/ceph-client.admin.111077.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115222.log.gz 2023-11-26T07:18:22.228 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111077.log.gz 2023-11-26T07:18:22.228 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120987.log 2023-11-26T07:18:22.228 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137158.log 2023-11-26T07:18:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120987.log: /var/log/ceph/ceph-client.admin.137158.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.120987.log.gz 2023-11-26T07:18:22.229 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137158.log.gz 2023-11-26T07:18:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135247.log 2023-11-26T07:18:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129800.log 2023-11-26T07:18:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135247.log: /var/log/ceph/ceph-client.admin.129800.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135247.log.gz 2023-11-26T07:18:22.229 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129800.log.gz 2023-11-26T07:18:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114053.log 2023-11-26T07:18:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119871.log 2023-11-26T07:18:22.230 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114053.log: /var/log/ceph/ceph-client.admin.119871.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114053.log.gz 2023-11-26T07:18:22.230 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119871.log.gz 2023-11-26T07:18:22.230 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130262.log 2023-11-26T07:18:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110460.log 2023-11-26T07:18:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130262.log: /var/log/ceph/ceph-client.admin.110460.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.110460.log.gz 2023-11-26T07:18:22.232 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.130262.log.gz 2023-11-26T07:18:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115586.log 2023-11-26T07:18:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125528.log 2023-11-26T07:18:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.115586.log: /var/log/ceph/ceph-client.admin.125528.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115586.log.gz 2023-11-26T07:18:22.233 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.125528.log.gz 2023-11-26T07:18:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114468.log 2023-11-26T07:18:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111978.log 2023-11-26T07:18:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114468.log: /var/log/ceph/ceph-client.admin.111978.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114468.log.gz 2023-11-26T07:18:22.233 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111978.log.gz 2023-11-26T07:18:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143119.log 2023-11-26T07:18:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124490.log 2023-11-26T07:18:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.143119.log: /var/log/ceph/ceph-client.admin.124490.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143119.log.gz 2023-11-26T07:18:22.235 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124490.log.gz 2023-11-26T07:18:22.235 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121718.log 2023-11-26T07:18:22.236 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116964.log 2023-11-26T07:18:22.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121718.log: /var/log/ceph/ceph-client.admin.116964.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121718.log.gz 2023-11-26T07:18:22.236 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116964.log.gz 2023-11-26T07:18:22.236 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141672.log 2023-11-26T07:18:22.236 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119641.log 2023-11-26T07:18:22.236 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141672.log: /var/log/ceph/ceph-client.admin.119641.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141672.log.gz 2023-11-26T07:18:22.237 INFO:teuthology.orchestra.run.smithi083.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.119641.log.gz 2023-11-26T07:18:22.237 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140560.log 2023-11-26T07:18:22.238 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143027.log 2023-11-26T07:18:22.238 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.140560.log: /var/log/ceph/ceph-client.admin.143027.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140560.log.gz 2023-11-26T07:18:22.238 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143027.log.gz 2023-11-26T07:18:22.238 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128998.log 2023-11-26T07:18:22.238 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123540.log 2023-11-26T07:18:22.239 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128998.log: /var/log/ceph/ceph-client.admin.123540.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128998.log.gz 2023-11-26T07:18:22.240 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123540.log.gz 2023-11-26T07:18:22.240 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132236.log 2023-11-26T07:18:22.240 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114329.log 2023-11-26T07:18:22.240 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132236.log: /var/log/ceph/ceph-client.admin.114329.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132236.log.gz 2023-11-26T07:18:22.240 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114329.log.gz 2023-11-26T07:18:22.240 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135021.log 2023-11-26T07:18:22.241 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119917.log 2023-11-26T07:18:22.241 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135021.log: /var/log/ceph/ceph-client.admin.119917.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.135021.log.gz 2023-11-26T07:18:22.241 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119917.log.gz 2023-11-26T07:18:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111134.log 2023-11-26T07:18:22.242 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114957.log 2023-11-26T07:18:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111134.log: /var/log/ceph/ceph-client.admin.114957.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111134.log.gz 2023-11-26T07:18:22.243 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114957.log.gz 2023-11-26T07:18:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113961.log 2023-11-26T07:18:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140875.log 2023-11-26T07:18:22.243 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.113961.log: /var/log/ceph/ceph-client.admin.140875.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113961.log.gz 2023-11-26T07:18:22.244 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140875.log.gz 2023-11-26T07:18:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125443.log 2023-11-26T07:18:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137893.log 2023-11-26T07:18:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125443.log: /var/log/ceph/ceph-client.admin.137893.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125443.log.gz 2023-11-26T07:18:22.245 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137893.log.gz 2023-11-26T07:18:22.245 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137847.log 2023-11-26T07:18:22.246 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134426.log 2023-11-26T07:18:22.246 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.137847.log: /var/log/ceph/ceph-client.admin.134426.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137847.log.gz 2023-11-26T07:18:22.246 INFO:teuthology.orchestra.run.smithi083.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.134426.log.gz 2023-11-26T07:18:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135634.log 2023-11-26T07:18:22.247 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121350.log 2023-11-26T07:18:22.248 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135634.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135634.log.gz 2023-11-26T07:18:22.248 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121350.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121350.log.gz 2023-11-26T07:18:22.248 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125199.log 2023-11-26T07:18:22.248 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115827.log 2023-11-26T07:18:22.249 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.125199.log: /var/log/ceph/ceph-client.admin.115827.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125199.log.gz 2023-11-26T07:18:22.249 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115827.log.gz 2023-11-26T07:18:22.250 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133204.log 2023-11-26T07:18:22.250 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116879.log 2023-11-26T07:18:22.250 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133204.log: /var/log/ceph/ceph-client.admin.116879.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133204.log.gz 2023-11-26T07:18:22.251 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116879.log.gz 2023-11-26T07:18:22.251 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111450.log 2023-11-26T07:18:22.251 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115781.log 2023-11-26T07:18:22.253 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111450.log: /var/log/ceph/ceph-client.admin.115781.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111450.log.gz 2023-11-26T07:18:22.253 INFO:teuthology.orchestra.run.smithi083.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115781.log.gz 2023-11-26T07:18:22.253 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142743.log 2023-11-26T07:18:22.255 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115003.log 2023-11-26T07:18:22.255 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.142743.log: /var/log/ceph/ceph-client.admin.115003.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142743.log.gz 2023-11-26T07:18:22.255 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115003.log.gz 2023-11-26T07:18:22.255 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120240.log 2023-11-26T07:18:22.255 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124720.log 2023-11-26T07:18:22.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135542.log 2023-11-26T07:18:22.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124720.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120240.log.gz 2023-11-26T07:18:22.256 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116145.log 2023-11-26T07:18:22.256 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124720.log.gz 2023-11-26T07:18:22.256 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135542.log: /var/log/ceph/ceph-client.admin.116145.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.135542.log.gz 2023-11-26T07:18:22.256 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116145.log.gz 2023-11-26T07:18:22.256 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120800.log 2023-11-26T07:18:22.257 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111561.log 2023-11-26T07:18:22.257 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120800.log: /var/log/ceph/ceph-client.admin.111561.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120800.log.gz 2023-11-26T07:18:22.258 INFO:teuthology.orchestra.run.smithi083.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111561.log.gz 2023-11-26T07:18:22.258 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122223.log 2023-11-26T07:18:22.258 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136730.log 2023-11-26T07:18:22.258 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122223.log: /var/log/ceph/ceph-client.admin.136730.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122223.log.gz 2023-11-26T07:18:22.258 INFO:teuthology.orchestra.run.smithi083.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136730.log.gz 2023-11-26T07:18:22.258 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103539.log 2023-11-26T07:18:22.259 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127555.log 2023-11-26T07:18:22.259 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.103539.log: /var/log/ceph/ceph-client.admin.127555.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103539.log.gz 2023-11-26T07:18:22.260 INFO:teuthology.orchestra.run.smithi083.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127555.log.gz 2023-11-26T07:18:22.260 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116651.log 2023-11-26T07:18:22.260 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136592.log 2023-11-26T07:18:22.260 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116651.log: /var/log/ceph/ceph-client.admin.136592.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116651.log.gz 2023-11-26T07:18:22.260 INFO:teuthology.orchestra.run.smithi083.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.136592.log.gz 2023-11-26T07:18:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117195.log 2023-11-26T07:18:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117956.log 2023-11-26T07:18:22.261 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.117195.log: /var/log/ceph/ceph-client.admin.117956.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117195.log.gz 2023-11-26T07:18:22.261 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117956.log.gz 2023-11-26T07:18:22.262 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109458.log 2023-11-26T07:18:22.262 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122763.log 2023-11-26T07:18:22.262 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.109458.log: /var/log/ceph/ceph-client.admin.122763.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.109458.log.gz 2023-11-26T07:18:22.263 INFO:teuthology.orchestra.run.smithi083.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122763.log.gz 2023-11-26T07:18:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143369.log 2023-11-26T07:18:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136778.log 2023-11-26T07:18:22.263 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.143369.log: /var/log/ceph/ceph-client.admin.136778.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143369.log.gz 2023-11-26T07:18:22.264 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136778.log.gz 2023-11-26T07:18:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114099.log 2023-11-26T07:18:22.264 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134518.log 2023-11-26T07:18:22.265 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114099.log: /var/log/ceph/ceph-client.admin.134518.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114099.log.gz 2023-11-26T07:18:22.265 INFO:teuthology.orchestra.run.smithi083.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.134518.log.gz 2023-11-26T07:18:22.265 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133669.log 2023-11-26T07:18:22.266 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126951.log 2023-11-26T07:18:22.266 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.133669.log: /var/log/ceph/ceph-client.admin.126951.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.133669.log.gz 2023-11-26T07:18:22.266 INFO:teuthology.orchestra.run.smithi083.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126951.log.gz 2023-11-26T07:18:22.266 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119779.log 2023-11-26T07:18:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127439.log 2023-11-26T07:18:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119779.log: /var/log/ceph/ceph-client.admin.127439.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119779.log.gz 2023-11-26T07:18:22.267 INFO:teuthology.orchestra.run.smithi083.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.127439.log.gz 2023-11-26T07:18:22.267 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114865.log 2023-11-26T07:18:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123779.log 2023-11-26T07:18:22.268 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114865.log: /var/log/ceph/ceph-client.admin.123779.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114865.log.gz 2023-11-26T07:18:22.268 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123779.log.gz 2023-11-26T07:18:22.269 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136048.log 2023-11-26T07:18:22.270 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113258.log 2023-11-26T07:18:22.270 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136048.log: /var/log/ceph/ceph-client.admin.113258.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136048.log.gz 2023-11-26T07:18:22.270 INFO:teuthology.orchestra.run.smithi083.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.113258.log.gz 2023-11-26T07:18:22.270 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131883.log 2023-11-26T07:18:22.270 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115735.log 2023-11-26T07:18:22.271 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.131883.log: /var/log/ceph/ceph-client.admin.115735.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131883.log.gz 2023-11-26T07:18:22.271 INFO:teuthology.orchestra.run.smithi083.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115735.log.gz 2023-11-26T07:18:22.271 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127089.log 2023-11-26T07:18:22.272 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135680.log 2023-11-26T07:18:22.272 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.127089.log: /var/log/ceph/ceph-client.admin.135680.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127089.log.gz 2023-11-26T07:18:22.273 INFO:teuthology.orchestra.run.smithi083.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135680.log.gz 2023-11-26T07:18:22.273 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121035.log 2023-11-26T07:18:22.273 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111449.log 2023-11-26T07:18:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121035.log: /var/log/ceph/ceph-client.admin.111449.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121035.log.gz 2023-11-26T07:18:22.274 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111449.log.gz 2023-11-26T07:18:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130989.log 2023-11-26T07:18:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118118.log 2023-11-26T07:18:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130989.log: /var/log/ceph/ceph-client.admin.118118.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.130989.log.gz 2023-11-26T07:18:22.275 INFO:teuthology.orchestra.run.smithi083.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118118.log.gz 2023-11-26T07:18:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128065.log 2023-11-26T07:18:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131981.log 2023-11-26T07:18:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128065.log: /var/log/ceph/ceph-client.admin.131981.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.128065.log.gz 2023-11-26T07:18:22.276 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131981.log.gz 2023-11-26T07:18:22.276 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138772.log 2023-11-26T07:18:22.277 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109652.log 2023-11-26T07:18:22.277 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.138772.log: /var/log/ceph/ceph-client.admin.109652.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138772.log.gz 2023-11-26T07:18:22.277 INFO:teuthology.orchestra.run.smithi083.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109652.log.gz 2023-11-26T07:18:22.277 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110754.log 2023-11-26T07:18:22.278 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118355.log 2023-11-26T07:18:22.278 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110754.log: /var/log/ceph/ceph-client.0.118355.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110754.log.gz 2023-11-26T07:18:22.278 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118355.log.gz 2023-11-26T07:18:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115873.log 2023-11-26T07:18:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133623.log 2023-11-26T07:18:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.115873.log: /var/log/ceph/ceph-client.admin.133623.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115873.log.gz 2023-11-26T07:18:22.280 INFO:teuthology.orchestra.run.smithi083.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133623.log.gz 2023-11-26T07:18:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121442.log 2023-11-26T07:18:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117149.log 2023-11-26T07:18:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121442.log: /var/log/ceph/ceph-client.admin.117149.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121442.log.gz 2023-11-26T07:18:22.281 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117149.log.gz 2023-11-26T07:18:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121097.log 2023-11-26T07:18:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132828.log 2023-11-26T07:18:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121097.log: /var/log/ceph/ceph-client.admin.132828.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.121097.log.gz 2023-11-26T07:18:22.282 INFO:teuthology.orchestra.run.smithi083.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.132828.log.gz 2023-11-26T07:18:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119074.log 2023-11-26T07:18:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132936.log 2023-11-26T07:18:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119074.log: /var/log/ceph/ceph-client.admin.132936.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119074.log.gz 2023-11-26T07:18:22.284 INFO:teuthology.orchestra.run.smithi083.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.132936.log.gz 2023-11-26T07:18:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141488.log 2023-11-26T07:18:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110092.log 2023-11-26T07:18:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141488.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141488.log.gz 2023-11-26T07:18:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110092.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110092.log.gz 2023-11-26T07:18:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129244.log 2023-11-26T07:18:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112546.log 2023-11-26T07:18:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.129244.log: /var/log/ceph/ceph-client.admin.112546.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.129244.log.gz 2023-11-26T07:18:22.286 INFO:teuthology.orchestra.run.smithi083.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112546.log.gz 2023-11-26T07:18:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119346.log 2023-11-26T07:18:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.137645.log 2023-11-26T07:18:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119346.log: /var/log/ceph/ceph-client.0.137645.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119346.log.gz 2023-11-26T07:18:22.287 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.137645.log.gz 2023-11-26T07:18:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120754.log 2023-11-26T07:18:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132337.log 2023-11-26T07:18:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.120754.log: /var/log/ceph/ceph-client.admin.132337.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.120754.log.gz 2023-11-26T07:18:22.288 INFO:teuthology.orchestra.run.smithi083.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.132337.log.gz 2023-11-26T07:18:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-26T07:18:22.299 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133807.log 2023-11-26T07:18:22.299 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.133807.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133807.log.gz 2023-11-26T07:18:22.299 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123239.log 2023-11-26T07:18:22.299 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123239.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123239.log.gz 2023-11-26T07:18:22.299 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135358.log 2023-11-26T07:18:22.299 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135358.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135358.log.gz 2023-11-26T07:18:22.300 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110759.log 2023-11-26T07:18:22.300 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110759.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110759.log.gz 2023-11-26T07:18:22.300 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131498.log 2023-11-26T07:18:22.300 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.131498.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131498.log.gz 2023-11-26T07:18:22.300 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114191.log 2023-11-26T07:18:22.300 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114191.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114191.log.gz 2023-11-26T07:18:22.300 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123099.log 2023-11-26T07:18:22.300 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123099.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123099.log.gz 2023-11-26T07:18:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119484.log 2023-11-26T07:18:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.119484.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119484.log.gz 2023-11-26T07:18:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127279.log 2023-11-26T07:18:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.127279.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127279.log.gz 2023-11-26T07:18:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109929.log 2023-11-26T07:18:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.109929.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109929.log.gz 2023-11-26T07:18:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141810.log 2023-11-26T07:18:22.303 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.141810.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141810.log.gz 2023-11-26T07:18:22.303 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112208.log 2023-11-26T07:18:22.303 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.112208.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112208.log.gz 2023-11-26T07:18:22.303 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121580.log 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.121580.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121580.log.gz 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131269.log 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.131269.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.131269.log.gz 2023-11-26T07:18:22.304 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138322.log 2023-11-26T07:18:22.305 INFO:teuthology.orchestra.run.smithi203.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-26T07:18:22.306 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.138322.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138322.log.gz 2023-11-26T07:18:22.306 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110414.log 2023-11-26T07:18:22.306 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110414.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110414.log.gz 2023-11-26T07:18:22.306 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118790.log 2023-11-26T07:18:22.306 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.118790.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118790.log.gz 2023-11-26T07:18:22.306 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112254.log 2023-11-26T07:18:22.307 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.112254.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112254.log.gz 2023-11-26T07:18:22.307 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109567.log 2023-11-26T07:18:22.308 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.109567.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109567.log.gz 2023-11-26T07:18:22.308 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132598.log 2023-11-26T07:18:22.308 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.132598.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132598.log.gz 2023-11-26T07:18:22.309 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128019.log 2023-11-26T07:18:22.310 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.128019.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128019.log.gz 2023-11-26T07:18:22.310 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116375.log 2023-11-26T07:18:22.311 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.116375.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116375.log.gz 2023-11-26T07:18:22.311 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126170.log 2023-11-26T07:18:22.312 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.126170.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126170.log.gz 2023-11-26T07:18:22.312 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-26T07:18:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113689.log 2023-11-26T07:18:22.324 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.2.113689.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113689.log.gz 2023-11-26T07:18:22.324 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130216.log 2023-11-26T07:18:22.325 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.130216.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130216.log.gz 2023-11-26T07:18:22.325 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134562.log 2023-11-26T07:18:22.326 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.134562.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.134562.log.gz 2023-11-26T07:18:22.326 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118002.log 2023-11-26T07:18:22.327 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.118002.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.118002.log.gz 2023-11-26T07:18:22.327 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.126841.log 2023-11-26T07:18:22.328 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.126841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.126841.log.gz 2023-11-26T07:18:22.328 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135293.log 2023-11-26T07:18:22.329 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135293.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135293.log.gz 2023-11-26T07:18:22.329 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135772.log 2023-11-26T07:18:22.330 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.135772.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.135772.log.gz 2023-11-26T07:18:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114568.log 2023-11-26T07:18:22.331 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.114568.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114568.log.gz 2023-11-26T07:18:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124398.log 2023-11-26T07:18:22.332 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.124398.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124398.log.gz 2023-11-26T07:18:22.333 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122717.log 2023-11-26T07:18:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.122717.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122717.log.gz 2023-11-26T07:18:22.334 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136002.log 2023-11-26T07:18:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.136002.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.136002.log.gz 2023-11-26T07:18:22.335 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123610.log 2023-11-26T07:18:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.123610.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123610.log.gz 2023-11-26T07:18:22.336 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110755.log 2023-11-26T07:18:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.110755.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110755.log.gz 2023-11-26T07:18:22.337 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111061.log 2023-11-26T07:18:22.338 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.111061.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111061.log.gz 2023-11-26T07:18:22.394 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-26T07:18:22.401 INFO:teuthology.orchestra.run.smithi203.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-26T07:18:22.463 INFO:teuthology.orchestra.run.smithi203.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-26T07:18:22.465 INFO:teuthology.orchestra.run.smithi083.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-26T07:18:22.471 INFO:teuthology.orchestra.run.smithi203.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-26T07:18:22.504 INFO:teuthology.orchestra.run.smithi203.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-26T07:18:22.505 INFO:teuthology.orchestra.run.smithi083.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-26T07:18:22.585 INFO:teuthology.orchestra.run.smithi203.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-26T07:18:22.587 INFO:teuthology.orchestra.run.smithi203.stderr: 2023-11-26T07:18:22.587 INFO:teuthology.orchestra.run.smithi203.stderr:real 0m0.580s 2023-11-26T07:18:22.587 INFO:teuthology.orchestra.run.smithi203.stderr:user 0m2.865s 2023-11-26T07:18:22.587 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m0.114s 2023-11-26T07:18:22.612 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-26T07:18:22.613 INFO:teuthology.orchestra.run.smithi083.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-26T07:18:22.635 INFO:teuthology.orchestra.run.smithi083.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-26T07:18:22.789 INFO:teuthology.orchestra.run.smithi083.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-26T07:18:22.791 INFO:teuthology.orchestra.run.smithi083.stderr: 2023-11-26T07:18:22.791 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m0.811s 2023-11-26T07:18:22.792 INFO:teuthology.orchestra.run.smithi083.stderr:user 0m3.420s 2023-11-26T07:18:22.792 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m0.396s 2023-11-26T07:18:22.792 INFO:tasks.ceph:Archiving logs... 2023-11-26T07:18:22.793 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353/remote/smithi083/log 2023-11-26T07:18:22.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-26T07:18:23.650 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353/remote/smithi203/log 2023-11-26T07:18:23.651 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-26T07:18:24.025 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-26T07:18:24.034 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-26T07:18:24.035 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-26T07:18:24.037 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-26T07:18:24.134 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-26T07:18:24.144 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-26T07:18:24.144 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:18:24.147 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:18:24.160 INFO:teuthology.orchestra.run.smithi083.stderr:bash: ntpq: command not found 2023-11-26T07:18:24.165 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-26T07:18:24.165 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2023-11-26T07:18:24.165 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 37 +1474us[+1474us] +/- 87ms 2023-11-26T07:18:24.165 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -895us[ -897us] +/- 62ms 2023-11-26T07:18:24.165 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -136us[ -136us] +/- 91ms 2023-11-26T07:18:24.165 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-26T07:18:24.166 INFO:teuthology.orchestra.run.smithi203.stderr:bash: ntpq: command not found 2023-11-26T07:18:24.169 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-26T07:18:24.170 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2023-11-26T07:18:24.170 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 +1473us[+1473us] +/- 87ms 2023-11-26T07:18:24.170 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -877us[ -863us] +/- 62ms 2023-11-26T07:18:24.170 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -135us[ -135us] +/- 91ms 2023-11-26T07:18:24.170 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-26T07:18:24.171 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-26T07:18:24.180 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-26T07:18:24.180 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-26T07:18:24.217 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-26T07:18:24.326 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-26T07:18:24.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-26T07:18:24.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:18:24.461 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-26T07:18:24.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:18:24.494 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-26T07:18:24.506 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-26T07:18:24.556 INFO:teuthology.task.internal:Duration was 1174.394289 seconds 2023-11-26T07:18:24.557 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-26T07:18:24.593 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-26T07:18:24.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-26T07:18:24.596 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-26T07:18:24.640 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-26T07:18:24.642 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-26T07:18:25.076 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-26T07:18:25.076 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2023-11-26T07:18:25.077 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-26T07:18:25.098 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2023-11-26T07:18:25.098 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-26T07:18:25.120 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-26T07:18:25.120 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-26T07:18:25.140 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-26T07:18:25.202 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-26T07:18:25.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-26T07:18:25.205 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-26T07:18:25.329 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-26T07:18:25.339 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-26T07:18:25.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-26T07:18:25.366 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-26T07:18:25.393 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-26T07:18:25.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:18:25.408 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:18:25.431 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2023-11-26T07:18:25.459 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2023-11-26T07:18:25.472 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:18:25.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:18:25.501 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:18:25.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:18:25.530 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-26T07:18:25.540 INFO:teuthology.task.internal:Transferring archived files... 2023-11-26T07:18:25.542 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353/remote/smithi083 2023-11-26T07:18:25.543 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-26T07:18:25.581 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353/remote/smithi203 2023-11-26T07:18:25.581 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-26T07:18:25.618 INFO:teuthology.task.internal:Removing archive directory... 2023-11-26T07:18:25.618 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-26T07:18:25.620 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-26T07:18:25.670 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-26T07:18:25.681 INFO:teuthology.task.internal:Not uploading archives. 2023-11-26T07:18:25.681 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-26T07:18:25.710 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-26T07:18:25.710 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-26T07:18:25.712 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-26T07:18:25.728 INFO:teuthology.orchestra.run.smithi083.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 26 07:18 /home/ubuntu/cephtest 2023-11-26T07:18:25.729 INFO:teuthology.orchestra.run.smithi203.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 26 07:18 /home/ubuntu/cephtest 2023-11-26T07:18:25.730 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-26T07:18:25.761 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-26T07:18:25.861 INFO:teuthology.nuke:Checking targets against current locks 2023-11-26T07:18:25.888 DEBUG:teuthology.nuke:shortname: smithi083 2023-11-26T07:18:25.889 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-26T07:18:25.911 DEBUG:teuthology.nuke:shortname: smithi203 2023-11-26T07:18:25.911 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-26T07:18:25.914 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:38.187444', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-26T07:18:25.937 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467353', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:38.186196', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-26T07:18:25.972 INFO:teuthology.orchestra.console.smithi083:Power off 2023-11-26T07:18:25.973 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-26T07:18:25.997 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2023-11-26T07:18:25.997 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:18:26.013 INFO:teuthology.orchestra.console.smithi203:Power off 2023-11-26T07:18:26.014 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-26T07:18:26.022 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2023-11-26T07:18:26.035 DEBUG:teuthology.orchestra.console.smithi203:power off output: Chassis Power Control: Down/Off 2023-11-26T07:18:26.036 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:18:26.057 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2023-11-26T07:18:30.023 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:18:30.058 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:18:30.134 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2023-11-26T07:18:30.168 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2023-11-26T07:18:34.137 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:18:34.168 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:18:34.249 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2023-11-26T07:18:34.277 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2023-11-26T07:18:38.250 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:18:38.278 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:18:38.360 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2023-11-26T07:18:38.388 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off 2023-11-26T07:18:38.461 INFO:teuthology.orchestra.console.smithi083:Power off completed 2023-11-26T07:18:38.489 INFO:teuthology.orchestra.console.smithi203:Power off completed 2023-11-26T07:18:38.594 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} duration: 1174.394288778305 failure_reason: 'Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=d790d891731748d79c13c7ba8ff278d1 status: fail success: false 2023-11-26T07:18:38.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-26T07:18:38.667 INFO:teuthology.run:FAIL